Will Darren Wilson, the police officer who shot and killed Michael Brown in Ferguson, Missouri, be charged with a crime? That decision is falling entirely to the St. Louis County Prosecuting Attorney’s Office. So too, probably, will the prosecution of a case if one proceeds. So far, county prosecutor Robert McCulloch has convened a grand jury to hear the evidence against Wilson without telling the jurors which charges he thinks should apply.

It’s at best difficult for local prosecutors to bring police officers to justice, and at worst impossible. In the wake of Brown’s death, as well as the recent choking of Eric Garner by police in Staten Island, it is a good time to consider taking certain charging and prosecuting decisions away from local district attorneys and placing them in the hands of special or federal prosecutors.

Local prosecutors in this country have absolute discretion over whom to charge, what charges to file, and how to proceed with cases. The responsibility is especially fraught when the subject of a potential indictment is a police officer. No group of people is more important and closely linked to prosecutors than the local officers with whom they work. Police are responsible for arrests, for investigations, and for gathering evidence. In many cases, like crimes involving drug possession, or when there are no other witnesses, a conviction hinges entirely on testimony and evidence presented by police. Staying on good terms with law enforcement is a prosecutor’s only chance of getting the evidence and cooperation needed to secure convictions in her hundreds of other cases. And although not official policy, conviction rates are key to advancement in many prosecutor’s offices.

And so, when local prosecutors weigh whether to go after police officers, they have an inherent conflict of interest that is both personal and professional. Even in a large city, the same officers are likely to testify in dozens of cases for a single prosecutor over the years. In smaller places, like St. Louis County, it is even more likely that charging a police officer means prosecuting a trusted colleague, if not a friend. In Ferguson there’s an additional source of concern about bias or the appearance of bias. Protesters have called for McCulloch to step down because his father was a police officer who was slain in the line of duty when McCulloch was 12 years old. McCulloch’s response to Missouri Gov. Jay Nixon’s suggestion that he remove himself from Wilson’s case? “Man up.”

It is not surprising that McCulloch is clinging to his authority, but that doesn’t mean it’s smart. In other circumstances, we ask lawyers and judges to take themselves out of cases—to recuse themselves, in legal jargon—when relationships are far more attenuated. For example, attorneys generally may not represent two clients accused of the same crime (at least, not without a waiver from each of them.) Law firms may not represent clients who want to sue corporations that the firm has represented, even if the representation was long ago and unrelated. Judges are asked to recuse themselves from cases in which they are perceived to have personal interests. Why does the glaring conflict local prosecutors have in deciding whether to prosecute the police escape attention?

Prosecutors have to contend not only with destroying relationships with individual cops, but with the pressure brought to bear by police unions. When the Chief Medical Examiner’s Office in New York ruled Eric Garner’s death by choking a homicide, two police unions struck back, calling the examiner’s ruling “political” and “flawed.” In order to protect their officers, these unions condemned a report by the very office they rely upon to investigate homicides.

When local prosecutors decide against prosecuting cops accused of wrongdoing, they face questions of legitimacy related to the racial disparities of our criminal justice system. If you think whites are overrepresented in police departments, take a look at prosecutors’ offices. One study of California prosecutors (and judges) noted that 80 percent of those in each group were white, while 60 percent of defendants were nonwhite. Racism or no, when white prosecutors send black men to prison while appearing to shield police, it looks like the decision-making process is rigged.

What’s the alternative to giving local prosecutors the power to decide whether to charge police officers and having to lead the cases against them? There are a few possibilities, any of which would be better than the current, conflict-riddled approach. One solution is to bring in a prosecutor from another county, within the same state, to decide whether to charge and lead a police prosecution. This is how decisions are made about whether to charge prosecutors themselves. It’s also what happened in Florida after the local prosecutor declined to charge George Zimmerman in the death of Trayvon Martin.

Another remedy is to put the prosecution of local police in the hands of federal prosecutors. Because U.S. attorneys do not rely as much on local law enforcement, their decision-making is at a greater remove. To save resources, the handoff to the federal government should be made at the outset of a case, rather than after a local prosecutor has declined to bring a case, or failed to convict. After a Ventura County jury acquitted four Los Angeles police officers of the brutal beating of Rodney King in 1992, the federal government stepped in and brought its own charges against the police; two of the four officers were then convicted. It would have been more efficient and confidence-inspiring, however, if the federal lawyers had been in charge from the start. Many local prosecutors will surely resist the changes I’m suggesting. They won’t want to cede the power to decide how to handle high-profile cases. But many of them are elected officials, so voters could push for change if they chose to.

The police do a difficult and dangerous job. But when they abuse power or break the law, they must be held to the same standard as every other citizen. Their closest allies aren’t well-positioned to do that. It’s time to recognize that, and find a substitute.