Calling for "oversight" can improve public confidence in the short term, but it's not a panacea for every public policy mess. Sometimes instead, or at least in addition, government actually must focus resources to solve a problem instead of merely "monitor" the continuing failure of existing public policies.
That's why I've come to think some of the solutions championed by reformers aimed at reducing forensic errors, including many folks I've worked with, and whose opinions I respect, somewhat conveniently ignore the depth of the crisis of credibility exposed by the National Research Council of the National Academy of Sciences, and simply don't go far enough. As an example, John Terzano from the Justice Project has a column at Fire Dog Lake titled "
Meaningful Oversight Necessary for Forensic Science" which essentially champions adding layers of bureaucracy for "oversight" and "accreditation" at crime labs "to ensure the objectivity and reliability of forensic science." He concludes:
the integrity of forensic evidence is too important to outsource oversight and quality standards entirely to professional trade organizations. Accreditation and professional certification are important first steps, but the responsibility for setting and ensuring quality standards, objectivity and independence ultimately resides with the state itself. A full solution will need to include more structural reform. One of these crucial steps is the creation of an independent oversight commission, staffed and funded to more closely supervise the work of forensic labs. This type of commission could set statewide quality standards that could build on the baseline afforded by professional associations, and could provide more rigorous, ongoing oversight to ensure that labs actually operate in a way that is consistent with the standards that exist on paper. Shifting forensic labs out from under the control of law enforcement agencies would address the subtle biases that can emerge when forensic workers see themselves on the law enforcement "team" instead of dispassionate and objective scientists. These safeguards and others are outlined in The Justice Project’s policy review Improving the Practices and Use of Forensic Science, and will help to ensure the objectivity and reliability of forensic testing and analysis.
Reliable forensic science is vital, and by making sure that the evidence is objective and valid, we will have a more efficient criminal justice system. Fixing these problems on the front end will reduce the chances that the state will have to spend more money and resources to correct the mistakes and injustices caused by forensic errors. At a time when California, along with the rest of the nation, is dealing with financial restraints, it is all the more imperative that legislators in all states make these improvements a priority. Forensic science can be a powerful tool, and meaningful structural reform is the only way to ensure that the best science is used in our courts.
Unfortunately, according to the NAS report, much so-called "forensic science" isn't really science-based at all but relies on subjective comparisons by people that have higher error rates than had been heretofore acknowledged.
Ironically, the issue came to light because of the development of DNA testing technology, which really an identify an individual to a scientific certainty (though there are still questions about how many points should be "matched" to get a completely reliable result). DNA testing technology was the result of years of rigorous research relying on the scientific method. With few exceptions (including the qualified exception of identifying controlled substances), that's not true of much of what goes on at the crime lab.
By comparison, many traditionally accepted forensic discipines such as fingerprints, handwriting analysis, toolmark comparisons, polygraph examination, tire-print matching, dog-scent lineups, and until recently, arson investigations, may really have no scientific basis at all because they were developed by cops, not scientists, and either have not been subjected to rigorous testing or in some cases (as with the polygraph and debunked arson theories) did not withstand it. These investigative methods have been used for years but remain essentially untested and in most cases courts don't even know their error rates. Maybe they work well, maybe they don't, but they've never been independently tested using the scientific method so nobody really knows.
In that context, what's to oversee? Perhaps some labs are sloppy. Maybe some are corrupt. Maybe some don't follow best practices. Maybe some need more independence from investigators. Maybe some have one or two bad apples who intentionally skew results. Certainly regulation could help with those problems.
But none of that changes the fact that some of the subjective comparison work done at crime labs has unacceptably high error rates. Some of it even qualifies as "junk science," even if courts over the years have held them to be reliable. No amount of oversight, however well-intentioned, can fix that problem, any more than "oversight" could ensure the quality of work by a voodoo priestess.
The missing component here is that verifiable, peer-reviewed scientific research actually needs to be performed on the validity of long-used forensic techniques, which eventually must be either validated or discarded. That's the stark reality facing forensic science today: It's a field surrounded by more questions about its validity than mere oversight can fix.
The hitch: if research must be performed, it must be funded, and nobody wants to talk about spending more research dollars when budget times are tight. In a recent roundup, I'd quoted a
story from the nation's largest forensic science conference in Seattle, whose focus much more than Terzano was on the dearth of funding to rectify the research vacuum:
"The theme of this meeting is 'Putting our house in order,' " said Thomas Bohan, the physicist-turned-forensics-expert who leads the 6,000-member organization.
A National Academy of Sciences (NAS) panel concluded last year that analysis of bite marks, blood spatters, handwriting and even fingerprints is not backed by the type of rigorous evidence that is standard in other scientific disciplines.
"The dominant message here ... is that the emperor really doesn't have all his clothes on," said Donald Kennedy, former president of Stanford University and an organizer of the NAS review.
Bohan said most forensic scientists have taken that message to heart.
The White House Office of Science and Technology Policy established a forensic-science subcommittee, and legislation will be introduced in Congress next month to bolster research and oversight of crime labs. But Bohan is impatient for progress.
"Everybody is talking about what to do," he said.
The Department of Justice (DOJ) is beginning to fund fundamental research in several areas, including ballistics and fire-debris analysis, said Michael Sheppo, leader of the forensic sciences at DOJ's research arm.
One project will seek to determine the error rate in fingerprint analysis — an area where law-enforcement experts had long insisted their record was perfect.
"That's a statement no scientist could accept," said Paul Giannelli, a legal-forensics expert at Case Western Reserve University School of Law.
Federal funding would be great, and I'm fairly confident it will come, eventually, though it hasn't been an Obama Administration priority. In reality, though, this isn't just a problem for the Department of Justice but for every level of government that relies on or operates crime labs, which is all of them involved with the justice system. Arguably, every crime lab should have their own research budget to evaluate their own work as they go. The task will require years-long commitment from state governments, the nonprofit sector, and in some cases even international cooperation through the UN and other multinational institutions. And it will not be completed tomorrow, or next year, or even five years from now, but really amounts to a gaping hole in scientific knowledge.
In particular, the Texas Department of Public Safety's crime lab system is large enough - and law enforcement relies upon it so heavily - that the state of Texas should fund independent, university-level research of its own accord. There's no excuse for waiting on the feds to make sure that our own state's crime-lab work is unimpeachably valid.
Outside of needed but unlikely legislative appropriations, the most obvious source of resources for this may be federal grants administered through the Governor's office. The state receives only one federal grant - $779,670 from the Coverdell grant program - that's specifically devoted to forensic science. But the Governor has access to other large pots of money that could also be spent on crime labs if his office chose to do so.
In the recent past, those monies have gone by the millions for the TDEX database, which is Texas' mega-database version of "total information awareness," for useless webcams along the border and for grants to border sheriffs that are mostly used to pay patrol officers overtime. But in the wake of the NAS report, the Governor's Criminal Justice Division would be wise going forward to shift a large chunk of that funding toward basic research in the forensic sciences.
Indeed, as with every crisis there is also an opportunity here. The NAS report set off what will inevitably become a once-in-a-generation reevaluation of forensic science, an Herculean task which will in all likelihood take a decade or two to fully flesh out. This research will be the subject of both federal and private grants and it'd be great if some Texas institution were to position itself as a leader on the subject, a development that would be much more likely if the Governor's Criminal Justice Division agreed to collaborate with other funding partners to leverage such an institute's startup.
That research could happen at some UT or A&M branch, or perhaps even represents an opportunity for smaller, corrections-oriented Sam Houston State, if they chose to seize it. A university which assembled the talent and expertise to vet forensics using verifiable, peer-reviewed methods would both perform the whole justice system a great service, set itself up for a seemingly inevitable spike in research funding in the area, and make a name for itself internationally in a critically important, cutting edge field.
Oversight is important and I don't want to diminish the need for greater regulation and more well-defined, evidence-based standards in forensic disciplines. But it can't substitute for actually knowing thanks to research and evidence that the methods overseen are actually valid in the first place.
Posts
