Expert Systems Help Labs Process DNA Samples

by Rhonda K. Roby

The products, manufacturers and organizations discussed in this article are presented for informational purposes only, and their discussion does not constitute product approval or endorsement by the U.S. Department of Justice.

Criminal justice professionals who work in the field of DNA analysis know that a backlog of convicted offender samples exists in our nation's laboratories. It takes a long time to analyze a DNA sample of a convicted offender. Two forensic analysts must visually review the sample and apply a set of standard operating procedures that can have many sets of rules. The procedures can be difficult to apply consistently. After the review is complete, the data must be entered into the national database.

Completing all the steps quickly is a formidable challenge.

New software programs called "expert systems" are helping increase the speed of the review process. Expert systems capture all possible circumstances that experts encounter when they do their jobs and dictate what the appropriate responses should be. For forensic analysts, expert systems not only allow them to get consistent, accurate results more quickly, they also help them review and upload many DNA profiles into the national database faster.

But can these expert systems help reduce the backlog of convicted offender samples that exists in our nation's laboratories?

According to the results of an evaluation by the National Institute of Justice (NIJ), they can. The NIJ Expert Systems Testbed (NEST) project, hosted by the Marshall University Forensic Science Center, evaluated the ability of three commercially available expert systems, designed specifically for forensic DNA laboratories, to rapidly and accurately review convicted offender single-source DNA samples for eventual upload into the national DNA database. Project researchers — including this author — found that the three programs will help reduce the backlog and ensure timely submission of data into the national database so ultimately more crimes can be solved.[1],[2]

What Are Expert Systems?

Expert systems are a subset of artificial intelligence. They store knowledge on how to respond to a particular result and, when a challenge is presented, use the stored knowledge in the program to respond with an explanation. The system, however, cannot completely replace a human expert because it contains only rules to solve most commonly encountered problems. The system must be able to recognize cases that are outside its rules and scope of knowledge or those in which there is a possible alternate judgment and alert a human expert reviewer.

Expert systems are developed with the close collaboration of the software development team and forensic science experts. Developers create the system by first conducting extensive interviews with experts in the field where the software will be used, writing algorithms to apply to the data and testing the programs with data appropriate for the software. The knowledge from these experts is then taken and applied consistently to the data review process by using the algorithms, automating the analysis and freeing analysts to focus on more complex challenges.

Use of this technology continues to increase in several industries as they become more aware of its benefits — for instance, expert systems are assisting physicians in making medical diagnoses, supporting NASA's space program and managing inventories for large factories. They serve as scheduling and planning tools when the everyday consumer makes an online airline reservation. A search engine finds all possible airline companies, determines the routes based on particular requests and attempts to find the requested time frame. Once the consumer chooses a particular flight plan, the system can determine the class of travel, price and seat assignments.

For forensic DNA analysis, expert systems could easily be one of the most important advances in analyzing convicted offender samples. By reviewing routine data rapidly and accurately, the system will allow analysts to focus on those samples with problems or "flags." Also, because DNA analysis requires considerable training and experience, software that interprets information with little or no work by the forensic analyst is a noteworthy advancement for the DNA community.

Rapid, Accurate Analysis

Our evaluation focused on the technical review of software programs that:

• Are publicly available for purchase.
• Are configurable, off-the-shelf software.
• Are housed in a laboratory.
• Can be used by people who are unfamiliar with computer code.
• Meet technical specifications outlined in the forensic standards.[3]

We evaluated three software packages that met these criteria:[4]

• GeneMapper^® ID v. 3.2 (Applied Biosystems, Foster City, Calif.).
• FSS-i³™ (Promega Corporation, Madison, Wis.), used with GeneMapper^® ID v. 3.2 for peak detection and sizing.
• TrueAllele^® System 2 (Cybergenetics, Pittsburgh).

We evaluated every step involved from buying the programs to running them. Our evaluation examined the time it takes to import data, run the application and complete the analysis. We also critically checked the flags, rules and features available with each program. We evaluated their ability to run without assistance by the analyst and to flag the samples needing human expert review. We further looked at the customer service and training provided by the vendors' technical staff.

Our team found that the three expert systems evaluated for single-source samples will help reduce the DNA convicted offender backlog.[5] We determined that each is able to rapidly and accurately conduct routine reviews (reviews of DNA samples without flags) and can significantly reduce the time spent in the human review of DNA profiles. When implementing one of these expert systems, only one analyst is needed to accept the good quality samples (with no expert system in place, two analysts are needed during a review), thereby reducing labor hours by half.

We determined, however, that the systems cannot fully replace a human expert or DNA analyst. The systems can be used as tools to evaluate single-source DNA samples and to alert the DNA analyst when results do not meet defined rules or there is a possible alternate judgment. But the final decision-making process in cases when results raise flags must remain in the hands of the trained DNA analyst.

As part of our evaluation, we also presented the features and limits of each system to help forensic analysts determine which program will most benefit their laboratories and to aid managers when deciding whether to purchase or subscribe to expensive software. Here is a brief summary of each expert system that we evaluated.

GeneMapper^® ID Software v. 3.2

We found the purchase of GeneMapper ID straightforward. The laboratory in which we evaluated the product purchases only software from the vendor; hardware is purchased separately. The consumer determines the number of licenses required by the laboratory and can self-install the software from a CD. The vendor provides a one-day on-site training on setup and basic software tools with purchases as well as technical support for telephone and electronic questions and free periodic webinars. More extensive training can be purchased. Our evaluation found many features in GeneMapper ID to be intuitive and the sizing and genotyping analysis to be simple and effective.

FSS-i³™ Expert Systems Software

With FSS-i³, hardware is purchased separately; the laboratory purchases only software from the vendor. The vendor offers on-site or company-site training at a separate cost. The five-day sessions include installation, optimization and training. The vendor also offers technical support for telephone and electronic questions. Overall, we found the operation of the software straightforward.

TrueAllele^® System 2

With TrueAllele, hardware is included with the software. The laboratory can renew its support and updates annually. The vendor provides two company-site training sessions: a two-day executive training session for laboratory managers and administrators and a four-day training session for scientists using the software. The vendor also provides technical support for telephone and electronic questions. During our evaluation, we found that we had a steeper learning curve for TrueAllele, but once we understood the setup, it was fairly straightforward and the analysis of the size standard and samples was simple and effective.

It is not the objective of the NEST project to identify the single best expert system for a laboratory. We recognize that crime laboratories are different, and each laboratory should consider its specific needs when choosing an expert system. Instead, the goal of the NEST project is to continue to give managers and analysts the tools they need to determine which expert system would best fit their laboratory and ultimately help reduce the backlog of convicted offender samples across the country so more crimes can be solved.

NIJ Journal No. 260, July 2008
NCJ 222905

About the Author

Rhonda Roby has been the NEST project technical director for NIJ since the project's inception in May 2005. For the past eight years, she has worked with software development teams and evaluated expert systems for forensic DNA analysis. Roby also conducts research for the Missing Persons Program at the University of North Texas Center for Human Identification, where she focuses on automation, development of assays and procedures, and mitochondrial DNA research. Back to the top.