|
Program History and Background
In 1986, the Centers for Disease Control and Prevention (CDC) implemented the Model Performance Evaluation Program (MPEP) to evaluate the performance of laboratories conducting testing to detect human immunodeficiency virus type 1 (HIV-1) antibody (Ab) by traditional testing methods (enzyme immunoassay, Western blot, etc.). Since that time, the MPEP has been expanded to include the evaluation of laboratory/testing site performance for HIV Ab by rapid testing methods and mycobacterium-related tests.
Program Purpose
High-quality HIV antibody rapid testing and Mycobacterium Tb drug susceptibility testing are essential to meeting the public health objectives for the prevention and control of these diseases. Prevention programs, diagnostic clinics, and seroprevalence studies rely on accurate antibody testing results to document HIV infection. Clinicians rely on accurate Mycobacterium Tb drug susceptibility testing to diagnose infection and develop appropriate treatment regimens. The impetus for developing this program came from the recognized need to assess the quality of retroviral/AIDS-related and Mycobacterium-related laboratory testing and to ensure that the quality of testing adequately met medical and public health needs. The objectives of the MPEP are to:
(1) Develop appropriate methods for evaluating quality in laboratory testing systems (including test selection, sample collection, and reporting and interpreting test results)
(2) Develop strategies for identifying and correcting testing quality failures
(3) Evaluate the effect of testing quality on public health.
Program Design
The MPEP is designed to analyze the steps in the total testing process and to identify the critical indicators of high-quality testing. Annually, the MPEP mails two performance evaluation sample panels each for HIV rapid testing, and Mycobacterium Tb drug susceptibility testing. Participant laboratories test the performance evaluation (PE) samples that CDC mails them in the same manner they test routine clinical specimens, and to report their testing results to CDC on specially designed forms. Aggregate data are derived from the testing results of the various methods provided to CDC by each of the laboratories. CDC compiles and analyzes the results and sends aggregate reports of testing results to all participant laboratories following each PE survey. In addition to sample panel surveys, participant laboratories complete a survey questionnaire every other year describing the characteristics of their laboratory and their testing practices.
The MPEP is not proficiency testing (PT). The MPEP provides detailed analysis of aggregate data and does not grade results from individual laboratories. Graded laboratory proficiency testing surveys are conducted, for example, by the College of American Pathologists (CAP) and the American Association of Bioanalysts (AAB). Of the approximately 850 laboratories enrolled in the MPEP for HIV-1 Ab testing, nearly 80% are also enrolled in programs sponsored by CAP, AAB, or other PT providers.
Participation
The PE surveys are provided free of charge to participant laboratories. The PE surveys depend upon a wide spectrum of laboratory participation to generate a representative data base. To accurately assess the quality of laboratory testing, we need the participation of laboratories representing all levels of performance. Past CDC experience shows that participation in PE surveys leads to improved testing performance. By encouraging participation in the PE surveys, CDC gives participant laboratories an opportunity for self-improvement and, in addition, a basis for developing prevention and intervention strategies. Interest in the program is quite high; approximately 90 percent of the laboratories receiving sample panels respond with results. This high rate of participation indicates strong support for achieving the goal of improving and maintaining high quality laboratory testing.
The benefits to participants include:
-
assuring accurate tests are being provided by the laboratory/testing site through external quality assessment
-
improving testing quality through self-evaluation in a nonregulatory enrvironment
-
testing well characterized samples from a source outside the test kit manufacturer
-
discovering potential testing problems so that procedures can be adjusted to eliminate them
-
comparison of testing results with others at a national and international level
-
ability to consult with CDC staff to discuss testing issues.
Recruitment for volunteer enrollment in the MPEP is ongoing by CDC (to date more than 20,000 laboratories/testing sites have been contacted). Not all of the laboratories/testing sites contacted perform the tests evaluated by the MPEP but of those that do perform these tests and participate in the MPEP, approximately, 750 perform HIV rapid testing, and 165 perform Mycobacterium drug susceptibility testing.
Reports
The reports sent to participant laboratories following each PE evaluation survey consist of tables and graphical figures, grouped for each performance evaluation sample, by test kit manufacturer, test method, WB band patterns, and IFA intensity, for example. A written analysis of results is included in each final report. The aggregate reports are also available on the World Wide Web at: http://wwwn.cdc.gov/mpep/default.aspx. Although the MPEP is not a regulatory program, laboratories find the aggregate reports of testing results beneficial in comparing their results with those from other laboratories. This practice affords each laboratory the opportunity for self-improvement and serves as a vehicle to accomplish an important objective of the MPEP: improving and maintaining high quality testing. Periodic brief reports highlighting important findings are sent to MPEP participants, usually after aggregate reports have been distributed. Reprints from CDC Morbidity and Mortality Weekly Report (MMWR) publications and from CDC authored peer-reviewed journal publications concerning retroviral or AIDS-related testing are also sent to participant laboratories.
Assessing Preanalytic and Post analytical Processes
To identify and assess barriers to high-quality laboratory testing in preanalytic and postanalytic steps of the total testing process for HIV and AIDS-related testing, CDC collaborates with the Association of Schools of Public Health to develop systematic analyses for identifying variables in the steps of the testing process. These analyses are used to assist in cataloging events that occur from the time tests are requested through specimen collection, laboratory analyses, and reporting of test results so that potential problems, particularly in the preanalytic and postanalytic steps of the testing process, can be identified and corrected. Some of these analyses include:
-
Identification of nonanalytic issues in HIV-1 antibody testing using blind proficiency testing.
-
Use of a nominal group process for improving the performance of laboratories conducting HIV antibody testing
-
Personnel competency assessments.
MPEP Focus on Laboratory Practices
Through PE survey data and ongoing research, the MPEP evaluates laboratory practices linked to high quality performance. The focus is on answering such questions as the following:
-
What are the testing practices and characteristics of laboratories that perform HIV- or Mycobacterium-related testing?
-
Do these practices affect quality in the laboratory testing process?
-
Is the high quality of laboratory testing dependent upon the nature of the laboratory itself, e.g., public health versus blood collection center versus hospital-based laboratory versus independent laboratory, numbers of samples tested, types and sequence of tests performed, internal and external quality assurance procedures implemented throughout the testing process, education and training of testing personnel, or experience of personnel performing the testing?
-
What strategies for correcting quality failures are most effective and how can they be implemented?
MPEP Products
Currently, we provide the following:
-
Reports to participants: These reports contain graphical figures and analyses of the PE sample survey and questionnaire results, which are useful indicators in detecting performance deficiencies and quality results. These reports can be found at: http://wwwn.cdc.gov/mpep/default.aspx.
-
Publications and presentations: MPEP Performance Bulletins addressing specific testing performance issues, MMWR articles describing specific aspects of performance results, and peer-reviewed journal publications directly addressing testing issues. A list of publications can be found at: http://wwwn.cdc.gov/mpep/rpubs.aspx. A list of presentations can be found at: http://wwwn.cdc.gov/mpep/rpres.aspx.
-
Statistical analyses: Examination of participant laboratory data to develop a profile of the characteristics which distinguish a laboratory testing process that performs consistently well from one that performs poorly. This information is essential for targeting strategies for improving the quality of laboratory testing.
Future Plans
Future plans of the MPEP include:
(1) Refinements in existing programs, such as performance panels to evaluate new or changing methods in performing HIV rapid testing, and use of multi-drug resistant Mycobacterium strains.
(2) Provision of improved online performance data submission
(3) Expansion of the survey design to broaden our knowledge about the level of quality in the preanalytic and postanalytic steps of the testing process
(4) Recommendations for intervention strategies for laboratory improvement
(5) Serve as a Center of Excellence for External Quality Assurance (EQA) activities, providing EQA consultative services to laboratories/testing sites within and outside the United States
|