Skip Navigation
 
Home | About CDC | Media Relations | A-Z Index | Contact Us
   
Centers for Disease Control & Prevention
CDC en Español 
Sexually Transmitted Diseases

ChlamydiaScreening Tests To Detect Chlamydia trachomatis and Neisseria gonorrhoeae Infections - 2002


Appendix C

Conducting Studies To Evaluate Chlamydia trachomatis and Neisseria gonorrhoeae Tests, Including Studies Required by the Regulations of the Clinical Laboratory Improvement Amendments of 1988 for Verifying or Establishing Test Performance Characteristics

When a test is a candidate to replace an established test, the sensitivities and specificities of the candidate and established tests should be compared. Recommendations for the conduct of test evaluation studies have been reviewed elsewhere (C-1–C-4). The imperfect sensitivity and specificity of reference standards complicate evaluation of C. trachomatis and N. gonorrhoeae tests. Discrepant analysis has been used universally to address this problem in the past. However, this approach has been criticized because of the likelihood of producing biased estimates of test characteristics (C-5–C-8). Key to advancing the evaluation of C. trachomatis and N. gonorrhoeae tests is the introduction of study designs that compare candidate and established tests in the same study by using an independent reference standard or by using approaches that do not require designation of a reference standard. Investigation of such designs for evaluation of C. trachomatis tests has begun (C-6, C-9–C-15).

The Clinical Laboratory Improvement Amendments of 1988 (CLIA) requires that a clinical laboratory verify or establish the performance of a test to be used in the medical care of persons when it is introduced into that laboratory (C-16). Verification of performance under CLIA applies to the introduction of tests that are Food and Drug Administration (FDA)-cleared, and the CLIA requirement is that the laboratory demonstrate that test performance (i.e., accuracy, precision, reportable range of results, and appropriate normal values) in that laboratory be equivalent to the performance claimed by the manufacturer in the FDA-cleared package insert. Establishing performance under CLIA applies to tests or modifications of tests that have not been FDA-cleared for a particular use. When a modification or use of a test has not been FDA-cleared, CLIA requires that the laboratory establish the performance characteristics of the test for that use. In addition to the performance specifications noted previously, establishing performance must include determinations of analytical sensitivity and specificity, to include interfering substances.

Such verification or establishment studies are relatively straightforward when the candidate test performs similarly to the established test. In such cases, the laboratory can verify or establish performance by using the established test as the reference standard if discrepancies between the candidate and established test are uncommon. If the candidate test performs differently from the established test, the laboratory is faced with the economic and logistical challenge of verifying or establishing the performance of the candidate test by introducing an independent reference standard. If the laboratory lacks an independent reference standard, a reference laboratory with an independent reference standard might be enlisted to 1) blindly test a suitable sample of verification or establishment study specimens with the reference standard tests or 2) supply a panel of specimens previously characterized by the reference laboratory. The reader is referred to the American Society for Microbiology CUMITECH series for detailed guidance regarding conduct of verification and establishment studies (C-17).

References
C-1. Irwig L, Tosteson ANA, Gatsonis C, et al. Guidelines for meta-analyses evaluating diagnostic tests. Ann Intern Med 1994;120:667–76.
C-2. Noe DA. Evaluation of the medical utility of classification studies [Chapter 4]. In: Noe DA, Rock RC, Grochow LB, Kahn SB, Perlin E, eds. Laboratory medicine: the selection and interpretation of clinical laboratory studies. Baltimore, MD: Williams & Wilkins, 1994:44–52.
C-3. Noe DA. Diagnostic classification [Chapter 3]. In: Noe DA, Rock RC, Grochow LB, Kahn SB, Perlin E, eds. Laboratory medicine: the selection and interpretation of clinical laboratory studies. Baltimore, MD: Williams & Wilkins, 1994:27–43.
C-4. Reid MC, Lachs MS, Feinstein AR. Use of methodological standards in diagnostic test research: getting better but still not good. JAMA 1995;274:645–51.
C-5. Hadgu A. Bias in the evaluation of DNA-amplification tests for detecting Chlamydia trachomatis. Statist Med 1997;16:1391–9.
C-6. Miller WC. Bias in discrepant analysis: when two wrongs don't make a right. J Clin Epidemiol 1998;51:219–31.
C-7. Lipman HB, Astles JR. Quantifying the bias associated with use of discrepant analysis. Clin Chem 1998;44:108–15.
C-8. McAdam AJ. Discrepant analysis: how can we test a test? J Clin Microbiol 2000;38:2027–9.
C-9. Alonzo TA, Pepe MS. Using a combination of reference tests to assess the accuracy of a new diagnostic test. Statist Med 1999;18:2987–3003.
C-10. Miller WC. Can we do better than discrepant analysis for new diagnostic test evaluation? [Editorial response]. Clin Infect Dis 1998;27:1186–93.
C-11. Johnson RE, Green TA, Schachter J, et al. Evaluation of nucleic acid amplification tests as reference tests for Chlamydia trachomatis infections in asymptomatic men. J Clin Microbiol 2000;38:4382–6.
C-12. Hadgu A, Qu Y. Biomedical application of latent class models with random effects. Appl Statist 1998;47(Pt 4):603–16.
C-13. Qu Y, Hadgu A. Model for evaluating sensitivity and specificity for correlated diagnostic tests in efficacy studies with an imperfect reference test. Journal of the American Statistical Association 1998;93:920–8.
C-14. Cheng H, Macaluso M, Vermund SH, Hook EW III. Relative accuracy of nucleic acid amplification tests and culture in detecting Chlamydia in asymptomatic men. J Clin Microbiol 2001;39:3927–37.
C-15. Black CM, Marrazzo J, Johnson RE, et al. Head-to-head multicenter comparison of DNA probe and nucleic acid amplification tests for Chlamydia trachomatis in women with use of an improved reference standard. J Clin Microbiol (in press).
C-16. Code of Federal Regulations. Title 42, Chapter IV, part 493—Laboratory requirements.
C-17. Elder BL, Hansen SA, Kellogg JA, Marsik FJ, Zabransky RJ. Verification and validation of procedures in the clinical microbiology laboratory [31]. In: McCurdy BW, ed. CUMITECH: cumulative techniques and procedures in clinical microbiology. Washington, DC: American Society for Microbiology, 1997.