Appendix 1

Diagnostic Accuracy Studies

Criteria

1. Screening test relevant, available for primary care, adequately described.
2. Credible reference standard, performed regardless of test results.
3. Reference standard interpreted independently of screening test.
4. Indeterminate results handled in a reasonable manner.
5. Spectrum of patients included in study.
6. Sample size.
7. Administration of reliable screening test.

Definition of Ratings Based on Above Criteria

Good: Evaluates relevant available screening test; uses a credible reference standard; interprets reference standard independently of screening test; assesses reliability of test; has few or handles indeterminate results in a reasonable manner; includes large number (>100) broad-spectrum patients with and without disease.

Fair: Evaluates relevant available screening test; uses reasonable although not best standard; interprets reference standard independently of screening test; has moderate sample size (50 to 100 participants), and includes a "medium" spectrum of patients.

Poor: Has important limitations, such as inappropriate reference standard, improperly administered screening test, biased ascertainment of reference standard, or very small sample size of very narrow selected spectrum of patients.

Randomized, Controlled Trials and Cohort Studies

Criteria

1. Initial assembly of comparable groups: randomized, controlled trials—adequate randomization, including concealment and statement of whether potential confounders were distributed equally among groups; cohort studies—consideration of potential confounders with either restriction or measurement for adjustment in the analysis; consideration of inception cohorts.
2. Maintenance of comparable groups (includes attrition, crossovers, adherence, and contamination).
3. Important differential loss to followup or overall high loss to followup.
4. Measurements: equal, reliable, and valid (includes masking of outcome assessment).
5. Clear definition of interventions.
6. Important outcomes considered.
7. Analysis: adjustment for potential confounders for cohort studies, or intention-to-treat analysis for randomized, controlled trials.

Definition of Ratings Based on Above Criteria

Good: Meets all criteria—comparable groups are assembled initially and maintained throughout the study (followup >80%), reliable and valid measurement instruments are used and applied equally to the groups, interventions are spelled out clearly, important outcomes are considered, and appropriate attention to confounders in analysis.

Fair: Studies will be graded "fair" if any or all of the following problems occur, without the important limitations noted in the "poor" category below: Generally comparable groups are assembled initially but some question remains as to whether some (although not major) differences occurred in followup, measurement instruments are acceptable (although not the best) and generally applied equally, some but not all important outcomes are considered, and some but not all potential confounders are accounted for.

Poor: Studies will be graded "poor" if any of the following major limitations exists: Groups assembled initially are not close to being comparable or maintained throughout the study, unreliable or invalid measurement instruments are used or not applied at all equally among groups (including failure to mask outcome assessment), and key confounders are given little or no attention.

Case-Control Studies

Criteria

1. Accurate ascertainment of cases.
2. Nonbiased selection of case-patients and controls, with exclusion criteria applied equally to both.
3. Response rate.
4. Diagnostic testing procedures applied equally to each group.
5. Measurement of exposure accurate and applied equally to each group.
6. Appropriate attention to potential confounding variable.

Definition of Ratings Based on Above Criteria

Good: Appropriate ascertainment of cases and nonbiased selection of case-patients and controls, exclusion criteria applied equally to case-patients and controls, response rate of 80% or greater, diagnostic procedures and measurements accurate and applied equally to case-patients and controls, and appropriate attention to confounding variables.

Fair: Recent, relevant, without major apparent selection or diagnostic work-up bias but with response rate less than 80% or attention to some but not all important confounding variables.

Poor: Major selection or diagnostic work-up biases, response rates less than 50%, or inattention to confounding variables.

---

Return to Contents
Proceed to Next Section