Appendix 2. Study Quality Rating Criteria

Diagnostic Accuracy Studies

Criteria:

Definition of ratings based on above criteria:

Good: Evaluates relevant, available screening test; uses a credible reference standard; interprets reference standard independently of screening test; reliability of test assessed; has few or handles indeterminate results in a reasonable manner; includes large number (> 100) broad-spectrum patients with and without disease.

Fair: Evaluates relevant available screening test; uses reasonable although not the best standard; interprets reference standard independent of screening test; moderate sample size (50-100 patients) and a "medium" spectrum of patients.

Poor: Has important limitations such as: uses inappropriate reference standard; screening test improperly administered; biased ascertainment of reference standard; small sample size of narrow selected spectrum of patients.

Randomized Controlled Trials (RCTs) and Cohort Studies

Criteria:

Definition of ratings based on above criteria:

Good: Meets all criteria: comparable groups are assembled initially and maintained throughout the study (followup > 80%); reliable and valid measurement instruments are used and applied equally to the groups; interventions are spelled out clearly; important outcomes are considered; and appropriate attention to confounders in analysis.

Fair: Studies will be graded "fair" if any or all of the following problems occur, without the important limitations noted in the "poor" category below: generally comparable groups are assembled initially, but some question remains whether some (although not major) differences occurred in followup; measurement instruments are acceptable (although not the best) and generally applied equally; some but not all important outcomes are considered; and some but not all potential confounders are accounted for.

Poor: Studies will be graded "poor" if any of the following major limitations exist: groups assembled initially are not close to being comparable or are not maintained throughout the study; unreliable or invalid measurement instruments are used or not applied at all equally among groups (including not masking outcome assessment); and key confounders are given little or no attention.

Case Control Studies

Criteria:

Definition of ratings based on above criteria:

Good: Appropriate ascertainment of cases and nonbiased selection of case and control participants; exclusion criteria applied equally to cases and controls; response rate equal to or greater than 80%; diagnostic procedures and measurements accurate and applied equally to cases and controls; and appropriate attention to confounding variables.

Fair: Recent, relevant, without major apparent selection or diagnostic work-up bias, but with response rate less than 80% or attention to some but not all important confounding variables.

Poor: Major selection or diagnostic work-up biases, response rates less than 50%, or inattention to confounding variables.

Return to Document