Skip Navigation
Department of Health and Human Services www.hhs.gov
 
Slide Tray
0 slides

Return to Slide Library

Slides

Add Search Results to Slide Tray Search:

"directness"

Slides: 1–12 of 35

Rating the Strength of Evidence From the CER

Defining Applicability. Applicability is defined as the extent to which the results observed in published studies are likely to reflect the expected outcomes when an intervention is applied to broader populations under real-world conditions. Similar terms include external validity, generalizability, directness, and relevance.

Defining Applicability

Judge Applicability and Strength of Evidence Separately. Applicability is best reported separately from quality or strength of evidence. Internal validity concerns are not altered by changes in perspective, but applicability is. Some evidence hierarchies (GRADE) consider applicability “or directness” when assessing the quality of evidence. For systematic reviews in which multiple perspectives (clinician, patient, policymaker) are considered, applicability should be assessed separately.

Judge Applicability and Strength of Evidence Separately

Consider the Contribution of an Individual Study to the Body of Evidence. This slide shows how characteristics of the individual study relate to domains evaluating the strength of the body of evidence. Each characteristic of the individual study is linked by an arrow to a domain for the strength of the body of evidence. The slide shows that the quality or internal validity of a study (i.e., limitations of study design and conduct) is one of several ways in which an individual study can contribute to the body of evidence. When evaluating the strength of the entire body of evidence, the quality of the study and the type of design influence the evaluation of the risk of bias for the entire body of evidence. The direction and magnitude of the results influence considerations of consistency of the body of evidence. The size of the study could result in nonsystematic or random error; it influences considerations of precision of the body of evidence. The relevance of the results to the key questions influences the evaluation of directness of the body of evidence. The relevance of the study sample to the population(s) of interest influences the assessment of applicability or generalizability of the body of evidence.

Consider the Contribution of an Individual Study to the Body of Evidence

This slide includes a table with two columns and three rows. Column 1, row 1: high. Column 2, row 2: High confidence that the evidence reflects the true effect. Further research is very unlikely to change our confidence in the estimate of effect. Column 1, row 2: moderate. Column 2, row 2: Moderate confidence that the evidence reflects the true effect. Further research may change our confidence in the estimate of effect and may change the estimate. Column 1, row 3: low. Column 2, row 3: Low confidence that the evidence reflects the true effect. Further research is likely to change our confidence in the estimate of effect and is likely to change the estimate. Column 1, row 4: insufficient. Column 2, row 4: Evidence is either unavailable or does not permit estimation of an effect. There is a footnote at the bottom that includes the following references: Guyatt GH, et al. BMJ 2008;336:924-6; Owens DK, et al. J Clin Epidemiol 2010;63:513-23; Samson  DJ, et al. AHRQ Comparative Effectiveness Review No. 20. Available at: http://www.effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?pageaction=displayproduct&productID=447.

Rating the Strength of Evidence From the CER: A Modification of the GRADE Methodology

Four Domains Used To Assess Relevant Studies
The following four major domains were examined: risk of bias (low, medium, high), consistency (no inconsistency, inconsistency present, unknown, or not applicable), directness (direct, indirect), and precision (precise, imprecise). Each key outcome on each comparison of interest was given an overall evidence grade based on the ratings for the individual domains.
Four Domains Used To Assess Relevant Studies
The following four major domains were examined: risk of bias (low, medium, high), consistency (no inconsistency, inconsistency present, unknown, or not applicable), directness (direct, indirect), and precision (precise, imprecise). Four of the outcomes (health-related quality of life, shoulder function, cuff integrity and time to return to work) for each comparison of interest were given an overall evidence grade based on the ratings for the individual domains.
Pages: [1] 2 3 Next