Translating Evidence into Recommendations

Slide Presentation from the AHRQ 2008 Annual Conference

---

On September 8, 2008, Tess Miller, Dr.P.H., made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (825 KB).

---

Slide 1

Translating Evidence into Recommendations

GRADE Lunch & Learn
AHRQ Annual Meeting
September 8, 2008
Tess Miller, Dr.P.H.

Slide 2

Be Transparent

• In how you collect & synthesize the evidence.
• In how you judge the evidence and make a recommendation.

Slide 3

Separate Roles

• One group collects & synthesizes the evidence.
• One group judges the evidence & makes a recommendation.

Slide 4

Provide Clear Instruction

• Purpose of the guideline/ recommendation.
  • Standardize practice; improve clinical outcomes; etc.
• Framework for Reviewing the Evidence.
  • PICO (Patient population; Intervention; Comparison; Outcomes.
  • Analytic Framework.

Slide 5

USPSTF Analytic Framework on Screening for a Disease

A diagram presenting the framework in screening for a disease.

• Persons at Risk.
• Screening.
  • Adverse Effects of Screening.
• Early Detection of Target Condition.
• Treatment.
  • Adverse Effects of Treatment.
• Intermediate Outcome.
• Association.
• Reduced Morbidity and/or Mortality.

Notes:

• The task force first develops an analytic framework for a topic. The framework not only provides a graphical depiction of the process, it makes explicit a number of essential considerations:
  1. The target population, intervention, comparison and most important outcomes.
  2. It depicts intermediate outcomes that may mediate the effects on or serve as surrogates for the more distal outcome.
  3. It identifies discrete questions which can be answered through a review of the evidence.
  4. It explicitly depicts downstream consequences of the initial intervention and possible subsequent interventions - the adverse as well as beneficial effects, and all of these together determine the overall balance of benefits and harms.
  5. Finally, it distinguishes studies which provide a direct link between an intervention and an outcome (top arrow) from less direct linkages between screening and reduced morbidity.
• From this analytic framework the task force develops key questions.

Slide 6

Judging the Strength of Evidence

• Guideline developers should clarify method(s) & language for judging the strength of the evidence.
• Guideline developers should consider the end user:
  • Who is it.
  • How will they use the guideline.
  • "Translate" the strength/quality of the evidence.

Slide 7

A sample table showing the "U.S. Preventive Services Task Force terminology to describe the critical assessment of evidence at 3 levels: individual studies, key questions, and overall certainty of net benefit of the Preventive Service."

Slide 8

A sample table showing the "Questions considered by the U.S. Preventive Services Task Force for evaluating evidence related both to key questions and to the overall certainty of the evidence of net benefit for the Preventive Service."

Slide 9

A sample table showing the "U.S. Preventive Services Task Force Recommendation Grid.*"

Slide 10

Translation Table for Assessing the Strength of a Body of Evidence

Evidence of Effectiveness	Quality of Execution	Design Suitability	Number of Studies	Consistent	Effect Size
1. Strong	Good	Greatest	?2	Yes	Sufficient
	Good	Greatest or Moderate	?5	Yes	Sufficient
	Good or Fair	Greatest	?5	Yes	Sufficient
	Meet criteria for sufficient evidence				Large
2. Sufficient	Good	Greatest	1	—	Sufficient
	Good or Fair	Greatest or Moderate	?3	Yes	Sufficient
	Good or Fair	Greatest, Moderate or Least	?5	Yes	Sufficient
3. Insufficient	Insufficient design or execution		Too few	No	Small

Notes:

• I won't review the algorithm for drawing conclusions in detail (it is presented in the AJPM supplement I discussed earlier) but this is it.
• The important point about this algorithm is that there are many different pathways to drawing conclusions about a body of evidence.
• For example, sufficient evidence can reflect either a relatively small number of very well designed and executed studies or a larger number of studies with lesser suitability of design or more limited quality of execution.
• Recommendations of the Task Force usually emerge directly from the strength of the evidence. For example, a strong body of evidence of effectiveness leads to a strong recommendation for the intervention. Likewise, evidence that the intervention is ineffective would lead to a recommendation against using that intervention.
• There are exceptions to these rules which I will discuss momentarily.
• To date, more than 70 recommendations have been made in diverse public health areas, and the resulting conclusions seem to have generally had good face validity.

Slide 11

Consider Computer Actionability

• Develop standards to use when there is no evidence.
  • Make no statement.
  • Consensus approach.

Slide 12

• Remember Transparency.
• Thank you.

Current as of January 2009

---

Internet Citation:

Translating Evidence into Recommendations. Slide Presentation from the AHRQ 2008 Annual Conference (Text Version). January 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/annualmtg08/090808slides/Miller.htm

---