U.S. Preventive Services Task Force

Slide Presentation from the AHRQ 2008 Annual Conference

---

On September 8, 2008, Kenneth Lin, M.D., Diana Petitti, M.D., M.P.H., and Tracy Wolff, M.D., M.P.H., made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (6.6 MB).

---

Slide 1

U.S. Preventive Services Task Force (USPSTF)

Kenneth Lin, M.D., Medical Officer, Agency for Healthcare Research and Quality (AHRQ)
Diana Petitti, M.D., M.P.H., Vice Chair USPSTF
Tracy Wolff, M.D., M.P.H., Medical Officer AHRQ

Slide 2

U.S. Preventive Services Task Force

• Independent panel of experts in primary care and prevention, multidisciplinary.
• Systematically reviews evidence for clinical preventive services implemented in a primary care setting.
• Makes recommendations on clinical preventive services in populations without recognized signs or symptoms of illness.
• AHRQ is mandated to convene and support USPSTF.
• Scientific support from Evidence-Based Practice Centers.
• Liaisons from primary care subspecialty societies and Federal agencies.

Notes:

• Presenter: Tracy.

Slide 3

Target Audiences

• Primary Care Clinicians and the Systems in which they function (including other clinicians).
• Academicians and Researchers.
• Quality Improvement Professionals and makers of tools that affect primary care practice.
• Health Care Policymakers and System Leaders.
• Employers and other Healthcare Purchasers.
• Members of the Public.

Slide 4

History of the Task Forces

The slide shows an image of a golden "20" hanging from a green ribbon with the words, U.S. Preventive Services Task Force 20th Anniversary 1964-2004 written.

• 1976—Canadian Task Force on PHE [Periodic Health Evaluation].
• 1984—USPSTF established by PHS [Public Health Service].
• 1996—Community Task Force.
• 1998—3rd USPSTF reconvened by AHRQ.
• 2001—Standing USPSTF Task Force.

Notes:

• Where did the Task Force come from?
• (Read above) PHE = periodic health exam.

Slide 5

Structure of USPSTF

The diagram shows how USPSTF works together with AHRQ and the Evidence-based Practice Centers (EPCs).

• AHRQ:
  • Convenes Administrative, research, and technical support for USPSTF.
  • Contracts to synthesize evidence for EPC.
• USPSTF and the EPCs:
  • Analytic Framework Development goes from USPSTF to EPC.
  • Evidence Presentation goes from EPC to USPSTF.
• USPSTF:
  • Makes Recommendations.

Notes:

• Here is a graphic that explains how AHRQ and the Task Force work together. The Task force is an independent body that receives technical support from AHRQ, a Federal agency. The Evidence-based Practice Centers (EPCs) also provide technical support to AHRQ and the Task Force through systematic reviews of the evidence.

Slide 6

Task Force Activities

• Provide evidence-based scientific reviews of preventive health services for use in primary healthcare delivery settings.
• Age- and risk-factor specific recommendations for routine practice.
• Primary and Secondary Prevention Recommendations:
  • Screening tests.
  • Counseling.
  • Preventive medications.

Slide 7

Recommendations Released in 2007 and 2008

• Aspirin/Nonsteroidal Anti-inflammatory Drugs (ASA/NSAIDs) to Prevent Colorectal Cancer.
• Chlamydia: Screening.
• Carotid Artery Stenosis: Screening.
• Hypertension (HTN) in Adults.
• Lipid Disorders in Children.
• Motor Vehicle Occupant Injuries: Counseling.
• Sickle Cell Disease in Newborns: Screening.
• Prostate Cancer.
• Asymptomatic Bacteruria: Screening.
• Bacterial Vaginosis (BV) in Pregnancy.
• Congenital Hypothyroidism.
• Chronic Obstructive Pulmonary Disease (COPD) Screening.
• Diabetes Type II: Screening.
• Gestational Diabetes Mellitus (DM): Screening.
• Newborn Hearing: Screening.
• Phenylketonuria (PKU): Screening.
• Adult Lipids: Screening.

Slide 8

USPSTF Topics in Progress

• ASA to prevent cardiovascular disease (CVD).
• Breast cancer—screening & preventive medication (PM).
• Breastfeeding.
• Cervical cancer screening.
• Colorectal cancer screening.
• Coronary heart disease (CHD)—risk factor screening.
• Dementia.
• Depression screening.
• Falls in the elderly
• Oral cancer screening.
• Tobacco counseling.
• Hepatitis B screening.
• Folic Acid for Neural Tube Defect Prevention.
• Hyperbilirubinemia—newborn screening.
• Lung Cancer.
• Multivitamins and supplements.
• Obesity.
• Osteoporosis—Screening.
• Physical Activity.
• Skin cancer—Screening.
• Sexually Transmitted Infections (STI)—counseling.
• Vision in Older Adults.

Slide 9

What's new?

• Updating previous recommendations.
• Addressing geriatric and child health recommendations.
• Federal Register notice for new topic nominations.
• Implementation—
  • Tools.
    • Pocket guide.
    • PDA.
    • Web site.
• New recommendation statement format.

Notes:

• Epc do methods piece on updating.
• AHRQ working on an in-house process.
• Rubber stamping vs. archive.
• Staged reviews.
• With the Centers for Disease Control & Prevention (CDC): HIV.
• Childhood obesity.
• Newborn screening?

Slide 10

Examples of USPSTF Resources

• Annual Pocket Guide to Clinical Preventive Services.
• One-page clinical summary of Recommendation Statement (RS).
• Adult Preventive Services timeline.
• ePSS [Electronic Preventive Services Selector].
• Publication of Recommendations in academic journals—Annals of Internal Medicine, Pediatrics.
• Partnerships with professional societies, ePocrates, Medscape.
• Patient brochures.

Slide 11

Employers and Policy Makers

• A Purchaser's Guide to Clinical Preventive Services—with the National Business Group on Health (NBGH) and CDC.
• Employer's Guide to Health Improvement and Preventive Services—with NBGH and Robert Wood Johnson Foundation.

Slide 12

• http://www.preventiveservices.ahrq.gov.
• Please visit our booth in the mAHRQet Place Café for examples of USPSTF resources.
• Please also attend: Session #66 USPSTF Making a Difference in Clinical Care—Tues, Sept. 9th 10-1130 AM.

Slide 13

Evidence and the USPSTF

Slide 14

Steps in the Recommendation Development Process.

1. Define questions and outcomes of interest using analytic framework.
2. Define and retrieve relevant evidence.
3. Evaluate quality of individual studies.
4. Synthesize and judge strength of overall evidence and draw conclusion about certainty.
5. Determine balance of benefits and harms.
6. Link recommendation to magnitude and certainty of net benefits.

Notes:

• Here are the steps the Task Force follows in its development of recommendations. I am going to go through these steps individually. The first step is defining the question and the outcome of interest for a topic (for example, prostate cancer).

Slide 15

Step 1: Analytic Framework on Screening for a Disease

The diagram presents the framework on screening for a disease.

• Persons at Risk.
• Screening.
  • Adverse Effects of Screening.
• Early Detection of Target Condition.
• Treatment.
  • Adverse Effects of Treatment.
• Intermediate Outcome.
• Association.
• Reduced Morbidity and/or Mortality.

Notes:

• The Task Force first develops an analytic framework for a topic. The framework not only provides a graphical depiction of the process, it makes explicit a number of essential considerations:
  1. The target population, intervention, comparison and most important outcomes.
  2. It depicts intermediate outcomes that may mediate the effects on or serve as surrogates for the more distal outcome.
  3. It identifies discrete questions which can be answered through a review of the evidence.
  4. It explicitly depicts downstream consequences of the initial intervention and possible subsequent interventions—the adverse as well as beneficial effects, and all of these together determine the overall balance of benefits and harms.
  5. Finally, it distinguishes studies which provide a direct link between an intervention and an outcome (top arrow) from less direct linkages between screening and reduced morbidity
• From this analytic framework the Task Force develops key questions.

Slide 16

Example: Analytic Framework for Prostate Cancer Screening

The diagram presents the framework on prostate cancer screening.

• Asymptomatic Men.
• Screen with prostate-specific antigen (PSA), digital rectal examination (DRE).
  • Adverse effects of screening:
    • False positive.
    • False negative.
    • Inconvenience.
    • Labeling.
• Early Prostate Cancer.
• Treat with radiation, prostatectomy.
  • Adverse effects of Rx:
    • Impotence.
    • Incontinence.
    • Death.
    • Overtreatment.
• Reduced prostate cancer morbidity, mortality.

Notes:

• Here is one example of an analytic framework for prostate cancer that includes screening of asymptomatic men with psa/dre to detect early prostate cancer, and adverse effects. Note that adverse effects can be associated with the screening process as well as harms from treatment.

Slide 17

Steps in the Recommendation Development Process

1. Define questions and outcomes of interest using analytic framework.
2. Define and retrieve relevant evidence.
3. Evaluate quality of individual studies.
4. Synthesize and judge strength of overall evidence and draw conclusion about certainty.
5. Determine balance of benefits and harms.
6. Link recommendation to magnitude and certainty of net benefits.

Notes:

• The next step is to define and retrieve relevant evidence.

Slide 18

Step 2: Define & Retrieve Relevant Evidence

• Create inclusion/exclusion criteria based on the key questions from the analytic framework.
  • Interventions (e.g. screening, counseling, meds).
    • Outcomes.
    • Populations.
    • Setting (generalizable to primary care).
    • Time period.
    • Types of studies.
• Sources of evidence.
  • PubMed, Cochrane, other database searches.
  • "Reference mining."
  • Hand searching topic-relevant specialty journals.
  • Recommendations from experts.

Notes:

• Steps in the process of defining and retrieving relevant evidence are (read above). Inclusion and exclusion criteria should be established before beginning the review.
• The goal of this step is to obtain quality evidence about the amount of harm and the amount of benefit for screening (or chemoprophylaxis or counseling) for a condition.

Slide 19

Steps in the Recommendation Development Process

1. Define questions and outcomes of interest using analytic framework.
2. Define and retrieve relevant evidence.
3. Evaluate quality of individual studies.
4. Synthesize and judge strength of overall evidence and make conclusion about certainty.
5. Determine balance of benefits and harms.
6. Link recommendation to magnitude and certainty of net benefits.

Notes:

• The next step is to evaluate the quality of individual studies.

Slide 20

Step 3: Evaluate Quality of Individual Studies

• Good:
  • Evaluates relevant available screening tests.
  • Uses a credible reference standard.
  • Interprets reference standard independently of screening test.
  • Large sample size, ~100 broad spectrum patients.
• Fair:
  • Evaluates relevant available screening tests.
  • Uses reasonable although not best standard;
  • Interprets reference standard independent of screening test;
  • Moderate sample size, ~50-100 "medium" spectrum patients.
• Poor: Has fatal flaw such as:
  • Uses inappropriate reference standard.
  • Screening test improperly administered.
  • Biased ascertainment of reference standard.
  • Very small sample size or very narrow selected spectrum of patients.

Notes:

• The Task Force uses a good, fair, poor scale for individual studies (read above).

Slide 21

Steps in the Recommendation Development Process

1. Define questions and outcomes of interest using analytic framework.
2. Define and retrieve relevant evidence.
3. Evaluate quality of individual studies.
4. Synthesize and judge strength of overall evidence and make conclusion about certainty.
5. Determine balance of benefits and harms.
6. Link recommendation to magnitude and certainty of net benefits.

Notes:

• The next step is to synthesize and judge the strength of the overall evidence.

Slide 22

Step 4: Synthesize and Judge Strength of Overall Evidence

• Evidence reports.
  • Evidence tables summarizing studies.
  • Narrative discussing overall strength of evidence.
• Meta-analysis.
• Modeling.
  • Decision analysis.
  • Projected outcomes table.
• Systematic reviews from others.

Notes:

• The Task Force uses several sources of information for synthesizing and judging the strength of the overall evidence. These sources include (read above).

Slide 23

Critical Appraisal Questions

• Do the studies have the appropriate research design to answer the key question?
• To what extent are the existing studies high quality?
• To what extent are the results of the studies generalizable (or "applicable") to the general U.S. primary care population and situation?
• How many studies have been conducted that address the key question? How large are the studies?
• How consistent/coherent are the results of the studies?
• Are there additional factors that assist us in drawing conclusions about the certainty of the evidence? (e.g., presence or absence of dose-response effects; fit within a biologic model).

Slide 24

Step 4: Synthesize & Judge Strength of Evidence for Each Key Question

• Convincing: Well-designed, well-conducted studies in representative populations that directly assess effects on health outcomes.
• Adequate: Evidence sufficient to determine effects on health outcomes, but limited by number, quality, or consistency of studies, generalizability to routine practice, or indirect nature of the evidence.
• Inadequate: Insufficient evidence to determine effect on health outcomes due to limited number or power of studies, important flaws in their design or conduct, gaps in the chain of evidence, or lack of information on important health outcomes.

Notes:

• The Task Force uses this scale to rate the strength of evidence for each key question.

Slide 25

Step 4: Synthesize and Judge Strength of Overall Evidence: Certainty

• Definition: The U.S. Preventive Services Task Force defines certainty as "likelihood that the USPSTF assessment of the net benefit of a preventive service is correct". The net benefit is defined as benefit minus harm of the preventive service as implemented in a general, primary care population. The USPSTF assigns a certainty level based on the nature of the overall evidence available to assess the net benefit of a preventive service.

Slide 26

Levels of Certainty: High, Moderate, or Low

• High: This conclusion is unlikely to be strongly affected by the results of future studies.
• Moderate: As more information becomes available, the magnitude or direction of the observed effect could change, and this change may be large enough to alter the conclusion.
• Low: The available evidence is insufficient to assess effects on health outcomes.

Slide 27

Steps in the Recommendation Development Process

1. Define questions and outcomes of interest using analytic framework.
2. Define and retrieve relevant evidence.
3. Evaluate quality of individual studies.
4. Synthesize and judge strength of overall evidence and make conclusion about certainty.
5. Determine balance of benefits and harms.
6. Link recommendation to magnitude and certainty of net benefits.

Notes:

• The next step is to determine the balance of benefits and harms of screening for a condition.

Slide 28

Step 5: Determine Balance of Benefits and Harms

• Estimate Magnitude of Net Benefit.
• Benefits of Service—Harms of Service = Net Benefit.
• 4 categories of Net Benefit:
  • Zero/Negative.
  • Small.
  • Moderate.
  • Substantial.

Notes:

• Simply put the Task Force estimates the net benefit of screening for a condition by subtracting the harms of screening from the benefits of screening.

Slide 29

Estimating Benefits: Projected Outcomes Table (COPD)

Smoking Status/ Age	NHANES I Number/10,000 with FEV1< 50% predicted	EPC pooled analysis Number of patients prevented from having >=1 COPD exacerbation	Number needed to screen (NNS)
Current smoker	207	12	833
Previous smoker	216	13	960
Never smoker	95	5	2000
Age 40-49	80	4	2500
Age 50-59	260	15	667
Age 60-69	370	22	455
Age 70-74	420	25	400

NHANES = National Health and Nutrition Examination Survey.

Notes:

• This is our best estimate of the benefits.

Slide 30

Estimating Harms: Issues

• Harms of prevention are real but hard to quantify.
• Include psychological and physical consequences of false-positives, false-negatives, labeling," overtreatment of "pseudodisease."
• Opportunity costs.
  • Time and effort required by patients and the health care system (may be substantial).
• Magnitude and duration of harm subjective, hard to compare to benefits.
  • NNH for well-defined harms (e.g., Gastrointestinal [GI] bleeds from ASA).

Notes:

• How does the Task Force assess harms?

Slide 31

Assessing Magnitude of Net Benefit

• No explicit criteria for magnitude.
• Substantial benefit: impact on high burden or major effect on uncommon outcome.
• Problems: requires evidence on harms and common metric for benefit and harms.
• Always requires judgment.

Notes:

• How does the Task Force assess the magnitude of benefit of screening for a condition?

Slide 32

Steps in the Recommendation Development Process

1. Define questions and outcomes of interest using analytic framework.
2. Define and retrieve relevant evidence.
3. Evaluate quality of individual studies.
4. Synthesize and judge strength of overall evidence and make conclusion about certainty.
5. Determine balance of benefits and harms.
6. Link recommendation to magnitude and certainty of net benefits.

Notes:

• The next step is to link the net benefits to a recommendation about a preventive services.

Slide 33

Step 6: Link recommendation to net benefits: USPSTF Grades of Recommendations

Certainty of Net Benefit	Magnitude of Net Benefit
	Substantial	Moderate	Small	Zero/negative
High	A	B	C	D
Moderate	B	B	C	D
Low	Insufficient

Notes:

• The Task Force uses this simplified grid to determine the grade of the recommendation. I will discuss in the next slide exactly what is meant by the letter grade. But I first want to explain the grid in the slide. The Task Force uses two measures, strength of overall evidence (good fair poor) and the estimate of net benefit (benefit minus harms) and using this grid determines a letter grade. For example if the evidence is good about the effectiveness of screening for condition x but there is more harms than benefits (negative net benefit) than the Task Force give a D recommendation.

Slide 34

Step 6: Link recommendation to net benefits: USPSTF Wording of Recommendations

• Grade A.
  • Grade Definition: The USPSTF recommends the service. There is high certainty that the net benefit is substantial.
  • Suggestion for Practice: Offer or provide this service.
• Grade B.
  • Grade Definition: The USPSTF recommends the service. There is high certainty that the net benefit is moderate or there is moderate certainty that the net benefit is moderate to substantial.
  • Suggestion for Practice: Offer or provide this service.
• Grade C.
  • Grade Definition: The USPSTF recommends against routinely providing the service. There may be considerations that support providing the service in an individual patient. There is moderate or high certainty that the net benefit is small.
  • Suggestion for Practice: Offer or provide this service only if there are other considerations that support offering or providing the service in an individual patient.
• Grade D.
  • Grade Definition: The USPSTF recommends against the service. There is moderate or high certainty that the service has no net benefit or that the harms outweigh the benefits.
  • Suggestion for Practice: Discourage the use of this service.
• Grade E.
  • Grade Definition: The USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of the service. Evidence is lacking, of poor quality, or conflicting, and the balance of benefits and harms cannot be determined.
  • Suggestion for Practice: Read "Clinical Considerations" section of USPSTF. Recommendation Statement. If offered the service, patients should understand the uncertainty about the balance of benefits and harms.

Notes:

• The Task Force uses letter grades for their recommendations.

Slide 35

• Diana's Slides.

Slide 36

Questions

Current as of January 2009

---

Internet Citation:

U.S. Preventive Services Task Force. Slide Presentation from the AHRQ 2008 Annual Conference (Text Version). January 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/annualmtg08/090808slides/Lin2.htm

---