Small group materials to accompany the lecture entitled "Understanding the Methods Used by the US Preventive Services Task Force in Developing Recommendations."
Student Guide

---

Background. Clinicians are frequently confronted with conflicting screening and prevention recommendations from a variety of sources. One source, the US Preventive Services Task Force (USPSTF), is thought to set the standard for evidence-based recommendations, though the methods used by the USPSTF to arrive at recommendations are not well known. An understanding of their methodological approach is useful for clinicians and clinician educators faced with making choices about whose guidelines to follow.

Target setting and audience: The materials presented herein are devised for an instructor-led small-group setting. The materials are applicable for use at all levels of training: undergraduate medical education, graduate medical education, and continuing medical education. These materials are linked to a teaching module appropriate for a large-group lecture setting.

---

By the end of this session, students will have met the following educational objectives:

1. Understand the major challenges in measuring and balancing benefits and harms of screening
2. Understand why clinical recommendations for cancer screening may conflict
3. Understand the methodology used by the US Preventive Services Task Force in developing recommendations related to breast cancer screening
4. Understand the interpretation and application of the US Preventive Services Task Force breast cancer screening recommendations

The estimated prep time for this small group is 90 minutes. The small group is scheduled for 120 minutes.

Clinical scenario: You are the director of the Women's Clinic at the General Hospital and are revising your guidelines for breast cancer screening. You note that revised recommendations by the US Preventive Services Task Force (USPSTF) seem to conflict in important ways with your current guidelines (based on the American Cancer Society guidelines). You are unsure what to do. You decide to read the new recommendation and some supporting documents before making any revisions. Here they are:

Screening for breast cancer: an update for the U.S. Preventive Services Task Force. Nelson HD et al; U.S. Preventive Services Task Force. Ann Intern Med. 2009 Nov 17;151(10):727-37, W237-42. Available at: http://www.uspreventiveservicestaskforce.org/uspstf09/breastcancer/brcanup.htm and
http://www.annals.org/content/151/10/727.full.pdf+html

Screening for breast cancer: U.S. Preventive Services Task Force recommendation statement. US Preventive Services Task Force. Ann Intern Med. 2009 Nov 17;151(10):716-26, W-236. Available at: http://www.uspreventiveservicestaskforce.org/uspstf09/breastcancer/brcanrs.htm and
http://www.annals.org/content/151/10/716.full.pdf+html

1. Clinical guidelines strive to balance benefits and harms. What are potential benefits and harms specific to breast cancer screening (as outlined in the USPSTF recommendation)? How might you rank these in terms of importance to population health? (10 minutes)

2. One cited harm of screening in the USPSTF recommendation is “overdiagnosis”. What does that mean specifically in the context of breast cancer screening? (10 minutes)

3. To assist the USPSTF in making its recommendation, a systematic review and meta-analysis were commissioned. You have read an abbreviated version of a much longer (95-page) review (Nelson et al). To determine the validity of a systematic review and meta-analysis for mammography screening, the following elements should be evaluated:

Element	Addressed?
Funding source disclosed	Yes (AHRQ)
Study protocol described	Yes
Search strategy described	Yes (in the full report)
Included and excluded papers listed	Yes (in the full report)
Publication bias addressed	Yes
Study quality assessed	Yes
Tests of heterogeneity performed	Yes
Sensitivity analyses performed	Yes (3 in full report)
Results reported such that analyses may be replicated	Yes (for women 39-49)

Look at the studies of mammography screening included in the meta-analysis (Appendix Table 1). What was the quality of the included studies? In what ways do the studies differ from each other? What are the challenges of combining studies in general and these studies in particular? How did the authors address limitations in combining results from studies that may have differed from each other in important ways? (10 minutes)

4. In the Nelson report, the following summary information from mammography screening trials is given for women aged 39-49 years:

Are any of the individual studies statistically significant? Does each study show a trend to benefit (based on the relative risk)? Which study do you think is weighted the least in the meta-analysis? The most? Do you think there is substantial heterogeneity? If so, which study do you think might contribute most to heterogeneity? (10 minutes)

5. The Age/Moss trial is the largest (over 150,000 women randomized) and was designed specifically to answer the question about mammography efficacy among women aged 39-41 years. Enrollees were randomized to being screened at ages 39-41 and annually thereafter until age 48 (i.e., for about 10 years) versus no screening. What can you conclude about the efficacy of mammography in women aged 39-49 on the basis of the reported results of the Age/Moss trial? What can you conclude about the efficacy of mammography in women aged 39-49 on the basis of the overall results of the meta-analysis? (10 minutes)

6. The USPSTF commissioned a decision analytic model to assist in weighing benefits and harms. In the report accompanying the guideline, the following table was provided (http://www.uspreventiveservicestaskforce.org/uspstf09/breastcancer/brcanart.htm):

Table 4. Benefits and Harms Comparison of Different Starting and Stopping Ages Using the Exemplar Model^a

Strategy	Average Screenings per 1000 Women	Potential Benefits (vs. No Screening)			Potential Harms (vs. No Screening)b
		Percentage of Mortality Reduction	Cancer Deaths Averted per 1000 Women	Life-Years Gained per 1000 Women	False-Positive Results per 1000 Women	Unnecessary Biopsies per 1000 Women
Comparison of different starting ages
Biennial screening
40-69 y	13,865	16c	6.1	120c	1,250	88
45-69 y	11,771	17c	6.2	116c	1,050	74
50-69 y	8,944	15	5.4	99	780	55
55-69 y	6,941	13	4.9	80	590	41
60-69 y	4,246	9	3.4	52	340	24
Annual screening
40-69 y	27,583	22c	8.3	164c	2,250	158
45-69 y	22,623	22c	8.0	152c	1,800	126
50-69 y	17,759	20c	7.3	132c	1,350	95
55-69 y	13,003	16c	6.1	102c	950	67
60-69 y	8,406	12c	4.6	69c	600	42
Comparison of different stopping ages
Biennial
50-69 y	8,944	15	5.4	99	780	55
50-74 y	11,109	20	7.5	121	940	66
50-79 y	12,347	25	9.4	130	1,020	71
50-84 y	13,836	26	9.6	138	1,130	79
Annual
50-69 y	17,759	20c	7.3	132c	1,350	95
50-74 y	21,357	26c	9.5	156c	1,570	1106
50-79 y	24,439	30	11.1	170	1,740	122
50-84 y	26,913	33	12.2	178	1,880	132

^a Results are from model S (Stanford University). Model S was chosen as an exemplar model to summarize the balance of benefits and harms associated with screening 1000 women under a particular screening strategy.
^b Overdiagnosis is another significant harm associated with screening. However, given the uncertainty in the knowledge base about ductal carcinoma in situ and small invasive tumors, we felt that the absolute estimates are not reliable. In general, overdiagnosis increases with age across all age groups but increases more sharply for women who are screened in their 70s and 80s.
^c Strategy is dominated by other strategies; the strategy that dominates may not be in this table.

How do various starting and stopping ages affect the balance of benefits and harms? How does biennial compared with annual screening affect the balance of benefits and harms? Compare biennial screening among women ages 40-69 years to biennial screening among women ages 50-69 years. What are the specific differences in benefits and harms? (20 minutes)

7. Review the USPSTF recommendation. What does the USPSTF conclude about mammography screening? What is their justification? (10 minutes)

8. A 40-year-old woman presents to your clinic for a periodic health examination. She is healthy and has no risk factors for any particular diseases. She does not smoke, is sexually active, and is not pregnant. Using the electronic Preventive Services Selector (http://epss.ahrq.gov/PDA/index.jsp), you note that screening for the following diseases gets an A or B recommendation: cervical cancer, hypertension, alcohol misuse, and obesity. Routine mammography is not recommended. She has read about the mammography controversy and wants to know more about the benefits and harms. Look at the following table from the Nelson report. These data were derived from the Breast Cancer Screening Consortium and include results from a single screening round.

What might you tell this patient about the likelihood of screen-detected breast cancer? false-positive testing? Additional imaging? Does every case of screen-detected breast cancer translate into a breast cancer death avoided? (10 minutes)

9. The USPSTF recommendations also state that a clinician taking the time to teach breast self-examination (BSE) should be discouraged since the harms outweigh the benefits (a “D” recommendation). The Nelson report summarizes the results of trials evaluating the benefit of teaching BSE on breast cancer mortality. Consider the results of the “good-quality” Shanghai trial. From the data presented, how certain are you that teaching BSE has no effect on breast cancer mortality? (5 minutes)

10. The USPSTF recommendations also state that there is insufficient evidence to recommend clinical breast examination (CBE) and an “I” statement was assigned. What does an “I” statement mean? What does the USPSTF suggest clinicians do when an “I” statement is assigned? (5 minutes)

11. One important component of health care leadership is arriving at consensus opinions. Within your group, determine a strategy to arrive at a consensus opinion about an optimal breast cancer screening strategy for the Women's Clinic at the General Hospital. Consider the following questions: (20 minutes)

• Should constrained resources (such as those in a public hospital) affect your decision?
• Would you make a different decision if mammograms were entirely free?
• Would you use formal cost analyses to assist in your decision?
• What are potential barriers to incorporating the USPSTF recommendation for patients aged 40-49 years at the General Hospital?

Acknowledgement

These materials were developed under an agreement between the Health Research & Education Trust and George F. Sawaya, MD as part of the “Knowledge Transfer/Implementation- Outreach to Health Professionals” project, no. HHSA 290200900014C sponsored by the Agency for Healthcare Quality and Research.

Dr. Sawaya is Professor of Obstetrics, Gynecology and Reproductive Sciences and Epidemiology and Biostatistics at the University of California, San Francisco. He was a member of the US Preventive Services Task Force from 2004-2008.

The information herein was current as of August 2010 as per the USPSTF website http://www.uspreventiveservicestaskforce.org/about.htm

---

Internet Citation:

Small group materials to accompany the lecture entitled "Understanding the Methods Used by the US Preventive Services Task Force in Developing Recommendations." Student Guide. October 2010. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/kt/webinars/tfmethodstudent.htm

---