Refining Evidence-based Recommendation Development
Janelle Guirguis-Blake, M.D.a; Ned Calonge, M.D., M.P.H.b; Therese Miller, Dr.P.H.c; Albert Siu, M.D., M.S.P.H.d; Steven Teutsch, M.D., M.P.H.e; and
Evelyn Whitlock, M.D., M.P.H.f
The U.S. Preventive Services Task Force (USPSTF) has developed new methods for evidence reviews and recommendation development. The most recent changes in the process include how the USPSTF solicits and prioritizes topics for review, updates evidence reviews and recommendations, and communicates with its audience.
This methods update was first published in the Annals of Internal Medicine. Select for copyright and
source information.
Contents
Introduction
Types of Recommendations
Types of Evidence Reviews
Communicating and Disseminating USPSTF Recommendations
Conclusion
References
Author Affiliations
Notes
Introduction
The U.S. Preventive Services Task Force (USPSTF) is an
internationally recognized, independent panel of nonfederal
experts in primary care, prevention, and research
methods that makes evidence-based recommendations to
guide the delivery of clinical preventive services. Convened
and supported by the Agency for Healthcare Research and
Quality (AHRQ), the USPSTF is charged by the U.S.
Congress to review the scientific evidence for clinical preventive
services and to develop evidence-based recommendations
for their delivery to the health care community.
The disciplines of USPSTF members include family medicine,
internal medicine, geriatrics, preventive medicine,
pediatric and adolescent medicine, obstetrics and gynecology,
nursing, psychology and behavioral medicine, public
health, and health policy.
Since its inception more than 20 years ago, the USPSTF
has worked to fulfill its mission by:
- Evaluating the benefits and harms of preventive services in apparently healthy persons on the basis of age, sex, and known risk factors for disease.
- Making recommendations about which preventive services should be provided routinely in primary care practice and which should not.
The USPSTF recommendations are intended to improve
both clinical practice and the health of patients. The
scope of the Task Force is specific: Its recommendations
address primary or secondary preventive services targeting
conditions of substantial burden in the United States and
are provided in primary care settings (or are available
through primary care referral).
Although the main audience
for USPSTF recommendations is the primary care
clinician, these recommendations also have relevance for
and are widely used by policymakers, managed care organizations,
public and private payers, quality improvement
organizations, research institutions, professional medical
organizations, specialist physicians, and patients.
The USPSTF is distinct from other groups that provide
recommendations for preventive services. It does not
create guidelines based on expert opinion, as do many nonprofit
advocacy organizations and professional groups. The
Task Force does not advocate for prevention, perform decision
analysis to routinely standardize the personal preferences
and values of patients, consider medicolegal issues or
the cost or coverage of services in making recommendations,
or set clinical standards or health policy. Instead, the
Task Force follows a unique and explicit methodology to
develop recommendations that pass a rigorous evidence-based
standard.1
Table 1 shows the Task Force's current
procedures for developing recommendations. The USPSTF
stands as an independent arbiter of the evidence and,
as such, has set the standard for evidence-based recommendations
for the delivery of clinical preventive services.
The process of making evidence-based recommendations
occurs in an environment in which many stakeholders,
often with competing interests, have their own preferences
for or ideas about the delivery of preventive services.
In such an environment, in which outside organizations
maintain a keen interest in what the Task Force recommends,
it is especially important for the USPSTF to maintain
transparency, accountability, and consistency to ensure
the independence and the integrity of their process
and recommendations.
This paper is 1 in a series presenting the refinements
that the USPSTF has undergone since its methodology was
last published in 2001. The Task Force processes of selecting
topics, synthesizing evidence, deliberating and voting
on recommendations, soliciting peer review, and finalizing
recommendations have evolved over time. The purpose of
this refinement is to continually improve the methods of
evidence-based review, to maintain transparency and objectivity,
and to increase USPSTF efficiency. Table 2 summarizes
the ways in which the USPSTF has refined its
processes to meet these and other aims.
Return to Contents
Types of Recommendations
New Topics
The Task Force solicits new topics for consideration
from the field through a periodic notice in the Federal
Register and solicitation of professional liaison organizations.
Task Force members may also generate new topics
for consideration.
The USPSTF first considers whether
newly nominated topics are within the scope of the USPSTF
(that is, a primary or secondary preventive service that is
relevant to primary care and addresses a disease with a
substantial health burden) and then prioritizes the topics
by using specific criteria:
- The public health importance of the condition to be prevented (burden of suffering and expected effectiveness of preventive services to reduce that burden).
- The potential for the USPSTF to affect clinical practice (based on existing controversy or the belief that a gap exists between evidence and practice).
The
USPSTF secondarily considers the need to balance the
portfolio of topics to address diverse groups, types of conditions,
and types of preventive services (for example,
screening, counseling, and preventive medication). The
USPSTF recommendation statement on routine use of aspirin
or nonsteroidal anti-inflammatory drugs for the primary
prevention of colorectal cancer, which appeared in
the 6 March 2007 issue of Annals of Internal Medicine,
represents a new topic nominated by the Centers for Disease
Control and Prevention (CDC).2
Updated Recommendations
To efficiently utilize available resources, the Task
Force has implemented new procedures to review previous
topics and update recommendations for continued inclusion
in the current Task Force library. To be consistent
with the standards of the National Guidelines Clearinghouse™
(http://www.guideline.gov), the process of revisiting and updating
a previous USPSTF recommendation begins approximately
3.5 years after that recommendation was released,
or earlier if a landmark study is published that could
change a current recommendation.
The USPSTF screens
topics under consideration to identify emerging scientific
issues and current clinical relevance, and then prioritizes
them by using the criteria described above, in addition to
considering the potential for new, recent evidence to
change a previous recommendation. The USPSTF then
recommends a targeted evidence update or a full evidence
update. A full evidence update systematically examines a
complete analytic framework of key questions by using recent
evidence, taking into account any need to reframe the
topic or focus of the recommendation since it was last
considered, whereas a targeted evidence update systematically
examines a subset of the key questions from the original
analytic framework.
Reaffirmation Recommendations
Some clinical preventive services, such as screening for
hypertension, have a strong, well-established evidence base
and are a routine part of clinical practice. Because it is
unlikely that new evidence will change USPSTF recommendations
for such services, the USPSTF reviews the evidence
for them in an expedited manner by conducting
literature searches that address benefits and harms and consulting
experts.
Some recommendations for clinical preventive services
fall within the scope of not only the USPSTF but also
other federal agencies. For example, adult and childhood
immunizations are addressed by the CDC Advisory Committee
on Immunization Practices (ACIP). In a few select
cases, the Task Force chooses to refer to such recommendations.
Although the USPSTF considers these recommendations
part of its portfolio of recommended clinical preventive
services, it refers clinicians to the ACIP active
evidence review process and recommendations for 2 reasons:
The USPSTF does not have adequate resources to
keep such recommendations current, and it does not wish
to duplicate the efforts of the ACIP.
Inactive Recommendations
The USPSTF considers some recommendations made
in previous years (for example, those for electronic fetal
monitoring, home uterine monitoring, and counseling for
dental disease) to be no longer current or priority topics.
These topics are regarded as "inactive" for various reasons.
- First, the USPSTF may consider such recommendations now to be outside its scope of work.
- Second, such recommendations may be judged to be no longer clinically relevant, because of changes in technology or clinical practice or because of new understanding of disease etiology or natural history.
- Finally, the topic of a recommendation may be judged to have low priority because it has limited potential to influence public health burden or clinical practice.
Currently inactive recommendations are identified on
the USPSTF Web site (http://www.preventiveservices.ahrq.gov).
Return to Contents
Types of Evidence Reviews
The USPSTF bases its recommendations on systematic
evidence reviews, which form the critical underpinnings
of its deliberations and decisionmaking. The
USPSTF members are intensively involved in the conceptualization,
content, and interpretation of these reviews.
The reviews are products of a partnership between members
of the USPSTF and Evidence-based Practice Center
(EPC), which conducts, synthesizes, and produces them.
The process is facilitated and coordinated by the staff of
the AHRQ, and in some cases, the AHRQ staff conducts
targeted evidence updates.
The USPSTF now uses 4 types
of reviews to support its recommendations:
- Full evidence reviews.
- Staged evidence reviews.
- Targeted evidence updates.
- Reaffirmation updates.
Recommendations for new topics
are informed by full evidence reviews (which may be, in
rare instances, staged evidence reviews). Updates of previous
recommendations are informed by 1 of 3 types of
reviews: full evidence updates, targeted evidence updates,
or reaffirmation updates. These 3 updated reviews represent
a new methodology for the USPSTF process. Table 3
provides descriptions and examples of the types of reviews.
Full evidence reviews begin with the development of
an analytic framework of key questions followed by a comprehensive
literature search. They then progress through
critical evaluation, qualitative or quantitative synthesis as
appropriate, and detailed documentation of methods and
findings. The steps in the USPSTF full evidence review
process are as follows:
- Creation of an analytic framework and key questions developed jointly by USPSTF members, EPC scientists, and AHRQ staff to guide the review process
- Determination of criteria for admissible evidence
- Evaluation of the evidence for internal and external validity of individual studies, study design and its relevance to key questions, consistency and coherence of the evidence, precision of the estimates of benefits and harms, and directness of the evidence to the key questions
- Estimation of the magnitude of benefits and harms for the preventive service in specific populations
- Assessment of the certainty of the evidence of the net benefit or harm for the preventive service in specific populations.
In select instances, critical gaps in the chain of evidence
become apparent during the full evidence review of a
new topic. The USPSTF may then request that the EPC
conduct the systematic review in a staged manner. Staged
reviews allow the USPSTF to determine whether it can
make a recommendation on the basis of the review results,
whether a full evidence review is required before it can
make a recommendation, or whether another product (such
as a commentary or an editorial) might be more appropriate
than a recommendation statement. Each staged review
is managed on a case-by-case basis, with the USPSTF determining
at each stage how to proceed.
One example of a
staged review was the USPSTF recommendation on
screening for hereditary hemochromatosis,3 a topic for
which there is an extensive literature on screening, penetrance
is poorly understood (but probably low), the incremental
benefit of earlier treatment is uncertain, and there
are important harms to screening and treatment. For this
review, the Task Force asked the EPC to report results for
a limited number of key questions, and then determined
whether the remaining key questions needed to be systematically
reviewed in order to vote on a recommendation.
For many topics for which the USPSTF has made a
previous recommendation, the USPSTF directs either the
staff of the EPC or AHRQ to conduct a targeted evidence
update rather than a full evidence update. The first step in
a targeted update is to identify the update key questions,
based on the analytic framework of the previous systematic
evidence review. Update key questions are critical questions
whose answers might result in the USPSTF making a
different recommendation based on new evidence. The researchers
conduct systematic evidence reviews for those
critical questions, limiting the literature search to studies
published since the prior full evidence review was finalized.
In these cases, updates can be completed by using this
targeted update process; the new information is evaluated
by using the established systematic review methods, and
results are integrated with the knowledge base from the
previous evidence review. The USPSTF considers this evidence
in updating its recommendations, rationale, and
clinical considerations. The soon-to-be-released recommendations
on screening for carotid artery stenosis and
screening for skin cancer are examples of targeted evidence
updates. (Further information is available at http://www.preventiveservices.ahrq.gov.)
For topics for which well-established evidence exists,
the reaffirmation evidence update involved in supporting
an updated recommendation is brief and includes literature
searches in PubMed® and the Cochrane database, performed
by AHRQ staff, on the benefits and harms of the
preventive service. The primary goal of the literature search
is to find new and substantial evidence that could change
the previous recommendation. The literature search uses the Medical Subject Heading terms from the previous evidence
review (if available) and searches for studies published
since the last review (3 months before the end date
of the previous search). For the literature search on benefits,
the search is limited to meta-analyses, systematic
reviews, and RCTs; for harms, the search includes meta-analyses,
systematic reviews, RCTs, cohort and case–control
studies, and large case series.
The USPSTF incorporates expert and peer review of
its background documents to confirm that all relevant literature
has been considered and that the evidence presented
for USPSTF consideration is accurate. The evidence
reviews on which USPSTF recommendations are based, as
well as the proposed recommendation statements, are reviewed
by a standard list of Federal agencies and professional
organizations. Additional reviewers for the evidence
reviews are identified by the EPC as national experts in the
field and investigators of sentinel trials. The USPSTF requests
that reviewers comment on the clarity, clinical usefulness,
and scientific accuracy of the recommendation
statement. The Task Force views its role as a decision
maker engaged in a deliberative process. Throughout this
process, the Task Force maintains its independence by
making these decisions without outside influence by professional
societies or governmental entities.
Return to Contents
Communicating and Disseminating USPSTF Recommendations
The clarity and comprehensibility of recommendations
are critical because of their widespread use. The USPSTF
and AHRQ are aware that the recommendations and the
letter grades used to define them may be misunderstood,
and these agencies are therefore taking pains to clarify and
refine them. AHRQ has conducted focus groups of
clinicians (2004–2006) to solicit feedback about the readability
and usability of the Task Force recommendations.
Various themes emerged, including requests for simplified,
succinct recommendations and an easier-to-use format
(boldface type, bulleted sections, and boxes to highlight
key information); recommendations of other professional
organizations to easily compare with the USPSTF recommendation;
and pointers to Web sites and references for
additional information on the topic. The new recommendation
grid and meanings for the letter grades appear in the
USPSTF methods update that appears in this issue.4
The recommendations of the USPSTF are widely disseminated
to professional audiences in relevant journals,
such as Annals of Internal Medicine; on the AHRQ
Web site (http://www.preventiveservices.ahrq.gov); in print
through the annual Guide to Clinical Preventive Services;
and in a Web-based Electronic Preventive Services Selector,
which is downloadable into personal digital assistant
devices. (Information on ordering AHRQ materials is
available on the AHRQ Web site [http://www.ahrq.gov/news/pubsix.htm], by telephone
at 800-358-9295, or by E-mail at AHRQpubs@ahrq.hhs.gov.)
Representatives from federal organizations (such as the
National Institutes of Health, the CDC, and the U.S.
Food and Drug Administration), professional organizations
(such as the American Medical Association, the American Academy of Family Physicians, the American
Academy of Pediatrics, and the American Academy of
Nurse Practitioners), and quality improvement organizations
are invited to observe the Task Force meetings; partner
organizations represent primary care clinicians, the primary
audience for the Task Force recommendations. The
roles of partner organizations are to inform the scope of the
topic, provide expert review of evidence reports and recommendation
statements, and assist with dissemination of
USPSTF recommendations to their members.
The end users of the recommendations have the opportunity
to respond to the USPSTF recommendations
and their accompanying evidence reviews through editorials
in peer-reviewed publications and through formal letters
to the editor of peer-reviewed journals. In addition,
letters can be sent to the Task Force through AHRQ. (Go to
http://info.ahrq.gov for the AHRQ mailing address and to
write an electronic letter to the agency.) These letters are an
important source of feedback to the USPSTF.
Return to Contents
Conclusion
The USPSTF believes that its recommendations and
reviews should be used to foster communication among
health care providers, patients, payers, employers, and
research organizations for the development of quality
improvement strategies. The USPSTF relies on partner organizations,
such as professional societies, and on policymakers
to use the USPSTF recommendations to improve
the delivery of evidence-based preventive services and,
when appropriate, to further research in areas identified by
the USPSTF.
As evidence-based reviews evolve, so too will the
USPSTF continue to refine and advance its methodology.
With limited resources and a growing body of literature
about clinical preventive services, the USPSTF must balance
its rigorous scientific standards with the demand for
up-to-date recommendations on a broad array of preventive
services. Likewise, to maximize implementation of
USPSTF recommendations, the USPSTF will continue to
refine its communication strategies. Responding to the
busy primary care clinician's need for clear, concise, evidence-based
recommendations on the delivery of clinical preventive
services is the mission of the USPSTF.
Return to Contents
References
1. Methods Work Group, Third U.S. Preventive Services Task Force. Current
methods of the U.S. Preventive Services Task Force: a review of the process. Am J
Prev Med 2001;20:21-35. [PMID: 11306229]
2. U.S. Preventive Services Task Force. Routine aspirin or nonsteroidal antiinflammatory
drugs for the primary prevention of colorectal cancer: U.S. Preventive
Services Task Force recommendation statement. Ann Intern Med 2007;146:361-4.
3. Whitlock EP, Garlitz BA, Harris EL, Beil TL, Smith PR. Screening for
hereditary hemochromatosis: a systematic review for the U.S. Preventive Services
Task Force. Ann Intern Med 2006;145:209-23. [PMID: 16880463]
4. Barton MB, Miller T, Wolff T, Petitti D, LeFevre M, Sawaya G, et al. How
to read the new recommendation statement: methods update from the U.S.
Preventive Services Task Force. Ann Intern Med 2007;147:123-7.
Return to Contents
Notes
Potential Financial Conflicts of Interest
Employment: S. Teutsch
(Merck & Co. Inc.); Stock ownership or options (other than mutual funds):
S. Teutsch (Merck & Co. Inc.).
Author Affiliations
a. Dr. Guirguis-Blake: Department of Family
Medicine, Tacoma Family Medicine Residency Program, University of
Washington, Tacoma, WA.
b.
Dr. Calonge: Colorado Department of Public Health and Environment, Denver, CO.
c.
Dr. Miller: Center for Primary Care, Prevention & Clinical Partnerships,
Agency for Healthcare Research and Quality, Rockville,
MD.
d.
Dr. Siu: Brookdale Department of Geriatrics and Adult Development,
Mount Sinai Medical Center, New York, NY.
e.
Dr. Teutsch: Merck & Company, Inc., West Point, PA.
f.
Dr. Whitlock: Kaiser Permanente Center for Health Research, Portland, OR.
Copyright and Source Information
This document is in the public domain within the United States. For
information on reprinting, contact Randie Siegel, Director, Division of
Printing and Electronic Publishing, Agency for Healthcare Research and Quality,
540 Gaither Road, Rockville, MD 20850.
Requests for linking or to incorporate content in electronic resources
should be sent to: info@ahrq.gov.
Source: Guirguis-Blake J, Calonge E, Miller T, Siu A, Teutsch S, Whitlock E. Current processes of the U.S. Preventive Services Task Force: refining evidence-based recommendation development. Ann Intern Med 2007;147:117-22.
Return to Contents
Current as of June 2007
Internet Citation:
Guirguis-Blake J, Calonge E, Miller T, et al. Current Processes of the U.S. Preventive Services Task Force: Refining Evidence-based Recommendation Development. Originally published in Ann Intern Med 2007;147:117-22. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/clinic/uspstf07/methods/currprocess.htm