Evaluation of the U.S. Preventive Services Task Force Recommendations for Clinical Preventive Services: Final Contract Report

You Are Here: AHRQ Home > About AHRQ > Mission & Budget > Evaluation of the U.S. Preventive Services Task Force Recommendations for Clinical Preventive Services > Executive Summary

Evaluation of the U.S. Preventive Services Task Force Recommendations for Clinical Preventive Services

Executive Summary

The Agency for Health Care Research and Quality (AHRQ) contracted with NORC at the University of Chicago to study the implementation of the U.S. Preventive Services Task Force (USPSTF) recommendations for clinical preventive services in a closed-panel health plan, open-panel health plan, hybrid health plan having both open- and closed-panel characteristics, and governmental health plan. The four health plan site selections serve to address AHRQ's interest in the adoption and integration of the USPSTF recommendations at the health plan level. This final report represents an analysis of qualitative interviews, prevention materials provided by sites, and other published literature. Overall, we report on findings from over 40 discussions with respondents from four different health plans. Our findings address the adoption, integration, delivery, and dissemination of the USPSTF recommendations in different types of health plans. We also describe the challenges faced by both health plan leadership and staff members in adopting, integrating, and delivering the USPSTF recommendations, and offer suggestions for improving dissemination of the recommendations.

Importance of the Current Study

In 1984, the U.S. Public Health Service created the USPSTF as an independent panel of experts in primary care and prevention that systematically review the evidence of effectiveness and develop recommendations for clinical preventive services. Although there is existing research regarding implementation and use of USPSTF recommendations at the individual clinician and group practice levels, there has been little research at the health plan level. On behalf of AHRQ, NORC had previously conducted an assessment of the use of USPSTF recommendations among community-based health care practices and found that physicians consider the recommendations valuable, but sometimes inconsistent with their local standard of care. Research has highlighted the need for examining the use of the USPSTF recommendations at the health plan (payer) level, which may be more effective in impacting usage at the physician level. Findings from this study offer unique insights into how the USPSTF recommendations are integrated into different types of health plans, and offers suggestions for improving dissemination of the recommendations.

Study Methods

Project activities were conducted using a tiered-qualitative approach divided into three separate but interrelated phases: 1) literature review and evaluation design; 2) semi-structured phone and in-person interviews; and 3) final analysis, including five thematic overviews focused on key cross-cutting themes related to the adoption, integration, and delivery of the USPSTF recommendations in health plans.

The evaluation design involved a review of the published and unpublished literature and the development of three key research questions to be explored further through qualitative methods. The three research questions are: 1) To what extent are clinical preventive services integrated into the four selected health care delivery systems? 2) What deficiencies exist in the delivery of clinical preventive services by these health care systems? and 3) How can AHRQ contribute to the increased implementation of USPSTF recommendations among health care delivery systems? The evaluation design provided the foundation for subsequent project activities, including the selection of the health plan sites.

The second major phase of the study involved in-depth semi-structured telephone and in-person interviews with staff from four health plans. To explore the adoption, integration, delivery, and dissemination of the USPSTF recommendations, NORC interviewed a total of 42 health plan staff were chosen based on their operational mix (e.g., open-panel plan, closed-panel plan, hybrid plan, and governmental plan) and willingness to participate in the study. Specifically, each health plan has different types of relationships with providers based upon employment versus contracting. We hypothesized that the structure of the health plan may impact the integration of the USPSTF recommendations and the ability of the plans to deliver the recommendations.

After participation was secured, NORC worked with the Medical Director and site contact to secure study participants. NORC specifically asked to interview individuals in several different kinds of positions, including Medical Directors, Directors of Quality Improvement, Directors of Health Information Technology (IT), Clinical Advisors,^a Quality Improvement Staff, and Health IT Staff. NORC developed an interview guide that is organized into modules reflecting the perspective of the particular respondent type (e.g. Medical Director, Director of Quality Improvement, Quality Improvement Staff, Director of Health IT, Health IT Staff, and Clinical Advisor). Given the different types of health plans and differences in the range of activities involved in the delivery of clinical preventive services, NORC peppered the interview guide with additional follow-up questions, when appropriate, that focused on topics or issues that were directly relevant to each health plan.

The interviews covered a range of topics that are important to the delivery and integration of the USPSTF recommendations. We asked individual respondents about their familiarity with the USPSTF recommendations, the adoption, integration, and delivery of the USPSTF recommendations in their health plans, challenges or barriers they face, and specific ways that AHRQ can improve the dissemination of the USPSTF recommendations to increase adoption rates at the systems-level. Prior to the interviews, NORC reviewed relevant prevention materials provided by each of the four health plans sites, and prepared a review of the literature on the USPSTF and its impact on the delivery of clinical preventive services.

Analysis activities, which culminated in the final report, drew from findings and themes across all project activities. NORC also developed five thematic overviews to further explore key cross-cutting themes related to the adoption, integration, and delivery of the USPSTF recommendations in health plans.

Background

The USPSTF provides clinicians with a valuable service, gathering and analyzing the available literature on preventive medicine and transforming it into sound evidence-based recommendations. Yet, despite the value and ease of attaining the USPSTF recommendations, patients are not receiving the recommended amount of preventive services. The literature shows that while physician time and the inability to prioritize recommendations are perhaps the greatest challenges to the delivery of clinical preventive services; additional factors such as implementation systems, quality improvement strategies, and the type of system in which patients seek care may also affect the rates of service delivery. Although there is existing research regarding implementation and use of USPSTF recommendations at the individual clinician and group practice levels, there has been little research at the health plan level.

Findings

Findings are discussed in three areas: 1) the adoption of the USPSTF recommendations, including familiarity with the recommendations, and the process of accessing, adopting and integrating the recommendations; 2) the integration and delivery of the USPSTF recommendations, including incorporating the recommendations into practice, major uses of health IT and quality improvement, the impact of reimbursement structures and other factors, and barriers to integration and delivery of the recommendations; and 3) improving dissemination of the USPSTF recommendations in health care systems.

I. Adoption of the USPSTF Recommendations in Health Plans

Familiarity with the USPSTF Recommendations

The synthesis of findings begins with the adoption of the USPSTF recommendations in health plans. We discuss respondents' familiarity with the recommendations, the methodology underlying the recommendations, and the USPSTF recommendation grading system. Overall, we found that the respondents were familiar with the USPSTF recommendations. However, reported familiarity varied dramatically. The majority of respondents indicated that they are "somewhat" or "fairly" familiar with the recommendations. Many respondents indicated that they have heard of the USPSTF recommendations, but do not know them well enough to cite specific examples. A few respondents indicated that they had not heard of the USPSTF recommendations at all prior to our interview. Our conversations indicate that most respondents were unfamiliar with the USPSTF's methodology. Many respondents were not aware that AHRQ disseminates a line of tools and products that incorporate the USPSTF recommendations, such as the Put Prevention into Practice materials, the Electronic Preventive Services Selector, a pocket manual of recommendations, and email updates.

Process for Adopting the USPSTF Recommendations

Each health plan has its own unique process for adopting clinical preventive services recommendations from the USPSTF and other sources. The closed-panel plan and governmental plan comply with guidelines for clinical preventive services issued by their respective systems-level or national headquarters. However, the degree of compliance with these nationally-determined guidelines—and relative autonomy to deviate—differs dramatically. In the open-panel and hybrid plans, the process of adopting the recommendations occurs at the plan level only. The importance of provider involvement in developing the guidelines is also a key finding across all four health plans.

The USPSTF recommendations play an important role in the process that health plans use to develop and adopt their own recommendations or policies for clinical preventive services. The health plans consult the USPSTF during their process of reviewing and adopting clinical preventive services recommendations. Expert opinion and group consensus decision-making are two other influences that impact the process used to select recommendations.

Accessing the USPSTF Recommendations

Few people regularly check for updated recommendations from the USPSTF. In fact, less than one quarter of respondents indicated that they regularly check for updated evidence and recommendations from the Task Force. Directors of Quality Improvement were most likely to check for updated evidence and recommendations from the Task Force. However, approximately 52% of participants indicated that they do not check for updated evidence.

The Director of Quality Improvement and Quality Improvement Department, more generally, were cited most often as the primary target audience of the recommendations followed by clinicians, the Medical Director, and finally, plan leadership at the systems-level or central office headquarters.

II. Integration and Delivery of the USPSTF Recommendations in Health Plans

Incorporating the USPSTF Recommendations into Practice

Each of the health plans delivers some of the USPSTF's "A" and "B" recommendations. None of the four plans systematically deliver all of the USPSTF recommendations. Many of the respondents were unfamiliar with which clinical preventive services recommendations are "A" and "B" recommendations. Others did not recognize the USPSTF grading scheme at all. As a result, these respondents had difficulty commenting on the delivery of the "A" and "B" recommendations in their health plans. In addition, another notable finding is that respondents indicated that some of the "A" and "B" recommendations are being delivered across health plans, but not due to the fact that they are highly recommended by the USPSTF. Rather, the recommendations are being systematically delivered because they coincide with Health Plan Employer Data and Information Set (HEDIS) specifications.

The USPSTF recommendations are integrated in health plan provider manuals on clinical preventive services, performance measures, or other publications. USPSTF recommendations are integrated electronically using health information technology tools such as electronic medical records (EMRs), clinical reminders, and order sets for clinicians. The USPSTF recommendations are incorporated into the plan's patient health education materials that are distributed to the member population.

Major Uses of Health Information Technology

Health IT plays a key role in the integration and delivery of clinical preventive services recommendations. Clinical decision support systems in health plans are highly relevant to the implementation of clinical preventive services recommendations. Health plans use EMRs, clinical reminders, and other health information technology tools such as order sets to not only integrate and deliver the USPSTF recommendations, but also track and monitor data for quality improvement purposes. Each of the health plans use health IT to integrate the USPSTF recommendations though their sophistication varies considerably. Governmental and closed-panel systems, where providers are employees of the plan, had the greatest integration of Task Force recommendations using health IT, followed by the hybrid system (in which approximately half of plan members access services through plan-affiliated providers and half through contracted providers). Finally, the open-panel system, which allows private physicians to contract with multiple health plans, had the least integration of the Task Force recommendations using health IT.

Major Uses of Quality Improvement

Quality improvement activities are employed at each of the health plans to increase the appropriate delivery of clinical preventive services. Health plans utilize health IT to measure and monitor the delivery of clinical preventive services for quality improvement purposes. However, respondents from all of the plans described that using data for quality improvement purposes can be challenging for a variety of reasons. Health plans noted a variety of barriers including claims lag, coding detail, unprocessed claims, inaccurate coding, and incomplete patient records.

Health plans have implemented quality improvement activities to increase the delivery of screenings for colorectal cancer, cervical cancer, and breast cancer, and improve the delivery of tobacco cessation counseling and flu immunizations. Many of the quality improvement techniques integrate health information technology. Specific quality improvement activities focused on improving delivery rates of clinical preventive services including provider report cards, internal work groups, monitoring and compliance, external programs and campaigns, patient education and outreach, barrier analysis, member satisfaction surveys, and practice patterns analysis. Respondents also described strategies used to encourage the implementation of quality improvement activities at the practice or clinician level including provider education and rewards.

The Impact of the Reimbursement Structure on the Delivery of Clinical Preventive Services

We found that the majority of the health plans utilize the reimbursement structure to reward the delivery of clinical preventive services—but to varying degrees. In the case of the hybrid plan and the governmental plan, individual physicians are financially rewarded for performing well on certain measures related to clinical preventive services. These plans indicated that their reimbursement structures have a "pay-for-performance" component, whereby financial incentives for medical teams and physicians are tied to health care quality. The open-panel plan is currently developing a pay-for- performance component for preventive health. The closed-panel plan rewards its medical teams based on performance, but does not financially reward individual physicians.

The Role of HEDIS in the Delivery of Clinical Preventive Services

For the past decade, the Health Plan Employer Data and Information Set (HEDIS) has been used to evaluate the quality of outpatient care in many large managed health care plans. Respondents occasionally confused the USPSTF recommendations and HEDIS measures. Only a handful of the respondents actually make a distinction between the USPSTF recommendations and HEDIS measures. Respondents across health plans conveyed that HEDIS strongly influences which clinical preventive services are provided and how frequently services are tracked and measured. It appears that the delivery of the USPSTF recommendations associated with HEDIS measures is evaluated and tracked more frequently than the delivery of the USPSTF recommendations that are not associated with HEDIS measures.

Perceptions of the USPSTF Recommendations

Health plan leadership and staff provided positive feedback on the USPSTF recommendations. The majority of respondents found that the USPSTF recommendations are packaged in a user-friendly way. Respondents described that the recommendations are "thorough," "easy to read," and "very easy to follow." Several respondents felt that AHRQ could improve upon the packaging of the USPSTF recommendations. Overall, the majority of Quality Improvement Directors, Quality Improvement staff, and Medical Directors were interested in learning more about the process behind the USPSTF recommendations.

Respondents indicated that the Task Force's prevention priorities aligned with other systems-level variables such as payer expectations, industry quality indicators, and consumer demand. A few respondents commented that prevention priorities are aligned with payer expectations and quality indicators, however, responses varied on the degree of alignment with these variables. Many respondents believe that the USPSTF's prevention priorities are not aligned well with consumer demand, citing that consumers have priorities and agendas that are not necessarily aligned with prevention. The majority of respondents indicated that they did not know whether the Task Force's prevention priorities are aligned with other state or Federal initiatives because they did not have a strong sense of the national and state priorities.

Barriers to the Adoption, Integration, and Delivery of the USPSTF Recommendations

Health plans faced a number of common barriers with regard to adopting, integrating, and delivering the USPSTF recommendations including: patient resistance, staff availability, barriers to the delivery of counseling recommendations, barriers to the integration of certain types of USPSTF recommendations, availability of clinical preventive services in the system, geographic barriers to care, information technology barriers, process barriers, lack of local control over the recommendations, and other barriers. These challenges are not due to fundamental issues with the USPSTF recommendations, but rather the result of larger systems-level challenges that health plans face with respect to adopting and integrating clinical preventive services recommendations. Many of these barriers were not unique to one particular health plan, but were recognized by respondents across plans.

III. Improving Dissemination of the USPSTF Recommendations in Health Plans

Improving the Utility of the Recommendations

Suggestions for improving the utility of the recommendations ranged from disseminating more information about the Task Force's methodology to certain members of the health plan staff to developing new prevention tools specifically designed for nurses delivering counseling recommendations. Other key suggestions include: creating procedure codes or performance measures that coincide with the USPSTF recommendations in order to ease the process of integration; providing cost information about preventive services and programs; providing more information about how decision-making should occur for people who are slightly outside of the recommendation's age limit; disseminating full paper copies of the recommendations; and developing new patient level prevention tools.

AHRQ's Role in Improving the Dissemination of the Task Force Recommendations

Respondents indicated that AHRQ could play a key role in improving the dissemination of the Task Force recommendations and methodology. First, many respondents indicated that it would be useful if AHRQ could do more to disseminate the Task Force recommendations. Respondents suggested that AHRQ could attend provider professional meetings and present on a few of the Task Force recommendations. A number of respondents also indicated that AHRQ should disseminate more information about the Task Force's methodology for selecting and grading recommendations.

Thematic Overviews

NORC prepared five stand-alone thematic overviews which explore several key cross-cutting themes related to the adoption, integration, delivery, and dissemination of the USPSTF recommendations.

The Impact of Pay-for-Performance on the Delivery of the USPSTF Recommendations.
The Role of Health IT in the Integration and Delivery of the USPSTF Recommendations.
Systems-Level Changes to Encourage the Delivery of the USPSTF Recommendations.
Delivering the USPSTF Recommendations in a Rural Health Care Setting.
The Impact of Health Plan Structures on the Delivery and Integration of the USPSTF Recommendations.

Each overview presents best practices or key findings across the four health plans. Data from qualitative interviews with health plan respondents, expert interviews, and relevant reports and scholarly literature were used to produce the overviews.

Conclusions

From the health plans' experiences, we compiled our key findings and lessons learned to inform the AHRQ Prevention Team; the USPSTF; researchers and policymakers; and health plan administrators, clinicians, and other implementers of clinical preventive health services recommendations. We synthesize our lessons learned, suggest next steps for AHRQ, and present areas for future research and analysis.

Lessons Learned

Our lessons learned are presented in four key areas. The first key area discusses the impact of health plan structures on the integration of the USPSTF recommendations.

The remaining areas focus on our study findings in relation to three evaluation research questions, which have guided the development and direction of the study: 1) To what extent are clinical preventive services integrated into the four selected health care delivery systems? 2) What deficiencies exist in the delivery of clinical preventive services by these health care systems? 3) How can AHRQ contribute to the increased implementation of USPSTF recommendations among health care delivery systems? Overall, lessons learned include:

1. Health plan structures impact the integration and delivery of the USPSTF recommendations

Different health plan models were selected to explore whether health plan structure has an impact on the integration and delivery of the USPSTF recommendations. We found that health plan structure clearly impacts a number of variables related to the integration and delivery of the USPSTF recommendations, including health information technology, quality improvement, the process for adopting recommendations, and provider incentives.

2. Clinical preventive services, and specifically the USPSTF recommendations, are integrated into all four selected health plans

The USPSTF recommendations for clinical preventive services are being integrated into each of the four health plans, though the degree of integration varies across plans. While it is more difficult to track whether the USPSTF recommendations are being delivered, our conversations with respondents indicate that the perception is that the "A" and "B" recommendations are a high priority at each of the plans. The USPSTF recommendations play an important role in the process that health plans use to develop and adopt their own recommendations or policies for clinical preventive services.

3. Health plans face common challenges with respect to the delivery of clinical preventive services

Health plans face a number of common challenges and barriers with regard to adopting, integrating, and delivering the USPSTF recommendations, and recommendations for clinical preventive services, more generally. Common challenges include: time constraints; patient resistance; staff availability; availability of clinical preventive services in some practice settings; geographic barriers to care; IT barriers; process-related barriers; and difficulties associated with adopting and integrating counseling recommendations.

4. AHRQ can contribute to the increased implementation of USPSTF recommendations within health plans

AHRQ can play a key role in improving the implementation of the USPSTF recommendations by (1) disseminating the recommendations and the USPSTF's methodology to health plan staff such as Directors of Quality Improvement, and (2) improving the dissemination of the line of tools and products that incorporate the USPSTF recommendations, such as the Put Prevention into Practice materials, the Electronic Preventive Services Selector, the pocket manual of recommendations, and email updates.

Next Steps for AHRQ

Finally, our recommendations for AHRQ's next steps include:

Enhance the visibility of the USPSTF and its recommendations. Our interviews demonstrated that health plan leadership is not aware of different strategies that AHRQ uses to disseminate its USPSTF tools and products to consumers. In order to improve the adoption and integration of the USPSTF recommendations, AHRQ should take steps towards improving the visibility of both the USPSTF and its recommendations for clinical preventive services. AHRQ should also consider presenting the role of the USPSTF recommendations at professional health research and policy conferences that reach a broader health care audience. AHRQ could sponsor a membership organization for USPSTF users, which would foster a unique and productive opportunity for dialogue about the USPSTF recommendations. Possible activities for the membership organization include a USPSTF membership conference which would foster dialogue about important and timely issues related to clinical preventive services recommendations.
Create new USPSTF products and publicize existing ones. Given respondents' desire to learn more about the methodology that the USPSTF uses to select and prioritize its recommendations, we propose that AHRQ develop a small brochure about the USPSTF's methodology for distribution across health plans. We also propose that AHRQ further disseminate and publicize the availability of its prevention tools and the opportunity to join the USPSTF listserv online.
Work more closely with health plan leadership. Respondents recommended that AHRQ work more closely with their health plan leadership, such as the Medical Directors and Directors of Quality Improvement. Specifically, respondents suggested that AHRQ and the USPSTF develop collaborative relationships with their health plans, similar to the existing partnerships that plans form with other professional organizations that issue clinical preventive services recommendations.
Educate consumers about the USPSTF recommendations and prevention. Several respondents indicated the importance of educating consumers about the USPSTF recommendations. Respondents suggested that the USPSTF include its recommendations on popular web news services such as WebMD, which provides timely health information and tools for health management.

Key Issues for Future Study

This study has elucidated important lessons learned for the adoption, integration and delivery of the USPSTF recommendations in different types of health plans. We identify four key areas that merit further investigation to assist AHRQ in moving forward.

The integration of the USPSTF recommendations in hybrid plans. Future studies should explore whether the mixed-model structure underlying hybrid health plans offers unique incentives for providers to integrate and deliver the USPSTF recommendations for clinical preventive services.
Use of hybrid plans as ideal study sites to investigate the impact of plan structure on the USPSTF recommendations. Hybrid plans are the ideal study sites to investigate the impact of plan structure on the USPSTF recommendations because their mixed-model nature allows for a comparison of the key variables of open and closed-panel health plans while also controlling for systems-level differences. Future research in hybrid plans has the potential to identify new directions and interventions to increase the integration and delivery of USPSTF recommendations across all types of health plans. Studies could compare the implementation and integration of the USPSTF recommendations in hybrid plans between plan-affiliated and contracted providers. More broadly, as health plan structures continue to evolve, research focused on hybrid plans offers an opportunity to explore preventive service delivery within both traditional and emerging health plan structures.
Health plan use of the USPSTF recommendations during times of change and controversy. Our findings suggest that health plans strategically consult the USPSTF recommendations during times of change (e.g. clinical preventive services recommendations evolve for new diseases and conditions, such as obesity) and controversy (e.g. new science prompts the USPSTF and other organizations to revisit current recommendations for clinical preventive services). Further research should explore whether health plans do, in fact, consult the USPSTF recommendations in a strategic manner, and consequently, how AHRQ can improve the dissemination and visibility of the USPSTF recommendations during climates of change and controversy to improve their potential for adoption and delivery in health plans.
Competing recommendations for clinical preventive services. Further research will be necessary to understand why certain specialty organizations are referenced for different types of clinical services, and whether there are trends across different types of health plans.
Why are certain "A" and "B" USPSTF recommendations consulted more than others? Further research should explore why some of the "A" and "B" USPSTF recommendations are rarely consulted. One possible way to explore this issue would be to create a detailed matrix of the clinical preventive services recommendations for different types of health plans, and identify the roots of the recommendations (e.g., the USPSTF, American Cancer Society, etc). Conducting a small number of follow-up interviews with the plans to discuss the trends identified in the matrix would reveal why the "A" and "B" USPSTF recommendations are being implemented for certain clinical conditions and not others.
The functions of a USPSTF membership organization for health plans. In order to increase the visibility of the USPSTF and its recommendations, it may be beneficial to establish a membership organization that would foster a regular dialogue about the USPSTF recommendations amongst health plan staff. Future research should explore the possible functions of a USPSTF membership organization, including its construction, activities, and potential to facilitate improvements in the dissemination and delivery of the USPSTF recommendations in health plans.

^a. For the purposes of this study, Clinical Advisors are primary care physicians or specialty care providers, who also serve in a leadership or broader prevention role at their health plan.

Return to Contents
Proceed to Next Section