By Somnath Saha, M.D., M.P.H.a,c, Thomas J. Hoerger, Ph.D.d,e, Michael P. Pignone, M.D., M.P.H.d,f, Steven M. Teutsch, M.D., M.P.H.g, Mark Helfand, M.D., M.S.a,b,c, and Jeanne S. Mandelblatt, M.D., M.P.H.h, for the Cost Work Group of the Third U.S. Preventive Services Task Force*.
Address correspondence to: Somnath Saha, M.D., M.P.H., Portland VA Medical Center (P3MED), 3710 SW U.S. Veterans Hospital Rd., Portland, OR 97207.
This article originally appeared in the American Journal of Preventive Medicine. Select for copyright and source information.
Abstract
Introduction
Potential uses of CEA in informing preventive service recommendations
Limits of using CEAs to inform preventive service recommendations
The USPSTF approach
Methods to review CEAs to inform recommendations for preventive services
Conceptual approach
Developing the instrument and process for abstracting data
Process for reviewing studies
Conclusions
Acknowledgments
References and Notes
As medical technology continues to expand and the cost of using all effective clinical services exceeds available resources, decisions about health care delivery may increasingly rely on assessing the cost-effectiveness of medical services. Cost-effectiveness is particularly relevant for decisions about how to implement preventive services, because these decisions typically represent major investments in the future health of large populations. As such, decisions regarding the implementation of preventive services frequently involve, implicitly if not explicitly, consideration of costs. Cost-effectiveness analysis summarizes the expected benefits, harms, and costs of alternative strategies to improve health and has become an important tool for explicitly incorporating economic considerations into clinical decision making.
Acknowledging the usefulness of this tool, the third U.S. Preventive Services Task Force (USPSTF) has initiated a process for systematically reviewing cost-effectiveness analyses as an aid in making recommendations about clinical preventive services. In this paper, we provide an overview and examples of roles for using cost-effectiveness analyses to inform preventive services recommendations, discuss limitations of cost-effectiveness data in shaping evidence-based preventive health care policies, outline the USPSTF approach to using cost-effectiveness analyses, and discuss the methods the USPSTF is developing to assess the quality and results of cost-effectiveness studies. While this paper focuses on clinical preventive services (i.e., screening, counseling, immunizations, and chemoprevention), the framework we have developed should be broadly portable to other health care services.
Keywords: economic models; costs and cost analysis; cost-benefit analysis; methods; MEDLINE; preventive health services; evidence-based medicine; practice guidelines.
At the close of the twentieth century, health care costs in the United States continued to rise steadily, accounting for 13.5% of the gross domestic product in 1998 (1), and debate on health care funding for the aging American population intensified. In this environment, preventive services often compete with one another and with diagnostic- and treatment-oriented care for increasingly constrained resources (2). While preventive services are often believed to save costs, delivery of most preventive services, with few exceptions (e.g., some immunizations), incurs net costs (3).
Cost-effectiveness analyses (CEAs) summarize the expected benefits, harms, and costs of adopting and translating a clinical recommendation into practice (4). The results of a CEA are typically presented as a ratio of the net costs to the net health outcomes of alternative intervention strategies, illustrated in the formula:
(C1-C2)/(O1-O2),
where C represents costs associated with an intervention, O represents outcomes, and 1 and 2 refer to alternative interventions. Costs associated with an intervention include the costs of the intervention itself plus those induced by the intervention (e.g., the costs of treating side effects), minus the costs averted because of the intervention (i.e., the costs of care for the prevented disease).
Outcomes in CEAs may be measured in different ways. Frequently, they are measured as life-years saved (LYS). While this measure accounts for how an intervention strategy affects mortality, it does not reflect the quality of life associated with different health outcomes. To capture the effect that intervention strategies have on both loss of and quality of life, the number of years with an illness or injury can be multiplied by a value weight from 0 (death) to 1 (full health) to generate quality-adjusted life-years (QALYs) (5). CEAs that use QALYs in the denominator of their cost-effectiveness ratios are often referred to as cost-utility analyses (CUAs), because they incorporate people's preferences, or utilities, for different states of health, illness, and injury. In our discussion, we treat CUAs as a subset of CEAs.
Economic analyses other than CEAs, such as cost-minimization analyses and cost-benefit analyses (CBAs), also provide information about the potential value of health services. CEAs differ from most of these types of analysis in that they describe how different strategies for allocating resources affect health outcomes (4). For the third U.S. Preventive Services Task Force (USPSTF), this focus on health outcomes reflects an essential element of the approach to reviewing evidence. CBAs also incorporate health outcomes, but convert outcomes into dollars. Because assigning dollar values to health outcomes is controversial and not frequently used in U.S. health policy, the USPSTF has chosen to focus primarily on CEAs.
By quantifying the immediate and downstream benefits, harms, and costs of interventions, CEA demonstrates the trade-offs involved in choosing among different intervention strategies to effect desired health outcomes. As such, CEAs may provide valuable information for those designing or implementing policies about preventive services. In this paper, we discuss uses of CEA to guide policies related to clinical preventive services, some of the limitations in using CEA to inform recommendations for services, and the approach and methods the USPSTF is developing to systematically review CEAs and incorporate cost-effectiveness data into its process for developing recommendations for the forthcoming third edition of the Guide to Clinical Preventive Services (the Guide), slated for publication in 2003.
Evidence-based recommendations are increasingly used to help determine which preventive and other services to include in clinical practice, public health programs, and benefits packages (6,7). Most of these recommendations have not systematically incorporated evidence related to cost-effectiveness. CEAs, however, can be used in several ways to inform and extend clinical service recommendations (Table 1).
[a] The current U.S. Preventive Services Task Force will not use cost-effectiveness analyses for this purpose.
One of the most important and common uses of CEA is to examine the costs and health benefits associated with alternative interventions to achieve a given health outcome. For instance, effective screening strategies for colorectal cancer include fecal occult blood testing (FOBT) (8), sigmoidoscopy (9), and possibly barium enema and colonoscopy (10). More effective screening strategies often cost more than less effective strategies. Depending on available resources, some might screen initially with FOBT, which provides moderate effectiveness at relatively low cost, while others might choose colonoscopy, which is more costly but may also be more effective (11-14). CEA makes explicit the trade-offs involved in these decisions.
CEAs can also help in selecting the most efficient application of effective interventions, such as the intervals between screening tests (15,16), the ages for starting or stopping a service (17), or the population subgroup likely to benefit most from a service (18). Early detection of cervical neoplasia provides a pertinent example. In a CEA of cervical cancer screening, Eddy (15,16) demonstrated that Pap testing every 3 years saved 97% of the lives that would be saved using annual screening and reduced costs by 67%. These results prompted several leading professional groups to issue a joint statement changing their recommendations to include triennial screening as a valid option (16). Other CEAs have noted that screening women aged >65 who have had limited prior testing or are at high risk for cervical cancer can save both health care dollars and lives (19). In contrast, screening low-risk women with a history of regular screening before age 65 increases the cost-effectiveness ratio more than tenfold over results for all women (20). These results suggest that recommendations for screening older women for cervical cancer should be stratified according to risk of disease and prior screening history.
CEAs can also be used to evaluate new technologies related to prevention. For example, tests for mutations in the BRCA1 and BRCA2 genes, which confer high lifetime risks of developing breast and ovarian cancer (21), can identify high-risk individuals, who can then be counseled about intensive surveillance for early disease detection or prophylactic surgery (22,23). It is not clear, however, how such tests should be used in general practice. In this situation, one can model the costs and health consequences of using BRCA testing in different populations and calculate the marginal cost and effectiveness. For example, one could conduct a CEA to compare the cost-effectiveness of screening all women versus screening only those with a strong family history of breast cancer. The results of such an analysis could help set policies about who should receive genetic testing and counseling.
CEAs can also help determine how factors that are not typically considered in clinical trials of intervention efficacy might influence the "real-world" effectiveness of a preventive service. For instance, there is evidence that doxycycline and azithromycin are equally efficacious in eradicating genital chlamydia infection in women (24). It might seem reasonable to recommend doxycycline as the antibiotic of choice, since it is less expensive than azithromycin. However, when one considers that azithromycin is given as a single dose and is therefore associated with higher rates of adherence than doxycycline, azithromycin may in some settings improve outcomes compared with doxycycline and therefore be the more cost-effective choice (25).
CEAs—more specifically, CUAs—can also be used to make explicit the impact of the target population's preferences for different health outcomes. By using QALYs as an outcome measure, CUAs account for the fact that most people prefer some states of health and illness to others. These preferences may need to be considered in deciding whether or not to implement a given preventive service. For instance, clinical trials typically report the effectiveness of tamoxifen in reducing overall morbidity and mortality as a balance between the benefits and harms of the drug, but fail to account for the different values that women place on various outcomes (e.g., breast cancer and endometrial cancer). In a CUA, the net effectiveness of the intervention is modeled as a weighted sum of benefits and harms, where the weights reflect women's preferences among these potential outcomes. Such information may be useful in establishing policies for large populations.
Finally, if CEAs are conducted in a standardized manner using the same units for measuring outcomes, preventive services can be ranked in a "league table," a listing of interventions in order of their costs of saving a year of life or QALY (26,27,28). In a setting of limited resources, these methods for prioritizing services could be used as a guide in providing services for a population to maximize overall health for a given investment.
Overall, CEAs are a dynamic tool for developing and adapting effective preventive service interventions to obtain the best value for the greatest number of individuals at risk for poor health outcomes. However, they also have shortcomings. In the following section, we present a summary of limitations associated with using CEAs to inform evidence-based recommendations.
Prior USPSTFs and other groups have chosen not to incorporate cost-effectiveness into their recommendations for several reasons, including a historical lack of standardized, high-quality CEAs; a paucity of CEAs for many preventive services; the questionable validity of QALYs in capturing preferences; a lack of transparency in the complex models used in many analyses; and concerns about the ethics and politics of rationing.
Until the last decade, there were few efforts in the United States to standardize the conduct of CEAs. As a result, many published analyses would not meet current criteria for high-quality CEA research (29). Lack of standardization has made it difficult to compare studies about a specific preventive service or across different interventions (30,31). In addition, for many services there has been a paucity of cost-effectiveness data, and for some services there are no economic evaluations at all.
Expanded use of QALYs as an outcome measure may improve the comparability of CEAs. Current use of QALYs, however, is limited by the shortcomings of existing preference measures, many of which are unvalidated and narrow in focus, not fully capturing the multifaceted nature of personal preference. QALYs as a measure are also not intuitively understood by most clinicians, policymakers, or patients. Moreover, results from a CEA using preferences of the general population to generate QALYs may not always be helpful to clinicians in their offices caring for individual patients with specific values that may differ from the average.
Using CEAs in making health decisions has also raised ethical concerns. CEAs typically assume that a year of life saved or a QALY for an infant is equal to that for a 70-year-old, or that a gain of 1 year for one person is the same as a gain of a tenth of a year for ten people. These assumptions could be considered the least biased and a protection against discriminating against any group, but they may also inadequately reflect societal and individual values (32,33).
Other ethical concerns relate to the use of CEA to ration health care services. Implicit in the use of CEAs in developing clinical recommendations is that societal health care resources are limited. In some industrialized countries this assumption, and the rationing of health care services, are made explicit. In the United States, CEAs have only rarely been used explicitly to set recommendations about services or funding levels. In the best-known case, the Oregon Health Services Commission ranked health care services according to their cost-effectiveness, in order to expand health care coverage among low-income populations within the constrained Medicaid budget (34). This approach was abandoned in the face of public criticism; many observers argued that rationing is not ethical, and that individuals should have access to all effective interventions (35,36). Patients and health professionals alike expressed concern about the prospect of potentially valuable services being withheld to save money. In Oregon, cost-effectiveness was ultimately used as one of several considerations in setting Medicaid-covered services. The Oregon experience illustrates that ranking services by their cost-effectiveness ratios may be a useful but not likely sufficient process for developing clinical service priorities. Most observers recognize that while CEAs are an important decision-making tool, they are not the only tool, and that many other factors, including ethical implications, public perceptions, and political and operational feasibility, need to be considered when prioritizing health care spending.
The perspective of a CEA—whose costs and benefits are considered in the analysis—is also important. A CEA using the societal perspective ideally incorporates an intervention's impact, in terms of costs and outcomes, on all members of society. If, however, an analysis is conducted from the perspective of a specific organization, such as a managed care provider, other costs (e.g., out-of-pocket patient costs and caregiver time) that are not relevant from the provider's perspective are not considered (30). Such an analysis may be useful only to a select group of decision-makers. Moreover, because studies from varying perspectives assess different sets of costs and outcomes, they are not directly comparable. The Panel on Cost-Effectiveness in Health and Medicine (4) has recommended that all CEAs include the societal perspective. One limitation of this approach, however, is that the societal perspective may not reflect the specific concerns of some parties interested in using CEAs, such as individual practitioners, patients, or health care organizations. Another limitation is that enumerating all of the costs society experiences is often more difficult than quantifying those experienced by a specific entity, such as a health care system.
For these and other reasons, the USPSTF has, in the past, not used CEAs to inform its recommendations. Over the past several years, however, the quantity and quality of CEAs have increased substantially (29). Theoretical refinement, methodologic advances, and the development of standards for reporting and conducting CEAs have improved their usefulness as decision-making tools. Moreover, as the cost of health care delivery has continued to rise, policymakers have found it increasingly difficult to decide about implementing clinical services solely on evidence of effectiveness, without consideration of costs. The third USPSTF has therefore decided to incorporate information from CEAs into its process for developing recommendations. While CEAs may never be able to address all of the complex issues involved in deciding how to allocate health care resources, their ability to quantify the trade-offs involved in choosing among different alternatives to improve health make them a valuable source of information in deciding which clinical preventive services to select and implement.
The USPSTF will conduct systematic reviews of CEAs to inform its recommendation process. These reviews will not replace the USPSTF's harms-benefits analyses and will not be conducted for every topic, but rather where relevant questions about cost-effectiveness exist. Most of these questions will address the trade-offs between two or more effective strategies for achieving a given health outcome; the cost-effectiveness of applying an intervention at varying intervals or to different target populations or risk groups; and the impact of factors such as adherence that may affect the costs and effectiveness of a given intervention. Examples of questions for which a CEA review might be initiated include the following: What is the cost-effectiveness of screening average-risk adults aged 20 to 30 for dyslipidemia? What is the comparative cost-effectiveness of universal versus selective screening for chlamydial infection among women aged <25? What is the impact of varying levels of adherence on the cost-effectiveness of colorectal cancer screening using sigmoidoscopy versus colonoscopy?
The USPSTF has set specific requirements for the use of CEAs in its recommendation process. First, there should be reasonable evidence that the intervention in question is effective. When definitive proof of effectiveness is difficult to achieve, the USPSTF may consider information from CEAs that make reasonable assumptions about the intervention's likely effectiveness. In general, however, the USPSTF will initiate a CEA review only for services where evidence of effectiveness exists.
Second, the USPSTF will use only economic analyses assessing the costs associated with achieving health outcomes. The results of studies that examine only intermediate measures, such as those reporting costs per patient screened or costs per case detected, are less interpretable in that the value of what is achieved for the costs accrued is less clear. These studies will therefore not be included in USPSTF CEA reviews.
Third, the USPSTF will focus primarily on CEAs that are conducted from the societal perspective. It will also give highest priority to studies in which valuations of outcomes, or QALYs, are derived from the perspective of general, rather than selected, populations (4). In this manner, consistent with the USPSTF perspective, CEAs used in making decisions should reflect the public interest, and not be biased by any group that stands to gain or lose by the implementation of a particular preventive service (26). Many CEAs include most but not all relevant costs accrued by society. Costs such as time and resources expended by patients and caregivers are frequently omitted. Such studies may nonetheless provide valuable information and will be considered. When reviewing these studies, the USPSTF will consider how including the relevant societal costs might have affected the studies' results.
Finally, the USPSTF will not create league tables to rank preventive services in order of cost-effectiveness. Although the USPSTF strives to provide as much information as possible to guide decision-makers in their use of preventive services, a preliminary review of the cost-effectiveness literature for several preventive services revealed that the quality and comparability of CEAs across services were not sufficient to allow direct comparisons and rankings. As standardization of methods and comparability of CEAs across preventive, diagnostic, and therapeutic services improves, future Guides may include league tables to allow for prioritizing services based on costs and expected benefits. In the meantime, Partnership for Prevention, a national nonprofit organization, has developed methods to estimate the relative value of services recommended by the second USPSTF to guide decision-makers in prioritizing effective preventive services (37).
To sum up, CEAs are valuable decision-making tools that, when properly applied, can help maximize efficiency and appropriateness in health care delivery. The USPSTF recognizes that CEAs have limitations and that their use in informing health policies is controversial. It also recognizes, however, that users of the third Guide, if not provided with information from CEAs, are still likely to use economic analyses, potentially in an unsystematic way. The USPSTF will therefore attempt to provide unbiased summaries of cost-effectiveness data where relevant, based on a systematic approach to, and critical appraisal of, the literature. In circumstances where information on cost-effectiveness is unavailable but is felt to be vital to the shaping of a recommendation, the USPSTF may conduct an original CEA.
Systematically reviewing CEAs shares much in common with the process for reviewing studies of intervention effectiveness (38). The goal in both cases is to identify the best available evidence regarding a specific question and to critically review and synthesize that evidence in order to answer the question in an evidence-based way. However, reviewing CEAs is also fundamentally different from reviewing effectiveness studies. For example, in a systematic review of the effectiveness of a single intervention, the effect sizes from several clinical trials, which represent unique samples of data gathered in a similar fashion, might be pooled in order to obtain a more accurate estimate of the intervention's effectiveness. It is difficult, however, to combine the results of several CEAs into a single cost-effectiveness ratio, because they often do not involve primary data collection and are frequently based on assumptions and models that vary in a way that may make combining results from multiple studies difficult or conceptually unsound. In addition, while the USPSTF's systematic reviews of intervention effectiveness involve developing an analytic framework of the various components of effectiveness and critically appraising the literature related to each component separately (38), CEAs typically use decision analytic modeling to include all the components of such a framework in a single study.
Why then, conduct a systematic review of CEAs rather than identify a single study that addresses the question at hand? CEAs vary widely in their methods and assumptions. Because of this variation, systematically reviewing CEAs provides several benefits. First, because CEAs draw on a variety of cost and effectiveness data sources to develop input parameters, a systematic review can identify which analyses use the best available evidence for key inputs and are therefore the most evidence based. Second, because the credibility of CEAs rests on their quality, a critical review of CEAs and a rating of the quality of each allow for identifying the most methodologically rigorous studies. Third, a comprehensive review can identify the studies that best address the question being asked. Fourth, comparatively assessing CEAs can help to identify variables and methods that significantly influence the estimated benefits and cost-effectiveness of an intervention. For instance, some CEAs might assume no harms from a given intervention, while others might assume that the intervention has significant harms. Comparing these studies side by side may provide insight into how the assumption or lack of assumption of harm affects the estimated benefit of the intervention. While some assumptions are varied within a single study using sensitivity analysis, most CEAs provide a limited number of sensitivity analyses. Thus, systematically reviewing CEAs may help identify, through a side-by-side comparison that amounts to a "virtual sensitivity analysis," the impact of different assumptions on the benefits of a given intervention. Finally, the more high-quality, independently conducted CEAs there are for a given intervention, the more convincing the evidence.