Strategic Plan

Center for Outcomes and Effectiveness Research, November 1998

Vision

To provide national leadership and direction for the development and use of outcomes and effectiveness research (OER) that supports improvements in patient outcomes.

Mission

To support improvements in health outcomes by conducting, supporting and facilitating research that is aligned with and informed by the needs of clinicians and patients in typical ("real world") practice settings.

Context

Since the establishment of the Agency for Health Care Policy and Research (AHCPR) in late 1989, interest among all stakeholders in the outcomes or end results of health care delivery has intensified. AHCPR's extensive and diverse portfolio of research has resulted in over 1,800 peer-reviewed publications, textbooks, recently established institutes and curricula, and new activity in outcomes research sponsored by multiple private-sector organizations and medical professional organizations. In addition, public concern that overzealous efforts to contain costs may have compromised the quality of care have stimulated new interest in translating the products of outcomes research into strategies to assess and improve health care quality.

Outcomes research was developed to address pragmatic questions confronting clinicians and patients daily:

• What works and what doesn't?
• For which patients?
• When in the course of an illness?
• At what cost?

The program at AHCPR has consisted of three major foci, complemented by a rich portfolio of complementary investigator-initiated research:

• The Patient Outcomes Research teams or PORTs (approximately 50 percent of the total investment).
• The Pharmaceutical Outcomes Research program.
• A program of centers focused on building capacity among minority health researchers and for the study of outcomes research for minority populations.

More recently, both within the second generation of PORTs (PORT IIs) and as discrete initiatives funded both by AHCPR alone (e.g., three trials examining the effectiveness of alternatives to hysterectomy for women with noncancerous uterine conditions) and in partnership with the National Institutes of Health (NIH), COER's portfolio has expanded to include selected clinical trials.

Criteria for the first generation (1989-93) of PORTs included a focus on specific conditions that are common, costly, and for which there was known to be well-documented evidence on substantial variation in practice. The PORTs were required to follow a very specific sequence and approach over a 5-year period. The second phase of PORTs (PORTIIs) offered investigators more flexibility in selecting research designs and methods, in part due to increasing recognition that the science base was much better developed in some areas (e.g., coronary artery disease) than others (e.g., most surgical conditions). The initial work quickly confirmed that the evidence for the vast majority of clinical practices was extremely limited.

Accomplishments to date include:

• Unprecedented overviews of health care delivery (i.e., an "aerial view of clinical practice" that provides an overview of services actually delivered or not delivered).
• Striking observations that challenge conventional wisdom and require further scrutiny.
• Findings in multiple clinical domains indicating that translation of medical knowledge into routine practice remains an enormous challenge (i.e., describing practice patterns and identifying challenges does not necessarily lead directly to improvements in practice).

Multiple studies have generated findings and measures that are now beginning to influence clinical practice on a large scale. Included in the current success stories that are just beginning to bear fruit are:

• The use of beta blockers to reduce mortality in patients who have recently had a heart attack.
• Decreased hospital admissions and improved hospital quality of care for individuals with pneumonia.
• Increases in the number of patients with a common heart rhythm disturbance who receive treatment to reduce strokes.
• Increases in the use of medication to improve lung function in premature infants.
• A patient-centered approach to the use of surgical treatment for men with enlarged prostates and a new emphasis on shared decisionmaking programs.
• Improved care for Medicare patients with cardiac disease.

There is now almost a routine acceptance by clinicians of the premise that scientific information should be applied to health care decisionmaking.

Outcomes research has also expanded the dimensions by which clinicians and patients evaluate the success of interventions to include those that are most important to individuals. OER has extended the focus of inquiry that informs health care decisionmaking from the individual patient to populations, from biometric readings only to measures that incorporate patient responses and values, and most importantly, from judgment based on some science and much experience to decisions grounded in scientific evidence relevant to heterogeneous populations.

From the outset, the opportunities for outcomes research to improve the end results experienced by individual patients as well as the measures by which improvement can be assessed have vastly exceeded available resources. COER's accomplishments to date underscore a collective need for better evidence to inform clinical practice—and a mandate to develop strategies that enhance the probability that better evidence will lead to enhanced health outcomes. In recent years, AHCPR has been joined by multiple private-sector organizations in conducting and supporting OER. This offers AHCPR and COER new challenges and opportunities in providing leadership for a burgeoning field.

Evolution of the sponsorship of OER is occurring concurrently with rapid changes in health care delivery, as well as comparable changes in information and communications technology that have important implications for the use and conduct of OER. In relation to the Agency's Strategic Plan, COER's focus is on supporting and conducting research that supports improvements in health outcomes. The primary audience for OER are clinical policymakers, including both individual clinicians and those who influence the landscape for clinical decisionmaking.

An essential component of all COER activities is summarized by the term "Strategic Science Development," a dynamic and explicit process that undergirds our efforts to support high quality research to promote improvements in health outcomes. This process is linked with an ongoing evaluation of the impact of AHCPR-supported OER and systematic analysis of OER supported by the private sector to inform future research initiatives, with the aim of aligning the products of the research enterprise with the needs of clinicians and patients.

A central challenge now and in the future is aligning the research enterprise, which has historically focused on publications, with the information needs of decisionmakers. (Select to access Figure 1 [4 KB].)

Where evidence has successfully altered clinical behavior, it has been combined with strategies and incentives for change at a local level. The challenge for the next generation of OER is to shift from a descriptive, exploratory enterprise to one that has a problem-solving focus (i.e., to move from "Gee Whiz" to "So What?!").

Achieving the greatest possible good with limited resources will require new approaches to identify the "outcomes of outcomes research" on an ongoing basis. It will also require a new conceptual approach for setting priorities for OER to clarify AHCPR's unique niche and enhance the sustainability of the entire enterprise.

Strategic Environment: 1998

COER's future planning must recognize important changes in the health care delivery system and within AHCPR itself. When the OER program was first initiated, some underlying premises included:

• New information on aggregate practice patterns could be presented to physicians in a nonthreatening fashion from a credible source, and their behavior would change. It cannot be overemphasized that this first premise is necessary but not sufficient to produce sustained changes in clinical practice.
• The primary audience, clinicians, are largely independent actors. Although physicians' socialization has not changed dramatically, in 1998 the vast majority of clinicians now work within one or more large groups. We now know that effective strategies to translate findings from OER into improvements in patient outcomes through clinical practice must link knowledge with supportive practice environments and incentives for change. Most successful efforts to improve outcomes have been conducted in hospital settings; we do not know yet how to extend these successes to decentralized organizations or how to take advantage of large practice organizations.
• AHCPR was the most important—and perhaps only—supporter of OER. In the past several years, an increasing number of private-sector organizations, predominantly but not exclusively in the pharmaceutical industry, have established "outcomes" units. An analysis conducted for COER by Lewin suggested that the unique Federal contribution in 1998 was the development of methods and tools, and support for training. (A report for the Institute of Medicine (IOM) commissioned by AHCPR in 1995 further identified a shortage of "outcomes researchers").
• "Effectiveness" research and clinical research are distinct entities, and (by inference) NIH is "not interested" in OER. There is substantial diversity across individual components of NIH. By 1998, several institutes, most prominently the National Cancer Institute (NCI), plan to establish "outcomes research" units, and the National Eye Institute (NEI) now requires a patient-reported visual function measure derived from the cataract PORT for all clinical trials. In addition, an increased number of clinical trials now include patient-reported outcome measures, and NIH is required by the Food and Drug Administration Modernization Act of 1997 (FDAMA) to establish a registry of trials. These developments suggest that AHCPR's unique contribution to OER may be to focus on individuals with multiple conditions.
• Further, effectiveness studies are simply equivalent to clinical studies that include heterogeneous patient groups. In 1998, it is clear that in addition to differences outside the system of care, differences within systems of care are as important to the intervention as the 'pure' clinical content.

Within AHCPR, COER's mission and work is most closely aligned with that of the Center for Organization and Delivery Studies (CODS), and the Center for Quality Measurement and Improvement (CQMI). Figure 2, labeled 'Key Focus of COER Research' depicts the interface between COER and CODS (16 KB). The interface and synergy between COER and CQMI is more challenging to articulate clearly. Better information regarding the relationship between the processes and outcomes of care can be used to expand the universe of available performance measures to assess—and improve—quality of care. However, this neglects the companion goal of OER to expand the metrics by which clinical interventions are assessed. To the extent that COER focuses its efforts on outcomes of care for individuals with multiple conditions, and can stimulate exploration of the relative benefits of using disease-specific vs. generic measures, COER and CQMI will have multiple opportunities for synergistic and complementary efforts. In addition, both COER and CQMI can benefit from joint understanding of factors that:

• Motivate interest in quality/outcomes improvement.
• Factors that enhance diffusion of knowledge and translation into practice change to prioritize combined and separate efforts.

Goal 1. Needs Assessment

Identify knowledge gaps where OER is needed and can improve patient outcomes.

Objectives

Combine expert opinion and empirical analysis (intramural assessments, synthesis of extramural research, and literature reviews) to determine where knowledge gaps and opportunities for improvement converge.

Strategies

1. Consult periodically with researchers and intermediate customers to establish an ongoing process for priority setting and identify opportunities for research and collaboration (Select to access Figure 3 [17 KB]).
2. Identify (on an ongoing basis) unique opportunities for AHCPR support and involvement, alone and in partnership with other funders.
3. Develop an empirical and conceptual assessment to inform priority setting.
4. Identify and followup unanswered questions from prior research investments. Questions to be answered include: Was the question worth answering? If so, have we or others supported needed follow-up studies? (If not, should we have funded the original study?) How can translation of findings be accelerated?
5. Develop specific strategies for use of convening function to provide leadership for OER with other funders. In addition to periodic meetings, develop plan for establishing an organizational structure to reinforce (e.g., Inter-Agency Committee on OER; Foundation for Health Outcomes Research)

Measures

• Based on the Outcome of Outcomes Research project and prior expert input, develop two publishable papers by March 1999 to clarify the next generation of OER for COER/AHCPR.
• Publish "RFI" (request for input) for CERTs.
• In fiscal year 1999, hold an initial meeting with invited researchers and intermediate customers (i.e., funders and users of OER) to gain input on how to structure an ongoing process for priority setting. Develop plan for "structural" options.
• Hire Visiting Scholar to assist in further development of conceptual framework for priority-setting that clarifies the interactions between biomedical science, clinical evaluation science, and health care delivery and financing.
• By the end of fiscal year 1999, develop an intramural research agenda that complements extramural research agenda.
• By the end of fiscal year 1999, develop and publish (internally or externally) a report that identifies unanswered questions from prior OER investments.

Goal 2. Knowledge Development (What and How)

Conduct, support, and facilitate the production of scientific knowledge that supports improvements in health outcomes.

Objectives

Conduct and support research that develops and validates the application of health outcomes measures and condition-specific information relevant to health care decisionmakers.

Strategies

1. Develop initiative for fiscal year 1999 that focuses on outcomes of care for older patients with chronic conditions, with a specific emphasis on vulnerable populations. (PORT III or "Vintage PORTs")
2. Support initiatives (as part of and separate from the above initiative) to develop innovative OER methodologies, including new health status measures, decision modeling, severity-of-illness measures, cost-effectiveness analysis, and statistical methods for effectiveness trials.
3. Support conferences and other activities that enhance synergy across individual projects (e.g., researchers focused on chronic care for older patients share information on methods and challenges).
4. Continue to refine emerging agenda for intramural research in COER, now including chronic illness and older women, cardiovascular OER, ethics, and cost-effectiveness analyses (CEA).

Measures

• By the end of 1998, develop an initiative (PORT III) focusing on outcomes of care for older patients with chronic conditions, with a specific emphasis on vulnerable populations.
• Bi-annually, write a paper ( for internal and/or external publication) summarizing/synthesizing the results of COER effort in supporting initiatives to develop innovative OER methodologies.
• Sponsor a Health Outcomes Symposium to push the boundaries of the current state of the science of outcomes measurement.
• Annually sponsor a conference to enhance interaction and synergy across individual projects.
• By the end of fiscal year 2000, develop and publish paper on the next frontier for OER (effectiveness trials and practice networks).
• Support and facilitate research to evaluate the appropriateness, effectiveness, and patient-reported outcomes of therapeutics, including pharmaceutical therapies (drugs and biological products).

Strategies

1. Support the identification of mutual areas of cooperation and opportunity with the Food and Drug Administration (FDA), Health Care Financing Administration (HCFA) , the Office of the Assistant Secretary for Planning and Evaluation (OASPE), the pharmaceutical industry, and other interested parties.
2. Design and implement the Centers for Education and Research Therapeutics (CERTs) program.
3. Support OER to produce new, innovative methods for determining the outcomes, appropriateness, and effectiveness of therapeutics.

Measures

• During fiscal year 1999, "formally" coordinate with FDA, HCFA, and OASPE to identify mutual areas of cooperation and opportunity.
• During fiscal year 1999, publish and disseminate summary of convergence of interests of stakeholders and incorporate into a Request for Applications (RFA) for CERTs.
• During fiscal year 1999, publish solicitation of RFA for CERTs and fund one or more centers in partnership with other stakeholders. RFA will include requirement for assessment of impact.
• Complete project on coverage decisions for Zyban.

Goal 3. Specific Emphasis: Clinical Economics/Ethics

Conduct, support and facilitate the production of knowledge that informs the efficient allocation of health care resources.

Objectives

A. Support development of tools to facilitate the conduct and comparability of CEAs.
B. Develop strategic assessment for performing CEA ("the CE of CEA").
C. Conduct and support research that promotes the use of CEA as a framework for decisionmaking.
D. Conduct and support research that examines the ethical implications of alternative criteria for health care resource allocation.

Strategies

1. Develop formal needs and opportunities assessment as per Goals 1 and 2, with a specific focus on tools needed for performing CEAs (e.g., development of a database for researchers' use for common cost inputs to CEAs).
2. Develop intramural capacity for targeting clinical decisions uniquely suitable for CEA.
3. Build capacity for studying ethical issues related to changes in the locus of decisionmaking for resource allocation.
4. Support CEA component of the National Emphysema Treatment Trial.
5. Work with CQMI to evaluate opportunities for applying CEA to quality improvement (QI) programs.
6. Identify the "right" opportunities to support CEA components of large trials funded by NIH and others.

Measures

• During calendar year 1998, complete a formal needs and opportunities assessment as per Goals 1 and 2, with a specific focus on tools needed for performing CEAs.
• By May 1999, recruit and hire national leader in CEA to lead cross-AHCPR effort.
• By the end of fiscal year 1999, publish Program Announcement articulating AHCPR's interests related to CEA.
• Conduct qualitative study to assess how decisionmakers balance cost and quality tradeoffs (i.e., rather than ask 'do you use CEA?' inquiry will focus on how decisions are made to inform use of CEA as a framework for decisionmaking).
• Work with internal staff and extramural community to develop supplement to Health Services Research on clinical applications of instrumental variables.
• Develop and implement quarterly ethics seminar series with the Center for Clinical Bioethics at NIH.
• Develop and implement strategy to encourage applications in ethics of resource allocation.

Goal 4. Translation

Support and facilitate the translation of scientific knowledge that leads to the optimal use of effective health care interventions.

Objectives

Promote the dissemination of OER to and uptake by "change agents," educational institutions, and clinicians.

Strategies

1. Develop internal system for frequent periodic assessment of key findings from extramural and intramural projects.
2. Disseminate OER findings at conferences, workshops, presentations, and briefings.
3. Explore strategies for inclusion of OER in educational curricula for health care professionals, including Continuing Medical Education (CME)/CE.
4. Develop and publish syntheses of findings and evidence of impact (both peer-reviewed papers and AHCPR documents or substrate for same).
5. Ongoing collaboration with Office of Health Care Information (OHCI) to identify and exploit opportunities for dissemination.

Measures

• By the end of fiscal year 2000, develop a system for periodically ascertaining the reaction(s) (e.g., changes in practice, changes to educational curricula) to key findings from AHCPR-supported extramural and intramural projects.
• Work with Research Translation Team on "Chartbook" for OER—slides and annotation.
• Complete COER Program Note.
• Publish two papers from the "Outcome of Outcomes Research."
• Update and synthesize results from Pharmaceutical Outcomes program.
• Publish the results of agenda-setting activities on ethical issues in paper for Health Affairs.
• By August 1999, publish paper with extramural team for mainstream clinical journal to describe the CEA for the National Emphysema Treatment Trial.
• Publish papers from intramural research: chronic illness and older women (conceptual overview—in press); analysis of subgroups with CHF as model for chronic disease; empirical analysis of Medicare's Health Outcomes Survey (HOS) and Consumer Assessment of Health Plans (CAHPS®) data for older women; conceptual and empirical analyses of strategies for assessing comorbidity in ambulatory care (minimum).
• Publish monograph that articulates the interactions and relationships between clinical evaluation science, biomedical science and health care delivery and financing.

Goal 5. Evaluation

Objective

Develop strategies for evaluating the impact of AHCPR-supported OER.

Strategies

1. Develop plan (with assistance of the Evaluation Officer) to assess:
   • Impact of AHCPR-supported OER.
   • The comparative successes and deficiencies of previously supported research—both topic area and mechanism.
2. Identify mechanisms for capturing translation of AHCPR-supported OER and impact of same.

Measures

• Complete first phase of "Outcomes of Outcomes Research" report (first draft to be completed and circulated for comments and feedback before the end of fiscal year 1998.
• Give Agency-wide briefing by end of calendar year 1998.
• Send "Outcome of Outcomes Research" report for external review.
• By the end of fiscal year 2000, develop internal expertise to facilitate identification of important findings. Such capability(ies) may be developed in conjunction with other operating units of AHCPR.
• To be developed.

Goal 6. Team Development

Recruit and develop staff with the requisite knowledge, skills, and abilities to accomplish COER's vision.

Objective

Build an interdisciplinary team that complement each other's abilities and who, together, can accomplish the Center's goals.

Strategies

1. Formalize Deputy position.
2. Recruit staff with appropriate knowledge, skills, and abilities.
3. Provide staff with appropriate educational and training opportunities to increase/improve their knowledge of OER and related knowledge, skills, and abilities.
4. Encourage staff to pursue activities of interest to them that are consistent with COER's mission.
5. Develop systematic internal approach for synthesizing results of previously and currently funded projects.

Measures

• By the end of fiscal year 1999, recruit two new staff members to expand expertise in outcomes measurement and CEA.
• During 1998 and thereafter, each member of COER will include an explicit goal for professional development in their Employee Performance Management System (EPMS).
• Develop Visiting Professor series (quarterly) for COER-wide professional development.
• During fiscal year 1999, investigate the feasibility of and options for developing a COER-wide plan for professional development.

Goal 7. Agency and Department Priorities

Contribute to and advance Agency and Departmental Initiatives, including Agency cross-cuts and special populations emphases.

Objective

Develop work plans for meeting/contributing to cross-cut and special population initiatives.

Strategies

1. Actively participate in working group meetings and specific plans for contributions and feedback to COER.
2. Provide prompt and useful responses to requests from cross-cut coordinators for minority health, children's health, and women's health.

Measures

• During fiscal year 1999, selected COER staff will develop explicit goals and expectations as part of their individual EPMS (in negotiation with cross-cut and special initiative coordinators).
• Work with child health coordinator on developmental (conceptual and strategic) work to establish foundation for major initiative in fiscal year 2000 on OER and children.
• Develop plan that articulates COER contributions to Department priorities in quality, child health and the race initiative.

Current as of November 1998

Internet Citation:

Strategic Plan, November 1998: Center for Outcomes and Effectiveness Research. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/coer/coerplan.htm

Center for Outcomes and Effectiveness Research
About AHRQ
AHRQ Home Page
Department of Health and Human Services