You Are Here: AHRQ Home > About AHRQ > Strategic Plan > Performance Plans and Performance Report

Performance Plans for FY 2003 and 2004 and Performance Report for FY 2002

Part 1. Overview of Performance Measurement

"What we really want to get at is not how many reports have been done, but how many people's lives are being bettered by what has been accomplished. In other words, is it being used, is it being followed, is it actually being given to patients? [W]hat effect is it having on people?"

—Congressman John Porter, Chairman, House Appropriations Subcommittee on Labor, HHS, and Education, 1998

Introduction and Rationale

The purpose of research is to produce information that can inform decisionmaking. There is increasing awareness among those who provide and receive health care services, those who pay for those services and those who are making policy decisions that health care should be research led and that the services which are provided should be evidence-based. As a result, research agencies must find a way to demonstrate the benefits of the research produced, not only in terms of how many research findings are published in professional journals but how the investment in research results in practical everyday applications that can be used by people who need information to make decisions about health care.

Demonstrating that research has led to tangible effects in the care provided to individual patients is difficult. Impact is not always immediate. For example, it may be several years for a health care organization, that has adopted a policy based on research funded by AHRQ, to learn what effect it has had on overall patient care. Knowing that some clinicians or health systems are changing their practice is different from knowing how overall practice patterns are being influenced and what the effect is on clinical outcomes.

Pyramid of Outcomes Model

To address the need to demonstrate the impact of research on people's health, AHRQ staff developed a "pyramid of outcomes" model that includes four levels of impact. At the base level is the impact on knowledge and further research development, at the top is the impact on patient outcomes.

This model of assessing impact of AHRQ-sponsored research forms the basis for the development of performance measures. Similarly, AHRQ must ensure that performance measures are developed to assess the impact of the research investment at all levels of the pyramid.

Performance measures aimed at the base of the pyramid focus on research that contributes to the health care knowledge base, leads to future research, or both. Research at this level includes the development of tools and research methods, instruments and techniques to assist clinical decisionmaking, and identify areas that do not have a sufficient evidence base. The process indicators that are developed to measure performance at this level of the pyramid assess the quantity or quality of activities that have the potential to contribute, at least indirectly, to helping AHRQ meet its strategic goals or to monitor the establishment of major new initiatives or implementation of improvements in core activities where significant resources are involved or the potential for significance of the ultimate impact is high.

Output indicators are developed aimed at capturing the impact at the second and third level of the pyramid. Research at the second level of impact is research that results in the creation of a policy or program by a professional organization, health plan, hospital, legislative body, regulator or accrediting organization. Similarly, impact at level three of the pyramid is defined as research that results in a change in what clinicians or patients do, or changes in a pattern of care.

AHRQ has developed outcome indicators to measure impact at the fourth level of the pyramid, that is impact on the quality of care, actual health outcomes, cost of treatment or access to health care. Often, however, the connection between a particular research project and health outcome is indirect and can take years to emerge. As a result, AHRQ has developed measures that utilize a "convergence of evidence" approach to establish a connection between research and outcomes. This involves identifying bodies of research which, when considered together, establish a connection between research and outcomes.

AHRQ Performance Indicators

Phase of Initiative	Year 1—research initiative starts	Years 3-5—results received	Years 3-10—results used in health care system
Indicator type	Process indicators	Output indicators	Outcome indicators
Indicator examples	Grants funded, creation of reports, partnerships formed	Publications, Web site, dissemination, research findings, reports, products available for use in health care system	Results of evaluation studies, users stories, analysis of trend/other data

Report/Plan Road Map and Budget Linkage

The AHRQ GPRA annual performance report and plans are aligned with the Agency's three budget lines:

Research on Health Care Costs, Quality, and Outcomes.
Medical Panel Expenditure Survey.
Program Support.

The first two budget lines are where Agency programs are funded. The goals associated with each of the budget lines represent core activities funded in each. The following two tables illustrate how the GPRA goals are aligned with the AHRQ budget lines.

Table 1, representing the GPRA goals for FY 2002, uses the cycle of research as a basic framework underpinning the development of goals and measures for AHRQ's budget line: Research on Health Care Costs, Quality and Outcomes.

Table 1: GPRA Framework FY 2002

What the Indicators Address	GPRA Goal
Budget Line 1: Research on Health Care Costs, Quality, and Outcomes
Cycle of Research Phase 1: Needs Assessment	GPRA Goal 1: Establish Future Research Agenda Based on User's Needs.
Cycle of Research Phase 2: Knowledge Creation	GPRA Goal 2: Make significant contributions to the effective functioning of the U.S. health care system through the creation of new knowledge.
Cycle of Research Phase 3: Translation and Dissemination	GPRA Goal 3: Foster translation of new knowledge into practice by developing and providing information, products, and tools on outcomes, quality, access, cost and use of care.
Cycle of Research Phase 4: Evaluation	GPRA Goal 4: Evaluate the effectiveness and impact of AHRQ research and associated activities.
Lead role for quality initiative	GPRA Goal 5: Support Department-wide initiatives to improve health care quality through leadership and research.
Budget Line 2: Medical Panel Expenditure Survey
Core MEPS activities	GPRA Goal 6: Collect current data and create data tapes and associated products on health care use and expenditures for use by public and private-sector decisionmakers and researchers.
Budget Line 3: Program Support
Agency management activities: contracts management and the AHRQ Intranet.	Goal 7: Support the overall direction and management of AHRQ.

Beginning in FY 2003, AHRQ has redesigned its strategic management system and revised its GPRA goals to align more closely with the Agency's strategic plan. Table 2 shows this revision and realignment and the strategies AHRQ will use to accomplish these goals.

Table 2: Revised GPRA Framework for FY 2003

What the Indicators Address	GPRA Goal and Strategies for Meeting the Goal
Budget Line 1: Research on Health Care Costs, Quality, and Outcomes
Strategic Goal 1: Support Improvements in the quality, safety and outcome of healthcare	To have measurable improvement in the quality, safety and outcome of healthcare for Americans. The National Healthcare Quality Report. The National Healthcare Disparities Report. HCUP Quality Indicators. Translating Research Into Practice (TRIP). Consumer Assessment of Healthcare Plans (CAHPS®). Accelerating the implementation of existing quality measures and safety practices. Developing capacity and new practices for quality and safety improvements. Detecting safety hazards and monitoring improvements in healthcare safety and quality. Centers for Education and Research on Therapeutics (CERTS). Evidence-based Practice Centers (EPCs). U.S. Preventive Services Task Force (USPTF). National Guidelines Clearinghouse™ (NGC).
Strategic Goal 2: Identify Strategies to Improve Access, Foster Appropriate Use and Reduce Unnecessary Expenditures	To develop the evidence base for policymakers and health systems to use in making decisions about what services to pay for, how to structure those services, and how those services are accessed. Excellence Centers to Eliminate Ethnic/Racial Disparities (EXCEED). Integrated Delivery System Research Networks (IDSRNs). Primary Care Based Research Networks (PBRNs). HIV Research Network. Healthcare Cost and Utilization Project (HCUP).
Strategic Goal 3: Build capacity to improve the quality of health care	To build the capacity for improving the Quality of Health Care Delivery through Research and Training. National Research Service Awards (NRSAs). Dissertation Research Grants. Career Development Awards. Minority Research Infrastructure Support Program (M-RISP). Building Research Infrastructure & Capacity Program (BRIC).
Budget Line 2: Medical Panel Expenditure Surveys
Core MEPS activities	To provide comprehensive, relevant and timely data on health care use and expenditures for use by public and private sector decisionmakers and researchers. Household Component (HC). Medical Provider Component (MPC). Insurance Component (IC).
Budget Line 3: Program Support
Enhance the value of AHRQ as the leader in Healthcare Outcomes, Quality, Cost, Use and Access	Maximize the value of AHRQ by developing efficient and responsive business processes, aligning human capital policies and practices with AHRQ's mission, building an integrated and reliable information technology infrastructure. Developing efficient and responsive business processes. Strategic management of human capital. Building an integrated and reliable information technology infrastructure. Integration of budget and performance.

This realignment will help AHRQ determine how well the basic knowledge which forms the core of AHRQ's work provides information that can be turned into actions by clinical decisionmakers, purchasers and providers who make decisions about what services to use and pay for and how to structure those services, as well as by policymakers.

Return to Contents
Proceed to Next Section