American business and government use performance measurement to improve efficiency and effectiveness, enhance the quality of products and services, and show accountability. Increasingly, public and private health organizations use performance measurement to link resources with outcomes, health risks, and various service and protection mechanisms. Their goal: To improve health in the United States.
In this issue, Prevention Report looks at how the U.S. Department of Health and Human Services (HHS) and State and local communities (see Spotlight) are applying performance measurement. This Focus article describes and shows how states can develop performance measures based on Healthy People objectives. A special Resources section provides many names and addresses, including World Wide Web sites, for more information.
Improving health care services through processes and measurement is nothing new. For years, accreditation of facilities and licensure and board certification of health professionals have been hallmarks of quality. The U.S. Preventive Services Task Force, the Office of Disease Prevention and Health Promotion (ODPHP), and panels of professionals convened by the National Institutes of Health, the Agency for Health Care Policy and Research, and other organizations have set clinical guidelines for health services for Americans of all ages.
In the 1990's, employers began monitoring the performance of health plans through the health Plan employer data information set (HEDIS). The Federal Government supported the expansion of HEDIS to measure performance of managed care organizations in serving Medicaid beneficiaries. The public health community has used the goals and objectives of Healthy People 2000 as a framework to drive the performance of Federal, State, and local public health, substance abuse, mental health, and environmental agencies.
As the examples below show, the emphasis is now on devising accountability and improvement strategies and outcome-focused data collection tools.
Performance measurement responds to the need to ensure efficient and effective use of resources, particularly financial resources. At HHS, it links the use of resources with health improvements and the accountability of individual stakeholders.
The most recent definition of performance measurement in the health literature comes from the Institute of Medicine:
". . . a continuing (and evolving) process-based in a context of shared responsibility and accountability for health improvement-for (1) selecting and using a limited number of indicators that can track critical processes and outcomes over time and among accountable stakeholders; (2) collecting and analyzing data on those indicators; and (3) making the results available to inform assessments of effectiveness of the intervention and the contributions of accountable entities."
This definition should be familiar to public health officials who have used Assessment Protocol for Excellence in Public Health (APEX-PH), an eight-step process for community health assessment and improvement, or know about the processes of total quality management (TQM) and continuous quality improvement (CQI).
The Government Performance and Results Act of 1993 (GPRA), a powerful initiative to reinvent government, aims to improve the efficiency and effectiveness of federally funded programs by holding Federal agencies accountable for spending public dollars. Under GPRA, Federal agencies must submit an annual performance plan, beginning with the President's fiscal year 1999 budget. The plan must include defined targets for performance goals, outcome indicators to measure progress toward the goals, a description of resources needed to meet the goals, a basis for comparing actual program results with the goals, a discussion of the process for validating the data that are collected, and an acknowledgment of the role of other parties in meeting goals.
GPRA's influence extends to the States, local jurisdictions, and other organizations that receive Federal funding. Such entities will be held accountable for describing the results that have been achieved with Federal dollars. For agencies using objectives such as Healthy People 2000 to track progress, the move to performance measurement should be a manageable next step.
Performance measurement still is being developed, and currently no single, agreed-upon approach exists. Since Prevention Report readers are familiar with tracking Healthy People objectives in Federal, State, and local agencies, the six steps in creating performance measures described below may be useful (see illustration). Understanding the similarities and differences between criteria for performance measures and health objectives will be helpful, too. Both performance measures and Healthy People objectives need to:
But performance measures also must:
Step One: Relate the performance measure to an important national, State, or local health priority area.
Healthy People 2000 is a consensus document that contains 319 health objectives for the Nation and can be used as a starting point for developing performance measures. In some public health programs, Healthy People objectives already have been adopted as program objectives.
For example, Congress first used Healthy People in the Omnibus Budget Reconciliation Act of 1989 to define reporting requirements under the Maternal and Child Health Block Grant. Beginning with the fiscal year 1991 report on Maternal and Child Health Activities and Health Status, State-specific information on infant mortality, prenatal care, and immunization rates is reported to Congress. The 1993 reauthorization of the Preventive Health and Health Services Block Grant required reporting based on Healthy People health status measures. Based on consultations with the States, a uniform data set was created based on Healthy People objectives in various priority areas, including tobacco, violence, and heart disease. The American Indian Health Care Improvement Act of 1992 also identified 61 Healthy People objectives to be included in reports to Congress.
Step Two: Measure a result that can be achieved in 5 years or less.
Performance measures need to reflect results that can be achieved in a relatively short timeframe (5 years or less) so that the impact of programs on the health of a population can be examined in real time. For health outcomes that cannot be achieved in a short timeframe, a measure scientifically linked to the outcome of interest can and should be used as the target result.
This is true in the area of chronic diseases, in which there are many long-term health status objectives of national importance that are unlikely to show changes in the first few years of health promotion activity. In fact, these objectives may not show changes for a decade or longer. Monitoring these changes is essential, but such measures cannot be used as a basis for evaluating performance.
For example, research shows that exercise, better nutrition, and smoking cessation can reduce the risk of coronary heart disease. Thus, improvements in exercise, nutrition, or smoking cessation can be used as performance measures.
Step Three: Ensure that the result is meaningful to a wide audience of stakeholders.
For example, the result in the Maryland illustration on the preceding page is meaningful to local schools, the State and local education departments, health agencies, parents, students, teachers, administrators, physicians and other health care providers, and many others.
Step Four: Define the strategy that will be used to reach a result.
Healthy People 2000 identifies some strategies for meeting long-term objectives for reducing risks and improving services. Many other documents and journals provide additional information about effective strategies. For coronary heart disease, the strategies are geared toward changing exercise, nutrition, and smoking behaviors. The Maryland illustration contains prototype performance measures from one of these areas: exercise.
When a strategy is community-based, gaining local input is essential. Local health departments often have a wealth of knowledge about how to communicate with community members and how to engage the community in developing strategies.
Step Five: Define the accountable entities.
Many organizations and individuals share responsibility for health improvements. Performance measures must go beyond describing areas of shared responsibility and identify results to be attained by each accountable entity.
In the further evolution of performance measurement, accountability needs exploration. Two key discussion points are how to define accountability better and how to ensure that successes and failures are treated as learning opportunities.
Step Six: Draft the performance measures and ensure that they meet statistical requirements of validity and reliability and have an existing source of data.
Working with epidemiologists and others at health agencies is essential in meeting
these requirements.
This example is based on the State of Maryland's Healthy Maryland 2000 document. Prevention
Report acknowledges Maryland's contribution to this special issue on performance
measurement.
STEP 1
| Relate the performance measure to an important national, State, or local health priority area. |
| Maryland has undertaken work related to the national health objective to reduce coronary heart disease deaths to no more than 100 per 100,000 people. |
STEP 2
| Measure a result that can be achieved in 5 years or less. |
| Maryland has identified an achievable result that is linked scientifically to the Healthy People 2000 Heart Disease and Stroke priority area: Increase the proportion of people who engage in light to moderate physical activity to at least 30 percent of the population. |
STEP 3
| Ensure that the result is meaningful to a wide audience of stakeholders. |
| Target stakeholders are essentially all Marylanders, with an emphasis on school-age children and people at high risk for diseases and medical conditions associated with physical inactivity (for example, persons with hypertension and high cholesterol). Stakeholders include principals, teachers, students, parent-teacher associations, the State education department, State and local health and recreational agencies, public health and medical professionals, and others. |
STEP 4
| Define the strategy that will be used to reach a result. |
The State of Maryland has selected four strategies:
|
STEP 5
| Define the accountable entities. |
| The accountable entities depend upon the strategies selected and the way in which a particular community is organized. For Maryland's strategy 2, these entities include schools, worksites, and community centers. For example, the Cecil County Public Schools have agreed to be accountable for specific tasks related to strategy 2 and are working in partnership with the Cecil County Health Department to offer healthy lifestyle programs to elementary school children. The programs, such as the Heart Challenge Course, bring teachers and food service workers together to promote healthy eating habits and physical fitness through educational games, classroom projects, and other activities that appeal to children. |
STEP 6
| Draft measures that meet statistical requirements of validity and reliability and have an existing source of data. |
| In consultation with biostatisticians and epidemiologists, organizations can draft measures that are statistically sound. One of Maryland's performance measures might be "Increase to 30 percent the proportion of students in each Cecil County elementary school who engage in light to moderate physical activity for 30 minutes or longer every school day by participating in school physical fitness activities." |
With its focus on the health of the population, Healthy People 2000 provides a framework for performance measurement. In terms of continuous quality improvement, this initiative has the longest history in HHS. The Public Health Service and its partners have used Healthy People for the past 18 years to set objectives and monitor targets that focus on reducing premature mortality and morbidity rates, as well as disabling conditions. (See Committee Actions 10 for a report on the Healthy People 2000 Consortium meeting.)
The Healthy People initiative reflects an unprecedented cooperative effort among government (Federal, State, and local agencies with public health, mental health, substance abuse, and environmental responsibility), voluntary and professional organizations, businesses, and individuals. Healthy People goals and objectives represent a national consensus about what it is possible to achieve through prevention. More than 10,000 people and groups commented on the draft documents of Healthy People 2000.
By engaging a broad range of partners to draft the objectives from 1987 to 1990, initiative planners hoped these groups ultimately would play a vital role in Healthy People 2000's success. And they have: 44 States have published year 2000 objectives, and 70 percent of local health departments reported to the National Association of County and City Health Officials that they are using elements of the Healthy People 2000 initiative.
As described above, Healthy People objectives have been incorporated into several Federal statutes, including the Omnibus Budget Reconciliation Act of 1989, the 1993 reauthorization of the Preventive Health and Health Services Block Grant, and the American Indian Health Care Improvement Act of 1992. Through such legislation, Congress has emphasized the importance of Healthy People as an initiative that tracks those health measures of greatest significance to the Nation. Healthy People objectives can serve as a framework for development of performance measures.
Healthy People contains many measures relevant to people doing performance measurement and provides direction for determining accountability. For example, about half of the objectives require action by people as individuals or as family members in the form of decisions about diet, exercise, alcohol consumption, tobacco use, and sexual practices. The other half of the objectives identify specific actors-for example, health care providers, schools, businesses, legislators, and government agencies. These objectives lend themselves to being adopted at a State or community level to track the contributions of various stakeholders in the community, such as whether physicians counsel about tobacco cessation or schools offer daily physical education or a comprehensive health curriculum.
Healthy People 2000 also has served as a catalyst for developing better data collection systems to measure the health status of Americans. The initiative has enhanced understanding of health status, risks to health, and use of clinical preventive services. These data now can provide the foundation to build even more refined performance measures.
There are, however, 75 objectives that lack a second data point by which to measure progress 7 years into the decade. Some data systems provide only one measurement a decade, for example, prevalence of depression, overweight, and dental decay. The frequency of data collection should be examined when developing performance measures.
As emphasis on performance measures grows, so will demand for better data and the timely release of those data. Monitoring performance will require as much current data as possible. Small area analysis with information by age, race, and gender will be a challenge. Greater emphasis on interventions will require new data collection among providers of services not traditionally surveyed. At a time of increasingly limited resources, engaging in performance measurement is a critical response to the public's demand for efficient use of public funds.
Lessons learned from the Healthy People experience can be applied to performance measurement. First, having agreement about what information needs to be measured has resulted in a commitment of resources to monitor and track the measures. Second, having a set of 10-year targets has served to direct the activities of stakeholders in the public and private sectors, even those over which the Federal Government has no direct authority.
Performance measurement offers a promise of improved management and improved outcomes. It builds on a long history and extensive experience in techniques to strengthen and improve government health programs. As the health community moves toward a future that includes performance measurement, program successes will follow.
The Healthy People initiative has served to focus the activities of many stakeholders
in the public and private sectors. These objectives can drive performance measurement by
creating targets toward which activities can be directed. In addition to health
objectives, performance measures need to include a short-term result, an improvement
strategy, and accountable entities. In addition, success with performance measurement will
rely upon the ability to create responsive data systems that generate timely data.
Back to ODPHP.