At different levels of government, performance measurement programs are underway. For example, Santa Clara County in California, the State of North Carolina, and several HHS agencies have launched performance measurement programs to evaluate health systems. These programs and processes of development and implementation are spotlighted as examples of approaches to performance measurement.
California's fifth largest county has developed comprehensive performance management (CPM), a county-wide system of managing public services based on measurable performance. CPM is both a product and a process. Santa Clara County has a training manual that uses an eight-step gameboard to teach department managers and staff how to develop CPM products and follow a management- employee involvement process. The products are missions, goals, and performance measures for the programs. The process requires managers and staff to participate fully as a team in discussions of what the program's picture of CPM should be.
The labor-management partnership has been important in implementing CPM. Through management-employee involvement committees (MEICs), every service unit or program forms a partnership between management and labor to assess the services provided in terms of the mission and goals, customers, and performance measures. In the health field, 12 MEICs operate in mental health, 3 in alcohol and drug services, 2 in public health (tuberculosis and teen pregnancy), 6 in the medical facility, and 3 in school-linked service (managed by public health but incorporating services from all health departments, as well as social services).
An example of a fully developed MEIC is the one in mental health on Clozaril/Risperidone. In this case, a committee and staff already existed for the purpose of reviewing severely mentally ill clients who were candidates for these antipsychotic drugs. The MEIC, which includes the committee and staff, has been able to create measures and modify the existing data system to collect the measures. This MEIC is well on the way to completion of its measurement process, though the process is expected to be fine-tuned over time.
As CPM enters its third year of development, approximately 40 percent of the county is using the process. Training is available for both staff and facilitators to assist MEICs in their work.
Santa Clara's next steps are gathering information to determine the degree of implementation and the need for support by its agencies, as well as revising its countywide mission and service priorities. With plans to tie performance measures to the budget by 1999, the county considers CPM an ongoing system of management in which services can be based on measurable results. (For additional information, write Mary Lou Fitzpatrick, CPM Coordinator, Office of the County Executive Administration, County of Santa Clara, County Government Center, East Wing, 11th Floor, 70 West Hedding Street, San Jose, CA 95110.)
Focusing on public health accountability, North Carolina is developing a three-level system, with Level I encompassing health status indicators; Level II, process and outcome measures; and Level III, the monitoring of activities.
Level I consists of a community wellness index (CWI), which provides a snapshot of health status in each county. The CWI has 30 outcome indicators divided into 8 categories, with the indicators generally consistent with or identical to Healthy People 2000 indicators. State officials have used the outcome indicators to generate composite rankings for each county and plan to disseminate broadly this local health status information to public health officials, the media, and the public.
Level II is the core of the accountability system, with 32 process and outcome measures that address how well the community is responding to local health needs. These measures are tied to the responsibilities of local health departments, as well as to those of the community, and include infant mortality rates and the receipt of well child care by Medicaid-eligible children. The State has considered demographics and geography, stratifying counties into four groups and generating rankings in each group. The Governor or the State Health Director will award superior performance in two categories: outstanding achievement and most improved. Low performers will meet with a team from the State Health Director's office to develop an action plan to be carried out and evaluated over 3 years.
Level III refers to the current monitoring activities, most of which are program specific or mandated by the Federal Government. By having a separate level of indicators, the State can ensure that required monitoring activities are carried out in a coordinated manner and that they complement but do not duplicate Level II assessments.
The State has been compiling data for accountability indicators through its health services information system and other sources (vital statistics, immunization registries) to provide counties with performance assessments. To create a more efficient data collection process, the State intends to evaluate its data system needs and to add essential elements and eliminate those that are not used. (For additional information, write Kevin Ryan, M.D., M.P.H., Chief, Women's Health Section, North Carolina Division of Maternal and Child Health, P.O. Box 27687, Raleigh, NC 27626-0587.)
Agencies in HHS are contributing to the evolution and consensus development for performance measurement. Since September 1996, HHS has been reviewing progress and shaping the future approach to performance measurement in selected public health programs. The HHS Performance Partnership initiative described herein has three core elements: performance measurement, administrative flexibility, and consolidation.
Performance Partnerships are an important theme for budget and legislative programs for this Administration. During 1997, the Department will continue to work in partnership with State and local governments to identify meaningful program outcomes and determine pertinent performance measures to assess program effectiveness. As described below, the success of these efforts depends on accommodating data and surveillance capacity issues and providing technical assistance and training.
In a related effort, the Department commissioned the National Academy of Sciences (NAS) to study existing performance objectives for specified grant areas, as well as the necessary data systems for implementing performance measurement.
NAS supports HHS' conversion from process objectives to more outcome-oriented measures but emphasizes the importance of data development. The draft NAS report points out the difficulty of articulating measures that accurately reflect success or failure in the program areas. The final report is expected in spring 1997.
The efforts of three HHS agencies-Centers for Disease Control and Prevention (CDC), Substance Abuse and Mental Health Services Administration (SAMHSA), and Health Resources and Services Administration (HRSA)-to move to performance measurement are described below.
CDC is developing health status indicators, uniform data sets, and public health surveillance; streamlining systems; and improving accountability for grants. Its State and local partners have called for better organization of reporting and data requirements.
In responding to its partners' requests, CDC has recognized the importance of focusing on outcomes for specific target populations to examine program effectiveness more directly.
By organizing nationwide meetings on performance measurement and collaborating with relevant committees, CDC has developed national recommendations for measuring program performance. The agency is attempting to integrate the development of performance measures reported by grantees and CDC (as required by the Government Performance and Results Act).
SAMHSA is working internally and with the States to advance performance partnership efforts. Discussions with the National Association of State Alcohol and Drug Abuse Directors and the National Association of State Mental Health Program Directors have ensured that SAMHSA's reauthorization addresses performance-based systems.
With the States and NAS, SAMHSA also is working to identify and implement specific performance measures for substance abuse and mental health. The Center for Substance Abuse Treatment has awarded contracts to 14 States to develop performance measures for substance abuse treatment. The Center for Substance Abuse Prevention is working with the States to identify prevention outcome indicators based on risk and resiliancy factors. In addition, the Center for Mental Health Services has a data development effort, the Mental Health Statistics Improvement Program, that is funding States to identify and use performance measures, including consumer-oriented indicators.
HRSA initiated the development of a performance management strategy with a full review of its programs and their readiness for measuring performance, using GPRA requirements as the basis for the review. HRSA is well along in its efforts for program-specific performance goals and measures within an integrated performance management strategy. HRSA linked its performance and strategic goals in the five Annual Performance Plans submitted for the 1998 budget and is extending these linkages for the 1999 submissions. In addition, the agency is realigning its strategic plan more closely to GPRA guidelines. HRSA is involving a wide range of partners and obtaining expert technical assistance.
An example of HRSA's efforts is the development of performance measures for its Maternal and Child Health Block Grant program. Started in summer 1996, the task was coupled with an effort to make technical revisions to the block grant application guidance from HRSA's Maternal and Child Health Bureau (MCHB). The committee responsible is led by the MCHB Director of the Office of State and Community Health. Activities to date have included compiling indicators from a variety of sources and categorizing them by population groups and by the MCHB Pyramid of Health Services (direct personal services, enabling services, population-based services, and infrastructure building). In March, MCHB presented its measures to the Association of Maternal and Child Health Programs and is scheduled to send draft guidance to States in April. Over the summer, MCHB will conduct pilot tests and by November have final guidance to States for use in 1998.
The Focus and Resources sections provide additional examples and opportunities to learn
more about performance measurement at local, State, and Federal levels. International
resources also are listed. Through performance measurement, the Federal Government and
organizations around the world increasingly are managing for results.
Back to
ODPHP.