Uses of Quality Measurement
Contents
Program Management
Accountability
Quality Improvement
Reporting Results
Want More Information?
A clear plan for using quality measurement is a prerequisite to a
successful quality measurement strategy. An established plan helps
ensure that decisions are made with the project purpose in mind.
This section:
- Describes the major uses of quality measurement.
- Identifies questions to help assess possible uses.
- Lists key quality measurement reporting issues.
Uses of quality measurement include:
Program Management. Overseeing key functions
to ensure that program goals are met and resources are used efficiently.
Accountability. Demonstrating achievement
of identified goals.
Quality Improvement. Devising and tracking the impact
of targeted interventions designed to improve health services.
Reporting Results. Reporting to a variety of stakeholders and other audiences.
Return to Contents
Program Management
Regular measurement of quality is an important program management
tool because it:
- Promotes the effective use of scarce resources and delivery of needed
services.
- Provides information needed to manage health plans, providers, and
other vendors and contractors.
- Provides a basis for offering incentives to these parties.
- Allows comparison of plan or program performance with that of other
plans or programs in the State or in other States.
- Allows the tracking of trends over time.
- Provides an objective basis for ongoing quality improvement programs.
- Facilitates customized reporting to multiple constituencies.
The Federal Balanced Budget Act of 1997 mandated
increased attention to performance monitoring as a tool for ensuring the
delivery of quality services in Medicaid and the State Children's Health
Insurance Program (SCHIP).
Online Resource: To review excerpts from the law, go to: http://www.ahrq.gov/chtoolbx/fbba97.htm
In response to this requirement, the Centers for Medicare & Medicaid
Services (CMS) formed the Performance Measurement Partnership Project
(PMPP) to select a common set of measures that can be used by Medicaid
and SCHIP programs to assess the quality of care. Final recommendations
of the PMPP, which is composed of State and Federal officials, are expected
in 2004.
The development of mandatory Title V performance measures by the Health
Resources and Services Administration’s Maternal and Child Health
Bureau and the development of Medicaid performance measures by CMS in
response to the mandates of the 1993 Federal Government Performance and
Results Act (GPRA) have also brought increased attention to program performance
issues.
Online Resources:
For more information on Title V performance measures, go to: http://www.ahrq.gov/chtoolbx/measure5.htm
For more information on the Maternal and Child Health Bureau, go to: http://www.mchb.hrsa.gov
Return to Contents
Strategy Development Questions
As you develop your program management strategy, here are some questions
to consider:
- How do you monitor health plans, providers, and outside vendors and
contractors? Could quality measurement help you hold them accountable?
- How do you enforce contractual requirements? Could quality measurement
provide you with better information?
- How do you learn whether your program is accomplishing its intended
goals? How do you know whether the decisions you make have their intended
impact?
- How do you build staff expertise in quality oversight and monitoring?
Could a quality measurement initiative help create and reward performance-oriented
staff expertise?
Return to Contents
Resources for Program Management
You may wish to explore two AHRQ program management
tools:
- The National Guideline Clearinghouse™ (NGC) is an
electronic resource of established, evidence-based guidelines for a
wide variety of health conditions, including some relating specifically
to children.
Online Resource: For more information on the National Guideline
Clearinghouse™, go to:
http://www.guideline.gov/
- AHRQ's evidence reports and technology assessments review the relevant
scientific literature on certain clinical care topics for others to use as
a basis for clinical guidelines and other quality improvement activities.
Some of these topics relate to child health issues. These resources are also
useful for quality improvement.
Online Resource: For a list of AHRQ's evidence report topics, go to: http://www.ahrq.gov/clinic/epcix.htm
For more information on these AHRQ Program Management Tools, go to:
http://www.ahrq.gov/chtoolbx/useful.htm
Return to Contents
Accountability
Accountability in publicly funded health care programs is a top priority.
The use of quality measurement allows program managers to:
- Hold health care plans and providers responsible for the services
they furnish.
- Assess the extent of their own accomplishments.
Example: Both providers and program managers may be held accountable
for seeing that children are receiving the appropriate number of well-child
visits or that children with special health care needs receive timely
care.
Those in the legislative and executive branches can also use quality
measurement information to:
- Assess the strengths and weaknesses of a program.
- Identify whether the decisions they have made are having the intended
results.
When developing a quality measurement strategy, consultation with
all stakeholders can help identify areas of interest.
Return to Contents
Strategy Development Questions
As you consider using quality measurement for accountability, some
questions to assess include:
- Who are your stakeholders? What are their needs and expectations?
Do they expect quality measurement reports? Could quality measurement
reports provide them with valuable information?
- What goals have your legislative and executive officials established
for your program? Could quality measurement information help them
assess whether these goals are being met?
- What are the requirements of your funders? Do they require quality
measurement reports? Could quality measurement reports provide
them with valuable information?
Return to Contents
Quality Improvement
Quality measurement provides one of the tools needed for effective quality
improvement initiatives. You can use quality measurement to establish
the initial or baseline level of performance and to remeasure performance
after the quality improvement intervention has begun.
Example: You can measure the current baseline rate of adolescent immunization,
the hospitalization rate for children with asthma, or parental satisfaction
with wait times in doctors' offices. You can then use the same performance
measurement tool to remeasure that same indicator after the quality improvement
intervention has been put in place and get a before-and-after comparison
of the results, such as:
- Have adolescent immunization rates improved?
- Has the hospitalization rate declined?
- Has satisfaction with wait times increased?
Without using quality measurement to assess the situation both before
and after the changes designed to improve performance, it is impossible
to determine whether quality is indeed improving or by how much.
Return to Contents
Strategy Development Questions
As you consider using quality measurement for quality improvement, some
questions to assess include:
- Have you and your stakeholders prioritized child health quality
improvement issues? Do you want to improve preventive care? Target
a particular health condition such as pediatric asthma?
-
Have you reviewed data sources that you already have on hand? Have
you looked at systematic surveys or prior reports for indications
of quality problems? Have you reviewed consumer or advocate complaints
and records of grievances or appeals to identify potential problems?
Have you reviewed recent reports from professional associations?
From other States?
-
Do you serve identifiable subgroups with likely child health quality
issues? Children in immigrant families? Children with special health
care needs? Identifiable racial or ethnic groups with potential health
care disparities?
- Does your State have particular environmental or other circumstances
that are likely to be connected with health problems that could
be addressed by health care providers? High lead paint or other likely
lead exposures? Hazardous waste sites?
Return to Contents
Quality Improvement Tools
Online Resource: AHRQ's Web site, QualityTools, provides links
to strategies and tools for quality improvement strategies.
Go to: http://www.qualitytools.ahrq.gov.
Return to Contents
Reporting Results
Quality measurement information is frequently reported to stakeholder
groups and other audiences. Such information can be complex. To ensure
that quality measurement reporting projects achieve their intended results,
careful planning is required. Some issues to include in such planning
are:
Return to Contents
Know Your Audience
There are a variety of potential audiences for quality measurement
reports, including:
- Legislators.
- State officials.
- Providers.
- Health plans.
- Advocates.
- Consumers.
Each audience has its own:
- Characteristics.
- Needs.
- Potential uses for the information.
Thorough knowledge of your intended audience is essential as you design
quality measurement reports. Note that designing a single report for
multiple audiences can result in a report that suits no one.
Example: A given report may be too detailed for consumers but overly
simplified for health plan representatives.
Design issues needing careful attention include:
Vocabulary. What vocabulary does the intended audience use on
a regular basis? Presenting your results in words that are familiar to
that audience is essential.
Interests. What does the intended audience want to know? What
questions need to be answered? To get attention, results must respond
to audience interests.
Likely Uses. What types of information does your intended audience
need? What will they do with the information? Do they need it for decisionmaking
or to carry out their responsibilities? Would they use it for individual
decisionmaking?
Format and Graphic Presentation. What will engage your audiences?
What are they used to? Different audiences are accustomed to differing
amounts and kinds of information. Some audiences will easily review dense,
multipage texts. Others get most of their information from electronic
media and will respond more readily to interesting graphics and colorful
presentations.
Online Resource: TalkingQuality is a resource on the latest research findings, real-world examples, and innovative ideas on ways to communicate complex information on health care quality to consumers. Developed by a work group of researchers with expertise in quality reporting, the Web site provides sponsors of quality reporting projects with reference materials and current evidence on what works best with consumers.
Go to: http://www.talkingquality.gov/
Return to Contents
Consider a Pilot Year
Collection and reporting of quality measures take:
- Time.
- Effort.
- Resources.
- Practice.
First-time quality measurement results typically reflect the process
of learning about the unexpected and dealing with the unanticipated.
Many quality measurement practitioners recommend that the first year
be defined as a pilot year, without public release of measurement results.
The advantages of having a pilot year include:
- Second-year results are more likely than first-year results
to reflect actual performance and will better serve the audience receiving
the reports.
- Providers or plans welcome the opportunity for a practice year without
adverse consequences.
Although public information requirements govern many situations, some
users have found that a clear statement of the purpose of a pilot year
has resulted in general willingness to wait for public results until
the second round of data collection is complete.
Return to Contents
Involve Your Stakeholders
At the very beginning of your work, consider creating an advisory group
composed of stakeholders that includes:
- Health plans.
- Providers.
- Consumers.
- Advocates.
Having such an advisory group:
- Allows you to keep them informed.
- Allows you to get their input on a timely basis.
- Allows them to voice their fears and hopes.
- Allows them to develop an interest in the success of the project.
- Provides input that enables timely course corrections and helps avoid
mistakes.
Airing plan/provider and consumer/advocate concerns in the same setting
can create:
- Mutual understandings.
- Realistic expectations.
- A common core of knowledge about performance measurement reporting.
When reports are released, knowledgeable stakeholders can help explain
their significance to the public.
Return to Contents
Consider Focus Groups
A stakeholders' advisory group can function as an informal focus group.
However, use of formal focus groups drawn from your target audience(s)
as you develop your project plan can:
- Provide additional and invaluable information about what group members
will find of interest.
- Indicate the best ways to communicate this information effectively.
If resources allow, using professional assistance in designing and administering
the focus group(s) can yield insights that will significantly increase
the effectiveness of the end report.
Return to Contents
Plan Your Dissemination Strategy
Clichés about excellent reports gathering dust apply to quality
measurement reports as well. A well-planned dissemination strategy needs
to be a part of every performance reporting plan, and it should be developed early
in the process. Answers to these questions should be included:
- For what audience(s) is this report intended? Where do they get their
information?
- What other stakeholders or audiences should also get this information?
How do they get their information?
- Do you want media coverage? If so, how can you get media attention?
If not, what will you say when you receive a media call with questions?
Return to Contents
Want More Information?
Many researchers have written on the uses of quality measurement
in child health programs:
Ferris TG, Dougherty D, Blumenthal D, et al. A report card on quality improvement for children's health care. Pediatrics 2001 Jan;107(1):143-55.
Abstract available on PubMed®:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=PubMed&list_uids=11134448&dopt=Abstract
Harris-Kojetin LD, McCormack LA, Jael EF, et al. Creating more effective
health plan quality reports for consumers: lessons from a synthesis of
qualitative testing. Health Serv Res 2001 Jul;36(3):447-76.
Abstract available on PubMed®:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=9776950
Homer CJ, Kleinman LC, Goldmann DA. Improving the quality of care for children in health systems. Health Serv Res 1998 Oct;33(34):1092-109.
Abstract available on PubMed®:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=9776950.
McManus MA, Graham RR, Fox HB, et al. How far have state Medicaid agencies advanced in performance measurement for children? Arch Pediatr Adolesc Med 2000 July;154:665-71.
Abstract available on PubMed®:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=Retrieve&db=pubmed&dopt=Abstract&list_uids=10891017.
National Committee for Quality Assurance. Ten steps for a successful report card project: producing comparative health plan reports for consumers.
Washington (DC); 1998 Oct.
Download report in PDF format (78.4 KB). Go to:
http://www.ncqa.org/Portals/0/Publications/Resource%20Library/10steps.pdf [PDF Help].
Simpson L. Quality of care: time to make the grade. Pediatrics 2001 Jan;107(1):171.
Abstract available on PubMed®:
http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?cmd=
Retrieve&db=pubmed&dopt=Abstract&list_uids=11134451.
Return to Contents
Internet Citation:
Child Health Care Quality Toolbox: Uses of Quality Measurement. July 2004. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/chtoolbx/uses.htm