Preface

By now it has become a truism to say that you can only improve those things that you can measure. In the world of quality, we have made significant progress in measurement. For many years, researchers, stakeholders, payers, and quality improvement and accrediting organizations have been laboring to get past the methodological, philosophical, and “small-p” political issues hampering common quality metrics. While there is much work to be done, we have made a lot of progress.

But when it comes to measuring efficiency—one of the six domains of quality identified by the Institute of Medicine—we have seen much less light than heat. There is a lot of recent activity, but little agreement about how to measure efficiency, much less how to improve it. We commissioned this report, modeled after AHRQ's Evidence Review series, as a comprehensive and impartial review of the evidence on efficiency measurement. Our goal was to identify, analyze, and classify current definitions, lay out a linguistic roadmap to help illuminate discussions, and identify some next steps. To accomplish this task, we enlisted a multidisciplinary team at RAND, supported by a very diverse and active Technical Advisory Group and countless other reviewers representing all stakeholder groups.

The Executive Summary, report, and appendices which follow lay out the approach, methodology, and findings. In this brief preface, we would like to highlight four of the most significant findings, and in particular to identify the implications for present use and future work: How can the findings from this report help improve our use and communication about current measures? What do they suggest about ways to improve the measurement of efficiency in the future?

Return to Contents

Findings, Lessons, and Implications

The Multiplicity of Perspectives on Efficiency

One major finding is that definitions of efficiency differ greatly depending on perspective, i.e., one's role as a payer, provider, consumer, etc.—proof of the adage that “where you stand depends on where you sit.” In most cases, individuals and firms will define efficiency as a relationship between what it costs them and what service or outcome they receive, rather than as a trait inherent in the provider. This difference in perspectives has important implications for transparency: Users of data on efficiency may not share the same perspective as those who generated the data. To facilitate communication under these circumstances, it would seem best to refer directly to the specific measure—cost per discharge, cost per episode, etc.—rather than using the term “efficiency” at all, and to be clear about whose costs are included in the calculation.

The Gap Between Peer-Reviewed Measures and Those in Use

A second finding is that there is almost no cross-over between the measures and methodologies in the fairly extensive peer-reviewed literature and the measures and methodologies in use. This finding presents a clear challenge to an agency such as AHRQ whose primary focus is facilitating creation and use of evidence-based measures, data, and information to improve care. An important priority for us in the next year will be finding ways to close the gap between research and practice in this particular domain.

The Silence of Quality in the Measures

A third finding of the report is that virtually none of efficiency measures, whether in the peer-reviewed literature, the grey literature, or the vendor products, includes the quality dimension. Quality is “assumed,” or is otherwise absent. This absence of a quality component, in fact, has led some such as the AQA to recommend using the word “cost” rather than efficiency to describe such measures. Regardless of whether one calls these measures cost (per the AQA definition) or efficiency (per the definition in this report), the implications are the same: When using these measures, it would seem most productive to pair each with its parallel quality measure. If there is no quality measure, and there is no quality dimension to the efficiency measure, it would be helpful to be clear and direct about this as well.

The Dearth of Validation for all Measures

A fourth finding is that the measures developed by researchers and those in common use do have one significant feature in common: a lack of validation or evaluation. This finding points to a clear need for more validation and evaluation of measures and their use.

Return to Contents

Next Steps

The widespread availability of credible and clear information on cost and efficiency is a critical component of transparency, and is also essential for improving efficiency within and across health care institutions and providers. A critical first step will be achieving clear and credible metrics. We hope this report helps establish some of the groundwork for this enterprise, and we look forward to working with all stakeholders on next steps. In the meantime, we also welcome your comments, suggestions, and input. They may be sent by mail to the Task Order Officer named below at: Agency for Healthcare Research and Quality, 540 Gaither Road, Rockville, MD 20850, or by e-mail to epc@ahrq.gov.

Carolyn M. Clancy, M.D.
Director, Agency for Healthcare Research and Quality

Irene Fraser, Ph.D.
Director, Center for Delivery, Organization and Markets
Agency for Healthcare Research and Quality

Herbert S. Wong, Ph.D.
Task Order Officer, Center for Delivery, Organization and Markets
Agency for Healthcare Research and Quality

---

Return to Contents
Proceed to Next Section