Chapter 2: Methods (continued)

Economic Efficiency for Society as a Whole

Thus far we have focused on efficiency from the perspective of specific entities within society. Efficiency for society as a whole, or "social efficiency," means that some entity can be better off only if some other entity is made worse off,¹⁰ that is, it concerns the allocation of resources across the entire society.

Efficiency from the perspective of each individual provider and intermediary is necessary for social efficiency, but it is not enough. Consider again the preceding example of the health plan and physicians. MD3 views himself as efficient because his total input cost is lowest among the three doctors. The health plan also views MD3 as efficient, because his charge to the plan is lowest. Nevertheless, the charge exceeds the input cost, perhaps because MD3 has a strong reputation and charges accordingly. This difference between charges and costs is a potential source of inefficiency for society as a whole. The delivery of additional services at a price above input costs and below current charges could be a win-win situation for the plan and doctors if the services are necessary and appropriate. Similarly, the price that employers are charged for health-plan coverage may exceed the plan's cost.

Society also includes those who need to consume health care. Consumers desire good health and hence value high-quality outcomes. Their interests diverge from those of providers and intermediaries in financial matters; consumers prefer to pay less for good health, so as to enjoy more of other goods (e.g., housing). Whether consumers obtain more or less of the value created by health care is not the issue, however.

The test for social efficiency is whether imperfect relations between various entities lead to situations in which the value to be shared among entities is less than was possible. The issue is whether society fails to make the most of win-win opportunities. Some examples are again helpful.

Providers may supply less output than is ideal for society as a whole. Take for example the "scale" of a hospital's operations. Higher volume is associated in some instances with better outcomes.¹¹ If "practice makes perfect," a hospital may nevertheless opt for a scale too small to exploit these benefits because reimbursement is not adequate or access to capital markets is limited. The hospital's perspective would not be aligned with that of society.

On the other hand, a hospital may supply more output than is ideal. Some observers believe that under the old paradigm of cost-based reimbursement, hospitals could make profits by investing in specialty services such as open heart surgery centers (see the literature review in Dranove and Satterthwaite, 2000¹²). The costs to society of redundant facilities were arguably not justified by their benefits. Evaluating the efficient supply of outputs raises an interesting question about perspective; taking the perspective of the nation, we might conclude that supply is excessive. If, however, one looks at a smaller geographic unit (state, county, metropolitan area), one might reach a different conclusion about the relationship between supply and societal need.

There could also be too much output due to consumer behavior. Consider a vision-impaired patient with a modest desire for cataract eye surgery. With generous vision insurance, the patient would opt for the surgery, because the benefit he experiences outweighs the cost he faces. It seems likely, however, that the costs to purchasers and society at large exceed the benefit.¹³ These examples demonstrate that the relationship between output and social efficiency is uncertain in general.

Moving beyond output, the fragmented structure of health financing in the U.S. has raised concerns about the system's administrative burden. This may be an issue of social rather than provider/intermediary efficiency. There is some evidence that billing-and-insurance-related costs are indeed substantial.¹⁴ A health plan probably does not weigh the impact of its decision to participate in a market on providers' administrative costs. There may therefore be more plans than is good for society as a whole. Working in the other direction, competition in the market for health insurance can lower prices, benefiting purchasers and consumers. Moreover, policies that "simplify insurance product design" may significantly restrict consumer choice.¹⁵

As a final example, consider the adoption of health information technology, such as computer physician order entry. This technology is expensive for doctors and its benefits vis-àvis higher quality and reduced cost are often shared with other entities. Thus, doctors will tend to invest less in health-information technology than would be desirable (efficient) for society as a whole. Some have followed this logic in advocating Medicare subsidies for adoption, suggesting that Medicare's perspective is closely aligned with that of society overall.¹⁶ Although any particular doctor plays a limited role in the health system and sometimes even in a patient's overall care, Medicare is involved in its beneficiaries' care across providers and over an extended period.

Taken together, these examples suggest that there are many reasons, unrelated to inefficiency from the perspectives of individual providers and intermediaries, why health care may be socially inefficient. Indeed, it has long been believed that this perspective is relatively problematic.¹⁷

Despite the importance of social efficiency in this context, we were unable to identify existing measures, as the next chapter explains. A potential explanation is that measuring social efficiency is quite challenging. In particular, a measure must account for the benefits and costs of a situation to all entities in society.

To the extent that entities desire to evaluate social efficiency, the development of adequate measures would need to be part of a future research agenda.

Return to Contents

Evidence Sources and Searches

Literature Searches/Search Strategy

The RAND Library staff performed the searches on Medline® and EconLit for articles. Members of the project team worked closely with the TEP and the librarians to refine the search strategy. We searched published articles in the English language, appearing in journals between the years 1990 and 2005, and involving human subjects. We also performed "reference mining" by searching the bibliographies of retrieved articles for additional relevant publications. All of these searches were conducted during December 2005. The search strategies can be found in Appendix B.

Vendors and Stakeholder Interviews

Because we expected some of the most well known efficiency measures might not appear in the published literature, we developed a list of organizations that we knew had developed or were considering developing their own efficiency measures. We used a purposive reputational sampling approach. This identified the eight leading vendors of proprietary efficiency measures and five national or regional leaders in quality and efficiency measurements and improvements. We contacted key people at these organizations in an attempt to collect the information necessary to describe and compare their efficiency measure to those we abstracted from articles.

The vendor organizations selected are major developers of proprietary software used as efficiency measurement tools. The stakeholder organizations selected are either national leaders in quality and efficiency measurement and improvement (e.g., The Leapfrog Group, AQA, and NCQA) or regional coalitions with a history of performance measurement and reporting (e.g., IHA in California and the Employer Health Care Alliance Cooperative, also known as The Alliance, in Wisconsin).

Return to Contents

Technical Expert Panel

This report was guided by a Technical Expert Panel (TEP). We invited a distinguished group of stakeholders and experts to participate in the TEP for this report. The TEP conference call was held in February 2005 and subsequent one-on-one conversations occurred between the project team and individual TEP members throughout the project. The TEP provided valuable feedback on the typology and possible organizations to contact. The TEP reviewed the final draft of this report. A list of the TEP members can be found in Appendix D.

Return to Contents

Title Screening, Article Review, and Selection of Individual Studies

Study Selection

Two researcher reviewers conducted the study selection process and selected studies for further review. Each reviewer independently reviewed the documents or studies and disagreements were resolved by consensus. Dual review was used at all stages of the project. The principal investigators and the experts involved in the project resolved any questions or needs for clarification that arose throughout the literature review. Reviewers screened all titles found through our Medline® and EconLit searches or that were submitted by content experts for pertinence to the key questions and therefore their relevance to this project. We adopted the following exclusion criteria that were applied at both the title/abstract and article screening phases:

• Cost-effectiveness of treatment or product.
• Effect of health on labor productivity.
• Efficiency is not stated as an outcome but implications of findings for efficiency are discussed.

Approved titles moved on to the article screening phase. We ordered all articles that were accepted and sent them out for further review based on topic area.

Data Abstraction

We designed a one-page data collection instrument specifically for this project and pilot-tested it with all reviewers. This screener (go to Appendix C) contained questions about the focus of the document, research topic, proprietary efficiency measures, location, and the type of paper. The article screener phase included the same exclusion criteria as the title review stage. Therefore, we excluded abstracts that clearly dealt with topics other than efficiency of the following entities:

• Clinicians (individual or group).
• Hospitals.
• Nursing homes.
• Long-term care hospital wards.
• Primary health centers.
• Systems (plans, medical groups, hospital chains, VA).
• Countries.
• Other providers.

Articles that focused on background or were reviews/meta-analyses were marked for separate examination, as described below. Project staff entered data from the forms into an electronic database and tracked all studies through the screening process.

Articles accepted at the screening stage were subjected to full abstraction using a standardized abstraction form. Some studies or documents described only measure development whereas others described use in an actual population. Due to the volume of articles accepted at the screening stage, the team only abstracted articles or documents accepted in the first round of screening that focused on efficiency measures in the United States exclusively. We did not include efficiency measures that were used to compare the United States with other countries.

Return to Contents

Peer Review Process

We identified 15 potential stakeholders who would be interested in using efficiency measures, and sent them a draft document for review. In addition, each TEP member was asked to review the draft. The list of reviewers and organizations can be found in Appendix D. A blinded list of all comments received, organized by section of the report, is presented in Appendix E, accompanied by our response to each comment.

Return to Contents
Return to Previous Section
Proceed to Next Section