This Web conference was offered on October 21, 2004. It was designed to help health care purchasers understand the evidence base for quality-based purchasing and public reporting, and to increase their understanding of an important tool set available to assist with evaluations of health care quality.
Penny Daniels: Good afternoon. Welcome to the Paying for Performance Web Conference. This is the first event in a two-part series titled "Buy-Right for Health Care Quality: Evidence and Indicators." This Web session will explore what we know about value-based purchasing as a strategy for improving health care outcomes and quality. The series is sponsored by the Department of Health and Human Services, Agency for Healthcare Research and Quality, often referred to by the acronym, AHRQ or "AHRQ". My name is Penny Daniels, and I will be your moderator for this session. I will begin by introducing today's panelists. They represent a broad range of interests, including government agencies, researchers, health care purchasers, and health care providers.
To set the context for today's discussion we will hear from AHRQ Director Carolyn Clancy. Carolyn will explain why we need to better match the level of payment for health care services with the level of quality we receive. Mark McClellan, administrator of the Centers for Medicare and Medicaid Services, or CMS, will then brief us on how the nation's largest health care purchaser, the Medicare program, is approaching this challenge.
And then joining us in our studio are three researchers who have been looking at this subject. They will talk about their latest work looking at the research findings on pay-for-performance, as well as the latest generation of programs. Our researchers are R. Adams Dudley from the University of California at San Francisco; Meredith Rosenthal from Harvard University; and Gary Young from Boston University. And then for reaction to these findings we are very pleased to be joined by Bob Galvin who is director of Global Health Care for General Electric Corporation based in Fairfield, Connecticut; and Stephanie Alexander, senior vice president for Health Informatics at Premier Inc. based in Charlotte, North Carolina. Welcome to all of you today.
Before we begin our discussion, I'd like to tell the audience a little bit about the format of this audio conference. Now while neither Carolyn Clancy nor Mark McClellan could actually join us in the studio today, we are very happy they were able to pre-record their comments. So we'll open with Dr. Clancy's audiotape and slides, followed by the short video tape provided by Dr. McClellan. We will then hear live from our three researchers; and finally Bob Galvin and Stephanie Alexander will give their perspective; he based on his experience representing a large employer, and she representing an alliance of non-profit hospitals and health systems.
And then after these brief presentations and comments, we will invite you, our listeners, to join our discussion and we'll explain in a little while how you can send us your questions and/or comments by phone or by E-mail.
In the meantime, a little bit of business here. If you have any Web-related technical difficulties during this event, please click the Help function in your window to troubleshoot your Web connection. If it seems that your slides are not advancing you may need to restart your browser, and log on again. If you're on the phone, dial * 0 to be connected to Technical Assistance. Also, if you have trouble with the audio stream or if you experience an uncomfortable lag time between the streamed audio and the slides, we encourage you to access our audio by phone. The number which is also on your screen is 1-877-407-8289, and this is the same number you'll call to ask questions when we get to the Q&A part of our program. So we now begin. To set the context for today's conference let's listen to AHRQ Director Carolyn Clancy's pre-recorded talk.
Carolyn Clancy: "Good afternoon. I'm Carolyn Clancy, the director of the Agency for Healthcare Research and Quality. Our mission is to improve the quality, safety, efficiency, and effectiveness of health care for all Americans. Recently there's been a great deal of excitement about the concept of "paying for performance," that is to say, rewarding high-quality providers appropriately in the marketplace as a strategy to improve quality of care for all who receive care.
And so today I'm very excited to let you know that we have some leaders in the field who have been studying and examining this issue, and I'm really excited that Mark McClellan, the new administrator of the Centers for Medicare and Medicaid Services, will also be making some introductory comments. I don't want to take any of the thunder away from our really terrific speakers today, but just wanted to offer some initial observations. The first is from the Institute of Medicine, and far too many reports including our National Health Care Quality Report, an annual report that was first published in 2003, the 2004 report will be coming out in the next few weeks--there are some very consistent messages. The first is that there are serious problems in quality of care. The Institute of Medicine said: "Between the health care quality we have, and the care we could have lies not just a gap, but a chasm."
I should also note as someone who has had some recent undercover experience as a hospital patient having a first-hand look at quality issues in the hospital that this is really about all of us. The Institute of Medicine has further noted: "that the problems don't come from bad people or health care professionals who aren't knowledgeable, but they really come from the fact that we have poor systems."
And moreover, that the chasm really derives from two overarching problems. The first is how we pay for care; the second is how we organize and structure care. In addition the report "Crossing the Quality Chasm," emphasized the need for improvements in Information Technology, an issue on which Secretary Thompson has led.
Now AHRQ's role in this work is very much focused on the measurement inside these initiatives. So for example, starting in 2003 we produce annual reports on the quality of health care in America, and disparities in health care associated with individual race, ethnicity, income, education, geography, and other factors. In addition to that we have supported the development of tools to assess local experience of care; for example, the Consumer Assessment of Health Plan Surveys, which now has modules for Health Plans. There's one under development for Hospital Care as well as for Nursing Home Care and other settings. We've also developed tools, including our inpatient indicators and patient safety indicators, to help hospitals improve quality of care. And in addition to that, we have some tools for measuring potentially avoidable admissions. So all initiatives to assess and improve quality really rely on measurement, and that's where AHRQ is a key science partner in your efforts."
Recently in a Health Affairs article in 2004, Meredith Rosenthal and colleagues identified 37 separate incentive plans representing 31 different payers. These are in-patient purchasers who can't wait for, sort of gradual programs to improve quality in a fairly incremental fashion. They're trying to jump-start their way to success, and Meredith Rosenthal will be presenting that work later in this Webcast. But I think what you'll hear is that the science has to catch up to the initiative of purchasers in the marketplace who want very much to improve quality of care right now."
I also want to make a couple of other comments about financial incentives, because I think we all want the highest quality of care possible, but sometimes when the term "financial incentives" is brought up, people express some reservations. The first comment I'd like to make is that all payment systems contain financial incentives. So it's not really a question of incentives versus no incentives; it's how those incentives are aligned with the outcomes and goals that we want from health care."
The second comment is that unintended incentives can be as strong as intended ones. Or to say it a different way, some leaders in health care quality have actually said that our current payment systems are toxic to producing high-quality safe health care."
And the final comment is that financial incentives are just one force shaping quality. That is to say, pay-for-performance is a very exciting topic, and I know many of you listening to this Webcast today are going to want to learn as much as possible about current initiatives. It is not the only solution or only strategy through which to improve quality of care. So with that, again I want to express my welcome and enthusiasm for the program today, and hope that you will continue to follow our progress as we move forward in improving quality of care for everyone."
Penny Daniels: Now we have a videotape from Mark McClellan. He will describe what the Centers for Medicare and Medicaid Services are doing related to quality-based purchasing; Mark McClellan, of course, is the CMS administrator. I want to point out that for those of you listening on the phone, you may or may not be able to hear this audio because it is on video. If that happens please bear with us through the video. Your audio will resume. Jim Lee, our technical director, now please roll the video.
[Video inaudible]
Our thanks to Mark McClellan for his insight into this very important topic. We'll now return to our panel of researchers to learn what they have been learning about quality-based purchasing.
First up, Dr. R. Adams Dudley is an associate professor at the Institute for Health Policy Studies at the University of California at San Francisco. He and his colleagues have just completed a major analysis of the research on pay-for-performance programs. Adams, how deep is the literature on the subject?
Dr. R. Adams Dudley: Well, Penny, unfortunately it's not very deep. Before we get into the research, I think we ought to talk a little bit about the various factors of incentive programs that could potentially influence the outcome, and you'll see that we'll come up with a lot of such factors. We'll describe a fair number of such factors, and then relative to the number of factors, there isn't that much literature.
Penny Daniels: What factors are they? What aspects do you think need to be studied?
Dr. R. Adams Dudley: Well, the characteristics of the incentive are clearly important. You can imagine how the magnitude on an incentive might influence response if it's a financial incentive, and also how it could change the response rates if you decide to release the information to the public or to other providers. That could create a reputational effect that might influence the response.
In addition, the cost of complying with whatever the quality guideline is could be important. So for example, if the quality goal is to give people an aspirin, that's not a very expensive thing to do. Aspirin is inexpensive, and giving the patient a pill one or a few times is not very difficult. Whereas if you're trying to get someone to quit smoking, that could involve a lot more resource commitment. You'd have potentially support groups or mediations or patches, and return visits to the providers. And so you can imagine that for a magnitude of a given incentive, the cost of complying could influence whether or not someone responds.
In addition, there are factors that are external to the incentive itself that are potentially important. There's the overall business environment: Is the environment mainly fee-for-service or capitation? And how well does the incentive fit into that environment? Are there other incentives around that could influence whether or not someone is able to pay attention to this one? It's conceivable there could be incentive cacophony where there's too many different programs asking you to do too many different things.
In addition, characteristics of the provider could be important. So if you're trying to give incentives to increase the use of preventive care, it's conceivable that providers who have been through medical school more recently have been exposed more to the importance of, and the approaches to, providing preventive care. Whereas some of us older folks, there was probably a little less of that when we were in medical school, and us dinosaurs might be a little slower to respond to incentives, just because we're maybe not quite as sure how to do that.
Similarly, you can imagine that organizational characteristics could be important. A provider could have every intention to want to continue on the preventive care idea, to want to make sure that everyone gets a mammogram, and if they're having to sift through a paper chart, and figure out whether or not their mammogram has been done, when it was done, what the results were, etcetera, that could be a lot more difficult than if the organization has a great computer system in place, and one can just click a box that says "Preventive Care", and see when the last mammogram was.
Penny Daniels: Going back to the importance of systems.
Dr. R. Adams Dudley: Right, and then in addition, patient characteristics matter. I'm a physician myself, and you'll ask any provider and they'll tell you there's some patients that want to go along with what you recommend. There are some patients who've read on the Internet why that's the wrong thing to do, and there are some patients who want to go along, but they can't get the ride to the clinic, or they can't get the ride to the mammogram appointment or something like that. So many patient factors that are external to the program could influence whether or not someone's able to achieve the quality goals that someone is targeting.
Penny Daniels: Okay, so Adams, you've been talking about what to think about when we're looking at the literature. Now when you actually look at it, what does it tell us?
Dr. R. Adams Dudley: There are only nine randomized trials of incentives available in the literature, and I can give you two general findings. Those are that, in the right setting, providers will respond to financial incentives; and also that in the right setting, providers will respond to reputational incentives.
To look at the first of these, the Vanderbilt University did a study in the mid-1980s that was very simple in its design, and represented a very small incentive. They offered residents in their pediatrics practice $2 per visit for providing well-child care versus a group that was randomized to $20 a month. And the folks who got an extra $2 every time they had a well-child care visit did more well-child care. So that's a very small incentive, and when an incentive that small can work in the right setting, you know that's it possible for financial incentives to work.
In addition, there have been studies that show that reputational incentives, releasing performance information to the public or to other providers, can also influence provider behavior. Here's an example: Hospitals in Wisconsin were randomized to either receiving a report about their obstetrical care, and there were also some other areas, but we'll focus on obstetrical care that was-they received a copy, and it was published in their local newspaper. Or they received a copy and it was confidential, and there was no release of the information to the public or they received no report at all. And the slide that you have in front of you shows the response, the difference in response rate to hospitals where they had within their obstetrical care they had a problem. They had low performance on hemorrhage after delivery of a new baby. And among those hospitals where that information was released to the public, 88 percent would then go onto Institute of Quality Improvement program in that specific area.
Penny Daniels: Wow, what a difference.
Dr. R. Adams Dudley: Right. Big difference, whereas those who received this report confidentially-still just as important a problem clinically-only 27 percent of them had a Quality Improvement Program in place when there was follow-up to check what their response was. And in those who had not received any reports, so they didn't have the information that this was a problem, they had even less activity going on, down at 9 percent.
Penny Daniels: Clear trend.
Dr. R. Adams Dudley: Right, so the main difference here in terms of incentive is, do I have a public report; that's a reputational incentive. And you see between the first part and the second part that it appears to have big effect in the right setting. So I think we can also focus on one other issue that we might be able to reach some conclusions about given even this relatively sparse literature; and that is, that uncertainty about whether or not I can actually achieve an incentive or have success trying to capture the incentive may also be an important aspect of these programs.
And I say that based on the following: When the incentive was structured as a fee-for-service (that is, you were told explicitly what service you were going to do, and how much you were going to get to it), it was almost always a positive study. And the one negative study was for getting people to quit smoking, which actually I'm a physician, I'm a pulmonologist, and that's a very hard thing to do. So I wasn't surprised that that study ended up being negative. I'm not sure how big the incentive would have to be to win on that one.
When the incentive was structured as a bonus for hitting a compliance target, by that I mean you get to, say, 80 percent, or you get to be among the top 10 percent of providers in the area, there was a little bit less response. There were more negative studies. And not only that, but among those two types it was always negative when the question was, "Can I be among the top 10 percent of the providers?" I think that might in part be because at least the way these were set up when pay-for-performance was a foreign thing, and people didn't know how their other competitors were doing and so forth, it would be very difficult in that setting to know, "What do I have to do to actually be able to achieve that compliance goal or that quality goal?" And therefore as an organization, it's hard to say, "Okay well let's make sure that we have exactly this plan, and make sure that we get to the goal," and so the result was negative study.
Penny Daniels: So it would be kind of like operating in the dark, that is.
Dr. R. Adams Dudley: I think the simple lesson is, you shouldn't ask people to operate in the dark. They're going to need a lot of information to decide whether or not it's worth them making the investments to achieve the quality goals you're trying to get them to.
Penny Daniels: Adams, given that so few studies have been done on this topic, what do you think are the big questions that still need to be addressed?
Dr. R. Adams Dudley: I think we don't know how big incentives need to be. It's one thing to pay residents $2-what is that? It's like the pizza. It's one thing to offer residents a slice of pizza for a visit, and we probably could have offered them a peanut butter cup, and it would have done. In the real world where people have a lot of other things that they need to work on-not that residents-I'm going to get a lot of calls from residents saying how hard it is. But with physicians who are further along, and practices that are sort of real-world practices, I think it remains an open question: How much of an incentive do we need to offer? Does it matter if the incentive that you're offering comes in a fee-for-service context, a capitative context, or some other general payment strategy? And should these incentives, all the incentives that were studied, for example, focus on individual providers? What's the performance of Dr. X? But if you think about some of the enabling factors that I mentioned like putting in a computer system, Dr. X doesn't usually do that alone. Usually that's a group or a hospital or a large organization deciding that it's worth making that investment. And so maybe incentives should be focused at the organizational level as well as the individual, or maybe some mix is needed. And we just don't know the answer to that.
In addition, almost all the incentives that are out there are for achieving a quality goal, and sometimes there may be enabling factors that we just need to get over the hump. Information Technology is one of them, and maybe there should be incentives for get-over-the-hump programs. And that's another thing that we don't know. And finally, and this is obviously very important to purchasers, "if I introduce incentives to improve quality care will that cost me money or save me money?" I wish I had the answer to that.
Penny Daniels: Very big question. Okay, well you've given us a lot to think about. Thank you, Adams. Our next speaker is Dr. Meredith Rosenthal, assistant professor in the Department of Health, Policy and Management at the Harvard University School of Public Health in Boston. Meredith, you have had the opportunity to look across a wide range of different pay-for-performance programs. Tell us-how widespread is this movement?
Dr. Meredith Rosenthal: Well, Penny, it's quite widespread, and as Dr. Clancy mentioned, we found about 37 programs when we were looking in 2003. The number is probably two or three times that now. And again, the issue is that after more than a decade of measuring quality, benchmarking, and public reporting, payers, health plans, and purchasers are getting frustrated with the slow pace of improvements. And so these programs are sort of the next effort among a wide variety of quality improvement efforts to try to get providers to meet these quality targets.
Penny Daniels: What are some of the typical components of pay-for-performance program, both for physicians and for hospitals?
Dr. Meredith Rosenthal: In the programs that we looked at, there were typically for physicians about five to ten HEDIS measures. (Health Plan Employer Data and Information Set measures.) These are sort of ambulatory care measures that most of us in research and working in health benefits are very familiar with now. These include things like cancer screening, the rate of cancer screening, the rate of appropriate treatment of people with diabetes and other chronic conditions. For physician's payments, these tend to focus around a set of these measures as well as some patient experience measures-overall patient satisfaction measures typically.
On the hospital side, we found that the programs were rewarding a much broader set of measures including both things, what we call process measures; that is, are you doing the right thing at the right time for a certain group of people, as well as outcome measures. Things like complication rates after surgery. In addition, they are looking at Information Technology explicitly in-hospital, looking at Leapfrog-type measures such as the use of computerized physician order entry systems.
Penny Daniels: You did a critical analysis of the first-generation of pay-for-performance programs; it was actually published in Health Affairs last spring. Given this analysis, what would you like to see in the next-generation of programs?
Dr. Meredith Rosenthal: The first thing that really struck us is that none of the programs were explicitly trying to reward quality improvement. So, as Adams was talking about before, the targets are set typically in terms of trying to read, 90 percent mammography rate for example, or trying to achieve status in the top quartile of the network. And so these targets are really set to reward providers that are doing well, now. So for those providers it's much easier for them to make the top; that is, the decision to do whatever that quality improvement is necessary. There's a much shorter way to go essentially to hit the target.
One big concern we have is that, that's going to stimulate quality improvement mostly in the high end of the market, and we want to see quality improvement among those physicians in hospitals that aren't doing well. Because of course, ultimately we'd like to improve the population health, and not just the health of folks who are lucky enough to go to the hospitals and physicians that really get it when it comes to some of these improvements.
The thing that we looked at also was the strength of those incentives. Again, going back to Adams' review, most of the payment incentives are on the order of about 5 percent on the physician's side of it. So about 5 percent of total fees or capitation payment was set aside for performance on quality measures. And we don't have good data on what the right number is. It seems a little small to really try to create a revolution in terms of new systems, new approaches to managing health care.
On the hospital side, the rewards that have been put forth so far are even smaller. Obviously the dollars are much bigger on the in-patient side, but we're still talking about 1 percent or 2 percent, and we question whether that's enough to really stimulate quality improvement of the magnitude that we're looking for here.
Penny Daniels: It sounds like we're already starting to see some trends. Do you think pay-for-performance is here to stay, Meredith?
Dr. Meredith Rosenthal: I think so, yes. I think again, we're all very conscious of this quality chasm, and the need to really get people moving, and to reward physicians and hospitals who have figured out ways to improve care. And so I think that these programs are going to continue to evolve. More payers are going to adopt them. And again, we'd like to see them focusing on quality improvement, to try to get improvement across the whole marketplace, and not just in one segment. As well as looking for more evaluation data back that will tell us how big an incentive needs to be in a particular setting, more consideration of larger incentives. But I do think these are here to stay, and I think it will be very exciting to see as the evaluation produces results, we can see change in the market, and improvement in the program.
Penny Daniels: Okay, Meredith, thank you so much. The third member of our panel is Dr. Gary Young, professor at Boston University School of Public Health. Gary is a member of the university Center for Organization, Leadership, and Management Research, which is funded by the Department of Veterans Affairs. Gary has talked about working on a very interesting study that is being supported by AHRQ and The Robert Wood Johnson Foundation. They're evaluating a set of pay-for-performance demonstration projects under the Rewarding Results program, which is a national initiative co-funded by RWJF, The Commonwealth Fund, and the California Health Care Foundation. Gary, what is this program? What is "Rewarding Results"?
Dr. Gary Young: "Rewarding Results" are for seven pay-for-performance programs that collectively cover several geographic regions of the U.S. The programs or demonstrations are testing innovative approaches to pay-for-performance, and will provide important lessons for health policymakers and managers. In six of the seven programs, the program sponsors are either a single health plan or a consortium of health plans. In the other program, "Bridges to Excellence," the sponsors are a consortium of employers including General Electric and Verizon.
Penny Daniels: How else are these seven programs different from each other?
Dr. Gary Young: The programs actually differ on a number of dimensions, for instance, the intended recipients of the incentive money, individual physicians in a couple of the programs, physician group practices and others, and hospitals in the case of the Michigan-based program. In those programs where the intended recipient is not an individual physician, we will closely be studying whether and how physician group practices and hospitals distribute incentive money to individual providers whose efforts are needed to achieve the quality target.
The programs also differ regarding the quality targets. All of the programs generally have targets focusing on preventive or primary care-oriented services. But across the programs, the specific targets for example, mammography, screening or diabetic eye exams, and the associated scoring systems differ-absolute threshold in some programs, percentage improvement in others.
Another difference among the programs is the structure and magnitude of the financial incentives. The majority of the programs offer bonuses. So providers have something to gain, but not lose regarding their quality performance. Across the programs, bonuses take the form of lump sum payments; in one program for example, a maximum payment of up to $5,000. But in others, also there are adjustments to fee schedules or bumps in capitated payments.
However, I'd like to also mention that in the Rochester-based program, the incentive is structured to expose providers to a downside risk, as well as an upside gain. Specifically, providers are subject to receiving between 50 and 150 percent of a withhold amount, which is about 20 percent of their reimbursements.
Penny Daniels: I understand that you did a theoretical model of what might influence how a physician will react to these incentives-in other words, whether they would pursue them. Tell us about that.
Dr. Gary Young: We are drawing from previous research. We've identified several factors that theoretically underlie providers' motivation and capability to pursue quality-related incentives successfully, and I'll briefly review these: awareness-providers must be aware they are eligible for financial incentives, and they must have a meaningful understanding of what they need to do to receive incentive money. Salience, financial salience-providers must perceive the financial incentive as being sufficiently large to offset the costs both monetary and psychological of pursuing the associated quality targets. There's also clinical credibility. Providers must believe that achieving the quality targets will improve the health status of their patients.
And then we have scope of control. Providers must have sufficient control over the clinical activities or actions necessary to achieve the quality targets. Then fairness. Providers must believe the incentive programs are being administered fairly among participants. Finally, we have unintended consequences. Providers must believe that pursuing quality-related incentives will not distract them from performing other important clinical activities for their patients.