[This Transcript is Unedited]
Hotel Monaco
225 North Wabash
Chicago, Illinois
DR. COLTIN: I'd like to welcome everyone to the meeting of the Quality Workgroup of the National Committee on Vital and Health Statistics. We will be starting with introductions. So I'd like to start here, and we'll go around to my right and cover all of the people at the table and then we'll go back to the people in the audience.
My name is Kathryn Coltin. I'm with Harvard Pilgrim Healthcare in Wellesley, Massachusetts and I am the Chair of the Quality Workgroup and a member of the full committee.
DR. EDINGER: I'm Stan Edinger. I'm with AHRQ. I'm also lead staff to the Quality Workgroup.
DR. COHN: I'm Dr. Simon Cohn. I'm a practicing physician and the national director for Health Information Policy for Kaiser-Permanente and a member of the committee.
MR. BLAIR: I'm Jeff Blair. I'm with the Medical Records Institute and I'm a member of the Committee.
MS. WILLIAMSON: Michelle Williamson, National Center for Health Statistics, CDC.
DR. KMETIK: I'm Karen Kmetik. I'm program director for Clinical Performance Evaluation and Improvement at the American Medical Association and a guest today. Thank you. Our offices are two blocks from here. Welcome to Chicago.
DR. FRIEDMAN: I'm Dan Friedman with the Massachusetts Department of Public Health and a member of the committee.
MS. KANAAN: I'm Susan Kanaan, writer for the committee.
MR. REILLY: I'm Tom Reilly from the Agency for Healthcare Research and Quality. I'm the director of the National Healthcare Quality Report.
DR. FITZMAURICE: Michael Fitzmaurice, Agency for Healthcare Research and Quality. I'm liaison to the National Committee, lead staff for the Secretary's Council on Private Sector Initiatives to Improve Security, Safety, and the Quality of Healthcare, and staff also to the Subcommittee on Standards and Security.
DR. COLTIN: We're going to go up to the audience.
MR. VANGEIST: Jonathan Van Geist, senior scientist in Medicine and Public Health at the AMA. I'm also involved in efforts to improve vital statistics.
DR. LOEB: I'm Jerod Loeb. I'm vice president for Research and Performance Measurement at the Joint Commission.
DR. CONLON: I'm Paul Conlon, vice president for Clinical Quality for Trinity Health. I'm here representing the AHA.
MS. ARISTA: My name is Irma Arista with the National Center for Health Statistics.
MR. KELLY: Ed Kelly, AHRQ.
MS. HOLMES: Julia Holmes from the National Center for Health Statistics.
MS. JACKSON: Debbie Jackson, National Center for Health Statistics, committee staff.
MS. JONES: Katherine Jones, National Center for Health Statistics and staff to executive committee.
DR. COLTIN: The first thing I want to do is cover some changes to the agenda and we were originally scheduled to take a break at 10:30 and to begin taking testimony on the National Healthcare Quality Report at 11:00. We're going to move that up a half hour. Our first panel is here and we will take a break at 10:00 and begin taking testimony at 10:30.
That will enable us, hopefully, to break for lunch a little earlier as well. The reason for that is that the panel that we had originally hoped to have this morning will not be held today. We had hoped to have representatives of several of the professional societies and other provider organizations, who are involved with developing and testing measures of quality at the individual provider level, come and talk with us about some of the data issues involved in trying to create those measures from existing data sources.
The consensus, as we went through the recruitment process, I think, was that this panel is a bit premature because many of the organizations are just beginning on some of those efforts, and given the sensitivity of the area, it was a little bit early to begin delving into lessons learned in that process as well.
Instead, we will be using the time to talk about next steps on the report that this committee will be preparing to summarize testimony that we've taken over the past 2-2.5 years on data issues and measuring quality in a number of sectors not simply at the individual provider level.
That was just one area. We have taken testimony around measures of health plans, measures of hospitals, measures of different types of services, mental health and substance abuse services, nursing home, home health and so forth. And our task right now is to synthesize the testimony that we've taken and try to make some recommendations about how to strengthen the data infrastructure needed to support measurement of quality across all of these different types of providers and settings and different types of care.
I guess one of the first things I'd like to do is review our current time line for our report and our writer, Susan Kanaan, has been working with us to pull information together. Susan, if it's okay, I'd like to turn to you to tell us what you think is reasonable at this point in time as a time line.
In deferring the individual provider level panel, I think we'll, in essence, eliminate that from our report because we're on a time schedule where it doesn't appear that they will be at a point to come in a speak to these issues before we're well along our time line of creating our report. That does create a bit of a gap, but we will acknowledge that in the report as such.
We can go on and take that testimony separately as another activity perhaps next year. The other thing we will be talking about beyond the time line will be assignments as to who will be taking the lead on writing various sections or gathering information for various sections of the report just to be clear on that.
And then, third, our process for developing and getting comment from the other members of the committee around the recommendations that we will be making based on the findings that are pulled together in this report. Susan, if you could just briefly remind us as to what the time line is that we're on at this point.
MS. KANAAN: Here's what I have drafted or kind of blocked out and this will be chance to kind of confirm it or change it. Kathy has already written an excellent and detailed outline. So that gets us off to a really good start and it's using that outlink one that we're figuring out who's going to do what.
I'm going to produce a kind of summary of the presentations and the themes that emerged from them going back to 1996 and have that ready for the committee to look at or the work group to look at the September meeting and hopefully, we'll be able to send that out to everyone in advance so that you'll have it with you.
And then, you're also going to be discussing your recommendations to some degree at your September meeting, and then I'm going to be working on sections 1 and 3 of the outline which doesn't mean anything to the rest of you, but the introduction and the actual discussion of the themes and, Kathy, I don't know that you're planning to do your work on your sections.
But, in any event, I'll have that ready in October and Kathy and I will go back and forth with that and it there will be a rough draft ready for the full committee to take a look at in November. I think it was still a little bit of an open question about whether you were going to try to get the entire report done by the end of this calendar year or we're going to come back to the committee in February. Our tentative plan was that the approval of the final report would be at the first meeting in 2003. I don't know if that's an acceptable time table to you, Kathy.
DR. COLTIN: I think that's probably where we're going to have to be. What I would like to do is to bring our draft recommendations, if possible, to the committee at the November meeting and begin that process and see whether we can reach consensus on the recommendations so that we can finalize the report for February. Comments? DR. COHN: Perhaps a naive question and I think you all know that I'm really not part of the work group, but sitting in today. I'm just curious as to what generally is the story line that you've developed for this report? Not necessarily recommendations, but what have you seen at the high level that needs to be communicated?
DR. COLTIN: I think that we've heard a lot from different sectors about some of the limitations of administrative data sets and the heavy reliance on administrative data sets as we drill down in trying to look at quality below the level of health plans.
Especially is you start thinking about measurement at the individual provider level, the sample sizes really preclude chart review to any great extent. It's not that you can't do it, but the cost implications are enormous. To the extent that you can minimize this by being able to get as much information as possible from administrative data sets and then only have to supplement that information with chart review, that becomes a preferable way to go.
In thinking about that and where are the limitations in the administrative data sets, there are really only 2-3 key data elements that would be useful and really important in trying to move measurement forward and we'll have some recommendations in some of those areas.
Some of them actually, there are some place holders in the transactions. They're just not required fields right now. I think that's one area where we heard a lot about the need to be able to do more with administrative data.
MR. BLAIR: Kathy, could you just clarify what you're thinking of when you say administrative data sets?
DR. COLTIN: We're thinking about the standard claims and encounter transactions primarily, the 837 and possibly also the 834. I think those are the two where there were some opportunities. Now, the alternative is think about the claims attachment transaction as another vehicle for potentially carrying additional information that might only be requested under certain circumstances where there might be an attachment to respond to those circumstances when they exist.
Those are some of the things we'll want to talk about and when we come up with some preliminary recommendations, the thought we're envisioning is between September and November, we would like to tie into any other meetings of the other subcommittees that may be going on so we can begin to take these recommendations and discuss them in some detail with the relevant subcommittees.
Certainly with the subcommittee on data standards and security, there are also recommendations that will raise privacy issues and we will want to take those to the subcommittee on privacy and confidentiality and talk with them about those recommendations about what's reasonable and feasible.
That's the process that we're going to try to go through if possible between the September and November meetings and bring a preliminary draft of recommendations to the full committee on the final day of the November meeting. Further follow up?
DR. COHN: I was sort of curious as I'm listening to this and I will apologize, either I've had too much caffeine or not enough, plus the couple hours time zone change from California. As I'm listening to this, I'm thinking in trying to put together ZIPPA and the MRI recommendations, the data recommendation in NHII.
I think there's a story here about fragmentation in healthcare, the fact that our data capabilities are pretty well broken in healthcare. Investigations that you've done probably near-term suggestions to move us into a greater understanding of what's happening in the healthcare system and maybe longer term recommendations that may be aligned more with what the NHII is thinking in terms of trying to think what the story line is here, but to tell people we need to rely more heavily on claims system data if we're, on the other side of our mouths talking about ---
DR. COLTIN: That was merely one recommendation.
DR. COHN: Sure.
DR. COLTIN: Clearly, there were a lot of comments that we heard about the hopes that people have for electronic medical records and what my be possible there and in particularly, the fact that there was a lot of heterogeneity around electronic medical systems that are in place.
What functionality should those provide in order support quality measurement? Can you do a population based query on an electronic medial system to identify all of the patients who have a certain condition or meet certain criteria to then begin to look at the elements of the care that that population is receiving. That's another area.
DR. COHN: I didn't mean to, in any way, I'm just trying to think of overall context of how the story gets told. Maybe it's not just short term. It's short and mid-term, recognizing some of the barriers you're describing.
DR. COLTIN: I think we will be talking about recommendations that have to do with administrative data, electronic medical record data, that have to do with survey data and that have to do with coding as well as those that have to do with developing capabilities that may not currently even be on the horizon.
MR. BLAIR: One suggestion, it might help to bring this report and our recommendations within this report into context with the NHII activities and the PMRI activities if choose to adopt similar terminology. For example, if we wind up referring to it as patient medical record information instead of electronic medical record, then we can begin to draw upon the recommendations that we've made there as well and the findings and problems we found there as well. It could fit together into a more coherent NCVHS.
DR. COLTIN: That makes sense. I appreciate that recommendation. That's where we're headed. We haven't yet even pulled together all of themes. That's the work Susan is doing between now and September. It's actually even a little premature for me to start saying this is what we're hearing, this is my recollection of some of the things that resonated with me across the various panels.
I think it will be very important for us to actually go through the formal step of pulling out the key pieces from the testimony and going over them in September and it may actually highlight some other themes that I've neglected to mention this morning. That's really where we're at with this at this point in time.
That sounds fine, Susan, in terms of the time line of being able to discuss those themes, being able to then say in which of those areas do we want to develop recommendations and then move forward with developing some preliminary recommendations that we can then test with the other subcommittees and discuss and hopefully bring a preliminary set to the full committee in November as well as having the work group being able to look at a draft of the report.
MS. GREENBERG: Good morning and I apologize for my late arrival. I just wanted to say it's really important to be bringing all this information together because we've had some really good sessions over the years. Also, I see some linkages with the functional status report that Susan also worked on for the committee. Much of what we heard about functional status was in the context of outcomes and quality and it focused more on administrative data, but also made recommendations related to PMRI.
DR. COLTIN: That's right and also the Medicaid Managed Care report there with some key recommendations in that report. At the time, we had tried to combine questions that we had about data issues related to quality with data issues more broadly in looking at Medicaid Managed Care and we did actually hear quite a bit about some of the issues in measuring quality for the Medicaid population and so we will draw on some of those previous recommendations as well and review them in that context.
What we were planning, as Susan mentioned, Susan is going to take the lead on putting together the introduction which really includes both a brief discussion of the national committee and the work group and its charge and what our goals are for the report. The next section is a background section that really tries to lay out the context.
That is where we will try tie into the NHII and other activities that the committee has had ongoing as well as what's going on out there in the marketplace, the trends around supporting consumer decision making and prudent purchasing and so forth.
That section is a section that I'm going to be taking the lead on, but hopefully with strong support from staff. That's where I will rely on Stan and others who are a support for others to help me pull together information to include in the background section.
DR. EDINGER: As a point on that and since Marjorie is here too, on the information on other parts of HHS as well as DOD and VA, Mike and I have worked on some of these issues too for data standards. One of the problems you run into is depending on how you do that, if you ask the agencies what are you doing and how are you doing it and getting it cleared sometimes, it probably will be a lot more than November before we ever get anything done.
I guess there are two approaches. One, we can write the NCVHS committee its interpretation of what it feels the agencies are performing and doing, in which case you may subvert the process of them actually buying into that you're saying about them or you can go through the process of having them all bless it in which case, it can take forever.
DR. COLTIN: I think that's something we should talk about at our September meeting when we have a preliminary framework for what we think the content will be in those areas.
DR. FITZMAURICE: Could I suggest that whatever we do probably we should send around to the agencies to say here's our timetable, if you can get us comments back by a certain date, we would appreciate it and continue our train down the track, but also give them a chance to hop on board.
DR. COLTIN: That sounds very reasonable.
MS. GREENBERG: What I was going to suggest is complementary to that and that is that possibly at the October Data Council meeting if John or someone is there representing the committee could give them a heads up that this report is being worked on and is coming out of the national committee and if you have a request from them consistent with what Mike said.
Ultimately, when the report is approved by the full committee with its recommendations, it will be presented to the Data Council. It's good to give them a heads up, particularly if you have a need from that and I'm sure Jim will be happy to accommodate that. If someone from the committee can't be there, one of the staff could do it, but hopefully someone from the committee can be there as well.
DR. COLTIN: That sounds good. As Susan mentioned, the third section of the report after the background is really this synthesis of the common thread of what we heard in the testimony and trying to identify obstacles and opportunities. That's the theme that we're looking at here and those will form the basis for the section on recommendations that we'll be talking about.
That's the area where we will hopefully spend the majority of our time at the September meeting is really thinking about these common threads and for which of them we want to develop recommendations. The final section is the recommendations. We talked a little bit about that process.
Unless anybody has any other questions or concerns or suggestions or advice about the process as we've laid it out, I think it is ambitious and it's likely that we end of going somewhat beyond November and will not be able to bring a final report to eh committee until its meeting in February, but I think it's not unreasonable.
DR. FRIEDMAN: Although we'd like to make Kathy a member for life, I don't believe the national committee has that option, but we do have precedent even if the Chair has rotated off the committee, bringing the Chair back when the report is presented. I think that we can deal with that even if it goes beyond November.
DR. COLTIN: That's fine. Thank you, Marjorie. I think that pretty much concludes what I wanted to say about the report. What I then wanted to do was just bring up a little be for the remainder of the day today. We have been asked by AHRQ to serve as a focal point for taking testimony on the National Healthcare Quality Report.
We will be doing that today. We will be providing a transcript of these proceedings to AHRQ as well as the minutes of the meeting summarizing what we think are the main themes that we heard from the presenter today, but the process of synthesizing all of the testimony will reside with ARHQ because they are also collecting comments through other source: through the Internet, and they will be receiving many comments just as occurred with the HIPAA process when comments came into the agencies directly and not just through the committee.
The summary of recommendations based on the comments will be produced by AHRQ and not by this committee. However, AHRQ would like to have us submit our own recommendations as well. That is something that we may want to schedule a telephone conference call to try to gain some agreement on before the September meeting, but I understand that it may not be too late if we can get these documents in at the September meeting.
So I'm hopeful that we can either have a preliminary conference call or we can perhaps use e-mail to try to at least try to formulate a draft set that we can then discuss at the September meeting and reach consensus on to forward a letter at that time. That is the process just so that the members know how this is going to work for today and what we will need to do between now and September.
MS. GREENBERG: It sounds to me like you're going to want a few hours at the full committee meeting in September?
DR. COLTIN: Yes. I don't know if it will be a few hours. I think possibly an hour would suffice for the recommendations on the National Healthcare Quality Report not our quality report. I think that we're going to need time for the work group to meet, but I'm not sure that we'll need time on the full committee agenda for September. We will in November.
MS. GREENBERG: I was thinking of the working session of the work group.
DR. COLTIN: Yes, we'll need a couple of hours for that. Absolutely. All right. We have finished even earlier than I thought. So unless anyone has any additional business for the work group, I think what we will do is take a break and will resume. Are all three of the presenters for the first panel here? We could even start earlier.
We can do that if that works out and then maybe we can start around 10:15 and take a break now. All right. Thank you.
(Short break at 9:40am.)
DR. COLTIN: Before we start, I have one brief announcement. That is that we are going to be taking public comment on the National Healthcare Quality Report beginning this afternoon. If there is anyone here in the room now who would like to offer public comments during that session, Jackie Adler at the table in the corner will have a list. We'd like you to sign up so that we have some idea of who will be making comments during the public comment period. If you can, please, see Jackie when you have a moment and put your name on the list.
For this section, we're going to hopefully have four presenters. We have three who are with us now and one who we expect shortly. The first presenter will be Paul Conlon representing the American Hospital Association. He will be followed by Jerod Loeb representing the Joint Commission and then we'll hear from Karen Kmetik representing the American Medical Association.
When he arrives, we will also hear from Jim Mortimer who is representing the Midwest Business Group on Health. Before we get into the panel, we're going to hear a brief overview of the National Healthcare Quality Report and Tom Reilly will lead that part of the discussion.
DR. REILLY: We want to talk today a little bit about the National Healthcare Quality Report as Kathy mentioned and we're specifically focusing in on the preliminary measures that we developed around the National Healthcare Quality Report and try to get some feedback around that.
Before I get going, I wanted to especially acknowledge my colleagues who have been working on this with us. This has been an inter-agency effort. We have a large coordinating group around the NQR that includes many, many agencies. We have subgroups who do a variety of different things. One of these is the measurer's subgroup and I want to acknowledge them, especially a couple of them who are here today: Irma Arispe from NCHS, Steve Clauser, who I think will be here shortly from NCI, Julia Holmes from NCHS, Ed Kelly from AHRQ, Beth Kosiak from CMS are all here today to get input from folks.
Just a bit of background, the National Healthcare Quality Report was mandated by Congress and this is basically the language of the legislation. Beginning in FY2003, the Secretary, acting for the Director, shall submit to Congress in any report on national trends and the quality of healthcare provided to the American people.
That's the extent of it, the mandate that's been put before us. It's obviously very broad and we spent a lot of time thinking about the mandate and what do we do with this mandate? How do we approach this mandate? We've been doing several activities to think about now we approach the mandate.
We developed a conceptual framework which I want to talk about, we've identified measures to populate that framework and those are the two things we especially want to focus on today. We also want to talk about identifying data sources for the measures, conducting research on report design and longer term developmental activities that we're trying to initiate again to provide a little context and where we're headed with things.
The focus of today's discussion is really on the measures that we've identified so far. Thinking about a conceptual framework. There's lots of ways to think about organizing a report like this, lots of ways to think about slicing and dicing material.
We contracted with the IOM to think about of the many approaches we might try, which one should we adopt for the National Healthcare Quality Report. The IOM formed a 14-member committee chaired by Drs. Roper and Epstein around this effort. They heard expert testimony from a variety of sources.
They did the IOM thing. They commissioned papers, had external review, and so forth and produced a final report to us called Envisioning the National Healthcare Quality Report that made a series of recommendations to us. One of those recommendations was to think about what sort of conceptual framework should we use in addressing a mandate.
The framework they recommended to us can be thought of as a matrix where the column of the matrix are basically the aims that the IOM laid out in the Chasm Report, their effectiveness, safety, timeliness, patient centeredness, and equity which is cross-cutting all of this and I'll talk about efficiency in just a little bit.
Those are the things the IOM had articulated as goals or aims to move forward on in the 21st century and things they encouraged us to track in the National Healthcare Quality Report. The rows of the matrix are basically the consumer perspectives on healthcare needs laid out by the Foundation for Accountability.
They are staying healthy, getting better, living with illness or disability and coping with the end of life. Basically, those rows remind us that we're not focusing on just one element of healthcare. We're not just focusing on hospitals, for example or just focusing on nursing homes.
We need to look at the full spectrum of consumer needs as we try to think about each of those aims for the healthcare system. The combination of those rows and column create cells. There are cells that represent measures around the effectiveness of helping people stay healthy. There is a cell around the effectiveness of helping people get better when they're sick and so forth and so on.
The basic drill is to identify measures to populate those cells. We were very happy with the framework that keeps us in line with a lot of the other stuff that's going on in the quality of measurement world right now and it's broad and inclusive and that's the kind of thing we were looking for.
It is very broad and very helpful and helps us organize our thinking, but it is very broad. We had to take a next step and begin fleshing out this framework. One of the things that the IOM had recommended in the Chasm Report for the HHS and then recommended again to us in the report ton the National Healthcare Quality Report is to start thinking about priority conditions or priority areas to focus your attention.
It's hard to do everything in all places. That being the case, we started thinking about that effectiveness column for example or the effectiveness of care for what? You start thinking about priority areas or priority conditions. That took us to start breaking up those column especially the effectiveness column because there is so much work and quality measures that has been done around effective so far.
We started to break it up into priority areas. There are lots of priority areas one could think about. We have put in place a process with the IOM again to try to think about how should we identify priority areas of the National Healthcare Quality Report and other activities in the department.
That process is going on now. We couldn't really wait for that process to come to conclusion before we started moving forward on the National Healthcare Quality Report. We had to identify some interim priority areas. There are lots of ways to think about doing this. We thought whatever the IOM comes up with, we'll at least roughly be aligned with the priority areas in Healthy People 2010.
We identified areas from Healthy People 2010 as the priority to try to align ourselves in filling out the matrix. We took the effectiveness column and identified a series of priority areas including cancer, chronic kidney disease, diabetes, heart disease, HIV, maternal and child health, mental health, respiratory disease, and long-term care.
Those were the interim priority areas we identified to begin our work. Within each of those priority areas, we then tried to think about what are the categories of care you want to capture in each of those cells. Just think about heart disease for a moment. What are the kinds of thins you want to capture about the effectiveness of care of helping people stay healthy having to do with heart disease? Screen for high blood pressure, high cholesterol, counseling on risk factors such as smoking cessation and so forth. What kinds of things do you want to capture around heart disease and helping people get better? Things like treatment of AMI and acute heart failure. What are the kinds of things you want to capture for heart disease and helping people live with disability or illness? The management of hypertension.
Within each of those then are measures you want to try to capture. As an example with the treatment of AMI, what measures do you want to use to try to capture the treatment of AMI and that's what reflected in the measure set that we identified. We tried to go through a process and I refer you in your handouts entitled National Healthcare Quality Report preliminary measure sets.
We sent out calls for measures. The IOM sent out a call to the private sector. We sent out a call to federal agencies asking for what measures would you like to recommend to us? We got all kinds of measures in. We sorted them into this framework and tried to go through a process of evaluating measures.
Just focusing on treatment of AMI, we have a whole set of measures recommended to us around the treatment of AMI. We tried to go through two processes in evaluating those measures. One was, we didn't want to recreate the wheel. If there is already an existing, consensus based measure set out there having to do with the treatment of AMI, we want to align with that.
If there's something that's already been developed that there has already been a large degree of consensus around, we want to capitalize on that. We don't want to recreate the wheel. We also want to try to evaluate the measures that were candidates for us using some criteria recommended to us by the IOM and those were important scientific soundness and feasibility. We have to be able to do the measurements.
If you flip as an example to page 4 in the preliminary measure st, we went through that drill. We had bunches of measures that people had recommended to us and we thought about what is the consensus based processes out there, these are things the Joint Commission has worked on around the treatment of AMI.
There is work coming together around the Medicare program and from the forum. We want to align around the measures identified by those processes. That being the case, those are the recommended measures in our preliminary measure set for the treatment of AMI.
There is a series of process measures and we try to include an outcome measure. In each of those categories, we try to identify a mix of process and outcome measures. That's the drill we went through generating each of those categories. We took the IOM matrix, tried to flesh it out, identify categories of care, and then identify measures within those categories.
Let's take another example looking at respiratory disease as an example. What are the things around helping people stay healthy? Influenza and pneumovac. Around respiratory disease and getting better? Treatment of pneumonia or upper respiratory infection. Around respiratory disease and living with illness? Management of asthma.
Within each of those are measures. Influenza immunization reflected on page 7. Process measures are immunization for high-risk populations under age 65, for those 65 and over, institutionalized adults with hospital admissions as an outcome.
I should orient you to the table a little bit more also. The first column in the table is a recommended measure from us. The second column is the national data source for national estimation. The next column is the source for state level estimation.
We've done a lot of consumer testing around what people want from the report. We hear from the Congressional types is this national level stuff is really nice, but what's happening in my state? We were pushing hard to identify state level data where we could.
The next column is a consensus process that aligns with the measures and we were trying not to recreate the wheel to identify existing consensus based processes for the different measures. There is a notes column. It give you the measure, national estimation, state level estimation and where we went to think about consensus.
We did that drill for each area and end up with this as our preliminary measure set. I invite you to take a look at it and see whether we've captured what's important or not? Have we missed important things or views or are we including some things that are not so important. In our stage right now, we're trying to get some input.
We broke up the effectiveness column in those priority areas. Lots of the work that has gone in quality measurement has been around the effectiveness of care. Safety, timeliness, and patient centeredness, you'll recall the original IOM matrix or the other column that were important for us.
The data development in those areas is less well developed than is the case in effectiveness. These measures tend to be more cross cutting in nature. As we move on through time, we see these areas becoming more condition specific, but for now we thought they had to be cross cutting in nature.
For safety, we tried to identify complications of care as a category in prescribing medications. Within there are measures for timely as basic access, getting appointments, and wanting times within each of those are measures.
Within patient centeredness, we have measures around the patient experience and let's take that as a final example. That is included on page 11, percent of patients reporting that doctors listen carefully, explain things clearly, show respect for what they had to say, and spent enough time with them.
the national estimation we go to MEPS. For state level estimation we to the national caps benchmarking database. We try to pursue that sort of national and state level estimation wherever we could. The final point I want to bring up around fleshing out the matrix is where possible, we plan to look at breakdowns of measures of effectiveness, safety, timeliness, patient centeredness by socio-demographic group, insurance status and so forth.
This is trying to capture that equity dimension which is cross cutting across all of the cells in the matrix. We will touch upon those at a high level in the National Healthcare Quality Report. Those differences will be explored in more detail in the National Healthcare Disparities Report.
Lert me just talk briefly about the process we've been through to identify measures to populate the framework and where we're headed. Our measure selection process was to establish a conceptual framework which we did with the IOM, that's the matrix. We identified a pool of candidate measures. I mentioned this previously.
The IOM put out a call to the private sector. We put out a call to federal agencies. We then mapped those candidate measures into the framework, re-evaluated the candidate measures with consistency with existing measure sets and through the IOM's measurement selection criteria.
That led us to identify the preliminary measure set which you have right here. Where we are now is we're trying to do an analysis around that preliminary measure set and that includes two things. We're trying to get input from stakeholders and other interested parties and that's why we're here today trying to hear from folks about what do you think. Are we on the mark or are we off the mark or where can we improve this?
At the same time, we're trying to do some estimation related analysis around these measures. Over the next few weeks, we hope to have input from a variety of different groups on what we've done, some number crunching done to take a look at the measures that we actually have proposed so far. At that point, we'll be prepared to make some final decisions about which measure we'll include in the final report.
Today, we especially want to get input from everyone around this preliminary measure set, but I want to talk just a little bit about our other activities to set a stage for where we're headed with things. In identifying the measures, we had to think about available data sources for the measures.
Again, one of our key criteria was feasibility, can we actually do the measurements? We're drawing from data from all over the department. We're doing population based data collection efforts, the National Health Interview Survey, and MEPS. There are establishment and provider based data collection efforts. There are the national Healthcare surveys that NCHS has for example.
There are administrative and regulatory data that we're using. There's the claims data and the MDS data from CMS. There is vital statistics and surveillance activities. We were trying to draw on lots of data from around the department.
The first report will rely heavily on existing centralized federal data sources. We've got to have the data in hand to turn the crank to crunch the numbers. As time goes on, we plan to increasingly utilize private sector data and other data sources where we can. The federal data sources are not the whole pool that would be useful for the National Healthcare Quality Report. As time goes on, we want to try to broaden our base on the what's represented in the NQR.
Let me talk briefly about some of the work that we've been doing around report design. We've been doing several activities. One is we did an inventory of products of similar reporting projects. We're not the first ones thinking about doing this kind of work. There's lots of people doing lots of things both within and outside the US.
We tried to do an inventory of those projects and try to understand what's been done to date. We've also been doing audience research around report design. Our first phase was on general design options. I just asked: What do you want this thing to be? It could be a lot of different things and we try to talk about how do you want this thing to work and what would be useful for you?
When we got through that phase, we talked about reorganization with folks and we will be talking about formatting as we go a little bit further down the line. As a result of that work, the inventory work and the audience research work, we thought about a reporting strategy.
There are several distinct audiences. I listed three here, but you can probably think about more audiences than this. Our primary audience is policy makers. It's a mandate to Congress and we have to report each year to Congress.
As we have done our audience research and we talk to people about what we want, there are really two groups that fall out. The first is what I grossly called the summarizes, what's the bottom line? Just give me the news. I don't want a lot of detail. Those tend to be the policy maker types.
A second group is what we call the analysis. Those are people who want all the details. They want all the tables. Let me look at the tables and plow through those myself. The product that you design to serve those two needs is quite distinct. We're trying to serve both of those needs as we can in the report to Congress.
We anticipate that they'll be two major parts to the report to Congress. The first is a set of appendices which will have very detailed tables breaking down each of those measures that we include in the preliminary measure set.
We anticipate having a national level table to attract trends and socio-demographic breakdowns as I mentioned and state level tables where we can and that will be a lot of tables. It's there for the analysts so they can plow through that and get through it. There will also be appendices around the specifications and so forth.
Based on those detailed tables, we'd write a summary report and that is for the summarizes, the policy types and that will be as short as we can make it. We want an executive summary of two pages, no more. If we can 2-3 pages with five take-away messages in there and then a short report, 50 pages if we can keep it to that, backing that up.
It would be a very short high-level summary report that draws upon the data appendices, but doesn't detail and march through each of those elements. That's not what we're trying to do for the summarizes. They want the bottom line.
As time goes on, we'd also anticipate creating a Web-based reporting tool. If you think about all of the tables, it's not really that efficient to plow through page after page after page of tables. It's much more efficient to have a warehouse of tables with front-end software where people can come in and drill down until their heart's content.
We have on our agenda developing a Web-based reporting tool like that, that could serve any level of person who wants to come in because they can go to different levels. It could be for policy types or analysts or the public if they so want it.
We also anticipate developing a report for the public which would be higher level summary even yet. As time goes on, you could anticipate developing other reporting products for other audiences and we'd like to move in those directions. We'll be happy in the first year if we just get the report to Congress done and then we'll build from there.
We've been working very hard to get things ready for the first report, but we're also trying to initiate longer term developmental activities. It's an annual report. As we're building the first report, we need to be thinking about what we need to do for future reports.
We're working in several areas. We want to support research and do research around quality measurement. One of the things that the IOM matrix is especially important and useful for us is it helps identify gaps, places where we don't have good measurement. As I'm sure many of you will notice, there are many areas in our preliminary data set that are not well developed. We need to work on those and working through this process and working with the IOM's process for us has helped us identify those gaps. It can help set an agenda for our quality measurement activities.
We're also trying to put in place international collaboration. We're not the only country to be doing this reporting activity. There are a number of other countries thinking about national reports on quality. To the extent that we can, we want to identify a core set of measures that can be done consistently across country.
Can the differences in health systems makes it difficult to identify what those might be. If we can only identify even a few that can be done consistently across countries, we all think that would be a tremendous step forward.
We want to develop work around expanding data sources. We want to work on enhancing existing data sources. The NCHS is working around developing some surveys to provide more quality and content for other purposes and the same with our MEPS survey. We want to continue to enhance those activities and identify additional data sources.
We want to move out in the direction of identifying private sector data that will be useful for the report. We want to develop and identify additional products to promote ongoing improvement in the report.
In summary, the National Healthcare Quality Report will provide information that monitors the nation's progress toward improved Healthcare quality. We're currently in the design phase for the first report in FY2003. Our underlying approach for the whole thing is this is an evolutionary process. We'll do the best we can on the first report, but then we'll build from there and continue to expand it.
As I mentioned, where we are now is getting input from folks around those preliminary measures. We here today to try to do that and we welcome your input. We're also accepting written comments up to August 25th and those should be sent to Ed Kelley and his e-mail address is ekelley@AHRQ.gov. With that, I'd be happy to answer questions, but I think Paul is on a bit of a short time frame. Perhaps we could move to Paul and I could answer any questions anybody has after Paul speaks. Will that work?
DR. COLTIN: That works okay.
DR. CONLON: Thank you. Ladies and gentleman, I am Paul Conlon, vice president for clinical quality and safety at Trinity Health. Trinity Health is the third largest Catholic healthcare system in the United States. We own or operate 46 hospitals across the US. We own 29 of those hospitals. We have 366 outpatient clinics, numerous long term care facilities, home healthcare, hospicing and senior housing programs from Silver Spring, MD to Fresno, CA.
We deliver 1 percent of the nation's babies and provide 1 percent of the nation's inpatient cardiovascular care. There are over 7,000 physicians on-staff in Trinity Health facilities. What we think is pretty important for the community to understand is that of those 7,000 only 500 are employed physicians. It's a private practice model and that's pretty unique.
My role at Trinity Health is monitor and measure our clinical quality and safety activities and to provide support to our clinical improvement activities on behalf of the system. My professional training is I'm a Doctor of Pharmacy and an attorney. There was mention made that I have to leave for part of this meeting. It's because we have a clinical quality contact conference call the fourth Thursday of every month at 12 noon (ET) and we're talking about ACORE measures at Trinity at noontime. I've got my staff starting the meeting and I'll join them after my testimony. If I get up and leave, it's not because of anything that anyone has said.
I am here today on behalf of the American Hospital Association's nearly 5,000 hospital, health system, network, and other health care provider members. Our member hospitals and health systems are committed to providing safe, effective care to the patients who entrust their lives with us. We are pleased to be able to contribute to the discussion of what a national report on the quality of health care should address. The AHA is committed to helping its members improve healthcare quality.
Since 1998, the Institute of Medicine has produced three major studies documenting that the quality of health care in the United States is quite good, but not as good as it could be. To focus and inform, Congress tasked AHRQ and I'll refer to you as AHRQ, with developing a national report on the quality of healthcare.
AHRQ is to be complemented for its efforts to reach out to interested parties in the development of this study. In this hearing and the open call for comments that was published in the FEDERAL REGISTER important steps in increasing interest in the National Quality Report and gaining health care's commitment to using it. AHRQ proposes to frame the report using four os the six aims Crossing the Quality Chasm, specifically, efficiency, effectiveness, safety, timeliness, patient centeredness. Efficiency is thought be beyond the scope of the quality report and will be dealt with more comprehensively in the the National Healthcare Disparities Report. Using four of the six themes articulated by the IOM to frame the report is wise.
It will the AHA and its members to integrate information from the National Healthcare Quality Report with other efforts that use the same framework. The four aims are to be correlated with aspects of care adapted from the framework from FACCT, a quality measurement group. These aspects of care include staying healthy, getting better, living with illness or disability, and end of life care as Ed has already outlined. The four aims and four aspects of care will form a matrix, which AHRQ proposes to fill in with measures for approximately 15 priority conditions when they have been identified.
AHRQ has asked the IOM to help identify, but they will not be ready in time to produce the initial report. AHRQ has worked with other parts of the Public Health Service to identify diseases on which it will report in the first iteration of the NQR. They are: cancer, chronic kidney disease, diabetes, heart disease, HIV/AIDS, mental health, respiratory diseases.
This list is an appropriate starting point. There are two additional general categories consisting of long-term care and maternal and long term child care which AHRQ proposes to report data on as well. However, AHRQ's proposed measures would only judge the effectiveness of care for these diseases, conditions, and general areas of care.
Unfortunately, the manner in which AHRQ intends to measure safety, timeliness, and patient-centeredness does not include specific data that will be helpful to healthcare providers particularly to help us understand how we are performing on these aspects of care for the patients with the identified priority conditions.
At AHA, we believe hospitals have to be able t to answer two fundamental questions about patient care: Did we do the right things? Did we do them well? If we can answer yes to both these questions, then we will know that we have served our patients well.
In effect, the NQR is an attempt to answer those questions for the nation. The proposed measures of effectiveness will indicate whether healthcare is doing the right things. The measures of safety, timeliness, and patient-centeredness will tell us if we are doing them well.
AHRQ's proposed list has a blend of outcome and process measures. It must be recognized that many of the outcomes are only partially within the control of health care. For example, the number of deaths from any disease or disorder is a reflection of many factors beyond the care provided, including the prevalence of certain environmental factors, aspects of individual behavior, and genetics.
Similarly, the number of new AIDS cases is an outcome measure that health care seeks to influence, but which will also be influenced by a variety of other social, biological, and cultural factors. As a matter of public health, it's important to know that many people are dying from a number of diseases. As a matter of public policy, the number of deaths from different diseases may help the President and Congress judge how to allocate scarce scientific and treatment resources.
From these outcome data, one cannot whether health care is in need of improvement or where their particular need is or whether social, environmental or behavioral factors need to be addressed. To know whether the right things are being done for patient, critical steps in the process must be measured.
Clearly, measuring every aspect of care would waste valuable resources. Only those elements of a process that are critical to achieving the desired outcome should be measured. We propose that each disease or disorder included in the NQR have a description of the target or goal, which would be developed using the data from the outcome measures. Then the quality of the health care would be judged based on how often the appropriate steps in the process were done and whether they were done well.
For example, under heart disease, the outcome measures that would be used might include acute myocardial infarction mortality within 30 days of admission, mortality after heart failure, hospital admissions for congestive heart failure, and percentage of hypertensive
patients with blood pressure under control.
The process measures that are addressed whether the health care system was performing the appropriate acts to prevent these undesirable outcomes might include appropriate screening for blood pressure and cholesterol, time to reperfusion and beta blockers at discharge for acute myocardial infarction patients, percentage of patients with heart failure whose left ventricular ejection fraction is measured, and percentage of those whose ejection fraction is judged too low and who then get ACE inhibitors at time of discharge.
Since AHRQ has not identified readily usable process of care measures for treating those with hypertension and congestive heart failure, these outcome measures might either be dropped in the initial report or included with a notation that have yet to be identified, but might be in subsequent reports.
If I may add a comment, process measures should directly related to the outcome measures. For some diseases or disorders, AHRQ has identified several outcome measures of interest and multiple process measures. We urge AHRQ to limit the number of outcome measures, and the number of should include only the critical few needed to asses key steps in treatment. The selected measures should be supported by clear and convincing evidence of their effect on care.
Our experience shows that there would be no more than five process measures per disease or disorder. If more are included, maintaining focus and priorities for improvement becomes difficult. For some of the diseases included in the report, ambulatory care sensitive diagnosis measures are being proposed.
While ambulatory care sensitive diagnoses are well known and understood in parts of the health care community, they are not well known and understood in the broader public arena. The text surrounding these measures must clearly identify that the problem addressed by these measures is likely to be the consequence of the individual's inability to access primary care and not one of inappropriate admissions.
The report should be clear that it's not discouraging appropriate hospital admissions, but is encouraging improved ambulatory care. AHRQ also proposes to include measures of post-acute and long-term care that were recently reviewed as part of a proposed measure set at the National Quality Forum. The results of these studies are expected between now and October.
For this reason, CMS asked the NQR to delay consideration of these measures until after the study results are available and they show clear and convincing evidence of the validity. The NQF board agreed with this recommendation. We would urge AHRQ to adopt the same position, don't adopt measures that aren't proven to be valid.
If AHRQ cannot wait to make decisions about these long term measures for he NQR, we urge ARHQ to leave these measures out of the first edition. This is a case where no information is better than unreliable or invalid information.
Finally, we would like to raise questions about a few of the specific measures that are under consideration. Under the Chronic Kidney Disease category, AHRQ is proposing to measure the number of patients on the waiting list for transplants and also the number of transplants performed. These numbers will likely tell us something about the lack of available organs, but not about the lack of the quality of the care provided to patients and, therefore, should be eliminated. They may be better classified as measure of access, but not process or effectiveness types of measures.
Under the Cancer and HIV/AIIDS categories, there are proposed measures looking at the proportion of people who died that were referred to hospice care and the number of days of hospice care they had received. Is referral to hospice the critical step in the process of care of the
terminally ill that warrants measurement? Or do the critical aspects of care include appropriate patient wishes regardless of whether that care is rendered in management of pain and respect of hospital, at home, in a long term care facility, or in hospice?
Of note, there are lots of inpatient programs on palliative care, bridging programs between palliative care and hospice programs that the absolute numbers relating to length of stay in the hospice program may not reflect the true care of the terminally ill.
Under mental health, a proposed process measure examines the number of patients with depression who received treatment. It is not clear whether the proposed measure had been tested and validated. Even if it has, it is more likely to measure whether people access the health care system rather than the effectiveness of care and should not be included as an effectiveness measure, but maybe better classified as an access measure.
In addition, the proposed measures of timeliness are measures of access rather than quality. Like quality, access is a critical policy issue that should be studied in a comprehensive manner to provide decision makers with needed information. We believe a separate report on access issues should be developed rather than including a few such measures in the particular quality report.
We would like to thank the NCVHS and AHRQ for the opportunity to discuss the important issues of what data elements are contained in the NQR and how that information is used to create real data for all who seek to improve care, either through changes in policy or changes in our performance.
We believe that the NQR will be most helpful if it can, I have three points here: 1. Identify specific outcomes for which improvement is sought as a way of providing focus for improvement efforts; 2. Assess whether the health care field is appropriately performing those steps in delivering that care; and, 3. Provide an assessment of how well we have provided those services were provided.
We encourage AHRQ to include the outcome measures in the report as information that helps the readers of the report understand why a certain disease or disorder was chosen for inclusion.
We further urge that only process measures that only process measures that reflect critical steps in caring for a patient should be included among the effectiveness measures. Although this will be a shorter list of measures than originally proposed, it will enable providers to focus their improvement efforts and that's the goal. We realize that this may leave gaps in the report. We pledge to work with AHRQ to develop measures and data to fill those gaps.
DR. COLTIN: Okay. Thank you. Does anyone have any requests for clarifications or follow up? The we'll proceed to hear from Jerod Loeb.
DR. LOEB: Thank you, ladies, for inviting me. As you can see, I'm Jerod Loeb. I'm vice president for research and performance measurement at the Joint Commission. It's the nation's oldest and largest health care accrediting body. We accredit about 18,000 health care entities in this country as well as internationally now.
In the realm of hospitals, we accredit just under 5,000 hospitals representing about 80 plus percent of the nation's hospitals but representing about 96 percent of the hospital beds. As you'll see from my comments, I've chosen to approach this somewhat differently that Paul and the AHA in that I would suspect that you would agree once you hear what I have to say that I'm probably at 5,000-8,000 fee as compared to Paul who is right at ground level.
I do that for several reasons. This is a preliminary reaction and we do intend to submit written comment to Ed by the appropriate deadline. What I've chosen to do is to provide some weighing here in the traditional academic sense for the questions that Tom and Ed posed for the individual speakers to reflect on, the first of which is: Do the initial measures meet the objective criteria?
I think there is great concordance here with the envisioning the National Healthcare Quality Report and I think this is a pretty good thing and it's a good grade. Are there gaps in the measure set? Sure there are, but we also understand at the same time this is only meant to be a preliminary measure set and as a preliminary set, it's a rousing A+ from our perspective.
Appropriateness of data sources. There are some conflicts with respect to included and excluded populations. I'm not exactly sure how that will be addressed. I'll go into more detail in a few moments on that, but perhaps most importantly, I want to make a very strong statement about the importance that AHRQ has placed in doing this in a collaborative manner and I think today is one example, but there are myriad of examples in which collaboration has been sought from the very beginning and what more could you ask for.
This is great. Lots of stakeholders have had input. Greg Myer told our Board of Commissioners this past weekend that the ultimate goal of the National Healthcare Quality Report must be to have a second report. I think in the realm of health care quality reports that have been issued over the last 20-25 years, that's a very important point.
We need to make sure that the quest for perfection is not going to stand in the way of simply being good or better stated, it is what it is. This is a preliminary report and as such, that's okay. This is a key challenge to AHRQ.
I can tell you from the perspective of being measured or the measure, measurement is not a neutral activity not is reporting performance measure data. It's as if you take your tongues and stick it into an electric socket repeatedly and it doesn't feel so good as many of you in this room know.
Another key challenge, held out philosophy. As you look across the folks that have been doing performance measurement. There really isn't a concordant focus or philosophy. Dr. Carbon had one and that's because he was the only one doing it at that time back at the turn of the century. He filed all of his data on small cards that were kept in his storage room. You can see him there doing that.
Another challenge. What should be measured. I think you would all agree with the statement that while we have dashboards on cars and we know what those indicators are going to be, we don't have a health care dashboard. Whether the National Healthcare Quality Report becomes one or not remains to be seen, but we all need to remember that what gets measured gets managed. What doesn't get measured oftentimes is not paid attention to.
Measurement creates distortions. The types of measures that are needed in the report, we believe very strongly that in the donabediean framework, there ought to be structure, process, and outcome measures first of which relates to the structures in health care.
The structures and the measures most likely are going to change over time. I think we need to be sensitive to that. Staffing effectiveness is on the radar screen quite prominently today at the national level.
Process measures. The processes and the measures are going to change over time as clinical information changes. Beta blockers post MI is obviously very important today whether it's going to be inter tension receptor blocking drugs or something else in the future is going to remain to be seen and we need to be sensitive to that.
As far as outcome measures, I think it's pretty clear that the outcomes are not going to change. Either patients are going to live or they're going to die, but the measures most likely will be changing over time. Most notably as the science of risk adjustment marches forward.
You can see the grades that I've given to the report at this point. It's a pretty solid pretty solid performance. To continue on at least in terms of my own thinking about performance measurement and health care, as I look at the horizon, I think there are leading coincident and lagging indicators.
This may be just another way of framing things, but it's an important way of framing things. Leading indicators really allow for stakeholders to understand something about predictability of care processes. They can relate to structures or processes. In the report, there are measures that address preventive care and resource use and including staffing is an important piece of this.
Coincident indicators are those that focus on performance, usually they're process measures and you've got a lot of those in the report and that's great. Use of beta blockers after MI is an example, but what about the rear view mirror, the lagging indicators?
They focus on performance, but usually performance that occurred yesterday or the day or month before. Traditionally, we think of these as outcome measures. As Paul stated earlier and for good reason, there are not very many outcome measures in the National Healthcare Quality Report as yet, but I would like to simply call attention to the fact that I believe that's a need for the future.
If you take the Mark Chasin classification of overuse, underuse and misuse with overdue the things that come to mind are antibiotic use, the geographic disparities and I don't think there's a good grounding yet in the National Healthcare Quality Report relating to that.
Underuse, clearly there are a host of measures. Beta blocker or ASE inhibitor use post MI or heart failure. In terms of misuse, there are a few, but we're not all the way where we need to be yet. There are some measures of selected post-op surgical complications and birth trauma, but we need to think more about that in the future.
One of the problems as I think AHRQ has already anticipated in looking at these multiple measure sets is they are different kinds of sets. The granularity in these sets very much differs. The measure sets have widely disparate amounts of granularity associated with them.
Another issue is data integrity. As you look at the various data sources, most people would agree that the level of data integrity or quality is going to vary depending on the source of the data. How do we address that in a report that assumes that the data integrity is similar across all sets.
What about populations? The measure sets include variable populations and that's going to be a difficult task in terms of the communicability and interpretabilty piece relative to performance measurement. As I look at the measure set, one of the questions I was asked to address sis where do you see gaps?
Any one of us can look at this from the perspective that we might occupy at any point in time and heavier different answers. There's no magical answer here, but as I look at it, it struck me that there really were not very many measures that reflect on surgical procedures and I think there is a great deal of interest in that in the public sector.
More emphasis might help. More emphasis on measures that are pertinent in the ambulatory setting. There is a great focus on measures outside the ambulatory setting, but I think we probably ought to begin looking more and certainly as it relates to safety at the ambulatory setting which is obviously a significant concern.
The focus on mental health while it's a starting point, it's looking exclusively at depression and doesn't focus on other important aspects of mental health care such as substance abuse. As I think about measure sets, this is a crazy construct I developed 7-8 years ago, as interlocking pieces of a puzzle.
The puzzle here is reflected as you see in this slide by IOM domains. This is just an example. Effectiveness, prevention, safety, patient centeredness. How do these pieces fit together? Ultimately, these pieces need to fit together and paint a picture.
As I was sitting over this past weekend with one of my daughters who is taking a computer class, she's entering high school in the Fall, she was looking over my shoulder at this and as I was trying to explain to her what I was doing, we together came up with a great idea and I need to give credit because I hired her to develop this next group of slides for me.
If you take these puzzle pieces and really think about it, each puzzle piece is reflective of a given measure set. Underneath this is a portrait and that's quality and the question is how do we get to quality? How many of these puzzle pieces do we need to uncover before we are able to determine what, in fact, lies underneath?
My guess is we can probably stop here. Most of you will appreciate that underneath these puzzle pieces is the Mona Lisa, but you don't need to uncover all these pieces. Measurement costs money. The push back relative to those that have to collect the data is significant. I have lots of bullet holes in my body to prove that point over the last 8 years or so that I've been at the Joint Commission.
The question of how we sort out accountability is a very critical question when one thinks about performance measure. Beta blocker measure is a great process measure post MI. To whom do we hold accountable? The physician who didn't write a prescription for beta blocker, the hospital who doesn't have an existing practice guideline to which it holds it's physicians responsible?
Is it the health plan that has multiple hospitals that it has to hold accountable? Where do we sort this out and is the same measure reflective of accountability at all of these levels or do we need different measures? I'm not sure there's a good answer to that in the National Healthcare Quality Report nor do I think there can be a good answer at the stage we find ourselves in health care today.
Part of the reason for that is you have to look at who built the measures. You've got the performance measurement experts that don't know anything about clinical medicine, the clinicians know everything about clinical medicine but nothing about performance measurement and the entrepreneurs who are doing this because there is a way to make money. They know nothing about performance measurement or clinical medicine and they care about making money.
There is this inherent conflict in terms of the measure sets that exist which creates a problem. I understand from the presentation you made, Tom, that consumers are not the primary audience, but they are one of the audiences. The question of whether consumer concerns will be met by the National Healthcare Quality Report is an important consideration.
In the old paradigm, consumers asked very simple questions, am I getting healthy, will I get better, and traditionally quality has been assumed, quality of physicians, hospitals, long term care facilities that I think we're in a very new paradigm today. The new questions are not only am I healthy and will I get better, but will I be safe or killed in my health care facility? Is this a good hospital, nursing home, doctor?
I don't think quality is any longer assumed on the part of the public. Those of us in Chicago certainly will know that just between Sunday, Monday, and Tuesday the CHIGACO TRIBUNE did a story which was designed to sell newspapers on nosocomial infections which has created a significant uproar in the community about whether hospitals are safe places relative to those infections.
We can see that there is a great deal of consumer interest in this at this point. Perhaps the final challenge, how to report the data? As I went back and thought about this, I pulled out my report card and you can calculate how old I am from this.
I hide my NY accent. This is my report card from September 1956. I was promoted and that was really critical, but if you look at how I was graded in the NYC school system, you will see I was graded with a grading scale that looked unsatisfactory, satisfactory, needs improvement, and SO which stood for unusually high level of achievement. I only had a couple of those, spelling and science.
If you compare these report cards and we probably have more experience in report cards in education that we do in any other aspect of society and perhaps we need to learn in health care from education. This is yet another daughter of mine, my younger daughter and you can see Rebecca in terms of her report card.
This is in her second grade and there were only three grades, needs to improve, progressing or consistently demonstrating. When I got this report card I fancy myself as an educator. I'm on Northwestern's faculty and have been for the last 25 years. I get this report and I was struck by the fact that she was only progressing in engages in the writing process.
Being a crazy suburban parent, the first thing I did was run to the school with my wife and asked should she be doing better than that? What does this mean? In health care, we get these report cards and go to the Net and look at healthgrades.com and we see 3-5 stars. How do we make these decisions?
A couple of years ago, 1999, this is Rebecca's Iowa test in 1999 in which you can see I was absolutely happy when I got this because I have 95 confidence intervals. I had everything I could possibly want. I had observed, expected, banding low in terms of low/high average.
As you can see, she did pretty good which I was really pleased with and I felt really positive. If you look at the upper right hand part of this slide, national percentile ranks in just about all the areas, she was way up there and I felt really good. I had comparative data, data that was specific to my daughter.
This is what we're going to need downstream in health care and the National Healthcare Quality Report is not going to provide this to us yet, but I think we need to think in that framing. Ultimately, this is Finagle's laws of information.
The information you have is ultimately going to prove to be not what you really wanted. The information you want is really going to be really not what you needed, but because what you really need simply can't be obtained. With that, I try to end on a happy note, but I'd be happy to answer any questions you might have.
DR. COLTIN: Does anyone have any requests for clarification or any more detail? It looks like everyone was happy with that. Now we'll hear from Karen Kmetik for the AMA.
DR. KMETIK: Thank you. I can never compete with Jerod's slide shows. I chose not to bring slides today. Instead, I'm going to speak to you in conversation. I want to applaud AHRQ for taking on this project. AMA and JCHO really support this activity.
Second, I want to thank NCHVS for inviting AMA to provide testimony today. The very first thing I want to say before I tell you a little bit about myself is my important message of the day is I applaud the fact that the measures in the report are not new measures.
I cannot tell you how important that is to those of us who are measure developers and who are trying very hard to work with one another to avoid redundancy and confusion in measures. That is a wonderful thing to see. I want to carry that forward.
Tom, you mentioned that in the future you'll be looking at where there are gaps and where measures don't exist and I just plead with everyone involved that when you do identify those that you come to us, to the other measure developers and let us know where the gaps are and we will try to rise to the occasion to create them rather than have another body begin to create measures. As we all know, it's very expensive. We can't be flying the same expert around the country. Let's put our best dollars and talents where they can best be met. That's my overarching message.
Just for a little background, you may not know, the AMA for the past four years has been convening the Physician Consortium for Performance Improvement, the Consortium. It's a group of methodologists and clinical experts. In Jerod's recent slide where he said sometimes you have one or other in the room, we have both because you do need both.
We've got clinical experts from over 50 national medical specialty societies. We have over 15 methological experts who come together and form work groups to develop performance measures to assist physicians in quality improvement. I'm sure everyone in this room has repeatedly read the wonderful Sulberg article telling us how important it is to define the purpose of our measures and that is our purpose to design measures that will assist physicians in quality improvement efforts.
As soon as I say that, we recognize that it is important to align those measures with measures for other purposes. JCHO has measures for a certain purpose. NCQA has measures for another purpose. The AMA with the Consortium is in a collaboration with JCHO and NCQA to make
sure that when we do develop a measure that might have a different purpose it, nonetheless, should be based on the same evidence.
To the extent possible, based on the same data elements. That is really the only way that we will get to a future of reducing the burden of data collection and perhaps coming up with some new data sources so that the quality report in the future can be based on some private sector sources.
Just to give you the very simple example that we always refer to, the HEDUS measure for hemoglobin is the percentage of patients with a hemoglobin value over 95 percent. The Consortium measure for hemoglobin is looking at the hemoglobin level of the patient over time.
The physicians tell us if you just tell me that "x" percent of my patients have a hemoglobin level of over 95 percent, that doesn't tell me a whole lot. I need to go back and know, by patient, what are the levels and what are the levels over time.
There is a different level of data analysis that needs to be done, but, nonetheless, both of us need the hemoglobin test value and date. That is the basic data elements that we need and the computer can crunch it either way. That is something that we're advocating and as we look to the future, may help inform the National Healthcare Quality Report in the future.
I did not bring handouts, but I encourage you to go to AMA Web site to see the latest Consortium measures. They're always updated and kept on our Web site. I also teach a course at the Loyola Graduate School of Nursing and last term I gave them three options to write their final papers on.
One of the options was you work at the AHRQ and you've been charged with designing a National Healthcare Quality Report. How are you going to do it? Not one student asked to do it. I was hoping to get some ideas for you, Tom. I am trying to bring this world to the graduate school level. So we'll see what we can get going some years from now.
One specific thing I want to say about the measures and we will provide very detailed comment by your deadline. The diabetes measures are listed in here as attributable to AMA, JHCO, and NCQA consensus process. I want to update everyone on a really exciting new diabetes measurement activity some of you may have heard about.
Most recently, we say to ourselves: the Consortium developed some nice diabetes measures for the physicians. We have the wonderful diabetes measures. We have a the D-quipped process that led to those and we have the AMA, JCHO, and NCQA collaboration that tried to come to the single data collection idea.
Why don't we once and for all mold these two activities into a true national effort for diabetes measurement and I'm proud to say we've done that. What you'll be hearing now is the alliance, that is the National Diabetes Quality Improvement Alliance. It includes all of the organizations who originally were part of D-quip:AMA, JHCO, and NCQA and includes several other organizations key to diabetes care and measurement.
There are 11 organizations total. We sat down and rectified, looked at every measure in the D-quip group, AMA/JHCO/NCQA group and came to consensus on what the measure should be both in quality improvement and pay level accountability. Those measures have recently been proposed to the national quality forum.
At this moment, they are out for vote by the NQF consensus process. I encourage you if you're a member of NQF, take a look at that and vote. Hopefully, vote yea. If you have questions, call me. We propose that the alliance be the keeper of the national diabetes measures.
We will make sure that we have kept up to date on the evidence. We will review them yearly, look at those folks using the measures and see their experience with it, but we will take it upon ourselves to fulfill that role. It's very exciting because you can really truly call it a national measurement set.
To the extent that that can be reflected as the consensus of those articles, it just takes it another level. Along those lines, the only area within the alliance's diabetes measures that I did not see reflected here was urine protein testing. I'm sure there's a reason and it probably has to do with the feasibility of the data.
It might be nice to note that kind of thing. Particularly when you've adopted a set that is so widely adopted by the alliance and hopefully soon by the national quality forum to further the understanding of the nation, a little footnote to say we've adopted all of those measures except one because we don't have that data source yet.
I want to concur with Paul Conlon regarding the nursing home measures. We very much supported the notion of NQF not going forward with those until we can benefit from the pilot study going on by CMS. It just makes sense. It's so close. Depending on your timing, you too can wait and benefit from that new knowledge, it just makes sense to us.
One other thing that would be useful to those of us who will be reviewing this over time and thing to provide positive input is to know a little bit more about the data sources. They're listed and can click on them and find out more, but even just a paragraph or two when you send these out. It's hard to separate the measure from the data source. It would be easier to see that alongside the measures.
Another thing that would be useful for those of us continually struggling to understand HIPAA, and thanks to Dr. Fitzmaurice. Every time I talk to him, I understand it a little more. It would be great to just have a few sentences here as to the fact, a simple statement that says the collection of this data and use of this data for the National Healthcare Quality Report is consistent with HIPAA, something to let us know how does this relate?
We talk to physicians. There are physicians who are very fearful mostly for lack of understanding. We're tying to improve that. Anything any of us in this room can do when we put out information about quality improvement and measurement, any sentence or two to explain the relationship to HIPAA is helpful.
MR. BLAIR: When you say HIPAA, is it the privacy?
DR. KMETIK: The privacy portion, yes. One other thing I want to make the committee aware of, Kathy talked about in your report talking about the long term goals as well. One of the goals that we're working very hard at the ANA is this functionality of the electronic medical record.
We firmly believe that it needs to fulfill the goal that the IOM has laid out. It's more than just asking the paper record and turning it electronic. It needs to be a source for quality improvement data. We just completed a feasibility study and we spent thousands of dollars and months of time to take an existing EMR and create that querying functionality so that the physician can now just push a button get those measure by patient at the point of care where we think if important.
They can push a deferent button and get it aggregated over all their patients for year or they can push another button and get it aggregated over their patients by health plan. The long term goal is that this may be the source of the data we need for everyone, that the physician's point of care.
Perhaps the physician can give it to the health plan to fulfill their HEDUS requirements. Perhaps in a wonderful world, it all aggregates out that we have a new private sector data sector data source for our National Healthcare Quality Report.
It may seem to you that it's not in our lifetime, but I have to believe it is. So just so you know, we're convening electronic medical records in October and this is something that we're pushing for, that these systems become functional in terms of being able to query them for quality improvement.
The last comment is that Kathy mentioned that you did have on your agenda some folks to speak about provider level measurement and that science nor experience is far enough along. I would just say that while that's true, there is a whole other world out there of helping providers to improve.
Until we're able to figure out the statistics, the small sample sizes, the risk adjustment to prepare physicians, maybe we can all jump on the bandwagon of helping them improve the quality of care. It requires good data, but we're not tied to having big sample sizes and all the other problems that you have in thing to do statistical comparisons.
With the EMR project, we're able to give doctors some data that they can use to improve care. That's a big wide open area that we can all help with until the other pieces are there that others are interested in. That's all I have. Thank you very much.
DR. COLTIN: I believe our fourth speaker has not yet arrived; is that true? Jim Mortimer? Do we have any questions or comments for any of the panelists at this point?
DR. REILLY: I just want to add one elaboration that I failed to mention earlier. One of the aims of the IOM laid out in the Chasm report was efficiency. In their report to us in the NQR, the recommended that we not include efficiency for a couple of reasons.
One is that efficiency measures is embedded in the other places in various ways. You capture overuse and underuse, you get at that and unit cost of services, they just didn't think we ought to go in that direction.
We've gotten some push back around you need to put some kind of context around the quality of measure stuff and you've got to include some assumptionat lease. We're probably going to do some of that in the first report. Mark McCullen as prepared a report for the IOM committee as part of their deliberations.
He laid out three options for thinking about efficiency moving from a resource consumption to more elaborate depictions of efficiency. We're going to go with that most general level that Mark talked about in his paper. We feel that at this stage, we have to frame it a little bit with some resource consumption measures as well.
DR. COLTIN: Any other questions or comments?
MR. BLAIR: Tom, I'm not a clinician. I read through the material when my wife read it to me over this weekend and she said this is a big matrix. She started to read it to me and it was so understandable. I just was very impressed, the logic and the relevancy just was understandable to a lay person.
Clearly, you have asked for feedback from a variety of medical professional groups and health care institutions and the AHA and all the rest. At least from within the medical domain, I don't think there's going to be a whole lot more to do to improve this. I think it's outstanding.
I was trying to think what could I offer that might be constructive. I have to go out of the box as a lay person. Forgive me if I do this and you may have already done this. I looked at the area of staying well and I thought of just a question. Have you solicited feedback from complementary and alternative medicine and have you solicited feedback from Environmental Protection Agency or environmental groups?
DR. REILLY: That's an excellent comment. We have not. That's what we ought to pursue. I think that's an excellent thought.
DR. COLTIN: Mike?
DR. FITZMAURICE: Question for Tom. The landscape keeps changing as we heard today. Groups form together when they have a common goal. Quality groups can mush together. Will there be a continuing section of the paperdio you anticipate in future years that will lay out the landscape of who's doing what in quality so that not only do we have a national measure, but a national reference for who is doing what.
DR. REILLY: I think that's one of the other pieces of feedback we consistently get when we talk to people about what do you want the report to be is they say to us frequently, this is all interesting, but what next? What do I do now? One of the things that the agency is trying to develop is some sort of Web based tool.
We hope to develop some collaboration of CMS around this that could be linked from the National Healthcare Quality Report. For example, to say if you're interested in diabetes measures, here's some information about quality improvement activity that might be going on that you can learn from to improve quality.
In the long term, you get that linkage between information we derive in the National Healthcare Quality Report to some kind of source where people can go for QI activity. That would be a great kind of contribution.
DR. FITZMAURICE: I wonder, too, if there is a license for a particular chapter or index on what's going on in the world? It sounds like it would be a five-year study and I would certainly volunteer.
DR. REILLY: We're always looking for volunteers.
DR. FITZMAURICE: Do you have a section or couple of paragraphs of what's going on in the world in terms of quality improvement?
DR. REILLY: I think we do want to have some reference in the report about talking about quality improvement. We'll probably refer folks to other sources outside to pursue that sort of endeavor. There will be some certainly. We can't just talk about measurement without also talking about what else is going on, but a lot of the activity will be in other settings and other documents.
DR. FITZMAURICE: I can see this not only being a focus for national leadership with all of these organizations contributing, but also an opportunity for worldwide leadership.
DR. REILLY: The international collaboration is very exciting.
DR. COLTIN: Anyone else? I'll take the prerogative to make a comment as well. I also applaud that work that's been done thus far. I have one question/suggestion. That has to do with the framework and the fact that you are getting some push back in the area of efficiency.
I'd like to give a little push back in the area around changing needs. My understanding of the framework that was used to represent consumer perspectives is that final category was broader than simply end of life care. It was reflective of changing needs. That suggests developmental needs in many cases over the course of a lifetime.
I could imagine categories from early childhood development and promoting healthy development through the adolescent transition to adulthood. There are standards that has been adopted by the AAP around anticipatory guidance in that area, menopause in women, and frail elders for instance.
Issues around that is a life stage issue before end of life. I would encourage you to think about that in events or you won't identify gaps in that area. I think there are gaps in that area throughout the life stage. Clearly, we don't want to meticulous everything that is part of normal human development. But there are clearly within each of those life stages where the Healthcare system does play an important role and we shouldn't overlook the opportunity to measure in some of those areas.
DR. FITZMAURICE: One thing you might think of, as you gather all of this information, you'll identify gaps and data that needs to be further discussed. I would hope you would consider having recommendations for additional research and additional consortium activity in the private sector.
The meetings may be supported by the government, but it's the private sector people getting together to develop new measures and responses to existing measure and to put an overall model onto it. It's not the government telling the private sector, here's what you do, but it's all of us looking at what we can do to make the health system better.
DR. COLTIN: Any other comments? I think we will take our break a little earlier than scheduled and let's reconvene about 1:00 or 1:30. How does that sound to people? And then at least our morning speaker will be here by then so we can at least get started earlier. Do you want to come back at 1:00 instead? Let's do 1:00.
(Lunch break at 11:36am.)
DR. COLTIN: Okay. I'm going ot call the afternoon session to order now. We have a second panel to offer testimony on the National Healthcare Quality Report. We'll be hearing from Jim Mortimer from the Midwest Business Group on Health, from David Bergman representing the Foundations for Accountabilities Child and Adolescent Health Measurement Initiative and from Walter Suarez who will be representing the National Association of Health Data Organizations. We'll start with Jim Mortimer.
DR. MORTIMER: Thank you and welcome to Chicago which is the home of Midwest Business Group on Health, an employer coalition with members that come together across an 11-state region to see what their role is in the healthcare system and to do things that will help them get better value for their health benefit dollars.
We have about 70 members. We've worked on healthcare quality issues since about 1985-6 trying to understand that aspect of what employers call healthcare purchasing. I'm pleased to have the opportunity to provide some remarks on the progress of the National Healthcare Quality Report and my points today may not be exactly in line with the kinds of things you're looking for on the data set, but they may be germane and I have a couple of comments on the data set.
The value of this report will go beyond Congress as a target audience. We expect that this will be a highly regarded and much needed piece of information for lots of different audiences when it comes out. I hope that the plans that go on will continue to look at that broad field of leaders and people who will benefit from this information.
The plans you have seem to be very consistent with the vision of the IOM and staying within that framework will produce a very useful end product. I would like to see us consider with the data set we have and the format of the ultimate report is how this information will be useful to consumers, patients, the covered population from an employer's standpoint and how it will be useful to healthcare purchasers.
I think they are two key audiences under the public column in your work. I would hope that the report shows individuals how healthcare varies. If we're going to have a national report, if we had a national average of a statistic that would be interesting, but that wouldn't tell us a whole lot.
If we were looking at these quality measures, we should be able to look at them at levels below the national average or national mean or median. Showing individuals, patients, consumers, Medicare beneficiaries, mothers and wives who are the ones who have the most to say about decisions in healthcare for a family unit and help them understand quality a little better so they don't take the quality of healthcare for granted.
By having this report or being able to look at this report on a Web site, they have a new appreciation of the degree of variation that exists in healthcare at the local level, institutional level, practitioner level, and when they see the report and see this extended variating, that they get some tips and further information on how to get more specific information and what questions to ask to pursue this in their own decision making as healthcare consumers.
This is a very important part of what we would hope for in this report. It should be variations in outcomes, process, and structure that people can see through this information. It should be posted in a place where people can know that they can find it when they need it. Consumers won't look at information until they're ready to use it.
If it is on a Web site and it's real easy to remember how to find that Web site, then they can go there and begin exploring the aspects of the information that will help them with their individual decision making. That's my first point to show information from the report that makes it clear why variation exists and help people get at that and drill down to the information that will help them make better decisions.
A second point is to prepare the report in a way that will show healthcare purchasers specific quality problems that they should be recognizing, things that need their attention. If purchasers pay for healthcare with no real information about the quality, good or bad, of healthcare, we'll continue to have a payment system that really provides no incentives or maybe even perverse incentives for improving the quality of care as chapter 8 in the Quality Chasm report says to us.
Purchasers include public and private employers. They include coalitions such as Midwest Business Group on Health and more than 90 groups like ours around the country. They include government agencies, DOD, Medicare/Medicaid, all of these organizations should be able to make use of the report as purchasers.
The problems that we see in the report should be national deficiencies. We should be able to see what the issues are across the country, but again, we should be able to drill down in some cases and see what the variation is at the state level or market area level.
Up until now, we found that the Dartmouth Atlas of Health Care is particularly useful for purchasers to understand local variation and get an idea which quality problems are effecting a given market area. The thing I would ask along these lines that's maybe beyond this scope or maybe it's what we were going to talk about when we measure efficiency is that for the quality measures we have in the report, we make an attempt to assess the cost of that quality measure as well.
If we have quality problems, they have impact on outcome, health status, complication rates. Those are the morbidity factors, the human factors. There is also the cost factor. Those quality problems have a cost, an avoidable cost if the problems are prevented.
I would see if we can't address an efficiency component with as many of the quality measures as possible. Just like there's a comment in the material about assessing the disparity component for some of the measures. Another criterion to be looked at for as many of the measures as we can.
Purchasers might be able to use those quality measures for their quality population and estimate the amount of preventable harm that goes along with each one of them, the impact of health status related to these measures, impact on productivity. If healthcare was done in an evidenced-based way and the direct cost of either correcting errors that have occurred in misuse, direct cost of overuse that really wasn't necessary or wise, underuse where people are spending more to get a better benefit. It goes all throughout the system.
If we can help translate these quality problems into cost terms, it will help purchasers understand their significance and they, in turn, will look for help translating them for consumers so that their covered populations can make better decisions.
In the measure set, the matrix that you have looks great. Just a couple of comments in the section on mental health, there is a notation about effective care. Mental health, depression that's identified in the plan is also a chronic illness. Having the management of depression in the next box below it would probably be germane.
It is something that does persist and people have lots of occurrences. I would note at the bottom of the matrix that something having to do with efficiency measures related to quality measures or some measure of the cost of that poor quality healthcare when the measure is off of its benchmark would be very useful to purchasers.
Helping these two audiences use the final report may be aided by writing a couple of other pieces to go with the report. I would think of writing a consumer's guide to using the report that could be published alongside the report to Congress and might tell people at language aimed at the consumer, what this means, how to think about some of the things in the report, how to access the report in the Web environment, how to use the Web tool, the talkingquality.gov work that's been done would be a place where that could be rooted as well.
I also think about writing and other piece, short maybe 10-12 pages that would be a purchaser's guide to using the report to help the kind of purchasers I mentioned earlier understand that it exists, what it means, collect case studies of purchasers who have been able to learn something from the report or apply it in some way to their thinking, priorities, decision making and keep publishing that as a tool to help these two audiences make access and decisions with the report.
I did not have the time in preparing these remarks to write them down and review them with our members. I would like to do that after today with your permission and circulate my statement to our members along with some of the material that you provided for comment and give them a deadline of the 21-22 of August to get their comments to me and I would like to send those comments to Dr. Kelley along with my statement and make that a part of the record. Thank you.
DR. COLTIN: Thank you. Any questions or follow up? Okay. Dr. Bergman.
DR. BERGMAN: Good afternoon. As noted, I'm a physician in practice in Northern California and I'm on the faculty at Stanford University. I'm here as a member of the executive committee for the Child and Adolescent Health Measurement Initiative known as CAMI. It's a cooperative development of individuals representing 80 major health care and advocacy organizations devoted to improving children's health care by increasing availability and use of reliable information on the quality of care provided to America's children and adolescents.
The organizations participating in CAMI are many. They include the NCQA, Joint Commission, Maternal and Child Health Bureau, AHRQ, CMS, CDC, Family Voices, Children Now, the American Academy of Pediatrics, the Harvard Pilgrim Health Plan and state officials from New York, Ohio, Massachusetts, Washington, and Florida among others.
CAMI operates under the administrative of and is staffed by the Foundation for Accountability. My comments today reflect input from both CAMI members and FACCT. I would like to begin by noting the CAMI and FACCT are very supportive of the purpose, design, and initial work towards progressing towards a national quality report.
The report is a bold and crucial step to increase public awareness about quality of care and create the basis for long term monitoring and improvement. We strongly support the use of the conceptual framework recommended by the IOM particularly because it anchors the assessment of quality to those aspects of care that are known to be most salient and relevant to consumers.
This consumer-oriented framework takes the report in a new and critically important direction. At the same time, it challenges all of us to select measures and reporting approaches that address the needs of patients and consumers, in particular, the needs of children and adolescents.
I suggest that for the National Healthcare Quality Report to be valued by public or Congress, we must first explicitly acknowledge the limitations of the initial available data and rapidly evolve to incorporate more responsive and relevant measures that those proposed here.
The history of health care measurement and development as led to the creation of numerous measures which have shown to be reliable and specific settings such as managed care, clinical trials, or for specific age or insurance groups but which have not been administered in nationally representative samples.
To understand and support the decision to make careful use of these measures including those drawn from HEDUS and Medicare chart audits. At the same time, the National Healthcare Quality Report simply reports on any good data which happens to be available, it may fail to fulfill the mandate of the IOM framework in inappropriately leading providers and others to focus their attention on what is now measured rather than what is important to the American public.
My comments today will address the scientific soundness, relevance of the permanent measure set, the robustness of the measures and the appropriateness of the proposed data sources. With respect to scientific soundness and relevance, we note and appreciate that the measures selected are sound measures for which a significant national database exist though a representative national data set is not always available.
Many of the measures are not relevant to the audience and goals intended, particularly children, adolescents and their parents. Children and adolescent health is unique characteristics differentiated from adult health and mandate the development of child specific measures.
Children pass rapidly through developmental stages of growth and this presents a challenge to measurement developers often requiring distinct measures for different age groups. For example, the screening test for developmental exposure of the hip will be different in newborns that it would be a 3-4 month old infant.
Children also depend on adults for access to care, consistency in the receipt of recommended treatment, and continuity of care over time. To adequately address the health of children, they must also address the health of parents and caretakers.
The work of CAMI and others has shown that parental well being is an important determinant of health and is seen by parents as essential to the assessment and quality of their child's health care. Children also have different patterns of health, illness, and disability.
They have fewer chronic conditions than adults to and with the exception of asthma, the number of children with particular chronic condition will be small. Quality measures for children with chronic illness require a non-categorical approach to achieve sufficient numbers for adequate assessment.
The use of screening instruments that identify children with chronic conditions such as the CAMI screen for children with special health care needs allows for quality measurement across large groups of children with different chronic conditions. We note that the only proposed measures specifically addressing health care provided to children and adolescents in the current national quality report are the measures and single items for hospitalization rates for gastroenteritis and asthma.
While these are important measures, they have barely begun to address the unique quality measurement needs of children. We have conducted over two dozen focus groups of parents and adolescents from all segments of society over the past six years. These focus groups are most concerned about the quality and interactions and education provided by their child's health care provider, particularly in such areas as counseling, guidance, and screening.
These are issues that are largely not addressed in the current draft of the report. The challenge this poses for the National Healthcare Quality Report will be to move beyond this preliminary measurement set to measures that are consumer relevant and derived directly from the experience of patients and families.
With respect to the robustness of the measures, the proposed set does not adequately address the range of demands recommended by the IOM framework particularly as they apply to children and families. We know that the preventive care and health promotion, the most common provider to children that often the most reliable and valid source of data is the patient or parent.
The almost exclusive reliance on preventive care events such as detailed immunization rates may lead to an inaccurate and incomplete judgments about the quality of preventive care provided for children in America today. Research has taught us that as a part of health promotion, parents seek information on sleep patterns, how to encourage learning and how to discipline their children.
Yet, fewer than half the families report receiving this care and greater than 50 percent report they would like more information on these topics. Quality measures such as CAMI's promoting healthy development survey needs data from parent to provide actual quality information for the clinician and information for the parent to better facilitate dialogues with their providers.
Similarly, for children with asthma, the use of measures such as asthma hospitalization rates does not provide actual information for the clinician or the health plan. However, the use of patient-based data as exemplified in the California Health Interview Survey, uses important information on symptom burden, missed opportunities for appropriate treatment and the identification of high-risk groups for poor asthma care.
We are concerned that narrowly specified process measures in this preliminary measurement set can lead to a teaching to the test rather than stimulating broad efforts at systemic improvement. Other measures are available today which are more relevant to consumers are actionable to improve care and can be analyzed for diverse populations and will provide a more robust assessment of care by the level of scientific rigor found in the measure selected.
With respect to the appropriateness of the data sources, for those measures that have been included in the report, the data measures are appropriate and valid for capturing quality of care information. However, the gaps and relevance in robustness noted can only be addressed by relying more heavily on the American public as the primary source of data.
This is particularly true in the area of child health, but with ample additional evidence from other domains such as adult chronic illness and end of life care. An over-reliance on clinical administrative data will fail to document quality in important demands such as health status and outcomes of the care that occur after the episode of care.
Patients and family members are unique sources for evaluating health care quality with respect to selected care processes, most outcomes, and certainly the experience of care. We recognize that many of the gaps and deficiencies in the proposed measurement set reflect the inconsistent state of measurement development in American health care and can not be corrected overnight or either within the first few years of the National Healthcare Quality Report's existence.
We are concerned that the first release of the report will set a de facto standard and expectation for further reports. It's absolutely critical that the gaps in the first report be seen as mandates for improved measurement specification and data collection.
To this end, we offer a few suggestions for both short term and long term improvements in the proposed report. With respect to short term improvements, the medical expenditure panel survey includes screening question which identify in each respondent family any resident children with a special healthcare need.
We recommend that all appropriate MEPS items such as timeliness of care and getting appointments as soon as parents want be stratified by the children's special care and health care needs status. The National Survey on Early Childhood Health and its measures of preventive and developmental care as well as its measures of patient experience should be concluded in the National Healthcare Quality Report.
This status attracts many of the Healthy People 2010 indicators as well care recommendations made by the AAP and the MCHB concerning smoking in the home, provision of anticipatory guidance and education of reading and injury prevention. We also propose that the National Healthcare Quality Report consider ambulatory care sensitive hospitalization measures beyond the two that are now included.
Scholarly researchers identified reliable elements for competing hospitalization rates for gastroenteritis, ENT problems, dehydration, and urinary tract infections. These composite rates for children aged 0-5 are robust enough to permit comparisons across racial groups in geographic areas, and may also be possible to develop measures such as admission rates for complications from clamydial disease that address prevention and effective care for adolescents.
With respect to long term improvements, the National Healthcare Quality Report should make explicit that quality health care for children is different than for adults. The number and natures of measures used in the report should reflect these differences.
There should be support for continuing testing and use of measures which address the needs of concern of parents and youth. Many widely used and high-quality measures exist, including the Picker surveys and CAMI surveys, early childhood development, children with chronic illness and adolescent risk behaviors.
The criteria used to review measures should be explicit and uniformly applied to all candidate measures. CAMI and FACCT are unequivocal supporters of the National Healthcare Quality Report effort. We observe that the work already performed is thoughtful, full, and rigorous. The very importance of this initiative heightens our concerns about the inevitable early gaps in content.
Paradoxically, a report which offers little new information and fails to speak to public concerns will find itself little used, valued, and unlikely to be funded in future years. For that reason, the publishers of this report should be candid and forceful about the very important gaps and data availability.
What is measured and reported will be understood by the public and by Congress as an answer to the question, what is quality health care. The report needs to make clear that the information we're presenting today is only a fragmentary picture of quality of health care in America.
We do know what the public wants to know and we must commit ourselves to developing the information infrastructure and will permit future reports to properly guide the improvement of the health system. Thank you.
DR. COLTIN: Thank you, Dr. Bergman. Any questions? Okay. We'll proceed to Walter Suarez.
DR. SUAREZ: Thank you. Madam Chair, Members of the Committee: I am Walter Suarez, Executive Director of the Minnesota Health Data Institute and Chair of the National Quality Committee of the National Association of Health Data Organizations. I want to thank you, on behalf of both the Institute and NAHDO, for the opportunity to offer testimony on the preliminary measure set being reviewed for use in the development of the National Healthcare Quality Report.
The Institute is a state public-private partnership dedicated to measuring and improving the quality and efficiency of health care services in Minnesota. We are one example of the many state health data organizations around the country working hard to gather and make available to consumer, purchasers, providers, plans and policymakers, comparative information about the quality of health care services delivered in our states.
The National Association of Health Data Organizations is a non-profit membership
organization created in the mid-80s that brings together state, federal, and private sector health data organizations to strengthen the nation's health information systems.
Over the past 17 years, NAHDO has served as a broker of expertise for state and national health systems development and to shape responsible health information policies. In early 2001, in response to the announcement of the development of the National Quality Report by AHRQ NAHDO established its National Quality Committee to advise the Agency on state-level implementation issues related to the measurement set to be used in the National Report. The Committee has been charged with: Representing state data organizations in the overall national quality dialogue;
Informing states on national quality report developments;
Proposing a collective strategy for promoting the incorporation and use of state-level information in the development of the National Report; and Defining the potential role of state health data organizations in regards to specific quality report recommendations.
Members of the NAHDO Quality Committee include state health data organization representatives from California, Hawaii, Massachusetts, Missouri, Minnesota, Pennsylvania, Texas, Utah, Wisconsin and Vermont. My testimony will focus on three major areas: Comments on the overall process and framework, comments on the proposed measurement set, addressing the questions posted by the conveners of this panel, and then finish up with some comments on the importance of the National Healthcare Quality Report for States and the need to include state-level data in the Report.
With respect to the overall process and framework, I want to start by praising AHRQ and its leadership team for the work they have done in developing and implementing a process to address what still seems to be an impossible task, providing an accurate picture of the state of the nation's health care quality.
We are very much in support of the direction and steps being taken by the Agency to create this important baseline measurement set for the first report. With respect to the conceptual framework of the report, while we recognize the limitations in terms of data availability, we still believe that the approach used by AHRQ to organize measures for specific conditions under the "Effectiveness" component of health care quality should be carried forward into the other components, Safety, Timeliness, Patient Centerdness.
For example, under "Safety" measures could be identified for complications of care for the core conditions being targeted by the report such as cancer, diabetes, heart disease, and so on.
Secondly, we believe the long-term care category under "Effectiveness" does not really reflect a health condition but a health care delivery setting. As such, we recommend to group it under other care delivery settings, such as hospitals, surgical centers, clinics, and others under the proposed alternative framework reporting.
In the process of identifying the final measures set for the first report, in addition to obtaining input from throughout the Department, we strongly recommend that AHRQ continue to reach outside the Department and engage more formally the data expertise available from health data organizations and other groups.
We will work hard over the coming months at our NAHDO National Quality Committee to provide input and expertise to AHRQ on the final selection of measures. We strongly believe that two other dimensions must be included, where possible, in this report, equity and efficiency.
We understand that AHRQ will be preparing a companion report on national healthcare disparities. But we still consider that, whenever possible, data on things like race/ethnic groups and other socio-demographic conditions should be reported in this National Quality Report.
We have similar thoughts about the efficiency dimension. We strongly support the inclusion in the report of measures that look at efficiency issues, such as resource consumption. With respect to reporting under alternative frameworks, we are also very supportive of the agency's intent to organize some of the data reported under the current framework around type of provider, hospitals, clinics, long-term care, and other settings.
Overall, we are very supportive of the selected priority health conditions for measurement and reporting cancer, chronic kidney disease, diabetes, heart disease, HIV/AIDS, Maternal and Child health, mental health and respiratory diseases.
As noted earlier, we recommend that the Long-term care category originally included in this list of conditions, be reported under an alternative framework by type of provider, along with other health care delivery settings.
We are also in agreement based on our best understanding of the description of each measure, with the draft list of measures included in the preliminary measure set. But we strongly recommend that the agency release soon a companion document to the preliminary measure set providing technical definitions of each measure numerator/denominator, or other as appropriate.
This detailed definition and description of the measure will allow for be are input by outside organizations and individuals who might not be familiar with how the source of data defines each measure. With respect to whether the preliminary measures meet the selection criteria of being valid, reliable, scientifically sound, and feasible, we believe that, in general, they all do. We did not see in the documentation what timeframe was expected to be used for the collection of the data.
We understand that in most cases, the timeframe will be whatever is most recently reported. That might create some confusion about what time is the report referring to. Thus, we recommend that a "timeliness" criteria be used in the evaluation of the final measure set. Perhaps data that is more than four years old should not be considered for inclusion.
Regarding gaps in the measure set, we believe that one of the areas to consider expanding even within the current preliminary measure set is child and Adolescent information. In some of the condition-specific areas, the focus is on the adult population. We believe that in areas such as respiratory disease, mental health, AIDS, and diabetes, data can be segregated for adults and children. We think grouping maternal and child data minimizes the reportability of data on children and adolescents.
Finally, in terms of appropriateness of data sources, we believe the identified data sources constitute at this point the best data available nationally on each of the areas. We recommend that, as the Agency gathers the data from all these various data sources, a list of identified data issues by source be created.
Importance of the National Healthcare Quality Report to States and Need to Include State-level Data.
Monitoring the delivery of high quality services continues to be a major priority for states. To the point that a number of them are now considering preparing their own State Healthcare Quality Report, following the steps of the National Report. That is why we believe that the National Healthcare Quality Report is going to serve states in a number of ways:
The national framework for quality measurement and reporting establishes the basis for states to begin producing a state version of the National Report. The Report will create a national reference data that will provide national benchmark data for states to use in their own state-level assessment and reporting of health care quality.
The National Report measure set will serve as a building block for states to expand on and advance state-level data collection, analysis and reporting. In this sense, states can dig deeper into specific topics for measuring and reporting health care quality.
Standardization of quality measurement sets for national and state reporting. In turn, we strongly believe that the impact of the National Healthcare Quality Report will significantly depend on its ability to deliver as much information as possible at the state level.
For every state, knowing that the nation's rate of mammogram within the past 2 years is X will be important. But it will be of much more significance to know (and here we would argue for both the states as well as for national policymakers), the mammography rate by state. This will I allow both states and national agencies and organizations to better target their resources.
We are very pleased to see that AHRQ has now identified in the preliminary measure set whether a state database exist for each measure. We believe one of the next steps is to ensure that the data to be reported at the state level meet one additional criterium: comparability.
We believe that in some instances there will be a need to adjust the data reported at the state level to be representative of the state's population demographic, socioeconomic, and others. For example, some of the Patient Centerdness measures come from surveys done on specific state populations Medicare, Medicaid, commercial populations.
These populations are not representative of the overall population of the state, and when reported state-by-state, the state comparisons will not be valid. We recommend that the agency use the Population Reference Sets developed by the Bureau of the Census for each state to adjust state-level data and make it reportable in the National Report.
In summary, we believe the efforts undertaken by the Agency to prepare and publish this report are commendable. The opportunity to continue to offer comments along the way, and in particular as the measure set gets finalized will be critical to the success and acceptability of the report. NAHDO's National Quality Committee will reconvene over the next several weeks and continue to work hard to provide more direct input to the Agency on the state components of the National Quality Report. I thank you again for the opportunity to provide this testimony.
DR. COLTIN: Thank, Dr. Suarez. Any other comments related to these presenters? Thank you. Now we are scheduled to move into the public comment period. Those of you who are in the audience might be interested in making public comments, I would ask that you sign the list that is at the table at the door so we have a record of who has commented. Then we will ask you to come to the table one at a time to make your comments. Do we have anyone signed up at this time? No.
We had publicized that we wouldn't begin the public comment period until 2:30. In the event that there are some organizations that are preparing to come at that time, what we can do is take a break at this point and reconvene at 2:30 and see if there is anyone here and then Tom and I have agreed to stay around and await anyone who might show up.
I will thank those who have commented already. Unless anyone has any additional comments that they would like to make, we'll take a break. Jeff, did you have something you wanted to say.
DR. REILLY: I just wanted to say I'd like to add my thanks to the panelists for their comments and thoughtful consideration of our questions. Thank you.
DR. COLTIN: Then we are on break to reconvene at 2:30.
(Short break at 1:50pm.)
DR. COLTIN: I'm going to adjourn the official component of the meeting, but Tom Reilly and I will stay around for the duration of the public comment period in the event that anyone does come and wish to offer comment on the report. Thank you.
(The meeting was adjourned at 2:35pm)