This Web-assisted audio teleconference series consisted of three events broadcast on July 25, 26, and August 1, 2001, via the World Wide Web and telephone. The program was designed to help State and local policymakers make policy decisions and allocate resources related to health care informatics. The User Liaison Program (ULP) of the Agency for Healthcare Research and Quality (AHRQ) developed and sponsored the program.
Cindy DiBiasi: This is a series of three Web-assisted teleconferences for State and local health policymakers sponsored under the User Liaison Program under AHRQ, the Federal Agency for Healthcare Research and Quality. My name is Cindy DiBiasi and I will be your moderator for today's session.
This is the third event of this User Liaison Program Web-assisted teleconference series on the “Role of Informatics in Improving Healthcare.” The healthcare field has the potential to benefit greatly from innovative applications of informatics. Everything from online access to health information, integrated electronic medical records, and computer-based information systems to provide practitioners with real-time assistance in their decision-making can significantly improve the quality of care and patient safety. It is important to separate the facts from the hype and better understand how information technology can be used to improve the delivery of healthcare.
Today's event will address “Getting Information into the Hands of Decision- Makers: Innovative Applications and Issues.” This Web-assisted teleconference will examine two related and innovative approaches to using informatics to make data, in this case, hospital discharge data, more readily available to policymakers and researchers. The goal is to support insightful, rapid turnaround, comparative analysis, both within and across States. We will also examine the results of a recent study of the accessibility and quality of health information available to consumers on the Internet and we will look at their implications for public policymakers.
On July 25 during the first Web-assisted teleconference in this series, “The Potential Impact of Clinical Informatics on Healthcare Costs, Quality, and Safety,” we examined the potential impact of specific clinical informatics interventions on the cost, quality, and safety of healthcare services. We also discussed the implications of developments in the areas of health informatics for State and local governments.
On July 26, we held the second Web-assisted teleconference in this series, “Using Informatics to Improve Program Performance: Examples of Innovative State Applications.” We looked at how information technology is being used within State-sponsored healthcare programs to improve access, enhance the quality and appropriateness, and reduce the cost of healthcare provided to program beneficiaries.
Today we are going to take a closer look at “Getting Information Into the Hands of Decision-Makers: Innovative Applications and Issues.” In the studio with me I have three experts who will be participating in our discussion.
Dr. Anne Elixhauser is a Senior Research Scientist with the Agency for Healthcare Research and Quality of the U.S. Department of Health and Human Services. Pete Bailey is Chief of Health and Demographics at the South Carolina State Budget and Control Board. Dr. Gretchen Berland is a Robert Wood Johnson Clinical Scholar and Grant Help Associate. Welcome everyone.
Before we begin our discussion, I do have a few housekeeping items to take care of. If at any point during this event you have Web-related technical difficulties, please use the “Tell” function to contact Tech Support. Also, if at any point in time you experience difficulty with the audio screen or if you experience an uncomfortable lag time between the streamed audio and slide presentation, please feel free to access the audio by your phone at 1-888-868-9080 and give the password “AHRQ Teleconference.”
Later in the call, our fine panel of experts will also be taking your questions. There are four ways you can communicate your questions to us. If you are on the phone, please listen for my instructions later and dial “14.” You may fax us your question at (301) 594-0380. You may E-mail us your question at info@ahrq.gov. And you may also directly type your question into the messaging field and hit “enter.” If you prefer not to use your name when you communicate with us, that is fine. We would like to know what State you are from and the name of your department or organization. Please indicate that regardless of the way in which you transmit your question.
We will have several audiotapes of this Web-assisted teleconference series available for purchase several weeks from now. I will be giving you further details about this at the end of today's show.
Finally, an archive of this Web-assisted teleconference will be available on the AHRQ ULP Web site. The URL is http://www.ahrq.gov/news/ulpix.htm. Now I think we are ready to turn to the important matter of “Getting Information into the Hands of Decision-Makers: Innovative Applications and Issues.”
Anne, I would like to start with you. Let's talk first about HCUP and HCUPnet. First, what is HCUP?
Dr. Anne Elixhauser: Cindy, HCUP is an acronym for the Healthcare Cost and Utilization Project. Basically, what HCUP consists of is data, tools for research and research reports. HCUP is a partnership between States, industry and AHRQ. What happens is that State data organizations and hospital associations collect hospital discharge data or administrative data from all the hospitals in their State. Currently, we have 24 States that are in the partnership and they provide their data to AHRQ.
We convert all their data into uniform format because, of course, every State has a different way of formatting their information and all their data. Currently, we have got over 60 percent of all hospital discharge summaries. Sixty percent of all hospital data is in HCUP.
We then make these databases available for research purposes. We also do analysis, we provide reports. Previously we have only made hard copy reports available, but then we realized that we could make the information much more available through other media like the Web.
Cindy DiBiasi: What is AHRQ doing to use developments in informatics technology to make the information more accessible and user-friendly for both policymakers and decisionmakers?
Dr. Anne Elixhauser: As I mentioned before, what we used to do is just make published reports available based on HCUP data. We realized a couple of years ago that we could provide a lot more information a lot more quickly through the Internet. So we went online about two years ago with a very simple format with a single database and we called it HCUPnet. All the changes that we have done on HCUPnet since then have been in response to user suggestions.
HCUPnet is a query system that provides detailed information on hospital statistics. It provides that information at the national level, the regional level and for some States that have agreed to participate, at the State level as well. It provides more information online than was ever before available. Plus, HCUPnet is really targeted to the non-researcher because it allows access to very detailed statistics about hospital data without requiring any special experience or any training.
Before HCUPnet, if you wanted this kind of information, you really would have to go out and buy the data. You would have to buy statistical software to get at the data, which would cost $1,000 or more. You would have to learn how to use the software and that is where the real expense comes in because it is fairly complicated. Then you are still not sure that you are doing it right and getting the right numbers. With HCUPnet, the information is right there at your fingertips.
You might call HCUPnet the great equalizer because information that formerly required a fair amount of statistical expertise is now available online to anybody who can use the Web.
Cindy DiBiasi: And who exactly is using HCUPnet and how are they using it?
Dr. Anne Elixhauser: HCUPnet is actually used in a lot of different ways. I have heard reports of, for example, a university professor who uses HCUPnet to teach his students how to think about research, how to frame research questions to get what they want out of a data set. Recently I spoke with a woman whose child was severely brain damaged because of high levels of bilirubin after birth. She is now spearheading a national campaign to start screening all infants routinely for bilirubin. She called our agency and wanted to get some statistics on how many babies this might affect. I showed her how she could get this information out of HCUPnet.
Basically, any time that you want information on hospital stays in the U.S., it is likely that HCUPnet could help you or at least get you started.
You could use HCUPnet to look at questions like, what are the most costly conditions that are treated in U.S. hospitals? HCUPnet will rank-order those for you. Let's say that you have some information in your own State that you would like to compare to some benchmarks and HCUPnet can provide you with national or regional averages. You can also compare your State to another State.
As I mentioned before, HCUP is a partnership and it is completely voluntary. Ten of the HCUP States have agreed to put their data in HCUPnet. It is likely that you will find a State that could provide a reasonable comparison to your own. HCUPnet basically lets you look at specific types of hospitals and specific types of patients.
Cindy DiBiasi: Why don't you show us, take us through it. It would be a better idea of how this all works.
Dr. Anne Elixhauser: OK. Sure. The first screen that you would see whenever you enter HCUPnet is “Welcome to HCUPnet.” This provides you with some links to other information about HCUP, but what you would do is you would hit the little blue button that says “Start HCUPnet.” That takes you to the database selection screen. This gives you an overview of all the databases that are available on HCUPnet. Like I said earlier, when we first started all we had available was national statistics. Since then we have expanded it to include a number of other databases. You can see those right there.
Instant Tables, for example, provides you with the kind of information that we used to provide just in hard copy publications. The most frequently asked-for type of information. The National Statistics allow you to get access to our Nationwide Inpatient Sample, what we call the NIS. This is a sample of about seven million discharge records that is taken from all the HCUP States. It is weighted to give you national estimates. The tab that says “Hospital Stays for Children Only”, that gives you access to a very special database that we just created that looks only at children's hospitalizations. It is the only database of its kind. It contains nearly two million hospital stays for children, and again, will give you national estimates.
The “Trends” tab will give information from '93 to '97 from the Nationwide Inpatient Sample. That is soon to expand to '98 and '99 data as well. On the “State Statistics” tab, it gives you detailed statistics from ten States who have agreed to participate. For States with comparable Web sites, we also have provided direct links to their Web site.
Why don't we go through a simple query using National Statistics? If you would then go to “National Statistics”, the first thing that you are asked to do is to decide what kind of information you want. Do you want information on all hospital stays or do you want information on specific diagnoses or specific procedures? In this example, we are going to pick “Procedures.” Then you are asked if you want to rank-order those procedures or if you want to choose specific ones. In this example, we are just going to choose a specific procedure. You would go to the next page after pressing “Next.”
Let's say that you want information on heart valve procedures. We choose here “Procedures on the Cardiovascular System”, and then if you click it again, you would then choose “Heart Valve Procedures.” Then you choose what kind of information you want. Do you want length of stay? Do you want total charges? Do you want in-hospital deaths? Do you want discharge data such as to another institution or home health care? Do you want aggregate charges, which is sort of the national bill for this procedure? Or do you just want the number of discharges?
In this example, let's say we are just interested in hospital deaths so we would click that one. Then you would go to the next screen and then you have an option of looking at statistics by certain patient characteristics. You could look at it by age group; you can look at it by sex, by primary payer including the uninsured here, and also look at it by the income. This is income from the patient's zip code. Let's say in this example that we just want to look at all patients. We don't click anything. Then you are given the option of looking at specific types of hospitals. You can look at hospitals by their ownership, by their teaching status, by whether they are located in a rural or urban area, by their bed size, either small, medium and large, or by the region of the country.
Let's say in this example all we want to look at is just by ownership and teaching status. So what we have done here is we are looking at in-hospital mortality for heart valve procedures by ownership and teaching status of the hospital. Then about that quickly, you would get to the results screen. You can see here that for for-profit hospitals who are non-teaching, they happen to have the highest death rate of about 8.7 percent compared with a little over 6 percent in not-for-profit hospitals. These numbers are not risk-adjusted, but they do provide you with a starting point.
Cindy DiBiasi: We are going to come back to you because I know there are going to be a lot of questions. I want to move now to Pete Bailey from South Carolina. Pete, your agency, the South Carolina State Budgeting Control Board is a member of HCUP. What benefit does your State realize from being a member of consortium?
Pete Bailey: It really has been a very valuable membership and it has been really nice to be a part of helping to create the National Data System and also to have access to national data because you need that kind of comparison between this State and other States. I think it is also fantastic to have access to, in my opinion, a very fantastic professional staff and our meetings with HCUP and around the staff. I can tell you it has been exciting to me to see the team that exists there.
They really do work well together and they are excited about what they do. Having access to that type of professional staff is really important. At the national meetings, there is a lot of brainstorming going on so you are in an environment with other States along with HCUP staff. A lot of research is discussed so you are on the front end of what is going on with research. Plus you are a part of it and a lot of it can be applicable to you at your State so you can carry it back home and do it at your State level if it is not done that way.
Then I think having HCUP gives you this national perspective to help. Many States have problems that are similar problems and having HCUP national gives you that ability to have someone help you work on those problems. Like for example, many States don't have military V.A. data and at the last meeting we talked about the fact that we should do that together. HCUP could be a lot of help. Same way in terms of say, South Carolina loses a lot of, its residents may go to North Carolina or Georgia, we don't know about those. Sooner or later as we get this whole national system built, that sharing will be really important. Of course, to overcome these types of sharing and that gives you the ability to do utilization rates so you know you have all your population.
Cindy DiBiasi: Specifically, how have you used HCUP data to do informed decision-making in your State?
Pete Bailey: Well, as I mentioned earlier, of course and I won't emphasize again, having national and State comparisons I cannot say enough about that, how important that is. As you have seen with Anne's presentation, now how quickly you can get that kind of information.
One of the most unusual examples that I have thought of in terms of using HCUP data involved another State. South Carolina several years ago, we are not a heavily managed care State, I think we have less than 20 percent of people in managed care. At that time, the environment was we are going heavily into managed care. So, of course the State got very, a lot of people got nervous and excited, what does it mean for us to go heavily into managed care? At that time, our Office of Research and Statistics, we had the responsibility to do demand and supply projections for physicians.
Of course the question comes up, if we go heavily into managed care, what will it mean in terms of demand? So, we were able to access another State's data that was heavily into managed care, a portion of that State that was almost completely managed care. Then a little more data than what was at HCUP because we actually got physicians' specialties. Then we could look under that State where there was strong managed care going on, look at their hospitalization use rate, their specialty. Look at it by the different groupings in the hospital like OB and the different surgical groups and the patient days. It was sort of like having a crystal ball because if you are going into managed care, you may look like them.
It is going to heavily affect your demands. It was fantastic to be able to have another State's data that has gone down the road that you might be going. A similar example could be used around disparities. Doing relative ratios, say, we are a heavily African-American and white population. If you looked at hospitalization rates for both of those and then looked at relative ratios, you would see those problems where we have got major disparity problems. It would be really interesting to do that in other States to see if your disparity problems are similar. And if not, why?
Cindy DiBiasi: You know they say imitation is the sincerest form of flattery. I understand that you have developed a South Carolina-specific Internet application that is somewhat similar to HCUPnet that allows individual providers to conduct customized analyses of their own hospital utilization and cost experience. Can you tell us a little bit about that?
Pete Bailey: We were really, really excited about this effort. South Carolina is a little different in that we are strictly a research and statistics organization in our data. As such, we have relationships, strong relationships with both the private and public side. We have a fantastic relationship with the South Carolina Medical Association. They actually gave us financial support to develop this secure Web site for physicians in South Carolina. It is PIN number, password protected. What we are able to do is put all of the inpatient hospital data out there so that a physician then could access his data and see how he compares with others. It is a wonderful thing to be able to do.
Cindy DiBiasi: How about taking us through an example of the type of analyses that could be done?
Pete Bailey: OK. The first thing, of course, when he comes on to the site, he has to do his PIN number and password. Once he gets past that, it pulls him into another screen. Let me say immediately that the way we developed this Web site was that we made up the numbers. The PIN numbers and passwords are made up. They are not connected to a physician in any way.
There is nothing on this Web site that is connected to a physician in any way so if someone broke in, they would be in a sea of numbers and not be able to get any information. So, on the first screen, there are several tabs up there where they can click on those tabs and look about, find out information about the site and the source of data and data quality and medical records coding and technical notes. The final tab which says “Your Data” is where he would click. He has a choice to look at the latest one year or then we put three years of data out there because small numbers are an issue. We are trying to overcome that.
Say he clicks on the latest three years, then what happens to him next thing he sees in terms of the screen is his top ten APDRG's for his practice. It shows him his number of patients, your patients and the percent of his patients by severity level. Then it shows Statewide numbers and the percent of patients by severity levels so he can immediately begin to see at the APDRG level if he is treating more severe patients than others.
So he makes a selection of the APDRG and then he can then select or go to another screen. I should have said that on that previous screen he selected (unclear) and delivery.
Cindy DiBiasi: Before you move forward, why don't you give us a little clarity on what APDRG is?
Pete Bailey: All Persons Defined Diagnostic Related Groups. It is pretty common, I think a lot of people understand that. Once he selects on the APDRG's, he is able to then, and you will see on the next screen the ability to select down to the ICD-9 level, the International Classification of Disease. He is able to see his comparisons there in terms of these ICD-9 codes and in terms of severity and whether or not he is treating more severe patients than others. In this particular case we selected low-cervical c-sections.
Then what pops out for him on the screen is of course, that particular ICD-9 code and the severity. So his final choice is to select what severity that he would like to look at. I think in this particular case we checked moderate in severity. That allows him to sort of go through and pick what is important to him in terms of what he wants to look at. Then his choices are to be able to look at length of stay, or discharge status and post-discharge events, resource use, and complications of care.
If he clicks on “Length of Stay”, then he begins to have several choices there. He can look at the average length of stay, he can look at length of stay (unclear), that is standard deviation, pre-surgical days, post-surgical days. He can look at the ICU [intensive care unit] and CCU [coronary care unit] use and days and progressive care use and days.
The other selection he can make, of course, would be the discharge status and post-discharge event comparison. If he made that selection, his choices are discharge status, home or discharge status to home help, or transferred to acute hospital or transferred to a non-acute hospital or transferred to a nursing home, or death in the hospital. Now, I might say there that we have the ability to link to vital records and have great relationships with the Vital Records Office. We eventually want to have deaths like within 30 days after hospitalization. Then he can check on re-admission as an inpatient within 30 days of discharge or visits to the ER [emergency room] within seven days of discharge. Those are two really good outcomes.
The next broad choice that I mentioned that he can do has to do with resource use. Basically, I have listed some of the billing categories that he can select also. He can go down and select any of those if he might want to look at comparative data. Finally, the complication of care comparisons. This is a great place where we have used HCUP. Their work on quality has been just tremendously useful to us. If we had had to do the research that they have done, it would have killed us.
We have depended heavily on the work that they have done. You will see on this particular slide, the complications of care. There are two pages on this. On the second page of those complications of care, you will see the obstetric indicators. We are really excited about this area, both from an OB and a pediatrician's standpoint. We think that as we link with Vital Records, we could really expand the usefulness of this kind of information. We are working with the Department of Health and Environmental Control over a Universal Screening Form for mothers. All of this eventually can be built in and expanded to really fantastic information.
Finally, depending on what he selects, the next screen then shows you the output that occurs that he would see on any of those indicators that he checked. If you take a look at this, this happens to be discharge visits to the ER within seven days of discharge. This particular physician you see had 11 patients and it turned out to be 5.5 percent of his patients. He can compare himself Statewide where it is 2.2 percent and see that is something that he might want to look at. He can see how he compares with a peer group and that is a hospital peer group based on bed size and the amount of physician specialty codes. He can compare himself with a geographical group.
Then we did comparisons of physicians that are at the 25th, 50th, and 75th percentiles. Again, looking at the physicians at the 75th percentile, their visits to the ER within seven days is 3.6 percent so you can this guy has to look at his data.
Cindy DiBiasi: Obviously, you are sold on the importance of this data. How have providers responded in your State?
Pete Bailey: It has really been fantastic. I think that this whole process is baby steps. I think you make a baby step and you learn a lot and you see greater potential. The physicians I think were really excited about the fact that someone cared enough about them to put all of their hospital data out there and that for the first time they have information. In most cases, there is not a level playing field. Payers come after them. They accuse them of things or they think they don't have the information to look. It is a fantastic thing to be able to do.
There was a lot of excitement about this. They were really pleased that we could do this. Immediately, the response was, “I want more! I want my office visit data out there, I want to know how that is related to hospitalization.” It really was exciting to hear their interest. The other thing that we learned that for a sizeable group of physicians, this was not useful because they didn't have that many hospitalizations. All numbers was a big problem. We did not even send it out to physicians unless they had ten hospitalizations. We felt it would be frustrating to have that small number. So we learned a lot too, that we do have to have more information out there.
The other thing that we learned is that OK, so you looked at your data for a particular quarter and you see, OK, what do you do next? There is no need to go back. There is nothing of interest to you if we don't have the ability for them to research or ask more questions. We know that we want to do that. We do have a public Web site that has query capability on both inpatient hospital data and emergency department data. We would do one differently for physicians and plan to have more query capability.
The other interesting thing is that it tied us really well to the medical schools. They want to base their whole continuing medical education program on this system because you can see where the problems are. It is a really great thing. We are making progress in that arena. We had even a lot of discussion concerning using this Web site to evaluate residency programs.
Cindy DiBiasi: We will be getting back to you because we have a lot of questions regarding this, but I do want to move on to Dr. Gretchen Berland. So far in the call we have focused primarily on applications using the Internet to get information into the hands of key decision-makers and policymakers and providers. There is another important group of decision-makers that we need to consider. Those are consumers who are increasingly looking to the net as an important source of health-related information. However, a recent study conducted by the Rand Corporation raises questions about the quality, accessibility and understandability of health information found on the Internet.
Gretchen, you were the lead author on that study. What was the impetus for the study?
Dr. Gretchen Berland: Well, Cindy, I think it is important to point out that the Internet is a big place. We know that in the year 2000, it was estimated that consumers could go to an estimated 25,000 health-related Web sites and look for information about health topics of interest. And those 25,000 Web sites offered nearly a billion pages of health information. We know it is a really big place. We also know that it is a busy place.
As you can see, according to Harris Interactives, in the last two years the number of individuals who have gone online to look for health information has doubled from 54 million to nearly 100 million people who report in the United States that they have gone to the Web for health information. We know that it is a big place and it is a busy place and it has attracted a lot of attention because of this. Many optimists write that access to health information on the Internet will really transform the doctor-patient relationship and transform healthcare.
Access to health information will motivate consumers to participate more in their care. There are also people who are concerned about the quality of health information on the Internet and worried that the information is incomplete and inaccurate and misleading and potentially harmful. That is sort of the size of, in terms of the vast size of the Internet, there has been a lot of concern written about the Web. Really to date, not a lot of studies have been conducted to really quantify some of the opinions that we have seen in the literature. Little is known about the accessibility, quality, and readability of some of the information you might find.
Cindy DiBiasi: Describe how you did the study.
Dr. Gretchen Berland: OK. In July of 2000, Rand got a grant from the California HealthCare Foundation. What we did was really broke the study down into three parts. The way to think about it, the first part of the study really says, ok, how easy is it for me to find health information on the Internet using search engines? That is the first part of our study.
The second part of our study says, well, if I land on information on the Internet, you can think of the Internet as a big swimming pool. You jump off one diving board and you land at a place. Well, where I land, is that comprehensive information, is that accurate information? That is related to the second part of our study.
The third part of our study really says, OK, well, if the information is there, is it going to useful for me? Am I going to be able to read this information? Am I going to be able to understand it? So that relates to the last question we ask which is, “What is the level of literacy required to understand information provided by these sites?”
Our first task really for the first part of the study was to pick, we know that there are thousands of health-related conditions.
We picked four for starters. Breast cancer, childhood asthma, depression and obesity. We picked those four for different reasons. We know that cancer is one of the most common reasons people go to the Web to look for health information. Childhood asthma because we know that unfortunately, it is increasing in prevalence. Parents often go to the Web to look for information about their children. Mental health is another; 20 percent of all users who go to the Web to look for health information look for information about mental health because there is often stigma associated with mental health illnesses. You can look for health information in the privacy of your own room. Obesity actually recently, according to (unclear) dialog, is the most popular reason people go to the Web to look for health information.
So we picked those four conditions. We then picked ten English language search engines and four Spanish language search engines and conducted standardized searches using simple search terms for each of those conditions and categorized the results of those searches.
Cindy DiBiasi: What did you find?
Dr. Gretchen Berland: What we found, what you can see here really shows that what we spent six months doing probably are what many people spend in the first five seconds searching when you go to your search engine. For those of you out there now, you can probably go to your search engine of interest and type in the words “breast cancer” and what we categorized over several months is really what people did in five seconds or less.
If you look here, what we did was we entered the search term and we then said, “Ok, well where do you get taken?” You either get taken to, we categorized that by counting the number of links. Do links take you to contents or more links? Are the links relevant? As you can see that about a third of the time the links that people follow to look for health information are just relevant. Then you say, “Well, does the link take me to health information? Does it take me to textual information?”
Well, we found that overall right now, finding health information on the Internet is somewhat of an obstacle course. Your chances of finding health information using English language search engines is about one in five. Your chances of finding health information using a Spanish language search engine are about one in eight. That means you are going to have to do a fair amount of sifting just to find information that is related to your topic of interest. So, the first part of our study.