Previous
Section Contents
Next
Section 
Outcome Measures for Home Healthcare
Dr. Shaughnessy characterized OASIS as a dataset and a means to an end. Many people were involved in its development, including 1,000 to 2,000 representatives of the homecare field.
It is important to recognize the unique features of homecare and how they relate to quality. The homecare environment is different from most other healthcare environments in that the provider is a "guest" of the consumer. This type of care is characterized by multidisciplinary care coordination, yet the team does not work together in the environment where care is actually provided. Functional outcomes in homecare were determined to be very important by patients, consumers, and providers. Every panel convened emphasized the importance of enabling the patient to function as well as possible to avoid institutionalization and maximize quality of life. Defining the episode of care and outcome interval has been challenging and always will be, in part because addressing intervening hospitalizations is difficult. Also, it is more difficult to monitor adherence to treatment and quality in homecare.
The development of OASIS began with an evaluation of quality measurement approaches for home healthcare and included an ongoing literature review, provider surveys, and consumer and provider input. The investigators considered measuring quality through outcomes, process, and structure, however, they reached a consensus that outcome measures are most appropriate for homecare, primarily because the purpose of this care is to effect patient well-being. Measuring outcomes provides information on the degree to which the system's purpose has been achieved.
The researchers initially considered an expansive set of 500-700 different outcome measures and narrowed these down based on the criteria of usefulness and practicality for clinicians, benefits to patients, measurability, and scientific issues. Several evaluation projects used these measures, for example, to study outcomes of care under fee-for-service versus health maintenance organization (HMO) plans.
Patient outcome was defined as a change in health status between two or more time points, generally the start of care and discharge from care. Because a great deal can happen between these two time points, Dr. Shaughnessy and his colleagues collected data on more frequent intervals. Yet they found that very little is lost by focusing only on the end points. The researchers tested several kinds of outcome measures. They selected measures that assess a change in health status and utilization outcome, because these make a great deal of sense to clinicians, they can be expressed as rates for outcome reporting, and dichotomies within them correlate strongly with more sophisticated or aggregated measures. These measures provide a relatively straightforward, easily understandable baseline for improvement.
Length of stay presents a unique dilemma in this field and should be taken into consideration as a risk factor. Over the period when homecare is provided, the patient's health status will change whether the individual receives healthcare or not. But duration of care is highly correlated with quality of care. The challenge is to identify the percentage of the outcome that is attributable to the natural progression of disease and disability. The remaining percentage can be attributed to the care provided over the interval. Risk adjustment can help approximate this.
William Golden, M.D., F.A.C.P., pointed out that the Home Health Prospective Payment System (PPS) came along as the OASIS measures were being developed. He asked about the impact of PPS on the conceptual framework. Dr. Shaughnessy replied that the Outcome Based Quality Improvement (OBQI) applications framework remained intact and independent of payment processes. The behavior of agencies was influenced by PPS, but the Interim Payment System (IPS) had a greater impact on OASIS than did the PPS. Once it became clear that resources would be more restrained, the reasons for collecting additional data were questioned. Dr. Golden asked whether the investigators ascertained that OASIS is relevant to the products that exist today. Dr. Shaughnessy promised to address this.
Ms. Terry asked about the identity of the specific risk adjusters for each outcome. Dr. Shaughnessy replied that the workbook includes only the number of risk factors for each outcome. Dr. Shaughnessy was reluctant to share the risk factors, because they are evolving. However, this evolution will not have a significant impact on the overall performance of the OASIS measures. Phyllis Fredland agreed that panel members need to know whether the appropriate risk factors were included to determine whether a measure will be useful. Given that so many risk factors are included, Dr. Bartlett suggested that Dr. Shaughnessy and his colleagues discuss particular risk factors when the panel addresses individual measures.
Dr. Shaughnessy explained that OBQI was meant to provide information to clinicians that they can use to enhance their patients' outcomes. This has happened. The OBQI report enables agencies to compare their performances on the 41 risk-adjusted outcomes to that of other agencies throughout the country and to their own performances in the preceding year.
The OBQI demonstrations were designed to determine whether OBQI works and whether it works relative to PPS. The national and New York State demonstrations took place between 1995 and 2001, and involved 54 (19 in New York State) agencies; several rounds of data collection, outcome reporting, and outcome enhancement; and 150,000 patients (100,000 in New York State). Participating agencies were asked to focus on specific target outcomes and reinforce or improve the care behaviors that produce them. The agencies in the demonstrations were advised to change no more than two outcomes to avoid overly diluting their energies. All of the demonstration agencies were also asked to select hospitalization as one target outcome.
The demonstrations produced favorable results. Hospitalization rates in the national demonstration sites decreased from 33 percent in Year 1 to 25.3 percent in Year 3. The hospitalization rates were relatively stable across participating states during this same period.
In the five-state Quality Improvement Organization (Q.I.O.) demonstration conducted since the advent of IPS and PPS, the results were similar to those of the national demonstration. Participating agencies improved their target outcomes by 6.7 percent but showed a modest decline in comparison outcomes. These agencies received less intensive technical assistance than those in the national demonstration project. The results reflect the validation of the OBQI process, which is effective even under the serious constraints currently imposed on homecare.
What can be done with the OASIS system is remarkable. Hundreds of providers have shown that they can make pronounced differences on behalf of their patients. The system has worked with hundreds of agencies and hundreds of thousands of patients.
In addition to the 41 measures, the OASIS dataset produces 13 adverse-event outcomes that have been useful for agencies. These cover events that occur infrequently. They should not be the basis of risk-adjusted outcome reports, because risk-adjusting events of such low frequency is almost impossible. Dr. Shaughnessy and his colleagues therefore commented that these outcomes should not be included with the 41 outcome measures.
Every outcome has a logistic regression model, with patients as the unit of analysis. Regression models were also estimated with agencies as the unit of analysis, although it is the patient-level models that are currently used for risk adjustment. The results of both of models were presented in the meeting workbook. Each patient-level outcome model has 10-60 risk factors. Each risk model was based on approximately 500,000 developed and then applied to another one million cases to see how well it performed. Every outcome measure applies to a specific domain for patients. For example, improvement in grooming applies only to patients who can improve in grooming.
Dr. Murtaugh wondered whether it would be simpler to use a single set of risk adjusters for all measures. Dr. Shaughnessy and his colleagues recently tried to identify common sets of risk adjusters. However, no common set explains variation as well as sets of risk adjusters that are specific to each outcome.
Consumer Testing
Margaret Gerteis, Ph.D., pointed out that consumer testing addresses most of the issues about importance and meaningfulness to consumers, and consumer perceptions of scientific validity. This aspect of face validity is important for reporting.
BearingPoint tested consumer perceptions of home health quality and variations in quality, the importance of OASIS measures, and their understanding of OASIS measures. BearingPoint also probed their responses to the public release of OASIS measures and whether they would seek and use this information.
One of the challenges was the need to obtain feedback on 54 measures. The investigators organized the measures into categories and translated them into plain language. They then conducted 31 in-depth interviews with Medicare beneficiaries and informal caregivers in Baltimore and Tampa, including people with and without homecare experience. They conducted focus groups with 26 hospital discharge planners and in-depth interviews with 24 generalist and specialist physicians.
Consumers were unaware of the scope and overall purpose of home health services. They also had a very limited understanding of choices and limited involvement in the selection process. They relied almost completely on hospital discharge planners (most had experienced homecare after leaving the hospital). Consumers judged quality (before seeing the OASIS measures) based mostly on personal traits of caregivers, not on agency performance. They had difficulty understanding that agency performance could be a concern.
Most physicians saw little difference in quality among agencies and attributed differences more to individual nurses than to agencies. Discharge planners were more likely to perceive differences among agencies based on prior experience, such as how well these agencies succeeded in keeping patients out of the hospital. Both physicians and discharge planners tended to judge quality based on their own personal experiences and feedback from patients.
Physicians generally relied on discharge planners to select homecare agencies for patients who were discharged from the hospital. In a small number of cases, families would request advice about homecare for family members. For both physicians and discharge planners, the most important factors in selecting an agency were having nurses who could see the patient when needed, offering the needed skills and services, providing coverage under the patient's health plan or insurance, and providing care in the patient's language.
In general, discharge planners were aware that they were not supposed to recommend specific agencies, and they tried to offer patients a choice. Some wanted to recommend one agency over another and tried to steer consumers toward that agency. Physicians often recommended specific agencies.
All three groups tended to focus on measures clearly linked to patient safety, health, and independence. Measures related to grooming and housekeeping were perceived by all groups as less important because they are less essential to health and because friends and family can often assist with these. The perceived importance among all three groups of specific measures varied according to a particular patient's needs and situation. Family caregivers and physicians felt that incontinence and mental health measures were important because they are usually the source of family caregiver burnout. Addressing them could help maintain the family's ability to care for the patient.
Consumers and professionals understood that the measures reflected desirable outcomes for a family member, but they had difficulty understanding how those measures related to the quality performance of an agency. Consumers tended to discount adverse outcomes, because they viewed patient death or hospitalization as obviously undesirable. They viewed discharge to the community as a desirable outcome, but did not understand how it could be used as a measure of quality. Consumers also questioned whether agencies could affect all of the outcomes measured and did not understand the stabilization measures. Because the desired outcome was improvement, consumers did not understand why staying the same mattered. They were confused by the behavioral health measures, because they perceived them as relating to dementia, which, they believed, meant that the patient should not be cared for in the home.
Consumers and professionals questioned the fairness of the comparisons, given the differences in case mix among agencies. Professionals were more likely to accept the validity of measures that could be objectively measured and questioned the validity of subjective measures. Professionals expressed concerns about reliability, given agency self-reporting.
Discussion
Ms. Clark asked whether the consumer cohort included a mix of persons. Dr. Gerteis replied that the consumers varied in their level of education, income, and ethnicity. The testing included a mix of beneficiaries, caregivers, and people who did and did not have homecare experience.
Dr. Golden asked whether the researchers solicited reactions to OASIS measures alone or asked open-ended questions. Dr. Gerteis replied that they asked open-ended questions about respondents' experiences with home healthcare, looked at the OASIS measures by category, and asked what else respondents would like to see.
Dr. Gerteis said that she and her colleagues tried to explain the purpose of the measures and presented scenarios of different kinds of patients and the types of services each might need. Professionals were shown how comparable measures were displayed on the Nursing Home Compare Web site, yet even physicians found it difficult to understand the concepts.
Dr. McGee was struck by the similarity between the BearingPoint study and a study she conducted with Medicare and Medicaid consumers on their reactions to Health Plan Employer and Data Information Set (HEDIS) measures as potential measures of performance.