In October 1999, long-term care experts met to help guide the long-term care research agenda and provide advice on alternative approaches to data collection for the Agency for Healthcare Research and Quality (AHRQ, then the Agency for Health Care Policy and Research). The meeting was cosponsored by AHRQ's Center for Organization and Delivery Studies (CODS) and Center for Cost and Financing Studies.
By William D. Spector, D.E.B. Potter, Jan De La Mare
Acknowledgements
Introduction
Day One
Long-Term Care Research Questions
Cost-Effectiveness
Equity/Access
Financial and Market Incentives
Consumer Issues
Quality Assurance
Methodology
Additional Topics of the Morning Discussion
Measuring Indicators of Quality
Dimensions of Quality
Quality of Care
Quality of Life
Process and Structural Measures
Risk Factor Measures
Discussion of Quality Indicators
Scope
What is a Quality Indicator?
Data Collection Issues
Quality-of-Life Measures
Medical Management
Potential of the MDS and OSCAR Data
Day Two
Summaries of Data Collection Systems
1996 Medical Expenditure Panel Survey-Nursing Home Component
Medicare Current Beneficiary Survey
National Nursing Home Survey
Online Survey, Certification, Reporting System (OSCAR) and Minimum Data Set
Institutional Consumer Assessment of Health Plans Survey (CAHPS®)
National Long-Term Care Survey
Survey Integration
Summary of Discussion About Data Needs
Cost-Effectiveness
Equity/Access
Financial and Market Incentives
Consumer Issues
Quality Assurance
Current Data
Pros and Cons of Person-Based and Facility-Based Sampling Designs
Overall Limitations of Current Data
Possible Strategies to Improve Data
Build a Broader Residential Frame
Increase the Long-Term Care Sample Size of MCBS
Conduct Specialized Periodic Surveys
Explore Use of Administrative Data
Conclusions
Appendices
1. Meeting Agenda
2. List of Participants
3. Pre-meeting Questionnaire
Tables
1. Framework for Long-Term Care Research Priorities
2. Measuring Long-Term Care Quality
3. Data Needs for Risk Adjustment
4. Comparison of Four National Data Sources (Text Version)
On October 14-15, 1999, AHRQ sponsored Future Directions for Residential Long-Term Care Health Services Research, a meeting to convene long-term care experts to help guide the Agency in developing its long-term care agenda. The meeting was sponsored by AHRQ's Center for Organization and Delivery Studies and Center for Cost (CODs) and Financing Studies (CCFS).
Specifically, the goals of the meeting were to:
The 2-day meeting was based on experts' responses to a series of questions collected prior to the meeting. The questions focused on policy and research priorities, national quality indicators, and data collection methods. (See Appendix 3 for pre-meeting questions.)
Participants were asked to list their top five research and policy questions concerning cost, quality, and outcomes of residential long-term care. Their responses fell into six categories:
Select to access Table 1. Research Questions.
Responses in this category reflected the need for better information on the most cost-effective ways to care for populations needing long-term care and better ways to provide a high quality of life. In some cases, responses stressed the cost side, and in others, they stressed the effectiveness side. Responses included concerns about how processes of care can be made more cost-effective as well as ways structure affects outcomes.
Examples of process concerns included approaches to providing primary care, rehabilitation, and prescription drugs, as well as assistive devices and special care units. Concerns about structure focused on the mix of staff, training of staff, and ways in which staff are organized, including approaches to supervision and communication among staff.
Participants also were concerned about the effect of ownership, including the impact of for-profit versus not-for-profit ownership and chain versus non-chain membership. In addition, experts expressed concern about the impact of the increasing vertical integration of long-term care businesses on cost and quality.
Equity issues included the distribution of long-term care across payment sources and sub-populations. Experts were concerned about the financial burden of long-term care and the cost of long-term care insurance premiums, and the implications of the growth in assisted living on equity and access.
Responses focused on incentives created by Medicaid and Medicare reimbursement and case mix reimbursement. Participants also mentioned the impact of the prospective payment system on the quality and cost of long-term care. Finally, they expressed interest in the role of market competition in fostering innovation and quality.
Participants expressed interest in research that studies consumer decisionmaking. They also mentioned efforts such as quality report cards to improve the quality of information that consumers receive.
Experts were interested in a number of quality assurance topics. They were concerned that currently, quality of care and outcomes of care are measured better than quality of life, although quality of life is a very important issue in residential care. They were interested in using outcomes to assess the quality of providers and to develop ways to assess the quality of the long-term care system as a whole.
Participants listed many different methodology issues related to improving measures, data collection, and forecasting. Several highlighted the importance of measuring quality-of-life outcomes, characterizing processes of care, and determining risk adjustment. In addition, participants ranked measuring consumer preferences as a high priority. In the data collection area, participants emphasized strategies for screening persons at high risk for long-term care use to increase sample size, linking administrative and survey data, and developing a broad residential frame.
Participants discussed the importance of developing measures that could be used to enhance consumers' ability to make informed choices based on their needs. They also noted the challenge of distinguishing between different residential settings of care (e.g., nursing home, assisted living, group homes, etc.). In addition, they mentioned the need to describe variations in residential care and to understand the context in which an individual resides. The role of subcontracting in nursing home care—whether staff are employed by the nursing homes or are subcontractors—also was noted as a gap in knowledge.
One participant underscored the importance of creating client-focused quality indicators rather than institutionally focused indicators. Others stressed the need to have facility-level quality measures, especially given that quality regulation is generally at the facility level.
Participants emphasized the importance of common measures for populations in different settings to allow for comparisons, regardless of where consumers are obtaining services. They also stressed the need to characterize variations in the market.
One goal of the meeting was to elicit expert opinion about quality indicators for residential long-term care that could be used to monitor changes in quality over time. We used a methodology that paralleled the one used to identify the most important policy issues. Prior to the meeting, we sent participants a set of questions and a proposed grid to help standardize responses. (Select Appendix 3 for pre-meeting questions.) The primary goal was to determine the dimensions of quality, and then to specify the most important measures within those dimensions.
We also asked participants, when appropriate, to specify the most important risk factors associated with those measures, time intervals needed for followup, and which risk factors would be likely to change in value over time. They were asked to think about four populations:
At the meeting, we presented a summary of the participants' responses (select to access Dimensions of Quality and Risk Factor Measures) and then discussed issues of feasibility and scope.
Respondents generally defined residential long-term care as all of the care provided in residential care settings. They included personal care for the traditional long-stay population, concerns related to quality of life associated with the home environment of the setting, and aspects of acute and preventive care. Prior to the meeting, we sorted their responses into two dimensions—quality of life and quality of care—and summarized the specific health areas that were specified within these broad dimensions.
Table 2 summarizes areas of health concerns identified by respondents. With respect to quality of care, we divided responses into seven categories: mortality, disability, discomfort, nutritional problems, infections, geriatric syndromes, and preventable complications. These represent the broad range of outcomes that may result from care provided in long-term care institutions. Three quality-of-life dimensions were distinguished using a taxonomy provided by Spilker and Revicki.1
When delineating specific health concerns, respondents typically discussed outcome measures, but some respondents included process and structural measures as well. The respondents generally kept their recommendations at a conceptual level rather than recommending specific instruments.
1. Bert Spilker and Dennis A. Revicki, "Taxonomy of Quality of Life" in Quality of Life and Pharmacoeconomics in Clinical Trials. Edited by Bert Spilker (Philadelphia: Lippincott-Raven Publishers, 1996), pp. 25-31.
Many residents in long-term care facilities are on a downward trajectory so that the goals of the care are delaying mortality and preventing excess morbidity and disability. Therefore, providers' ability to minimize mortality and functional decline is an important factor for judging quality. Respondents stressed particular areas of disability, including:
Respondents also mentioned problems with nutrition (e.g., excess weight loss and dehydration) and infections (e.g., pneumonia, septicemia, and urinary tract infections). A number of other outcomes that could be viewed as geriatric syndromes were also mentioned, including pressure ulcers, incontinence, contractures, and sleep disturbances. In addition, respondents listed pain control as another important aspect of long-term care. The final category included preventable negative outcomes such as some hospitalizations and emergency room use and medical complications, including complications from medicines.
The second dimension of quality of life included areas of concern that are affected more directly by the home environment of institutions rather than the care provided, although care may have an indirect effect on these outcomes as well. These areas of concern can be categorized as follows: personal-internal, personal-social, and external-societal.2 Within the personal-internal domain, the respondents listed choice, dignity, privacy, and meeting life preferences; within the personal-social domain they included social interaction in and outside the facility; and within the external-societal domain, they included safety/abuse, neglect, and community participation.
2. Spilker and Revecki, pp. 25-31.
Although most respondents focused on outcome measures when delineating quality domains, some respondents directly discussed process measures of quality. Rather than speaking about changes in health (e.g., formation of pressure ulcers), they stressed the importance of monitoring in some manner the care (or lack of care) provided and perhaps using guidelines. They specifically mentioned skin care and periodic turning, range of motion exercises, toilet training, and use of restraints. Other examples included the availability of endurance and strength training and appropriateness and timeliness of assessments. One respondent discussed the importance of maintaining updated medical records in the facility to better reflect care in other settings such as hospital and ambulatory visits. Collection of process measures are needed to help fill the gap in the literature that demonstrates the relationship between process measures and outcomes.
Some respondents discussed the importance of including structural measures of quality. These included measures of staffing, the housing environment, planned activities, and other characteristics of facilities (e.g., size and ownership type). Staffing concerns included staff-to-patient ratio measures for nurses and nurse practitioners, physicians, therapists, and social workers; staff turnover and level of compensation; training; and concern about long hours.
Participants were also asked to delineate the most important risk factors. We were interested in risk factors because health outcomes result from the interaction of care provided and the health of the individual being treated. Outcomes can be converted into quality indicators to the extent that the effectiveness of the care can be separated from the influence of the risk factors. This is usually done using "expected outcomes" by controlling for risk factors at the patient level and aggregating to the level of interest. Consequently, it is important to collect data on the most important risk factors.
Table 3 lists important risk factors that participants mentioned prior to the meeting. Based on our questions, participants could have linked risk factors to specific outcomes. Typically, we did not get that level of specificity from the respondents, so the synthesis of responses indicate which risk factors are needed overall for risk adjustment.
Responses about the most important risk factors included demographic characteristics (age, race, and gender), ADLs, mental problems (e.g., cognitive deficits and depression and/or anxiety), physical impairments, sensory impairments, communication problems, diagnoses, terminal prognosis, and socioeconomic factors (e.g., income and marital status). Some respondents also mentioned length of stay.
There was some disagreement about when risk factor information should be collected. Some stressed collection at admission, while others thought that values would change over time so that the admission value, especially for long stayers, would no longer be relevant. Including risk factors that are not collected at admission can be problematic because they may reflect previous levels of quality.
Only a small number of participants responded to the additional questions concerning the appropriate followup interval for quality indicators or the risk adjusters that they thought would be likely to change over time. Respondents generally felt that 3-month, 6-month, and 1-year intervals were sufficient to capture change over time. If respondents made a distinction concerning followup intervals, they thought that quality-of-life measures did not need to be followed any more frequently than every 6 months. Respondents noted that age, disability, and behavior problems are likely to have higher mean values in future years, reflecting the aging of the population and increasing disability levels of residential long-term care residents. It is particularly important, therefore, to control for these variables when making comparisons of outcomes over time.
Meeting participants felt that the list we presented included most of the outcome dimensions. Measures of functioning were seen as central to a study of outcome indicators. There was less consensus about measures of cognitive functioning, however. These are more difficult to use as quality indicators because it is less certain that a facility can influence cognition.
Nevertheless, some participants viewed certain aspects of cognition as more malleable than others. Measures of confusion and depression may be more attributable than pure cognition. It was pointed out that some cognitive problems can be reversible, and inappropriate use of drugs may lead to over-sedation. Many believed that more research is needed for measuring cognition to isolate malleable domains from those that are not.
Additional measures of staffing were also discussed. The discussion focused on the changing roles of staff and the changing mix of functions that staff perform. One participant gave an example of nurses doing some tasks that respiratory therapists normally perform. This may suggest that less skilled staff are being substituted for more skilled staff, indicating that staff ratios should be supplemented by an analysis of functions performed.
Finally, one panel member recommended that we include respect in the quality-of-life measures because it is a separate construct from dignity.
Discussion about an appropriate definition of a quality indicator occurred several times throughout the meeting. Usually, there are two schools of thought among researchers interested in quality: the outcome advocates and the process advocates. Most participants seemed to take a middle ground between these two positions. They felt that for an outcome measure to be used as a quality indicator, there should be evidence that the outcome is malleable and that a facility has control over changing that particular outcome. If a process (or structural) measure is used as a quality indicator, there should be evidence that it affects resident outcomes. The strictness of these standards and the degree to which indicators should be risk adjusted would depend on how the measures are used.
Participants pointed out that national data collection strategies rely heavily on proxy respondents and records. For household-based data collection, it is common for one respondent in each household to answer questions about all household members; for surveys in residential settings, staff are interviewed and records are used. These have been the approaches of the Medical Expenditure Panel Survey Nursing Home Component (MEPS NHC), the National Nursing Home Survey (NNHS), and the Medicare Current Beneficiary Survey (MCBS).
The general feeling was that these approaches work well for collecting expenditure data, but would not work well for developing quality-of-life measures. There was concern that changes in data collection protocols could be very expensive. For example, if the survey involved a structured walk-around and observation of the environment, it would involve a major change in training and instruments, and could also affect the level of cooperation received from the facilities.
Participants cited the development of quality-of-life indicators for residential settings as a high priority. They felt that these measures are not ready to be used now, but that this is an important area for research. They also expressed concern that if quality of life is not monitored while other areas are, quality in this area may suffer.
The group discussed the difficulties of getting valid quality-of-life information from records, or even getting valid responses directly from residents, especially those who are cognitively impaired. There is some evidence that staff in assisted living facilities can provide reliable data on physical and cognitive functioning, communication, and behavioral problems of residents, but not good measures on mood, psychosocial, and activity measures. However, participants agreed that proxy responses are not valid for quality-of-life measures.
With respect to interviewing residents, some participants cautioned against using mental status exams to screen sample persons because mental deficits are very multidimensional. Persons with some deficits may be totally capable of answering certain questions. In addition, it was suggested that if persons with mental deficits are interviewed, the questionnaire should be less complicated than for cognitively intact persons.
The group mentioned that Rosalie Kane is developing quality-of-life measures for nursing home residents under a contract with the Health Care Financing Administration (HCFA) and suggested that this effort might be worth following. HCFA also has a new contract with Research Triangle Institute to develop performance measures for intermediate care facilities for the mentally retarded (ICFMRs).
Participants voiced additional concerns about quality-of-life measures. One concern was the low response rate that would be likely because of a high proportion of cognitively impaired persons. Another concern was how to obtain individual preferences. Discrepancy score approaches were discussed, but the problem of differential expectations makes discrepancy scores difficult to compare. Measures of preferences for everyday living are being developed at the Philadelphia Geriatric Center. This approach seeks to establish preferences before cognitive impairment occurs. One participant suggested that some measures of quality of life are more difficult to measure than others. For example, it may be easier to measure hotel aspects of a facility (e.g., quality of food, physical environment) than to observe neglect and abuse.
In the quality-of-life discussion, participants also stressed the need to have positive measures as well as negative measures (e.g., happiness, not just a focus on depression; meaningful social interactions, not just any social interactions).
A second priority area was management of acute and chronic conditions. The discussion focused on the type of data needed and the importance of being able to compare data across different settings. In some cases, measures involve blood samples, such as hemoglobin A1C to assess the management of diabetes. Participants expressed concern that these types of measures would involve new data collection approaches, and the reliability of these data have not been studied.
Another concern was expressed about the availability of complete medical records. These are generally not available in assisted living facilities, and may not be complete in nursing homes because some of the care is provided through outside clinics. These records are not duplicated in nursing facilities.
A final concern was that there may be small sample size problems resulting in large standard errors for measures that are based on relatively rare conditions. This is especially problematic if measures are facility specific and are used to compare the quality of facilities rather than making comparisons for large populations over time.
The minimum data set (MDS) in nursing homes contains diagnosis and symptom information that may allow some condition-specific measures to be developed for a given condition; however, the ability to develop meaningful quality indicators with these data has not been studied and is likely to be quite limited. For example, for some measures (e.g., pressure ulcers and urinary tract infections), the MDS uses the reference point of "last week." This is a limitation because assessment of infections should be done over the entire stay. It is important to track infections over the entire stay because there can be multiple occurrences and some infections can be cured after a short period of time. The 1996 MEPS Nursing Home Component collected all occurrences of infections, pressure ulcers, and fractures.
Participants discussed a number of plans to augment the MDS data as well as the potential to merge these data with other HCFA data. Currently, MDS is relevant for long-stay nursing home residents only. A subacute care MDS is being developed, but it is not likely to be complete within the next 5 years. To help assess changes in outcomes for short-stay residents, the frequency of assessments for the Medicare population will be increased to Day 4, Day 11, and then in 60 days. Drug data in nursing homes will be collected by October 2000. In addition, a discharge assessment will be done.
The MDS currently does not have discharge outcome information on every resident. The process for getting MDS information from the repository to researchers has not yet been worked out.
In addition, HCFA's Online Survey Certification and Reporting (OSCAR) data are being redesigned. This information is often used to link facility variables with MDS data. There is interest in collecting staffing data, for example, at more points in time. There was also discussion about the usefulness of developing quality measures by linking MDS to Medicare enrollment and claims data and Medicaid claims data.
Most of the discussion related to limitations of information and the fact that these data apply only to nursing home residents. For example, lack of availability of important information (e.g., advanced directive or living will information) in the Medicare data presents a limitation of using these data alone, but some of these limitations could be eliminated when combined with MDS data.
Discussion also focused on Medicaid data that recently were mandated for submission to HCFA. Although these data provide more details on treatments than Medicare data, they are only relevant for long-stay residents. The group expressed concern that the Medicare data currently do not include data from managed care, so as managed care grows, the bill files only will include a very select population. Although patients are typically disenrolled from managed care when entering a long-term care facility, the timing varies by plan and State.
The goal of day two was to get advice from panel members concerning data collection and approaches to integrating national residential care surveys. During the morning session, presenters highlighted the major features of recent national long-term care surveys. During the afternoon session, participants discussed data needs—within the context of the enumerated research goals—and the pros and cons of various data collection approaches.
The following summaries describe the presentations of various data collection systems, as well as a presentation on data integration. Table 4 (Table Version, Text Version) compares the central design features for some of the data systems discussed. (An earlier version of this table was compiled by AHRQ staff for distribution at the meeting to facilitate discussion.)
Sponsored by the AHRQ and the National Center for Health Statistics (NCHS), the MEPS NHC survey collects basic and ancillary charges for nursing home care and associated billing amounts, amounts received, and payment sources for a sample of residents who spent any time in a nursing home during the calendar year 1996. It provides income and asset information, typically collected from next-of-kin community respondents, and captures transitions in care for the year.
The design is a multi-stage probability design, with nursing homes sampled at the initial stage and persons at the final stage. Data are collected by computer-assisted personal interviewing (CAPI) methods. The 1996 MEPS NHC included a sample of about 800 nursing homes and 6,000 persons—a little more than half were sampled as January 1 residents and the remainder sampled as 1996 admissions.3
3. For additional information, see Porter, DEB. Design and methods of the 1996 Medical Expenditure Panel Survey Nursing Home Component. Rockville (MD): Agency for Health Care Policy and Research; 1998, MEPS Methodology Report No. 3. AHRQ Publication No. 98-0041, and the AHRQ MEPS Web site, http://www.meps.ahrq.gov.
Although the survey does not collect quality measures directly, it collects a large number of health status measures at two points in time (90-day followup for admissions and 1-year followup for January 1 residents), which would allow for analyses of health outcomes. The health status questionnaires were designed to mirror HCFA's MDS instrument.
In addition, the MEPS NHC collects data on any occurrence of infections, pressure ulcers, fractures, and hospitalizations; medicines prescribed over the year, and use of health services for periods of nursing home residence. Data on facility characteristics include nurse staffing (agency and employees), special care units, accreditation, availability of specially trained providers, vaccinations for influenza and pneumonia, and services provided to residents and nonresidents. Facility data, at the unit level, can be linked to persons.