Patients who are sicker differ somewhat from healthier patients in their comparative ratings of health plans, as do those who are older or younger. Early results suggest that these differences might sometimes be large enough to consider presenting plan ratings separately by subgroups of patients, according to a study of the Consumer Assessments of Health Plans Survey (CAHPS®) supported by the Agency for Healthcare Research and Quality (HS09205).
CAHPS® was originally developed with support from AHRQ. Subsequently, it has been implemented in many sites where its use has been required by large employers, employer groups, State and Federal Government agencies, and accreditation organizations. Since its launch in 1997, CAHPS® has become increasingly important as a tool for evaluating and comparing health plans.
In this study, researchers at Harvard Medical School analyzed responses from 8,204 adults who described their experiences with their health plans in the Washington State demonstration of CAHPS®. The survey asked State employees to rate 20 health plans offered by the Washington State Employees Benefits Bureau. The analysis focused on responses to four global rating items on a scale from 0 (worst possible) to 10 (best possible) for the plan, health care received overall, personal doctor, and specialist.
Overall, a case mix model that adjusted for between-plan differences in respondents' perceived health status and age explained from 3 to 7 percent of the variation in individual CAHPS® scores. Although the case mix effects were significant, the magnitudes of the adjustments for case mix were typically modest and did not greatly affect the relative ranking of health plans. However, in a few cases the adjustments were large enough to substantially affect a plan's ranking (as large as 44 percent of the standard deviation of plan means for rating of health care). The researchers conclude that despite the limited impact of case mix adjustment, it was nonetheless worthwhile in the interest of fairness to plans with adverse case mix.
Analyses of interactions between plan and patient characteristics (age and health status) suggested that effects of these characteristics might vary from plan to plan, although not enough to reverse the direction of the relationships or to substantially affect the validity of the case mix model. The effects for individual patients, however, could be more substantial, indicating that some plans are more successful than others in equalizing the gap between ratings by sicker and healthier members. The researchers recommend that similar analyses be applied in larger implementations of CAHPS® in which these effects could be assessed more precisely.
See "Does the effect of respondent characteristics on consumer assessments vary across health plans?" by Alan M. Zaslavsky, Ph.D., Lawrence Zaborski, M.S., and Paul D. Cleary, Ph.D., in the September 2000 Medical Care Research and Review 57(3), pp. 379-394.
Return to Contents
Proceed to Next Article
540 Gaither Road Rockville, MD 20850