Summary
GAO was asked to review the Centers for Medicare & Medicaid Services (CMS) initiative to publicly report additional information on its "Nursing Home Compare" Web site intended to help consumers choose a nursing home. GAO examined CMS's development of the new nursing home quality indicators and efforts to verify the underlying data used to calculate them. GAO also reviewed the assistance CMS offered the public in interpreting and comparing indicators available in its six-state pilot program, launched in April 2002, and its own evaluation of the pilot. The new indicators are scheduled to be used nationally beginning in November 2002.
CMS's initiative to augment existing public data on nursing home quality has considerable merit, but its planned November 2002 implementation does not allow sufficient time to ensure the indicators are appropriate and useful to consumers. CMS's plan urges consumers to consider nursing homes with positive quality indicator scores, in effect, attempting to use market forces to encourage nursing homes to improve the quality of care. However, CMS is moving forward without adequately resolving important open issues on the appropriateness of the indicators chosen for national reporting or the accuracy of the underlying data. To develop and help select the quality indicators, CMS hired two organizations with expertise in health care data--Abt Associates, Inc. and the National Quality Forum (NQF). Abt identified a list of potential quality indicators and tested them to verify that they represented the actual quality of care individual nursing homes provide. GAO's review of the available portions of the report raised serious questions about the basis for moving forward with national reporting at this time. NQF, which was created to develop and implement a national strategy for measuring health care quality, was hired to review Abt's work and identify core indicators for national reporting. To allow sufficient time to review Abt's validation report, NQF agreed to delay its recommendations for national reporting until 2003. CMS limited its own evaluation of its six-state pilot program for the initiative so that the November 2002 implementation date could be met. Early results were expected in October 2002, leaving little time to incorporate them into the national rollout. Despite the lack of a final report from NQF and an incomplete pilot evaluation. CMS has announced a set of indicators it will begin reporting nationally in November 2002. GAO has serious concerns about the potential for public confusion by the quality information published, especially if there are significant changes to the quality indicators due to the NQF's review. CMS's proposed reporting format implies a precision in the data that is lacking at this time. While acknowledging this problem, CMS said it prefers to wait until after the national rollout to modify the presentation of the data. GAO's analysis of data currently available from the pilot states demonstrated there was ample opportunity for the public to be confused, highlighting the need for clear descriptions of the data's limitations and easy access to impartial experts hired by CMS to operate consumer hotlines. CMS has not yet demonstrated its readiness to meet these consumer needs either directly or through the hotlines fielding public questions about confusing or conflicting quality data. CMS acknowledged that further work is needed to refine its initiative, but believes that its indicators are sufficiently valid, reliable, and accurate to move forward with national implementation in November 2002 as planned.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
No director on record
No team on record
No phone on record
Recommendations for Executive Action
Recommendation: To ensure that publicly reported quality indicator data accurately reflect the status of quality in nursing homes and fairly compare homes to one another, the Administrator of CMS should delay the implementation of nationwide reporting of quality indicators until there is greater assurance that the quality indicators are appropriate for public reporting--including the validity of the indicators selected and the use of an appropriate risk-adjustment methodology--based on input from the NQF and other experts and, if necessary, additional analysis and testing.
Agency Affected: Department of Health and Human Services: Centers for Medicare and Medicaid Services
Status: Not Implemented
Comments: Although CMS stated it would use GAO's report to help improve the initiative over time, the agency implemented the national reporting of quality indicators during November 2002, prior to gaining greater assurance that the quality indicators are appropriate for public reporting.
Recommendation: To ensure that publicly reported quality indicator data accurately reflect the status of quality in nursing homes and fairly compare homes to one another, the Administrator of CMS should delay the implementation of nationwide reporting of quality indicators until a more thorough evaluation of the pilot is completed to help improve the initiative's effectiveness, including an assessment of the presentation of information on the Web site and the resources needed to assist consumers' use of the information.
Agency Affected: Department of Health and Human Services: Centers for Medicare and Medicaid Services
Status: Not Implemented
Comments: Although CMS stated it would use GAO's report to help improve the initiative over time, the agency implemented the national reporting of quality indicators during November 2002, prior to completing a more thorough evaluation of the pilot.
