Evaluation of VISTA Performance Profiles and Non-VISTA Measures
Eve A. Kerr MD MPH
VA Ann Arbor Healthcare System
Ann Arbor, MI
Funding Period: December 1999 - November 2001

BACKGROUND/RATIONALE:
The VA has incorporated extensive diabetes measures in their External Peer Review Program (EPRP) quality measurement system, which is based on medical record abstraction.
While EPRP is VA’s gold standard and provides the most reliable diabetes quality information currently available, chart abstraction is very expensive. Certain process and intermediate outcome data (for example, whether a lab test was performed and its value) are available from the Veteran’s Integrated Health Systems Technology and Architecture (VISTA) system. However, the validity and reliability of these data are unknown.

OBJECTIVE(S):
The specific goals of the project include: (1) to examine the correlation of diabetes measures derived from VISTA measures with those derived from medical record and patient survey, and (2) to assess the ability of each of the measures to accurately identify facilities with higher or lower quality.

METHODS:
The project compares diabetic quality measures in the VA obtained from three existing data sources: VISTA, medical records and patient surveys (using data already collected by the VA Office of Quality and Performance (OQP) as part of a quality assurance activity). We have compare the quality measures derived from each data source to determine how they correlate and how each contributes to the variation in quality scores across facilities.

FINDINGS/RESULTS:
Success rates were higher for process measures derived from medical record versus automated data (e.g., 78% vs. 68% for LDL measured; 84% vs. 78% for A1c measured). This difference narrowed for intermediate outcome measures (e.g., 79% vs. 76% for LDL<130; 86% vs. 88% for A1c<9.5%). Agreement for measures derived from the medical record compared to automated data was moderate for process measures (e.g.,A1c measured, kappa=0.61) but high for intermediate outcome measures (e.g., A1c<9.5%, kappa=0.92). Hybrid measures, which use automated data supplemented with medical record data, yielded success rates similar to those of medical record based measures. Hybrid process measures would require medical record review in only 50% of cases. Process measures, but no intermediate outcome measures, showed significant variation attributable to the facility, regardless of the data source.

IMPACT:
We found that agreement between medical record and automated data was generally high. Nonetheless automated data tended to underestimate the success rate in process measures for diabetes. Applying hybrid methodology yielded results consistent with the medical record but required less data to come from medical record reviews. Despite the high rates in overall performance, further research should examine the underlying reasons for facility level variation in diabetes process measures in order to craft appropriate quality improvement programs.

PUBLICATIONS:

Journal Articles

1. Heisler M, Smith DM, Hayward RA, Krein SL, Kerr EA. Racial disparities in diabetes care processes, outcomes, and treatment intensity. Medical Care. 2003; 41(11): 1221-32.
2. Kerr EA, Smith DM, Hogan MM, Hofer TP, Krein SL, Bermann M, Hayward RA. Building a better quality measure: are some patients with 'poor quality' actually getting good care? Medical Care. 2003; 41(10): 1173-82.
3. Kerr EA, Smith DM, Kaplan SH, Hayward RA. The association between three different measures of health status and satisfaction among patients with diabetes. Medical Care Research and Review. 2003; 60(2): 158-77.
4. Heisler M, Smith DM, Hayward RA, Krein SL, Kerr EA. How well do patients' assessments of their diabetes self-management correlate with actual glycemic control and receipt of recommended diabetes services? Diabetes Care. 2003; 26(3): 738-43.
5. Kerr EA, Smith DM, Hogan MM, Krein SL, Pogach L, Hofer TP, Hayward RA. Comparing clinical automated, medical record, and hybrid data sources for diabetes quality measures. Joint Commission Journal on Quality Improvement. 2002; 28(10): 555-65.
6. Heisler M, Bouknight RR, Hayward RA, Smith DM, Kerr EA. The relative importance of physician communication, participatory decision making, and patient understanding in diabetes self-management. Journal of General Internal Medicine. 2002; 17(4): 243-52.
7. Kerr EA, Krein SL, Vijan S, Hofer TP, Hayward RA. Avoiding pitfalls in chronic disease quality measurement: a case for the next generation of technical quality measures. American Journal of Managed Care. 2001; 7(11): 1033-43.
8. Asch SM, Kerr EA, Lapuerta P, Law A, McGlynn EA. A new approach for measuring quality of care for women with hypertension. Archives of Internal Medicine. 2001; 161(10): 1329-35.

DRA: Chronic Diseases, Health Services and Systems
DRE: Quality of Care, Technology Development and Assessment
Keywords: Diabetes, Quality assessment, Risk adjustment
MeSH Terms: none