BACKGROUND/RATIONALE:
Our previous VA HSR&D funded research has established that computerized vignettes are a valid measure of actual clinical practice for measuring the quality of care in the outpatient setting (IIR 95-014 and IIR 98-118-1). Vignettes have several advantages: they are inherently case-mix adjusted, have greater criterion validity than the medical record, are inexpensive relative to other measures and allow for cross-system comparisons of care. As a next step, we propose to use vignettes to improve clinical practice and health outcomes in the VA outpatient setting.
OBJECTIVE(S):
This is a pilot study to first measure the cross-sectional relationship between variations in physicians’ vignette scores and aggregated scores of individual physicians’ patient health outcomes; and second to (longitudinally) determine whether feedback of vignette scores improves physicians’ clinical performance as measured by vignettes
METHODS:
1. Study Design: Physicians will complete computerized vignettes for four conditions – diabetes, coronary artery diseases (CAD), chronic obstructive pulmonary disease (COPD), and depression. We will collect retrospective outcomes data and develop composite outcome measures for on two conditions, Diabetes and CAD. For the longitudinal analysis, only vignette data will be collected and fed back to providers. Feedback will consist of specific data on vignette outcome scores for the individual physicians and for the sites overall.
2. Site Selection: Primary care clinics at 2 VAMCs
3. Study Population and Sampling: We will enroll primary care physicians at 2 VAMCs. 30 consenting physicians will be prospectively randomized into two groups. One group will receive feedback of their vignette scores, and the other group will serve as control, receiving no feedback.
4. Variables and Measurement Instruments: Computerized vignettes measuring clinical practice completed by the physicians for diabetes, CAD, COPD, and Depression and a composite health outcome measures from the medical records of these physicians’ patients with diabetes and CAD.
5. Data Collection Strategy and Timeline: Vignettes will be administered to all physicians at baseline, with feedback of scores 3 months later and readministration of vignettes 9 months thereafter to measure the trend in improvement. The composite outcome data will only be collected at baseline only.
6. Data Analysis: The statistical analysis will compare the effects within the context of an analysis of covariance (ANCOVA) model. The analyte is the quality of care physicians give to patients with four common conditions. The relationship between vignette scores and patient outcomes will be modeled accounting for clustering effects. The prospective experimental design will be used to quantify possible differences between the intervention and control groups. The data will be analyzed using a three-way crossed, one-way nested ANCOVA model where the covariate is the baseline vignette score. This model can be used to look at case effects, by domain, level of training, and by site.
FINDINGS/RESULTS:
PI status has been transferred to Sharad Jain, MD; Dr. Peabody continues to serve as a project advisor. Next steps involve hiring project staff, patient recruitment, and data collection.
IMPACT:
It is anticipated that our study will provide important information as to whether feedback from clinical vignettes will 1) improve provider practice and 2) health outcomes. This information will help the VA system improve the quality of care that veterans receive.
PUBLICATIONS:
Journal Articles
- Peabody JW, Luck J, Jain S, Bertenthal D, Glassman P. Assessing the accuracy of administrative data in health information systems. Medical Care. 2004; 42(11): 1066-72.
