|
BACKGROUND/RATIONALE:
Within VA QuERI, and in implementation science more broadly, researchers and funders are emphasizing the need to use well-developed theoretical models to guide research and quality improvement activities. Conceptual models, in principle, help ensure that key determinants of implementation effectiveness, including contextual factors, are recognized and assessed, so that key facilitators are ensured and barriers are diagnosed and addressed.
However, conceptual models can only guide implementation to the extent that variables contained therein are accurately and efficiently measured. Accordingly, IHD-QuERI has developed survey scales to measure key variables from a well-developed conceptual model: the Promoting Action in Research Implementation in Health Services (PARIHS) model. These scales are intended to assess organizational readiness to change in preparation for implementation projects or in the early stages.
The instrument developed by IHD-QuERI is among few available for implementation in health services, and has generated considerable interest among other queris and academics from other institutions. The instrument has been used in preliminary analyses of three quality improvement projects and the data appear to be promising. However, the instrument has yet to be rigorously validated using established psychometric assessments.
OBJECTIVE(S):
The objective of the present project is to rigorously validate the instrument and make any necessary revisions. Specifically, the project will:
1. Conduct item analysis to test scale reliability and convergent and divergent validities;
2. Examine the underlying factor structure of the organizational readiness to change scales using exploratory factor analysis; and
3. Test the predictive validity of organizational readiness to change scales against expert panel evaluation based on cardiac care plans developed for a national QI initiative.
METHODS:
Data:
For each research aim, we will analyze survey data collected for three quality improvement projects: 1) the Acute Coronary Syndrome Objective 4 assessment of cardiac care plan implementation (n = 66); 2) the VISN 19 clinical reminders project (n = 13); and 3) the VISN 23 ICU project (n = 40). Surveys were administered to clinicians and staff involved in quality improvement projects. Overall response rate was approximately 53%. For Aim #3, we will include data from expert panel evaluations of cardiac care plans developed for the VHA Cardiac Care Initiative to assess the predictive validity of the RTC scales by comparing them to the expert panel evaluations.
Each data set includes responses for a 78-item instrument measuring readiness-to-change in three major sections: Evidence was assessed with 3 subscales, context with 6 and facilitation with 9. Each subscale comprised 4 - 5 items. Subscales were based on specific dimensions elaborated in the PARIHS model.
Analysis:
We will conduct three sets of analyses (Nunnally and Bernstein 1994; Bernard 2000): 1) item analysis, 2) exploratory factor analyses, and 3) multivariate regression analysis
Aim 1: Item analysis comprises three measures of items' correlations to a given subscale to determine if items within scales correlate as predicted, and fail to correlate across scales as predicted. (1) Cronbach's alpha will be calculated for scale reliability. (2) Item-rest correlations will be calculated as an indicator of convergent validity and to identify items that do not correlate well with a given scale and could be dropped for parsimony. (3) Item-rest correlations will then be compared to items' correlations to other scales to assess the divergent validity of scales.
Aim 2: Exploratory factor analysis will be conducted to determine how many underlying "factors" might be present, and their relationships to each other and to corresponding items. We will examine the factor structure of the data to identify items of inadequate representation of factors. The emergent factor structure will be compared to the structure predicted in the survey design in order to assist refining existing items or adding new items to achieve a more complete representation of the PARIHS model.
Aim 3: Predictive validity will be assessed using multivariate regression analyses to examine the association of readiness-to-change scores derived from the quality improvement projects with an independent assessment of readiness to change, controlling for facility characteristics such as academic affiliation and level of cardiac care services. The independent assessment of readiness to change will come from expert panel evaluation of the appropriateness and completeness of an 11-point cardiac care plan that each VHA facility and VISN was required to develop as part of the VHA Cardiac Care Initiative. The purpose of the initiative was to ensure that acute coronary syndrome patients received guideline concordant care.
FINDINGS/RESULTS:
No results at this time.
IMPACT:
VHA personnel and facilities stand to benefit from having access to a validated survey that could help predict how prepared a facility is to make a given practice change. The survey is in the public domain and available for use by VA and is already being used by several quality improvement projects within the VA.
VHA patients, whose data will be used to construct the facility-level quality of care indicators, benefit indirectly by helping fashion a more reliable tool for guiding implementation of evidence-based practices, which are intended to improve the quality, effectiveness and efficiency of care.
This research will also benefit the broader health services community by potentially providing a validated instrument for assessing organizational readiness to change.
PUBLICATIONS:
None at this time.
DRA:
Health Services and Systems
DRE:
Communication and Decision Making
Keywords:
Organizational issues
MeSH Terms:
none
|
