You Are Here: AHRQ Home > About AHRQ > AHRQ 2008 Annual Conference > Slide Presentation (Text Version)

Hospital Quality Indicators in Iowa Rural Hospitals

Slide Presentation from the AHRQ 2008 Annual Conference

On September 10, 2008, Pengxiang (Alex) Li, made this presentation at the 2008 Annual Conference. Select to access the PowerPoint® presentation (302 KB).

Presented by: Pengxiang (Alex) Li
Co-authors:
- Marcia W. Ward
- Paul James
- John E. Schneider

Slide 1

Hospital Quality Indicators in Iowa Rural Hospitals

Bethesda, 2008 AHRQ Annual Meeting Maryland.
Support grant: Agency for Healthcare Research and Quality Grant #HS015009.

Slide 2

Background

Hospital quality indicators were used to provide a perspective on hospital quality of care.
- AHRQ Inpatient Quality Indicators (IQIs).
- AHRQ Patient Safety Indicators (PSIs).
Our analyses focus on
- Acute Myocardial Infarction (AMI) in-hospital mortality (IQI-15).
- Four PSIs (PSI-5, PSI-6, PSI-7, and PSI-15).

Slide 3

Outline

Comparison of Iowa urban and rural hospitals on AMI in-hospital mortality.
- James PA, Li P, Ward MM. Myocardial infarction mortality in rural and urban hospitals: Rethinking measures of quality of care. Annals of Family Medicine 5:105-111, 2007.
Association between Critical Access Hospital (CAH) conversion and patient safety indicator performance.
- Li, P., Schneider, J. E. & Ward, M. M., (2007) Effect of Critical Access Hospital Conversion on Patient Safety. Health Services Research 42(6):2089-2108.
Exploration of a potential reason of patient safety change associated with CAH conversion.
- Li, P., Schneider, J. E. & Ward, M. M., Effects of Critical Access Hospital Conversion on the Financial Performance of Rural Hospitals. Inquiry (in press).

Slide 4

How do Iowa urban and rural hospitals compare on AMI in-hospital mortality?

James PA, Li P, Ward MM. Myocardial infarction mortality in rural and urban hospitals: Rethinking measures of quality of care. Annals of Family Medicine 5:105-111, 2007.

Slide 5

Introduction

Observational studies find that the quality of care for myocardial infarction (MI) patients admitted to rural hospitals is substandard (Sheikh 2001, Baldwin 2004).
- Lower volumes of MI patients in rural hospitals.
- Lacking cardiologists.
- Lacking support services.

Slide 6

Introduction

Validity of these observational studies has been questioned.
- Unbalanced comparison groups.
  - Patients admitted to rural hospitals tend to be older, poorer, in poorer health, and have greater number of comorbidities (Baldwin 2004, Chen 2000, Frances 2000).
- Referral patterns of rural provider.
  - Empirical study showed that less severe patients were referred to urban hospitals (Metha 1999).
- Unmeasured confounding may account for differences in patient outcomes.

Slide 7

Objectives of the study

To compare characteristics of MI patients admitted to rural and urban hospitals.
To examine in-hospital mortality between rural and urban hospitals among MI patients.
- Using traditional risk adjustment techniques (Logistic regression).
- Using instrumental variable methods (IV).

Slide 8

Methods-Data

Discharge data from Iowa State Inpatient Dataset (2002 & 2003).
Inclusion criteria:
- A principal diagnosis of MI (ICD-9-CM: 410.01-410.91).
- Eighteen years or older.
Exclusion criteria:
- The hospital identification number was missing (n=9).
- Patient's whose home county was not in Iowa (n=1,248).
- Patients' zip code was missing (n=14).
- Patients' sex was missing (n=1).
- Our primary analyses also excluded patients discharged or transferred to another short term general hospital for inpatient care (n=1,618).
Most of our analyses are based on 12,191 MI patients.

Slide 9

Methods: Variables

Dependent variable:
- In-hospital mortality.
Independent variables:
- Urban vs Rural hospitals that patients admitted to:
  - Urban: 27 hospitals.
  - Rural: 89 hospitals.
- Payer: e.g. Medicare, private insurance, self-pay.
- Admission type: e.g. emergency.
- Race.
- Risk adjustment index:
  - Charlson comorbidity index.
  - All Patient Refined Diagnosis Related Groups (APR-DRGs) risk index.

Slide 10

Methods: Traditional Analytic Approach (Logistic Regression)

Univariate analyses of group comparisons.
- Chi-square tests for dichotomous data.
- ANOVAs for continuous data.
Logistic regressions for multiple regression analyses.

Slide 11

Methods: Pitfalls with Logistic Regression

Using administrative inpatient data, one cannot control all patients' risk factors (e.g. severity of illness):
- If unmeasured variables are related to selection of the hospital, the estimates of the hospital-specific contribution to mortality will be biased.
- For example, elderly MI patients with severe comorbid conditions, which are unmeasured in administrative data, might prefer to remain in the rural hospitals.
  - As a result, a higher risk-adjusted mortality rate in rural hospitals might simply be due to more severe patients in rural hospitals.
Using administrative inpatient data, one cannot control all patients' risk factors (e.g. severity of illness):
- If unmeasured variables are related to selection of the hospital, the estimates of the hospital-specific contribution to mortality will be biased.
- For example, elderly MI patients with severe comorbid conditions, which are unmeasured in administrative data, might prefer to remain in the rural hospitals.
  - As a result, a higher risk-adjusted mortality rate in rural hospitals might simply be due to more severe patients in rural hospitals.

Slide 12

Approaches to Minimize Bias

Collect all the relevant patient-level variables: very costly.
Randomized controlled trial.
- Not feasible to this study.
Instrumental variable (IV) estimation:
- An econometric technique which enables us to obtain unbiased estimates of treatment effects in observational studies.
- An example: Wehby (2006) found that using the logistic regression model, early initiation of prenatal care is associated with a higher probability of low birth weight (LBW).
  - Unmeasured confounders: women at a higher risk demand more (or early) prenatal care compared to those at lower risk.
  - IV estimations showed that early time to prenatal care initiation is associated with a lower probability of LBW.

Slide 13

The Instrumental Variable (IV) estimation

IVs are used to achieve a "pseudo-randomization."
- The instrumental variable technique can extract variation in the focal variable (rural hospital selection) that is unrelated to unmeasured confounders, and employ this variation to estimate the causal effect on an outcome.
Assumptions for IV(s):
- IV(s) should correlate with treatment variable (choice of rural hospital).
- IV(s) should not be correlated with the unmeasured confounders.

Slide 14

Methods: Instrumental Variable Technique

Instrumental Variable = Patients' distance to the nearest urban hospital
- The distances between each patient's home and all urban hospitals in Iowa were obtained by calculating the distances between the centroids of each patient's resident zip code and all urban hospitals' zip codes.
- Similar to Brooks (2003) approach, instrumental variables in the study are dummy variables that group patients based on the their distance to the nearest urban hospital.

Slide 15

Methods—IV Technique: First assumption

Patients who live closer to an urban hospital are more likely to choose an urban hospital than those who live farther away.
- Partial F-statistics for the IVs in the first stage regression.
- Small values of first-stage F-statistics imply failure of assumption 1.
- Rule of thumb: F>10 indicates good association (Staiger 1997).

Slide 16

Methods—IV Technique: Second Assumption:

Distance to the nearest urban hospital is not associated with the severity or pre-morbid risks of patients with MI.
- Descriptive comparison between two groups of patients classified by IV.
  - If the instrument is independent of the unmeasured confounders, it should also be independent of observed risk factors (e.g. age, and comorbidity index).
- Over—identifying restrictions tests.
  - The null hypothesis is that the IV is not correlated with unmeasured confounders.

Slide 17

Methods: IV Technique

To examine the robustness of our findings:
- We used a range of patients' groups for the instrumental variable (2, 4, 8, and 12 groups).
- We varied the independent variables.
The syslin two-stage least squares (2SLS) procedure in SAS 9.1 was used to do IV estimation.

Slide 18

Results—Table 1: Baseline characteristics of MI patients admitted to rural and urban hospitals

Table compares rural versus urban MI patients showing that rural patients are about 13 ½ years older, have 1 ½ times the Charlson comorbidity and APR-DRG risk index, and about double the in-hospital mortality.

Slide 19

Results—Table 2: Baseline characteristics of MI patients transferred out of rural hospitals or staying in rural hospitals

Table comparing rural patients who stay versus those who transfer out of rural hospitals show that those whostay are about 11 years older, have about 25% more secondary diagnoses, about 505 more Charlson comorbidity, and about double the APR-DRG risk index.

Slide 20

Results—Table 3: Odds ratios of in-hospital mortality among MI patients admitted to urban hospitals or to rural hospitals, using logistic regression models (n=12,191)

Table shows that urban patients have a lower mortality risk than rural patients.

Slide 21

Results—Table 4: Characteristics among MI patients grouped by distance to the nearest urban hospital

Table with these column headings: Variables—Distance to nearest urban hospital<=14.08 miles (median)—Distance to nearest urban hospital > 14.08 miles—p-value.

Mean Distance to the nearest urban hospital (miles)—4.94—34.20—<0.0001.
Percent of patients admitted to urban hospitals (%)—99.4—77.07—<0.0001.
Age—68.89—72.02—<0.0001.
Male (%)—58.65—57.45—0.18.
Black (%)—1.95—0.08—<0.0001.
Number of secondary diagnoses—5.72—5.53—<0.0001.
Charlson comorbidity index—0.72—0.72—0.67.
APR-DRG risk index—0.07—0.07—0.48.
In-hospital mortality rate (%)—7.07—7.52—0.34.

Slide 22

Results—Table 5: Instrumental variable estimates of the difference of in-patient mortality between urban and rural hospitals

Table with these column headings: IV models (n=12,191—Number of groups for instrumental variable—F statistic instrumental variable—P-value for identifying restrictions tests—Coefficients—P-value.

Unadjusted—2—1540.16—n/a—0.0199—0.34.
Unadjusted—4—642.65v0.65—0.0269—0.16.
Unadjusted—12—184.31—0.13—0.0269—0.16.
Adjusted for demographic variables—2—1568.24—n/a—0.0127—0.58.
Adjusted for demographic variables—4—652.86—0.80—0.0081—0.69.
Adjusted for demographic variables—12—187.14—0.10—0.0065—0.75.
Adjusted for demographic variables and Charlson comorbidity index—2—1539.9—n/a—0.0090—0.69.
Adjusted for demographic variables and Charlson comorbidity index—4—642.51—0.92—0.0053—0.80.
Adjusted for demographic variables and Charlson comorbidity index—12—184.29—0.12—0.0040—0.84.
Index adjusted for demographic variables and APR—DRG risk—2—1694.27—n/a—0.0034—0.87.
Index adjusted for demographic variables and APR—DRG risk—4—640.61—0.92—0.0069—0.72.
Index adjusted for demographic variables and APR-DRG risk—12—202.50—0.01—0.0063—0.74.

Slide 23

Results: Sensitivity analyses

Repeat analyses in different samples:
- Excluding transferred in MI patients.
- Three-year state inpatient datasets (2001 to 2003).
Different IV estimation method:
- Two-stage residual inclusion method to account for the endogeneity in nonlinear (logistic) model.
- Bivariate Probit model (using Stata 9.0).
The results are consistent with IV estimation in Table 5.

Slide 24

Discussion

This study confirms earlier studies:
- MI patients admitted to rural hospitals were older and sicker than their urban counterparts.
- Traditional models all indicate significantly higher in-hospital mortality for those admitted to rural hospitals.

Slide 25

Discussion

Our findings suggest that the traditional logistic regression models are biased:
- Admissions to rural or urban hospitals are likely to be confounded by unmeasured patient variables.
- Referral patterns in rural hospitals.
  - Younger and less sick patients are transferred to urban hospitals.
  - The clinical judgment about transfer of rural senior patients with MI may rely on different criteria.

Slide 26

Discussion

Patient preferences are likely to play a significant role in transfer decisions for older MI patients.
- May reflect personal choice or existing serious comorbidities.
- Serious cases may choose to remain close to home.
- The transfer patterns may reflect rural doctors respecting their patients' wishes.
Using in-hospital MI mortality to measure quality of care in rural hospitals is problematic.

Slide 27

Limitations of the study

The results of the IV estimation can only be generalized to patients for whom distance affects their choice.
- The conclusion cannot be applied to MI patients bypassing rural hospitals and seeking care in urban hospitals.
The findings for hospitals in one State may not generalize to other States.
Analyses of in-hospital mortality rates may not generalize to mortality rates after hospitalization.

Slide 28

Conclusions

Mortality from MI in rural Iowa hospitals is not higher when controlled for unmeasured confounders.
Current risk-adjustment models may not be sufficient when assessing hospitals that perform different functions within the healthcare system.
Unmeasured confounding is a significant concern when comparing heterogeneous and undifferentiated populations.

Slide 29

Did conversion to Critical Access Hospital (CAH) status affect patient safety indicator performance?

Slide 30

Background

In order to protect small, financially vulnerable rural hospitals, the Medicare Rural Hospital Flexibility Program of the 1997 Balanced Budget Act allowed hospitals meeting certain criteria to convert to critical access hospitals (CAH).
This changed their Medicare reimbursement mechanism from prospective (PPS) to cost-based.
One objective of the policy was to increase the quality of care in these hospitals.

Slide 31

Timeframe for Conversion to CAH

Graph shows decrease in timeframe for Rural PPS hospitals conversion to CAH from 1997 through 2005.

Slide 32

Patient Safety

Cartoon from Buffalo News depicts three doctors in patient's hospital room, reading a report. The caption reads: "This report says medical errors such as indecipherable prescriptions cause the deaths of 98 patients a year, or is that 98,000?" It's hard to read this. In any case, we're supposed to report them, or is that repeat them?

Slide 33

4 PSIs and Composite

AHRQ recommends suppressing the estimates if fewer than 30 cases are in the denominator.
Only five patient safety indicators are able to provide PSI measures for all rural Iowa hospitals.
- PSI-5: foreign body left during procedure.
- PSI-6: iatrogenic pneumothorax.
- PSI-7: selected infections due to medical care.
- PSI-15: accidental puncture or laceration.
- PSI-16: transfusion reaction.
  - Too rare to provide variability to differentiate hospitals in Iowa.
A composite patient safety variable was created by summing the four PSIs (PSI-5, PSI-6, PSI-7, and PSI-15).

Slide 34

Number of Hospitals Having Better or Worse Performance after CAH Conversion

Graph shows better and worse performance of PSI-5, PSI-6, PSI-7, PSI-15, and a composite score of the 4 PSIs on a scale of 0-45.

Slide 35

Cross-sectional Analyses

Cross-sectional comparisons showed that CAHs had better performance than rural PPS hospitals on 4 of the 5 PSI measures.
However, the difference in patient safety indicators might be due to differences in patient mix, hospital characteristics besides CAH conversion, and differences in markets and environment.

Slide 36

Multivariable Analyses

We used multivariable Generalized Estimating Equations (GEE) models and sensitivity analyses to control for the impact of patient case mix, market variables, and time trend.
GEE models showed that CAH conversion was associated with significant better performance in PSI-6, PSI-7, PSI-15 and composite PSI.
Findings were robust among sensitivity analyses using different samples and different methods.

Slide 37

Conclusions

CAH conversion in rural hospitals resulted in enhanced performance in PSIs.
We speculate that the likely mechanism involved an increase in financial resources following CAH conversion to cost-based reimbursement for Medicare patients.

Slide 38

How did Critical Access Hospital conversion affect rural hospital financial condition?

Li, P., Schneider, J. E. & Ward, M. M., Effects of Critical Access Hospital Conversion on the Financial Performance of Rural Hospitals. Inquiry (in press).

Slide 39

Objectives

To study the effects of CAH conversion on Iowa rural hospitals' operating revenue, cost, and profit margin.

Slide 40

Study Sample and Study design

Sample
- Eight year (1997-2004) panel data for 89 Iowa rural hospitals (rural PPS hospitals and CAHs).
- Unit of analysis is hospital-year.
Study design
- Quasi-experimental designs that use both control groups and pretests.
- Panel data regression with fixed hospital effects.

Slide 41

Models

Ad hoc models:
- Revenue_it=f(CAH_it,P_jt,Y_it,X_it).
- Cost_it=f(CAH_it,W_jt,Y_it,X_it).
- Margin_it=f(CAH_it,W_jt, P_jt,Y_it, X_it).
Variables:
- CAH_it: hospital status (CAH or rural PPS) for i^th hospital in year t.
- P_it: output prices for i^th hospital in year t.
- W_it: input prices for i^th hospital in year t.
- Y_it: output volume for i^th hospital in year t.
- X_it: other variables for i^th hospital in year t that empirically affect dependent variables.

Slide 42

CAH variables

One dummy variable:
- CAH=1, if the hospital is in CAH status.
Three dummy variables:
- CAH1_it=1, if the hospital is in the first year of CAH status, otherwise CAH1_it=0.
- CAH2_it=1, if the hospital is in the second year of CAH status, otherwise CAH2_it=0.
- CAH3_it=1, if the hospital is in CAH status for more than 2 years, otherwise CAH3_it=0.
Comparison group: Rural PPS

Slide 43

Other covariates

P_it: output prices for i^th hospital in year t
- Medicare Part A (hospital) adjusted average per capita cost (AAPCC) as proxy of hospital output price (county level).
W_it: input prices for i^th hospital in year t
- Hourly wages for registered nurses (county level).
Y_it: output volume for i^th hospital in year t
- Total number of acute discharges, total number of outpatient visits, and average length of stay of acute discharges.
- The squared and cubed output measures and interaction terms will be included.

Slide 44

Other covariates

X_it: other variables for i^th hospital in year t that empirically affect dependent variables
- Hospital size (number of beds).
- Hospital case-mix.
  - Hospital mean DRG weight, percent of emergency visits, and percent of Medicare and Medicaid days among acute inpatient days.
- Variables reflecting the hospital market (we assumed the county to be the relevant geographic market of hospital services.).
  - Herfindahl-Hirschman Index (HHI), per capita income, and population density in the county in which the hospital is located.
- Year dummy variables which will adjust the effects of unmeasured, time-specific factors.
Revenue and expense functions were log transformed.

Slide 45

Data Sources

Iowa Hospital Association Profiles.
Iowa State Inpatient datasets.
Area Resource File.
Centers for Medicare and Medicaid Services.
American Hospital Association Annual Survey Database.
Bureau of Labor Statistics.

Slide 46

Result

Table shows changes in rural hospital patient care revenue, expense, and operating margin associated with CAH conversion, 1998-2004.

Slide 47

Table 2

Table shows changes in rural hospital patient care revenue, expense, and operating margin during the first, second and third plus years of CAH conversion, 1998-2004.

Slide 48

Results

Operating revenue:
- No change in the first year of conversion (paid an interim rate).
- Significant increases since the second year of CAH conversion.
Operating expenses:
- CAH conversion is associated with significant increase in hospital operating expenses.
- Hospitals increase expenses in the first year of conversion.
Operating Margin:
- Significant drop in the first year of conversion.
- Significant increase since the second year of conversion.
Sensitivity analyses showed similar results.

Slide 49

Conclusions

CAH conversion in rural hospitals resulted in better patient safety.
Rural hospital CAH conversion was associated with significant increases in hospital operating revenues, expenses and margins.

Slide 50

Summary: Limitations of measures

In-hospital mortality:
- Substantial unmeasured confounders.
Patient Safety Indicators:
- Only small number of indicators can be applied to rural hospitals.
- Changes of indicators might reflect changes in coding or reporting in administrative data.
We need hospital quality indicators specifically for rural hospitals.

Slide 51

Thank you
Questions?

Contact information:
- Pengxiang (Alex) Li
- University of Pennsylvania
- penli@mail.med.upenn.edu

Current as of January 2009

Internet Citation:

Hospital Quality Indicators in Iowa Rural Hospitals. Slide Presentation from the AHRQ 2008 Annual Conference (Text Version). January 2009. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/about/annualmtg08/091008slides/LiWard.htm