Chapter 7. Trending: Comparing Results Over Time

Many hospitals that have administered the hospital patient safety culture survey have indicated that they intend to re-administer the survey on a regular basis to track changes in patient safety culture over time. For the 2008 Comparative Database Report, some of the hospitals that previously administered the survey and submitted data for the 2007 report also submitted data for the 2008 report based on a follow-up survey of their staff. While the overall benchmarks presented earlier in this report reflect only the most recent survey data from all 519 participating hospitals, we have data from two administrations of the survey for 98 hospitals which allows us to examine trends over time for these hospitals. This chapter presents the results from trend analyses comparing patient safety culture survey results for these 98 hospitals since their previous administration. Changes in scores of 5 percent or greater are highlighted.

Highlights For the 98 hospitals with trend data, the average length of time between previous and most recent survey administrations was 14 months (range: 7 months to 19 months). The average change in percent positive scores between administrations on the patient safety culture composites was a slight increase of 2 percent (ranging from 0 to 3 percent change). 38 percent of trending hospitals increased by 5 percent or more on Overall Perceptions of Patient Safety, Frequency of Events Reported, and Nonpunitive Response to Error. 27 percent of hospitals decreased in percent positive scores by 5 percent or more on Staffing. The average percent of respondents giving their work area/unit a patient safety grade of “A-Excellent” and “B-Very Good” increased over time by 7 percent. The average percent of respondents reporting one or more events increased slightly over time by 3 percent.

When reviewing the results in this chapter, it is important to keep in mind that the trending results from these 98 hospitals represent only a small proportion of the total number of database hospitals, and therefore the trending data should be viewed as preliminary. In addition, there are a number of complex reasons why survey scores might change, or not change, over time. Important factors to consider are whether the hospital implemented patient safety initiatives between survey administrations and the length of the time period between administrations. Survey methodology issues can also play a big role in score changes. Low survey response rates for either the previous or most recent administration, changes in the number of staff asked to complete the survey, or changes in the types of staff asked to complete the survey, will make it difficult to interpret changes in scores over time. We provide descriptive information about some of the factors that may have affected changes in scores where possible.

Characteristics of the 98 Trending Hospitals

Table 7-1 displays summary statistics from the previous and most recent survey administrations from the 98 trending hospitals. As shown in the table, the average number of respondents increased in the most recent survey administration (from an average of 182 to 213 respondents). Overall average response rates were similar between previous and most recent administrations, but fewer hospitals indicated that they surveyed all staff or a sample of all staff in their most recent administration. Additional characteristics of the 98 hospitals are below:

• Most of the 98 trending hospitals (68 percent) administered the survey to the same types of staff in their previous and most recent administrations.
• The average change in response rate from the previous administration was -2 percent (range: one hospital had a decrease in response rate by 63 percent and one had an increase by 47 percent).
• The average length of time between previous and most recent survey administrations was 14 months (range: 7 months to 19 months).

As shown in Table 7-2, most of the trending hospitals (72 percent) were small hospitals with 99 beds or fewer. The 98 trending hospitals therefore disproportionately represent a much larger percentage of small hospitals compared to the 2008 database overall (55 percent have 99 beds or fewer) and compared to AHA hospitals (53 percent have 99 beds or fewer).

Tables 7-3 and 7-4 show that most of the 98 trending hospitals were non-teaching (87 percent) and that they were about equally divided between government and non-government ownership. Again, these distributions vary when compared to the 2008 database overall (74 percent non-teaching and 24 percent government-owned) and when compared to AHA hospitals (77 percent non-teaching and 26 percent government-owned). Therefore, the trending hospitals disproportionately represent a larger percentage of non-teaching hospitals and a larger percentage of government-owned hospitals.

Description of Trending Statistics

Before presenting results on the changes in survey scores over time, we provide an explanation of the trending statistics that are presented. Table 7-5a shows examples of the statistics shown in this chapter. The tables show the average percent of respondents who answered positively in the most recent survey administration (left column) and the previous administration (middle column) for the 98 trending hospitals only. The change over time [Most Recent score minus (-) Previous score] is shown in the right column as a negative number if the most recent administration showed a decline, or a positive number if the most recent administration showed an increase.

Table 7-5b shows additional trending statistics that are provided. The maximum increase and maximum decrease show the scores for the hospitals with the largest average percent positive score increase and the hospitals with the largest decrease. The average increase and decrease of percent positive scores across the 98 trending hospitals is also shown. The average increase was calculated by only including hospitals that had an increase in their most recent score; hospitals that showed no change or decreased were not included when calculating the average increase. Similarly, the average decrease was calculated by only including hospitals that had a decrease in their most recent score; hospitals that showed no change or increased were not included when calculating the average decrease.

The pie charts in Charts 7-1, 7-2, and 7-3 show the percent of hospitals that increased or decreased 5 percent or more on the composites, patient safety grades, and events reported respectively.  The percent of hospitals that increased or decreased less than 5 percent are represented as “Did not change.”

Composite and Item-level Trending Results

Table 7-6 presents trending results showing average percent positive scores on each of the 12 patient safety culture composites from the 98 trending hospitals. Percent positive scores for their most recent and previous data administration/submission are shown, and so are the change over time, the hospital scores with the maximum increase and maximum decrease, and the average increase and decrease over time across the 98 hospitals. Table 7-6 also shows that there was a slight overall increase in the average change in percent positive scores over time on the patient safety culture composites (average 2 percent, ranging from 0 to 3 percent change). For hospitals with increases in scores over time, average increases ranged from 5 to 8 percent. For hospitals with decreases in scores, average decreases ranged from -4 to -6 percent.

The item-level trending results in Table 7-7 show that the average change in item-level percent positive scores over time on the patient safety culture items ranged from a decrease of 2 percent to an increase of 3 percent. For hospitals with increases in item scores over time, average increases ranged from 5 to 11 percent. For hospitals with decreases in item scores, average decreases ranged from -5 to -9 percent.

Trending results from the item that asks respondents to give their hospital work area/unit an overall grade on patient safety are shown in Table 7-8. The average percent of respondents giving their work area/unit a patient safety grade of “A-Excellent” and “B-Very Good” increased over time by 7 percent.

Trending results from the item that asked respondents to indicate the number of events they had reported over the past 12 months are shown in Table 7-9. The average percent of respondents reporting one or more events increased slightly over time by 3 percent.

Pie Charts of Trending Results

The pie charts in Chart 7-1 show the percentages of hospitals that increased, decreased, or did not change by 5 percent or more on the 12 patient safety culture composites. Thirty-eight percent (38 percent) of trending hospitals increased by 5 percent or more on Overall Perceptions of Patient Safety, Frequency of Events Reported, and Nonpunitive Response to Error. Twenty-seven percent (27 percent) of hospitals decreased in their percent positive scores by 5 percent or more on Staffing.

Chart 7-2 shows that similar percentages of hospitals either increased in the percent of respondents providing patient safety grades of “A-Excellent” or “B-Very Good” (44 percent increased), or did not change (41 percent of hospitals did not change or changed less than 5 percent). Only 14 percent of the hospitals decreased by 5 percent or more.

Chart 7-3 shows that similar percentages of hospitals either increased in the percent of respondents reporting one or more events (38 percent), or did not change (42 percent of hospitals did not change or changed less than 5 percent). Twenty percent (20 percent) of the hospitals had a decrease of 5 percent or more.

Additional Trending Analyses

To explore potential reasons why some hospitals had increases in their patient safety culture scores over time, we examined the relationship between hospital characteristics—such as bed size, ownership, and teaching status—and changes in patient safety culture scores over time. Relationships were examined by calculating correlations between changes in patient safety culture scores and hospital characteristics. Correlations are a type of statistic that convey the extent to which two variables have a linear relationship. Correlations range from a low of 0 to a high of 1.00 and can be either positive or negative. The closer the correlation is to 1.00 (or -1.00), the greater the degree of association between the variables.

Teaching status and ownership were not found to be related to changes in scores, however bed size was moderately related. The smaller the hospital bed size, the greater the number of patient safety culture composites that increased by at least 5 percent (correlation: r = -.33, p < .05). In addition, smaller hospitals had greater average overall change across their composite scores compared to larger hospitals (correlation: r = -.29, p < .05). Smaller hospitals also had greater increases in their composite scores on Overall Perceptions of Patient Safety compared to larger hospitals (correlation: r = -.24, p < .05). It is important to keep in mind that these findings are very preliminary; particularly given that the trending hospitals disproportionately represent a much larger percentage of small hospitals (72 percent had 99 or fewer beds).

We also examined whether hospitals that improved on nonpunitive response to error also had increases in the number of respondents who reported at least one event in the past 12 months. This finding was supported; hospitals that increased their percent positive score on nonpunitive response to error also tended to have an increase in the number of respondents who reported at least one event in the past 12 months (correlation: r = .20, p < .05).

Appendix C:  Trending Results by Respondent Characteristics

Part III of the report contains Appendix C that shows trends over time for the 98 hospitals that administered the survey and submitted data twice. Average percent positive scores from the most recent and previous administrations are shown on the survey composites and items, broken down by the following respondent characteristics:

• Appendix C: Trending Results by Respondent Characteristics
  • 1-Work area/unit.
  • 2-Staff position.
  • 3-Interaction with patients.

Highlights of the findings from the breakout tables in this appendix are provided on the following page.

Note: Because there were fewer than 20 hospitals in most hospital breakout categories, trending results are not shown by hospital characteristics to ensure hospital confidentiality.

Highlights from Appendix C: Trending Results by Respondent Characteristics Respondent Work Area/Unit (Tables C-1, C-3, C-4) Respondents in Pediatrics had the largest increases in positive response over time on 9 of the 12 patient safety culture composites (average increase across the 9 composites was 11 percent). Respondents in Anesthesiology had the largest decreases in positive response over time on 7 of the 12 patient safety culture composites (average decrease across the 7 composites was 13 percent). Pediatrics had the largest average percent of respondents who increased over time in giving their work area/unit a patient safety grade of “Excellent” or “Very good” (a 21 percent increase—from 60 to 81 percent); Anesthesiology had the largest decrease over time (a 7 percent decrease—from 89 to 82 percent) Anesthesiology had the largest average percent of respondents who increased over time in their reporting of one or more events in the past year (a 13 percent increase: from 35 to 48 percent); the largest decrease in percent reporting was in Obstetrics (an 11 percent decrease from 65 to 54 percent). Respondent Staff Position (Tables C-5, C-7, C-8) Dieticians had the largest decreases in positive response over time on 7 of the 12 patient safety culture composites (average decrease across the 7 composites was 8 percent). Pharmacists had the largest average percent of respondents who increased over time in giving their work area/unit a patient safety grade of “Excellent” or “Very good” (a 15 percent increase—from 67 to 82 percent); Dieticians had the largest decrease over time (a 9 percent decrease—from 83 to 74 percent). Pharmacists had the largest average percent of respondents who increased over time in their reporting of one or more events in the past year (a 12 percent increase from 69 to 81 percent); the largest decrease in percent reporting was with Therapists (Respiratory, Physical, Occupational, Speech) (a 5 percent decrease from 43 to 38 percent). Respondent Interaction With Patients (Tables C-9, C-11, C-12) There were no noticeable composite differences over time based on respondent interaction with patients (all were increases over time of 3 percent or less). There were no noticeable differences in patient safety grade over time based on respondent direct patient interaction (difference was 1 percent; a 7 percent increase for respondents with direct interaction and a 6 percent increase for respondents without direct interaction). There were no noticeable differences in the number of events reported over time based on respondent interaction with patients (difference was 3 percent; a 5 percent increase for respondents with direct interaction and a 2 percent increase for respondents without direct interaction).

Return to Contents
Proceed to Next Section