Notes: Description of Data Cleaning and Calculations
This section provides additional detail regarding how various statistics
presented in this report were calculated.
Data Cleaning
Each participating hospital was asked to submit cleaned,
individual-level survey data. However, as an additional check, once the
data were submitted, response frequencies were run on each hospital's
data to look for out-of-range values, missing variables, or other data
anomalies. When data problems were found, hospitals were contacted and
asked to make corrections and resubmit their data. In addition, each
participating hospital was sent a copy of their data frequencies for the
hospitals to verify that the data set received was correct.
New: In order to keep the database current, data more than 3½ years old
are removed from the database. Thus, 65 hospitals that administered the
survey prior to January 1, 2006, were dropped from the database.
Response Rates
As part of the data submission process, hospitals were asked to provide
their response rate numerator and denominator. Response rates were
calculated using the formula below.
Response Rate =
Number of complete, returned surveys
Number of surveys distributed - Ineligibles
Numerator = Number of complete, returned surveys. The
numerator equals the number of individual survey records submitted to the
database. It should exclude surveys that were returned blank on all
non-demographic survey items, but include surveys where at least one
non-demographic survey item was answered.
Denominator = The total number of surveys distributed
minus ineligibles. Ineligibles include deceased individuals or those who
were not employed at the hospital during data collection.
As a data cleaning step, we examined whether any individual survey
records submitted to the database were missing responses on all of the
nondemographic survey items (indicating the respondent did not answer any
of the main survey questions). Records where all nondemographic survey
items were left blank by the respondent were found (even though these blank
records should not have been submitted to the database). We therefore removed
these blank records from the larger data set and adjusted any affected
hospital's response rate numerator and overall response rate
accordingly.
Calculation of Percent Positive Scores
Most of the survey's items ask respondents to answer using 5-point
response categories in terms of agreement (Strongly agree, Agree, Neither,
Disagree, Strongly disagree) or frequency (Always, Most of the time,
Sometimes, Rarely, Never). Three of the 12 patient safety culture
composites use the frequency response option (Feedback and
Communication About Error, Communication Openness, and Frequency of Events
Reported). The other nine composites use the agreement response
option.
Item-Level Percent Positive Response
Both positively worded items (such as "People support one another
in this work area") and negatively worded items (such as "We
have patient safety problems in this work area") are included in the
survey. Calculating the percent positive response on an item is different
for positively and negatively worded items:
- For
positively worded items, percent positive response is the
combined percentage of respondents within a hospital who answered
"Strongly agree" or "Agree," or
"Always" or "Most of the time," depending on
the response categories used for the item.
For example, for the item "People support one another in this
work area," if 50 percent of respondents within a hospital Strongly
agree and 25 percent Agree, the item-level percent
positive response for that hospital would be 50% + 25%= 75% positive.
- For
negatively worded items, percent positive response is the
combined percentage of respondents within a hospital who answered
"Strongly disagree" or "Disagree," or
"Never" or "Rarely," because a negative
answer on a negatively worded item indicates a positive
response.
For example, for the item "We have patient safety problems in
this work area," if 60 percent of respondents within a hospital Strongly
disagree and 20 percent Disagree, the item-level percent
positive response would be 80 percent positive (i.e., 80 percent of
respondents do not believe they have patient safety problems
in their work area).
Composite-Level Percent Positive Response
The survey's 42 items measure 12 areas or composites of patient
safety culture. Each of the 12 patient safety culture composites includes 3
or 4 survey items. Composite scores were calculated for each hospital by
averaging the percent positive response on the items within a composite.
For example, for a 3-item composite, if the item-level percent positive
responses were 50 percent, 55 percent, and 60 percent, the hospital's
composite-level percent positive response would be the average of these
three percentages or 55 percent positive.4
Item and Composite Percent Positive Scores
To calculate your hospital's composite score, average the
percentage of positive response to each item in the composite. Here is an
example of computing a composite score for Overall Perceptions of
Patient Safety:
- There are four items
in this composite—two are positively worded (items A15 and A18)
and two are negatively worded (items A10 and A17). Keep in mind that
DISAGREEING with a negatively worded item indicates a POSITIVE
response
- Calculate the
percentage of positive responses at the item level (an example is in
Table 1).
In this example, there were 4 items with percent positive response
scores of 46 percent, 52 percent, 46 percent, and 56 percent. Averaging these
item-level percent positive scores results in a composite score of .50 or
50 percent on Overall Perceptions of Patient Safety. In this example, an
average of about 50 percent of the respondents responded positively on the
survey items in this composite.
Once you calculate your hospital's percent positive response on
each of the 12 safety culture composites, you can compare your results with
the composite-level results from the 885 database hospitals.
Minimum Number of Responses
New to the 2010 database report, we enacted several new rules regarding
a minimum number of responses for calculating the percent positive scores.
First, we only calculated percent positive scores for hospitals that had at
least 10 completed surveys. Second, item-level results were only calculated
when there were at least three responses to the item. If a hospital had
fewer than three responses to a survey item, the hospital's score for
that item was set to missing. Third, if a hospital had fewer than five
respondents in a breakout category (e.g, work area/unit, staff position,
direct interaction with patients), no statistics were calculated for that
breakout category (i.e., all scores were set to missing). For example, if a
hospital had five respondents indicating they worked in the Anesthesiology
unit and four respondents indicating they worked in Pharmacy, that hospital
would be included in the statistics displayed for Anesthesiology units but
not in those displayed for Pharmacy units. These minimums also apply to the
statistics displayed in Appendixes B and D (results by respondent
characteristics).
Percentiles
Percentiles were computed using the SAS® software default method.
The first step in this procedure is to rank order the percent positive
scores from all the participating hospitals, from lowest to highest. The
next step is to multiply the number of hospitals (n) by the percentile of
interest (p), which in our case would be the 10th, 25th, 50th, 75th or 90th
percentile.
For example, to calculate the 10th percentile, one would multiply 885
(the total number of hospitals) by .10 (10th percentile). The product
of n x p is equal to "j+g" where "j" is the integer
and "g" is the number after the decimal. If
"g" equals 0, the percentile is equal to the percent positive
value of the hospital in the jth position plus the percent positive value
of the hospital in the jth +1 position, divided by two [(X(j) +
X(j+1))/2]. If "g" is not>
equal to 0, the percentile is equal to the percent positive value of the
hospital in the jth +1 position.
The following examples show how the 10th and 50th percentiles would be
computed using a sample of percent positive scores from 12 hospitals (using
fake data shown in Table 2). First, the percent positive scores are sorted
from low to high on Composite "A."
10th percentile
- For the 10th percentile,
we would first multiply the number of hospitals by .10 (n x p =
12 x .10 = 1.2).
- The product of n x p
= 1.2, where "j" = 1 and "g" = 2. Since
"g" is not equal to 0, the 10th
percentile score is equal to the percent positive value of the hospital
in the jth +1 position:
- "j"
equals 1.
- The 10th
percentile equals the value for the hospital in the 2nd
position = 48 percent.
50th Percentile
- For the 50th
percentile, we would first multiply the number of hospitals by .50: (n
x p = 12 x .50 = 6.0).
- The product of n x p
= 6.0, where "j" = 6 and "g" = 0.
Since"g" = 0, the 50th percentile score is equal to the
percent positive value of the hospital in the jth position
plus the percent positive value of the hospital in the jth
+1 position, divided by two:
- "j"
equals 6.
- The 50th
percentile equals the average of the hospitals in the 6th and 7th
position (64%+66%)/2 = 65%.
4 Note that this method for calculating
composite scores is slightly different from the method described in the
September 2004 Survey User's Guide that is part of the original
survey toolkit materials on the AHRQ Web site. The guide advises computing
composites by calculating the overall percent positive across all the items
within a composite. The updated recommendation included in this report is
to compute item percent positive scores first, and then average the item
percent positive scores to obtain the composite score, which gives equal
weight to each item in a composite. The Survey User's Guide will
eventually be updated to reflect this slight change in methodology.
Return to Contents
Proceed to Next Section