Notes:
Description of Data Cleaning and Calculations
This section provides additional detail about how various
statistics presented in this report were calculated.
Data Cleaning
Each participating medical office was asked to submit
cleaned, individual-level survey data. As an additional check, once the data
were submitted, response frequencies were automatically run on each medical
office's data to find out-of-range values, missing variables, or other data
anomalies. When data problems were found, data submitters were required to make
corrections and resubmit their data. Each submitter was shown a copy of their
data frequencies to verify that the data set received was correct. Missing responses
and "Does Not Apply" or "Don't Know" responses are not part of the results.
Response Rates
As part of the data submission process, medical offices were
asked to provide their response rate numerator and denominator. Response rates
were calculated using the formula below.
Response
Rate = Number of complete, returned surveys / Number
of surveys distributed − Ineligibles
Numerator = Number of complete, returned surveys. The
numerator equals the number of individual survey records submitted to the
database. It excludes surveys that were returned blank on all nondemographic
survey items but includes surveys where at least one nondemographic
survey item was answered.
Denominator = The total number of surveys distributed
minus ineligibles. Ineligibles include deceased individuals or those who were
no longer employed at the medical office during data collection.
As a data cleaning step, we examined whether any individual
survey records submitted to the database were missing responses on all of the
nondemographic survey items (indicating that the respondent did not answer any
of the main survey questions). Records where all nondemographic survey items
were missing were excluded from the medical office's numerator. Medical offices
were included in the database only if they had a numerator of at least 5 after
this data cleaning step.
Response Categories
Most of the survey's items ask respondents to answer using
5-point response categories in terms of agreement (Strongly agree, Agree,
Neither, Disagree, Strongly disagree) or frequency (Always, Most of the time,
Sometimes, Rarely, Never). Three of the 10 patient safety culture composites,
consisting of 12 items, use the frequency response option (Communication
Openness, Patient Care Tracking/Follow-up, and Communication About
Error).
The 13 noncomposite items use 6-point frequency response categories.
The nine Patient Safety and Quality Issues items use a frequency scale ranging
from "Not in the past 12 months" to "Daily" (Not in the past 12 months, Once or
twice in the past 12 months, Several times in the past 12 months, Monthly,
Weekly, Daily). The four Information Exchange With Other Settings items use
similar response options ranging from "No problems in the past 12 months" to "Problems
daily" (No problems in the past 12 months, Problems Once or twice in the past
12 months, Problems several times in the past 12 months, Problems monthly,
Problems weekly, Problems daily).
Item-Level Percent
Positive Response
Both positively worded items (such as "Staff support one
another in this medical office") and negatively worded items (such as "Staff use
shortcuts to get their work done faster") are included in the survey.
Calculating the percent positive response on an item is different for
positively and negatively worded items:
-
For positively worded items with 5-point response scales, percent
positive response is the combined percentage of respondents within a medical
office who answered "Strongly agree" or "Agree," or "Always" or "Most of the
time," depending on the response categories used for the item.
For example, for the item "We have
enough staff to handle our patient load," if 50 percent of respondents within a
medical office responded Strongly agree and 25 percent responded Agree,
the item-level percent positive response for that medical office would be 50% +
25%= 75% positive.
-
For negatively worded items, percent positive response is
the combined percentage of respondents within a medical office who answered
"Strongly disagree" or "Disagree," or "Never" or "Rarely," because a negative
answer on a negatively worded item indicates a positive response.
For example, for the item "Mistakes
happen more than they should in this office," if 60 percent of respondents
within a medical office responded Strongly disagree and 20
percent responded Disagree, the item-level percent positive response
would be 80 percent (i.e., 80 percent of respondents do not believe mistakes
happen more than they should in this office).
Percent positive scores for the Patient Safety and Quality
Issues items, as well as the Information Exchange With Other Settings items,
were calculated differently than the other survey items. The percent positive
score for these 13 items are the sum of the three response options that
represent the smallest frequency of occurrence. For Patient Safety Quality
Issues items these are not in the past 12 months, once or twice in the past 12
months, and several times in the past 12 months. For Information Exchange With
Other Settings items, the three responses are no problems in the past 12
months, problems once or twice in the past 12 months, and problems several
times in the past 12 months.
Composite-Level Percent Positive Response
The survey's 51 items measure 10 areas or composites of
patient safety culture, information exchange with other settings, and patient
safety and quality issues. The 10 patient safety culture composites include three
or four survey items. Composite scores were calculated for each medical office by
averaging the percent positive response on the items within a composite. For
example, for a three-item composite, if the item-level percent positive
responses were 50 percent, 55 percent, and 60 percent, the medical office's
composite-level percent positive response would be the average of these three
percentages, or 55 percent positive.
Item and Composite Percent Positive Scores
To calculate your medical office's composite score, average the
percentage of positive response to each item in the composite. Here is an
example of computing a composite score for Staff Training:
- This composite has three items. Two are positively worded (items #C4 and
#C7) and one is negatively worded (item #C10). Keep in mind that DISAGREEING
with a negatively worded item indicates a POSITIVE response.
- Calculate the percentage of positive responses at the item level (go to
example in Table 1).
This example includes three items, with percent positive
response scores of 46 percent, 56 percent, and 48 percent. Averaging these
item-level percent positive scores results in a composite score of .50 or 50
percent on Staff Training. In this example, an average of about 50 percent of
the respondents responded positively to the survey items in this composite.
Once you calculate your medical office's percent positive
response for each of the 10 patient safety culture composites, you can compare
your results with the composite-level results from the 934 database medical
offices.
Percentiles
Percentiles were computed using the SAS® software
default method. The first step in this procedure is to rank order the percent
positive scores from all the participating medical offices, from lowest to
highest. The next step is to multiply the number of medical offices (n) by the
percentile of interest (p), which in our case would be the 10th, 25th,
50th, 75th, or 90th percentile.
For example, to calculate the 10th percentile,
one would multiply 934 (the total number of medical offices) by .10 (10th
percentile). The product of n x p is equal to "j+g" where "j" is the integer
and "g" is the number after the decimal. If "g" equals 0, the percentile is
equal to the percent positive value of the medical office in the jth
position plus the percent positive value of the medical office in the jth
+1 position, divided by 2 [(X(j) + X(j+1))/2]. If "g" is not
equal to 0, the percentile is equal to the percent positive value of the medical
office in the jth +1 position.
The following examples show how the 10th and 50th
percentiles would be computed using a sample of percent positive scores from 12
medical offices (using fake data shown in Table 2). First, the percent positive
scores are sorted from low to high on Composite "A."
10th percentile
-
For the 10th percentile, we would first multiply the number
of medical offices by .10:
(n x p = 12 x .10 = 1.2).
-
The product of n x p = 1.2, where "j" = 1 and "g" = 2. Since "g" is not
equal to 0, the 10th percentile score is equal to the percent
positive value of the medical office in the jth +1 position:
- "j" equals 1.
- The 10th percentile equals the value for the medical office in
the 2nd position = 48%.
50th percentile
-
For the 50th percentile, we would first multiply the number
of medical offices by .50:
(n x p = 12 x .50 = 6.0).
-
The product of n x p = 6.0, where "j" = 6 and "g" = 0. Since "g" = 0, the
50th percentile score is equal to the percent positive value of the medical
office in the jth position plus the percent positive value of the medical
office in the jth +1 position, divided by 2:
-
"j" equals 6.
- The 50th percentile equals the
average of the medical offices in the 6th and 7th positions
(64%+66%)/2 = 65%.
Return to Contents
Proceed to Next Section