Chapter 4. Establishing Data Collection Procedures
Once you have decided how you want the surveys distributed and returned and have established a medical office point of contact (POC), you need to make several decisions regarding your data collection procedures. This chapter includes strategies for publicizing your survey and maximizing your response rate. It also outlines methods for tracking responses and collecting data.
We strongly recommend publicizing the survey before and during data collection. Publicity may include posting flyers or posters in the medical office, promoting the survey during staff meetings, and, if possible, sending staff E-mails and posting information about the survey on an office Intranet. Be sure to advertise that the survey is supported by medical office or health system executive leaders, or both.
If you sponsor the survey in multiple medical offices in a larger practice or health care system, practice or system leaders can promote the survey in any meetings attended by POCs in the participating medical offices. After the survey is underway, if response rates are low in some offices, it may be useful to have medical office POCs share their strategies for promoting the survey with others. Also, physician response rates typically are lower than those of staff. Midway through data collection, you might consider having the senior executive in the office, practice, or health system send a special E-mail to all physicians in participating offices, thanking them if they have completed the survey and encouraging the others to do so.
If you have publicized your survey well and your survey cover letter includes important information about the purposes of the survey, we think sending a prenotification letter announcing the upcoming survey is optional. However, if you conduct a Web-based survey (where sending E-mail messages is easy) or you are unsure that all providers and staff know about the upcoming survey, you may want to send a prenotification letter. The letter should be signed by your medical office or health system senior executive, or both. If an outside vendor is handling the data collection duties, use the letter as an opportunity to introduce the vendor.
Return to Contents
We recommend following a basic data collection approach to achieve high response rates. Achieving a high response rate is important for drawing valid generalizations about your medical office from the survey data you collect. Surveys are used to infer something about a particular population. There must be enough survey respondents to accurately represent the medical office before you can legitimately present your survey results as a reflection of your medical office's safety culture.
If your response rate is low, there is a danger that the large number of staff who did not respond to the survey would have answered very differently from those who did respond. Therefore, an overall response rate of 50 percent or more should be your minimum goal. The higher the response rate, the more confident you can be that you have an adequate representation of the staff's views. The approach we recommend involves carrying out the following steps in the order presented:
- First survey. About 1 week after you have begun publicizing the survey (or several days after sending a prenotification letter if you decide to do that), distribute a survey packet to each provider and staff member in your medical office. The packet should include the survey, a supporting cover letter, and a return envelope that the respondent can seal. If you want staff to return their surveys by mail, rather than returning them to the medical office POC or dropping them in a special box in the office, include a preaddressed postage-paid envelope to make it easy for respondents to return their surveys.
- Reminder card or letter. One week after distributing the survey, distribute a reminder card or letter to providers and staff, thanking those who have already responded and reminding others to please respond. Reminders can also be sent by E-mail to employees with individual work E-mail addresses they can access from their office computers. If your medical office is small, you may choose to have the POC remind staff members simply by speaking to them.
- Second survey. The next step is to distribute a second survey to everyone in your medical office:
- Single small- or medium-sized medical offices: Distribute the second survey 1 week after distributing the reminder notice.
- Multiple or large medical offices: Distribute the first reminder 2 weeks after the first survey and distribute the second survey 2 weeks after distributing the first reminder.
Include a cover letter in the second survey packet thanking those who have already responded and reminding others to please complete the second survey. (If you have chosen to put individual identification numbers or identifiers on your survey, you can distribute second surveys only to nonrespondents.)
- Data collection closeout. One week after distributing the second survey, end data collection and begin preparing for data cleaning and analysis.
Consider Using Incentives To Maximize Response Rates
Offering incentives can be a good way to increase responses to a survey because respondents often ask, "What's in it for me?" You may want to offer individual incentives, such as a raffle for cash prizes or gift certificates, or you can offer group incentives, such as catered lunches for medical offices with at least a 75 percent response rate. Be creative and think about what would motivate your staff to complete the survey. |
Return to Contents
Whether or not you are surveying a single medical office or multiple medical offices, you need to decide whether to use individual respondent identification numbers or identifiers on your surveys. In addition, if you are surveying multiple medical offices, you need to decide how to track returned surveys by office. We provide recommendations and discussion regarding these issues. First, we discuss our recommendation regarding the use of individual identifiers; then we discuss options for tracking surveys by office. Finally, we provide important guidelines that must be followed if you choose to use individual identifiers.
Individual Identifiers
Whether you administer the survey in a single medical office or in multiple medical offices, we recommend that you conduct an individually anonymous survey. Do not use individual identifiers (usually a number or code) on the survey to track survey respondents and nonrespondents. Also, do not ask respondents to provide their names on completed survey forms. The advantage of individually anonymous surveys is that fewer respondents will refuse to participate because of concerns that identifiers will be used to figure out their individual responses to the survey. Understand that confidentiality concerns are even stronger in smaller medical offices. You want to ensure that respondents feel comfortable reporting their true perceptions and confident that their answers cannot be traced back to them.
If you decide not to use identifiers, you will need to distribute reminder notices and second surveys to all eligible staff, with instructions to disregard the reminder and second survey if the first survey was completed and mailed. You may receive phone calls from respondents who completed and returned their survey, wondering why they received followup materials. You can tell them that you have no information about who responded, so followup materials were distributed to everyone. You can tell them to disregard the materials and then remove their names from further followup. You will run a small risk that someone may complete and return more than one survey.
Office Identifiers
If you are surveying multiple medical offices, you will want to be able to produce feedback reports for each office. Therefore, you need to be able to identify which returned surveys came from which medical office. We offer a few ways of doing that. Our suggestions vary depending on the number of offices you are surveying and your preferences regarding how respondents will return their surveys.
Vary survey color. If the number of offices you are surveying is not too large, you can print the survey on different colored paper for each office. Then respondents can return the survey either within the office or to a headquarters or vendor address and their surveys will still be identifiable by office.
Restrict options for returning surveys.You can instruct respondents to return their surveys in sealed envelopes to their medical office POC or to drop boxes in the medical office. The POC can then batch the returned surveys, put them in a package that has the office name or an office identifier on it, and send the package to a designated project leader or to a designated vendor. Under this option, some staff may not participate in the survey because they would prefer, for reasons of confidentiality, to send their completed surveys directly to someone outside their medical office.
Use a medical office identifier. You can include a medical office identifier on the surveys when they are printed by giving each medical office a unique form number as an identifier. The form number would be the same for all surveys in each medical office but would differ across medical offices. For example, if you are surveying three medical offices, you would use Form 1, Form 2, and Form 3 to identify these medical offices. Print the identifier in an unobtrusive location on the survey (e.g., lower left corner of the back page). Because the words "Form 1,", "Form 2," etc., are part of the printed document, they do not stand out. Be aware, however, that some respondents will be so concerned about the confidentiality of their responses that they might mark out the medical office identifier.
Return to Contents
Guidelines When Using Individual Identifiers
You may decide, particularly if your medical office is quite large or you are surveying multiple medical offices, that the advantages of using individual identifiers outweigh the advantages of individually anonymous surveys. For example, if you use individual identifiers to track response, you can distribute reminders and followup materials only to nonrespondents, thus reducing your costs and eliminating the possibility of someone completing more than one survey. However, it is possible that some respondents will de-identify their own surveys by removing or marking out the identifiers anyway.
If you use individual identifiers, you must adhere to careful procedures to protect the confidentiality of the information linking individual staff names to the identification numbers or codes. You will need to ensure that only key project personnel have access to information linking individual names or groups to the identification numbers or codes. You will also need to destroy all information linking names to identifiers at the end of data analysis. |
Return to Contents
The following materials will need to be developed and assembled in preparation for survey distribution. Personalizing outer envelopes and letters (e.g., addressed to "Dear John Doe") sometimes promotes higher response rates (Dillman, 2007). Care should be taken, however, to prevent names from appearing on the actual survey forms.
Office/Practice/System Point-of-Contact Letter
You should send a letter to any persons designated as an office or system-level POC describing the purposes of the survey and explaining their role in the survey effort. Print the letter on official letterhead, dated with month/year, signed by the medical office senior executive. If a medical office practice or system is sponsoring the survey in multiple offices, the letter could be signed by a practice or system senior executive or by both the medical office senior executive and a practice or system senior executive.
We also recommend that you provide the POC with a simple data collection protocol that describes POC tasks, along with a proposed timeline. (Select for a sample data collection protocol in Appendix A for a single, medium-sized medical office. The schedule can be adjusted if your office is small or large or you are surveying multiple medical offices.) In most medical offices, POCs will be busy with their regular responsibilities. The protocol will help them stay on schedule.
Publicity Materials
We recommend preparing and posting informational flyers or posters in your medical office and sending E-mail notices when possible to publicize the upcoming survey. Your publicity materials can help legitimize the survey effort and increase your response rate by including some or all of the following types of information:
- Endorsements of the survey from your medical office leaders
- Clear statements about the purpose of the survey (to measure staff attitudes and opinions about patient safety in their medical office) and how the collected data will be used (to identify ways to improve safe patient care and quality of care)
- Assurances that only summary data will be reported, thus keeping individual responses confidential
- Introductions to the survey vendor, if you have chosen to use a vendor
- Contact information for the designated POC in the medical office
Cover Letter in First Survey Packet
The cover letter that is included in the first survey packet should be on official medical office letterhead and should address the following points:
- Why the medical office is conducting the survey and how survey responses will be used.
- How much time is needed to complete the survey.
- Assurances that the survey is voluntary and can be completed during work time.
- Confidentiality or anonymity assurances.
- Suggested reply timeframe and how to return completed surveys.
- Incentives for survey participation (optional).
- Contact information for the medical office POC (and practice/system-level POC, if applicable).
In the cover letter, or on the survey form, ask staff to complete the survey within 7 days, but do not print an actual deadline date on the letter or survey. Sometimes data collection schedules get delayed, and you do not want to reprint letters or surveys because they are outdated. In addition, sometimes people will not complete a survey if they notice that it is beyond the deadline date.
Sample Cover Letter Text
The enclosed survey is part of our medical office's efforts to better address patient safety and the quality of care for our patients. All providers and staff in the medical office are receiving this survey. It will take about 10 to 15 minutes to complete, and your individual responses will be kept confidential. Only group statistics, not individual responses, will be prepared and reported.
Please complete your survey and return it WITHIN THE NEXT 7 DAYS. When you have completed your survey, please [provide return instructions]. [Optional incentive text: In appreciation for participation, staff who complete and return their surveys will receive (describe incentive).]
Please contact [POC name and job position] if you have any questions [provide phone number and E-mail address]. Thank you in advance for your participation in this important effort |
Reminder Notice
Medical office POCs can distribute reminder notices (e.g., a reminder message on a half-page of cardstock) after the first survey administration to all office providers and staff (if the survey is anonymous). The notice should ask them to please complete and return their surveys and should include a thank you to those who have done so already. If you are using identifiers to track responses, you can distribute the notices to nonrespondents only. In small medical offices, POCs may opt to remind providers and staff simply by talking to them individually or in a group.
Cover Letter in Second Survey Packet
The contents of the second survey cover letter should be similar to the first cover letter but should have a different beginning. For example, if you are conducting an anonymous survey, you will have to distribute second surveys to everyone, so you might say: "About X days ago a copy of the Medical Office Survey on Patient Safety was distributed to you and other staff at your medical office. If you have already returned a completed survey, thank you very much and please disregard this second survey packet."
If you are using individual identifiers, you will be able to identify respondents and remove them from the list of staff receiving followup materials, so you might begin as follows: "About X days ago a copy of the Medical Office Survey on Patient Safety was distributed to you and other staff at your medical office. Because we have not yet received a completed survey from you, we are enclosing a second copy of the survey (if you recently returned your survey, thank you and please disregard this second survey packet)."
Surveys
If you are conducting an anonymous survey and plan to send second surveys to everyone, print at least twice the number of surveys as staff in your sample. If you are tracking responses and will send second surveys only to nonrespondents, you may print fewer surveys. For example, if there is a total of 20 providers and staff in your medical office and your medical office's survey response history typically results in a 40 percent response to the first survey, you would need to print 20 first surveys and 12 second surveys (20 staff x 60% nonrespondents = 12), for a total of 32 printed surveys.
Labels
You will need labels for the outside of each survey packet. If the surveys are distributed within the medical office, the labels can include only the names of providers or staff members in the medical office. Even if the survey itself is completed anonymously, it is a good idea to label a survey packet with the name of each staff member. Self-addressed return labels should be used on return envelopes. Labels also may be used to place identifiers on the surveys if you have chosen to use identifiers.
Envelopes
You will need a set of outer envelopes to distribute the surveys and a set of return envelopes that can be sealed for the return of completed surveys. Preprint the return address on the return envelopes (or use labels). To make sure that the cover letter, survey, and return envelope fit without folding or bending, use slightly larger outer envelopes. Calculate the number of envelopes based on the number of initial and followup surveys to be sent.
Postage
If staff are instructed to return their surveys by mail, weigh the survey and the return envelope to ensure that you have adequate postage on the return envelopes. When calculating the total cost of postage, be sure to base the amount on the number of initial and followup surveys to be mailed.
Return to Contents
Tracking Surveys
You, or your vendor, will need to follow survey response rates by tracking surveys as they are returned. Tracking returned surveys can be done very simply with a spreadsheet software program. If you plan to use survey identifiers, create a separate row for each individual identifier. Create columns across the top of your spreadsheet for the date the initial survey is distributed, the date the returned survey is received (so that respondents can be excluded from followup reminders), and the distribution dates for any first reminders and second surveys.
Calculating Preliminary Response Rates
Calculate a preliminary response rate for each round of followup—for example, at the time of the reminder message and the second survey—to track your response progress. To calculate preliminary response rates during data collection, divide the number of returned surveys (numerator) by the number of surveys distributed (denominator) minus any surveys sent to ineligible providers and staff:
Number of surveys returned
Numbers of surveys distributed – ineligibles
Consider a person ineligible if he or she was away on leave from the medical office during the entire data collection period or if the person's employment in the office ended before the start of data collection. For example, you may have provided a vendor with an initial list of all staff and providers. However, between the time you provided the list and the start of data collection, a staff member went on medical leave for 3 months and a receptionist resigned. Those two staff members should be removed from the distribution list and from the denominator in your response rate calculation.
At the end of data collection, after you have examined all returned surveys, you will need to adjust your last preliminary response rate to reflect decisions made about whether a survey is complete or incomplete (go to Chapter 6 for a discussion of how to calculate the final official response rate for your medical office).
Return to Contents
To ensure that you receive as many responses as possible, plan to continue data collection for at least 1 full week after the second survey is distributed. Referring to the project timeline in Figure 1 (Chapter 2 ), allow about 6 weeks from the start to the end of data collection if you are surveying a single, medium-sized medical office, and about 8 weeks if you are surveying a large office or multiple medical offices. If any unexpected circumstances arise during data collection that delay the return of completed surveys, you may want to hold the data collection period open longer. However, once the final cutoff date arrives, close out data collection and begin preparing the data for cleaning and analysis.
Return to Contents
Proceed to Next Section
540 Gaither Road Rockville, MD 20850