Chapter 4. Establishing Data Collection Procedures

Once you decide how you want the surveys distributed and returned and you establish a nursing home point of contact (POC), you need to make several decisions regarding your data collection procedures. This chapter includes strategies for publicizing your survey and maximizing your response rate and outlines methods for tracking responses and collecting data.

Publicize and Promote the Survey

We strongly recommend publicizing the survey before and during data collection. You can, for example, post flyers or posters in the nursing home, promote the survey during staff meetings, and, if possible, send staff E-mails and post information about the survey on a nursing home Intranet. Be sure to advertise that the survey is supported by nursing home or health care system executive leaders, or both.

If your health care system is sponsoring the survey in multiple nursing homes, system leaders can promote the survey in any meetings attended by designated POCs in the participating nursing homes. After the survey is underway, if response rates are lagging in some nursing homes, it may be useful to have POCs in nursing homes with high response rates share their strategies for promoting the survey with others. Also, physician response rates often are lower than those of staff and other providers. Midway into data collection, you might consider having the senior executive in the nursing home or at system headquarters send a special E-mail to all physicians in your survey population, thanking them if they have completed the survey and encouraging the others to do so.

If you have publicized your survey well and your survey cover letter includes important information about the purposes of the survey, we think distributing a separate prenotification letter announcing the upcoming survey is optional. However, if you are unsure whether all providers and staff know about the upcoming survey, you may want to send a prenotification letter to announce the survey effort before administering the survey. The letter should be signed by a nursing home or health care system senior executive, or both. If an outside vendor is handling data collection, use the letter as an opportunity to introduce the vendor.

Return to Contents

Follow Standard Survey Administration Steps

We recommend following a basic data collection approach to achieve high response rates. Achieving a high response rate is important for drawing valid conclusions about your nursing home from the survey data you collect. Surveys are used to infer something about a particular population. There must be enough survey respondents to accurately represent the nursing home before you can legitimately present your survey results as a reflection of your nursing home's safety culture.

If your response rate is low, there is a danger that the large number of staff who did not respond to the survey would have answered very differently from those who did respond. Therefore, an overall response rate of 50 percent or more should be your minimum goal. The higher the response rate, the more confident you can be that you have an adequate representation of the staff's views. The approach we recommend for paper surveys involves carrying out the following steps in the order presented:

1. First survey. About 1 week after you begin publicizing the survey (or several days after sending a prenotification letter if you decide to do that), distribute a survey packet to each eligible provider and staff member in your nursing home. The packet should include the survey, a supporting cover letter, and a return envelope that the respondent can seal. If staff will return their surveys by mail, rather than returning them to the nursing home POC or to a special box in the nursing home, the return envelope in the survey packet should be a preaddressed postage-paid envelope. This will make it easier for respondents to return their surveys.
2. Reminder postcard or letter. Two weeks after distributing the survey, distribute a reminder postcard or letter to staff, thanking those who have already responded and reminding others to respond. Reminders can also be sent by E-mail to employees with individual work E-mail addresses. If your nursing home is small, you may choose to have the POC remind staff members simply by talking to them.
3. Second survey. Two weeks after distributing the reminder, distribute the second survey. Include a cover letter in the second survey packet thanking those who have already responded and reminding others to please complete the second survey. (If you have chosen to put individual identification numbers or identifiers on your survey, you can distribute second surveys only to nonrespondents.)
4. Data collection closeout. At the end of the second week following distribution of the second survey, end data collection and begin preparing for data cleaning and analysis.

Consider Using Incentives To Maximize Response Rates Offering incentives can be a good way to increase responses to a survey because respondents often ask, "What's in it for me?" You may want to offer individual incentives, such as a raffle for cash prizes or gift certificates. Another option is group incentives, such as catered lunches for nursing homes with at least a 75 percent response rate. Be creative and think about what would motivate your staff to complete the survey.

Return to Contents

Decide Whether To Use Survey Identifiers

Whether or not you are surveying a single nursing home or multiple nursing homes, you need to decide whether to use individual respondent identification numbers or identifiers on your surveys. In addition, if you are surveying multiple nursing homes, you need to decide how to track returned surveys by each individual nursing home. We provide recommendations and discussion regarding these issues. First, we discuss our recommendation regarding the use of individual identifiers; then we discuss options for tracking surveys by nursing home. Finally, we provide important guidelines that must be followed if you choose to use individual identifiers.

Individual Identifiers

Whether you are administering the survey in a single nursing home or in multiple nursing homes, we recommend that you conduct an individually anonymous survey and do not use individual identifiers (usually a number or code) on the survey to track survey respondents and nonrespondents. Also, do not ask respondents to provide their names on completed survey forms. A strong potential advantage of individually anonymous surveys is that fewer respondents will refuse to participate because of concerns that the identifiers will be used to figure out their individual responses to the survey. Confidentiality concerns may be stronger in smaller nursing homes. You want to make sure that respondents feel comfortable about reporting their true perceptions and confident that their answers cannot be traced back to them.

If you decide not to use identifiers, you will need to distribute reminder notices and second surveys to all eligible staff, with instructions to disregard the reminder and second survey if the first survey was completed and mailed. You may receive phone calls from respondents who completed and returned their survey, wondering why they received followup materials. You can tell them that you have no information about who responded, so followup materials were distributed to everyone. You can tell them to disregard the materials and then remove their names from further followup. You will run a small risk that someone may complete and return more than one survey.

Nursing Home Identifiers

If you are surveying multiple nursing homes, you will want to be able to produce feedback reports for each nursing home. Therefore, you need to be able to identify which returned surveys came from which nursing home. We offer a few ways of doing that. Our suggestions vary depending on the number of nursing homes you survey and your preferences regarding how respondents will return their surveys.

Vary survey color. If the number of nursing homes you are surveying is not too large, you can print the survey on different colored paper for each nursing home. Then respondents can return the survey either within the nursing home or to a headquarters or vendor address and their surveys will still be identifiable by nursing home.

Restrict options for returning surveys. You can instruct respondents to return their surveys in sealed envelopes to their nursing home POC or to drop boxes in the nursing home. The POC can then batch the returned surveys, put them in a package that has the nursing home name or nursing home identifier on it, and send the package to a designated project leader or designated vendor. Under this option, some staff may not participate in the survey because they prefer, for reasons of confidentiality, to send their completed surveys directly to someone outside their nursing home.

Use a nursing home identifier. You can include a nursing home identifier on the surveys at the time they are printed by giving each nursing home a unique form number as an identifier. The form number would be the same for all surveys in each nursing home but would differ across nursing homes. For example, if you are surveying three nursing homes, you would use Form 1, Form 2, and Form 3 to identify these nursing homes. Print the identifier in an unobtrusive location on the survey (e.g., lower left corner of the back page). Because the words "Form 1," Form 2," etc., are part of the printed document, they do not stand out. Be aware, however, that some respondents will be so concerned about the confidentiality of their responses that they might mark out the nursing home identifier.

Guidelines When Using Individual Identifiers You may decide, particularly if your nursing home is quite large or you are surveying multiple nursing homes, that the advantages of using individual identifiers outweigh the advantages of individually anonymous surveys. For example, if you use individual identifiers to track responses, you can distribute reminders and followup materials only to nonrespondents, thus reducing your costs and eliminating the possibility of someone completing more than one survey. However, it is possible that some respondents will deidentify their own surveys by removing or marking out the identifiers anyway. If you use individual identifiers, you must adhere to careful procedures to protect the confidentiality of the information linking individual staff names to the identification numbers or codes. You will need to ensure that only key project personnel have access to information linking individual names or groups to the identification numbers or codes; and destroy all information linking names to identifiers at the end of data analysis.

Return to Contents

Develop and Assemble Survey Materials

The following materials will need to be developed and assembled in preparation for the survey distribution. Personalizing outer envelopes and letters (e.g., addressed to "Dear John Doe") sometimes promotes higher response rates (Dillman, 2007). Care should be taken, however, to prevent names from appearing on the actual survey forms.

Office/System Point of Contact Letter

You should send a letter to any persons designated as a nursing home or system-level POC describing the purposes of the survey and explaining their role in the survey effort. Print the letter on official letterhead, dated with month/year, signed by the nursing home senior executive and/or health care system senior executive.

We also recommend that you provide the POC with a simple data collection protocol that describes their tasks along with a proposed timeline (for a sample data collection protocol, go to Appendix C). In most nursing homes, POCs will be busy with their regular responsibilities. The protocol will help them stay on schedule.

Publicity Materials

We recommend preparing and posting informational flyers or posters in the nursing home and sending E-mail notices when possible to publicize the upcoming survey. Your publicity materials can help to legitimize the survey effort and increase your response rate by including some or all of the following types of information:

• Endorsements of the survey from your nursing home leaders.
• Clear statements about the purpose of the survey (to measure nursing home safety culture) and how the collected data will be used (to measure nursing home staff attitudes and perceptions about resident safety culture in their nursing home).
• Assurances that only summary data will be reported, thus keeping individual responses confidential.
• An introduction about the survey vendor, if you have chosen to use a vendor.
• Contact information for the designated POC in the nursing home.

Cover Letter in First Survey Packet

The cover letter that is included in the first survey packet should be on official nursing home letterhead and should address the following points:

• Why the nursing home is conducting the survey and how survey responses will be used.
• How much time is needed to complete the survey.
• Assurances that the survey is voluntary and can be completed during work time.
• Confidentiality or anonymity assurances.
• Suggested reply timeframe and how to return completed surveys.
• Incentives for survey participation (optional).
• Contact information for the nursing home POC (and system-level POC, if applicable).

In the cover letter, or on the survey form, ask staff to complete the survey within 7 days, but do not print an actual deadline date on the letter or survey. Sometimes data collection schedules get delayed, and you do not want to reprint letters or surveys because they are outdated. In addition, sometimes people will not complete a survey if they notice that it is beyond the deadline date.

Sample Cover Letter Text The enclosed survey is part of our nursing home's efforts to better address safety and quality care for our residents. All staff in the nursing home are receiving this survey. It will take about 10 to 15 minutes to complete, and your individual responses will be kept confidential. Only group statistics, not individual responses, will be prepared and reported. Please complete your survey and return it WITHIN THE NEXT 7 DAYS. When you have completed your survey, please [provide return instructions]. [Optional incentive text: In appreciation for participation, staff who complete and return their surveys will receive (describe incentive).] If you have any questions, please contact [POC name and job position] at [provide phone number and E-mail address]. Thank you in advance for your participation in this important effort.

Reminder Notice

Nursing home POCs can distribute reminder notices (e.g., a reminder message on a half-page of cardstock) after the first survey administration to all eligible nursing home providers and staff (if the survey is anonymous). The notice should ask them to please complete and return their surveys and should include a thank you for those who have responded. If you are using identifiers to track responses, you can distribute the reminders to nonrespondents only. In small nursing homes, POCs may opt to remind staff simply by talking to them individually or in a group.

Cover Letter in Second Survey Packet

The contents of the second survey cover letter should be similar to the first cover letter but should have a different beginning. For example, if you are conducting an anonymous survey, you will have to distribute second surveys to everyone. You might say: "About [X] weeks ago a copy of the Nursing Home Survey on Patient Safety Culture was distributed to you and other staff in this nursing home. If you have already returned a completed survey, thank you and please disregard this second survey packet."

If you are using individual identifiers, you will be able to identify respondents and remove them from the list of staff receiving followup materials, so you might begin as follows: "About [X] weeks ago a copy of the Nursing Home Survey on Patient Safety Culture was distributed to you and other staff in this nursing home. Because we have not yet received a completed survey from you, we are enclosing a second copy of the survey (if you recently returned your survey, thank you and please disregard this second survey packet)."

Surveys

If you are conducting an anonymous survey and plan to send second surveys to everyone, print at least twice the number of surveys as staff to be surveyed. If you are tracking responses through individual identifiers and will send second surveys only to nonrespondents, you may print fewer surveys. For example, if your nursing home has a total of 80 staff and your nursing home's survey response history typically results in a 40 percent response to the first survey, you would need 80 first surveys and 48 second surveys (80 staff x 60% nonrespondents = 48), for a total estimate of 128 printed surveys needed.

Labels

You will need labels for the outside of each survey packet. If the surveys are distributed within the nursing home, the labels could include only the names of providers or staff members in the nursing home. Even if the survey itself is completed anonymously, it is a good idea to label a survey packet for each staff member. Self-addressed return labels will be used on return envelopes. Labels also may be used to place identifiers on the surveys if you have chosen to use identifiers.

Envelopes

You will need a set of outer envelopes to distribute the surveys and a set of return envelopes that can be sealed for the return of completed surveys. Preprint the return address on the return envelopes (or use labels). To make sure that the cover letter, survey, and return envelope fit without folding or bending, use slightly larger outer envelopes. Calculate the number of envelopes based on the number of initial and followup surveys to be sent.

Postage

If staff are instructed to return their surveys by mail, weigh the survey and the return envelope to ensure adequate postage on the return envelopes. When calculating the total cost of postage, be sure to base the amount on the number of initial and followup surveys to be mailed.

Return to Contents

Track Responses and Calculate Preliminary Response Rates

Tracking Surveys

You, or your vendor, will need to follow survey response rates by tracking surveys as they are returned. Returned surveys can be tracked very simply with a spreadsheet software program. If you plan to use survey identifiers, create a separate row for each individual identifier. Create columns across the top of your spreadsheet for the date the initial survey is distributed, the date the returned survey is received (so that respondents can be excluded from followup reminders), and the distribution dates for any first reminders and second surveys.

Calculating Preliminary Response Rates

Calculate a preliminary response rate for each round of followup—for example, at the time of the reminder message and the second survey—to track your response rate progress. To calculate preliminary response rates during data collection, divide the number of returned surveys (numerator) by the number of surveys distributed minus any surveys sent to ineligible providers and staff (denominator):

(Number of surveys returned) divided by (Numbers of surveys distributed – ineligibles)

Consider a person ineligible if he or she was away on leave from the nursing home during the entire data collection period or if the person's employment with the nursing home ended before the start of data collection. For example, you may have provided a vendor with a list of all staff and providers on your survey distribution list. However, between the time you provided the list and the start of data collection, a staff member went on medical leave for 3 months and a nurse resigned. Those two staff members should be removed from the distribution list and from the denominator in your response rate calculation.

At the end of data collection, after you have examined all returned surveys, you will need to adjust your last preliminary response rate to reflect decisions made about whether a survey is complete or incomplete (Go to Chapter 5 for a discussion of how to calculate the final official response rate for your nursing home).

Return to Contents

Close Out Data Collection

To ensure that you receive as many responses as possible, plan to continue data collection for 2 full weeks after the second survey is distributed. Referring back to the project timeline in Chapter 2, allow about 9 weeks from the start to the end of data collection. If any unexpected circumstances arise during data collection that delay the return of completed surveys, you may want to hold the data collection period open longer. However, once the final cutoff date arrives, close out data collection and begin preparing the data for cleaning and analysis.

Return to Contents
Proceed to Next Section