The objective of the research described in this paper was to develop and improve a Nonresponse Followup (NRFU) instrument for the 2010 U.S. Census. This research is unique in that multiple pretesting methods were used in the development of a single instrument in two different languages: English and Spanish.
The NRFU instrument was originally developed and tested as a computer-assisted personal interview (CAPI) instrument. The U.S. Census Bureau’s NRFU operation is critical to gathering data from both English- and Spanish-speaking households who do not return their self-administered census forms. The instrument was developed using a series of pretesting methods, including usability and cognitive testing, behavior coding, and an observational study of the administration of test versions of both the English and Spanish instruments. Though many of the questions had already been tested in the context of a self-administered paper form, this was the first time pretesting was conducted using the interviewer-administered questions as they were scripted to be read from a CAPI instrument.
This paper discusses overarching results of three rounds of English cognitive testing, two rounds of Spanish cognitive testing, two rounds of usability testing in both languages, two rounds of behavior coding of the instrument in both languages, and an observational study of the administration of the NRFU interview in the field in both languages. The application of mixed pretesting methods to the development of one survey instrument is an all-too-uncommon situation. This paper presents lessons learned about the types of findings made possible by the different pretesting methods, and offers the unique opportunity to examine issues of equivalency between a source and a translated version of a survey instrument through multiple measures.
