Cognitive interviewing techniques are increasingly being used in questionnaire development to assess respondents' comprehension, their ability to retrieve information, and the processes they us in doing so. The results are then applied to construct new questions or to revise existing ones. Cognitive techniques have thus been used primarily at the "front end" of questionnaire development, prior to the pre- or field test. Because of the time needed to conduct and analyze such interviews, typically only a very small number of purposively- chosen respondents is interviewed at the questionnaire development phase. Recently researchers have begun to use "debriefing questions" following a standardized interview to assess respondents' reactions to the interview and, sometimes, to evaluate comprehension as well. This paper argues that debriefing questions can be used as a cognitive tool to assess the quality of the interview, and that because they can be administered to a large, representative sample, they are an important "back-end" supplement to the one-to-one interviews used in the early phases of questionnaire development. The paper (1) reviews the literature to see how respondent debriefing questions have been used in other surveys; (2) presents results of a respondent debriefing conducted in conjunction with the pretest of the Food Security Supplement to the Current Population Survey, and (3) recommends that such debriefings be made a routine part of questionnaire development and evaluation, along with such techniques as behavior coding.
Questionnaire Development, Debriefing Questions, Reliability, Comprehension
Hess, Jennifer C. and Eleanor Singer. (1995). The Role of Respondent Debriefing Questions in Questionnaire Development. Statistical Research Division Working Papers in Survey Methodology (#95-18). U.S. Census Bureau. Available online at <http://www.census.gov/srd/papers/pdf/sm9518.pdf>.
[PDF] or denotes a file in Adobe’s Portable Document Format. To view the file, you will need the Adobe® Reader® available free from Adobe.
This symbol indicates a link to a non-government web site. Our linking to these sites does not constitute an endorsement of any products, services or the information found on them. Once you link to another site you are subject to the policies of the new site.