Interviewers are in a unique position to evaluate the merits of survey questions. Not only do they obtain very useful feedback from respondents in the course of administering questionnaires, more experienced interviewers can often draw on their accumulated knowledge of survey interactions to identify--during the pretesting stage of questionnaire development--questions that are likely to be difficult for interviewers to read and/or for respondents to comprehend. In this paper, we discuss our experiences with interviewer debriefings (i.e., focus groups, structured questionnaires) as one of four general methodologies used in a field test of alternative versions of a major economic survey. The paper is organized as follows. In the introductory section, we review methodological research in the area of questionnaire pretesting that has made use of interviewer debriefings. In the second section of the paper, we describe our experiences with the interviewer-debriefing methodology. The context for this discussion, as alluded to above, is the work that was done by a team of researchers in evaluating alternative versions of questions for the Current Population Survey (CPS). In the final section, we conclude with a general discussion of this method's utility, identifying what we believe are its strong points and limitations under various pretesting/contextual conditions. We also propose a new, composite debriefing technique--one that combines the use of focus groups and a structured debriefing questionnaire--that addressess some of the recent criticism of interviewer debriefings as a pretesting methodology.
To request a copy of this paper please write at srd@census.gov Also please include your name, address, telephone number, and title and number of paper.
