An Application of Rasch Analysis to Questionnaire Design:
Using Vignettes to Study the Meaning of "Work" in the
Current Population Survey
Elizabeth A. Martin, Pamela C. Campanelli, and Robert E. Fay
KEY WORDS: Rasch measurement model, response scale, question meaning, respondent debriefing
ABSTRACT
As part of the research to develop a new questionnaire for the Current Population Survey (CPS), a special respondent debriefing study was conducted in 1988 to learn more about how respondents understand and formulate their answers to the labor force questions. A sample of approximately 2,300 respondents answered standardized debriefing questions after they had completed the main CPS interview. As part of the debriefing interview, respondents were presented with five vignettes describing various work activities and asked whether the person described should be reported as working or not. In order to understand respondents’ interpretations of work, the authors apply the Rasch measurement model to the cross-classification of responses to the vignettes. The paper addresses two questions: (1) is there an underlying dimension of meaning which accounts for the diverse patterns of responses to the vignettes? and (2) to the extent that responses to the vignettes are structured by a coherent concept of “work,” is the response structure constant across the population?
Analysis of the vignette data suggest that there is an underlying dimension of meaning which accounts for responses to four of the five vignettes. One of the items does not “fit” in a scale with the others. Respondents vary along a dimension of inclusiveness: at one end are those who hold a strict definition of work, and at the other are respondents who are willing to include marginal activities as work. Additional interactions between particular pairs of items also must be included in the model to achieve a satisfactory fit. These interactions indicate that respondents may use particular heuristics (e.g., cash payment) to decide which activities should be considered “work.”
The paper further finds that there are significant variations in interpretations of work among subgroups defined by age and education.
The paper concludes that vignettes can be useful for learning how a survey concept is applied in practice in situations of interest to the questionnaire designer, and that Rasch analysis can help illuminate the coherence and meaning of concepts used in surveys.
CITATION: Elizabeth A. Martin, Pamela C. Campanelli, and Robert E. Fay. 1991. “An Application of Rasch Analysis to Questionnaire Design: Using Vignettes to Study the Meaning of ‘Work’ in the Current Population Survey.” The Statistician 40: 265-276.
