Challenging Research Issues in Statistics and Survey Methodology at the BLSProblem Statement: How do Respondent Cognitive Processes and Respondent Interviewer Interaction Affect Data Quality in the American Time Use Survey (ATUS)? Key Words: time-use diary, respondent cognitive processes, respondent recall strategies, behavior coding, respondent-interviewer interaction, conversational interviewing. Contact for further discussion: Background and DefinitionsSince 2003 the ATUS, sponsored by the Bureau of Labor Statistics, has produced annual estimates of the amounts of time people spend in activities such as paid work, childcare, volunteering. The ATUS collects information about time spent in activities beginning at 4:00 am on the previous day up until 4:00 am of the interview day. Interviewers use a set of scripted open-ended questions to walk respondents chronologically through the prior 24-hour day. Respondents are asked to recall their daily activities, how long each activity took, who was in the room with them, and where the activity took place. This 24-hour recall diary is used in tandem with computer-assisted telephone interviewing (CATI) for data collection. Early BLS research on the 24-hour recall diary explored the cognitive processes that respondents used to reconstruct their prior day and how cognitive processes were related to data quality. BLS and Westat researchers testing the 24-hour recall diary instrument were concerned about possible variation in data quality associated with differences in how respondents interpret "activities," the effects of highly salient activities, routine activities, and schemata (or everyday activities) on recall; and heuristic or rule of thumb strategies for estimating activity duration (Forsythe, 1997). Findings from cognitive interviews indicated that participants recalled activities and time durations more easily when their activities and times were atypical and/or when they had an appointment or time commitment. Some participants used a visualization technique as part of the recall process, while others used a decomposition strategy — recalling a larger block of time or major activity to set time boundaries, then recalling what happened during the time period. Participants who lived by the clock reported by the clock, but others had difficulty identifying starting and stopping times, and reported time durations instead. Since this early testing, BLS has not conducted further formal research on the cognitive processes associated with the questions used and tasks undertaken in completing the 24-hour recall diary. Additional research on respondents' cognitive processes might aid in developing instrument and interviewer interventions to assist respondents in their task and ultimately reduce measurement error. In addition, research on respondent and interviewer interaction during diary data collection is another important and unexplored area of methodological study. ATUS interviewers are trained in conversational interviewing techniques, which allow for interventions with a respondent to help him or her stay on track when remembering the day's activities, and activity sequences and timing. For example, after a respondent says that he or she ate dinner for an hour, the interviewer might check back, "that brings us up to 7:00 pm, does that sound about right?" Interviewers are also trained in ways to facilitate recall; one such technique is called working backwards. If a respondent can't remember what he or she did right after dinner, interviewers can ask, "what is the next thing you remember doing (and what time did that take place)?" and then prompt respondents to think of what he or she did in between. This working backwards technique may help respondents fill in the gaps, as may visualization techniques, such as an interviewer asking respondents to picture where they were in order to remember forgotten activities. Research to evaluate how conversational interviewing and specific recall techniques are used by interviewers and whether the techniques are successful in helping respondents reconstruct their day could help refine ATUS procedures and data quality. One method that might be used to study both respondent cognitive processes and respondent and interviewer interaction is behavior coding. This technique has been successfully utilized with event history calendar data collection (Belli, 2004) to understand how interviewers ask questions and provide clarification and feedback to respondents, how respondents interpret questions and recall answers, and how interviewers and respondents interact during the survey task. Issues
Related Questions:
References:Belli, R., Lee, E.H., Stafford, F.P., and Chou, C.H. (2004). "Calendar and Question-List Survey Methods: Association Between Interviewer Behaviors and Data Quality." Journal of Official Statistics 20: 185-218. Forsythe, B. (1997). "Cognitive laboratory research plan for collecting time use data." Washington, D.C.: Bureau of Labor Statistics, Office of Survey Methods Research. Horrigan, M. & Herz, D. (2004). "Planning, designing, and executing the BLS American Time-Use Survey." Monthly Labor Review, 127, 3-19 Juster, F.T. & Stafford, F.P. (Eds.). (1985). Time Goods and Well Being. Ann Arbor, MI: Institute for Social Research. Juster, F.T. (1986). "Response Errors in the Measurement of Time Use." Journal of the American Statistical Association 81, 390-402. National Academy of Sciences. (2000). Time-Use Measurement and Research: Report of a Workshop. Washington, DC: National Academy Press. Phipps, P. and Vernon, M. (2008). "24 Hours: An Overview of the Recall Diary Method and Data Quality in the American Time Use Survey." In Robert F. Belli, Frank P. Stafford, and Duane F. Alwin (Eds.), Calendar and Time Diary Methods in Life Course Research. Thousands Oaks, CA: Sage. Expected Publication 2008. Robinson, J.P. (1985). "The validity and reliability of diaries versus alternative time use measures." In F.T. Juster and F. P. Stafford (Eds.), Time Goods and Well Being (pp. 33-62). Ann Arbor, MI: Institute for Social Research, University of Michigan. Robinson, J.P. (1999). "The Time-Diary Method: Structures and Uses." In W.E. Pentland, A.S. Harvey, M.P. Lawton, and M.A. McColl (Eds.), Time Use Research in the Social Sciences (pp. 47-87). NY: Kluwer Academic/Plenum
Last Modified Date: July 30, 2008 |
Tools |
Calculators |
Help |
Info |