Seam Bias in the 2004 SIPP Panel: Much Improved, but Much Bias Still Remains
Jeffrey C. Moore
KEY WORDS: dependent interviewing, event history calendars, measurement error, memory cues, spells, transitions
ABSTRACT
Panel surveys generally suffer to some extent from seam bias, the tendency for estimates of change measured across the “seam” between two successive survey administrations to far exceed change estimates measured within a single interview. Seam bias strikes at a core utility of retrospective panel surveys, because it means that reports of the start and end dates of spells of important characteristics (e.g., program receipt, health insurance coverage, etc.) are likely to contain substantial measurement error. Much research has documented the existence of seam bias; attempts to reduce it, however, have generally met with only limited practical success.
The U.S. Census Bureau recently completed a multi-year research program to improve the Survey of Income and Program Participation (SIPP) questionnaire, a main goal of which was to reduce seam bias. The specific questionnaire revision intended to accomplish this was a much more extensive and focused use of dependent interviewing (DI) procedures. New DI procedures were incorporated into the SIPP questionnaire with the launch of the most recent SIPP panel in February 2004. This paper describes those procedures, and examines their impact on seam bias in the first several waves of the 2004 panel for a number of characteristics (e.g., government transfer program participation, school enrollment, employment, health insurance coverage, etc.), through a comparison with the same estimates derived from the 2001 panel. Initial analyses, using preliminary, unedited data files, found clear evidence of a positive change with the new procedures (Moore, et al. (2008)). This paper repeats those analyses on edited, public-use data files, the results of which point to the same three general conclusions: First, seam bias is substantially lower in 2004 than it was in 2001. Second, the seam bias improvements are attributable to the new DI procedures. Third, however, notwithstanding the clear improvements, seam bias still afflicts SIPP 2004 panel data to a substantial extent. While there is good reason to expect that additional refinements to the DI procedures would yield further seam bias improvements, the results also leave little doubt that even under the most optimistic scenario, those refinements would yield some additional reductions in seam bias but would by no means eliminate it. An almost inescapable conclusion is that for most characteristics of interest to SIPP – school enrollment being a notable and instructive exception – the traditional question-by-question interviewing approach, using calendar months as cues, may simply be limited in its ability to capture high quality retrospective reports of spell start and end dates. Rather than focusing on any additional fine-tuning of that approach, future redesigns of the SIPP program should consider new alternatives such as event history calendar methods, which are more attuned to the basic organization of human memory, and which thus hold out promise as a better way to yield high quality retrospective spell data.
CITATION:
