In August 1996, welfare reform legislation was passed and the Census Bureau was mandated to conduct a survey to evaluate welfare reform and its impact on the nation. The Survey of Program Dynamics (SPD), a longitudinal demographic survey, is designed to accomplish that goal. The SPD uses a sample of original respondents from the 1992 and 1993 panels from the Survey of Income and Program Participation (SIPP). The SPD is conducted each Spring from 1997 through 2001. The first implementation of the questionnaire designed specifically for the SPD is scheduled for Spring 1998. There are two components to the SPD. The "core" survey instrument obtains information on such topics as employment, earnings, income sources and amounts, program participation, eligibility, educational enrollment, job training, disability, health care utilization, health insurance, children's enrichment activities, child care, child support, contact with absent parents, food security, marital relationship and conflict, and adult depression. Most questions in the survey are about the past calendar year, with some exceptions. The second component of the SPD is a self-administered questionnaire (SAQ) designed for persons 12-17 years of age. The SAQ is designed for administration through a personal audio-cassette player. Topics covered in the SAQ include family routines, housework and chores, relationship with parents, parental monitoring, contact with nonresidential parent, minor problem behaviors, substance use, knowledge of and attitudes towards welfare programs, marriage and childbearing, sexual initiation and contraception. (Adolescents 12-13 years of age are not asked the sexual initiation and contraception questions.) The primary purpose of the SPD pretest was to evaluate the SPD survey instrument, the adolescent SAQ, and some of the logistical, operational, and procedural aspects of the survey. Another purpose of the pretest was to obtain timer data so we could determine whether the length of the survey necessitated cuts in content prior to the 1998 SPD. Field pretesting of the SPD was conducted between October 6-22, 1997 in four regional office areas: Boston, Kansas City, Los Angeles, and Atlanta. Personal visit interviews were conducted using computer- assisted personal interviewing (CAPI). Pretest evaluation sources include debriefings of Field Representatives, interviewing observation reports, and review of taped interviews in which FR/respondent interactions were analyzed to detect problematic items. In addition, response analysis and respondent debriefing analysis was conducted (by Child Trends, Inc.) on the data obtained from the adolescent SAQ. This report details recommendations for changes in question wording and sequencing. We also provide suggestions for enhancements to FR training and the FR manual. Some procedural issues, such as, the use of respondent flashcards and the labor force activity worksheet are also addressed. The majority of suggested revisions are minor question wording changes. Nearly all of these recommendations were agreed to by the sponsoring divisions (POP/HHES) and can be implemented in time for the 1998 SPD. In some cases, our suggested revisions include more complex changes to the sequencing of questions within a series or between series. We also requested changing a series currently designed at the person level to a household level design, in an effort to reduce both FR and respondent burden. Many of these structural suggestions were agreed to by the sponsoring divisions, however, most of the more complex revisions cannot be implemented for the 1998 SPD due to resource and time constraints. They will be held for 1999 implementation. Given the size of the SPD instrument and the quantity of suggested revisions, it is not possible to summarize all the recommendations in the executive summary. Among the more significant recommendations resulting from the pretest are the following: Eliminating the Labor Force Activity Worksheet, replacing it with a calendar similiar to that used in SIPP; Resequencing the labor force series, so data for last year are collected prior to data for "last week"; Providing a response option of "retired" in selected labor force items and skipping such persons out of inappropriate items to reduce interviewer and respondent burden; Reordering the employment series so the name of employer is obtained prior to the "employer loop" (which obtains detail information for each employer); Including household-level income screeners for the income source module to reduce the number of irrelevant questions asked of specific income groups; Including a household-level income screener for high income households for the food security module to eliminate unnecessary respondent burden; Screening out inappropriate subgroups (men, never divorced adults, etc.) from questions regarding WIC, alimony, disability income, etc; Revising the item that identifies type of health insurance coverage to be asked at a person level instead of a household level to reduce interviewer and respondent burden; Revising question wording in almost all modules to clarify concepts and terms to increase respondent comprehension and reduce confusion, to reduce task difficulty and decrease burden, to clarify reference periods, etc. The report also details the successful administration of the adolescent supplement. Very few revisions are recommended for the 1998 SAQ. Pretest timer obtained from DSD indicates that no cuts to the core SPD are needed to meet the targeted 60 minute per household interview time for the 1998 SPD. Timer data revealed the average core pretest interview took 55.5 minutes, once outliers on both ends were removed. (The average when outliers were not removed was 62.19 minutes.)
