Computer Assisted Interviewing for SAMHSA's National Household Survey on Drug Abuse

1. Introduction

The National Household Survey on Drug Abuse (NHSDA) is designed to yield yearly cross-sectional estimates of substance use and abuse for subpopulations defined by age and racial/ethnic status for the U.S. civilian, noninstitutionalized population aged 12 or older. The annual survey is sponsored by the Office of Applied Studies (OAS) within the Substance Abuse and Mental Health Services Administration (SAMHSA) and is currently conducted by the Research Triangle Institute (RTI). Prevalence estimates are produced of the use of alcohol, tobacco, marijuana and hashish, cocaine, heroin, inhalants, and hallucinogens, as well as of the nonmedical use of psychotherapeutic drugs. Prior to 1999, national samples were used to control the relative standard error of estimates for four subgroups: (a) the population as a whole; (b) four age groups: 12 to 17, 18 to 25, 26 to 34, and 35 or older; (c) three racial/ethnic groups: non-Hispanic/all other races, non-Hispanic/black, and Hispanic;¹ and (d) the cross-classification by age group and racial/ethnic group. In general, the sample was selected in four stages consisting of (a) primary sampling units (PSUs), (b) area segments within PSUs, (c) dwelling units within the area segments, and (d) persons within occupied dwelling units. To select the sample persons, interviewers visited each sample dwelling unit and screened the dwelling unit to determine the number of people in each of the age groups and the household's racial/ethnic status. Sample selection tables, printed on paper, were then consulted to determine who, if anyone, was selected for the survey. In most households, no one was selected; in the others, either one or two persons were selected.

From the inception of the NHSDA until 1999, the data were collected face to face using paper-and-pencil interviews (PAPIs) with a combination of interviewer-administered questions and respondent-completed answer sheets. Interviewers collected the data in the respondents' homes and attempted to obtain a private location for the interview. The respondents' anonymity and privacy of their responses were protected by separating the identifying information from their survey responses. Respondents were also assured that their identities and responses would be handled with the strictest confidence. The more sensitive questions were self-administered either by having individual respondents complete answer sheets on their own or by having them mark their answers as the interviewer read the questions aloud to them. Parental permission was required before interviewing a respondent under 18 years of age.

In 1999, the NHSDA sample was expanded and redesigned to permit using a combination of direct and model-based small area estimation procedures that allow SAMHSA to produce estimates for all 50 States and the District of Columbia (DC). In addition, computer-assisted data collection procedures were adopted for both screening and interviewing. This report summarizes the research to develop these computer-assisted screening and interviewing procedures. Information on the redesigned sample and details regarding the small area estimation techniques are not covered in this report, but may be found in other reports available from SAMHSA (2000).

The computer-assisted applications were developed for a number of reasons. First, during the screening, interviewers made both intentional and unintentional errors. It was possible to influence who was selected for the survey by manipulating the order in which respondents in the household were listed, and interviewers sometimes made mistakes when they consulted the sample selection tables. For example, in the 1997 survey, 75 cases could not be used because of incorrect sample selections. To prevent these errors and to permit more complex sampling algorithms for selecting sample persons, an electronic screening application was developed using a handheld Newton computer.² Second, interviewers and respondents made errors when completing the questionnaire, and the privacy afforded by the self-administered answer sheets was less available to those respondents who did not read well and had to depend upon the interviewer for assistance in completing the self-administered portions of the interview. To alleviate these difficulties, a computer-assisted interviewing (CAI) application was developed that employed both computer-assisted personal interviewing (CAPI) for sections of the interview that had traditionally been administered by interviewers, and audio computer-assisted self-interviewing (ACASI), for sensitive questions that had been asked using self-administered answer sheets.

In 1996, 1997, and 1998, SAMHSA sponsored a variety of research studies that explored the usefulness of using CAI procedures to reduce errors in the NHSDA. The research included a small-scale field experiment conducted in late 1996 that examined the operational feasibility of using CAI procedures for the NHSDA. This 1996 experiment examined perceptions of privacy, the length of the interview, the effect of CAI on the interviewing environment, and the quality of the data provided. Following the field experiment, a series of laboratory studies addressed methods for resolving inconsistent responses, ways for asking the questions on the frequency of use in the past 12 months, and reactions to different voices for ACASI interviews. In late 1997, a large-scale field experiment was conducted. The 1997 experiment examined the ACASI portion of the CAI interview by looking at (a) different ways of asking contingent questions, (b) the ability of respondents to resolve inconsistent responses, and (c) the effect of giving respondents multiple chances to report use of particular substances. In addition, the 1997 study examined the use of a handheld computer to conduct the household screening and sample selection activities. Finally, in the summer of 1998, a series of cognitive laboratory studies examined procedures for asking additional questions on tobacco use, and in August 1998, a field test of a revised electronic screening application and CAI procedures was conducted.

This report covers a variety of NHSDA field experiment topics. To start, Chapter 2 gives a brief history of research on the NHSDA, and Chapter 3 offers further background information, including a literature review and an overview of critical design and operational issues. Chapter 4 focuses on the 1996 feasibility experiment and cognitive laboratory research, while Chapters 5 through 9 delve into the 1997 field experiment. Specifically, Chapter 5 summarizes the design and conduct of the 1997 effort, Chapter 6 compares CAPI/ACASI with PAPI for selected outcomes, Chapter 7 describes the effect of ACASI experimental factors on prevalence and data quality, Chapter 8 details the development and testing of an electronic screener, and Chapter 9 describes the operation of the 1997 field experiment. The next two chapters offer insights into the willingness of NHSDA respondents to be interviewed (Chapter 10) and the effect of NHSDA interviewers on data quality (Chapter 11). Chapter 12 is devoted to further refinement of the CAI procedures during the 1998 laboratory and field testing of a tobacco module. Please note that the exhibits, which are numbered according to the sections that they follow, are grouped immediately after the sections in which they are discussed.

¹ In the interest of readability for this report, "white" is used to indicate "non-Hispanic/all other races" and "black" to indicate "non-Hispanic/black."

² The Newton computer is an Apple product (Message Pad 2000). The U.S. Government does not endorse or favor any specific commercial product or company. Trade, proprietary, or company names appearing in this publication are used only because they are considered essential in the context of the studies reported herein.

This page was last updated on June 16, 2008.

SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States.