Services Research Outcomes Study (SROS)

1. EXTERNAL VALIDITY OF SURVEY RESPONSES
   • Table A.1 - Comparison between self report on drug use and urinalysis in SROS

EXTERNAL VALIDITY OF SURVEY RESPONSES

An examination of the relationship between self-report on arrests and recorded arrest history, as measured by state records of arrests, is presented in NTIES (Gerstein et al.,1997). Comparison of self-reports and official arrest records indicated that self-reports and arrest records were highly but not perfectly concordant (80 percent). Underreporting of arrest records was most frequent among individuals interviewed in prison or jail and among males under 25 years of age.

The overall validity of various kinds of self-report data has been of general concern to survey researchers. Errors in self-report have been noted in studies of various types of behaviors, including voting (Abelson, Loftus, and Greenwald, 1992) and the receipt of health care (Loftus, Smith, Klinger, and Fielder, 1992). Variations have been explained by the social desirability bias as well as by autobiographical memory processes.

Various researchers have demonstrated that distortion of responses related to social desirability may vary across groups (Callahan, 1968; Weiss, 1968). More recent work reinforces the importance of the context and characteristics of the group of respondents. For example, in a study of persons using a walk-in clinic for immediate medical care, 72 percent of persons with cocaine-positive urine denied recent cocaine use (McNagny and Parker, 1992). Alternatively, Lundy, Gottheil, Weinstein, Sterling, and Serota (1995) reported that those who did not complete substance abuse treatment had significantly higher rates ofunderreporting of drug use than did those who completed treatment. Wish et al. have also suggested that the validity of client self-reports may differ by drug (1997).

SROS has compared respondent’s self-reports of drug use with a more objective measure: urinalysis. Although both hair analysis and sweat analysis technologies hold promise for the future, to date, urinalysis has been most typically used to determine the accuracy of self-report. Urinalysis, when performed properly in quality-controlled laboratories (a NIDA-certified laboratory was used in SROS), provides an objective criterion to which self-report of drug use can be compared. It should be noted that the EMIT (Enzyme-Multiplied Immunoassay Technique) test used for the SROS urinalysis has some tendency toward false positive, not false negative, reports. As a result, the direction of error in this external validity study is toward overestimating drug use and underestimating the accuracy of respondents’ self reports of drug use in the population studied.

The ability to detect substances in urine is limited by the length of time it takes the body to clear drug metabolites. This time periods are short for alcohol, whose "dwell time" in the body is short, while heroin and cocaine metabolites remain longer (i.e., three days). Thus, urinalysis underestimates alcohol use.

SROS conducted follow-up interviews with 1,799 respondents and collected urine samples at the end of the interview for 1,364. That number constitutes 76 percent of the sample interviewed, which included respondents who were in jails/prisons whose management did not permit them to provide urine for analysis.

Table A-1 displays the results of urinalyses. The external validity check indicated that the self-reports of drug use by the SROS sample were predominantly confirmed by urinalysis. Agreement between self-report and urinalysis ranges from 89.7 percent to 98.5 percent for illicit drugs in the past week and from 86.2 percent to 99.0 percent in the past 24 hours. The major exception is alcohol, where concordance is 64.1 percent for the past week and 76.8 percent for the past 24 hours. As expected, self-report of alcohol use is far higher than by urinalysis.

Table A.1 - Comparison between self report on drug use and urinalysis in SROS
Results of Comparison	Time Period	Amphet-amine	Cocaine/Crack	Opiates Including Heroin	Alcohol	Methadone
Agreement: Self Report and Urine	past week	98.5%	89.7%	93.8%	64.1%	97.8%
	past 24 hours	99.0%	86.2%	92.8%	76.8%	97.6%
Urine Positive; Negative Self Report	past week	0.6%	9.1%	5.0%	1.4%	1.3%
	past 24 hours	0.8%	13.8%	6.7%	1.9%	1.7%
Urine Negative; Positive Self Report	past week	0.9%	1.2%	1.2%	34.5%	0.9%
	past 24 hours	0.2%	0.0%	0.5%	21.3%	0.7%

It is important to note that underreporting of drug use occurs more for cocaine/crack and heroin than for other drugs tested. Also, the level of that underreporting is higher for the 24-hour period than for the past week, probably due to the greater accuracy of urinalysis for the shorter time period.

This page was last updated on June 03, 2008.

SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States.