2001 National Household  Survey on Drug Abuse

Appendix B: Statistical Methods and Limitations of the Data

B.1 Target Population

An important limitation of the National Household Survey on Drug Abuse (NHSDA) estimates of drug use prevalence is that they are only designed to describe the target population of the survey—the civilian, noninstitutionalized population aged 12 or older. Although this population includes almost 98 percent of the total U.S. population aged 12 or older, it excludes some important and unique subpopulations who may have very different drug-using patterns. For example, the survey excludes active military personnel, who have been shown to have significantly lower rates of illicit drug use. Persons living in institutional group quarters, such as prisons and residential drug treatment centers, are not included in the NHSDA and have been shown in other surveys to have higher rates of illicit drug use. Also excluded are homeless persons not living in a shelter on the survey date, another population shown to have higher than average rates of illicit drug use. Appendix E describes other surveys that provide data for these populations.

B.2 Sampling Error and Statistical Significance

The national estimates, along with the associated variance components, were computed using a multiprocedure package, SUrvey DAta ANalysis (SUDAAN) Software for Statistical Analysis of Correlated Data, which was designed for the statistical analysis of sample survey data from stratified, multistage cluster samples (RTI, 2001). The final, nonresponse-adjusted, and poststratified analysis weights were used to compute unbiased design-based drug use estimates.

The sampling error (i.e., the standard error [SE]) of an estimate is the error caused by the selection of a sample instead of conducting a census of the population. Sampling error is reduced by selecting a large sample and by using efficient sample design and estimation strategies, such as stratification, optimal allocation, and ratio estimation.

With the use of probability sampling methods in the NHSDA, it is possible to develop estimates of sampling error from the survey data. These estimates have been calculated in SUDAAN for all estimates presented in this report using a Taylor series linearization approach that takes into account the effects of the complex NHSDA design features. The sampling errors are used to identify unreliable estimates and to test for the statistical significance of differences between estimates.

B.2.1 Variance Estimation for Totals

Estimates of proportions, , such as drug use prevalence rates, take the form of nonlinear statistics where the variances cannot be expressed in closed form. Variance estimation for nonlinear statistics in SUDAAN is performed using a first-order Taylor series approximation of the deviations of estimates from their expected values.

Corresponding to proportion estimates, , the number of drug users, , can be estimated as

 ^,

where is the estimated population total for domain d, and p hat sub d is the estimated proportion for domain d. The SE for the total estimate is obtained by multiplying the SE of the proportion by , that is,

 ^.

This approach is theoretically correct when the domain size estimates, , are among those forced to Census Bureau population projections through the weight calibration process. In these cases, is clearly not subject to sampling error.

For domain totals, , where is not fixed, this formulation may still provide a good approximation if it can be reasonably assumed that the sampling variation in is negligible relative to the sampling variation in . In most analyses conducted for prior years, this has been a reasonable assumption.

For a subset of the tables produced from the 2001 data, it was clear that the above approach yielded an underestimate of the variance of a total because was subject to considerable variation. In these cases, a different method was used to estimate variances. SUDAAN provides an option to directly estimate the variance of the linear statistic that estimates a population total. Using this option did not affect the SE estimates for the corresponding proportions presented in the same sets of tables.

B.2.2 Suppression Criteria for Unreliable Estimates

As has been done in past NHSDA reports, direct survey estimates considered to be unreliable due to unacceptably large sampling errors are not shown in this report and are noted by asterisks (*) in the tables containing such estimates found in the appendices. The criterion used for suppressing all direct survey estimates was based on the relative standard error (RSE), which is defined as the ratio of the standard error (SE) over the estimate.

Proportion estimates () within the range [0 < < 1], rates, and corresponding estimated number of users were suppressed if

RSE[(-ln()] > 0.175 when < 0.5

or

RSE[(-ln(1 - )] > 0.175 when > 0.5.

Using a first-order Taylor series approximation to estimate RSE[(-ln()] and RSE[(-ln(1 - )], the following was obtained and used for computational purposes:

SE()/	> 0.175 when < 0.5
-ln()

or

SE()/(1-)	> 0.175 when > 0.5   .
-ln(1-)

The separate formulas for < 0.5 and > 0.5 produce a symmetric suppression rule (i.e., if is suppressed, then so will 1 - ). This ad hoc rule requires an effective sample size in excess of 50. When 0.05 < < 0.95, the symmetric property of the rule produces a local maximum effective sample size of 68 at = 0.5. Thus, estimates with these values of along with effective sample sizes falling below 68 are suppressed. A local minimum effective sample size of 50 occurs at = 0.2 and again at = 0.8 within this same interval, so estimates are suppressed for values of with effective sample sizes below 50.

Prior to the 2000 NHSDA, these varying sample size restrictions sometimes produced unusual occurrences of suppression for a particular combination of prevalence rates. For example, in some cases, lifetime prevalence rates near = 0.5 were suppressed (effective sample size was < 68 but > 50), while not suppressing the corresponding past year or past month estimates near = 0.2 (effective sample sizes were > 50). To reduce the occurrence of this type of inconsistency, a minimum effective sample size of 68 was added to the NHSDA suppression criteria starting in 2000. As approached 0.00 or 1.00 outside the interval (0.05, 0.95), the suppression criteria still required increasingly larger effective sample sizes. For example, if = 0.01 and 0.001, the effective sample size must exceed 152 and 684, respectively.

Also new to the NHSDA starting in 2000 were minimum nominal sample size suppression criteria (n = 100) that protect against unreliable estimates caused by small design effects and small nominal sample sizes. Prevalence estimates were also suppressed if they were close to 0 or 100 percent (i.e., if < .00005 or if > .99995).

Estimates of other totals (e.g., number of initiates) along with means and rates (both not bounded between 0 and 1) were suppressed if RSE() > 0.5. Additionally, estimates of the mean age at first use were suppressed if the sample size was smaller than 10 respondents; moreover, the estimated incidence rate and number of initiates were suppressed if they rounded to 0.

The suppression criteria for various NHSDA estimates are summarized in Table B.1 at the end of this appendix.

B.2.3 Statistical Significance of Differences

This section describes the methods used to compare prevalence estimates in this report. Customarily, the observed difference between estimates is evaluated in terms of its statistical significance. "Statistical significance" refers to the probability that a difference as large as that observed would occur due to random error in the estimates if there were no difference in the prevalence rates for the population groups being compared. The significance of observed differences in this report is generally reported at the 0.05 and 0.01 levels. When comparing 2000 and 2001 prevalence estimates, the null hypothesis (no difference in the 2000 and 2001 prevalence rates) can be tested against the alternative hypothesis (there is a difference in prevalence rates) using the standard difference in proportions test expressed as follows:

where ₁ = 2000 estimate, ₂ = 2001 estimate, var(₁) = variance of 2000 estimate, var(₂) = variance of 2001 estimate, and cov(₁, ₂) = covariance between ₁ and ₂.

Under the null hypothesis, Z is asymptotically distributed as a normal random variable. Calculated values of Z can therefore be referred to as the unit normal distribution to determine the corresponding probability level (i.e., p value). Because there is a 50 percent overlap in the sampled segments between the 2000 and 2001 NHSDAs, the covariance term in the formula for Z will, in general, be greater than 0. Estimates of Z, along with its p value, were calculated using SUDAAN, using the analysis weights and accounting for the sample design as described in Appendix A. A similar procedure and formula for Z were used for estimated totals and for comparing prevalence estimates for different population subgroups from the same data year.

When examining the effects of subgroup variables with more than two levels on a prevalence measure, a X  ² test of independence of the subgroup and the prevalence variables was conducted first to control the error level for multiple comparisons. If the X  ² test indicated some significant differences, the significance of each particular subgroup comparison discussed in the report was tested as indicated above. SUDAAN analytic procedures were used in all tests to properly account for the sample design.

B.3 Nonsampling Error

Nonsampling errors can occur from nonresponse, coding errors, computer processing errors, errors in the sampling frame, reporting errors, and other errors not due to sampling. Nonsampling errors are reduced through data editing, statistical adjustments for nonresponse, close monitoring and periodic retraining of interviewers, and improvement in various quality control procedures.

Although nonsampling errors can often be much larger than sampling errors, measurement of most nonsampling errors is difficult or impossible. However, some indication of the effects of some types of nonsampling errors can be obtained through proxy measures, such as response rates and from other research studies.

B.3.1 Screening and Interview Response Rate Patterns

Response rates for the NHSDA were stable for the period from 1994 to 1998, with the screening response rate at about 93 percent and the interview response rate at about 78 percent (response rates discussed in this appendix are weighted). In 1999, the computer-assisted interviewing (CAI) screening response rate was 89.6 percent, and the interview response rate was 68.6 percent. A more stable and experienced field interviewer (FI) workforce improved these rates in 2000 and continued in 2001. Of the 171,519 eligible households sampled for the 2001 NHSDA main study, 157,471 were successfully screened for a weighted screening response rate of 91.9 percent (Table B.2). In these screened households, a total of 89,745 sample persons were selected, and completed interviews were obtained from 68,929 of these sample persons, for a weighted interview response rate of 73.3 percent. A total of 13,478 (16.5 percent) sample persons were classified as refusals or parental refusals, 4,681 (5.3 percent) were not available or never at home, and 2,657 (4.9 percent) did not participate for various other reasons, such as physical or mental incompetence or language barrier (Table B.3). Tables B.4 and B.5 show the distribution of the selected sample by interview code and age group. The weighted interview response rate was highest among 12 to 17 year olds (82.2 percent), females (74.6 percent), blacks and Hispanics (75.0 and 78.8 percent, respectively), in nonmetropolitan areas (76.7 percent), and among persons residing in the Midwest (74.4 percent) (Table B.6).

The overall weighted response rate, defined as the product of the weighted screening response rate and weighted interview response rate, was 61.5 percent in 1999, 68.6 percent in 2000, and 67.3 percent in 2001. Nonresponse bias can be expressed as the product of the nonresponse rate (1-R) and the difference between the characteristic of interest between respondents and nonrespondents in the population (P_r - P_nr). Thus, assuming the quantity (P_r - P_nr) is fixed over time, the improvement in response rates in 2000 and 2001 over 1999 will result in estimates with lower nonresponse bias.

B.3.2 Inconsistent Responses and Item Nonresponse

Among survey participants, item response rates were above 97 percent for most questionnaire items. However, inconsistent responses for some items, including the drug use items, were common. Estimates of substance use from the NHSDA are based on the responses to multiple questions by respondents, so that the maximum amount of information is used in determining whether a respondent is classified as a drug user. Inconsistencies in responses are resolved through a logical editing process that involves some judgment on the part of survey analysts and is a potential source of nonsampling error. Because of the automatic routing through the CAI questionnaire (e.g., lifetime drug use questions that skip entire modules when answered "no"), there is less editing of this type than in the paper-and-pencil interviewing (PAPI) questionnaire used prior to the NHSDA redesign in 1999.

In addition, logical editing is used less often because with the CAI data, statistical imputation is relied upon more heavily to determine the final values of drug use variables in cases where there is the potential to use logical editing to make a determination. The combined amount of editing and imputation in the CAI data is still considerably less than the total amount used in prior PAPI surveys. For the 2001 CAI data, for example, 6.7 percent of the estimate of past month hallucinogen use was based on logically edited cases and 6.6 percent on imputed cases, for a combined amount of 13.3 percent. In the 1998 NHSDA (administered using PAPI), the amount of editing and imputation for past month hallucinogen use was 60 and 0 percent, respectively, for a total of 60 percent. The combined amount of editing and imputation for the estimate of past month heroin use was 5.7 percent for the 2001 CAI and 37.0 percent for the 1998 PAPI data.

B.3.3 Validity of Self-Reported Use

NHSDA estimates are based on self-reports of drug use, and their value depends on respondents' truthfulness and memory. Although many studies have generally established the validity of self-report data and the NHSDA procedures were designed to encourage honesty and recall, some degree of underreporting is assumed (Harrell, 1997; Harrison & Hughes, 1997; Rouse, Kozel, & Richards, 1985). No adjustment to NHSDA data is made to correct for this. The methodology used in the NHSDA has been shown to produce more valid results than other self-report methods (e.g., by telephone) (Aquilino, 1994; Turner, Lessler, & Gfroerer, 1992). However, comparisons of NHSDA data with data from surveys conducted in classrooms suggest that underreporting of drug use by youths in their homes may be substantial (Gfroerer, 1993; Gfroerer, Wright, & Kopstein, 1997).

B.4 Incidence Estimates

For diseases, the incidence rate for a population is defined as the number of new cases of the disease, N, divided by the person time, PT, of exposure or

IR =	N .
	PT

The person time of exposure can be measured for the full period of the study or for a shorter period. The person time of exposure ends at the time of diagnosis (e.g., Greenberg, Daniels, Flanders, Eley, & Boring, 1996, pp. 16-19). Similar conventions are applied for defining the incidence of first use of a substance.

Beginning in 1999, the NHSDA questionnaire allows for collection of year and month of first use for recent initiates. Month, day, and year of birth are also obtained directly or imputed in the process. In addition, the questionnaire call record provides the date of the interview. By imputing a day of first use within the year and month of first use reported or imputed, the key respondent inputs in terms of exact dates are known. Exposure time can be determined in terms of days and converted to an annual basis.

Having exact dates of birth and first use also allows the person time of exposure during the targeted period, t, to be determined. Let the target time period for measuring incidence be specified in terms of dates; for example, the period 1998 would be specified as

t = [t₁, t₂) = [1 Jan 1998, 1 Jan 1999),

a period that includes 1 January 1998 and all days up to but not including 1 January 1999. The target age group can also be defined by a half-open interval as a = [a₁, a₂). For example, the age group 12 to 17 would be defined by a = [12, 18) for persons at least age 12, but not yet age 18. If person i was in age group a during period t, the time and age interval, L_t,a,i, can then be determined by the intersection:

L_t,a,i = [t₁, t₂) [DOB_i MOB_iYOB_i + a₁,DOB_iMOB_iYOB_i + a₂),

assuming the time of birth can be written in terms of day (DOB_i), month (MOB_i), and year (YOB_i). Either this intersection will be empty (L_t,a,i = Ø) or it will be designated by the half-open interval, L_t,a,i = [m_1,i, m_2,i), where

m_1,i = Max{t₁, (DOB_iMOB_iYOB_i + a₁)}

and

m_2,i = Min{t₂, (DOB_iMOB_iYOB_i + a₂).}

The date of first use, t_fu,d,i, is also expressed as an exact date. An incident of first drug d use by person i in age group a occurs in time t if t_fu,d,i [m_1,i,m_2,i). The indicator function I_i (d,a,t) used to count incidents of first use is set to 1 when t_fu,d,i [m_1,i,m_2,i) and to 0 otherwise. The person-time exposure measured in years and denoted by e_i(d,a,t) for a person i of age group a depends on the date of first use. If the date of first use precedes the target period (t_fu,d,i < m_1,i), then e_i(d,a,t) = 0. If the date of first use occurs after the target period or if person i has never used drug d, then

If the date for first use occurs during the target period L_t,a,i, then

Note that both I_i (d,a,t) and e_i(d,a,t) are set to 0 if the target period L_t,a,i is empty (i.e., person i is not in age group a during any part of time t). The incidence rate is then estimated as a weighted ratio estimate:

where the w_i are the analytic weights.

Prior to the 1999 survey, exact date data were not available for computing incidence rates. For these rates, a person was considered to be of age a during the entire time interval t, if his/her a^th birthday occurred during time interval t (generally, a single year). If the person initiated use during the year, the person-time exposure was approximated as one-half year for all such persons rather than computing it exactly for each person.

Because of the new methodology, the incidence estimates discussed in Chapter 5 are not strictly comparable with the estimates before the 1999 NHSDA. The estimates in this report are based on retrospective reports of age at first drug use by survey respondents interviewed during 1999 to 2001. Because they are based on retrospective reports as was the case for earlier estimates, they may be subject to some of the same kinds of biases.

Bias due to differential mortality occurs because some persons who were alive and exposed to the risk of first drug use in the historical periods shown in the tables died before the 1999-2001 NHSDAs were conducted. This bias is probably very small for estimates shown in this report. Incidence estimates are also affected by memory errors, including recall decay (tendency to forget events occurring long ago) and forward telescoping (tendency to report that an event occurred more recently than it actually did). These memory errors would both tend to result in estimates for earlier years (i.e., 1960s and 1970s) that are downwardly biased (because of recall decay) and estimates for later years that are upwardly biased (because of telescoping). There is also likely to be some underreporting bias due to social acceptability of drug use behaviors and respondents' fear of disclosure. This is likely to have the greatest impact on recent estimates, which reflect more recent use and reporting by younger respondents. Finally, for drug use that is frequently initiated at age 10 or younger, estimates based on retrospective reports 1 year later underestimate total incidence because 11-year-old (and younger) children are not sampled by the NHSDA. Prior analyses showed that alcohol and cigarette (any use) incidence estimates could be significantly affected by this. Therefore, for these drugs only 2000 age-specific, and not overall, estimates were made. Likewise for these drugs, 1999 estimates were made using 2001 NHSDA data and 1998 estimates were made using 2000 and 2001 NHSDA data.

B.5 Serious Mental Illness Estimates

For the 2001 NHSDA, mental health among adults was measured using a scale to ascertain serious mental illness (SMI). This scale consisted of six questions that ask respondents how frequently they experienced symptoms of psychological distress during the 1 month in the past year when they were at their worst emotionally. The use of this scale is based on a methodological study designed to evaluate several screening scales for measuring SMI in the NHSDA. These scales consisted of a truncated version of the World Health Organization (WHO) Composite International Diagnostic Interview Short Form (CIDI-SF) scale (Kessler, Andrews, Mroczek, Üstün, & Wittchen, 1998), the K10/K6 scale of nonspecific psychological distress (Furukawa, Andrews, Slade, & Kessler, in press), and the WHO Disability Assessment Schedule (WHO-DAS) (Rehm et al., 1999).

The methodological study to evaluate the scales consisted of 155 respondents selected from a first-stage sample of 1,000 adults aged 18 or older. First-stage respondents were selected from the Boston metropolitan area and screened on the telephone to determine whether they had any emotional problems. Respondents reporting emotional problems at the first stage were oversampled when selecting the 155 respondents at the second stage. The selected respondents were interviewed by trained clinicians their home using both the NHSDA methodology and using a structured clinical interview. The first interview included the three scales described above using audio computer-assisted self-interviewing (ACASI). Respondents completed the ACASI portion of the interview without discussing their answers with the clinician. After completing the ACASI interview, respondents were then interviewed using the 12-month nonpatient version of the Structured Clinical Interview for DSM-IV (SCID) (First, Spitzer, Gibbon, & Williams, 1997) and the Global Assessment of Functioning (GAF) (Endicott, Spitzer, Fleiss, & Cohen, 1976) to classify respondents as either having or not having SMI.

The data from the 155 respondents were analyzed using logistic regression analysis to predict SMI from the scores on the screening questions. Analysis of the model fit indicated that each of the scales alone and in combination were significant predictors of SMI and the best fitting models contained either the CIDI-SF or the K6/K10 alone. Receiver operating characteristic (ROC) curve analysis was used to evaluate the precision of the scales to discriminate between respondents with and without SMI. This analysis indicated that the K6 was the best predictor. The results of the methodological study are described in more detail in a forthcoming paper (Kessler et al., in press).

To score the items on the K6 scales, they were first coded from 0 to 4 and summed to yield a number between 0 and 24. This involved transforming response categories for the six questions (DSNERV1, DSHOPE, DSFIDG, DSNOCHR, DSEFFORT, and DSDOWN) given below so that "all of the time" is coded 4, "most of the time" is coded 3, "some of the time" 2, "a little of the time" 1, and "none of the time" 0, with "don't know" and "refuse" also coded 0. Summing across the transformed responses obtains a score with a range from 0 to 24. Respondents with a total score of 13 or greater were classified as having a past year SMI. This cutpoint was chosen to equalize false positives and false negatives.

The questions comprising the K6 scale are given below:

DSNERV1

Most people have periods when they are not at their best emotionally. Think of one month in the past 12 months when you were the most depressed, anxious, or emotionally stressed. If there was no month like this, think of a typical month. During that month, how often did you feel nervous? All of the time Most of the time Some of the time A little of the time None of the time DK/REF

Response categories are the same for the following questions:     

DSHOPE	During that same month when you were at your worst emotionally . . .how often did you feel hopeless?
DSFIDG	During that same month when you were at your worst emotionally . . . how often did you feel restless or fidgety?
DSNOCHR	During that same month when you were at your worst emotionally . . . how often did you feel so sad or depressed that nothing could cheer you up?
DSEFFORT	During that same month when you were at your worst emotionally . . . how often did you feel that everything was an effort?
DSDOWN	During that same month when you were at your worst emotionally . . . how often did you feel down on yourself, no good, or worthless?

Table B.1 Summary of 2001 NHSDA Suppression Rules
Estimate	Suppress if:
Prevalence rate,  , with nominal sample size, n, and design effect, deff	The estimated prevalence rate, , is < 0.00005 or > 0.99995, or > 0.175 when < 0.5, or > 0.175 when > 0.5, or Effective n < 68, or n < 100 where Effective n = Note: The rounding portion of this suppression rule for prevalence rates will produce some estimates that round at one decimal place to 0.0 or 100.0 percent but are not suppressed from the tables.
Estimated number (numerator of )	The estimated prevalence rate, , is suppressed. Note: In some instances when is not suppressed, the estimated number may appear as a 0 in the tables; this means that the estimate is > 0 but < 500 (estimated numbers are shown in thousands).
Mean age at first use, , with nominal sample size, n	RSE() > 0.5, or n < 10
Incidence rate,	Rounds to < 0.1 per 1,000 person-years of exposure, or RSE() > 0.5
Number of initiates,	Rounds to < 1,000 initiates, or RSE() > 0.5

Source: SAMHSA, Office of Applied Studies, National Household Survey on Drug Abuse, 2001.

Table B.2 Weighted Percentages and Sample Sizes for 1999 to 2001 NHSDAs, by Screening Result Code

Source: SAMHSA, Office of Applied Studies, National Household Survey on Drug Abuse, 1999, 2000, and 2001.

Table B.3 Weighted Percentages and Sample Sizes for 1999 to 2001 NHSDAs, by Final Interview Code, among Persons Aged 12 or Older

Source: SAMHSA, Office of Applied Studies, National Household Survey on Drug Abuse, 1999, 2000, and 2001.

Table B.4 Weighted Percentages and Sample Sizes for 1999 to 2001 NHSDAs, by Final Interview Code, among Youths Aged 12 to 17

Source: SAMHSA, Office of Applied Studies, National Household Survey on Drug Abuse, 1999, 2000, and 2001.

Table B.5 Weighted Percentages and Sample Sizes for 1999 to 2001 NHSDAs, by Final Interview Code, among Persons Aged 18 or Older

Source: SAMHSA, Office of Applied Studies, National Household Survey on Drug Abuse, 1999, 2000, and 2001.

Table B.6 Response Rates and Sample Sizes for the 1999 to 2001 NHSDAs, by Demographic Characteristics

Source: SAMHSA, Office of Applied Studies, National Household Survey on Drug Abuse, 1999, 2000, and 2001.

This page was last updated on June 16, 2008.

SAMHSA, an agency in the Department of Health and Human Services, is the Federal Government's lead agency for improving the quality and availability of substance abuse prevention, addiction treatment, and mental health services in the United States.