|
 |
|||||||||||
 |
|
|||||||||||
|
|
||||||||||||
A coordinated five-year sample design was developed for 1999 through 2003. The sample design for the 2002 main study, as a subsample of the five-year study, consisted of a deeply stratified, multi-stage, area probability design. Exhibit 2.1 presents details of the sample design.
The coordinated 1999–2003 design calls for 50 percent overlap in first stage units (area segments) between each successive year of the five-year study following completion of the 1999 survey.
The first stage of the sample selection procedures began by geographically partitioning each state into roughly equal-sized field interviewer (FI) regions. These regions were formed as a means of stratification so that each area would yield roughly the same expected number of interviews during each data collection period. This partitioning divided the United States into 900 FI regions made up of counties or groups/parts of counties.
These FI regions were subdivided into smaller geographic areas—called segments—that served as the primary sampling units. In general, segments consisted of adjacent Census blocks and were equivalent to area segments selected at the second stage of selection in NSDUHs conducted prior to 1999. A total of 96 segments per FI region were selected (with probabilities proportional to size): 24 to field the five-year study and 72 to serve as backups in case of sample depletion or to field any supplemental studies SAMHSA may request. For the 2002 survey, a total of 7,200 segments within the 900 FI regions were selected. Of the total, 3,600 segments were overlap segments used during the 2001 survey, 3,576 segments were new, and 24 segments were duplicates of segments used in previous years. For this last category, the same area had been listed previously under a different segment identification number, so the original listing was used instead of relisting the same area.
After selecting these new areas, the process of counting and listing (C/L) the dwelling units (DUs) within each new segment ensued. Segments to be used in 2002 were listed between April and November of 2001. Once all DUs for a particular quarter were listed, the second-stage selection process identified sample dwelling units (SDUs) for inclusion in the study.
At the final stages of selection, five age group strata were sampled at different rates. These five strata were defined by the following age group classifications: 12–17, 18–25, 26–34, 35–49, and 50 years old and over. No race/ethnicity groups were purposely over-sampled for the 2002 main study. However, consistent with previous NSDUHs, the 2002 NSDUH was designed to over-sample younger age groups.
Preparations for C/L field activities began with the decision to use the existing NSDUH data collection management structure to supervise counting and listing. All current Field Supervisors (FSs) were asked to handle the administrative tasks for the listers hired for their area. These tasks included completion of the initial hiring process, segment assignment, managing the timely completion of segments, and weekly approval of time and expense reports. (Exceptions occurred in a few states to allow those FSs and their field staff to concentrate solely on screening and interviewing work. In those states, traveling lister teams completed the C/L work.) For technical supervision such as how to handle a specific segment, all listers contacted the manager for Counting and Listing (C/L Manager) for answers and advice.
Beginning in March 2001, FSs recruited listing staff from their existing staff of field interviewers. Experienced listers not currently working as NSDUH interviewers were also available for hire. A total of 335 listers were hired, certified, and worked from April through November 2001, to complete counting and listing operations for the 2002 NSDUH.
All hired listers received a home study training package containing a memorandum and materials including a project C/L manual; C/L video tape; hire letter; Data Collection Agreement; 2002 NSDUH C/L Project Specification Sheet; and a certification packet which included questions about procedures as well as path-of-travel exercises. Staff had two weeks upon receipt of this package to complete the certification test and return it to RTI for evaluation. Of the 347 training packages distributed, only12 hired listers did not pass the certification test. They received feedback about their efforts including copies of the questions missed but were not allowed to work as listers. All certified listers received their bulk listing supplies.
A select group of nine listers were chosen to serve as traveling listers. All nine of these listers had served on the traveling listing team during the previous year's C/L work. These travelers reported directly to a Traveling C/L Manager who provided administrative supervision in addition to managing their workload and assignments.
A group of RTI survey specialists attended classroom training in June to learn C/L procedures. Training included detailed instruction in proper C/L protocol and the completion of actual segments selected for the state of North Carolina.
Newly certified listers were then authorized to begin their C/L assignments. All listers sent their completed assignments directly to the Sampling Department at RTI where they were carefully edited. Feedback was provided to any listers who had significant errors. Problem segments were either refielded (for correction of major errors) or were corrected by sampling staff through discussions with the lister. In some cases, the lister returned to the segment to review the items in question.
Prior to the start of actual C/L field work, segment packets were assembled at RTI. Each packet contained maps of the selected area, listing forms, and blank segment information sheets. A copy of the maps remained at RTI for reference when assisting with problems encountered in the field.
Beginning in April, segment kits were assigned and sent to those listers who had completed the certification process and were ready to begin listing. Once the remaining staff became certified, they received an assignment as well. Listers recorded the address or description of up to 400 dwelling units (DUs) in each segment.
To reduce the time required to count and list segments, several procedures were implemented to maximize efficiency. In many cases the "count" step was eliminated: the lister could immediately list the segment unless during the initial trip around the boundaries of the segment it was apparent the segment had experienced additional construction or the lister determined that the segment was large (i.e., 400+ DUs). As had been done on prior rounds of the NSDUH, a rough count procedure was allowed for segments containing large geographic land areas, large DU counts (400+ DUs), or significant growth in residential DUs (typically, 1,000+ DUs). This procedure permitted listers to obtain an approximate count of residential DUs in these segments from secondary sources—such as the post office, fire department, or county or city planning office—without having to conduct an exact count.
If a lister came across a segment that needed subsegmenting, the lister called in the initial DU counts to RTI's Sampling Department, who could sometimes subsegment it over the telephone (any segment with more than 400 DUs generally required subsegmenting). In cases involving traveling listers, the telephone subsegmenting process allowed the lister to—in one trip—count and list a segment with 400 or more DUs, rather than experiencing a delay of one or two weeks and necessitating a second trip to the segment. For difficult subsegmenting tasks, the segment materials were sent to RTI to be handled directly by sampling personnel. Of the 3,576 new segments listed for the 2002 survey, 429 required subsegmenting. When obvious and possible, sampling staff completed any needed subsegmenting prior to the assignment of the segment to the lister, although the majority of subsegmenting occurred during the listing process.
The counting and listing of almost all of the segments was completed by the end of November 2001 (the exceptions involved a few access problems or late segments that had to be returned to the field for re-listing). Once the segments were listed and the completed segment kits were received at RTI, an editing process of the completed materials checked for and deleted any DUs located outside segment boundaries, ensured that listing sheets matched segment sketches/maps, and verified that proper listing order and related listing rules were observed. During this editing process, the sampling staff also checked all subsegmenting that occurred in the field to ensure it was done correctly.
Listed DUs were keyed into a computer control system. A selection algorithm selected the specific sample dwelling units (SDUs) to be contacted for the study. Prior to the beginning of the appropriate quarter, FSs assigned segments (or partial segments) to their interviewing staff. Interviewers received all assigned SDUs on their Newton handheld computer. Each selected unit and the next listed unit (for use as a sample check to capture missed dwelling units during screening and interviewing) were also printed on Selected DU Lists. These lists, along with copies of the handwritten listing forms and maps, were distributed to the assigned field staff before the start of each quarter.
During the screening process, Field Interviewers (FIs) were trained to identify any unlisted DUs that existed within the SDU or within the interval between the SDU and the next listed DU. If the missed DUs were housing units, they were automatically entered into the Newton (up to established limits) and selected for participation. At most, the FI could independently add five missed DUs per SDU and a maximum of ten missed DUs per segment. If the FI discovered more than these amounts or if the missed DUs were group quarters units, the FI called the FS. The FS then either called RTI's Sampling Department for further instructions or instructed the FI to call the Sampling Department directly, depending on the situation.
While no upper-limit was placed on the total number of DUs that could be added to a segment by RTI's Sampling Department, the FIs were instructed to notify RTI of any significant listing problems. In a small number of segments, portions of these segments had to be re-listed during the screening and interviewing phase. Table 2.2 indicates the number of segments that experienced added DUs, as well as the total number of added DUs for the 2002 NSDUH.
In many of the major urban areas, field staff had some difficulties gaining access to locked buildings, and listers in particular had some trouble listing very large public housing complexes. Access in some suburban areas proved problematic as well; more and more planned communities have intercoms, guarded gatehouses or entryways outfitted with cameras and scrambled buzzer systems. Access to military bases, college dormitories, and large retirement communities also proved problematic at times. Based on experience, these types of access problems were expected. Special mechanisms or protocols were in place to handle them promptly and in some cases avoid them entirely.
Access problems were typically resolved through effective follow-up efforts of supervisory staff, including situation-specific letters of request and in-person visits by the Field and/or Regional Supervisors. In particularly difficult situations, SAMHSA offered additional support via special refusal conversion letters or telephone follow-ups by the Project Officer.
As in past years, the often problematic access to military bases was handled with a formal and standardized approach for 2002. Through joint RTI/SAMHSA efforts, a contact person within the Pentagon for each branch of the service was identified. These individuals were advised in advance of base selections for the year. They then notified the base commanders regarding RTI's need to access these bases for both listing and screening/interviewing work. Additionally, standard letters and informational packages were sent by RTI staff to help obtain access to all selected bases. These efforts were effective: access to all selected bases was secured.
Access to colleges and universities is sometimes problematic. RTI used several standard approaches to accommodate the concerns of school administrators. Having standardized letters available that addressed reoccurring issues with a variety of attachment options was very effective.
Most schools requested or required only a letter stating the sponsor and the purpose of the study, and identifying the lister or data collection staff. However, some schools wanted more complete information and the right to approve the field data collection procedures and personnel working in and around their campuses. Most of these situations resulted in packages being sent that contained:
Included with all letters and packets was an endorsement letter signed by the presidents of Duke University and the University of North Carolina at Chapel Hill. In the end, all of the private educational institutions expressing concerns cooperated in the counting and listing phase of the 2002 NSDUH.
Nine segments were identified during the counting and listing phase as difficult to access during months with unusual weather. Including 18 overlap segments from the 2001 study, there were a total of 27 segments in 2002 with access issues. Most involved roads made impassable by snow during the winter months. Others involved roads inaccessible due to rain, and one or two isolated locations involved water-only access that often froze during the winter months. If segments with weather or geographic access problems were selected for a quarter in which the access would be a problem (generally Quarters 1 or 4), the segment was switched with a segment in the same region for an appropriately paired time period. For example, inaccessible first quarter segments were switched with second quarter segments in the same region that would be more accessible during the first quarter; fourth quarter segments were switched with more easily accessed third quarter segments. Generally the "switched" segment was selected because it had more accessible road surfaces, was more urban, or had fewer inaccessible roads.
In a few locations, such as some areas in Alaska, there were no segments that were better for reassignment during the problematic time period. When that happened, staff made prompt assignments, emphasized early completion of the work, and tried to plan around good weather forecasts to accomplish the field work as early in the period as possible.
Exhibit 2.1
2002 NSDUH Sample Design Summary
First Stage of Selection for the Main Study: Segments
The 2002 design provided for estimates by state in all 50 states and the District of Columbia. States should therefore be viewed as the "first level" of stratification as well as a reporting variable. Eight states, labeled the "big" states in Table 2.1, had a sample designed to yield 3,600 respondents per state. The remaining 43 "small" states1 had a sample designed to yield 900 respondents per state.
The larger sample sizes obtained at the state level, along with small area estimation techniques refined under previous NSDUH contracts, enabled the development of estimates for all states, for several demographic subgroups within each state (i.e., age group and race/ethnicity group), and for some Metropolitan Statistical Areas and a few small areas in the "big" states.
The "second level" of stratification defined contiguous geographic areas within each state and also corresponded in size to the annual assignment for a single field interviewer (FI). These FI regions were of approximately equal population size in terms of allocated sample.
Additional implicit stratification was achieved by sorting the first-stage sampling units by an MSA/SES (Metropolitan Statistical Area/socioeconomic status) indicator2 and by percentage of non-Hispanic white. The first stage sample units for the 2002 NSDUH were selected from this well-ordered sample frame.
For the first stage of sampling for the 2002 NSDUH, each of the FI regions was partitioned into noncompact clusters of dwelling units by aggregating adjacent Census blocks. Consistent with the terminology used in previous NSDUH studies, these geographic clusters of blocks were referred to as segments. On average, segments were formed so that they contained at least 175 dwelling units and were constructed using 1990 Decennial Census data supplemented with revised population counts obtained from outside sources. A sample dwelling unit in the NSDUH refers to either a housing unit or a group quarters listing unit (such as a dormitory room or a shelter bed).
A sample of segments was selected within each FI region, with probabilities proportionate to a composite size measure and with minimum replacement. Segments were formed so that they contained sufficient numbers of dwelling units to support three annual NSDUH samples. This allowed half of the segments used in any given year's main sample to be used again in the following year as a means of improving the precision of measures of annual change. This also allows for any special supplemental sample or field test that SAMHSA may wish to conduct in any given NSDUH year within the same segments.
In order to coordinate the sample selection for 1999 through 2003, 96 segments were selected within each FI region. An equal probability subsample of eight segments was used for the 2002 NSDUH. These eight segments were randomly assigned to quarters and to two waves within each quarter. The waves used in the 2002 NSDUH were designated as Waves 4 and 5. Wave 4 segments were used for the 2001 and 2002 surveys. New dwelling units (i.e. those not previously selected for the 2001 study) were selected from the Wave 4 segments for 2002. Wave 5 segments were new for 2002 and will be used again for the 2003 survey.
Data from roughly one-fourth of the final sample of respondents was collected during each calendar quarter. This important design feature helped control any seasonal bias that might otherwise exist in drug use prevalence estimates and other important NSDUH outcome measures of interest.
Second Stage of Selection for the Main Study: Listed Lines
Before any sample selection within selected segments began, specially-trained staff listed all dwelling units and potential dwelling units within each newly selected area segment. A dwelling unit is either a housing unit for a single household or one of the eligible noninstitutional group quarters that are part of the defined target population. The listings were based primarily on observation of the area segment and could include vacant dwelling units and units that appeared to be dwelling units but were actually used for nonresidential purposes. The objective of the listing was to attain as complete a listing as possible of eligible residential addresses; any false positives for residences were eliminated during the household screening process after the sample was selected.
The sampling frame for the second stage of sample selection was the lines of listed dwelling units and potential dwelling units. After accounting for eligibility, nonresponse, and the third-stage sample selection procedures (including a response rate adjustment for the effect of the $30 incentive), it was determined that 182,250 lines were needed to obtain a sample of 67,500 responding persons distributed by state and age-group. During the study's implementation, however, a total of 178,013 lines were selected and yielded a final respondent sample of 68,126 (as shown in Table 2.1). These lines were selected among lines not used in the 2001 survey (overlap segments) and the complete list of dwelling units (new segments).
As in previous years, if an interviewer encountered any new dwelling unit in a segment or found a dwelling unit missed during the counting and listing activities, the new/missed dwellings were selected into the NSDUH using a half-open interval selection technique.3 That selection technique eliminated any frame bias that might have been introduced because of errors and/or omissions in counting and listing activities and also eliminated any bias that might have been associated with using "old" segment listings.
Third Stage of Selection for the Main Study: Persons
After dwelling units were selected within each segment, an interviewer visited each selected dwelling unit to obtain a roster of all persons aged 12 and over residing in the dwelling unit. This roster information was then used to select zero, one, or two persons for the survey. Sampling rates were pre-set by age group and state. Roster information was entered directly into the electronic screening instrument (the Newton) which automatically implemented this third stage of selection based on the state and age group sampling parameters.
Using an electronic screening instrument also provided the ability to impose a more complicated person-level selection algorithm at the third stage of selection. As a result of this unique design feature, any two survey-eligible people within a dwelling unit had some chance of being selected—i.e., all survey eligible pairs of people had some non-zero chance of being selected. This design feature is of interest to NSDUH researchers because it allows analysts to examine how the drug use propensity of one individual in a family relates to that of other family members residing in the same dwelling unit (e.g., the relationship of drug use between a parent and child). In 2002, a parameter was added to the person selection process that increased the number of selected pairs within dwelling units without unduly diminishing response rates.
As illustrated in Table 2.1, at the third stage of selection, 80,581 people were selected from 136,349 screened and eligible dwelling units. A total of 68,126 completed interviews were obtained from these 80,581 selected persons.
Expected Precision of NSDUH Estimates
The multi-stage, stratified NSDUH design has been optimally constructed to achieve specified precision for various person subpopulations of interest. These SAMHSA-specified, precision requirements call for the expected relative standard error on a prevalence of 10% not to exceed the amounts listed below.
For the main study:
To achieve these precision requirements and meet state sample-size requirements, the optimal person-level sample distribution by strata was determined that minimized data collection costs while simultaneously meeting the above-specified precision requirements for several critical NSDUH outcome measures.
The precision constraints in the design optimization models were set up using local area predictions of drug use from a project involving small area estimation techniques to generate local area estimates from 1991–1993 NSDUH data. Drug use estimates across strata were appropriately scaled to reflect the generic 10% prevalence.
Table 2.1
Sampling Summary of 2002 Main Study NSDUH
| Statistic | Small States | Big States | Total |
|---|---|---|---|
| Total Sample | |||
| FI Regions | 516 | 384 | 900 |
| Segments | 4,128 | 3,072 | 7,200 |
| Selected Lines | 102,255 | 75,758 | 178,013 |
| Eligible Dwelling Units | 85,793 | 64,369 | 150,162 |
| Completed Screening interviews | 78,748 | 57,601 | 136,349 |
| Selected Persons | 45,534 | 35,047 | 80,581 |
| Completed Interviews | 38,828 | 29,298 | 68,126 |
| Average Per State | |||
| FI Regions | 12 | 48 | |
| Segments | 96 | 384 | |
| Selected Lines | 2,378 | 9,470 | |
| Completed Interviews | 903 | 3,662 | |
| Interviews Per Segment | 9.41 | 9.54 | |
| Average Per State And Quarter | |||
| Segments Per FI Region | 2 | 2 | |
| Interviews Per FI Region | 18.81 | 19.07 | |
| Interviews Per Segment | 9.41 | 9.54 | |
| Total States | 43 | 8 | 51 |
| Total Interviewers (approximate number that varied by quarter) |
516 | 384 | 900 |
Note:
"Small" states refers to states where the design yielded 903 respondents on average. "Big" states refers to states where the design yielded 3,662 respondents on average.
Table 2.2
Segments with Added Dwelling Units
2002 NSDUH
| Number of Added DUs per Segment (X) |
Number of Segments with X Added DUs |
Cumulative Number of Added DUs* |
|---|---|---|
| 1 | 499 | 499 |
| 2 | 172 | 843 |
| 3 | 76 | 1071 |
| 4 | 43 | 1,243 |
| 5 | 26 | 1,373 |
| 6 | 13 | 1,451 |
| 7 | 9 | 1,514 |
| 8 | 10 | 1,594 |
| 9 | 5 | 1,639 |
| 10 | 6 | 1,699 |
| 11 | 1 | 1,710 |
| 12 | 1 | 1,722 |
| 13 | 1 | 1,735 |
| 14 | 1 | 1,749 |
| 17 | 1 | 1,766 |
*Total number of added DUs = 1,766
|
|
|
 |
Click to Return to OAS Home Page |
|
This page was last updated on May 16, 2008.
|
|
