|
NAEP Sample Design NAEP Sample Design
A new sample design is created for every National Assessment of Educational Progress (NAEP). Because each year's design differs, this section of the website discusses each separately; however, some general facts can be said about all NAEP assessments. In each assessment year, a sample of students in designated grades within both public and private schools is selected for assessment. The geographic scope always covers the entire United States and—in some assessment years—U.S. territories and possessions. Since 1990, the samples of public schools and their students in each state have been large enough to provide state-level estimates. In all cases, the selection process has utilized a probability sample design in which every school and student has a non-zero chance of being selected, and standard errors can be calculated for the derived estimates. The selection of a sample of students for the assessment involves a complex multistage sampling design with the following stages:
The Common Core of Data (CCD) file a comprehensive list of operating public schools in each jurisdiction that is compiled each school year by the National Center for Education Statistics (NCES)—is used as the sampling frame for the selection of sample schools. The CCD also contains information about grade span and enrollment of each school, as well as its location. As part of the selection process, in years when relatively large samples are selected, public schools are combined into groups known as strata on the basis of various school characteristics related to achievement. These characteristics include the physical location of the school, extent of minority enrollment, state-based achievement scores, and median income of the area in which the school is located. Stratification of public schools occurs within each state. Combining schools within strata provides a more ordered selection process with improved reliability of the assessment results because each stratum group is adequately represented in the sample. Generally, a sample of approximately 100 grade-eligible public schools is selected within each jurisdiction (or about 7,000 to 10,000 all together); within each school, about 60 students are selected for assessment. Both of these numbers may vary somewhat, depending on the number and enrollment size of the schools in a jurisdiction, as well as the scope of the NAEP program in the particular assessment year. Students are sampled from a roster of individual student names, not by entire classrooms. The total number of schools is a function of the number of grades to be assessed, the number of subject areas to be assessed, and the number of states that choose to participate in the program. The national sample of private schools in all grades is drawn from a list compiled through the Private School Survey (PSS), which is a mail survey of all U.S. private schools carried out biennially by the Census Bureau under contract to NCES. In years in which relatively large samples are to be selected, private schools are classified by type (e.g., Roman Catholic, Lutheran), and are grouped for sampling by geography (Census division), degree of urbanization of location, and minority enrollment. About 700 private schools, on average, are included in a national sample, with up to 60 students per school selected for assessment. When the samples are to be representative only at the national level, the procedure for selecting both public and private schools typically involves a somewhat different process. Rather than selecting schools directly from lists of schools, the first stage sampling involves selecting a sample of some 50 to 100 geographic areas called primary sampling units (PSUs). A PSU is a geographic area comprising a consolidated metropolitan statistical area (CMSA), a metropolitan statistical area (MSA), a New England County Metropolitan Area (NECMA), a county, or a group of contiguous counties in the United States (including Alaska, Hawaii, and the District of Columbia.) The entire Untied States was divided into 1,027 PSUs in 2001. Each PSU met a minimum size requirement and was contained within one of four geographic regions. Within the set of selected PSUs, public and private school samples are selected using similar procedures to those described above for the direct sampling of schools from lists. The samples are clustered geographically which results in a more efficient data collection process. The selection of PSUs is not necessary when the sample sizes are large in each state. Last updated 18 June 2008 (MH) |
||||||||
