|
NAEP Sample Design NAEP Sample Design
For each assessment cycle, a sample of students in designated grades within both public and private schools is selected for assessment. The student population to be assessed is located in schools throughout the United States. From time to time, students in selected U.S. territories and Department of Defense Dependents Schools (DoDDS) are also included. The National Center for Education Statistics (NCES) specifies which territories and DoD schools are to be included in a particular assessment year. In some years (for example, 2001), assessments are provided only for the Nation as a whole, and for limited sub-entities, such as Regions and Divisions. In these years the student samples are limited in size and distributed throughout the Nation. In other years (for example, 2000, 2002, and 2003), assessments for public schools are also provided for individual states and other jurisdictions (for example, Bureau of Indian Affairs, territories and DoD), in which case, larger samples of students are selected from schools within each jurisdiction to meet the reliability requirements for the jurisdiction. In these "state assessment" years, the jurisdiction samples are combined to produce the required national estimates. In all cases, the sample selection process utilizes a probability sample design in which each school and each student has a known probability of being selected. This design also provides for the calculation of standard errors for the derived estimates. Public School Selection in State Assessment YearsThe selection of a sample of public school students for state assessment involves a complex multistage sampling design with the following stages:
The Common Core of Data (CCD) file, a comprehensive list of operating public schools in each jurisdiction that is compiled each school year by the National Center for Education Statistics (NCES), is used as the sampling frame for the selection of sample schools. The CCD also contains information about grade span, enrollment, and location of each school. In addition to the CCD list, a set of specially sampled jurisdictions is contacted to determine if there are any newly formed public schools that were not included in the lists used as sampling frames. Considerable effort is expended to increase the survey coverage by locating public schools not included in the most recent CCD file. As part of the selection process, public schools are combined into groups known as strata on the basis of various school characteristics related to achievement. These characteristics include the physical location of the school, extent of minority enrollment, state-based achievement scores, and median income of the area in which the school is located. Stratification of public schools occurs within each state. Combining schools within strata by such selected characteristics provides a more ordered selection process with improved reliability of the assessment results. On average, a sample of approximately 100 grade-eligible public schools is selected within each jurisdiction; within each school, about 60 students are selected for assessment. Both of these numbers may vary somewhat, depending on the number and enrollment size of the schools in a jurisdiction, and the scope of the assessment in the particular year. Students are sampled from a roster of individual names, not by whole classrooms. The total number of schools selected is a function of the number of grades to be assessed, the number of subjects to be assessed, and the number of states that elect to participate in the program, as well as the desired precision of the survey estimates, and the average number of students to be selected within a sampled school. Private School Selection in State Assessment YearsIn years in which state-level samples are drawn for public schools, private schools are classified by type (Catholic, Lutheran, Conservative Christian, non-sectarian, and other private schools), and are grouped for sampling by geography (Census division), classification by urbanization (multi-category type of location), and minority enrollment. About 700 private schools, on average, are included, with up to 60 students per school selected for assessment. These samples are not large enough to support state-level estimates for private schools. Thus, inferences for private schools are limited to the national level, even in years when public school assessments are state-specific. A national sample of private schools in all grades is then drawn from a list compiled through the Private School Survey (PSS), which is a mail survey of all U.S. private schools carried out biennially by the Census Bureau under contract to NCES. This survey is primarily a census (all schools are canvassed), but the survey also includes a sample from an area frame to provide full coverage. The PSS list is updated for new schools only for a sample of Catholic dioceses because of the lack of an organizational structure akin to school districts for non-Catholic private schools. National-Only Assessment YearsIn years when the NAEP samples are intended only to provide representation at the national level and not for each individual state, the public and private school selection process involves an additional stage of sampling. Rather than selecting schools directly from lists of schools, the first stage of sampling involves selecting a sample of some 50 to 100 geographic primary sampling units (PSUs). Each PSU is composed of one or more counties. PSUs vary in size considerably, and the number of counties that comprise a PSU can change over time with geographic shifts in the population. Generally about 1,000 PSUs cover the total U.S. population, from which a sample is selected as the first stage of the NAEP sample. Within the set of selected PSUs, public and private school samples are selected using similar procedures to those described above for the direct sampling of schools from lists. The school samples are clustered geographically, which results in a more efficient data collection process. The selection of PSUs is not necessary when the sample sizes are large in each state, as in state assessment years. Last updated 11 March 2009 (RF) |
||||||||
