NAEP Analysis and Scaling

			Initial Activities Estimation of NAEP Score Scales Estimation of Population and Student Group Distributions Scale Linking and Transformation to the Reporting Metric Summary Statistics for Scale Scores of Groups Drawing Inferences From NAEP Results Describing NAEP Scale Score Distributions

The goal of the analysis of NAEP data is to summarize the performance of groups of students. NAEP analysis consists of several steps.

Initial activities include the calculation of simple counts and percentages, as well as classical test statistics. The purpose of initial analysis activities is threefold. First, initial activities verify the accuracy of the data and data files used in the analysis. Second, initial activities provide the first indication of aspects of the data and analysis that require special consideration and attention. Finally, the initial item analysis provides initial starting values for use in the scaling process. Some of the initial activities are conducted without student weights or with preliminary student weights, but final student weights are used whenever possible.

After the initial activities are completed, NAEP score scales are created via Item Response Theory (IRT) and scale score distributions are estimated for groups of students. When the score scales are created, parameters describing the item response characteristics are estimated. In years in which state assessments take place, the same score scales are used for both national and state assessment results. Because NAEP is not concerned with individual test results, NAEP does not produce individual test scores but does produce estimates of scale score distributions for groups of students. The resulting scale score distributions describing student performance are transformed to a NAEP scale, and summary statistics of the scale scores are estimated. Statistical tests are used to make inferences about the comparisons of results for different groups of students or for different assessment years. Finally, NAEP scale score distributions are described via National Assessment Governing Board achievement levels and/or item mapping procedures. Subjects for which the Board has established achievement levels include civics, geography, mathematics, reading, science, U.S. history, and writing.

Separate analysis plans are developed for each NAEP special study and for long-term trend assessments. Often these plans include all of the steps described above. For specific studies special approaches might be taken.

Last updated 17 July 2008 (DR)

Printer-friendly Version