NAEP Item Development ProcessThe National Assessment of Educational Progress (NAEP) is based on content frameworks and test specifications developed by the National Assessment Governing Board. The National Center for Education Statistics (NCES) oversees the development of the assessment items and scoring rubrics. Both agencies employ extensive reviews by multiple groups at various stages. The Governing Board and NCES play crucial roles in ensuring NAEP’s quality and that it is developed through an extensive process. Phase I. The Governing Board Framework and Test Specification Development The Governing Board is responsible for developing the framework and test specifications that serve as the blueprint for the assessment. The Governing Board utilizes panels of business representatives, members of the general public, local and state policymakers, curriculum specialists from state and local education agencies, practitioners, noted researchers in the disciplines, and educators. Three committees (a steering committee for policy guidance, a planning committee for content expertise, and a technical advisory committee to deal with psychometric issues relevant to large-scale assessments) produce two documents:
These documents are reviewed in multiple public forums by educators and the general public. State curriculum specialists and testing directors also provide reviews. Both the Governing Board Assessment Development Committee and the full Board approve the framework and assessment specifications. Phase II. Item Development NCES calls on many constituents to guide the process and review the assessment. The NAEP contractor begins this process by developing more than twice the number of needed items. This item pool is then subjected to:
Then a standing committees of content experts, state and local education agency representatives, teachers, parents, and representatives of professional associations reviews the items. Each standing committee considers:
This is followed by a state item review where further feedback is provided. Items are then revised and submitted to the Governing Board assessment development committee for approval prior to field testing. The field test is used to finalize the testing instrument. Items are revised based on student data, and the item set is once again subjected to review following the same procedure described above. Then, a final set of test items is assembled for NCES and the Governing Board review and approval. Phase III. Assessment After the main assessment, items are once again examined. In the rare cases where item statistics indicate any remaining problems, an item is dropped from the assessment. The remaining items are secured for reuse in future assessments, with a subset of items publicly released.
|