Item Development Process

			Background Questions Geography Cognitive Items Mathematics Cognitive Items Reading Cognitive Items Science Cognitive Items U.S. History Cognitive Items

NAEP assessments include cognitive items, test questions that are designed to assess what students know and can do, and background questions. Cognitive items are based on the framework and specifications documents for each assessment subject. These types of items include multiple-choice items, constructed-response items scored dichotomously, and constructed-response items scored polytomously. Background questions gather information such as time spent studying. NAEP subject areas include the arts, civics, economics, foreign language, geography, mathematics, reading, writing, science, U.S. history, or world history.

The item-development steps¹ for each subject area are as follows:

1. The National Assessment Governing Board provides content frameworks and item specifications in each subject area.

2. Instrument development committees in each subject area provide guidance to NAEP staff about how the objectives can be measured given the constraints of resources and the feasibility of measurement technology. The committees make recommendations about priorities for the assessment (within the context of the assessment framework) and the types of items to be developed.

3. Items are chosen for the assessment through an extensive selection process that involves the input of practitioners from across the country and members of the instrument development committees.

4. Specialists with subject-matter expertise and experience in creating items according to specifications develop and review the assessment questions.

5. The items and accompanying scoring guides are reviewed and revised by NAEP test development staff and external test specialists.

6. Representatives from the state education agencies meet and review all items and background questionnaires that are scheduled to be part of the state assessment.

7. Editorial and fairness reviews are conducted as required by NCES.

8. Field test materials are prepared, including those necessary to secure clearance by the federal Office of Management and Budget.

9. A field test is conducted in many of the states and jurisdictions slated to participate in the following year's operational assessment.

10. Representatives from state education agencies meet and review the field test results for all items selected for the state assessment.

11. Based on the field test analyses, new items for each subject area are revised or modified where necessary. The items once again undergo the full range of reviews.

12. Each subject area instrument development committee approves the selection of items to include in the following year's operational assessment.

13. After a final review and a check to ensure that each assessment booklet and each block meet the overall guidelines for the assessment, the booklets are typeset and printed.

¹ Item development steps for the background questionnaires are different and are explained in the background questionnaire section.

Last updated 12 June 2008 (MH)

Printer-friendly Version