Measuring the Status and Change of NAEP State Inclusion Rates for Students with Disabilities

November 2008

Authors: Sami Kitmitto and Victor Bandeira de Mello

Executive Summary

Since the late 1990s, participation rates of students with disabilities (SDs) in the National Assessment of Educational Progress (NAEP) from different states have fluctuated. To address concerns that these changes may affect the validity of reports on achievement trends, NAEP has

• instituted policies for providing test accommodations for students with disabilities;
• developed a methodology to correct for the bias resulting from changing inclusion rates, and
• implemented procedures to increase the number of students with disabilities who are included as test-takers, such as better training of field staff, better procedures to assign proper accommodations for students, and improved communications with schools.

States’ procedures for including and accommodating students with disabilities are also evolving.

To measure whether these strategies and changes are associated with higher state-by-state inclusion rates, we have developed two distinct approaches for comparing state inclusion rates with one another and gauging progress in their improvement over time. Both approaches rely on regression analysis to estimate the relationship between a student’s characteristics and the probability that the student is included on the NAEP assessment. One approach, the nation-based one, estimates one regression using data pooled from all states. The other, the state-specific approach, estimates the regression separately for each state. The relationships are estimated using individual-level data and are then used to establish expectations (or predicted probabilities) for the inclusion of students with disabilities with different characteristics. Individual-level predicted probabilities are aggregated to the state level to form state-level expected inclusion rates. The two approaches examined changes in inclusion rates from 2003 to 2005 and from 2005 to 2007 for grades 4 and 8 mathematics and reading assessments.

For the comparison between 2005 and 2007 described in this report, the two approaches produced similar results when comparing the indices of baseline status of inclusion and change over time:

• The majority of states did not make a statistically significant change in the rate of inclusion.
• Among states that did show a significant change, most were less inclusive in 2007 than in 2005.
  • For the nation-based approach: 8 out of 15 states for mathematics grade 4 were less inclusive in 2007 than in 2005; 17 out of 19 states for mathematics grade 8; 18 out of 26 states for reading grade 4; 21 out of 25 states for reading grade 8.
  • For the state-specific approach: 17 out of 19 states for mathematics grade 8 were less inclusive in 2007 than in 2005; 12 out of 22 states for reading grade 4; 14 out of 18 states for reading grade 8.
  • The exception was for the state-specific approach for mathematics grade 4, where more of the states with significant changes had increases: 8 out of 15.
• Most of the states whose inclusion rate significantly increased in 2007 had a relatively low inclusion rate in 2005.
  • All states with significant increases in inclusion rates in 2007 had relative inclusion rates in the bottom 50 percent in 2005 with the exception of one state for the nation-based method for grade 8 mathematics.
• States whose inclusion rate significantly decreased in 2007 had varied relative inclusion rates in 2005.
• The expected (predicted) inclusion rates varied from state to state by grade and subject.

Back to Top

• PDF 1 of 5 contains:
  Table of Contents
  Executive Summary
  Introduction
  -also includes front matter
  (532K PDF)
• PDF 2 of 5 contains:
  Chapter 1: Methodology
  Includes full description of state-specific and nation-based approaches
  (1411K PDF)
• PDF 3 of 5 contains:
  Chapter 2: Data (Summary Statistics)
  Chapter 3: Caveats and Cautions in Interpretation
  Chapter 4: Results (state-specific, nation-based, and comparisons)
  (864K PDF)
• PDF 4 of 5 contains:
  Appendix A: Calculation of Standard Errors
  Appendix B: Changes in Inclusion Rates from 2003 to 2005
  Appendix C: Recentering Measures
  (1102K PDF)
• PDF 5 of 5 contains:
  Appendix D: SD Background Questionnaires
  (593K PDF)
• The complete PDF of the report: Measuring the Status and Change of NAEP State Inclusion Rates for Students with Disabilities (3947K PDF)

NCES 2009-453 Ordering information

Suggested Citation
Kitmitto, S., and Bandeira de Mello, V., (2008). Measuring the Status and Change of NAEP State Inclusion Rates for Students with Disabilities (NCES 2009–453). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, Washington, D.C.

For more information, learn about how NAEP works to increase inclusion for students with disabilities in large-scale assessments and about the potential effects of exclusion upon educational assessments.

Back to Top

---

Last updated 01 December 2008 (NB)