December 9, 2005

Honorable Douglas D. Christensen
Commissioner of Education
Nebraska Department of Education
301 Centennial Mall, South, 6th Floor
P.O. Box 94987
Lincoln, Nebraska 68509-4987

Dear Commissioner Christensen:

Thank you for submitting Nebraska's assessment materials for review under the standards and assessment requirements of the Elementary and Secondary Education Act (ESEA), as amended by the No Child Left Behind Act of 2001 (NCLB). We appreciate the efforts required to prepare for the peer review and hope that the process provides useful feedback that will support your State's efforts to monitor student progress toward challenging standards.

External peer reviewers and U.S. Department of Education (ED) staff evaluated Nebraska's submission and found, based on the evidence received, that it did not meet most of the statutory and regulatory requirements of Section 1111(b)(3) of ESEA. The status of the Nebraska's assessment system is Final Review Pending. In this status, a State must clearly articulate to ED how it will meet the remaining requirements and be able to fully implement its standards and assessment system by the end of the 2005-2006 school year. Nebraska must also demonstrate that it has the capacity to:

• Provide guidance and technical assistance to local school districts on the NCLB standards and assessment requirements,
• Review district evidence of compliance with the NCLB standards and assessment requirements using professionally recognized standards and practices for assuring assessment alignment with standards, validity and reliability, and
• Approve local district systems of standards and assessments, which will allow students to achieve to high standards.

Summaries of the critical elements that met the NCLB standards and assessment requirements follow. The summaries also provide Nebraska with the critical elements that have been approved and will not need further review as well as those that partially met the requirements.

1.0 - Academic Content Standards: Nebraska meets this requirement at the State level for reading/language arts, mathematics and science. However, since Nebraska is a local control State and districts have the authority to adopt their own standards, no evidence was provided that all school districts had either adopted local academic content standards and/or adopted the State academic content standards in reading/language arts, mathematics and science in grades 3-8 and high school. As a result, we cannot conclude that all districts in Nebraska have approved academic content standards.

2.0 - Academic Achievement Standards: Nebraska does not meet this requirement in reading/language arts, mathematics or science. Independent analyses conducted under contract to the State of Nebraska indicate that the four performance levels adopted by Nebraska cannot be accurately reported. No evidence was provided regarding the extent to which all districts had approved academic achievement standards for reading/language arts and mathematics for each of their assessments in grades 3-8 and high school for regular and alternate assessments, for writing (grades 4, 8 and 11) or science.

3.0- Full Assessment System: Nebraska does not meet the requirement for assessments in grades 3-8 in reading/language arts and mathematics for the regular and alternate assessments. No evidence was provided regarding the extent to which all districts had approved academic assessments for reading/language arts and mathematics for all required grades (3-8 and high school) for regular and alternate assessments. Nebraska meets this requirement for writing.

4.0 - Technical Quality: Nebraska does not meet this requirement. The six Quality Criteria are not sufficiently rigorous to determine the validity, reliability, and bias of the variety of assessments used in each district for regular or alternate assessments.

5.0 - Alignment: Nebraska did not provide evidence that all districts meet this requirement for regular and alternate assessments in reading/language arts, mathematics or writing. The six Quality Criteria are not sufficiently rigorous to determine the alignment of district academic assessments to State or local academic content and achievement standards adopted by the local districts.

6.0 - Inclusion: Nebraska does not currently meet the requirement for inclusion of students on local (reading/language arts) and State (writing) assessments. Nebraska will not report inclusion rates for both reading/language arts and mathematics assessments assessed during the same year until after test results are generated for the 2005-06 school year. However, at grade 11 the percentages of students with disabilities and English language learners not tested exceeded 8% in mathematics for school year 2003-04, the most recent year reported.

7.0 - Reporting: Nebraska does not meet the NCLB assessment reporting requirements. Student performance must be reported using three performance levels (rather than the two currently reported). In addition, the performance levels must be content competency specific in reporting student performance results.

Additional evidence is needed to show how Nebraska meets the critical elements identified as partially or not meeting the requirement under the NCLB standards and assessment peer review guidance. More detailed information about the additional evidence that will be needed in order to be in compliance is outlined beginning with page four.

When the required additional evidence has been submitted, it will be subject to peer and ED staff review. A second peer review will need to be scheduled consistent with the previously announced dates. Enclosed with this letter are detailed comments from the peer review team that evaluated the Nebraska assessment materials. The peer reviewers are experts in the areas of standards and assessments; they reviewed and discussed the State's submission of evidence and prepared a consensus report that is documented as the Peer Notes. I hope you will find the reviewers' comments and suggestions helpful.

We look forward to working with Nebraska to support a high-quality assessment system. If you would like to discuss this further, please do not hesitate to call Zollie Stevenson, Jr. (202-260-1824) or Darla Marburger (202-260-2032) of my staff.

Sincerely,

Henry L. Johnson

Attachment

cc: Ms. Marilyn Peterson Dr. Pat Roschewski

For full approval, Nebraska will need to submit acceptable evidence that the assessment systems developed by local school districts meet nationally professionally recognized standards and practices in terms of the development of academic achievement standards, within district comparability, technical quality, alignment and reporting of results (beyond the categories of "met" and "not met") of the reading/language arts and mathematics assessments being used for NCLB accountability in grades 3-8 and high school. External studies commissioned by Nebraska have indicated that there are significant weaknesses in the technical quality and alignment of the district assessments. Nebraska has acknowledged that their decision rules for making technical quality determinations are not psychometrically correct… and that the State has yielded on some technical correctness to gain acceptability by district personnel (#22, page 14 of the Nebraska evidence submission).

Further, Nebraska must clarify the number of districts that have implemented their systems of assessment for reading and mathematics in grades 3-8, the number of school districts who plan to have all of those assessments in place during the 2005-06 school year, and the number who will not implement the 3-8 and high school reading and mathematics assessments during 2005-06 (regular and/or alternate assessments).

More specific information is provided below:

1.0 - ACADEMIC CONTENT STANDARDS

• Nebraska must provide evidence that local districts had formally approved/adopted by May 2003 or since that date, challenging academic content standards in reading/language arts and mathematics in all grades 3-8 and the 10-12 ranges.
• Nebraska staff must submit evidence that it has taken steps to ensure that all districts developing their own academic content standards in reading, mathematics, and science have met the requirements regarding challenging and rigorous standards that encourage the teaching of advanced skills. The current practice of reviewing a single standard in each content area to make judgments related to all standards in that area is not sufficient.
• Since the academic content standards for grades 5 and 8 in science are the same, the State should identify the guidance it has provided to school districts to define grade level expectations with more specificity.
• The State should provide information on how they validate stakeholder involvement in the development of the local district and State academic content standards including numbers participating in each district, areas of expertise, and whether individuals representing students with disabilities and English Language Learners were involved.

2.0 - ACADEMIC ACHIEVEMENT STANDARDS

• Nebraska must indicate how the State will address the concerns reported in several studies that the State commissioned that Nebraska's four levels of achievement cannot be reported accurately.
• Nebraska must describe the steps it will take (and provide the timeline) to ensure that academic achievement standards, including alternate achievement standards, developed by local school districts meet the requirements of the law regarding achievement levels, rigor, and that differentiation between levels are valid and reliable.
• Nebraska staff must submit evidence of the process used to ensure that all performance descriptors are content competency and grade specific. Student performance reports must show the student's performance in reading/language arts and mathematics in relationship to the content competencies (related to the academic content standards) that have been achieved at each performance level.
• Nebraska staff must submit evidence that it has approved the alternate achievement standards for all districts that are using alternate assessments aligned with alternate academic achievement standards and describe the process used for approval.
• Nebraska staff should provide evidence that local districts included stakeholders in the development of academic achievement standards.

3.0 - FULL ASSESSMENT SYSTEM

• Nebraska staff must provide information on the full array of the assessment options being implemented by its school districts, the extent to which districts employ one or more of these options, and whether all grades and content standards in reading and mathematics are assessed annually for each school district.
• Nebraska staff must provide evidence that all assessments used for accountability within each school district meet the necessary technical requirements, alignment with content standards, comparability of results, and equivalency in terms of content coverage, difficulty, and quality.
• Nebraska staff must provide evidence that local district assessments yield coherent information for students across grades and subjects within each school district.
• Nebraska staff must provide evidence that the assessments within a local district are comparable within the district.
• Nebraska staff must provide information on how they will address the various studies commissioned by the State that indicate higher order thinking skills and understanding of challenging content are not assessed in all districts.

4.0 - TECHNICAL QUALITY

• Nebraska staff must indicate how and when it will implement a review process for local district assessment systems that will meet professionally recognized standards and practices for assuring validity and reliability. Results of external studies clearly show that Nebraska's six Quality Criteria are not sufficient to determine validity, reliability, and bias of the assessments used in local districts.
• Nebraska staff must conduct a full review of district assessments, academic content and achievement standards to determine the validity and reliability of those assessments in line with professionally recognized standards and practices.
• Nebraska staff must indicate how it will address conclusions resulting from its various studies that identify assessment system limitations.
• Nebraska must provide information on how it will evaluate the quality and use of accommodations administered at the local district level.

5.0 - ALIGNMENT

• Nebraska staff must provide evidence to show that local district assessments are aligned with State or local standards. Insufficient detail was provided regarding how local districts through their District Portfolios met the alignment requirements. Criteria 1 of the six Quality Indicators was not sufficiently explained. The connection between Criteria 1 and professionally recognized standards and practice for standards and assessment alignment was not adequately addressed in the evidence submission.
• Nebraska staff should provide specific information on how the assessments reflect State or local standards in terms of a coherent approach, comprehensiveness and range, content and process, degree and pattern of emphasis, and range of academic achievement.
• Nebraska staff must provide information on how they will address the findings in the Year Three Report-Comprehensive Evaluation of Nebraska's School-Based, Teacher-led Assessment and Reporting that approximately one-fourth of the districts reviewed indicated that the assessments were comprised of items or tasks that do not reflect a match to the standard they were intended to measure.

6.0 - INCLUSION

• Nebraska staff must indicate the measures the State will take to ensure greater student participation in local district assessment administrations to meet the inclusion requirements of NCLB. The report provided indicated that participation rates are low in certain subgroups.
• Nebraska must provide more information and documentation about the alternate assessment for ELLs, including how it is equivalent to the district testing for other students. This assessment must also meet the requirements for NCLB if the results are to be used for accountability purposes.

7.0 - REPORTING

• Nebraska staff should verify that the reporting system facilitates appropriate, credible and defensible interpretation and use of its assessment data. Further, Nebraska should ensure the accuracy of all reports generated related to the assessment requirements of NCLB.
• Nebraska staff must report and disaggregate for all assessment results as required by NCLB.
• Nebraska staff must report assessment results in State report cards by the required academic achievement standards, not just "met" and "not met". Districts and schools must also meet NCLB assessment results reporting requirements. Further, student performance reports must indicate student's performance in reading/language arts and mathematics in relationship to the content competencies (related to the academic content standards) that have been achieved at each performance level.

Requirements for Alternate Assessments

For full approval of the State assessment system, the alternate assessments used by school districts in Nebraska's current alternate assessment must meet the following requirements:

• Yields results separately in reading and math.
  • Provides clear guidelines to all LEAs for student participation.
  • Designed and implemented in a manner that supports use of results for Adequate Yearly Progress (AYP).
• Aligned with State content standards.
• Assessment design is appropriate for school accountability measure (e.g., results are comparable across schools and districts: evidence includes test blueprint, test administration manual or administrator training materials, scoring rubric and scoring procedures).
• Local assessments must also meet the requirements in 200.3(c) of the NCLB standards and assessment regulation.
• Provide evidence of the technical quality of district alternate assessments.
  • Includes evidence of validity, reliability, accessibility, objectivity, and consistency with nationally recognized professional and technical standards.
  • Includes description of the standard-setting process employed; the judges that participated in standard setting and their qualifications; and State adoption of the resulting alternate achievement standards.
• Reports results to teachers and parents in a manner consistent with the alternate achievement standards.

Additional Evidence Needed for Alternate Assessment

For full approval, Nebraska will need to submit acceptable evidence that each school district has adopted alternate assessments that are in compliance with the following areas:

2.0 - ALTERNATE ACADEMIC ACHIEVEMENT STANDARDS

• Documentation of the standard-setting process employed.
• Documentation of performance descriptors that reflect specific competencies.
• Documentation of judges that participated in the standard setting and their broad -based qualifications.
• For AYP purposes, documentation that shows how reading and math are used for the alternate academic achievement standards.
• Clarification of the grade levels of the alternate assessments.

3.0 - FULL ASSESSMENT SYSTEM

• Documentation that the State's alternate assessment is designed and implemented in a manner that supports use of results for AYP.

4.0 - TECHNICAL QUALITY

• Evidence that procedures are in place that address data quality control.
• Documentation of the scoring rubric and scoring procedures.
• Evidence that the assessment design is appropriate for school accountability measures (e.g., results comparable across schools and districts).
• Evidence of consistency of the scoring decisions including how the alternate assessment scores are interpreted for student classifications by proficiency levels.
• Evidence of reliability of proficiency level determinations.
• Evidence that alternate assessment scores for reading and math are clear and separate and exclude non-academic IEP goals.

5.0 - ALIGNMENT

• Evidence of the review of the alignment of the content standards with the alternate assessment.

7.0 - REPORTING

• Supporting information to help parents understand the individual student reports.
• Individual student reports for the alternate assessments must have the following qualities:
  • Provide valid and reliable information for each student on the academic content and achievement standards expected of all students.
  • Display this information in a format that is understandable to parents, teachers, and principals.
  • Provide detailed information on student achievement to the level that maintains the validity and reliability of the assessment (subject, competency, skill, etc.).

Return to state-by-state listing