Interpreting NAEP Science ResultsOverview of the Assessment Overview of the AssessmentAs the only federally-funded ongoing assessment of student science achievement on a national scale, the NAEP science assessment reflects the content framework and expert perspectives about science and its measurement. Read more about what the assessment measures, how it was developed, who took the assessment, and how the assessment was administered. This page describes elements of the main NAEP science assessment, and does not apply to the long-term trend science assessment (which was discontinued in 1999). Read more about the difference between the main and long-term trend NAEP assessments. The 2005 science results presented on the website are based on representative samples of students for the nation and for participating states and other jurisdictions. Approximately 305,000 students from approximately 15,800 schools participated in the 2005 NAEP science assessment at grades 4, 8, and 12. The national results reflect the performance of students attending both public and nonpublic schools, while the state and jurisdiction results reflect only the performance of students attending public schools. Beginning in 2002, the NAEP national sample was obtained by aggregating the samples of public school students from each state and jurisdiction, and then supplementing the aggregate sample with a nationally representative sample of students from nonpublic schools, rather than by obtaining an independently selected national sample. As a consequence the national sample size increased, and smaller differences between years or between groups of students were found to be statistically significant than would have been detected in previous assessments. In keeping with past practice, all statistically significant differences are indicated in the current web results pages. Comparisons are made to results from previous years in which the assessment was administered. In addition to the 2005 results, state and national results are reported from the 1996 and 2000 assessments. The more recent national results (those from 1996 or later) are based on administration procedures in which testing accommodations were permitted for students with disabilities and for English language learners. Read more about NAEP's policy of inclusion. Comparisons between results from 2005 and those from assessment years in which both types of administration procedures were used (1996 and 2000) are discussed based on the results when accommodations were permitted, even though significant differences in results when accommodations were not permitted may be noted in figures and tables. Changes in student performance across years or differences between groups of students in 2005 are discussed only if they have been determined to be statistically significant. Reporting the Assessment—Scale Scores and Achievement LevelsThe results of student performance on the NAEP science assessment are presented on this website in two ways: as average scores on the NAEP science scale and as the percentages of students attaining NAEP science achievement levels. The average scale scores represent how students performed on the assessment. The achievement levels represent how that performance measured up against set expectations for achievement. Thus, the average scale scores represent what students know and can do, while the achievement-level results indicate the degree to which student performance meets expectations of what they should know and be able to do. The national results for 1996, 2000, and 2005 are summarized on the NAEP composite science scales, which are composed of subscales for each of the three fields of science (i.e., Earth, physical, and life). The subscales summarize student performance across all three types of questions in the assessment (multiple-choice, short constructed-response, and extended constructed-response). The composite scales were developed at each of grades 4, 8, and 12 by weighting the separate subscales based on the relative importance of each content area in the NAEP science framework. The composite scale at each grade ranges from 0 to 300. While the scale score ranges are identical, the scale was derived independently at each grade. Also, scales were weighted differently at different grades in determining the overall scale. Therefore, average scale scores across grades cannot be compared. For example, equal scale scores on the grade 4 and grade 8 scales do not imply equal levels of science achievement. Achievement-level results are presented in terms of science achievement levels as adopted by the National Assessment Governing Board, and are intended to measure how well students' actual achievement matches the achievement desired of them. For each grade tested, the Governing Board has adopted three achievement levels: Basic, Proficient, and Advanced. For reporting purposes, the achievement-level cut scores are placed on the science scales, resulting in four ranges: below Basic, Basic, Proficient, and Advanced. The Board established its achievement levels in 1996 based upon the science content framework and standard-setting process involving a cross section of educators and interested citizens from across the nation who were asked to judge what students should know and be able to do relative to the content reflected in the NAEP science framework. As provided by law, the Acting Commissioner of Education Statistics has determined that the achievement levels are to be considered developmental and should be interpreted and used with caution. However, both NCES and the Governing Board believe these performance standards are useful for understanding trends in student achievement. Description of Science Performance by Item Maps for Each GradeThe performance of fourth-, eighth-, and twelfth-graders can be illustrated by maps that position question descriptions along the NAEP science scale. The descriptions used on these maps focus on the science skill or knowledge needed to answer the question. For multiple-choice questions, the description indicates the skill or knowledge demonstrated by selection of the correct option; for constructed-response questions, the description takes into account the skill or knowledge specified by the different levels of scoring criteria for that question. Approximately 25 science questions per grade have been selected and placed on an item map for grade 4, grade 8, and grade 12. Results are EstimatesThe average scores and percentages presented on this website are estimates because they are based on representative samples of students rather than on the entire population of students. Moreover, the collection of subject-area questions used at each grade level is but a sample of the many questions that could have been asked. As such, NAEP results are subject to a measure of uncertainty, reflected in the standard error of the estimates. The standard errors for the estimated scale scores and percentages in the figures and tables presented on this website are available through the NAEP Data Explorer. NAEP Reporting GroupsResults are provided for groups of students defined by shared characteristics—gender, race or ethnicity, eligibility for free/reduced-price school lunch, student-reported highest level of parental education, students with disabilities, and students identified as English language learners. Based on participation rate criteria, results are reported for subpopulations only when sufficient numbers of students and adequate school representation are present. The minimum requirement is at least 62 students in a particular group from at least five primary sampling units (PSUs). However, the data for all students, regardless of whether their group was reported separately, were included in computing overall results. Explanations of the reporting groups are presented below. GenderResults are reported separately for males and females. Race/EthnicityIn all NAEP assessments, data about student race/ethnicity is collected from two sources: school records and student self-reports. Before 2002, NAEP used students' self-reports of their race and ethnicity on a background questionnaire as the source of race/ethnicity data. In 2002, it was decided to change the student race/ethnicity variable highlighted in NAEP reports. Starting in 2002, NAEP reports of students' race and ethnicity are based on the school records, with students' self-reports used only if school data are missing. Information based on student self-reported race/ethnicity will continue to be reported in the NAEP Data Explorer. In order to allow comparisons across years, assessment results presented are based on school-reported information for six mutually exclusive racial/ethnic categories: White, Black, Hispanic, Asian/Pacific Islander, American Indian (including Alaska Native), and Other. Students who identified with more than one of the first five categories or had a background other than the ones listed were categorized as Other. Eligibility for Free/Reduced-Price School LunchAs part of the Department of Agriculture's National School Lunch Program, schools can receive cash subsidies and donated commodities in turn for offering free or reduced-price lunches to eligible children. Based on available school records, students were classified as either currently eligible for the free/reduced-price school lunch or not eligible. Eligibility for free and reduced-price lunches is determined by students' family income in relation to the federally established poverty level. Students whose family income is at or below 130 percent of the poverty level qualify to receive free lunch, and students whose family income is between 130 percent and 185 percent of the poverty level qualify to receive reduced-price lunch. The classification applies only to the school year when the assessment was administered (i.e., the 2004–05 school year) and is not based on eligibility in previous years. If school records were not available, the student was classified as "Information not available." If the school did not participate in the program, all students in that school were classified as "Information not available." Parental EducationParents' highest level of education is defined by the highest level reported by eighth-graders and twelfth-graders for either parent. Fourth-graders' replies to this question were not reported because their responses in previous studies were highly variable, and a large percentage of them chose the "I don't know" option. Student with Disabilities (SD)Results are reported for students who were identified by school records as having a disability. A student with a disability may need specially designed instruction to meet his or her learning goals. A student with a disability will usually have an Individualized Education Program (IEP), which guides his or her special education instruction. Students with disabilities are often referred to as special education students and may be classified by their school as learning disabled (LD) or emotionally disturbed (ED). English Language Learners (ELL)Results are reported for students who were identified by school records as being English language learners. (Note that English language learners were previously referred to as limited English proficient (LEP). RegionPrior to 2003, NAEP results were reported for four NAEP-defined regions of the nation: Northeast, Southeast, Central, and West. As of 2003, to align NAEP with other federal data collections, NAEP analysis and reports have used the U.S. Census Bureau's definition of "region." The four regions defined by the U.S. Census Bureau are Northeast, South, Midwest, and West. The Central region used by NAEP before 2003 contained the same states as the Midwest region defined by the U.S. Census. The former Southeast region consisted of the states in the Census-defined South minus Delaware, the District of Columbia, Maryland, Oklahoma, Texas, and the section of Virginia in the District of Columbia metropolitan area. The former West region consisted of Oklahoma, Texas, and the states in the Census-defined West. The former Northeast region consisted of the states in the Census-defined Northeast plus Delaware, the District of Columbia, Maryland, and the section of Virginia in the District of Columbia metropolitan area. Therefore trend data by region are not provided for the 2005 science assessment. The table below shows how states are subdivided into these Census regions. All 50 states and the District of Columbia are listed. Other jurisdictions, including the Department of Defense Educational Activity schools, are not assigned to any region.
Exclusion RatesForty-four states and Department of Defense schools participated in the 2005 science assessment. To ensure that the samples in each state are representative, NAEP has established policies and procedures to maximize the inclusion of all students in the assessment. Every effort is made to ensure that all selected students who are capable of participating meaningfully in the assessment are assessed. While some students with disabilities (SD) and/or English language learners (ELL) students can be assessed without any special procedures, others require accommodations to participate in NAEP. Still other SD and/or ELL students selected by NAEP may not be able to participate. Local school authorities determine whether SD/ELL students require accommodations or shall be excluded because they cannot be assessed. The percentage of SD and/or ELL students who are excluded from NAEP assessments varies from one jurisdiction to another and within a jurisdiction over time. Read more about the potential effects of exclusion rates on assessment results. See additional information about the percentages of students with disabilities and English language learners See the types of accommodations permitted for students with disabilities and/or English language learners at the national and state levels. Exclusion rates for other subjects, as well as rates of use of specific accommodations, are available. Statistical SignificanceThe differences between scale scores and between percentages discussed in the results on this website take into account the standard errors associated with the estimates. Comparisons are based on statistical tests that consider both the magnitude of the difference between the group average scores or percentages and the standard errors of those statistics. Throughout the results, differences between scores or between percentages are discussed only when they are significant from a statistical perspective. All differences reported are significant at the 0.05 level with appropriate adjustments for multiple comparisons. The term "significant" is not intended to imply a judgment about the absolute magnitude of the educational relevance of the differences. It is intended to identify statistically dependable population differences to help inform dialogue among policy makers, educators, and the public. Cautions in InterpretationsUsers of this website are cautioned against interpreting NAEP results in a causal sense. Inferences related to subgroup performance or to the effectiveness of public and nonpublic schools, for example, should take into consideration the many socioeconomic and educational factors that may also impact performance.
|
||||||||||||
