Highlights from NCEE (Archive)

Highlights from Effectiveness of Selected Supplemental Reading Comprehension Interventions: Impacts on a First Cohort of Fifth-Grade Students
There are increasing cognitive demands on student knowledge in middle elementary grades where students become primarily engaged in reading to learn, rather than learning to read. Children from disadvantaged backgrounds often lack general vocabulary, as well as vocabulary related to academic concepts, that enable them to comprehend what they are reading and acquire content knowledge. They also often do not know how to use strategies to organize and acquire knowledge from informational text in content areas such as science and social studies.

Although multiple techniques for direct instruction of comprehension in narrative text have been well demonstrated in small studies, that is not as much evidence on the effectiveness of teaching reading comprehension within content areas. Improving the ability of disadvantaged students to read and comprehend text is an important element of federal education policy aimed at closing the reading achievement gap. The Institute of Educations Sciences has undertaken a rigorous evaluation of curricula designed to improve reading comprehension.

The Effectiveness of Selected Supplemental Reading Comprehension Interventions: Impacts on a First Cohort of Fifth-Grade Students Students reports on the impacts of four reading comprehension curricula on fifth-grade reading comprehension. The curricula were selected to represent similar approaches to teaching reading comprehension strategies in the United States. Although the details of each curriculum differ, the four curricula share common strategies for improving reading comprehension, such as: Summarizing textual information into essential points; Generating Questions as student and teachers read together; use of Engaging Text; Cooperative Learning where students work together in groups; Teacher Modeling of comprehension strategies are used; and Direct Explanation of strategies.

The four curricula are Project CRISS (developed by Creating Independence Through Student-Owned Strategies); ReadAbout (developed by Scholastic Inc. and delivered primarily through a computer program); Read for Real (developed by Chapman university and Zaner-Bloser); and Reading for Knowledge (adapted by the Success for All Foundation for inclusion in this study). The effects of the curricula relative to a randomly-assigned control group are based on the Group Reading Assessment and Diagnostic Evaluation (GRADE) which is a general reading comprehension assessment, as well as tests of comprehension of science and social studies passages developed by the Educational Testing Service for this study.

This report contains the following key findings:

Curriculum Implementation

• During summer and early fall 2006, over 90 percent of treatment teachers were trained to use the curricula (91 percent of Read for Real teachers, 96 percent of Reading for Knowledge teachers, and 100 percent of Project CRISS and ReadAbout teachers).
• In the spring of 2007, over 80 percent of teachers reported using the curriculum they were assigned to use.
• Classroom observation data show that teachers implemented 55 to 78 percent of the strategies deemed important by the developers for implementing each curriculum. ReadAbout and Project CRISS teachers implemented, on average 71 and 78 percent of such strategies respectively. Reading for Knowledge teachers implemented 48 and 65 percent of the strategies deemed important for the two types of instructional days that are part of the curriculum. Read for Real teachers implemented 55 and 71 percent of the strategies deemed important for the two types of instructional days that are part of that curriculum.

Achievement Effects

Overall, average test scores in schools randomly assigned to use the four curricula were not statistically significantly higher than scores in the control schools. There were no positive effects on the GRADE or on the science or social studies reading comprehension assessments.

• Reading comprehension for each of three curricula (Project CRISS, ReadAbout, and Read for Real) was not significantly different from the control group.
• Students in schools using the Reading for Knowledge curriculum scored statistically significantly lower than control group students on the science reading comprehension assessment and on a composite score of all three assessments. Negative impacts were found for students whose teachers had more than 10 years of teaching experience, and for students in classrooms with a lower reading strategy guidance scale score based on classroom observations.

This study is the largest of its kind ever to use an experimental design to study a variety of reading comprehension curricula. This report is based on the reading comprehension achievement of fifth-graders in 10 districts and 89 schools during the 2006-07 school year. The 89 schools in this report are in ten districts that are geographically dispersed in eight states. The districts and schools that participated in the study were statistically significantly more disadvantaged, larger, and more urban than the U.S. average. However, this is not a representative sample of districts and schools in the U.S., because participating sites are likely to be unique in ways that make it difficult to select a representative sample. Eligible districts were willing to use all four of the study’s curricula and allowed the curricula to be randomly assigned to their participating schools.

A second report on this study for the 2007-08 school year will be based on the reading comprehension achievement of a second cohort of fifth-graders using three of the curricula. The evaluation did not include Reading for Knowledge in the second year. The reading comprehension of fifth-grade students participating in the first year of the evaluation will be also be assessed at the end of the sixth grade. This report will also include information from classroom observations of fidelity to each curricula as well as classroom practices across the curricula.

Browse to http://ies.ed.gov/ncee/pubs/20094032/