- Executive Summary:
PDF (76 KB)
- Final Report: PDF (5.5 MB)
PDF (76 KB)
PDF (5.5 MB)
The No Child Left Behind Act of 2001 (PL 107-110) established the Reading First Program (Title I, Part B, Subpart 1), a major federal initiative designed to help ensure that all children can read at or above grade level by the end of third grade. Reading First (RF) is predicated on scientifically researched findings that high-quality reading instruction in the primary grades significantly reduces the number of students who experience reading difficulties in later years.
The Reading First program's overarching goal is to improve the quality of reading instruction— and thereby improve the reading skills and achievement of children in the primary grades—by providing substantial resources at both the state and local levels. The intent is to ensure that teachers in kindergarten through third grade use reading programs and materials that are research-based. Additionally, Reading First intends to increase access to and the quality of professional development for all teachers of these grades, including special education teachers, to ensure that they have the necessary skills to teach these researched-based reading programs effectively. An important provision of the RF legislation is that professional development be made available to all schools, not only schools that received RF funding. A third emphasis is on using assessments, both to monitor progress and to identify students' reading problems early on. Reading First is intended to help prepare classroom teachers to screen for, identify, and overcome barriers to students' ability to read at grade level by the end of third grade. More specifically, the programs and the professional development provided to school staff must use reading instructional methods and materials that incorporate the five essential elements of effective primary-grade reading instruction, as specified in the legislation: 1) phonemic awareness; 2) decoding; 3) vocabulary development; 4) reading fluency, including oral reading skills; and 5) reading comprehension strategies.
All 50 states and other jurisdictions1 have been awarded Reading First grants. To date (April 2006), states have awarded subgrants to approximately 1,550 local school districts and, in turn, these districts have provided funds to approximately 5,200 schools nationwide. Because grants to states were awarded over an extended time period and states differed in the amount of time they allotted to their competitive subgrant processes, districts and schools are at various stages of implementing their Reading First programs.
The enabling legislation for RF requires the U.S. Department of Education (ED) to contract with an outside entity to evaluate the program's implementation (Section 1205). To meet this requirement, the Department contracted with Abt Associates in October 2003 to design and conduct the Reading First Implementation Evaluation, which addresses the following questions:
The five-year study has produced this interim report based on data collected during the 2004–05 school year as well as analyses of extant data sources; it will also produce a final report in 2007 based on data from the 2006–07 school year and updated extant data. This interim report addresses questions 1 and 2. Question 3 requires an analysis of longitudinal data and will be addressed upon completion of the second wave of data collection in 2007, as will questions 4 and 5. Below we summarize key findings from the evaluation, using the following data sources:
The non-RF Title I school sample was constructed purposefully to provide a context for understanding how reading programs in a sample of Reading First schools differ from those in schools serving similar populations of students.2 The non-RF sample includes only Title I schoolwide project (SWP) schools with at least 40 percent of the students eligible for free or reduced price lunches, which is comparable to the RF school population. The two groups of schools are demographically similar in staff experience, attendance rates, mobility, and stability of enrollment. RF schools are, however, on average, larger than the Title I schools, and have larger proportions of K–3 students reading below grade level.
We can make comparisons between RF and non-RF Title I samples, but because the two samples are not matched they cannot be assumed to be equivalent. Thus, the differences between the groups discussed in this report cannot be attributed to the Reading First program.
Key FindingReading First schools appear to be implementing the major elements of the program as intended by the legislation, such as providing scientifically based reading instruction in grades K–3, increased amounts of time for reading instruction, interventions for struggling readers, wider use of classroom-based reading assessments, and more professional development activities. |
Key FindingsReading First schools received both financial and nonfinancial support from a variety of external sources. During the 2004–05 school year, the median annual amount of funds RF schools received to implement their reading program was $138,000. In addition, Reading First schools have multiple external resources, in addition to RF funds, to support the |
Most RF schools also received Title I funds (91 percent) and district funds (79 percent) to support their reading programs. Exhibit E-1 illustrates that beyond financial support, according to principals, RF schools received substantially more external assistance than did Title I schools on selecting instructional programs (76 percent vs. 56 percent), diagnosing needs of struggling readers (70 percent vs. 50 percent), conducting demonstration lessons (71 percent vs. 48 percent), and reviewing the effectiveness of reading programs (71 percent vs. 47 percent).
Key FindingClassroom reading instruction in RF schools is significantly more likely to adhere to the RF legislation than that in Title I schools. Reading instruction encompasses the amount of instructional time, use of appropriate reading materials, and implementing reading activities and strategies supported by scientifically based reading research. |
The Guidance for the Reading First Program states that schools "should consider the allocation of time, including a protected, uninterrupted block of time for reading instruction of more than 90 minutes per day." Significantly more RF schools than Title I schools reported having a reading block for each of grades K through 3 (grades 1–3; 98 percent vs. 92 percent, kindergarten; 98 percent vs. 88 percent, Exhibit E-2). Teachers in Reading First schools reported, on average, that they spent significantly more time on reading than did teachers in non-RF Title I schools—a difference of about 19 minutes per day, or almost 100 minutes per week. Teachers in newly funded RF schools were also significantly more likely than teachers in non-RF Title I schools to report that they had increased the amount of time spent on reading from the 2003–04 to the 2004–05 school years (61 percent vs. 35 percent).3
Staff in RF schools also reported (using a five-point scale4) having significantly more time set aside than staff in non-RF Title I schools to use assessment data to plan instruction (3.88 vs. 3.38); to observe reading instruction in other classrooms (2.21 vs. 1.96); to collaborate on reading lesson planning and instruction (4.23 vs. 3.88); and to receive instruction themselves from a reading coach (once a month on average for RF schools, vs. four or fewer times per year in non-RF Title I schools).
Reading First schools reported that they have made substantial changes to their reading programs since they received their RF funds in the 2004–05 school year (Exhibit E-3). Newly funded RF schools were significantly more likely than Title I schools to have adopted a new core reading program (39 percent vs. 16 percent), to have added new intervention programs for struggling readers (74 percent vs. 43 percent), to have added new supplementary materials (69 percent vs. 58 percent), and to have adopted new materials for English Language Learners (43 percent vs. 29 percent).
Staff in mature RF schools were significantly more likely than staff in non-RF Title I schools to rate (on a five-point scale) the following statements about their schools' reading programs more positively:
Despite evidence of greater alignment with SBRR, staff in mature RF schools recognized their lack of experience working with some materials. They were significantly more likely than staff in non-RF Title I schools to rate (also on a five-point scale) the following statements more negatively: 1) Teachers are experienced with supplemental reading materials (3.29 vs. 3.58); and 2) Teachers are experienced with reading intervention materials (3.19 vs. 3.39). This may reflect the fact that many of these materials are new in RF schools.
Reading instructional activities and strategies in RF schools appear to be aligned with the tenets of the Reading First program. Differences in instructional environments between mature RF and non-RF Title I schools were manifested in a variety of ways.5 RF teachers across all grades (K– 3) rated as central a greater proportion of SBRR-aligned practices than did teachers in Title I schools: These statistically significant differences were modest in size (e.g., second grade: 76 percent vs. 72 percent), although the difference for kindergarten teachers was somewhat larger (77 percent vs. 68 percent).
When asked specifically about the centrality of instruction in the five dimensions of reading, similar patterns emerge. There are differences that are modest in magnitude. Kindergarten and first-grade teachers rated as central to their instruction a significantly higher proportion of scientifically based practices in teaching phonemic awareness and decoding than did teachers in Title I schools (kindergarten: 91 percent vs. 86 percent; first grade: 85 percent vs. 81 percent). RF third-grade teachers rated as central to their instruction a significantly higher proportion of scientifically based practices in teaching vocabulary and fluency than did their Title I counterparts (vocabulary, 75 percent vs. 70 percent; fluency. 56 percent vs. 47 percent). There were no differences between RF and Title I teachers in their centrality ratings related to teaching comprehension skills.
Key FindingsRF teachers in three grades (kindergarten, second, and third) were significantly more likely than their counterparts in Title I schools to place their struggling students in intervention programs. Based on principal reports, in both RF and Title I schools, there was no time delay between identifying students who need interventions and the provision of services to those students. RF and Title I schools were also similar with respect to planning and coordinating instruction for ELL students. Although significantly more RF teachers reported receiving professional development in helping struggling readers than did Title I teachers, teachers in both RF and Title I schools recognized the challenge of providing effective instruction to struggling readers; 80 percent of teachers in both groups reported that they need additional professional development on this topic. |
Exhibit E-4 indicates that RF schools were more likely to rely on progress monitoring (98 percent vs. 90 percent) and reading coach recommendations to identify struggling readers (92 percent and 55 percent) than were Title I schools. Although there were no differences in principal reports of the use of diagnostic tests, RF teachers across all grades were significantly more likely to rely on diagnostic assessments to determine their struggling readers' core deficits than were teachers in Title I schools (74 percent vs. 64 percent).
Despite these differences in the identification of struggling readers, there were no substantive or statistically significant differences between RF and Title I schools in terms of availability of intervention services for struggling readers; about 80 percent of both RF and Title I principals reported that reading intervention services are available when needed. Nor did the average waiting time for students' receipt of services differ significantly between RF and Title I schools; two-thirds of the principals reported no wait time for students in need of intervention. On average, identified students received services within approximately one week.
RF and Title I schools are more similar than different with respect to coordinating instruction for ELL students. There were no significant differences between RF and Title I teachers in reported time set aside to coordinate instruction with ELL staff; in fact, more than one-third of teachers who have ELL students in their classrooms (in both groups of schools) reported that no such time is specifically set aside. Only about 10 percent of teachers reported that they had weekly meetings with ELL staff to coordinate reading instruction for their struggling ELL students. It is important to note here, however, that some schools may not have any ELL teachers on staff.
RF and Title I schools do differ in the provision of special education services to struggling readers. Title I schools were significantly more likely than RF schools to have a certified special education teacher provide recommendations to plan instruction for struggling readers (83 percent vs. 72 percent). RF teachers in kindergarten, first, and second grades were significantly more likely than Title I teachers to report that time is not set aside for coordination of the reading instruction provided to their special education students (kindergarten: 51 percent vs. 44 percent; first grade: 42 percent vs. 35 percent; second grade: 37 percent vs. 30 percent).
RF teachers were more likely to report increased amounts of time for struggling readers to practice skills in several dimensions of reading. RF teachers in three of four grades reported providing practice in significantly more reading dimensions than did teachers in Title I schools (kindergarten: 2.75 vs. 2.66; second grade: 2.83 vs. 2.78; third grade: 2.71 vs. 2.45). These differences, while statistically significant, are substantively quite small. RF third-grade teachers were significantly more likely than Title I teachers to provide struggling readers with extra practice in decoding (92 percent vs. 82 percent) and fluency (97 percent vs. 90 percent); there were no significant differences for the other grades.
In terms of materials, RF teachers were also more likely to draw upon resources external to the core reading program to support struggling readers. Kindergarten and third-grade teachers in RF schools were significantly more likely than their counterparts in Title I schools to use materials that supplement the core reading program (kindergarten teachers: 70 percent vs. 62 percent; third-grade teachers 74 percent vs. 66 percent). There were no significant differences in first or second grade. RF teachers in first and second grade were significantly more likely than teachers in Title I schools to provide in-class help in reading for ELL students (66 percent vs. 55 percent). Kindergarten and third grade were not significantly different.
Key FindingsAssessment plays an important role in reading programs in both RF and non-RF Title I schools. Reading First schools received more outside assistance in selecting assessments than Title I schools. At the same time, staff in both RF and Title I reported that the district is largely responsible for the selection and interpretation of reading assessments. There were some differences in the types of assessments teachers in Reading First schools and teachers in non-RF Title I schools found useful. RF teachers were more likely to identify assessments from their core or supplementary reading programs as useful than are teachers in Title I schools. In contrast, Title I teachers were more likely to report that informal assessments are useful than are RF teachers. Teachers in RF schools were more likely to report applying assessment results for varied instructional purposes (e.g., for planning grouping, progress monitoring and identifying struggling readers) than their Title I counterparts. |
Reading First schools received significantly more outside assistance (from district, state, publisher, university expert, etc.) than did non-RF Title I schools in selecting assessment instruments for their K–3 reading program (76 percent vs. 56 percent) and interpreting assessment results (82 percent vs.70 percent).
Overall, there were many similarities in RF and non-RF Title I principals' reports of who is responsible for the selection and interpretation of reading assessments. More than three-quarters of both RF and non-RF Title I principals identified the district as responsible for selecting assessments. There were, however, some differences. Half of RF principals (51 percent) also reported state responsibility for selecting assessments, compared to 31 percent of non-RF Title I principals. In 54 percent of non-RF Title I schools, the principal also held this responsibility, compared to only 40 percent of RF principals.
Nearly all, about 90 percent, Reading First and non-RF Title I principals reported that they, as principals, were responsible for interpreting assessment results. In 93 percent of RF schools, the reading coach also assumed responsibility for this task, compared to only half of Title I principals (52 percent). This significant difference reflects the fact that, in contrast to Reading First schools, Title I schools were much less likely to have a designated reading coach.
Eighty-four percent of RF teachers reported that they had regularly scheduled, formal time set aside to use assessment data to plan instruction, compared with 74 percent of non-RF Title I teachers.
Most teachers in both RF and Title I schools named at least one assessment that they found useful in placing or grouping students (90 percent), determining student mastery of skills (89 percent), and identifying the core deficits of struggling students (85 percent).
In general, RF teachers were significantly more likely to identify formal assessments as useful than Title I teachers. In contrast, Title I teachers were more likely to report the usefulness of informal assessments than were RF teachers. Across assessment purposes, Reading First teachers were more likely than their Title I counterparts to identify assessments from the core or supplementary reading program (e.g., for determining student mastery of skills, 50 percent vs. 38 percent) or standardized tests (e.g., for placing or grouping students, 50 percent vs. 44 percent), such as the DIBELS (e.g., for identifying the core deficits of struggling students, 33 percent vs. 10 percent).
In contrast, across the three uses of assessments, Title I teachers were significantly more likely to identify informal assessments as useful than were Reading First teachers (e.g., for determining student mastery of skills, 38 percent vs. 28 percent). Specifically, Title I teachers were more likely to report classroom-based assessments (e.g. for identifying the core deficits of struggling students, 24 percent vs. 17 percent) and running records or miscue analysis (e.g., for placing or grouping students, 19 percent vs. 10 percent) as useful than Reading First teachers.
Most K–3 teachers in both Reading First and Title I schools named at least one assessment that they found useful in placing or grouping students (90 percent), determining student mastery of skills (89 percent), and identifying the core deficits of struggling students (85 percent). However, a significantly greater percentage of RF teachers reported that they use assessment results to organize instructional groups (83 percent vs. 73 percent), to determine progress on skills (85 percent vs. 78 percent), and to identify students who need reading intervention services (75 percent vs. 65 percent) than did Title I teachers.
Key FindingsPrincipals in Reading First schools were significantly more likely to report having a reading coach than were principals of non-RF Title I schools.6 Coaches in RF schools were significantly more likely to provide teachers with various supports for their reading instruction than were coaches in non-RF Title I schools. |
Reading First schools were significantly more likely to have a reading coach (98 percent vs. 60 percent) than were non-RF Title I schools, as reported by the principal, reflecting the fact that nearly all states required RF schools (but not non-RF Title I schools) to have a reading coach. Of the Title I schools that reported having reading coaches, 88 percent have a coach that is doing the central activities of a reading coach.7 In Reading First schools, reading coaches are intended to work primarily with teachers rather than directly with students. It is therefore noteworthy that a significantly smaller percentage of reading coaches in RF schools reported that providing direct reading instruction to students is absolutely central to their work: 29 percent, compared to 53 percent in non-RF Title I schools (Exhibit E-5).
Reading coaches in mature Reading First schools were significantly more likely than their non-RF Title I counterparts to rate the following teacher support activities as absolutely central to their work: providing training or professional development (95 percent vs. 87 percent), coaching staff on a range of topics (92 percent vs. 83 percent), organizing professional development (87 percent vs. 67 percent), and facilitating grade-level meetings (67 percent vs. 47 percent).
Reading coaches in mature Reading First schools were significantly more likely than their non-RF Title I counterparts to report that the following administrative support activities were absolutely central to their work: compiling reading assessment data (88 percent vs. 67 percent) and administering or coordinating reading assessments (86 percent vs. 67 percent). About 75 percent of both mature RF and non-RF Title I schools reported participating in school leadership team meetings, and about 70 percent of both types of schools reported managing reading instructional materials, as absolutely central to their work (with no significant differences). Significantly more reading coaches in mature RF schools, compared with coaches in non-RF Title I schools, characterized participating in professional development (97 percent vs. 86 percent) as absolutely central to their work.
Key FindingsRF staff received significantly more professional development than did Title I staff. RF teachers were more likely to have received professional development in the five dimensions of reading instruction as well as in overall teaching strategies. Indeed, RF teachers reported feeling better prepared to teach the five dimensions of reading than their Title I counterparts. Activities attended by RF teachers were more likely to have structural attributes conducive to a successful experience, such as incentives and follow-up activities, than those attended by teachers in non-RF Title I teachers (Corcoran, 1995; Corcoran, et al., 2003; Garet, et al., 1999; Learning First Alliance, 2003). |
Significantly more Reading First teachers attended professional development workshops related to reading than did non-RF Title I teachers (94 percent vs. 81 percent). Further, RF teachers, on average, reported having spent significantly more time attending professional development activities—conferences, workshops, college courses—in the past year than did teachers in non-RF Title I schools (40 hours vs. 24 hours).
Based on teacher reports, professional development activities attended by RF teachers, as compared with non-RF Title I teachers, were significantly more likely to:
RF principals, reading coaches and teachers reported having participated in significantly more professional development activities to improve their knowledge of the five dimensions of reading instruction than did Title I teachers (Exhibit E-6). For example, significantly more RF than Title I teachers received professional development in phonemic awareness (85 percent vs. 62 percent). RF teachers rated themselves (on a five-point scale) as significantly better prepared to teach the following skills than did Title I teachers: phonemic awareness, 4.13 vs. 3.66; decoding, 3.86 vs. 3.35; vocabulary, 3.79 vs. 3.40; comprehension, 3.80 vs. 3.54; and fluency, 3.83 vs. 3.40.
RF teachers were significantly more likely than teachers in Title I schools to report having received professional development assistance on administering and using assessments (85 percent vs. 67 percent), interpreting assessment data (90 percent vs. 68 percent), and using diagnostic tests to guide instruction (66 percent vs. 49 percent).
Reading coaches and principals also received professional development to support them in their roles. RF reading coaches were significantly more likely than their non-RF Title I counterparts to report having received professional development assistance on how to help teachers make reading instruction systematic and explicit (78 percent vs. 52 percent), and on the essential components of SBRR instruction (90 percent vs. 60 percent). Both of these topics are central to the Reading First program. Similarly, principals in RF schools were significantly more likely to report having received professional development in all five dimensions of reading than were principals in Title I schools (p < .05 for all five dimensions).
These findings provide some preliminary evidence to suggest that Reading First is being implemented in schools and classrooms as intended by the legislation. For the most part, funds are awarded to appropriate districts and schools. States are providing appropriate supports, particularly in terms of professional development related to reading, and in the selection and use of assessments to inform instruction. Reading First schools appear to have established instructional environments to support SBRR-based reading instruction. In K–3 classrooms, the reading programs implemented by teachers in Reading First and non-RF Title I schools appear to be different in a variety of ways, including instructional time, resources, instructional planning and collaboration, use of assessments, and focus on the five dimensions of reading instruction. Taken together, these findings provide some initial evidence to suggest that Reading First schools are carrying out the objectives of the Reading First legislation. Future analyses, after the second round of survey data collection in 2007, will examine how implementation of these elements changes over time and how student achievement patterns in RF schools may differ from those in non-RF Title I schools.
1. State Education Agencies (SEAs) were eligible to apply for RF grants. Other jurisdictions eligible include District of Columbia, the schools of the Bureau of Indian Affairs, and American Samoa. Guam and Northern Mariana Islands received grants through the consolidated grants to insular areas.
2. The most rigorous design option available for this evaluation would have been to identify a group of non-RF schools matched to RF schools on key demographic and achievement characteristics to minimize differences between RF and non-RF schools and thereby approximate a random assignment experiment. However, because RF schools, by definition, are among the lowest performing schools in their respective districts, matched comparison schools could include better performing schools. Also, RF schools could likely differ from similar non-funded schools because, often, they had to demonstrate motivation, and this factor could influence any observed instructional differences between RF and comparison schools.
3. For this comparison, we used the newly funded RF schools since the mature RF schools may have already increased their instructional time in their first year of implementation, 2002–03. That said, about 45 percent of the mature RF schools reported increasing their instructional time in the 2004–05 school year.
4. The scale represents how often schools reported time being set aside during the school year: 1 = Not at all, 2 = 1–4 times, 3 = 5–8 times, 4 = Once a month, and 5 = Once a week or more.
5. We constructed six composites to summarize teachers' ratings of the centrality of a series of instructional activities associated with the following reading dimensions and other instructional features: 1) phonemic awareness and decoding; 2) vocabulary; 3) comprehension; 4) fluency; 5) use of scientifically based instructional strategies and materials; and 6) negative alignment with scientifically based reading research. (See Appendix D for a list the specific items included in each composite.) Scores were computed for each composite based on the percentage of instructional activities specified in that composite that a teacher rated as "central to their instruction."
6. Most states require RF schools to employ reading coaches to support teachers' reading instruction; this is not the case for non-RF Title I schools.
7. Because respondents from non-RF Title I schools who completed the Reading Coach Survey reported a multiplicity of job titles, we used their responses to two survey questions to determine their inclusion in the comparison group of reading coaches: How central is each of the following activities? 1) "Coaches staff on a range of topics"; and 2) "Organizes professional development for K–3 teachers." Respondents who answered a 3 ("somewhat central") or above (on a 5-point scale) for at least one of these two items were included in the comparison group of reading coaches from Title I schools for these sets of analyses regarding reading coach responsibilities. As a result, 34 reading coach respondents from Title schools, the equivalent of 940 weighted respondents, were excluded from these analyses.
Last Modified: 07/24/2006
|