Elementary School Math Professional Development Impact Evaluation

Contractors: American Institutes for Research; Harvard University; Measured Decisions, Inc.; Clowder Consulting

Background/Research Questions:

Title II, Part A of the current Elementary and Secondary Education Act provides monetary resources to improve teacher quality. One way to affect teacher quality, and thus teacher effectiveness, is by providing in-service teacher training, also called professional development (PD). Currently there is relatively little rigorous empirical evidence on the components of an effective PD program (i.e., the importance of emphasizing content knowledge and how that knowledge might be delivered to teachers and then infused into the classroom). At the same time, there has been renewed policy interest in fostering a globally-competitive STEM workforce, which will require teachers to effectively impart mathematics to their students. Of particular concern are upper elementary school teachers (e.g., grades 4 and 5), who are typically less likely to have strong mathematical training and yet are asked to teach math topics that are foundational for more advanced courses in middle school and beyond.

IES conducted a random assignment evaluation of an intensive math PD intervention focused on enhancing teacher content knowledge and integrating this knowledge into the classroom. The intervention had three components: Intel Math, Mathematics Learning Communities (MLC), and Video Feedback Cycles (VFC). Intel Math, the core of the intervention, is a widely used, 80-hour professional development course focused on developing teachers' understanding of K–8 mathematics. The MLC was aligned to the Intel Math program and offered teachers opportunities to connect their learning to the classroom through a professional learning community that focused on student thinking in a series of 5 two-hour sessions. The VFC provided further support for connecting their learning to the classroom by allowing teachers to practice what they learned three different times using actual lessons that were videotaped and reviewed by analysts with expertise in assessing the mathematical quality of instruction. For each round of videos, analysts shared the results with a district-based math coach (who also led the MLCs), and the coach provided 1-on-1 feedback to the teacher, linking the classroom practice to the relevant MLC and Intel Math content. This study focused on two primary questions:

• What is the average impact on (a) teachers' content knowledge, (b) teachers' classroom practices, and (c) student achievement of offering a specialized professional development intervention (in this case, Intel Math, Mathematics Learning Communities, and Video Feedback Cycles) relative to "business-as-usual" professional development?
• How is the professional development intervention implemented?

Design:

Approximately 200 volunteer fourth-grade math teachers from 6 districts around the country participated in this study. Within each participating school, fourth-grade teachers were randomly assigned to receive either the PD intervention or business-as-usual PD. The Intel Math component consisted of 80 instructional hours during the summer, covering K-8 math topics aligned with the common core state standards. About 90 percent of the course was focused on content, while 10 percent was focused on pedagogy. The MLC component consisted of 10 instructional hours split into 5 interactive sessions throughout the fall, and involved activities such as analyzing student work to help teachers integrate content learned from Intel Math into the classroom. The VFC component consisted of 3 instructional hours split into 3 1-on-1 sessions throughout the fall, and involved practicing the delivery of three different lessons and receiving content-based feedback on the quality of instruction using expert coders and math coaches to help integrate content learned from Intel Math and MLCs into the classroom. Tests of math content knowledge were administered to all study teachers in the summer (as baseline), fall (after Intel but prior to MLCs and VFCs), and spring (after the entire intervention has been delivered). Extant state math assessment data for these teachers' students were collected from districts for the pre and post study periods. In addition, study teachers were observed using the Mathematical Quality of Instruction (MQI) classroom observation protocol once in the fall (after Intel but prior to MLCs and VFCs) and twice in the spring (after the entire intervention has been delivered). These data collection activities helped inform the study's first research question. Additional teacher survey data and data collected by the implementation team helped inform the second question.

Cost/Duration: $8,157,019 over 4.5 years (September 2012–March 2017)

Current Status: This study has been completed. A report and brief have been released (see below for key findings).

Key Findings: A report entitled Focusing on Mathematical Knowledge: The Impact of Content-Intensive Teacher Professional Development was released on September 28, 2016. This report presents all findings from this study, including the following key findings:

• The PD had a positive impact on teacher knowledge: Average scores on a study-administered math test were 21 percentile points higher for teachers who received the study PD than for those who did not;
• The PD had a positive impact on some aspects of instruction: Average ratings of teachers' use and quality of math explanations in the classroom were 23 percentile points higher for teachers who received the study PD than for those who did not; and
• The PD did not have a positive impact on student achievement: Students of teachers who received the PD scored, on average, 2 percentile points lower than control teachers' students on both a study-administered math test and state assessment. In general, this difference was not statistically significant.

A brief entitled Does Content-Focused Teacher Professional Development Work? Findings from Three Institute of Education Sciences Studies was released on November 2, 2016. This brief synthesizes findings from this study and two prior IES studies of teacher professional development in middle school mathematics and elementary school reading. Key lessons learned include:

• The PD improved teachers' knowledge and some aspects of their practice;
• Improving teachers' knowledge and practice did not lead to positive impacts on student achievement;
• Most of the measured aspects of teachers' knowledge and practice were not correlated with student achievement; and
• The consistent pattern of findings suggests that future studies might seek to better understand on what aspects of teacher knowledge and practice PD should focus, and how PD can achieve a larger impact on knowledge and practice that also impacts student achievement.