Impact Evaluation of the Teacher Incentive Fund

Contractor: Mathematica Policy Research, Inc.; Vanderbilt University

Background/Research Questions:

The Teacher Incentive Fund (TIF) was renamed the Teacher and School Leader Incentive Program with reauthorization of the Elementary and Secondary Education Act (ESEA) in the Every Student Succeeds Act (ESSA). The purpose of the TIF program is to develop and implement performance-based compensation systems (PBCSs) for teachers, principals, and other personnel in high-need schools.

Research indicates that high quality teachers are critical to raising student achievement in low-performing schools, but schools most in need often have difficulty in attracting and retaining high-quality teachers. Performance pay is a policy promoted by the TIF program to improve the quality of educators in high-need schools. This evaluation studies performance pay that provides substantial and differentiated bonus pay to high-performing educators in low-performing schools with high-need students. The study will address the following:

• What are the characteristics of all 2010 TIF districts and their performance-based compensation systems? What implementation experiences and challenges did TIF districts encounter?
• How do teachers and principals in schools that did or did not offer pay-for-performance bonuses compare on key dimensions, including their understanding of TIF program features, exposure to TIF activities, allocation of time, and attitudes toward teaching and the TIF program?
• What is the impact of pay-for-performance bonuses on students' achievement on state assessments of math and reading?
• How do pay-for-performance bonuses affect educator mobility, including whether mobility differs by educator effectiveness?

Design:

The evaluation effort includes two separate teams: the implementation team to support fidelity of implementation to each grantee's proposed performance based compensation system (PBCS) consistent with the evaluation study, and the evaluation team to rigorously study the PBCS policy. Study schools were randomly assigned within a grant to either implement all components of the PBCS or the PBCS with a 1% across-the board bonus instead of the differentiated effectiveness incentive component of the PBCS. Data collection includes a district survey of all 2010 TIF grantees and the following data collection for schools participating in the random assignment study: teacher and principal surveys, teacher and principal school assignment records, student Record Information (such as student demographics and student test scores), and grantee interviews. Annual data collection will be used to document implementation information as well as to conduct impact analyses.

Cost/Duration: : $13,700,000 over 6 1/2 years (December 2009–August 2017)

Current Status: Data analyses are underway for the fourth year of program implementation for a report to be released in summer 2017. The first report was released on September 16, 2014. (http://ies.ed.gov/ncee/pubs/20144019/index.asp). The second report was released on September 24, 2015. (http://ies.ed.gov/ncee/pubs/20154020/index.asp). The third report was released on August 24, 2016 (http://ies.ed.gov/ncee/pubs/20164004/index.asp). The final report will be announced on http://ies.ed.gov/ncee.

Key Findings: The third report provides ongoing implementation information and initial impacts on student achievement.

The main findings among all TIF districts with 2010 awards are:

• Similar to the previous two years, most districts (88 percent) implemented at least 3 of the 4 required program components for teachers.
• By the third year, reported implementation challenges decreased with no more than one-fifth of TIF districts reporting any major challenges.

For the subset of 10 districts that agreed to participate in a random assignment study, key findings on the effect of pay-for-performance on educators include the following:

• After three years of TIF implementation, average student achievement remained 1 to 2 percentile points higher in schools that offered pay-for-performance bonuses than in schools that did not. This difference was equivalent to a gain of about four additional weeks of learning.
• At least half of the evaluation districts each year met the grant guidance for awarding differentiated performance bonuses for teachers. However, in each year, no more than 2 of the 10 districts awarded bonuses for teachers that were substantial or challenging to earn.
• Teachers' understanding of performance measures continued to improve, but only about 60 percent of teachers correctly reported being eligible for a performance bonus. In addition, teachers believed that the maximum bonus they could earn was no more than two-fifths the size of the actual maximum bonus that districts awarded, a finding similar to previous years.