Long-Term Impact Study
A new quasi-experimental study of Leading Educators’ work provides new proof for the power of teacher professional development, finding significant results for students.
To date, there’s been limited evidence that teacher professional learning (PL) can meaningfully improve teacher practice and student outcomes at scale. This study explores what allowed teacher leaders in three cities to get lasting results in math and literacy using 10 years of data end rigorous methods.
About the Program
During the 2015-16 and 2016-17 school years, Leading Educators ran a fellowship program for teachers and school leaders across Kansas City, Louisiana, Memphis, and Washington, D.C. that was designed to provide ongoing professional development on addressing mathematics and English language arts (ELA) standards in equitable, differentiated ways.
Over two school years, teacher leader fellows attended summer institutes, quarterly workshops, and monthly coaching meetings designed to develop the beliefs, knowledge, and skills to lead development for peer teachers. These teacher leaders, who often still taught a smaller number of students themselves, led weekly or bi-weekly collaboration around shared content area learning, lesson practice, data review, and lesson improvement to improve student learning across the school.
The Results
This study used ten years of student data to mesaure the effects from teacher participation in Leading Educators programming. There were 529 schools in the comparison group and 29 schools in the treatment group for this study. Treatment schools had a higher proportion of students who identified as Hispanic and Black, students who were English language learners, and students who were identified as neurodiverse learners.
Students
in schools where teachers participated in Leading Educators programming made statistically significant improvements in math and ELA proficiency that considerably exceeded the average effect size for elementary and middle school interventions.
Math achievement
showed a 28% increase (8.5 percentage points) over the 4-year period in the percentage of students proficient or advanced.
ELA achievement
showed a 17% increase (5.3 percentage points) over the 4-year period in the percentage of students proficient or advanced. The effect was significant at the 10% level.
This impact evaluation aimed to answer important questions about the magnitude and sustainability of teacher development programs and specific intervention features. Using a difference-in-differences approach allows us to be confident that a change happened and feel more certain about what caused that specific change to occur.
Results represent strong evidence of a positive impact in student proficiency in both ELA and Math across the partnership years and two years after. The estimated effects, which range from 0.15 to 0.63 standard deviations, are considered medium to large and of substantial importance in education by authors of different frameworks for interpreting education effects. These are impressive findings when we take into account the scalability and the low cost of an intervention that targeted a small proportion of teachers per school.
The impact on student proficiency was highest in the second year and for teachers enrolled for longer time.
The improvements are higher and statistically significant in the first two years following the beginning of the program, with the highest impact found in year 2. In the two years that follow, the effects are not statistically significant.
Beside higher effects in year 2, the study also found higher statistically significant effects when teachers were enrolled for longer than the average enrollment time of 15 months in both ELA and Math. These two results represent strong evidence that more time in the program is better.
Schools do better when district or charter group leaders are involved. Schools that had at least one district or charter leader enrolled alongside teachers had a larger statistically significant effect compared to schools where only teachers enrolled.
There is strong suggestive evidence that a smaller number of teacher leaders is better than a larger group. One explanation for this phenomenon could be connected to differences in teacher recruitment methods and teacher readiness. We found that programs where teachers enrolled individually had more prepared teachers on average (measured by years of experience, education level and baseline knowledge) compared to programs where teachers enrolled as teams. Another explanation is connected to coaching time. In the context of the fellowship model, a larger group of teachers translated into less individualized coaching time, which would reduce the effect on teacher and students.
How do these results compare to other research?
In this commentary piece, researcher Dr. Matthew P. Steinberg, an Associate Professor of Education and Public Policy in the College of Education and Human Development and the Schar School of Policy and Government at George Mason University, explains the research on teacher professional development and how these findings compare.
We acknowledge the importance of understanding our impact on teacher professional practice and student performance. We hope to share what we have learned about the design and implementation features that get real results with districts and partners who are looking to make similar changes in their communities.
We offer this paper as a call to action for other professional learning partner organizations to intentionally and rigorously examine their internal operations and impact. When we know better, we must do better.
