Are We Misunderstanding the Evidence? The Hidden Pitfalls of Education Research

June 16, 2025 | The Learning Counsel

A growing number of researchers across social sciences, including education, are grappling with a fundamental challenge: how we interpret the results from research. The way we’ve traditionally analyzed and understood study results has some critical flaws that can lead us astray.”

In a new opinion piece for The Learning Counsel, Rebecca Taylor-Perryman, managing director of research and data at Leading Educators, weighs the challenges of replicating evidence-based improvement models and the missed opportunities to learn from programs that didn’t show positive evidence.

At the heart of the issue lie some common research practices related to how we determine if a finding is “real.”

Citing Leading Educators’ strong evidence from Harlem, she offers a few ways to raise the bar for evidence quality and how it gets used:

1. Demand multiple studies: Ask vendors and partners for results from several impact studies, not just a single promising one.
2. Consider the context: Inquire about who was included in the study and who was not.
3. Be skeptical of large claims: Exercise caution with interventions reporting very large effect sizes (remember, a moderate effect is generally between 0.05 and 0.20 standard deviations) and ask how those estimates were calculated.
4. Prioritize transparency: Work with partners who are open about the limitations of their data and can explain their impact in ways that resonate with the realities of learning.