Do Leaders Really Understand the Research They Promote?

How often have you heard the words “research-based” used in your presence? Are you tired of hearing them? If you often drop those words in a conversation, are you sure you are using them correctly? Have you read the research, or just merely looked at the Cliff Note’s version of it?

Unfortunately, there are too many educators who promote the idea of “research based,” but who only look at the numbers that are assigned to the research, or have a lack of understanding in how the research was supposed to be used in the classroom.

For examples, we need to go no further than when Carol Dweck recently clarified the growth mindset (read Carol Dweck’s commentary on the topic here), and Howard Gardner had to dispel the myths around multiple intelligences. Additionally, it happens often with the Visible Learning work of John Hattie, who is a leading researcher in the field of education.

Visible Learning is about making the learning visible for students through the use of learning intentions and success criteria. It is founded on deep research around meta-analysis, which is a mathematical approach to combine many large studies, the veracity of which relies on effect sizes.

Over the years that I wrote the Finding Common Ground blog, I focused many posts on John Hattie and his Visible Learning research. For full disclosure, this is not meant to be a plug for Hattie’s work, nor for mine, because I have worked with him for the last four years. It’s just that I’ve come across a troubling trend when I speak to school and district leaders about his research. Leaders look at the influences with the highest effect sizes, and then dictate that teachers should use those, without understanding how the influence looks in action nor the research behind it.

The power of Hattie’s research is that he brings so many studies together and therefore can show how much of an impact specific influences can have on learning. However, that is where issues begin to arise, not for Hattie’s work, but rather how it is interpreted and then used by school and district leaders and teachers. And it serves as a cautionary tale.

To avoid the pitfalls of misusing research there are three areas that leaders should examine before they ever bring another “research-based” initiative to their schools again.

1. Don’t focus too much on the numbers

In educational research, a .40 typically refers to a year’s worth of growth for a year’s input. Hattie refers to this .40 as the hinge point. All of his 251 influences have an effect size attached to them. Some of the influences are well above a .40 and many of them are well below a .40. There are leaders who believe that their teachers should only focus on influences that are above a .40, and some suggest that their teachers should only focus on those above a .60. After all, the larger the effect size, the bigger the impact. Unfortunately, this is flawed thinking, because it’s not always about the effect size, but how we use the influence. It is the story about how the influences overlap that matters and this is why Hattie called his model of overlap “Visible Learning.”

Educators need to look at more than just larger effect sizes.

2. The Shiny New Toy isn’t for everyone…especially if you use it wrong

Too often leaders, whether they use Hattie’s research or not, chase after the shiny new toy, that everyone is talking about or they use—whatever new “research-based” solution is currently in vogue–because they assume it’s going to revolutionize their schools. Unfortunately, they implement the shiny new toy incorrectly, and it results in no revolutionary change. In order to implement anything properly, we must understand our current reality or that shiny new toy will never, ever work. We shouldn’t chase after shiny new toys because they look good on the shelf; we should chase after them because we need them, and know how to use them.

When it comes to Hattie’s research—or any other educational researcher for that matter—it’s about helping educators understand what works best for them at that moment in time, and making sure we all have a common understanding of how that particular influence should be implemented. After all, we may choose an intervention with a larger effect-size and implement it poorly.

3. All show, and no go

Leaders tend to call out which influence might work best based on their gut feeling, but very few have read the research deeply to learn how those influences should work. Regardless of whether the influence has a large effect size or not, it’s important to understand what the research says about those particular influences. This, however, also means that leaders have to focus on implementing a small number of influences for an extended period of time as opposed to randomly throwing out into the conversation their favorite influence of the week.

In the End

If leaders are going to position themselves as the “lead learners” where they may dictate what works best, they should have an understanding of what they dictate before they dictate it.

---

Peter DeWitt is the author of several books including Collaborative Leadership: 6 Influences That Matter Most (Corwin Press), and the newly released School Climate: Leading With Collective Efficacy.

https://www.edweek.org/ew/articles/2015/09/23/carol-dweck-revisits-the-growth-mindset.html