Visible Learning Effect Sizes When Schools Are Closed: What Matters and What Does Not 

With the prospect of schools being out for a while, I will review some of the Visible Learning influences that may help understand what really matters and what does not matter as much.  

First, does it matter that students are not in the physical place called school? 

There is a body of research on the effects of losing school time, but we need to start by distinguishing between effects from school holidays and school closures. Holidays are for recharging the batteries for students and teachers. If your system is moving to holidays, do not worry about schooling: Let the teachers and students recharge their batteries doing other activities.  

The effects from school holiday are very small on students, and there is little reason to believe that the length of the school year has much effect at all. Note that the so-called vacation effect (-.02), summer school length effect (.08), the summer school effect (.19), and the effect of modifying school calendars (.08) are low.   

There is no meta-analysis of the effect of the length of the school year, but there are traditional reviews, and the effect is tiny. From the PISA results, Australia has one of the longest school days and school years across all countries, and the USA is close behind. If we take out one term/semester of 10 weeks, those countries still have more in-school time compared to Finland, Estonia, Korea, and Sweden, which all outscore Australia and the USA on PISA.  

There is data on the effect of teacher strikes and lengthy shut outs—and again the message is that the effects are very low, especially for students below middle school, but they increase after middle school, especially in math. 

Let’s recall the effects of the Christchurch earthquakes in 2011, which severely disrupted access to schools. There was a rush to online learning with a cry was for special dispensations for upper high school examinations. As advisor to the Qualifications Authority that oversaw these exams, I argued we should not give special dispensation. I based this on strike research, which showed no effects at this upper school level, with positive effects in some cases. Sure enough, the performance of Christchurch students went up, and as schools resumed, the scores settled back down. Why? Because teachers tailored learning more to what students could NOT do, whereas often school is about what teachers think students need, even if students can already do the tasks. 

  So, the messages are: 

• Do not panic if our kids miss 10 or so weeks 
• If your system has school holidays, then have a holiday from schooling 
• Worry more about subjects in which parents have the least skill and about subjects and tasks where parents make kids skill and drill and lose the thrill (especially math) 
• Make sure to provide opportunities to learn what students do not know and do not engage them in busy work  
• It is not the time in class, but what we do in the time we have, that matters 

What’s the effect of the home? 

The most likely implication of school closures relates to equity. Students who come from well resourced families will fare much better than those from lower resourced families: The effect of home resources is powerful (d = .51). I have rarely met a parent who does not want to help the child, but some do not have the skills. Remember, we made schooling compulsory because teachers are better at teaching than parents.  

We need to be doubly concerned about those students who most need teacher expertise—those from homes where parents are least likely to be teachers, students with special needs who require specialized instruction, those who already do not like learning at school, and those who come to school primarily to be with their friends (for them learning alone is a killer). 

The good news is that learning at home is related less to the family structure (LGBTQIA, one two parent, adopted parents), immigrant status (d = .05), or whether one or two parents are working (.03), and more to the parents’ skills at becoming school teachers. 

Let’s look at the effect sizes of homework. The effect goes up when students are asked to practice something they have already learned at school, and down when they are asked to engage in learning by themselves or with parent help. Opportunities to practice—yes, projects—no. In the current situation, students should not be doing “homework” since there are no regular classes, but should instead be engaged in projects where there is attention to the precious knowledge and not to completing some task at any level of quality and without learning new knowledge, new relations between ideas, or deeper understanding. 

Home schooling might seem apt in the current situation, but sadly it is not. The majority of students who are home schooled in the USA and Australia are in families that are more religious, more conservative, more white, more affluent, and have better educated parents. Yes, home schooled students score about one grade level higher than their school peers and perform at the 80th percentile (when their peers, by definition, score at the average 50th). However, students who are home schooled already start at the 85th percentile, so they do no better or worse at school or home. Perhaps the takeaway is that only parents who are well educated and who have attributes of good teachers are likely to home school with some success. It is the teaching qualities of the parents that matter—and for many kids, their parents are not great teachers of school-related work. 

The home factors that do matter include parental involvement (.43) (and note this includes father involvement [d = .21], so let’s not presume that it is the mother who is responsible alone for teaching during this crisis), family communication quality (d = .56), and especially parental expectations (.70). 

So, the climate of the home for learning matters: high expectations and high levels of communication (talk, talk, talk, listen, listen, listen). It needs to allow for errors and mistakes as opportunities to learn, not opportunities to do it again with the hope that the second time it will magically become right. Any learning should include opportunities for students to give feedback about their learning and to receive feedback about where to go next. This is a key skill of teachers, but often less so of parents. 

Let me pause and note three other factors about parents: 

• Any sense by the child that they are under parental surveillance leads to decreased impact of learning at home. Parents cannot become learning police, demanding sustained concentration until completion. Teachers are not police, nor should parents be. 
• Parental involvement decreases once reading is attained, so more attention to the pedagogical aspects in non-reading tasks becomes important 
• The negative effects on learning increase for subjects where parents are least likely to be skilled or have knowledge, such as math, science, and history  

So, what does all this mean?   

Our students are differentially advantaged or not as a function of their homes. A benefit of schooling is to reduce these inequities in home resources, skills, and opportunities. Parents can be, but are not always, great teachers of school–related topics. A great teacher has deep pedagogical content knowledge, skills in motivation and engagement, skills in dealing with disruptions and boredom and distractions. Parents may want to expel their students from the home school, and many will give up. It is hard to teach some kids! The current crisis might make parents more appreciative of the skills of teachers, and we can use the pandemic as an opportunity to increase the esteem of the teaching profession: Perhaps all teachers deserve a massive pay rise from the expertise they demonstrate over the next six months. 

We need to find ways to ensure that students at home engage in the optimal tasks; not just busy tasks, not just projects that keep them entertained, not boring repetitive activities. The choice of task matters critically, but let me remind you of the Christchurch earthquake finding: It is the choice of tasks relative to where students are now and where they need to go next that advances their learning. Use the technology for great diagnosis; share scoring rubrics and success criteria up–front with students before they get too involved in the task; be clear (teacher clarity matters more when students are not in front of you to correct, cajole, give instant feedback to), and evaluate progress as you do in the physical classroom. Also, build formative evaluation opportunities into the tasks. 

Remember, if students get stuck, do not know what to do next, or make errors, you should not depend on the parents to know about the errors or what best to do next. We do not want parents giving feedback in a way that ends up with them doing the work! Take ownership of feedback as much as possible. 

Create as many opportunities for social interaction, not just between you and the student, but using technology for students to work, share, interact, and learn together, as you so often do in the regular classroom. Learning at home need not be a lonely activity, with the only or even primary resource the parent. 

We know the effect size of technology remains low and has been so for the last 50 years. As Dylan Wiliam has often said, technology is the revolution that is still coming! The effect of distance learning is small (.14) but that does not mean it is NOT effective—it means it does not matter whether teachers undertake teaching in situ or from a distance over the internet (or, like when I started in my first university, via the post office). What we do matters, not the medium of doing it. 

I do note that the highest effects of digital technology are interactive videos (.54), intelligent tutoring systems (.51), in writing (.42), and in mathematics (.35). The lowest effects are the presence of mobile phones (-.34—turn them off), and the presence of one-on-one laptops (.16). However, most of these studies have been undertaken in classrooms and thus are perhaps not so relevant in this crisis. 

Perhaps the most exciting use of technology for our current situation is the increasing power of social media to enhance learning. Marie Davis (2018) has explored asking students to use social media, such as Edmodo, to have students send questions and talk about what they do not know. They are more likely to do this on social media than directly to the teacher. What an opportunity to exploit in the current situation! 

I know every school has answers to deliver via electronic means (I am stunned at the number of “experts” who are talking AT teachers regarding technology use right now!), but we need to view technology use like planning lessons and creating resources: It is the means and starting point, not the core, of teaching. It is the decisions we make as students are learning, as we listen to them think aloud, as we give them alternate strategies and help them work with others to jointly advance learning, as we formatively evaluate our impact, that are important. Use social media as much as possible to make these connections. 

Help parents see their role as creating routines for learning and allowing their children to not know: Tell them that there is no point having their kids learn stuff they already know, not knowing is a sign of readiness and excitement of learning. Create opportunities for teacher-student and student-student interaction. 

But keep in mind that there is a work balance not only for students but also for teachers. Deliver mini-classes using social media, make them clear, and provide oodles of opportunities for feedback. Make it skill–based and provide exciting ways to practice, and not just project–based which can (but does not need to) lead to busy work with little learning. Worry more about subjects that parents are least likely to be able to help with, like math and science, and encourage kids and parents to read, read, read and also talk about their reading, so the story is important, the  vocabulary is stretched, and then simultaneously keep teaching the skills of reading to make reading pleasurable.   

Distance Learning 

Distance learning shows a very low effect size, but that means that it does not matter whether you are distant or not and should not be interpreted as “distance is disastrous.” What is more important is the methods of teaching, not the media. 

My thoughts about online learning are: 

• Optimize the social interaction aspects (we do not want to be talked at, but learn with) 
• Check for understanding (listen to the feedback from the students about their learning even more when you do not have the usual cues of the classrooms 
• Make sure there is a balance between the precious knowledge and the deep thinking (too often online favors the former over the latter) 

Bottom line: Understand what it means to be a learner online. When the usual peer interactions are often not as present, the teacher’s observational skills are different, and there is too often an over-emphasis on content and repetition. 

And the teachers… 

And who is looking after teachers during this time? How can we:  

• Know our impact from a distance 
• Learn how we as groups of teachers can evaluate, discuss, and work together 
• Discover ways to enhance the collective efficacy of all (now with the parents)  
• View this pandemic as an opportunity to learn more about how to work with students from afar, outside of our normal comfort zones of the classroom and school 
• Engage with parents to realize we as educators have unique skills and expertise (and are happy to share them), and not get upset if students are not spending 5-6 hours every day in the belief that school at home is but a mirror of the typical school day   

Over the next months many students will look for support from a trusted adult, seek help from peers, seek ways to express their feelings, and establish some sense of routine. Many of us will go through feelings of shock and disorganization. There will be altruistic or heroic phases and claims, and honeymoon periods of high morale and action and optimism, but some will feel angry, displaced, and lonely.  

Rob Gordon is an Australian who has studied the effects on schools from bushfires and talks about the desired state of fusion: “In this state of fusion, members identify with each other because they share the same experience; they feel strong emotional attachments because of what they have undergone together and rapidly develop a shared culture of stories, symbols, and memories” (2004).   Make sure every child experiences this fusion and becomes an insider in the online classroom. That schools and classrooms already existed helps develop this bond, but now to maintain it. Build on symbols, rituals, and identity; model connectedness; use stories and other artefacts so that when schools reopen there is a symbol, a thing, an event—a mosaic, a play, stories, collective memories of the at-home socially distancing experience. 

We know from the aftermath of Hurricane Katrina in New Orleans how important it is for teachers and school leaders to be visible, decisive, trustworthy, respected, and willing to engage in frontline work (Porche, 2009). After Katrina, those who had prior history of problems were more likely to show symptoms of traumatic stress, depression, sadness, anger, anxiety, and loneliness—for both students and teachers (Osofsky, Osofsky, & Harris, 2007). The effect on student achievement was not as great as many expected. Students were out of school between three and seven weeks and many had no school work in this time: There was a drop of -.17 from Katrina, but “what is more surprising is how quickly the Parish evacuees recovered from the experience and actually began to see gains in test scores” (Sacerdote, 2012, p. 131; see also Payne, McCaffrey, Kalra, & Zhou, 2008, who showed a drop of only .06 in statewide test scores from the outage). 

Schools, no matter via what medium, can be hubs of response and recovery, a place to support emotional recovery and promote social togetherness—and this is as important as any achievement gains. It would be wonderful to use this pandemic as an opportunity to learn about learning from afar, so share stories of success of teachers and students learning from this crisis, pay particular attention to below average or special needs students, discover how to develop collective efficacy among teachers and school leaders, and use this experience to learn how to best work with all students. 

---

References

Porche, Demetrius J. “Emergent leadership during a natural disaster: A narrative analysis of an acute health care organization’s leadership.” (PhD diss., Capella University 2009).