In June, the journal Educational Philosophy and Theory published “A Critique of John Hattie’s Theory of Visible Learning” by Thomas Aastrup Rømer. To quote from the paper’s abstract:
First, I argue that the theory is a theory of evaluation that denies education as such. Second, I show that there are problems with the dependent variable, learning, i.e. the effect of a given intervention. Thirdly, I show that Hattie’s theory belongs to the radical constructivist paradigm. Thus, the problems of constructivism, i.e. problems of normativity and the outside world, walks directly into Hattie’s concept of teaching, resulting in a double breakdown of the essence of teaching. Fourth, I argue that Hattie’s concept of feedback has a centralizing trend which ultimately has the potential to transform the country’s educational activities into a big hierarchical and data-driven organism. Finally, I show that Hattie’s reference to Karl Popper’s theory of “three worlds” is based on a highly problematic reading, where Popper’s objective world is reduced to a subset of a radical subjectivity.
In the following blog post, John Hattie responds to each section of the paper.
Most of Rømer’s critique is based on a reading of Visible Learning (2009), and fails to keep up to date with the improvements, modifications, extensions, and other contributions (especially the curriculum based books).
1. Hattie’s theory is an example of what happens when a theory of evaluation marginalizes and even destroys or forgets educational theory and practice
Rømer calls the VL a “rather simplistic theory of evaluation,” is “about measurement and behaviorism.” He defines evaluation in a way that then leads to his critique – “a retrospective investigation of the degree of realization of a given set of goals of a well-defined intervention, policy or program” whereas I see evaluation more Scriven-driven as evaluating the merit, worth, and significance – in this case of the impact on students (where I am careful to talk about impact in a far more generalized notion than achievement and test scores). Unlike Rømer, I have no difficulties seeing the VL work as being embedded in evaluation (which many have claimed to be the trans-discipline of all!).
Yes, I see a major goal “of learning is that learners should be able to be ‘self-monitoring, self-evaluative, self-assessing and self-learning’” – this is what is often meant by life-long learning, meta-cognition etc. Yes, I reference evaluators, but also reference many non-evaluators despite Rømer’s claims. Education is more, however, than evaluation – there is a what, where, who, and so much more (as noted throughout the books). Rømer then confuses evaluation with measurement, and teaching is more than an intervention (as noted throughout and chapters on teaching methods and approaches)!
1.1 The exclusion of education
Rømer notes that I do not exclude wider notions of “education,” completely ignores my wider work by focusing only on one of the Visible LearningTM books (there are now 16). I never said the “teacher must disappear into the learner” and he mistakes the claim about teaching the student to become more like the teacher for his own reductionist claims. He then stretches the claims by his own view (misinterpretation) that the “teacher must leave behind even the love of discipline, profession, science, and content, in favor of a self-love that is expressed in an idea of the student who tries to become an evaluation teacher.” What a stretch.
Yes, I start with an example of evidence-based medicine, but then education is indeed also an evidence–based discipline – it is using this evidence that is more the issue. Like Rømer, there are limits to any analogy, although education is full of distractions, full of claims with limited if any evidence, and too often claims are made with no recourse to evidence of impact on students. Indeed, one of the major claims in Visible Learning is that it is not difficult to find “some” evidence of positive impact – as nearly all interventions have some positive impact! The claim is to move far beyond merely asking for evidence.
His example of “school” being “independent” is his claim. I make a strong case for influences of the school, but point out that these influences are not as powerful (on average) as many others. To say, “learning becomes something outside ‘school’ is nonsense.” And I make much play that it is NOT reasonable, as Rømer claims to merely “combine the variables a, b, and c to get d > .4.” I never said that “all schools anywhere in the world could be the “same.”
1.2 The overall mechanism of destruction
This section is a straw person by Romer – first, I do not “replace education by evaluation theory” – he makes this claim. He misses the point when he parodies the 138 as “simple and atomized;” some of them are very broad, there is much overlap as I claim throughout, and it is hard to see how all this leads to the idea that Visible Learning “swallowed the very thing that it should evaluate”.
2. Problems of validity in the dependent variable
Rømer’s major point is: “Even if we accept the idea of ‘learning’ as an evaluation effect, Hattie does not make a sufficiently precise definition of his dependent variable, that is, his effect variable: ‘achievement outcome/learning out- comes/learning’.” But then he notes I have a very broad notion – including surface and deep learning, and have written extensively (as have many others cited) on these topics. It is indeed a “serious criticism” of Rømer that he ignores this associated work but prefers his own narrow interpretations of the meanings of achievement/learning. As I noted in Chapter 10 of Visible Learning, I agree that too much schooling and research is overly focused on surface learning – indeed have written about the “grammar of surface” in our schools and systems.
3. Hattie’s theory is a twofold attack on the concept of teaching and on the European cultural tradition
3.1 The dissolution of the teacher and the reconstruction of the teacher on the premise of dissolution
Yes, I do claim “a special kind of student-centeredness—the fact that the teacher must SEE learning through the student’s eyes” but never reduced it to student construction or have the teacher “integrate himself into the student” or that the teacher becomes the non-teacher! He then roams into a critique of constructivist theory claiming that my work is “a radical constructivist theory.” But I never take the outside world out of the model (students and teachers bring their skill, will and thrill to the classroom), in Visible Learning there is reference to “social, scientific, political or economic aspects of educational life,” yes there is little reference to Bildung (but see the forthcoming book on this with Steen Larsen), and it surely not credible to claim that because a cover has a picture of the “brain” that I reduce all to the inner workings of the brain!
3.2 The break from Western tradition
Yes, I have little regard for passivity, but Rømer’s jumps into his own beliefs seem unrelated to the claims in Visible Learning, and it is strange jump to his conclusion that “Hattie’s theory could thus be classified as a kind of transition in the process of self-oblivion in Western culture.”
4. Hattie’s concept of feedback has a built-in centralizing effect
Again, Rømer ignores my many writings on feedback, claiming my notion is “philosophically unreflected” and has no connection with Socratic dialog or anything similar. I do not accept Rømer’s reductionism of my notion of feedback to a ‘deductive and an inductive aspect.’ I have spent much research time on how individuals receive feedback, have written extensively on the variability of the effects of feedback, and his claim about the full-page figure as technical, atomistic misses the thrust of the picture – there are many instances, implications and impacts of feedback! (see Hattie & Clarke, 2019).
Yes, Rømer cites me as saying “feedback is not primarily about making art, science, and culture visible to the student: it is more about making learning visible to the teacher” (although I say “not only,” not “not primarily”). And I do not ever say that “Feedback are bits of d > .4 information coming to the teacher from the learner …” His claims about the hierarchy like implementation of d>.40 is a mistaken parody. Nowhere does it state that feedback “is about collecting information and data from lower levels of learning,” and his picture of a huge international post-human sucking up of >.40 is his invention.
5. Hattie radicalizes an already reductionist reading of Popper’s ‘three world theories’
There is indeed little reference to Popper’s world 3 (“of cultural and scientific objects”) in Visible Learning, as that was not the focus of this book; and I never claimed that “the ‘self-monitoring learner’— takes place on the level of World 3.” Most of Rømer’s claims relate to Bereiter, and confuse my claims about impact on learning as if there is but one notion of “learning” – which can include cultural and scientific claims and objects.
6. Conclusion
Most of the claims in Rømer are from a rather narrow reading of Visible Learning. Yes, it is based on a theory of evaluation, but does not “throw away educational concepts and relationships.” All advances in our field are based on critique, and the comments in Rømer can add to this debate – but a wider reading of my work other than one book seems minimal. I thrive on critique, hard benefited from such critique, and I trust I have improved my writings because of critique.
