An expert’s experience of working with the OECD
Every three years, the journalists of national media across the world sit with pens poised to pass judgment on how the schools of their nations have performed on the Program for International Student Assessment – otherwise known as the PISA test. This high profile test simply benchmarks the educational performance of 15 year olds in different countries in relation to each other. But the results are also used to determine who is strong and who is weak, the nations that are on the rise and those that are on the wane.
At the end of each triennial cycle, the often unlikely titans who turn out to top the rankings are heralded to the shame of the rest. In 2000, it was Finland, then it was Singapore, and then Shanghai. Leaders of advanced economies wring their hands in public after being beaten by Eastern European competitors, as if they were captains of industry who had been outsmarted by peasants and potato eaters. Many countries, The Economist naughtily claimed, are prone to PISA envy. Some, like Germany after the first PISA test results, go into PISA shock.
The PISA assessments are designed and administered by the Organization for Economic Cooperation and Development (or OECD). Headquartered in Paris, the OECD consists of 34 predominantly Western European nations (but also including the US, Canada, Chile and Mexico in the Americas as well as Australia, New Zealand, Korea and Japan) dedicated to promoting market economies and political democracy. Set up just after World War II to help administer the U.S. Marshall Plan, the OECD now addresses a wide range of social policy matters, including education, where 65 nations and territories currently participate in its PISA tests.
In recent years, PISA and therefore OECD have been subject to mounting critique by educators. In May, a letter signed by a large number of academics worldwide denounced PISA for major sampling errors, for driving short term policy efforts among countries trying to outdo each other on the tests, for promoting teaching to the test by marketing packages of PISA-like items to schools, and for leading educational systems to focus on economic outcomes at the expense of other equally worthy outcomes that serve the broader development of humankind.
These criticisms have some validity, though they ignore the point that PISA and OECD draw great attention to equity issues as well as overall rankings, and that the findings of PISA generally provide little or no support for the market driven and test-centered solutions that are preferred by many lower ranking countries and of which many academics are also critical.
There is also another side to the educational work of the OECD as well as its PISA tests, that is also stimulated by those assessments. At the request of countries and their Ministers of Education, OECD undertakes reviews of aspects of those countries’ educational policies and practices and it conducts thematic reviews of particular policy areas such as equity or leadership across countries. These reviews are based on a methodology where an OECD team including outside experts undertakes background work on the educational performance of the country in question (including but not restricted to PISA data), along with collection of information on the country’s economic performance, educational policy history, and patterns of poverty and inequality. The appointed review team then undertakes an intensive visit to the country spanning at least one full week that includes visits to schools, teacher education programs, inspection and assessment services, local school districts, teachers’ and administrators’ organizations, university and Ministry research departments, and the offices of Ministers and their senior staff. I have participated as an expert in two of these reviews.
In January 2007, I went to Finland as part of an OECD thematic review of the relationship between policies and practices of educational leadership and system-wide educational performance. Finland had been a top performer on PISA since the origin of the tests and was being inundated with visitors, yet the reasons for the nation’s high performance were something of a mystery, including among Finns themselves.
In a chapter we published in 2008 and in a later chapter of my book with Dennis Shirley on The Global Fourth Way: the quest for educational excellence (Corwin 2012), we were able, for the first time to provide one of the first coherent and globally known narratives of the reasons for Finland’s success. These included a strong vision of and value for public education in which almost Finnish children participate as the creator of Finland’s future society; resulting high status for the country’s teaching profession whose members are stringently selected through rigorous university-based teacher education programs that confer Masters degrees on all of them; a widespread culture of collaboration in curriculum development among teachers in each school district; an equally robust culture of collaboration among all partners in strong local municipalities where most curriculum and other policy decisions are made; and a system of widespread cooperation and trust instead of US-style test-based accountability.
Without this and other reports by OECD and others on the reasons for Finland’s high performance on PISA, there would be no material to underpin the praise that many US educators and academics have heaped on the Finnish system in opposition to the ineffective US educational policies of market competition and standardization that have yielded no success on PISA.
In October 2013, I participated as one of two international experts in another OECD team to review the school improvement strategy of Wales. Sandwiched between the Irish Sea on one side and England on the other three, this nation of just over 3 million people has the highest poverty rates in the United Kingdom and ranks in the bottom third of all participating countries on PISA. Wales is the lowest ranking of the four UK countries on PISA and it is the only one to differ from the rest by a statistically significant degree.
Our review, published in April 2014, paid tribute to Wales’ strong showing on equity among OECD countries, and to its fully comprehensive system of secondary schools where there is negligible private sector participation, and where there are no English-style academies or free schools (the equivalent of US charter schools). These are features that the country shares in common with high performing Finland. As we noted in our report, Wales had also abolished standardized testing after it had taken over control of its educational system from the UK government in London – a direction that seemed to be consistent with practice in Finland. But, in contrast to Finland, as the country’s own plans revealed, its social challenges were high and its educational performance was in need of improvement. So what were the areas of difficulty, and what policy options could we offer to address them? We proposed four.
1. Meet the learning requirements of its students and deliver equity and quality. We proposed developing more differentiated and personalized approaches to teaching and learning in order to address inequities, along with the provision of more qualified support staff to work alongside teachers with challenging students.
2. Build professional capital and a culture of collective responsibility for improved learning for all students. This involved developing strategies to attract high quality teachers into teaching, including extended an existing government-funded Masters program for early career teachers, and also strengthening university-based teacher education (instead of replacing or eroding it with a market of other kinds of providers). Building professional capital involved reducing the feelings of overload among teachers who felt they had to implement too much too fast. One way to achieve this, we said, was to learn from the mistakes of other jurisdictions and sequence the literacy and numeracy strategy into literacy, then numeracy, or vice versa. Last the professional capital of teachers could be developed further by devising specific ways to build strong social capital among teachers and leaders across schools so that more help for improvement would come from one another rather than from outside intervention. All this would work better, we said, if developing leadership at every level was treated as a system priority and driver of change, not an afterthought to it.
3. Create a coherent assessment and evaluation framework:
After devolution from England, Wales disposed of its standardized tests. But getting rid of a bad thing is not the same as instituting a good thing. Teachers had little training in classroom assessments, processes to moderate those assessments across teachers were not strong, and grade inflation began to occur year upon year as teachers felt they should be improving all the time. The government reacted by reintroducing standardized testing for grades 2 to 9, so we advised reviewing and refining this strategy by working to minimize unintended effects such as teaching to the test, and suggested that in time the tests may be reduced to a couple of grades, like the highest performing countries, using the money saved to provide higher quality training in classroom assessments.
4. Define and implement policy with a long-term perspective: The point in raising standards is not to push to get a higher ranking on PISA 2015 for its own sake, and we discouraged policy makers from only doing this. Instead, as the BBC news headlined our review: “Welsh government lacks education ‘long-term vision’” of the kind of learners it wants its students to be. Overall, we said, the Welsh government should, “Develop a shared vision of the Welsh learner, reflecting the government’s commitment to quality and equity, and translate it into a small number of clear measurable long-term objectives”.
It was an inspiring vision, in the midst of economic crisis, that galvanized Finland into valuing its teachers and basing the future of the country upon their expert shoulders. Wales has Finland’s strong record on equity and also its widespread commitment to and investment in public education and public good by all the society. What it is missing is the leadership capital and professional capital that could be ignited by such a vision.
Despite all the controversy around OECD’s PISA testing, in its work with actual nations and systems, you do not see OECD promoting more US-style market-based systems of school competition or teacher preparation. You do not see arguments for more standardization, or for driving national reform efforts by panic-driven demands to score higher on PISA. In our reports, and ones like them, the point is to learn from PISA and many other indicators, and to work with countries in the quest to achieve greater quality and equity in their systems that is in line with the strongest principles and best practices of the highest performing systems. The US could learn a lot more from reading and reflecting on these country reviews than from weeping and wailing about where it ends up on the international rankings.
