• Written by Alan Reid, Professor Emeritus of Education, University of South Australia

Since the OECD’s Programme for International Student Assessment (PISA) began in 2000, published results have sent commentators and politicians in some countries into meltdown. The release of the 2018 test results last week was no exception.

Out of 79 participating countries and economies in 2018 Australia came equal 11th (with countries including NZ, UK and US) in reading (in 2015 it was 12th); equal 13th (with countries including US, UK and Germany) in science (in 2015 it was tenth); and equal 24th (same as the OECD average, and NZ and Russia) in maths (in 2015 it was 20th).


Read more: Aussie students are a year behind students 10 years ago in science, maths and reading


The Daily Telegraph claimed Australian schools “are failing”. The Australian bemoaned Australia had “plunged in global rankings”; and business leaders told Australian educators to “lift your game”.

According to education minister Dan Tehan, “alarm bells should be ringing”. Tehan claimed he would raise the issue with state and territory ministers for education at the Education Council meeting in Alice Springs this week.

In particular, Tehan said he wanted to “take a chainsaw to the curriculum” and “put literacy and numeracy back to the heart” of it.

There are two striking features of the reactions to the PISA results. First, it seems broadly accepted PISA defines educational quality when it only tests three subject areas and the methodology itself is questionable.

And second, the results inevitably produce a flurry of policy responses, none of which have been specifically tested as a means to improve our PISA scores.

There are serious problems with PISA scores

PISA has seemingly become the arbiter of education quality in Australia and around the world. When the results are released, the numbers are broadly accepted as accurate measures of the quality of the world’s education systems.

This ignores the chorus of concerns growing for years about PISA’s serious methodological problems which test a stratified sample of 15 year olds in three curriculum areas.

These same concerns were raised by 100 educational researchers around the world in a public letter to the Director of PISA at the OECD in 2014.

Some such concerns relate to the problems of making an international test neutral when it is converted to many different languages and cultures. Other concerns come from educational statisticians and researchers who argue the validity and reliability of the tests themselves are at best dubious and at worst render the league tables “useless”.


Read more: The PISA world education test results are about to drop. Is Australia getting worse?


There are also issues with the sampling process. Some participating countries only include certain parts of the country or exclude more schools and students from the tests than others. This clearly makes it more difficult to compare countries.

Mainland China is an example. For instance, when Shanghai finished top in each of the three domains in the 2012 PISA tests, Tom Loveless from the Brookings Institute in Washington argued the students tested weren’t representative of the students in the city.

He wrote Shanghai’s PISA-tested students didn’t include thousands from poor rural areas whose parents had moved to the city in search of work.

The OECD’s Education Director Andreas Schliecher, admitted to the UK’s House of Commons Education Select Committee that, in fact, only 73% of students were represented in the Shanghai province’s sample of students in the 2012 test.

Loveless’ also analysed the recent results, which were reported in the Washington Post. He noted the four Chinese provinces participating in 2018 (Beijing, Shanghai, Jiangsu and Zhejiang) did significantly better than the four (Beijing, Shanghai, Jiangsu and Guangdong) that participated in 2015.

Loveless wrote “the typical change in a nation’s scores is about ten points. The differences between the 2015 and 2018 Chinese participants are at least six times that amount…”. He hypothesised this had something to do with the change in provinces selected for testing.

These and other problems with PISA’s methodology suggest it is foolhardy to accept the test results as precise readings of educational quality in any country, or for ranking countries.

Knee-jerk policy fixes will only take us backward

The other feature of the reaction to PISA results is the litany of policy proposals to fix the problems PISA has supposedly unearthed.

Minister Tehan confined his strategies to “stripping back” the “crowded” Australian curriculum and focusing on “the basics”, as well as fast tracking professionals from other fields into teaching.

But there is no obvious link between the PISA results and the strategies proposed and certainly no analysis of what information PISA offered to support them. Even if PISA data are taken seriously, we are obliged to investigate the reasons for educational outcomes before designing policies to address the problems.

And although PISA only tests three areas of the curriculum, the strategies proposed apply to the whole curriculum and, it seems, to the whole of schooling.

Minister Tehan should explain what part of the PISA data convinced him that the Australian curriculum should be pared back to “the basics”. And if that data exists, why should it apply to every other area of the curriculum apart from the three areas tested by PISA: maths, science and reading?

Or is PISA simply being used as a convenient vehicle for introducing favoured policy positions?

I am not suggesting some of the PISA data cannot usefully contribute to the ongoing effort to enhance the quality of Australian education. There is a strong case to be made for sharing educational ideas and practices with other countries.

But superficial, knee-jerk readings of international standardised test data are more likely to impede than advance quality improvement.

The meeting of the Education Council this week could use the opportunity to do two things when they reach the agenda item on PISA.

First, it could set in train a process for achieving national agreement about an approach to educational accountability which goes beyond the simplistic reliance on standardised tests. This would start with the purposes of accountability, including supporting schools to enhance education quality, and aim to provide the community with rich information about educational progress.


Read more: Estonia didn't deliver its PISA results on the cheap, and neither will Australia


Inevitably new approaches would mean broadening our evidence options. These should be both qualitative as well as quantitative. They should also support teachers in their work rather than impose time consuming form-filling. And they should be based on trust in the professional expertise of our teachers.

Second, the Education Council should institute a review of PISA which might consider the flaws in the testing regime, and ways to overcome these. This might help ensure policy, media commentary and research based on PISA results would acknowledge the limitations of the tests and be more tentative about using PISA as the sole arbiter of what constitutes quality in education.

These two steps would help break the destructive cycle of the release of PISA tests results every three years, followed by the barrage of criticising educators and the work of schools, as well as a flurry of political, ideologically pre-determined policy proposals.

After all, it is somewhat ironic that the policy makers who for so long have enforced a standardising educational policy regime, simply double down on it when their own standardising measures have deemed such policies to have failed.

Australia needs a new education narrative, and work on it should start this week at the Education Council.


Alan Reid is the author of “Changing Australian Education: How policy is taking us in the wrong direction and what can be done about it”, out now from Allen and Unwin.

Alan Reid has received funding from the Australian Research Council.

Authors: Alan Reid, Professor Emeritus of Education, University of South Australia

Read more http://theconversation.com/pisa-doesnt-define-education-quality-and-knee-jerk-policy-proposals-wont-fix-whatever-is-broken-128389

All You Need to Know About Trenchless Technology

For many years, the traditional sewerage lines and pipe developments were not enough due to the long wait and cracking. The traditional sewer pipe repairs involved cracking the earth to find the par...

News Company - avatar News Company

Before we rush to rebuild after fires, we need to think about where and how

A primary school in East Gippsland was burnt down in the current bushfire crisis. While Premier Daniel Andrews immediately committed to rebuilding the school as it was, media reported the local CFA ca...

Mark Maund, PhD Candidate, School of Architecture and Built Environment, University of Newcastle - avatar Mark Maund, PhD Candidate, School of Architecture and Built Environment, University of Newcastle

Australian sea lions are declining. Using drones to check their health can help us understand why

Australian sea lions (Neophoca cinerea) are one of the rarest pinnipeds in the world and they are declining. Jarrod Hodgson, CC BY-NDAustralian sea lions are in trouble. Their population has never rec...

Jarrod Hodgson, PhD Candidate, University of Adelaide - avatar Jarrod Hodgson, PhD Candidate, University of Adelaide

With costs approaching $100 billion, the fires are Australia's costliest natural disaster

It’s hard to estimate the eventual economic cost of Australia’s 2019-20 megafires, partly because they are still underway, and partly because it is hard to know the cost to attribute to de...

Paul Read, Climate Criminologist & Senior Instructor/Lecturer, Faculty of Medicine, Monash University - avatar Paul Read, Climate Criminologist & Senior Instructor/Lecturer, Faculty of Medicine, Monash University

In cases of cardiac arrest, time is everything. Community responders can save lives

Cardiac arrest can occur with little or no warning in people who were previously healthy, including young people. From shutterstock.comEach year more than 24,000 Australians experience a sudden cardia...

Bill Lord, Adjunct Associate Professor, Monash University - avatar Bill Lord, Adjunct Associate Professor, Monash University

So the government gave sports grants to marginal seats. What happens now?

When Australians pay their income tax, they assume the money is going to areas of the community that need it, rather than being used by the government to shore up votes for the next election. This is...

Maria O'Sullivan, Senior Lecturer, Faculty of Law, and Deputy Director, Castan Centre for Human Rights Law, Monash University - avatar Maria O'Sullivan, Senior Lecturer, Faculty of Law, and Deputy Director, Castan Centre for Human Rights Law, Monash University

The Olympics have always been a platform for protest. Banning hand gestures and kneeling ignores their history

It is the year of the Tokyo Olympics, and the International Olympic Committee was quickly out of the blocks with new guidelines regarding athlete protests. The IOC is worried the biggest stories of...

David Rowe, Emeritus Professor of Cultural Research, Institute for Culture and Society, Western Sydney University - avatar David Rowe, Emeritus Professor of Cultural Research, Institute for Culture and Society, Western Sydney University

Where Can You Get Weed By Ordering It Online?

Nowadays, everyone wants to get their hands on some weed. Marijuana has become legalized in a lot of countries worldwide. People wait in lines for days to buy some. You couldn’t have imagined that...

News Company - avatar News Company

Hidden women of history: Catherine Hay Thomson, the Australian undercover journalist who went inside asylums and hospitals

Catherine Hay Thomson went undercover as an assistant nurse for her series on conditions at Melbourne Hospital. A. J. Campbell Collection/National Library of AustraliaIn this series, we look at under...

Kerrie Davies, Lecturer, School of the Arts & Media, UNSW - avatar Kerrie Davies, Lecturer, School of the Arts & Media, UNSW

Sick and Tired of Your Dead End Job? Try Teaching!

Tired of the same old grind at the office? Want an opportunity to impact lives both in your community and around the world? Do you love to travel and have new experiences? Teaching English is the perfect job for you! All you need is a willingness to ...

News Company - avatar News Company

The Impact of an Aging Population in Australia

There’s an issue on the horizon that Australia needs to prepare for. The portion of elderly citizens that make up the country’s overall population is increasing, and we might not have the infrastructure in place to support this. Australians h...

News Company - avatar News Company