• Written by Alan Reid, Professor Emeritus of Education, University of South Australia

Since the OECD’s Programme for International Student Assessment (PISA) began in 2000, published results have sent commentators and politicians in some countries into meltdown. The release of the 2018 test results last week was no exception.

Out of 79 participating countries and economies in 2018 Australia came equal 11th (with countries including NZ, UK and US) in reading (in 2015 it was 12th); equal 13th (with countries including US, UK and Germany) in science (in 2015 it was tenth); and equal 24th (same as the OECD average, and NZ and Russia) in maths (in 2015 it was 20th).

Read more: Aussie students are a year behind students 10 years ago in science, maths and reading

The Daily Telegraph claimed Australian schools “are failing”. The Australian bemoaned Australia had “plunged in global rankings”; and business leaders told Australian educators to “lift your game”.

According to education minister Dan Tehan, “alarm bells should be ringing”. Tehan claimed he would raise the issue with state and territory ministers for education at the Education Council meeting in Alice Springs this week.

In particular, Tehan said he wanted to “take a chainsaw to the curriculum” and “put literacy and numeracy back to the heart” of it.

There are two striking features of the reactions to the PISA results. First, it seems broadly accepted PISA defines educational quality when it only tests three subject areas and the methodology itself is questionable.

And second, the results inevitably produce a flurry of policy responses, none of which have been specifically tested as a means to improve our PISA scores.

There are serious problems with PISA scores

PISA has seemingly become the arbiter of education quality in Australia and around the world. When the results are released, the numbers are broadly accepted as accurate measures of the quality of the world’s education systems.

This ignores the chorus of concerns growing for years about PISA’s serious methodological problems which test a stratified sample of 15 year olds in three curriculum areas.

These same concerns were raised by 100 educational researchers around the world in a public letter to the Director of PISA at the OECD in 2014.

Some such concerns relate to the problems of making an international test neutral when it is converted to many different languages and cultures. Other concerns come from educational statisticians and researchers who argue the validity and reliability of the tests themselves are at best dubious and at worst render the league tables “useless”.

Read more: The PISA world education test results are about to drop. Is Australia getting worse?

There are also issues with the sampling process. Some participating countries only include certain parts of the country or exclude more schools and students from the tests than others. This clearly makes it more difficult to compare countries.

Mainland China is an example. For instance, when Shanghai finished top in each of the three domains in the 2012 PISA tests, Tom Loveless from the Brookings Institute in Washington argued the students tested weren’t representative of the students in the city.

He wrote Shanghai’s PISA-tested students didn’t include thousands from poor rural areas whose parents had moved to the city in search of work.

The OECD’s Education Director Andreas Schliecher, admitted to the UK’s House of Commons Education Select Committee that, in fact, only 73% of students were represented in the Shanghai province’s sample of students in the 2012 test.

Loveless’ also analysed the recent results, which were reported in the Washington Post. He noted the four Chinese provinces participating in 2018 (Beijing, Shanghai, Jiangsu and Zhejiang) did significantly better than the four (Beijing, Shanghai, Jiangsu and Guangdong) that participated in 2015.

Loveless wrote “the typical change in a nation’s scores is about ten points. The differences between the 2015 and 2018 Chinese participants are at least six times that amount…”. He hypothesised this had something to do with the change in provinces selected for testing.

These and other problems with PISA’s methodology suggest it is foolhardy to accept the test results as precise readings of educational quality in any country, or for ranking countries.

Knee-jerk policy fixes will only take us backward

The other feature of the reaction to PISA results is the litany of policy proposals to fix the problems PISA has supposedly unearthed.

Minister Tehan confined his strategies to “stripping back” the “crowded” Australian curriculum and focusing on “the basics”, as well as fast tracking professionals from other fields into teaching.

But there is no obvious link between the PISA results and the strategies proposed and certainly no analysis of what information PISA offered to support them. Even if PISA data are taken seriously, we are obliged to investigate the reasons for educational outcomes before designing policies to address the problems.

And although PISA only tests three areas of the curriculum, the strategies proposed apply to the whole curriculum and, it seems, to the whole of schooling.

Minister Tehan should explain what part of the PISA data convinced him that the Australian curriculum should be pared back to “the basics”. And if that data exists, why should it apply to every other area of the curriculum apart from the three areas tested by PISA: maths, science and reading?

Or is PISA simply being used as a convenient vehicle for introducing favoured policy positions?

I am not suggesting some of the PISA data cannot usefully contribute to the ongoing effort to enhance the quality of Australian education. There is a strong case to be made for sharing educational ideas and practices with other countries.

But superficial, knee-jerk readings of international standardised test data are more likely to impede than advance quality improvement.

The meeting of the Education Council this week could use the opportunity to do two things when they reach the agenda item on PISA.

First, it could set in train a process for achieving national agreement about an approach to educational accountability which goes beyond the simplistic reliance on standardised tests. This would start with the purposes of accountability, including supporting schools to enhance education quality, and aim to provide the community with rich information about educational progress.

Read more: Estonia didn't deliver its PISA results on the cheap, and neither will Australia

Inevitably new approaches would mean broadening our evidence options. These should be both qualitative as well as quantitative. They should also support teachers in their work rather than impose time consuming form-filling. And they should be based on trust in the professional expertise of our teachers.

Second, the Education Council should institute a review of PISA which might consider the flaws in the testing regime, and ways to overcome these. This might help ensure policy, media commentary and research based on PISA results would acknowledge the limitations of the tests and be more tentative about using PISA as the sole arbiter of what constitutes quality in education.

These two steps would help break the destructive cycle of the release of PISA tests results every three years, followed by the barrage of criticising educators and the work of schools, as well as a flurry of political, ideologically pre-determined policy proposals.

After all, it is somewhat ironic that the policy makers who for so long have enforced a standardising educational policy regime, simply double down on it when their own standardising measures have deemed such policies to have failed.

Australia needs a new education narrative, and work on it should start this week at the Education Council.

Alan Reid is the author of “Changing Australian Education: How policy is taking us in the wrong direction and what can be done about it”, out now from Allen and Unwin.

Alan Reid has received funding from the Australian Research Council.

Authors: Alan Reid, Professor Emeritus of Education, University of South Australia

Read more http://theconversation.com/pisa-doesnt-define-education-quality-and-knee-jerk-policy-proposals-wont-fix-whatever-is-broken-128389

Best Road trips Tours You Might Take in 2020/2021

You probably know that a great road trip provides the level of adventure and pleasure like few other things. Not only do you get a lot of freedom, but you might also realise that it’s much che...

Diana Smith - avatar Diana Smith

3 Fall Landscaping Ideas to Keep Your Yard Looking Good

The leaves are falling, which means your yard is a mess. Check out these 3 fall landscaping ideas to keep your yard looking good year-round. In about a month fall leaves will begin to cover your ...

Sarah Williams - avatar Sarah Williams

The Victorian government has allocated $60 million to mental health. But who gets the money? 

The second wave of the coronavirus and the resulting restrictions have impacted all Victorians. Many are struggling, but some are struggling more than others. In response to the increasing number of p...

Chris Maylea, Senior Lecturer, Law and Social Work, RMIT University - avatar Chris Maylea, Senior Lecturer, Law and Social Work, RMIT University

Got someone with coronavirus at home? Here's how to keep the rest of the household infection-free

ShutterstockAlthough some positive signs suggest Victoria’s second wave may be slowing, we continue to see large numbers of COVID-19 cases recorded every day. Most people who test positive for C...

Thea van de Mortel, Professor, Nursing and Deputy Head (Learning & Teaching), School of Nursing and Midwifery, Griffith University - avatar Thea van de Mortel, Professor, Nursing and Deputy Head (Learning & Teaching), School of Nursing and Midwifery, Griffith University

Want to see a therapist but don't know where to start? Here's how to get a mental health plan

ShutterstockLast week, the Australian government announced it will provide ten extra Medicare-subsidised psychological therapy sessions for Australians in lockdown areas due to COVID-19.In such a stre...

Louise Stone, General practitioner; Clinical Associate Professor, ANU Medical School, Australian National University - avatar Louise Stone, General practitioner; Clinical Associate Professor, ANU Medical School, Australian National University

A new community case of COVID-19 in New Zealand is a matter of when, not if. Is the country prepared for it?

AAP/Nick PerryOur latest modelling shows New Zealanders need to redouble their efforts to keep track of people they meet, if we are to have a chance of controlling any re-emergence of COVID-19.The dir...

Michael Plank, Professor in Mathematics, University of Canterbury - avatar Michael Plank, Professor in Mathematics, University of Canterbury

Young men are more likely to believe COVID-19 myths. So how do we actually reach them?

ShutterstockIf the media is anything to go by, you’d think people who believe coronavirus myths are white, middle-aged women called Karen.But our new study shows a different picture. We found me...

Carissa Bonner, Research Fellow, University of Sydney - avatar Carissa Bonner, Research Fellow, University of Sydney

No, the extra hygiene precautions we're taking for COVID-19 won't weaken our immune systems

ShutterstockDuring the COVID-19 pandemic we’re constantly being reminded to practise good hygiene by frequently washing our hands and regularly cleaning the spaces where we live and work.These p...

Vasso Apostolopoulos, Pro Vice-Chancellor, Research Partnerships, Victoria University - avatar Vasso Apostolopoulos, Pro Vice-Chancellor, Research Partnerships, Victoria University

How to keep your contact lenses clean (and what can go wrong if you don't)

ShutterstockYou’re rushing and accidentally drop a contact lens on the bathroom floor. Should you:a) run it under the tap and pop it in?b) spit on it and do the same?c) use the cleaning solution...

Nicole Carnt, Scientia Senior Lecturer,  School of Optometry and Vision Science, UNSW, Sydney, UNSW - avatar Nicole Carnt, Scientia Senior Lecturer, School of Optometry and Vision Science, UNSW, Sydney, UNSW

News Company Media Core

Content & Technology Connecting Global Audiences

More Information - Less Opinion