• Written by Hassan Vally, Associate Professor, La Trobe University
It can be difficult to work out whether you should believe a study's reported findings. GaudiLab/Shutterstock

Who doesn’t want to know if drinking that second or third cup of coffee a day will improve your memory, or if sleeping too much increases your risk of a heart attack?

We’re invested in staying healthy and many of us are interested in reading about new research findings to help us make sense of our lifestyle choices.

But not all research is equal, and not every research finding should be interpreted in the same way. Nor do all media headlines reflect what was actually studied or found.

So how can you tell? Keep these five questions in mind when you’re reading media stories about new studies.

1. Has the research been peer reviewed?

Peer review is a process by which a study is checked by experts in the discipline to assess the study’s scientific validity.

This process involves the researcher writing up their study methods and results, and sending this to a journal. The manuscript is then usually sent to two to three experts for peer review.

If there are major flaws in a study, it’s either rejected for publication, or the researchers are made to address these flaws.

Although the peer-review process isn’t perfect, it shows a study has been subjected to scrutiny.


Read more: Peer review has some problems – but the science community is working on it


Any reported findings that haven’t been peer reviewed should be read with a degree of reservation.

2. Was the study conducted in humans?

Findings from studies conducted in animals such as mice or on cells in a lab (also called in vitro studies) represent the earliest stage of the scientific discovery process.

Regardless of how intriguing they may be, no confident claims about human health should ever be made based on these types of study alone. There is no guarantee that findings from animal or cell studies will ever be replicated in humans.

3. Are findings likely to represent a causal relationship?

For a study to have relevance to our day-to-day health, the findings need to reflect a causal relationship rather than just a correlation.

If a study showed that coffee drinking was associated with heart disease, for example, we want to know if this was because coffee actually caused heart disease or whether these to things happened to occur together.

In a number of studies that found this association, researchers subsequently found that coffee drinkers were more likely to be smokers and therefore, these results were more likely to reflect a true causal relationship between smoking and heart disease.

Just because something is common among coffee drinkers, doesn’t mean coffee caused it. Africa Studio/Shutterstock

In observational studies, where researchers observe differences in groups of people, it can sometimes be difficult to disentangle the relationship between variables.

The highest level of evidence regarding causality comes from double-blind placebo controlled randomised controlled trials (RCTs). This experimental type of study, where people are separated into groups to randomly receive either an intervention or placebo (sham treatment), is the best way we can determine if a something causes disease. However it, too, is not perfect.

Although other types of studies in humans play an important role in our understanding of health and disease, they may only highlight associations that are not indicative of causal relationships.


Read more: Clearing up confusion between correlation and causation


4. What is the size of the effect?

It’s not enough to know that an exposure (such a third cup of coffee or more than nine hours of sleep a night) causes an outcome, it’s also important to clearly understand the strength of this relationship. In other words, how much is your risk of disease going to increase if you are exposed?

If your risk of disease is reported to increase by 50% (which is a relative risk), this sounds quite frightening. However, if the original risk of disease is low, then a 50% increase in your risk may not represent a big actual increased risk of disease. A 50% increased risk of disease could mean going from a 0.1% risk of disease to your risk being 0.15%, which doesn’t sound quite so dramatic.


Read more: What you need to know to understand risk estimates


5. Is the finding corroborated by other studies?

A single study on its own, even if it’s a well-conducted randomised controlled trial, can never be considered definitive proof of a causal relationship between an exposure and disease.

As humans are complex and there are so many variables in any study, we can’t be confident we understand what is actually going on until findings are replicated in many different groups of people, using many different approaches.

Until we have a significant body of evidence that is in agreement, we have to be very careful about our interpretation of the findings from any one study.

What if these questions aren’t answered?

Switch news sites or try to see the original study. Fizkes/Shutterstock

If the media report you’re reading doesn’t answer these questions, consider changing news sites or looking at the original paper. Ideally this would be linked in the news article you’re reading, or you can search PubMed for the article using a few keywords.

The journal article’s abstract should tell you the type of study, whether it was conducted on humans and the size of the effect. If you’re not blocked by a paywall, you may be able to view the full journal article which should answer all of the questions you have about the study.


Read more: Where's the proof in science? There is none


Hassan Vally does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

Authors: Hassan Vally, Associate Professor, La Trobe University

Read more http://theconversation.com/is-this-study-legit-5-questions-to-ask-when-reading-news-stories-of-medical-research-117836

Carbon pricing: it's a proven way to reduce emissions but everyone's too scared to mention it

Opposition leader Anthony Albanese sought to claim the climate policy high ground last week with his commitment to a net-zero emissions target by 2050. But figures on Australia’s emissions fro...

Tony Wood, Program Director, Energy, Grattan Institute - avatar Tony Wood, Program Director, Energy, Grattan Institute

'I don't want anybody to see me using it': cashless welfare cards do more harm than good

ShutterstockThe Australian government touts compulsory income management as a way to stop welfare payments being spent on alcohol, drugs or gambling. The Howard government introduced the BasicsCard...

Greg Marston, Head of School, School of Social Science, The University of Queensland - avatar Greg Marston, Head of School, School of Social Science, The University of Queensland

'They wouldn't let me call anybody': women in mental health wards need better protection from sexual assault

ShutterstockMental health inpatient units should be safe and healing places. But we’ve found women staying in these units are being threatened, harassed and sexually and physically assaulted by...

Juliet Watson, Senior Lecturer, Housing and Homelessness, RMIT University - avatar Juliet Watson, Senior Lecturer, Housing and Homelessness, RMIT University

Stone tools show humans in India survived the cataclysmic Toba eruption 74,000 years ago

Christina Neudorf, Author providedAbout 74,000 years ago a volcanic eruption at what is now Lake Toba in Sumatra, Indonesia, created one of the most dramatic natural disasters of the past 2 million y...

Chris Clarkson, Professor in Archaeology, The University of Queensland - avatar Chris Clarkson, Professor in Archaeology, The University of Queensland

Five Australian universities get the bulk of philanthropic donations

Wes Mountain/The Conversation, CC BY-NDPhilanthropy is a growing source of revenue for Australian universities. It’s essential to advancing quality research, equity and learning. On average, Au...

Omer Yezdani, Director, Office of Planning and Strategic Management, Australian Catholic University - avatar Omer Yezdani, Director, Office of Planning and Strategic Management, Australian Catholic University

Older and poorer: Retirement Income Review can't ignore the changing role of home

natasaelena/ShutterstockThe assumption that retired people have minimal housing costs underpins the settings of our retirement incomes system. But the real state of housing for older Australians toda...

Emma Dawson, Honorary Fellow, School of Social and Political Sciences, University of Melbourne - avatar Emma Dawson, Honorary Fellow, School of Social and Political Sciences, University of Melbourne

The jobs market is nowhere near as good as you've heard, and it's changing us

We are continually being told that more of us are employed than ever before. Reserve Bank Governor Philip Lowe points out (correctly) that a higher proportion of us are in jobs than at any other tim...

Michael Keating, Visiting Fellow, College of Business & Economics, Australian National University - avatar Michael Keating, Visiting Fellow, College of Business & Economics, Australian National University

100 years of The Cabinet of Dr. Caligari: the film that inspired Virginia Woolf, David Bowie and Tim Burton

Decla-Bioscop AGBerlin. February 26 1920. A new silent film, The Cabinet of Dr. Caligari, is released to unsuspecting German audiences and quickly becomes a worldwide sensation. “When will I d...

Ben McCann, Associate Professor of French Studies, University of Adelaide - avatar Ben McCann, Associate Professor of French Studies, University of Adelaide

Compact Comforts: The Big Benefits of Small Cars

With the ever-increasing rise in fuel prices, more and more people are switching to compact cars in 2020. Small cars are renowned for having the lowest road tax and the best fuel economy. However, t...

News Company - avatar News Company

Sick and Tired of Your Dead End Job? Try Teaching!

Tired of the same old grind at the office? Want an opportunity to impact lives both in your community and around the world? Do you love to travel and have new experiences? Teaching English is the perfect job for you! All you need is a willingness to ...

News Company - avatar News Company

The Impact of an Aging Population in Australia

There’s an issue on the horizon that Australia needs to prepare for. The portion of elderly citizens that make up the country’s overall population is increasing, and we might not have the infrastructure in place to support this. Australians h...

News Company - avatar News Company