.

  • Written by Hassan Vally, Associate Professor, La Trobe University
It can be difficult to work out whether you should believe a study's reported findings. GaudiLab/Shutterstock

Who doesn’t want to know if drinking that second or third cup of coffee a day will improve your memory, or if sleeping too much increases your risk of a heart attack?

We’re invested in staying healthy and many of us are interested in reading about new research findings to help us make sense of our lifestyle choices.

But not all research is equal, and not every research finding should be interpreted in the same way. Nor do all media headlines reflect what was actually studied or found.

So how can you tell? Keep these five questions in mind when you’re reading media stories about new studies.

1. Has the research been peer reviewed?

Peer review is a process by which a study is checked by experts in the discipline to assess the study’s scientific validity.

This process involves the researcher writing up their study methods and results, and sending this to a journal. The manuscript is then usually sent to two to three experts for peer review.

If there are major flaws in a study, it’s either rejected for publication, or the researchers are made to address these flaws.

Although the peer-review process isn’t perfect, it shows a study has been subjected to scrutiny.


Read more: Peer review has some problems – but the science community is working on it


Any reported findings that haven’t been peer reviewed should be read with a degree of reservation.

2. Was the study conducted in humans?

Findings from studies conducted in animals such as mice or on cells in a lab (also called in vitro studies) represent the earliest stage of the scientific discovery process.

Regardless of how intriguing they may be, no confident claims about human health should ever be made based on these types of study alone. There is no guarantee that findings from animal or cell studies will ever be replicated in humans.

3. Are findings likely to represent a causal relationship?

For a study to have relevance to our day-to-day health, the findings need to reflect a causal relationship rather than just a correlation.

If a study showed that coffee drinking was associated with heart disease, for example, we want to know if this was because coffee actually caused heart disease or whether these to things happened to occur together.

In a number of studies that found this association, researchers subsequently found that coffee drinkers were more likely to be smokers and therefore, these results were more likely to reflect a true causal relationship between smoking and heart disease.

Just because something is common among coffee drinkers, doesn’t mean coffee caused it. Africa Studio/Shutterstock

In observational studies, where researchers observe differences in groups of people, it can sometimes be difficult to disentangle the relationship between variables.

The highest level of evidence regarding causality comes from double-blind placebo controlled randomised controlled trials (RCTs). This experimental type of study, where people are separated into groups to randomly receive either an intervention or placebo (sham treatment), is the best way we can determine if a something causes disease. However it, too, is not perfect.

Although other types of studies in humans play an important role in our understanding of health and disease, they may only highlight associations that are not indicative of causal relationships.


Read more: Clearing up confusion between correlation and causation


4. What is the size of the effect?

It’s not enough to know that an exposure (such a third cup of coffee or more than nine hours of sleep a night) causes an outcome, it’s also important to clearly understand the strength of this relationship. In other words, how much is your risk of disease going to increase if you are exposed?

If your risk of disease is reported to increase by 50% (which is a relative risk), this sounds quite frightening. However, if the original risk of disease is low, then a 50% increase in your risk may not represent a big actual increased risk of disease. A 50% increased risk of disease could mean going from a 0.1% risk of disease to your risk being 0.15%, which doesn’t sound quite so dramatic.


Read more: What you need to know to understand risk estimates


5. Is the finding corroborated by other studies?

A single study on its own, even if it’s a well-conducted randomised controlled trial, can never be considered definitive proof of a causal relationship between an exposure and disease.

As humans are complex and there are so many variables in any study, we can’t be confident we understand what is actually going on until findings are replicated in many different groups of people, using many different approaches.

Until we have a significant body of evidence that is in agreement, we have to be very careful about our interpretation of the findings from any one study.

What if these questions aren’t answered?

Switch news sites or try to see the original study. Fizkes/Shutterstock

If the media report you’re reading doesn’t answer these questions, consider changing news sites or looking at the original paper. Ideally this would be linked in the news article you’re reading, or you can search PubMed for the article using a few keywords.

The journal article’s abstract should tell you the type of study, whether it was conducted on humans and the size of the effect. If you’re not blocked by a paywall, you may be able to view the full journal article which should answer all of the questions you have about the study.


Read more: Where's the proof in science? There is none


Hassan Vally does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.

Authors: Hassan Vally, Associate Professor, La Trobe University

Read more http://theconversation.com/is-this-study-legit-5-questions-to-ask-when-reading-news-stories-of-medical-research-117836

Is your horse normal? Now there’s an app for that

Vet: are you happy? Horse: neigh. evilgurl/Flickr, CC BY-NC-SASince ancient times, horse behaviour, and the bond between horses and humans, has been a source of intrigue and fascination. The horse-l...

Paul McGreevy, Professor of Animal Behaviour and Animal Welfare Science, University of Sydney - avatar Paul McGreevy, Professor of Animal Behaviour and Animal Welfare Science, University of Sydney

Small histories: a road trip reveals local museums stuck in a rut

Berry, and other tourist towns, are out of step with modern museum curation which is trying to include Aboriginal communities and their stories. ShutterstockAboriginal and Torres Strait Islander read...

Jen Saunders, Phd candidate, University of Wollongong - avatar Jen Saunders, Phd candidate, University of Wollongong

Curious Kids: how are stars made?

Stars come into existence because of a powerful force of nature called gravity. ESA/Hubble & NASA, Acknowledgement: Judy SchmidtIf you have a question you’d like an expert to answer, send it...

Orsola De Marco, Astrophysicist , Macquarie University - avatar Orsola De Marco, Astrophysicist , Macquarie University

What is perimenopause and how does it affect women's health in midlife?

Perimenopause lasts months for some women, and years for others. from www.shutterstock.comAll women know to expect the time in life when their periods finish and they reach menopause. Many might even...

Gita Mishra, Professor of Life Course Epidemiology, Faculty of Medicine, The University of Queensland - avatar Gita Mishra, Professor of Life Course Epidemiology, Faculty of Medicine, The University of Queensland

Vital signs. Our compulsory super system is broken. We ought to axe it, or completely reform it

We're taking money from people, letting it fall through the cracks, and spending no less than we were on pensions. ShutterstockThe just-announced inquiry into Australia’s retirement income syste...

Richard Holden, Professor of Economics, UNSW - avatar Richard Holden, Professor of Economics, UNSW

Might consciousness and free will be the aces up our sleeves when it comes to competing with robots?

Our advantage lies in incommensurables, and it'll grow in importance. Franck V. on UnsplashThe rise of artificial intelligence has led to widespread concern about the role of humans in the workplaces ...

Allan McCay, Law Lecturer, University of Sydney - avatar Allan McCay, Law Lecturer, University of Sydney

Should I stay or should I go: how 'city girls' can learn to feel at home in the country

Shutterstock/The ConversationA move to the country is often presented in popular culture as an idyllic life, a place where you can escape the pressures of the city. It’s in television shows su...

Rachael Wallis, Lecturer and Honorary Research Fellow, University of Southern Queensland - avatar Rachael Wallis, Lecturer and Honorary Research Fellow, University of Southern Queensland

Grattan on Friday: Storm clouds avoid the bush, darken over the economy

National Farmers' Federation president Fiona Simson says she doesn't think the government has a drought policy. ShutterstockGovernment sources insist shock jock Alan Jones didn’t drive Thursday&...

Michelle Grattan, Professorial Fellow, University of Canberra - avatar Michelle Grattan, Professorial Fellow, University of Canberra

Julianne Schultz appointed chair of The Conversation

Professor Julianne Schultz AM FAMA has been appointed chair of The Conversation Media Group, following the retirement of Harrison Young. Since becoming chairman in April 2017, Harrison has improved ...

Misha Ketchell, Editor & Executive Director, The Conversation - avatar Misha Ketchell, Editor & Executive Director, The Conversation

Cats are not scared off by dingoes. We must find another way to protect native animals

New research suggests feral cats can probably outsmart dingoes. Wikimedia/AAPFeral cats are wreaking havoc on our native wildlife, eating more than a billion animals across Australia every year. But ...

Bronwyn Fancourt, Adjunct Research Fellow, University of New England - avatar Bronwyn Fancourt, Adjunct Research Fellow, University of New England

Curious Kids: does chewing gum stay inside you for years?

Swallowing a lot of gum can cause it to stick together or stick to food in your gut. www.shuttershock.com, CC BYIf you have a question you’d like an expert to answer, send it to curiouskids@th...

Jerry Zhou, Lecturer, School of Medicine, Western Sydney University - avatar Jerry Zhou, Lecturer, School of Medicine, Western Sydney University

Don't believe your ears: 'enhancing' forensic audio can mislead juries in criminal trials

Audio used as evidence in criminal trials can often be unreliable.  Many criminal trials feature forensic evidence in the form of audio recordings, typically from bugging houses or cars, or intercep...

Helen Fraser, Adjunct Associate Professor, University of New England - avatar Helen Fraser, Adjunct Associate Professor, University of New England

The case for 'inclusion riders' in creative industries: what Australian discrimination law says about quotas

In March last year, Frances McDormand won the Academy Award for Best Actress. In her acceptance speech, she drew attention to the female nominees in the room and left them with two final words: &ldq...

Liam Elphick, Adjunct Research Fellow, Law School, University of Western Australia - avatar Liam Elphick, Adjunct Research Fellow, Law School, University of Western Australia

The Portal review: can meditation change the world?

The Portal uses individual stories of meditative transformation to suggest a bigger change is possible. SuppliedThe Portal follows six individuals who undergo a personal transformation from trauma an...

Peggy Kern, Associate professor, University of Melbourne - avatar Peggy Kern, Associate professor, University of Melbourne

Why white married women are more likely to vote for conservative parties

Women’s perceptions of 'gender linked fate' were contingent on two dimensions: their race and their marital status. ShutterstockThe polls were wrong in the last US and Australian federal electi...

Leah Ruppanner, Associate Professor in Sociology and Co-Director of The Policy Lab, University of Melbourne - avatar Leah Ruppanner, Associate Professor in Sociology and Co-Director of The Policy Lab, University of Melbourne

Thoughts and prayers: miracles, Christianity and praying for rain

In a speech in Albury last month, Prime Minister Scott Morrison told his audience that he was praying for rain in drought-affected areas. “I pray for that rain everywhere else around the count...

Philip C. Almond, Emeritus Professor in the History of Religious Thought, The University of Queensland - avatar Philip C. Almond, Emeritus Professor in the History of Religious Thought, The University of Queensland

Prime Minister's science prizes awarded for algebra expertise, anti-cancer research and excellence in science teaching

Cheryl Praeger was awarded the 2019 Prime Minister’s Prizes for Science. She has spent more than four decades inspiring a love for maths in others, and has created a vast body of academic work i...

Michael Hopkin, Science + Technology Editor, The Conversation - avatar Michael Hopkin, Science + Technology Editor, The Conversation

Curious Kids: is it OK to listen to music while studying?

Does music usually put you in a better mood? That might help you try a little bit harder and stick with challenging tasks. Shutterstock I am in year 11 and I like to listen to music when I am studyin...

Timothy Byron, Lecturer in Psychology, University of Wollongong - avatar Timothy Byron, Lecturer in Psychology, University of Wollongong

A requiem for Reformasi as Joko Widodo unravels Indonesia's democratic legacy

It’s deeply ironic that Indonesia’s third president, BJ Habibie, died on September 11 – less than a week before the national legislature passed a law that gutted the highly-regarded ...

Tim Lindsey, Malcolm Smith Professor of Asian Law and Director of the Centre for Indonesian Law, Islam and Society, University of Melbourne - avatar Tim Lindsey, Malcolm Smith Professor of Asian Law and Director of the Centre for Indonesian Law, Islam and Society, University of Melbourne

Sick and Tired of Your Dead End Job? Try Teaching!

Tired of the same old grind at the office? Want an opportunity to impact lives both in your community and around the world? Do you love to travel and have new experiences? Teaching English is the perfect job for you! All you need is a willingness to ...

News Company - avatar News Company

The Impact of an Aging Population in Australia

There’s an issue on the horizon that Australia needs to prepare for. The portion of elderly citizens that make up the country’s overall population is increasing, and we might not have the infrastructure in place to support this. Australians h...

News Company - avatar News Company

LifeStyle

Questions to ask yourself before buying your watch

There are more and more watches on the market. And more and more brands are trying to seduce consu...

How to Thoroughly Prepare Children for a Professional Photoshoot at a Studio

Children are only young for a moment, which is why, for a lot of parents, it's essential to take a...

What to Expect at the University of Florida Tour

The University of Florida is a dream college for most aspiring students. Not only because of its p...

7 Professions that Will Be Huge in the Next Decade

In order to embark on a career path that requires a lot of training and experience, you might ne...