Did you last see a scientific paper? I mean a physical one. An older academic in my previous university department used to keep his journals in boxes. On entering his office, you would be greeted by a wall of his roosters and packets of various journals. It was odd, but there was a way to find the paper you were looking for if you didn't keep your journals organised.
The time has passed for cornflakes boxes. Since the very first scientific journal was inaugurated in 1665, the majority of research has been submitted, reviewed and read online. It was an essential part of the story of Covid-19 and was often devoured on social media. Hard copies of journals are seen as curiosities or not at all.
The system for how we publish science is largely unchanged despite the internet. We still have scientific papers, we still send them off to peer reviewers, and we still have editors who give the ultimate thumbs up or down as to whether a paper is published in their journal.
There are big problems with this system. The issue of publication bias is related to the fact that reviewers and editors are more likely to give a scientific paper a good write-up and publish it in their journal if it reports positive or exciting results. Scientists go to great lengths to hype up their studies, lean on their analyses so they produce better results, and sometimes even commit fraud in order to impress those important people. This changes our view of what happened.
Is it time to stop worrying about stress?
The way journals work could be changed. Maybe the decision to publish could be made based on the methodology of the study, rather than on the results. Maybe scientists could just publish all their research, and journals would decide which results get out into the world. Maybe we could get rid of the scientific papers altogether.
The publication section of a scientist's CV is important because they want to have more papers published under their name. It might sound crazy to suggest we could do without them. The problem is that obsession. It's difficult to get the contents of a peer-reviewed paper right because of the sacred status of a published paper.
The reality of scientific research is messy. Weird, unexpected numbers are thrown up in studies. A traditional paper makes you dumb things down. If you are working towards a big, milestone goal of a published paper, the temptation is ever-present to file away a few of the jagged edges of your results to help tell a better story. Many scientists admit in surveys that they make their results into attractive looking papers, but distort the science along the way.
Some fields of science are already using online notebooks instead of journals – living documents instead of living fossils
Consider changes. There are errors in scientific papers. More than 50% of the psychology papers ran through had one specific statistical error, and more than 15% had an error serious enough to overturn the results. You have to write in to the journal, get the attention of the busy editor, and get them to issue a new, short paper that formally details the correction in a paper. Many scientists are ignored by journals when they request correction. It's too much hassle to fix the errors in the scientific literature that haven't been corrected.
Consider the data. Sharing the raw data that formed the basis of the paper was not easy back in the day. The data can be uploaded to an open repository in a few clicks. We act as if we live in the past because papers don't always have the data attached.
The solution to all these problems is the same, use the internet. We can change the paper into a website that will show the results of the study. Everyone can see the full process from data to analysis to write-up, and anyone can reproduce the full analysis and check they get the same numbers.
This would be an improvement over the current situation where the analysis and writing of papers goes on in private, with scientists having the option to make their results public or not. It's how science really is if you throw sunlight on the whole process. It would be easy to add in new data if you were running a long-term study on the climate or child development.
Sign up to our Inside Saturday newsletter for an exclusive behind-the-scenes look at the making of the magazine's biggest features, as well as a list of our weekly highlights.
There are obstacles to big changes. It is easy to write a Word document with your results and send it in to a journal, as we do now, but it is harder to make a notebook website that weaves together the data, code and interpretation. How would peer review work in this scenario? It has been suggested that scientists could hire red teams to dig into their notebook sites and test their findings. Who would pay for the system is up for debate.
We have made amazing progress in many areas of science, but we are still stuck with the flawed model of publishing research. The paper's name evokes back to a simpler time. Some fields of science are using online notebooks instead of journals, and living documents instead of fossils. It is time for the rest of science to follow suit.
Naomi Oreskes wrote Why Trust Science.
Chris Chambers wrote The Seven Deadly Sins of Psychology.
Richard Harris wrote a book called "How Sloppy Science creates Worthless Cures, Crushes Hope, and Wastes Billions."
All Rights Reserved © 2024
