Retraction inaction: How the pandemic has exposed frailties in scientific publishing

In a new article published in Monash Lens this month, Cochrane Australia Co-Director Steve McDonald explores the extent and implications of retractions in scientific publishing throughout the Covid-19 pandemic.

Relatively early in the COVID-19 pandemic, three scientific papers were published about the new, highly-contagious virus that have since become notorious. All used fake or suspect patient data, and were either retracted by the prominent medical journals that published them or removed altogether.

Unfortunately, the damage was already done, despite the retractions, because the papers had already been cited by other researchers in the field, and reported on in mass media.

Even worse, the papers continued to be cited even after being retracted by the journals in question. Retraction is supposed to safeguard against error and misconduct, and should stop bogus or incorrect research from impacting scientific ideas and clinical practice, but that’s not how it played out in these high-profile cases, during a global pandemic.

A new investigation involving Monash University’s health evidence unit, Cochrane Australia – in the School of Public Health and Preventive Medicine – looked at retractions among the more than 270,000 COVID-19 papers that have been lodged online since the start of the pandemic. The 212 retracted papers investigated were cited 2697 times, a median of seven times per paper.

A quarter of these retracted papers reported clinical findings relevant to patient care – almost 90% of citations of these papers referenced the retracted paper without mentioning it had been retracted, and 80% were published after the retraction.

Co-director of Cochrane Australia, senior research fellow and PhD candidate Steve McDonald was a co-author on the paper, which used Retraction Watch and the “cited by” function in Google Scholar to track retractions and citations.

Lead author Gideon Meyerowitz-Katz is a Sydney public health expert, epidemiologist and University of Wollongong PhD candidate.

It’s the first time a link between the pandemic and retractions to clinical studies has been examined.

Three significant examples

Those big three examples of suspected fraudulent or unverifiable data comprise the bulk of the citations for the 212 retracted papers. They are:

• A paper published in The Lancet in May 2020 claiming the drug hydroxychloroquine increased the chances of death from COVID-19 at a time when the drug was largely untested. The paper was retracted after two weeks.

• A paper published in the New England Journal of Medicine in May 2020 claiming a link between heart disease and an increased risk of death from COVID-19, but with no elevated risk from certain heart drugs previously thought to be harmful. This paper was retracted after five weeks.

• A preprint paper published in April 2020 claiming a link between COVID-19 deaths and low Vitamin D, in SSRN Electronic Journal, which despite using bogus data and mysterious authors was cited by the British Medical Journal and the British Heart Foundation, and reported as a media story by The Sun (UK) and The Daily Mail. This paper was removed after two months.

“With COVID,” says McDonald, “we saw this push to get information out quickly, and with many more people doing and rapidly publishing COVID research, there’s been a spike in retractions. The number of retracted COVID publications is now more than 250. It’s true there was definitely a need to get information out there, but in doing so publishing processes were often compromised.”

Medical journals became vulnerable to haste.

“They can be in a situation where they want a splash, they want a bit of a scoop, to raise their own profile. So they’re in some ways conflicted – if they’re sent a paper they know is going to attract media attention, then they have an interest in publishing that. But at the same time, they obviously need to ensure the research itself is rigorous and reliable.”

The rise of the ‘Surgisphere’

The Lancet and the New England Journal of Medicine papers – hydroxychloroquine, heart disease – became known as the “Surgisphere” case; The Guardian revealed governments and the World Health Organisation had acted on the papers’ findings, and also uncovered Australian links.

The same group of authors, under the auspices of Surgisphere, had also earlier published a paper claiming the drug ivermectin led to fewer deaths in patients hospitalised with COVID-19 – a curious contrast to their “hydroxychloroquine-is-bad” thesis.

“The Lancet and the New England Journal of Medicine published data supposedly collected from an international registry of more than 600 hospitals,” says McDonald, “but it transpired within days that no one had heard of this registry; they weren't sure where the data had come from. And so within a very short period of time, both papers were retracted.

“This case shows that this isn’t just a problem that might affect low-impact, obscure journals. It really is across the board. What happened with COVID is that some of the processes that journals have around due diligence for peer review, and statistical checking, were expedited at a pace that wouldn't typically be the norm.”

“[The journals] published data supposedly collected from an international registry of more than 600 hospitals, but it transpired within days that no one had heard of this registry; they weren't sure where the data had come from.”

McDonald says preprint servers for academic papers also hosted dubious COVID-19 science, as the vitamin D study demonstrates. Preprint servers allow authors to make available early versions of research papers before peer review or journal publication.

Overall, the investigation found a spike in retracted papers during the 2020-21 peak in research and publication, but that most citations to these papers were not “critical”.

“In theory, when people cite retracted studies, they should be citing them in a critical way, alluding to the fact that these papers have been retracted because the research is unreliable.

“But what we found was that actually in a lot of these cases, even if the author team who cites the retracted paper were doing so long after the paper had been retracted, they weren't citing it as a retraction. They were using it as evidence that ‘this particular intervention is effective’, or ‘there's nothing wrong with that research’. So they were uncritically citing retracted papers.”

Findings should serve as a warning

He says the investigation, and the impact of the pandemic on the medical science community, should be a warning, adding that the retraction process with journals was already very “slow and clunky” before COVID-19, often taking years – “clumsy, inept and slow paced,” according to the paper, which cites one retraction process (about concussion in sports, also with an Australian link) that took 10 years.

The basic process is that peers or colleagues, or other scientists, may encounter what they think is fraudulent data, or plagiarism, or incorrect analyses, and the journal editor will be told.

“The process can take a long time for those issues to be addressed,” he says.

Sometimes, depending on the nature of the complaint, it goes back to the institutions or universities where the authors are based, “which is when it can become quite problematic”, McDonald says, “because if you’re making accusations of fraud, or issues like that, then there needs to be internal investigations within the university. And that itself can take time.”

The pandemic has exposed frailties in scientific publishing that should serve as a warning to all researchers.

“Blindly citing papers – irrespective of where they’re published – without first assessing their reliability or retraction status can falsely elevate poor and possibly fraudulent research, potentially harming the very people the research should be helping”, McDonald says.

*This article was first published on Monash Lens on 17 October 2022.