Debate on Hydroxychloroquine’s efficacy: Why we need to be careful about research papers

Researchers across the world are working at a breakneck speed to find evidence related to COVID-19 therapies, vaccines and diagnostics. They are trying to find what causes it and how the disease is evolving.

In the past week, medical journal Lancet retracted a peer-reviewed observational study on Hydroxychloroquine (HCQ), which led WHO and many other countries to halt clinical trials midway. In a statement last week, authors of the study Prof Mandeep Mehra, Zurich cardiologist Frank Ruschitzka and Dr Amit Patel said their reviewers were not able to conduct an independent and private peer review, suggesting that they could not vouch for the accuracy of the data. The Lancet study relied on a database from a US company, Surgisphere.

The study published on May 22 concluded that HCQ had no benefit in the treatment of COVID-19 patients; in fact, it causes more harm in the form of heart diseases and death. The Lancet study is the largest so far and claims to have analysed data of 96,000 people from 671 hospitals across six continents. The observational study compared the patients who had received the drug with those who did not. The study is a peer-reviewed one, which adds a safety net compared to preprint studies.

To be sure, the retraction of the Lancet study and the restart of HCQ trial by WHO do not mean that the debate on HCQ's efficacy is settled. This debate will have to be settled by robust randomised clinical studies that are free of selection bias and confounding.

How credible is the research?

The HCQ study published by Lancet can be an aberration, but it raises questions on how scientific studies are done during the COVID-19 pandemic.

Researchers across the world are working at a breakneck speed to find evidence related to COVID-19 therapies, vaccines and diagnostics. They are trying to find what causes it and how the disease is evolving.

While there is pressure on researchers to find answers, the problem with science is that it is a slow and gradual process. At the end of the day, science is a human enterprise and is prone to wrong interpretations and incorrect results. Eventually, all this adds to our better understanding.

Traditionally, scientific papers are peer-reviewed by independent researchers with relevant expertise before they are published in journals. The independent reviewers, who wrote critiques of the study, pointed out any deficiencies in methodology of the study and sought some additional data. This used to take months before a study is published in a scientific journal. Despite this rigour, there is no guarantee that the study is perfect.

The emergence of pre-print servers like bioRxiv and medRxiv is allowing researchers to publish papers without peer review. In fact, there has been a flood of these studies.

The Nature has quoted saying that there are around 3,000 papers published on bioRxiv and medRxiv related to coronavirus research. Some of these studies are just computational, or ecological studies. Many are observational studies prone to selection bias and confounding.

These studies are picked up by the media without mentioning the caveats stated in the study or pointing out the limitations of study methodology. For example the initial study on HCQ for COVID-19 is a preprint, even the one linking BCG vaccine to spread of COVID-19 is a preprint. There are other sensational studies like smokers having reduced risk of acquiring COVID-19 and many others.

Governments and their leadership who are desperate to find quick fixes, have made decisions based on assumptions made by these studies. Social media also amplifies sensationalism.

It may be time for us to take the studies with a pinch of salt.