On its face, it was a major finding: Antimalarial drugs touted by the White House as possible COVID-19 treatments looked to be not just ineffective, but downright deadly. A study published on 22 May in The Lancet used hospital records procured by a little-known data analytics company called Surgisphere to conclude that COVID-19 patients taking chloroquine or hydroxychloroquine were more likely to show an irregular heart rhythm-a known side effect thought to be rare-and more likely to die. Within days, large randomized trials of the drugs screeched to a halt. Solidarity, the World Health Organization's (WHO's) megatrial of potential COVID-19 treatments, paused recruitment into its hydroxychloroquine arm.
But just as quickly, the results have begun to unravel-and Surgisphere, which provided patient data for two other high-profile COVID-19 papers, has come under withering online scrutiny from researchers and amateur sleuths. They have pointed out many red flags in the Lancet paper, including the astonishing number of patients and details about patient demographics and dosing that seemed implausible. "It began to stretch and stretch and stretch credulity," says Nicholas White, a malaria researcher at Mahidol University in Bangkok.
As Science went to press, The Lancet issued an Expression of Concern, noting "serious scientific questions" about its paper. Hours earlier, The New England Journal of Medicine ( NEJM) issued an Expression of Concern about a second study using Surgisphere data, published on 1 May. The paper reported that taking certain blood pressure drugs including angiotensin-converting enzyme (ACE) inhibitors didn't increase the risk of death among COVID-19 patients, as some researchers had suggested. The journal asked the authors "to provide evidence that the data are reliable."
A third study using Surgisphere data is also under fire. In an April preprint, Surgisphere founder and CEO Sapan Desai and coauthors concluded that ivermectin, an antiparasitic drug, dramatically reduced mortality in COVID-19 patients. In Latin America, where ivermectin is widely available, that study led some officials to authorize use of the drug, creating a surge in demand.
Chicago-based Surgisphere has not publicly released data underlying the studies. On 2 June, Desai told Science through a spokesperson that he was "arranging a nondisclosure agreement that will provide the authors of the NEJM paper with the data access requested by NEJM." And in a 29 May statement, Surgisphere defended the integrity of its research and said it was pursuing "an independent academic audit" of its results in The Lancet. The journal and non-Surgisphere authors also said data reviews were underway.
The episode has left leaders of halted hydroxycholoroquine trials weighing whether to restart. "The problem is, we are left with all the damage that has been done," says White, a co-investigator on a halted trial for COVID-19 prevention. It will now be hard to recruit people to key studies, he says. "The whole world thinks now that these drugs are poisonous."
Desai co-authored the Lancet paper with cardiologist Mandeep Mehra of Harvard University's Brigham and Women's Hospital (BWH), cardiologist Frank Ruschitzka of University Hospital Zürich, and cardiac surgeon Amit Patel, who listed affiliations with the University of Utah and HCA Research Institute in Nashville, Tennessee. (Mehra and Patel referred inquiries to BWH. Ruschitzka did not respond to requests for comments.) The authors describe an analysis of electronic health record data from patients already treated for COVID-19 at 671 hospitals on six continents-nearly 15,000 people prescribed chloroquine or hydroxychloroquine, alone or in combination with an antibiotic, and a control group of 81,000 other patients. After adjusting for potentially confounding factors, the researchers found the risk of dying was 9.3% for the control group versus 23.8% for those getting hydroxychloroquine alongside an antibiotic.
In a 25 May media briefing, WHO Director-General Tedros Adhanom Ghebreyesus cited the results in announcing a "temporary pause" in Solidarity's hydroxychloroquine arm. Regulators in France and the United Kingdom also instructed investigators, including White's team, to halt enrollment in trials. And Sanofi said it would temporarily stop recruiting patients to two trials of its hydroxychloroquine formulation.
Other researchers immediately took issue with the analysis. The study does not properly control for the likelihood that patients getting the experimental drugs were sicker than the controls, says Matthew Semler, a critical care physician at Vanderbilt University. And White notes anomalies in the data. Although 66% of the patients were reportedly treated in North America, the reported doses tended to be higher than the guidelines set by the U.S. Food and Drug Administration. And the authors claim to have included 4402 patients in Africa, but it seems unlikely that African hospitals would have detailed electronic health records for so many patients, White says. The study also reported more deaths in Australian hospitals than the country's official COVID-19 death statistics, The Guardian reported. On 29 May, The Lancet issued a correction saying a hospital assigned to the study's "Australasia" group should have been assigned to Asia and updating a supplemental table. "There have been no changes to the findings of the paper," it says.
The brief response left some researchers frustrated. "This was very, very annoying," says James Watson, a statistician at Mahidol who on 28 May published an open letter-now signed by more than 140 researchers-that calls for the release of Surgisphere's hospital-level data, an independent validation of the results, and publication of the peer-review comments that led to the Lancet publication. " The Lancet encourages scientific debate and will publish responses to the study, along with a response from the authors," a journal spokesperson said in a response.
On 2 June, many of the same researchers and others published an open letter to NEJM and the authors of the ACE inhibitor study, citing similar problems in that paper. It notes inconsistencies including a discrepancy between the small number of hospitals in each country that are said to have shared patient data with Surgisphere and the high proportion of those countries' confirmed COVID-19 cases included in the study.
Oddities also appear in the ivermectin study, says Carlos Chaccour of the Barcelona Institute for Global Health. There's evidence that ivermectin, the key weapon in the global campaign against river blindness, also has antiviral properties. The 6 April preprint, co-authored by Patel, Desai, and Mehra, along with David Grainger of the University of Utah, used Surgisphere data reportedly collected at 169 hospitals around the world between 1 January and 1 March. It included three patients in Africa who received ivermectin-even though only two COVID-19 cases had been reported in all of Africa by 1 March, Chaccour and two colleagues note in a recent blog post.
Chaccour says after he inquired about the discrepancy, the authors posted a second, longer version of the manuscript on 19 April, containing data collected between 1 January and 31 March. The new manuscript reported that ivermectin reduced the need for mechanical ventilation by 65% and slashed the death rate by 83%. But the revision had other problems, Chaccour and his colleagues wrote in their blog post. For example, the data shown in a figure were wildly different from those reported in the text. (Grainger also did not reply to a request for a comment.)
In response to the ivermectin study the Peruvian Ministry of Health modified its COVID-19 treatment protocol to include ivermectin (as well as hydroxychloroquine) for mild and severe cases of COVID-19; demand for the drug in Peru has surged. In Trinidad, Bolivia, the city government aimed to hand out more than 350,000 free doses of ivermectin after the country's Ministry of Health authorized its use against COVID-19.
Surgisphere's sparse online presence-the website doesn't list partner hospitals by name or identify its scientific advisory board, for example-has prompted intense skepticism. Physician and entrepreneur James Todaro of the investment fund Blocktown Capital wondered in a blog post why Surgisphere's enormous database doesn't appear to have been used in peer-reviewed research studies until May. Chaccour asks how such a tiny company-LinkedIn lists only a handful of employees-was able to reach data-sharing agreements with hundreds of hospitals around the world.
Desai's spokesperson says the company has 11 employees and has been developing its database since 2008.
The potential of hydroxychloroquine for treating COVID-19 has become a political flashpoint. French microbiologist Didier Raoult, whose own widely criticized studies suggested a benefit from the drug, derided the Lancet study in a video posted on 2 June, calling the authors "incompetent."
For scientists running randomized trials of hydroxychloroquine, an urgent question has been how to respond to the paper and the ensuing flap. A trial funded by the U.S. National Heart, Lung, and Blood Institute opted to keep running after its data and safety monitoring board (DSMB) reviewed safety data from already enrolled participants, says Semler, a co-investigator on the study. WHO's paused Solidarity trial is awaiting similar review from its DSMB, says Soumya Swaminathan, the organization's chief scientist.
The controversy is an unfortunate distraction, says Miguel Hernán, a Harvard epidemiologist and co-investigator on an ongoing trial of hydroxychloroquine in Spain and Latin America. "If you do something as inflammatory as this without a solid foundation, you are going to make a lot of people waste time trying to understand what is going on." Chaccour says both NEJM and The Lancet should have scrutinized the provenance of Surgisphere's data more closely before publishing the studies. "Here we are in the middle of a pandemic with hundreds of thousands of deaths, and the two most prestigious medical journals have failed us," he says.
Correction (4 June 2020): An earlier version of this story said one problem with the ivermectin study, according to Carlos Chaccour and his colleagues, was the strikingly low mortality rate, 21%, of COVID-19 patients who needed mechanical ventilation. It stated that a case series in the New York City area found that 88% of COVID-19 patients who needed ventilation died. However, that number was based on a paper in JAMA that was later corrected because the actual mortality for this group was much lower. The sentence has been removed from the story.
All Rights Reserved © 2024
