And so it continues, the saga of hydroxychloroquine for COVID-19. It’s been a while since I’ve written about this particular drug. Indeed, it’s hard to believe that it’s been well over three months since I railed against the FDA’s premature issuance of an emergency use authorization (EUA) for hydroxychloroquine to treat coronavirus and over three weeks since I noted that the FDA had finally backtracked and revoked its EUA for hydroxychloroquine. In the age of the pandemic, weeks seem like months, and months seem like years. Be that as it may, there have been…developments…that have led me to conclude that an update is in order. The saga of hydroxychloroquine and chloroquine remain, as it has from the beginning, a cautionary tale about medical science in the age of a pandemic. The saga also reminded me how, in the context of this pandemic, not only has science become politicized, but narratives often not grounded in science saturate news coverage, which brings us to this Tweet from President Trump:
Yes, this is a study that comes from my neck of the woods, Henry Ford Hospital, and it’s not just Trump who is promoting it as vindication. It’s Peter Navarro:
Tuesday morning, economist and Trump advisor Peter Navarro walked onto the White House driveway and promptly brought a political cloud back onto the FDA.
Speaking to a White House pool reporter, Navarro said that four Detroit doctors were, based on a single disputed study, filing for the FDA to again issue an emergency authorization for hydroxychloroquine, the anti-malarial pill that President Trump hyped for months as a Covid-19 treatment over the objections of his own scientists. Then, while avoiding directly calling for the FDA to OK the drug, blasted the agency. He said its decision to pull an earlier authorization “was based on bad science” and “had a tremendously negative effect” on doctors and patients.
The question in the title of this article (“Can the FDA remain independent?”) is an excellent one, but let’s take a look at what’s going on here, which is very evident based on the Trump campaign website, where the copywriter is practically frothing at the mouth with glee over this study:
A new study from the Henry Ford Health System finds that hydroxychloroquine “significantly” decreased the death rate of patients with coronavirus. It also found that patients did not suffer heart-related side effects from the drug. “The data here is clear that there was a benefit to using the drug as a treatment for sick, hospitalized patients,” said Steven Kalkanis, CEO of the Henry Ford Medical Group.
This is fantastic news. Fortunately, the Trump Administration secured a massive supply of hydroxychloroquine for the national stockpile months ago. Yet this is the same drug that the media and the Biden campaign spent weeks trying to discredit and spread fear and doubt around because President Trump dared to mention it as a potential treatment for coronavirus.
The media and the Democrats fear-mongering around hydroxychloroquine became so bad that doctors told NPR they were having a difficult time even recruiting patients to enlist in clinical trials to study it. “The fact that President Trump is touting this drug means some people are now invested in the idea that hydroxychloroquine won’t work,” one said. “We’re talking about a treatment. Who would be rooting for us not to find the therapy, for God’s sakes?” asked another.
The new study from the Henry Ford Health System should be a clear message to the media and the Democrats: stop the bizarre attempts to discredit hydroxychloroquine to satisfy your own anti-Trump agenda. It may be costing lives.
Yes, this is basically the same message that was being promoted three months ago, when I noted that science-based medicine has no chance against Donald Trump, Peter Navarro, and Dr. Oz (the last of whom was also promoting hydroxychloroquine as a panacea for COVID-19 at the time). During that time period, there was a drip-drip-drip of negative studies. True, they weren’t randomized, double-blind clinical trials, but rather observational studies (like the one being touted by Donald Trump and Peter Navarro, as you will see). Still, they were enough to lead me to conclude that it was unlikely that hydroxychloroquine would be found in randomized trials to have significant activity against COVID-19. This is starting to be confirmed with, for instance, the publication of a a randomized controlled clinical trial of the drug as post-exposure prophylaxis that was entirely negative. This was followed by two more, first, a Spanish post-exposure prophylaxis trial that was also negative. Then there was the Recovery Trial from the UK, which failed to find a benefit from hydroxychloroquine in hospitalized patients treated with the drug, leading to the revocation of the EUA.
Before that, hydroxychloroquine was being relentlessly and shamelessly touted as a highly effective treatment—cure, even—for COVID-19 its promotion by a French “brave maverick scientist” and the President. How did this come about? I’ll repeat the story, because it’s clear if you know the real story just how flimsy the evidence base was. Based on an observation of 80 patients full of confirmation bias, Chinese doctors in Wuhan noted that no patients with lupus erythematosis became ill with COVID-19 and hypothesized that the chloroquine or hydroxychloroquine that they were taking might be the reason. (These drugs are also mildly immunosuppressive, hence their use to treat autoimmune diseases.) Of course, during a pandemic, it is people who are immunosuppressed are the very people who most rigorously obey orders to practice social distancing and self-quarantine and thereby protect themselves from infection. Be that as it may, the Chinese doctors started using the antimalarial drugs, and anecdotal evidence of success was reported, leading to randomized clinical trials that were announced by the Chinese government to have been “promising.” None of this stopped China from incorporating these drugs into its recommended regimen. The World Health Organization followed suit, as did several countries, and thus was born a new de facto standard of care for COVID-19 based on, in essence, no evidence other than some in vitro evidence that the drugs inhibit replication of SARS-CoV-2, the virus that causes COVID-19, anecdotes, and incredibly weak clinical trial evidence.
The evidence was basically as weak as this study from the Henry Ford Hospital System by the Henry Ford COVID-19 Task Force, by lead author Samia Arshad and corresponding author Marcus Zervos. Here is the text. Basically, it’s a retrospective observational study. It was not randomized. It was not double blind. The exposures studied included receipt of hydroxychloroquine alone, hydroxychloroquine in combination with azithromycin, azithromycin alone, or neither, and the primary outcome measure was in-hospital mortality. The study involved 2,541 patients with a median hospitalization time of 6 days (interquartile range, IQR 4-10 days) with a median age of 64 years (IQR 53-76 years). The subjects were 51% males, 56% African-American, and the median followup was 28.5 days (IQR: 3-53 days). Multivariable Cox regression models and Kaplan-Meier survival curves were used to compare survival among treatment groups while controlling for demographics (e.g., age, gender), preexisting medical conditions (e.g. CVD, lung disease) and clinical disease severity. The authors reported that in-hospital mortality was 18.1% (95% confidence interval:16.6%-19.7%); by treatment: hydroxychloroquine + azithromycin, 157/783 (20.1% [95% CI: 17.3%-23.0%]), hydroxychloroquine alone, 162/1202 (13.5% [95% CI: 11.6%-15.5%]), azithromycin alone, 33/147 (22.4% [95% CI: 16.0%-30.1%]), and neither drug, 108/409 (26.4% [95% CI: 22.2%-31.0%]). Primary cause of mortality was respiratory failure (88%); no patient had documented torsades de pointes (a potentially fatal heart rhythm disturbance that can be caused by hydroxychloroquine). From Cox regression modeling, predictors of mortality were age >65 years (HR:2.6 [95% CI:1.9-3.3]), white race (HR:1.7 [95% CI:1.4-2.1]), CKD (HR:1.7 [95%CI:1.4-2.1]), reduced O2 saturation level on admission (HR:1.5 [95%CI:1.1-2.1]), and ventilator use during admission (HR: 2.2 [95%CI:1.4-3.3]). Hydroxychloroquine provided a 66% hazard ratio reduction, and hydroxychloroquine + azithromycin 71% compared to neither treatment (p < 0.001).
It didn’t take long for Twitter denizens to note that there were some huge potential biases in this study. First:
Yes, patients in the hydroxychloroquine and the hydroxychloroquine+azithromycin groups were twice as likely to be given a steroid. Why is this relevant? Simple. It was recently reported in a randomized clinical trial that a steroid, dexamethasone, resulted in improved survival in COVID-19 patients requiring oxygen support. At the time I wrote about it, the study hadn’t yet been published, but now it is on medRxiv.
There were also a number of other potential confounders. An accompanying editorial by Lauren J MacKenzie, Emily G. McDonald, and Steven Y.C. Tong on the Henry Ford Hospital study noted the potential confounding factor of the steroid use and discussed other problems with the trial as well. First, there was “immortal time bias,” which is a bias that can be introduced when some time-dependent variables are not modeled:
First, the precision of the results is impacted by immortal time bias, since several time-dependent covariates were not modelled in this manner. Fortunately, since the average time to receipt of treatment was only 1 day, this bias may be small; nonetheless, it favors treatment and should be taken into consideration.
Then there were a number of other factors not taken into account:
Second, there is an important potential for residual confounding because there are a number of prognostic factors (e.g. frailty, residence in long term care, or “do not resuscitate” orders), potentially important markers of disease severity (e.g. ferritin, C-reactive protein (Zeng et al., 2020), troponins (Vrsalovic and Vrsalovic Presecki, 2020), and D-dimer (Zhang et al., 2020), and co-administration of potentially beneficial therapies (e.g. anticoagulants (Paranjpe et al., 2020) that were not included in the analysis.
This third confounder, however, is most likely the main bias that could well have produced a false positive result:
Third, confounding by severity or indication (Kyriacou and Lewis, 2016) is likely. While there was a hospital treatment protocol in place, unmeasured clinical factors likely influenced the decision not to treat 16.1% of patients, in a center where 78% received treatment. These factors are often difficult to capture in an observational study. Were the decision to withhold treatment related to poor prognosis (e.g. palliative intent), it stands to reason that patients receiving neither hydroxychloroquine nor azithromycin would have the highest mortality. Indeed, the non-treated group had an overall mortality that was higher than the rate of admission to the ICU (26.4% vs. 15.2%), suggesting that many patients were not considered appropriate for critical care. Such being the case, their care may have differed in other substantive ways that was also associated with death (e.g. terminal illness or advanced directives limiting invasive care). In the hydroxychloroquine treatment groups, the inverse was true with mortality lower than the rate of admission to the ICU (16.1% vs. 26.9%). While a propensity score analysis might further account for some differences between treatment groups, this approach is still limited to the information available in the dataset.
In this unrandomized study, confounding by severity or indication almost certainly at least significantly contributed to the observed result. Particularly telling is the observation that the mortality was higher than the rate of admission to the ICU; that almost certainly meant that there were a lot of patients who were considered so unlikely to survive that they weren’t admitted to the ICU and instead underwent only palliative care.
Fourth, the chronological time point during the course of the pandemic whereby patients were managed was not included in the study. As the Henry Ford Health System became more experienced in treating patients with COVID-19, survival may have improved, regardless of the use of specific therapies. Hospital-specific guidelines regarding COVID-19 screening eligibility, as well as the availability of COVID-19 testing may have also changed over time, introducing additional chronological bias.
In other words, this study has a lot of issues, the sorts of issues that can plague retrospective studies of this sort, particularly retrospective studies in the fast-changing treatment milieu of a pandemic, where treatment protocols are evolving at a record pace, sometimes based on solid evidence but more often based on anecdote and clinical experience as more objective data are pending. I was also rather amused by this passage in the accompanying editorial:
Overall, the authors should be commended for rapidly compiling and analyzing data from a large cohort of COVID-19 patients. Clinicians worldwide ought to be acknowledged for their best effort to care for patients in uncertain times and in the absence of proven therapies. It is, however, very sobering to note that the number of patients in this single observational study would have made a substantive contribution to any randomized controlled trial. While all healthcare providers feel a clinical imperative to offer patients treatment, there was little evidence to justify a hydroxychloroquine protocol at the outset of the pandemic. It is a failing of healthcare systems and research infrastructure that the protocolization of unproven therapies is exponentially easier to execute than participation in pragmatic randomized controlled trials. Moving forward, we encourage academic centers to commit to participating in the necessary clinical trials that will establish high quality evidence for safe and effective therapies in the shortest possible time.
This is very similar to what I’ve been saying all along and more than a bit sarcastic. I like it. Yes, hydroxychloroquine became the de facto standard of care for COVID-19 patients based on a questionable anecdote about patients with rheumatoid arthritis taking the drug in Wuhan supposedly never becoming ill with coronavirus that led to clinical trials whose results were not published but reported verbally and led to the Chinese government to include hydroxychloroquine as a recommended treatment for coronavirus. This then led other governments, as the pandemic progressed, to take on this recommendation, thinking that the Chinese must have known what they were talking about. Then a “Brave Maverick” French doctor named Didier Raoult took up the banner and touted a combination of hydroxychloroquine and azithromycin as, in essence, a cure-all for COVID-19, leading other “brave maverick doctors” to do the same, leading to some tech bros to start touting it, which then brought it to Donald Trump’s attention. The rest, unfortunately, is history. The authors are exactly right. There was never any evidence sufficiently compelling to justify using hydroxychloroquine for COVID-19 outside of the auspices of a clinical trial. Unfortunately, that hasn’t stopped researchers from basically going hog wild on hydroxychloroquine:
The analysis, conducted in partnership with Applied XL, a Newlab Venture Studio company, found that one in every six trials was designed to study the malaria drugs hydroxychloroquine or chloroquine, which have been shown to have no benefit in hospitalized patients.
“If the goal was to optimize the likelihood of figuring out the best treatment options, the system is off course,” said Robert Califf, the head of clinical policy and strategy at Verily Life Sciences and Google Health and a former commissioner of the Food and Drug Administration. The findings show, he said, that too often studies are too small to answer questions, lack real control groups, and put too much emphasis on a few potential treatments, as occurred with hydroxychloroquine.
Indeed, the analysis found many of the studies are so small — 39% are enrolling or plan to enroll fewer than 100 patients — that they are unlikely to yield clear results. About 38% of the studies have not actually begun enrolling patients.
The amount of resources and effort expended on hydroxychloroquine based on so little data and prior plausibility has been staggering. One wonders what other avenues of research far more likely to result in effective treatments have been delayed or foregone because of all the attention lavished on hydroxychloroquine.
The hydroxychloroquine cultists very much remind me of acupuncturists. Just as acupuncture believers do when rigorous randomized, double-blind, sham acupuncture-controlled clinical trials fail to show an effect of acupuncture greater than placebo, hydroxychloroquine cultists retreat to studies with less rigor when randomized controlled trials don’t show what they want. In this case, they’re ignoring the at least two negative randomized trials, one for prophylaxis and one for hospitalized patients, that failed to find a beneficial effect due to hydroxychloroquine and latching on to this bias-prone retrospective study from Henry Ford Hospital as “proof” that Donald Trump was right all along. Unfortunately, once we have randomized clinical trial results, studies like the Henry Ford Hospital study really don’t tell us much of anything worthwhile any more.