I likened ivermectin to acupuncture a couple of weeks ago. The approach to evidence is the reason why the comparison came to me. As more and more high quality evidence from randomized clinical trials has failed to find a therapeutic effect from using ivermectin to treat COVID-19, its advocates point to positive studies that are less rigorous. More and more high quality studies with appropriate sham acupuncture placebo groups fail to find a benefit for treating anything, which is exactly what acupuncture advocates have been doing as more and more high quality studies with appropriate sham acupuncture placebo groups fail to find a benefit for treating anything. They have been citing lower quality studies, which may not be blinded, or even randomized. Citing pragmatic studies is putting the cart before the horse. Outside of clinical trials and all the rigid protocols and selection criteria, a situation, pragmatic studies are intended to see how well a treatment that has been shown to work in high quality randomized controlled clinical trials works. When randomized controlled trials fail to show a benefit when ivermectin is used to treat COVID-19, ivermectin advocates use the same sorts of excuses, such as claiming that medicine is biased and there is a double standard. The double standard doesn't favor what the advocates of ivermectin think it does.
I mentioned an RCT of ivermectin for COVID-19 in my post two weeks ago, but I didn't discuss it much because it hadn't been published and had only been reported on by the Wall Street Journal. The study was published in the New England Journal of Medicine last Thursday, and it was a resoundingly negative trial, without even a hint of efficacy. Was it a good trial? Absolutely not. There is no trial. It was large and well-designed and showed no effect from the early treatment of COVID-19 with ivermectin. It was one more drop in the negative RCTs for ivermectin. The most famous and largest RCTs of ivermectin for COVID-19 were either incompetently carried out or even fraudulent, and as I mentioned, ivermectin for COVID-19 is looking increasingly like acupuncture.
Why bring this up again after you wrote about it? The case of ivermectin is a very good example of the utility of evidence-based medicine, and it occurred to me that the purpose of this blog is to highlight the benefits of evidence-based medicine.
Let's go back to the beginning to explain the differences between the two. I will explain how the differences apply. The difference is that the prior probability of a treatment working in evaluating clinical evidence is taken into account.
The very first post ever on this website was written by Dr. Steven Novella.
All of science describes the same reality, and therefore it must (if it is functioning properly) all be mutually compatible. Collectively, science builds one cumulative model of the natural world. This means we can make rational judgments about what is likely to be true based upon what is already well established. This does not necessarily equate to rejecting new ideas out-of-hand, but rather to adjusting the threshold of evidence required to establish a new claim based upon the prior scientific plausibility of the new claim. Failure to do so leads to conclusions and recommendations that are not reliable, and therefore medical practices that are not reliably safe and effective.
This is why the authors of this blog strongly advocate for science-based medicine – the use of the best scientific evidence available, in the light of our cumulative scientific knowledge from all relevant disciplines, in evaluating health claims, practices, and products.
What did Steve mean by that? It is an estimate of how likely a proposed treatment is to produce a positive result when tested in a clinical study. It is an estimate of the prior probability of a null hypothesis being rejected in a study and a statistically significant difference between the treatment group and placebo group indicating that the treatment works. I am writing for a lay audience and realize that this is boiling it down a bit.
Clinical trials can have a lot of noise even when they are designed and carried out perfectly. The prior plausibility of a treatment is important in interpreting clinical trial results. I will explain more in the next section. In 2008 the original intent was to look at the evidence for treatments advocated as part ofcomplementary and alternative medicine. I have argued that the newer names for CAM are nothing more than a re-enactment of a quackery.
Many examples of alternative medicine have been discussed over the years. You might think that our favorite is homeopathy. The reason is simple. The Law of Infinitesimals states that the stronger a remedy gets, the more likely it is that the original remedy is not left. It is arguably the best example of a treatment with no prior plausibility, given that most of the remedies are either water or other diluent. For any disease or medical condition, several well-established laws of physics and chemistry would have to be spectacularly wrong. It is true that there are other alternative medicine treatments that have the same level of implausibility. I use it so often because it is an excellent teaching example, because it is so common and ubiquitous. I have noticed that a lot of people don't know what homeopathy is. Many people think that homeopathy is just another form of medicine. It is an excellent way to test placebo vs. placebo in clinical trials, because many homeopathy remedies are just water used to make sugar pills.
How do you consider prior plausibility in RCTs?
We've talked about the differences between the two methods, which use frequentist statistics and Bayesian reasoning.
The only valid method of investigation in medicine is the RCT, and I have long complained about obscene worship of it. It was a term that I first learned about when I was taught to counter misinformation about the H1N1 flu vaccine. There is a lot of methodolatry in EBM, and it is part of the reason why treatments like acupuncture are very, very low. I would like to take this opportunity to quote the co- founder and former regular of SBM, Dr. Kimball Atwood, who discussed why the two organizations should be synonymous. It is intended to complete it, or at least to fill in its blind spot.
He argued elsewhere.
That discussion made the point that EBM favors equivocal clinical trial data over basic science, even if the latter is both firmly established and refutes the clinical claim. It suggested that this failure in calculus is not an indictment of EBM’s originators, but rather was an understandable lapse on their part: it never occurred to them, even as recently as 1990, that EBM would soon be asked to judge contests pitting low powered, bias-prone clinical investigations and reviews against facts of nature elucidated by voluminous and rigorous experimentation. Thus although EBM correctly recognizes that basic science is an insufficient basis for determining the safety and effectiveness of a new medical treatment, it overlooks its necessary place in that exercise.
I hope you can see where I'm going with this. In the post from which I drew that quote, Dr. Kimball explained more about what he was talking about. It also discusses how an estimate of prior probability can affect the result of a clinical trial.
I think I'll put a table here that we like to cite.
The calculation was made for the probability based on prior probability.
The table shows how prior probability affects the probability of a given p-values. There is one thing to notice. The lower the prior probability, the less likely it is that a result will be for a standard p-value less than or equal to 0.05. In the case of a treatment with a very low prior probability/plausibility, even results with fairly low p-values bear serious questioning. The mathematical formulas and reasoning used to derive these numbers are in the reference for those who are more knowledgeable.
The situation is worse for something like homeopathy, in which the prior probability based on its scientific impossibility is zero, than it is for the table above. In order to address a common criticism of our argument that RCTs of highly improbable/implausible treatments are akin to testing whether magic works as medicine, I will quote Steve Novella and myself from a paper we co-authored.
It should also be noted that ‘biologically plausible’ does not mean ‘knowing the exact mechanism’. What it does mean is that the mechanism should not be so scientifically implausible as to be reasonably considered impossible. In other words, the mechanism should not violate laws and theories in science that rest on far sturdier and longerestablished foundations than imperfect, bias-prone clinical trials. For example, homeopathy violates multiple laws of physics with its claims that dilution can make a homeopathic remedy stronger and that water can retain the ‘memory’ of substances with which it has been in contact before [9]. Thus, treatments like homeopathy should be dismissed as ineffective on basic scientific grounds alone. That is why we propose the term science-based medicine (SBM) as opposed to evidence-based medicine (EBM). SBM restores basic science considerations to EBM and is what EBM should be.
I'm assuming that most of the readers of the SBM accept that basic science is all that is needed to reject the idea of energy healing as a treatment. I'm assuming that most of the readers will accept that, for example, acupuncture is still incredibly implausible, with a prior probability of less than 1%. I would say less than that. Basic science can be enough to reject a proposed therapy if it is so implausible as to be impossible and not worth testing in RCTs. The treatments that appeared promising before being tested in humans failed.
What about ivermectin? That doesn't fall into the same category as other therapies. It is a drug that is very effective against diseases caused by roundworm. The discoverers won the prize for that indication. Is it possible that ivermectin could work against a viral illness? The coronaviruses that cause COVID-19 were the subject of a proposed mechanism for its antiviral activity. The mechanism was based on studies of cell culture.
These are reasonable considerations. In the last two sections of this post, I hope to convince you that the prior probability for ivermectin was always very, very low.
How did the idea of ivermectin being an effective treatment for COVID-19 come about? The idea of using hydroxychloroquine as a mild immunosuppressive drug to treat autoimmune diseases took hold early in the pandemic. In early 2020 Chinese physicians reported that a group of 80 patients who were taking hydroxychloroquine did not catch COVID-19. During an epidemic, the patients most likely to follow the recommendations of public health authorities are those who are immune suppressed. The Chinese government published an expert consensus recommendation for patients with COVID-19 in February, based on anecdotal reports and small clinical trials. A number of nations followed suit. President Donald Trump and Dr. Mehmet Oz joined the French scientist who started flogging hydroxychloroquine as a cure for COVID-19. Unfortunately, the rest is history. By the summer of 2020, it was clear that hydroxychloroquine was not effective against COVID-19.
The idea that ivermectin could be used to treat COVID-19 is not new. I discussed this evidence several months ago, when a conspiracy theory was being spread that Paxlovid was a new drug from Pfizer. I will recap briefly.
It has been known for a long time that ivermectin can prevent coronaviruses from replicating. The nucleus is the location of the transport of the proteins. In the case of SARs-CoV-2 infections, the nucleus is important for the completion of its lifecycle. This is the thing. It takes high concentrations of drugs to prevent the replication of HIV and the Dengue virus. The need for a high concentration of drug to inhibit the importin that was needed for SARS-CoV-2 replication was examined in the original Australian paper.
What is the problem? This review summarizes basic pharmacology.
As noted, the activity of ivermectin in cell culture has not reproduced in mouse infection models against many of the viruses and has not been clinically proven either, in spite of ivermectin being available globally. This is likely related to the pharmacokinetics and therapeutic safety window for ivermectin. The blood levels of ivermectin at safe therapeutic doses are in the 20–80 ng/ml range [44], while the activity against SARS-CoV2 in cell culture is in the microgram range. Ivermectin is administered orally or topically. If safe formulations or analogs can be derived that can be administered to achieve therapeutic concentrations, ivermectin could be useful as a broad-spectrum antiviral agent.
The IC50 is the concentration that produces 50% of maximal inhibition of a process. This is a serious problem, even if the paper proposes ivermectin as a treatment.
A dose of 12 mg twice daily alone or in combination with other therapy for 5–7 days has been proposed as a safe therapeutic option for mild, moderate or severe cases of Covid-19 infection.10 The time to reach maximum plasma concentration of 20–50 ng/ml, after a dose of 6 or 12 mg, respectively is approximately 4 h.
This is the reason that ivermectin doesn't work against COVID-19. It requires a concentration that is 66- to 199-fold higher than in the blood. The review concluded that an ivermectin analogue that is either more active or can achieve a higher concentration in the bloodstream is worth investigating. ivermectin was not a good candidate for a treatment for the disease. The situation is just as bad as the one I discussed for the claim that ivermectin should be considered a promising drug for COVID-19. ivermectin was never a promising candidate for an anti-viral drug to treat COVID-19.
How do we translate this into a probability? Someone suggested what is probably an accurate assessment during the discussion of the NEJM trial.
Yup. As liked to say, it wasn't homeopathy-level implausibility, but it was highly implausible.
— David Gorski, MD, PhD (@gorskon) March 31, 2022
I believe that a pretest probability of less than 1% is accurate. I watched the NEJM study on Thursday.
To accept that such a mechanism might be operative, the clinical trial results have to be highly compelling and pristine. None of the clinical trials for #ivermectin even came close to that standard.
— David Gorski, MD, PhD (@gorskon) March 31, 2022
If ivermectin were to be useful against COVID-19, it would have to work by a different mechanism than the one described in the study. It is possible that such a previously unknown mechanism could be used, but it is highly unlikely. Steve Novella once said that the clinical trial results would have to be pristine in terms of very compelling results from very well-designed and executed clinical trials.
This never applied to any clinical trial of ivermectin for COVID-19, not even the seemingly strongly postive ones, and that's even leaving aside the discovered incompetence and likely fraud in the largest trial.
These negative results never surprised those of us with SBM tendencies. Even the best clinical trials likely had a low probability of success because of the low prior probability that ivermectin is effective against COVID-19. They weren't the only trials. Most trials were equivocal or negative, and, as is the case with acupuncture, the larger and better designed study, the more likely it was to be negative, which leads me to an adage that I frequently use on Twitter.
It is very low prior plausibility.
+
Clinical studies are equivocal.
Is that what it is?
Drug doesn't work for the proposed indication
Any effect that is small and inconsistent will not be clinically useful.
It has been a long time since we talked about the differences between the two, but I felt that a discussion like this was necessary. The reason, of course, has been the pandemic and how the EBM has treated highly implausible COVID-19 therapies in the same way that it has treated highly implausible CAM therapies, through the lens of methodolatry.
The discussion of ivermectin is an example of why it is important in all areas of medicine. Even if you accepted the rationale proposed at face value, ivermectin was not an effective therapy for COVID-19. Basic pharmacology is what it is. A drug that only affects the target at a concentration that is at least 70% higher than the highest blood concentration of the drug that can be safely achieved using standard dosing is not likely to be an effective treatment. It is a general principle that most highly effective drugs do not affect the target at micromolar or g/ml concentrations. Candidate drugs that only affect their target at high concentrations are not likely to be useful drugs.
If a pharmaceutical company had developed a drug like ivermectin for COVID-19, it would have been abandoned after in-vivo testing showed that it needed a high concentration to be effective. A drug company would not pursue a candidate compound like this unless it was a base molecule that could be modified to bind the desired target at a lower concentration or to achieve higher blood levels. Until recently, a number of scientists who I respect said that ivermectin probably doesn't work, or that there was a strong evidence base. Basic pharmacology is important.
I realize that ivermectin believers won't accept an argument against them from a SBM -based argument. There were many reactions to the NEJM study.
And this:
First thing you do with any study, especially one that contradicts all other studies is look to see who funded it. Rainwater was a friend of Warren Buffett and Walt Disney; Deep state thru and thru. So this study is suspect.https://t.co/G2txJzxTKe
— Caoimhe (@CaoimheEithwen) April 1, 2022
All of them fell under the category of rationales, which is what this meme is about.
One shouldn't expect that the arguments for and against the benefits of the two therapies would sway believers in either. It is my hope that the thinking of SBM will sway people who don't really take into account prior plausibility in evaluating RCT evidence and therefore take a long time to reach a conclusion about a treatment. I would argue that it is unethical to begin another RCT of ivermectin to treat COVID-19. The trial would be all risk with no realistic potential for any patient to benefit. I'm not sure if it's ethical to continue to recruit patients to RCTs.
I plan on adding the example of ivermectin for COVID-19 to my long-used examples of homeopathy and acupuncture when I talk about the differences between the two. It never should have been. The case of ivermectin has shown me the error of my ways if I have been guilty of not applying it to conventional medicine as much as I do to CAM.
All Rights Reserved © 2024
