I tried to argue that the case of ivermectin, the antihelminthic medication that is used in both animals and humans to prevent and treat diseases caused by roundworms, became the focus of a miracle cure. I will not revisit the argument because I pointed out that ivermectin had a very low plausibility and probability of working in a randomized controlled trial in humans. When used to treat COVID-19, ivermectin is indistinguishable from placebo. Ivermectin is a great medicine for roundworm infections, but it doesn't work against COVID-19. If you take basic science into account, there is no reason to 888-276-5932 888-276-5932 888-276-5932 888-276-5932. I compared ivermectin to acupuncture, a treatment which high quality RCTs increasingly find no effect distinguishable from placebo/sham treatments but for which advocates have pivoted to lower quality evidence. I came across an article last week that reinforced this point. An international panel including patients, clinicians, researchers, acupuncture and surgery trialists, statisticians, and experts in clinical epidemiology and methodology was published in The BMJ. I might need to take a break from writing about COVID-19, but this consensus article is an example of how important it is to take into account prior plausibility. It's not all bad. The article seems to advocate more rigorous science. As I read it, I noticed how the authors focused on the importance of double blinding and proper use of sham acupuncture rather than asking if it has any effect on any condition. I think this betrays the belief among many people. The article begins. The original and revised Standards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA)10 11 focus on reporting not optimal trial conduct. Other acupuncture guidance covers type of trial (efficacy,12 effectiveness13), design, and lack of representativeness (include an international panel14), but is not informed by comprehensive and systematic evidence synthesis. To address the most prevalent design and methodological concerns in current acupuncture randomised trials, an international panel including patients, clinicians, researchers, and trialists in acupuncture, surgery, statistics, patient engagement, and clinical epidemiology developed guidance for research teams planning acupuncture trials. To inform the guidance, we conducted a systematic survey15 exploring characteristics associated with acupuncture treatment effects. And: We established a steering committee including frontline acupuncture clinicians, acupuncture trialists, clinical trial methodologists, and a statistician (XHJ, JPL, LXL, CMW, LT, YQZ, RMJ, and GHG) with extensive experience in acupuncture and trial methodology. The steering committee recruited an international expert panel of 27 experts (from Asia, Europe, America, and Australia), including patients, frontline acupuncture clinicians, acupuncture and surgical trialists, clinical trial methodologists, and statisticians. Acupuncture trialists and methodologists were appointed to the panel on the basis of their h index and assessment of their research expertise, and frontline clinicians on the basis of their clinical experience and reputation. If there is one thing that we have been discussing for a long time, it is how pseudoscientific, unscientific, and religion-based treatments love to set up pseudo medical organizations. The doctor dubbed them "pseudo medical pseudoprofessional organizations" back in 2008. Naturopaths have done it, as have a number of other pseudomedical specialties, resulting in highly dubious evidence-based guidelines for breast cancer. Let's look at the good, the bad, and the ugly. There is no correlation between Clint Eastwood's performance in this movie and the acupuncture study. Let's concede what's good about the guidelines. First, the authors emphasize points that are important for designing a trial.
Choosing the research question
Consideration 1: Is the question important?
The Good…
The rationale for the research question should be established by trialists.
Enrolling patients.
Consideration 2: Who should the participants be?
People who are most responsive to the intervention could be included in a trialists objective. A broad population could be included with a primary pragmatic or practical objective.
There are possible differences in effect across patient groups.
For pragmatically oriented trials include heterogeneous populations, with prior specification of subgroup analyses to consider hypothesised effect modification.
Consideration 1 is very important. The lack of a plausible mechanism and the observation that the more rigorous a clinical trial of acupuncture is, the more likely it is to be a negative study, makes us think that no question involving acupuncture is important anymore. I accept that not everyone would agree with this assessment.
For the sake of argument, we should postulate that we have an important question about acupuncture. The next consideration is important. What do the authors mean by people who are most responsive to intervention? My point is simple, how do you know if a person is likely to respond to acupuncture? People more responsive to placebo effects in general might be more responsive to the ultimate in theatrical placebo.
There is mention of pragmatic trials. Calculating the efficacy of an intervention in the general population, outside of the strictly defined inclusion and exclusion criteria of a clinical trial and the strictly enforced protocols for monitoring outcomes is called pragmatic trials. The difference between RCTs and pragmatic trials is that RCTs don't include a placebo group, include a wider range of people, and sometimes are not even randomized. There aren't always control groups. Why? Steve Novella and I have emphasized for a long time that pragmatic trials operate under a very important assumption, that the treatment being studied has been proven to be safe and effective in RCTs. The idea behind pragmatic trials is to assess how well a treatment works in the wild, not to determine whether it works in the first place. I often say that doing a pragmatic trial of a treatment that has never been shown to work is putting the cart before the horse. There is no placebo control and pragmatic trials are very prone to results due to placebo effects.
It is reasonable to include heterogeneous populations in a pragmatic trial. That is the point of a pragmatic trial, at least one of the main points. It is mandatory to prespecify subgroup in order to avoid post hoc analyses and data mining that find apparent effects that aren't real. Post hoc analyses should be used as hypothesis-generating exercises, not as hypothesis testing exercises.
The discussion of engaging patients, the duration of followup, and the choice of outcomes are examples of The Good.
Let's move on to The Bad.
The bad.
The paper shows the biases of the people who wrote it. Let's start with a rather telling omission. It is not an omission. The authors acknowledge an issue in a sidelong way without actually addressing the most important analysis to avoid it. Adherence to treatment is what I'm referring to.
Consideration 8: How should trialists deal with adherence?
When designing a trial that primarily takes the individual patient’s perspective, trialists might consider implementing strategies to achieve optimal adherence. When trialists take a public health perspective, there is no need to implement strategies to increase adherence.
Istention to treat analysis anyone? Adherence might be a problem in RCTs, but not once do they mention an intention-to-treat analysis. The recent negative RCT of ivermectin for COVID-19 that I discussed included more than one intention to treat analyses. The authors of this study seem to be unable to type the words.
We get handwaving like this.
Thus, when designing a trial that primarily takes the individual patient’s perspective, trialists should consider implementing strategies to achieve optimal adherence. One strategy is to engage patients to help design the trial. Patients can identify and improve onerous trial design, therefore increasing participation.60 Another strategy is to include a run-in period before randomisation and only randomise patients who are highly compliant in the run-in.
Including a run-in period is highly feasible in drug trials when patients may be offered a placebo during the run-in or when the trial is focused on morbid or fatal events and short term exposure will have no influence on the outcome. The situation is more complicated in acupuncture trials focusing on symptoms in which initial exposure to treatment during a run-in may complicate inferences about what occurs after randomisation. This is probably why acupuncture trials rarely use run-in periods to enhance adherence.
This is good, but not enough. Again, I find it telling that there is no importance to the intention-to-treat analysis in the paper. The per-protocol analysis involves only patients who complete the entire clinical trial according to the protocol, and they are counted towards the final results. Best practice in RCTs includes both an intention to treat analysis and a per-protocol analysis.
The actual intervention is next.
Selecting the intervention
Consideration 4: Who should perform the intervention?
Trialists should report the expertise of the acupuncturists. A trial that aims to show whether an acupuncture treatment can work under ideal conditions will choose the most expert practitioners available. Trialists aiming to establish the effect of treatment in ordinary practice will select clinicians with typical levels of expertise.
Consideration 5: What specific acupuncture technical features should be considered if aiming to design trials for maximum treatment effect?
If aiming for maximum treatment effect, trialists should select a high frequency of acupuncture treatment sessions and penetrating type of acupuncture (manual and electroacupuncture) over lower frequency or non-penetrating (transcutaneous electrical acupoint stimulation (TEAS), laser, acupressure) acupuncture.
The authors emphasize the importance of expertise in an intervention for surgery, procedure-based interventions, and the like. It makes me think of how pseudoscientific specialties like acupuncture love to mimic real medicine and claim that expertise matters even if the entire rationale behind them is based on prescientific vitalism. The recommendation is for high Frequency over low Frequency and lower Invasiveness.
For this point, notice how the authors have mixed together the two. I like to point out that this is a bait-and-switch, because it is contrary to the ancient practices that people use to justify sticking needles into people to treat them. Transcutaneous electrical nerve stimulation might work for some indications. I've always believed that the reason that acupuncturists started hooking up needles to a current source was because they realized that they didn't do much.
It gets worse.
The ugly.
The discussion of the best comparator to use in a clinical trial of acupuncture is the most obvious propaganda aspect of this set.
Choosing the comparator
Consideration 6: Are trialists interested in the specific effect or the overall (specific and non-specific) effect of acupuncture?
Trialists should blind data collectors, outcome assessors, and data analysts and carefully consider the desirability of a sham that leads to underestimation of acupuncture’s treatment effects in clinical practice.
Fine tuning the flexibility of intervention and comparator
Consideration 7: To what extent should the trialist choose a flexible intervention and comparator?
Trialists with a primary explanatory or mechanistic focus might specify that practitioners administering both the intervention and the control use a highly standardised approach. Trialists with a primary pragmatic or practical focus might include clinicians with varied techniques reflecting practice in the community and instruct them to use their usual treatment approaches.
Consideration 6 is obvious, but notice the wording. The authors would like to see the data collector, outcome assessors, and data analysts, but what about the patients and the acupuncturists? In that case, they only weakly advocate that a sham should be considered. Imagine a new medication being tested. Imagine my argument, "Trialists should blind data collectors, outcome assessors, and data analysts and carefully consider the desirability of a sham that leads to underestimation of the drug's treatment effects in clinical practice."
The treatment of a placebo control is to reduce nonspecific effects. The authors consider what is normally considered a desirable effect of using a placebo, to control for nonspecific effects that might produce a negative result. That's because it's a placebo. Even if they don't admit it, they do seem to realize it, even as they invoke all sorts of caveats regarding which sham to use or even whether to use a sham at all.
In drug trials, which are closer to a explanatory or mechanistic design, trialists often use a placebo to achieve blinding. Acupuncture trials, however, face multiple challenges when using placebo or sham acupuncture for blinding.15 It is nearly impossible to blind clinicians who deliver acupuncture, although one device exists that allows practitioner blinding but limits the acupuncture to a rather superficial version (maximum 5 mm insertion).45Participants who have had previous acupuncture experience may also be hard to blind.4748
Whether to attempt blinding in acupuncture trials depends on the hypotheses and objectives of a trial. Here, we will refer to a treatment’s biological effects as specific effects and placebo effects as acupuncture’s non-specific effects. Trialists examining explanatory questions should include sham acupuncture control with adequately blinded participants to differentiate acupuncture’s specific and non-specific effects.45
The results of such explanatory studies require careful interpretation, however. The clinical and basic science literature support the possibility of specific effects generated by sham acupuncture4950— that is, a failure to show a difference between real and sham acupuncture may be because the sham has effects closely related to that of the intervention. Therefore, when using sham control, trialists must consider the possibility of a specific effect generated by the sham and thus underestimating the effects of acupuncture in clinical practice compared with no intervention or other interventions such as drugs.
It is not nearly impossible for blind acupuncturists to use acupuncture. I will concede that it is not easy to do, but it is nearly impossible, and it betrays a very common attitude among acupuncture researchers. Steve Novella and I have written about these studies before, and they have written about them before. It is almost trivial to make a current source that looks like it is working. The subjects won't be able to tell the difference because the current is so low.
The authors try to claim that there are specific effects from sham, which is of course. Even though the best practice is to use sham/placebo control in RCTs, the authors suggest that it isn't actually scientifically proven. Why bother?
I want you to observe what I mean.
Previous studies of shams have focused on pain and chronic pain and conducted univariable analyses (web appendix 2, 2.2.2 (2)). We used multivariable analyses in our systematic survey, adjusting for other potential factors such as treatment frequency, flexibility of the acupuncture regimen, and sample size. Like a previous systematic review,51 we found the type of sham did not influence acupuncture’s effect.15 This is perhaps unsurprising considering the finding of meta-epidemiological studies in areas beyond acupuncture: results of such studies have proved inconsistent, with one recent thorough review showing no systematic effect of blinding.52
Considering all the evidence, the effect of the type of sham remains uncertain and might vary with the type of medical condition. Over 90% of acupuncture trials focus on pain, quality of life, function, and other symptoms. These are all subjective outcomes in which blinding may be particularly important.15 Trialists should therefore carefully consider the desirability of a sham in randomised trials investigating efficacy and, if desirable, the nature of the sham in relation to their trial’s specific objectives. Trialists should also be aware that the use of non-penetrating needle shams mandates the use of the same device in the real acupuncture group. These devices may impede real acupuncture treatment effects.
The authors cite a meta-epidemiological study that claims to claim that, in contrast to everything we know about the use. As the subjectivity of the outcome being assessed increases, the chances that a lack of blinding could contribute to a positive outcome but didn't reach statistical significance. The problem with this study is that few studies included in the meta-analyses actually assessed whether blinding was effective, with the authors noting that all instances the credible intervals were wide.
The authors mention it.
We did not expect to find that our study does not firmly underpin standard methodological practice. Further, our results are coherent with other meta-epidemiological studies that have reported similar results. The implication seems to be that either blinding is less important (on average) than often believed, that the meta-epidemiological approach is less reliable, or that our findings can, to some extent, be explained by lack of precision.
I would suspect the latter two explanations, given the poor quality of most acupuncture studies and the fact that meta-epidemiological studies are still rather ill-defined. It's true that blinding matters less for death than it does for more subjective outcomes, such as pain, but few are the acupuncture studies that look at such outcomes.
Again, the authors seem to realize that it is a placebo.
Acupuncture therapies are multifaceted interventions that incorporate patient-practitioner interactions, treatment theories, tools to deliver the stimulations, manipulation techniques, and points selection. The trial objectives should determine the flexibility of the intervention or control.
The patient-practitioner interactions are one of the strongest contributors to placebo effects.
More rigorous RCTs that incorporate best practices of RCTs, such as sham controls, placebos, and double blinding, rely on less rigorous pragmatic trials, which are more prone to bias and placebo effects. The shift to citing lower quality evidence is relevant when considering ivermectin trials as higher quality evidence increasingly comes back negative. Similarly, so is the tendency of acupuncturists to make excuses for why they don't want to do more rigorous studies, such as casting doubt on the scientific legitimacy of sham interventions as the best comparators. In the case of ivermectin trials, there are endless excuses in the form of this.
The bar of evidence for a positive effect should be higher for both interventions because of basic science. We found that advocates of incredibly implausible treatments always cite lower quality evidence and even try to tilt the field to increase the likelihood that their treatment will appear to be valid by science.
You can buy an e-book.
Dr. Hall has a video course.
The text is powered by the internet.
Powered by
TranslatePowered by Translate
All Rights Reserved © 2024
