Methodolatry and COVID

It was an interesting comment.

An interesting comment was left in a previous article about a video made by a doctor. Squirrelelite is a reader and he says that this doctor.

Mentioned a tweet from Matthew Yglesias suggesting that lots of people would have died if we had waited to do an RCT to approve boosters as Gruber and Krause wanted. He then goes on to claim that 2 or 3 hundred thousand people would have quickly signed up for the trial to get access to a booster shot, ignoring the huge cost and time delay to enroll 5-8 times as many people as were in the original vaccine trials. And he states that it would “only” take 11 months to run that trial. He claims we don’t have effectiveness results on 12-15 year olds without such a trial and also ignores these results from back in February or this result from January… (He) claims Gruber and Krause were “vindicated” but he doesn’t talk about any numbers to support that.

I promise they are reporting it fairly, because I didn't have the strength to watch the video Squirrelelite provided. It's true that many doctors have lamented that there have not been large randomized controlled trials to study boosters and other pandemic interventions. Squirrelelite referred to two scientists who resigned from the FDA last summer in protest of the decision to offer a booster to all adults. There wasn't enough evidence to support this. They shouldn't be called anti-vaccine for their skepticism, but they haven't been vindicated.

It is easier to have an idea for an RCT than it is to actually do it.

I like RCTs. All doctors do. In a fantasy world with unlimited time and money, but without ethics, an RCT would yield the optimal answer to many medical questions. They have the ability to randomize patients and minimize bias. They are a remarkable achievement and have a fascinating history. Every intervention that can be studied via an RCT should be studied.

Significant ethical and practical limitations prevent us from doing this in the real world. It's easier to have an idea for an RCT than it is to actually do it.

• The safety of the subjects in RCTs is paramount, as it should be, especially given the history of Nazi atrocities and the Tuskegee Study. Basic ethics and Institutional Review Boards (IRB) strictly limit what can be studied via an RCT. RCTs can only be used to study interventions that have a reasonable chance of benefiting patients or when there is true uncertainty about which treatment is better, which is termed clinical equipoise. There’s never been or will be an RCT showing that smoking harms or that parachutes help. Having said this, there are many fewer “parachutes” in medicine than we often believe. Medicine is full of treatments that everyone “knew” worked, until they were found to be useless in an RCT. To repeat, I firmly believe that every intervention that can be studied via an RCT should be.
• Only the largest RCTs can reliably detect rare treatment harms or benefits. The rarer the event, the larger a study must be to detect it, something termed a study’s power. Generally, rare events can only be detected once the studied treatment has been given to large numbers of people outside an RCT. Real-world studies have taught us much more about rare vaccine harms and benefits than the RCTs.
• RCTs are hard to do for rare diseases. It’s challenging to enroll enough patients.
• RCTs are of relatively short duration. Any treatment harm or benefit that occurs many years later won’t be detected via an RCT.
• RCTs are slow. A doctor who has an idea for an RCT can’t just start one the next day. It takes time to design the trial, to get funding, to have it evaluated and approved by an IRB, to set up the infrastructure, to enroll patients, to run the study, and to synthesize the data. Most RCTs take many years to from the time they are conceived to when the final data changes clinical care, which is obviously problematic in a pandemic that arrived like a tornado. As I will discuss, the COVID vaccine trials had several unique features, including the rapid spread of COVID, that allowed them to proceed faster than most RCTs.
• RCTs are expensive, and funding can be hard to come by, especially for studies where no one can profit based on the outcome. The COVID vaccine trials were unique in that full funding was immediately available.
• An RCT can sound perfect right until the moment you actually have to enroll patients into it. Those of us who have done this know that recruitment can be very difficult. Some patients are eager to do so, but many don’t want to be a “guinea pig.” A normally hard process is much harder in a pandemic. One researcher, who has actual experience running an RCT this pandemic, said, “I think of what my clinical research team went through to enroll people in that trial, and I thought my nurses were going to die. One of them got covid and got sick. Imagine trying to do that on a daily basis, multiple patients, some of them facing intubation, none of them have their families.” Patients or their families have to be able to give informed consent, which can be impossible for delirious, intubated COVID patients and their distraught families. As another researcher said, “There was one day our system had 84 deaths. And then you’re going to ask me to potentially put them on a placebo? It’s just really heart wrenching, talking with families, if your patients are able to communicate — and you’re dealing with all these deaths.” Given all of this, it’s no surprise that clinical trials can fail to recruit enough patients. A proposed RCT of hydroxychloroquine and azithromycin run by the NIH closed after enrolling just 20 patients, or 1% of their goal. I’m sure that study looked great on paper. Graveyards are full of dead RCTs and there’s research into why trials fail. The vaccine trials were unique in that boatloads of volunteers, like me, beat down the door to enter them. I am skeptical that hordes of twice-vaccinated people would be similarly eager to enter into a booster trial. Anyone who acts as if recruiting a large number of subjects into an RCT is a trivial matter while a pandemic rages is either clueless or dishonest.
• RCTs require an enormous infrastructure and multiple experts with different skills. Pfizer has this. You don’t. Most RCTs occur at multiple locations, and teams have to be in place at every site. One of the mRNA vaccine trials occurred at 152 sites worldwide. The vaccines weren’t just a scientific triumph, the RCTs were also logistical marvel. To get a sense of how complicated RCTs are, one article listed the following costs:  regulatory affairs, site identification and selection, site contracting and payments, site initiation and activation, site management, onsite monitoring, drug safety management, drug logistics, biological sample logistics, clinical supplies logistics, medical writing, site close-out, project management, study files/document management, data management, statistics, quality control, communication with central CRO/sponsor, and pass-through costs, which includes, trial insurance policies for each country, shipping: physical files to sites, site contracts, and tumor/blood samples, blood tubes and shipping packages, office supplies: files, paper, and printing, payments to sites per enrolled patient (to cover clinical procedures and laboratory tests), publication fees, ethics committee evaluation fees, site contract fees, regulatory authority evaluation fees, travel costs for selection, initiation, routine monitoring, and close-out visits, central pathology and radiology reviews, translational/biomarker studies, coordinating investigators, drug manufacturing and testing, drug distribution services, EDC license and service fees, web tools (imaging platforms, eTMF), document translations, and data and Safety Monitoring Board (DSMB). 
• Given their complexity, RCTs don’t always smoothly. Serious problems can sometimes arise and small hiccups are more common than not. Vaccine trials in particular have to close to flawless. Those who enjoy spreading fear about vaccines will magnify the slightest imperfection to cast doubt on the whole project. This happened with the original COVID vaccine RCTs, much to the delight of anti-vaxxers. A rushed or sloppy RCT would do much more harm than good
• RCTs, including the vaccine RCTs, commonly exclude key segments of the population, such as pregnant women or immunocompromised patients. As such, other study designs are used to evaluate the vaccine in these populations. The exclusion of pregnant women from most RCTs is a source of controversy.
• RCT participants, who have to be savvy enough to learn about the trial in the first place, may not be representative of the general population. They are generally healthy and motivated enough to travel back and forth to the study site on a regular basis. The most vulnerable people, those who benefit from the booster the most, would likely be underrepresented in an RCT, thus skewing the results. Racial and ethnic minorities have also been underrepresented in clinical trials.
• An RCT done in one area may not generalize to another. For example, an RCT of masks in Bangladesh relied on “role-modeling by community leaders”. Its results would not be applicable to areas where community leaders oppose masks.
• Unlike any other condition in medicine, the disease being studied changes rapidly in a pandemic. The vaccine RCTs were done with a variant that is long gone. Does anyone believe these RCTs, where the vaccine was 95% effective at stopping COVID, have any relevance to our individual and policy choices today? Of course not. These RCTs rapidly became historical relics, and we’ve learned much more about the vaccines through real-world studies. An RCT of other interventions that was initiated at the pandemic’s onset, would likely be obsolete due to the more contagious variants that are circulating today.

Researchers use different types of observational studies to investigate topics that are difficult or impossible to study via an RCT.

They have a greater potential for spurious associations because they don't select patients randomly. It is possible that sicker people are more likely to get a vaccine than healthy people. The make vaccine appear less effective than it actually is if you don't control for this discrepancy.

Most researchers acknowledge their limitations in their papers. There are a lot of really bad observational studies out there, and discussing their flaws is a major topic here at Science Based Medicine.

Worship methodolatry

Methodolatry is a technique used by anti-vaxxers who claim that only RCTs can provide reliable information. If there was an RCT showing, anti-vaxxers might pretend they would embrace vaccines. They reject the validity of existing RCTs, and it is clear that if their proposed RCT were done, they would demand another.

Anti-vaxxers may point out that there is no evidence that the vaccine prevents cancer. This is true. The RCTs of the vaccine used a surrogate endpoint, which can be a cancer precursor. It would not be ethical to let people remain in the placebo group once evidence emerged of the vaccine's safety and efficacy, as the outcome would not be possible to study in the time frame of an RCT.

Methodolatry isn't just a problem for anti-vaxxers. Two years ago, serious people wrote an article about the effectiveness of the vaccine. The trials were not designed to determine effectiveness against the disease. There were no reported cases of cancer in any trials.

It is still uncertain whether HPV vaccination prevents cervical cancer as trials were not designed to detect this outcome, which takes decades to develop… There are too few data to clearly conclude that HPV vaccine prevents cervical intraepithelial neoplasia grade 3.

The huge amount of non-trial research evidence that enables most scientists to conclude that the vaccine works was discussed in the rebuttal.

It wasn't a great leap to think that if a vaccine prevented a cancer precursor, it would also prevent cancer. There is clear evidence that the vaccine is very effective at preventing cancer. The people who reject this evidence are guilty of methodolatry.

Absence of evidence is not evidence of absence. When asked to provide evidence for their latest miracle cure, it is often invoked by quacks. There is no evidence that magic beans cure cancer, but there is no evidence that magic beans don't. They are fond of pretending that at some point in the future, science will study magic beans and their cancer-healing properties will be revealed.

The absence of evidence is not evidence of absence. Someone wouldn't be wrong if they said gun violence had decreased after aliens destroyed all guns. We are allowed to believe reasonable things with less evidence than Massive RCTs.

There would have been a real cost of waiting if a large RCT had shown the vaccine prevented cancer. Doctors used less-than- perfect data to create the vaccine that protects millions of people.

This was not controversial amongst doctors before. Only cranks who wrote articles like The Truth is Out: Gardasil vaccine coverup exposed and Can pharma improve on nature. The vaccine against the human papilloma vaccine was opposed. If a doctor said we should not use the vaccine until there was an RCT showing it reduced mortality, it would be seen as a crank.

Things are not the same with COVID.

Methodolatry is studies that are impossible.

Anti-vaxxers protest that studies showing the safety of individual vaccines are useless because there has never been an RCT of the entire vaccine schedule. They wonder how anyone could be against an RCT of the full vaccine schedule.

They are aware that no parent would sign their baby up for an RCT and that it would not be approved by the IRB. It would be impossible to carry out a study without these obstacles. That is the point.

Studies overwhelmingly show that vaccines benefit children. They want to diminish the credibility of the studies. Parents who volunteer their children for RCTs of vaccines are vilified by anti-vaxxers. That is how ridiculous they are.

Anti-vaxxers are able to identify gaps that an enormous RCT could fill. The RCTs that have been done are flawed. They are sometimes felt to be too small in size or short in duration. There are many studies that show vaccines are safe and effective, but the only study that matters is the RCT.

The point of those who want to spread fear about vaccines is to reject the good studies that have already been done. When an observational study claims to reveal vaccine flaws, anti-vaxxers have no issue with non-RCTs, even when the methods are questionable. These studies are considered to be bombshells.

The image is from https://virologydownunder.com.

There are RCTs in a pandemic.

The RECOVERY trial showed a dirt-cheap steroid saved lives in severely ill patients. RCTs were used to evaluate 9 other treatments. The researchers who demonstrated that surgical masks slowed COVID also proved that it was possible to investigate these sorts of interventions via an RCT, at least in Bangladesh.

There should have been more of these RCTs. The CDC hopes to have a strike force that can quickly test interventions. Signing up for RCTs is a heroic and vital effort where their safety is a priority. Medical progress would stop if these volunteers were not there.

Everyone who worked on or volunteered for the RCT deserves our gratitude. It was a lot of work.

I realize that we can study every intervention this way, though I wish more interventions had been studied via RCTs. Our public health leaders had to make choices because we don't have an RCT Strike Force. You can disagree with their choices, but they couldn't have studied everything via an RCT. It would take a lot of money and time to convince people to get the first two doses of an RCT.

It is not easy to say which interventions should be studied in a raging Pandemic. They are needed to evaluate new vaccines. I'm happy that this was our top priority. Proper evaluation of medications is necessary. I didn't give my patients ivermectin or hydroxychloroquine because of some lousy observational studies. Without a well-done RCT showing otherwise, I assumed these drugs would cause more harm than benefit.

Other interventions have a different risk/benefit profile and favor their use even without large RCTs. Such is life. Time is not an unlimited luxury in a Pandemic and we can't let perfect be the enemy of good. Dr. Greenhalgh wrote:

Regulators were right to suggest a booster without an RCT.

The evidence we have about the value of boosters is very good. A booster was shown to be safe and effective at stopping COVID. The vaccine prevented hospitalizations and deaths, but doctors minimized it. This is true, but like those who claim there is no evidence that the vaccine stops cancer, it is not wise or helpful. It is potentially dangerous. Many people don't know how much the third shot can help them.

It wasn't a great leap to think that if vaccine stopped COVID infections, it would also lower deaths and hospitalizations. Large observational studies show that boosters are effective at limiting severe disease. The vaccine helps stop the spread of the virus. We are allowed to believe in things that are reasonable.

The risks were not high. The worst case scenario would have been an unnecessary vaccine dose for millions of people. This is problematic, but not a tragedy for those who received the vaccine. The vaccine-myocarditis rate is lower with the booster and patients do well in a short time.

If regulators could have snapped their fingers and willed a massive booster RCT into existence, they would have. Since they don't operate in Fantasyland, they had to make a decision based on the best evidence at the time.

If enough people volunteered to let it be run, we might just now be getting the results, assuming they demanded an RCT large enough to show the booster prevented hospitalizations and deaths. The trial at the start of the Pandemic was closed after just 20 people were signed up.

A booster RCT that started last summer would have begun before the arrival of variant that seem better at evading the immune system. It is not clear if the results of the first two doses would be relevant today.

Regulators were right to suggest boosters without benefit. Millions of people would have been at risk if a booster RCT had been run. The delay would have been very costly and the information gained would have been of little use.

The main issue with boosters is that most people don't need one. This is not a surprise. After an RCT showed the booster was extremely effective in preventing COVID, contrarian doctors thought it was funny to sit together in a studio and proudly declare they were unmasked, unboosted, and unrepentant. The results of a massive RCT showing the booster failed were not discussed. They don't require any evidence to trash vaccines, but they do require massive RCTs to advocate for them. They assumed that the booster prevented severe COVID because there wasn't a massive RCT showing it.

. There is no evidence for these doctors.

Doctors who actually treat COVID patients know the best way to save lives and prevent suffering is to encourage every eligible person to get a booster, not to spread unwarranted doubt about the solid evidence we have that they are beneficial. Boosters have been shown to be of benefit for every age group studied so far, even teens.

If influential doctors hadn't made a public mockery of it all, more people would have taken it.

A double standard.

They could have used their large platforms to educate the public on the strengths and weaknesses of various study designs. They could have told people that having an idea for an RCT is much easier than doing an RCT.

They have deceived their audience into believing that only enormous RCTs can determine the value of pandemic interventions, that enormous RCTs are trivial endeavors, and that more enormous RCTs weren't done because the people in charge were stupid and apathetic. They undermined confidence in all the measures that were used to control the virus and public health in general.

A doctor proposed that public health restrictions should not last more than three months if they can be proven beneficial in an RCT. If RCT can prove that washing your hands before returning to work is beneficial, then restaurants will have to get rid of their bathroom signs. The next time the guy uses the bathroom at work, he can determine his personal risk level. He is doing anything wrong, but there is no data.

When one sees the double standard they have for when data is deemed reliable, the gimmick becomes clear. The doctors who call for an RCT for every proposed measure to limit the virus are perfectly content to use weak observational data, or no data at all, when it bolsters their argument that children should be left vulnerable to the virus. The information on vaccine side-effects comes from observational studies, but contrarian doctors discuss it all the time. The validity of this data was not questioned because it did not come from RCTs. The rigor of an RCT is demanded only with purported vaccine benefits.

RCTs are often deemed insufficient as with the booster trial. The vaccine was very effective in preventing COVID. In their missives, contrarian doctors either pretended these studies didn't exist or they lied about it. They said they were too small to inform the safety of a vaccine that will be given to millions.

It wasn't a great leap to think that if vaccine stopped COVID infections, it would also lower deaths and hospitalizations. There is clear evidence that the vaccine is very effective in preventing these rare but devastating outcomes in children. contrarian doctors pretend they don't exist because they show the vaccine is beneficial. They don't believe in reasonable things.

An impossible study.

The vaccine benefits children despite the fact that both RCTs are real-world studies. A doctor said last week.

If Pfizer and Moderna want these vaccines to be given to children, they should first conduct a randomized controlled trial that shows that they reduce hospitalization and all-cause mortality. They failed to do so for adults. They should not get away with that for our children.

I am going to surprise this doctor with something. I encourage you to do the same when a doctor suggests an RCT. It would be nice to have an RCT showing that the vaccine saves children's lives.

Would it be possible to run this RCT?

Children rarely die from COVID. A child dying in a vaccine is very rare. It would take one of the largest and longest RCTs in the history of medicine to evaluate the vaccine's effect on all-cause mortality. This trial requires an immense number of unvaccinated children whose parents would be willing to enroll them in an RCT with a 50% chance they would get vaccine.

As they say, it's funny.

There are more obstacles. Would an IRB approve a study where children are left unvaccinated? Absolutely not. Would ethical doctors enroll their patients in the study? Absolutely not. Would anyone pay for it? Absolutely not. Would the results be available in a useful time period? Absolutely not.

There is no way this study can be run, like the anti-vaxxers called for an RCT of the entire vaccine schedule. Again, that's the point. When contrarian doctors don't like the results of the many studies showing the vaccine is beneficial for children, they call for a bunch of RCTs.

It makes sense that the individual who demanded this impossible RCT was one of the doctors who wanted unvaccinated children to contract COVID. His efforts were successful. He should take a victory lap.

Instead, after 1,500 dead children died and tens of thousands more were hospitalized, he beclowning himself to avoid acknowledging the overwhelming evidence that the vaccine can reduce these severe outcomes. He can't point to any evidence that the vaccine is riskier than the virus. He can't do anything except wish away unwanted studies, demand an absurd RCT, and proclaim that there is no evidence in the world that would convince him children should be protected from a potentially deadly virus. He is happy to leave children vulnerable to carbon dioxide rather than say he was wrong.

This impossible RCT is an un serious demand from an un serious person, and it fools parents into believing the vaccine hasn't been tested, when nothing could be further from the truth. The vaccine rates for children are abysmally low thanks to this and many other myths spread by contrarian doctors.

Pfizer and Moderna are not the only ones who want these vaccines to be given to children. Doctors who care for sick children want them to bevaccinated as well. The consequences of myths spread by doctors are real. The swine flu has more to do with making videos on their laptop than it does with making videos on the internet.

Doctors who call for an RCT for everything don't usually run a single RCT on anything.

Suddenly, I have a feeling. It is familiar to those of us who thought it was worthwhile to think about these issues prior to the epidemic. It's all just pathetic variations of Will HPV Vaccination prevent Cervical Cancer blended with a dose of Can pharma improve on nature? There is a vaccine for the human immunodeficiency virus, also known as the "Hp Vaccine." Which one is better?

Therian doctors are able to identify gaps that an enormous RCT could fill. The RCTs that have been done are flawed. They are sometimes felt to be too small in size or short in duration. There are many studies showing vaccines are safe and effective, but the only one that matters to contrarian doctors is the RCT.

The entire point of those who spread fear about vaccines and fetishize immunity is to reject the RCTs that have already been done. When an observational study claims to reveal vaccine flaws, contrarian doctors have no issue with non-RCTs, even when the methods are questionable. These studies are considered to be bombshells.

It isn't because doctors are ignorant or lazy, but because the studies are expensive, slow, and really hard. It is no surprise that contrarian doctors always call for someone else to do an RCT. It's easy to point a camera at yourself and complain that other doctors didn't run enough RCTs. It is extremely difficult to run an RCT.

Doctors who call for an RCT for everything generally haven't run a single RCT on anything.