With Covid studies, the quality of the evidence matters | David Spiegelhalter and Anthony Masters

In The Adventure of the copper beeches, the detective says, "Data! Data! Data! I cannot make bricks without clay. Recent claims of benefits from wearing masks and using ivermectin depended on low-quality clay.

It cannot make silk purses out of sows' ears, but it can pool the results from many studies. A recent British Medical Journal review looked at six studies and estimated a 53% reduction in risk. The smallest effect was a reduction in the number of infections. An analysis of US states00057-2/fulltext and a survey of about 8,000 Chinese adults in early 2020 were theaviest studies, but their editorial highlighted the risks of confounding variables and the impossibility of disentangling the effects of infections. This review found a similar reduction in hand washing.

The estimated effect dissipated after removing studies with a high risk of bias.

It is difficult to imagine a randomized trial of the effects of mask-wearing. It should be easier to evaluate ivermectin, a cheap, anti-parasitic drug, which is heavily promoted as an overlooked treatment for Covid-19, if you look at these. A meta-analysis found that ivermectin reduced Covid-19 mortality by over 50%. There have been concerns over major errors and fraud in some analyses after a preprint server withdrew the largest study. After removing studies with a high risk of bias, the estimated mortality effect dissipated, so no firm conclusions can be drawn.

Huge effort has been put into scrutinising ivermectin research, even leading to an intriguing observation the drug appears to work in countries with a high prevalence of roundworm, whereas what is needed are the kind of good-quality studies that finally laid the claims about hydroxychloro Fortunately, these are under way.

The Winton Centre for Risk and Evidence Communication is chaired by David Spiegelhalter. The Royal Statistical Society has a statistical ambassador.