Some doctors and epidemiologists have criticized a study used by the Victorian government as evidence to support its strict mask mandate.
Experts stressed the importance of wearing masks indoors and how masks reduce transmission. However, the Kirby Institute and Burnet Institute jointly conducted a flawed study.
The July medical journal Plos One published the study that used newspaper photos and surveys to evaluate mask compliance and its impact on Covid rates.
Victoria's mask mandate has been controversial. It requires that people wear masks whenever they are outdoors, even in open spaces. However, there is strong evidence that indoor transmission risks are significantly higher, particularly within households.
Daniel Andrews, the premier of Victoria, announced that Victorians would be prohibited from taking off their masks to drink alcohol outside. However, it can be taken off to eat and drink non-alcoholic beverages. The government used the Plos One study to justify its policy. This applies even when there is no one around.
The study examined images of people in public spaces from The Age newspaper's digital archive. These were taken between 10 July 2020 and 2 August 2020, before and after the introduction of the mandatory mask policy in the state's long lockdown.
These photos, which were taken in public places in Melbourne such as shopping centres and streets, were used to calculate how many people wore masks in public over three time periods. A survey was also done by the researchers.
The paper showed that the mandatory use of masks significantly increased public use and was associated with a substantial decline in COVID-19 new cases.
Burnet institute released a press release stating that the world's first study found that mask-wearing had turned the pandemic around in an instant and was the most important tool to turn the tide of Victoria's second Covid wave in 2020.
In light of increasing evidence of outdoor transmission, it is important that masks be worn indoors and out during an outbreak of Covid-19, a paper author stated in a press release.
News.com.au raised concerns about the methodology of the paper's study and called the study into question. According to news.com.au, the Victorian health department had used the study to support its mandate.
Kyle Sheldrick, a researcher, first raised concerns about this paper. He said that there were many methodological issues with the study.
Sheldrick stated that it is difficult to draw causal inferences from photographs taken by news photographers for a specific publication. News photographers don't just take photos of random people in the area. They are assigned to photograph specific stories or editorial lines. It is difficult to imagine a more biased group.
Sheldrick stated that there was an inconsistency between the methods described, and the results reported. He has made public his concerns.
He stated that although the newspaper's archivist checked the 14-day period for any additional unpublished photos, which is very little, and analysed all the images, the search results are not described. The authors, however, only describe 44 images.
Professor Peter Collignon, an infectious diseases physician, agreed with Sheldrick. He said observational studies were the most weakest type of study. They are often unable to prove cause and effect, and they are susceptible to confounding factors that are hard to control but could have an impact on the results.
Collignon stated that he thought it was a mistake for this study to be used in any policy.
There are many flaws in this methodology, and it isn't high-quality evidence. You need randomised controlled, double-blind studies to get high-quality evidence. This study is not high-quality, and it would fall under any evidence criteria.
Gideon MeyerowitzKatz, an epidemiologist at University of Wollongong said that the study was not robust enough to establish a causal link between masking, masking, and a decline of Covid cases.
He said that although the authors used two methods for estimating the change in masking behaviour over time, none of them gave any clear indication of how many people were using masks in the state or city. These photos are not representative of the entire population. The survey was conducted in a very short time span and only 18 people were included after the 23rd July.
Register to Receive the Top Stories from Guardian Australia Every Morning
He said that a randomised controlled trial in Bangladesh showed that mask policies could have a reasonable effect on cases. It was around 10%. He said that the flaws found in one study do not necessarily mean that masks don't work.
The Burnet Institute, which models for the Victorian government, released a statement describing Sheldrick as a PhD student. It also stated that it supported the study.
We understand that not everyone agrees with our work. However, others, including peer-reviewers, thought differently, according to the statement.
Our paper found that there was a significant decline in cases after the introduction of mandatory mask policies in Melbourne. It was implemented in isolation of other restrictions, which allowed for an exclusive assessment of its potential. This was unusual, and it is why we called it a world-first at the time.
Sheldrick stated that he was disappointed with the response of the institute.
He said that the specific criticisms are not addressed in the response. For example, the photos of the before and after are taken in the afternoon. The mandate photos were taken before lunch. So it is impossible to compare them meaningfully.
Professor Catherine Bennett, Deakin University's chair of epidemiology, stated that the study was a valuable contribution. Although she acknowledged that the researchers had only looked at photographs and data for a short time, and assumed a linear relationship among masks and reduced Covid cases, which is unusual for epidemic curves, the study had proven the mask policy was effective in influencing public acceptance.
She said that it was a valuable contribution to track the progress and ensure that the policy is implemented in practice.
