Botched Coverage of the Waterloo Mask Study
Looks like many reporters didn't even read past the abstract.
I’m not a scientist, but I’ve been in the media business long enough to find out that most drive-by media coverage of scientific studies is not coverage of scientific studies. It’s coverage of press releases about scientific studies and 10-minute interviews with study authors and other loquacious academics and public health officials whose contact information reporters have on a permanent digital post-it note. People who can thoughtfully critique a study and care that you don’t botch the coverage are not sought after by reporters. People with credentials who will return your calls are.
The University of Waterloo’s mask study that found cloth masks were “only 10% efficient” is no different. Here’s a sample of the headlines:
Study supports widespread use of better masks to curb COVID-19 indoors (same article as found in Healthshots)
A study suggests that N95 and KN95 masks can prevent Covid-19 better than cloth masks
Common Facemasks Filters Only About 10 Percent Of Breathed Aerosol Droplets: Study
STUDY: POPULAR BLUE SURGICAL MASKS OFFER CLOSE TO ZERO PROTECTION AGAINST COVID
The paradox of news readership is that you only know the headlines are misleading if you’ve read the document the article is about or some kind of summary (Supreme Court rulings, dossiers, bombshell emails, etc.). In this case you need to have read at least the study’s abstract and at least skimmed the methodology, in which case you already know more about the study than about 95% of news articles are going to tell you.
This study didn’t measure COVID transmission or “protection” from COVID. It didn’t measure aerosols breathed out of a human being. It did not use SARS-COV-2 in any way. Not even close.
A gem of a reporter named Travis Fain at WRAL shows us how it’s done, however:
Study: Mask type really matters, ventilation may matter more
A completely fair and appropriate headline! The whole article is head and shoulders above any other press coverage I’ve browsed, including his summary of the study’s methodology and findings. He also got input from a virologist, an epidemiologist, and a chemist. If I were an editor (ah, someday), I’d snap up “Travis Fain, statehouse reporter” in a heartbeat and double his pay.
Here’s Fain’s summary:
University of Waterloo researchers used a sealed room, a masked mannequin and atomized olive oil to simulate breathing in their study. They used lasers to measure the aerosols, tiny particles that can float in the air, getting past masks and circulating in the room.
Then they added ventilation, courtesy of an air purifier equipped with the sort of high-powered HEPA filters used in labs and on airplanes.
Though the virus behind COVID-19 is present in larger droplets, which all masks help block, it's also thought to transmit through these much smaller aerosols. In fact a separate study released last week suggests airborne/aerosol transmission may be the dominant form of transmission.
The University of Waterloo study reached several conclusions:
N95 masks and similar masks filter exhaled air much better than cloth and surgical masks. Cloth and surgical masks caught 10 to 12 percent of aerosols breathed out in the experiment. Various N95, KN95 and R95 masks stopped 46 to 60 percent.
Even N95 masks leak, primarily around the nose, allowing aerosols to circulate. This led to "notably higher" concentrations of aerosols more than six feet away from the mannequin.
N95 masks with valves lost half their ability to stop aerosols. N95's without valves remain the recommended choice, "if worn correctly," but a loose-fitting N95 "provides a negligible" filtration efficiency.
The study found "increased ventilation/air-cleaning capacity significantly reduces the transmission risk in an indoor environment, surpassing the apparent mask filtration efficacy." The university put that another way in a press release on the study: "Even modest ventilation rates were found to be as effective as the best masks in reducing the risk of transmission.*"
*The study authors are playing a little fast and loose here. They don’t know how aerosol dispersion translates to sars-cov-2 transmission in real life (much less from a real person); they’re making an assumption.
How many reporters who covered this study would tell you the study was done with olive oil? How many could tell you the study found a loose-fitting KN95 had less than 4% filtration efficiency? How many would tell you that an air exchange rate of 2.45 room volumes per hour was associated with aerosol concentrations better than an R95 (basically an N95 that can be used with oily particles), and that the authors consider ~2 room volumes to be a “relatively low air change rate?”
Reporters are bad at covering science not because they are bad at science per se, but because they are lazy and/or on tight deadlines and on top of that, they let their biases lead them around by the nose. (The latter is a mistake I’ve made in the past and paid dearly for it.) Consider the smear of ivermectin, a drug that won a Nobel prize in 2015 for its treatment of river blindness with billions of doses administered, as “a horse dewormer,” or the deceitful coverage of hydroxychloroquine in the spring of 2020.
If you want to know why half the country wears face coverings like they’re some kind of magic talisman, it is because they (understandably) rely on the news media to interpret science for them. Journalists and public health officials are the trusted middle-man between academia and the laypeople. The problem is that there’s very limited accountability on scientific coverage, since most people can’t or won’t read the literature or check the data themselves. Other things, like emails, speeches, or videos can be more easily referenced to see if the media is covering something honestly or not. Scientific literature and data (like epi curves) are fertile ground for propaganda. Very few people are going to meticulously check your work and call you out on any B.S., and if they do, the vast majority of the public isn’t going to hear the critique. In other words, propaganda about a pandemic can be far more effective than propaganda about anything else non-science related. And if the CCP has taught us anything, it’s that propaganda allows you to accumulate and hold on to a massive level of power to enforce totalitarian policies and promote right-think.
This is a huge problem with no quick fix, in large part because our public schools’ have done a crap job teaching children to think critically and educate them to a level where they are conversant in major scientific concepts like sample size, statistical significance, control groups, etc. Perhaps the rise of school choice will make up some of this deficit. In the meantime, there are a few things that can help mitigate your susceptibility to pseudo-scientific bat guano, assuming you’re already conversant in said concepts:
Read the abstract
Try to grasp the methodology.
Read as much of the study you can (we all have lives)
Read science articles from the bottom up (paragraph by paragraph).
Review the sources (click on the links).
Check against official data (state covid-19 hospitalization curves, for example)
Don’t get suckered by novel statistics. For example, if Arizona hit a 7-day record high of rolling case averages, that doesn’t necessarily mean covid-19 is “out of control.” Look at the context: temporally (the case curve), how high testing levels are, how high neighboring states’ per capita cases are, the percent positive rate, etc.
Don’t get suckered by computer modeling studies, which are highly susceptible to GIGO (garbage in, garbage out).
Check biases, vested interests of academics and officials pushing narratives to reporters.
It seems obvious, but check the author even has a credible source.
Now, go forth and call bullshit.
Botched Coverage of the Waterloo Mask Study
You start to think about derivatives here: the university's release describing the study is one step removed from the study and the people who did it. The media coverage that follows the university's press release is a second step removed. Then, the short, biased summary of/link to the media coverage that you see on Twitter and Facebook is a third step removed. At every step, the study will get more and more misrepresented.
I am a scientist, and I can confirm that most drive-by media coverage of scientific studies is not coverage of scientific studies! Even university press releases will often mischaracterize a study done by their own faculty (and then mass media bouncing off those releases doesn't have a chance). Two examples that immediately come to mind were:
1. A modeling study from some University of Michigan folks early in COVID that built into their model that masking + social distancing would yield a transmission reduction of 90% (or something like that, I forget the exact number). The university press release said the study demonstrated the importance of masking. THAT IDEA WAS BUILT INTO THE ASSUMPTIONS OF THE MODEL YA PUNKS. But the university press release essentially mischaracterized the study. You think mass media is going to do a better job after that?
2. Of course the now infamous Duke masking study that lacked a control group. That gigantic scientific error didn't appear to give, again, even the university itself any pause in promoting the study as a "masking works" study.