No, masks don't protect at the individual level
Calling bullshit on this nonsense
Since the Cochrane Review on masking came out saying that, essentially, there is no evidence that masking does anything to prevent the spread of COVID, I’ve seen on Twitter and now today in this Slate piece arguments that say, “Well, masks may not help at the population level, but they still help an individual”. Quote from slate:
“The Cochrane Review tells us two important things. First, there have been very few high-quality studies examining the effectiveness of masks during the COVID pandemic, and second, from the little high-quality data we do have, we don’t see large impacts of masking in preventing viral infections on the population level,” Jennifer Nuzzo, director of the pandemic center at Brown University School of Public Health, told Slate. “This doesn’t necessarily mean masks don’t protect individuals. But it could mean that the way they’re used at the population level is not effective. We need more randomized trials to understand why.”
This is, in a word, bullshit. The reason this is surely bullshit is because it is (almost) always easier to pick up a signal at a population level than at an individual level. The opposite of what they are currently saying would be way more plausible, i.e. it would be much more likely to be in a situation where we could not detect/prove any effect at an individual level, yet we see a clear signal at a population level. This is simply the nature of aggregation of data.
You can add together a large number of insignificant values and arrive at significant value. For example, suppose we required (somehow) everyone in the world to check that their tires were properly inflated before they began to drive anywhere. Trying to measure the impact on the probability of an accident for a single driver for a single time driving would surely result in a value so close to zero that we would not have any confidence saying this practice did anything. However, if we aggregate all the data for all drivers for many trips, it’s very possible that we would be able to detect a small, but statistically significant reduction in traffic accidents.
On the other hand, it is not possible (except in a case I’ll mention below*), to aggregate data that is significant at an individual level and not be significant at a population level. Back to our example above… suppose we proved that by checking the air in his tires, T Coddington reduces his chance of an accident by 5% each time he drives. How in the world, do you add up a bunch of T Coddingtons and claim and arrive at “no impact” at a population level? That is essentially what the folks saying masks might work at an individual level, but not population level want you to believe. Masks work for Jim, Bob, Sally, and Betsy each individually, but not for the group Jim, Bob, Sally and Betsy. Bullshit.
*The exception here would be if the individual variables were negatively correlated, resulting in them “canceling” each other out at the aggregate level. This case would imply that while masks reduced the risk for Jim & Bob, they increased the risks for Sally & Betsy. I don’t believe this is the argument these folks are missing.
Readers with probability & statistics backgrounds, please tell me if I have a blind spot here. How else can one plausibly say that the effect is significant at an individual level, but when I add up all the individuals, it loses significance.?
So, now I need to add "Slate" to my long list of BS publications and channels. The Atlantic, The BBC, CNN, MSNBC, The New York Times, Sky News UK, The Guardian, The Times, etc. etc.
It's getting hard to keep track of them all.
Thanks for the refresher on stats re aggregated v singular but I think you may have taken Nuzzo out of context. I'm not convinced she was proposing what you elegantly refuted. Maybe I am being too generous but her statement reminds me of the seatbelt saga. There was a reasonable signal in the auto racing sector suggesting that seat belts saved lives. Drivers wore a double sash and lap belt fitted snuggly. So eventually single lap and sash were rolled out in consumer vehicles, with some countries mandating their use. Surprisingly to some, in contrast to the racing industry, population level signals were marginal at best. Then along came inertia reel seat belts and the safety signal strengthened markedly. Why, because now the device was, at a population level, taking the confounder of 'fitted incorrectly' out of the aggregated data. To be fair to Nuzzo on this matter, numerous mask wearers reluctantly wore ill fitted masks or trusted the blind faith talisman effect of wearing a mask irrespective of material quality or bypass flow. So, just like with seatbelts, if a majority don't wear a device as is necessary for correct functioning, then a population study will not reveal the true capacity of a fit for purpose device. The noise of confounding factors may drown out any signal attributes of the device. This Cochran review looked at mask studies in the same way it looked at ivermectin studies. With masks, there was no QAQC on materials or fitting and with ivermectin a majority of included studies were ludicrously treating after 5 days of symptoms. Individual studies and meta analyses fail to elucidate effectiveness of interventions if context and nuance is ignored. What the Cochran review does tell us is that mask mandates or masking end masse doesn't protect the population. The 'why they don't protect' at population level has not been answered. But equally this review does not determine whether a snugly fitting, well constructed, hygienically worn N95 mask offers no protection to an individual. Another study may do so but not this review.