The Office of National Statistics (ONS) in the UK has recently published data on mortality by vaccination status. When 1st published, they warned that the data could not be used un-adjusted to estimate vaccine effectiveness (VE) because it did not account for various confounders in the data. Now, the ONS has come back to make
"Also why is the data only up to March when they have the data to Dec '22?"
Good catch. They indeed have data in the spreadsheet through December. A skeptic might think perhaps using the additional data, based on their own calculations, reduced their estimated VE. Luckily I'm not skeptical by nature 🤣
Picky is good. Are you 1) Simply asking for the cumulative numbers for each or 2) Pointing out that an area curve is probably not the right display (the data displayed are discrete values, not continuous, so bars for each month are probably more appropriate)?
If it's the former, I can add that very easily. If it's the latter, I really don't feel like replacing all the charts 🤣
If we can get relative areas for actual and projected values, can we use that to estimate how far off ONS' projections are and also estimate VE effectiveness?
Even if values are discrete rather than continuous, they should still be useful for error estimation.
Might be a misunderstanding here. What I have plotted is not the ONS' projected values (at least not their explicitly stated ones), so we would not be looking at the "how far off" they were. The projected values above are my back calculation of what deaths, without a vaccine, would have been implied to occur given the VE estimates the ONS is providing. If ONS says VE is 90%, and there are 10 deaths among that group, they are implicitly saying there would have been 100 deaths if none of those people were vaccinated.
To explain further... what I am essentially saying here is that in order for one to have confidence in the VE estimates from the ONS, you have to believe the path of the orange about is the best estimate of how COVID deaths would have proceeded had there never been a vaccine. I find that path highly implausible, but others may have a different opinion, but no one should be able to escape the logic of this.... you can't choose one or the other, arithmetic implies that either both are true (VE estimate, orange path), or neither are true.
I believe Figure 3 definitely is a powerful display of the Healthy User Bia.. Correcting for this is exactly what they claim to (or should) be doing in their estimated VE. The fact that they are seemingly correcting for this and yet we still see the (to me) totally implausible implications above leaves me unsure what's really going on.
Mar 10, 2023·edited Mar 10, 2023Liked by T Coddington
Agreed, I think they should be correcting for this in estimated VE but I think figure 3 shows "the amount of confounding left in our model" i.e. it how badly they have failed to fully correct for all confounders, e.g. Healthy User Bias in (see notes under 4. Non-COVID-19 mortality above figure 3):
"As coronavirus (COVID-19) vaccination should not provide protection against non-COVID-19 mortality, we can use non-COVID-19 mortality as a control outcome to assess the amount of confounding left in our model. The risk of death would not be expected to differ between vaccination status groups if all confounding factors were accounted for, the vaccine has no effect on non-COVID-19 mortality, and all deaths caused by COVID-19 were accurately classified as deaths involving COVID-19. Given instances of adverse events are rare, we can assume that the non-COVID-19 risk of death should be similar to, or close to, zero if there is no residual confounding."...
"Since we do not expect the estimates to be primarily related to the effect of the vaccine, we refer to this as "reduction in risk of death" (compared with the unvaccinated population) rather than vaccine effectiveness."...
"However, even when including all adjustments for confounding factors, we observe a reduction in risk of non-COVID-19 death for vaccinated groups compared with the unvaccinated population. The risk of non-COVID-19 death is lower for all vaccinated groups compared with unvaccinated groups except the "over three months after the first dose" group.
This indicates the presence of residual confounding, despite taking into account recent socio-demographic factors and different sources of health data."
Now that I understand how to read Figure 3 better, it strikes me as crazy how much residual confounding there appears to be, even after accounting for a long list of factors. Makes me wonder what else is really going on. For example, suppose the population of unvaccinated was significantly underestimated, would that explain this? I honestly don't know, need to think it through.
Thanks. Something feels wrong to me about his method of adjustment, but I can't put my finger on it yet.... will give it some more thought. I feel like the hazard ratios need to be weighted given way more folks die of non-covid reasons than of covid but I will admit that this is not my 1st language in terms of statistical analysis. Fee free to email me if you have ideas or want to discuss.
Lol. It's obviously because of how well the NPIs worked until the vaccine came along?! 🤪
You also live much longer if you're jabbed.
https://nakedemperor.substack.com/p/take-the-jab-and-live-forever
Also why is the data only up to March when they have the data to Dec '22?
"Also why is the data only up to March when they have the data to Dec '22?"
Good catch. They indeed have data in the spreadsheet through December. A skeptic might think perhaps using the additional data, based on their own calculations, reduced their estimated VE. Luckily I'm not skeptical by nature 🤣
Exactly, wouldn't want that sort of data with the Spring boosters coming up
Did you calculate the area under the curve for each spike section (2020, 2021, and both actual and projected 2022)? I guess I'm kind of picky.
Picky is good. Are you 1) Simply asking for the cumulative numbers for each or 2) Pointing out that an area curve is probably not the right display (the data displayed are discrete values, not continuous, so bars for each month are probably more appropriate)?
If it's the former, I can add that very easily. If it's the latter, I really don't feel like replacing all the charts 🤣
If we can get relative areas for actual and projected values, can we use that to estimate how far off ONS' projections are and also estimate VE effectiveness?
Even if values are discrete rather than continuous, they should still be useful for error estimation.
Might be a misunderstanding here. What I have plotted is not the ONS' projected values (at least not their explicitly stated ones), so we would not be looking at the "how far off" they were. The projected values above are my back calculation of what deaths, without a vaccine, would have been implied to occur given the VE estimates the ONS is providing. If ONS says VE is 90%, and there are 10 deaths among that group, they are implicitly saying there would have been 100 deaths if none of those people were vaccinated.
To explain further... what I am essentially saying here is that in order for one to have confidence in the VE estimates from the ONS, you have to believe the path of the orange about is the best estimate of how COVID deaths would have proceeded had there never been a vaccine. I find that path highly implausible, but others may have a different opinion, but no one should be able to escape the logic of this.... you can't choose one or the other, arithmetic implies that either both are true (VE estimate, orange path), or neither are true.
Right.
@TCoddington
What to you make of Figure 3: Reduction in risk of non-coronavirus (COVID-19) death by vaccination status compared with unvaccinated
Is this the underlying "Healthy User Bias"? Seems like it would reduce "corrected" Vaccine Effectiveness against Covid-19 to less than 50%.
I believe Figure 3 definitely is a powerful display of the Healthy User Bia.. Correcting for this is exactly what they claim to (or should) be doing in their estimated VE. The fact that they are seemingly correcting for this and yet we still see the (to me) totally implausible implications above leaves me unsure what's really going on.
Agreed, I think they should be correcting for this in estimated VE but I think figure 3 shows "the amount of confounding left in our model" i.e. it how badly they have failed to fully correct for all confounders, e.g. Healthy User Bias in (see notes under 4. Non-COVID-19 mortality above figure 3):
"As coronavirus (COVID-19) vaccination should not provide protection against non-COVID-19 mortality, we can use non-COVID-19 mortality as a control outcome to assess the amount of confounding left in our model. The risk of death would not be expected to differ between vaccination status groups if all confounding factors were accounted for, the vaccine has no effect on non-COVID-19 mortality, and all deaths caused by COVID-19 were accurately classified as deaths involving COVID-19. Given instances of adverse events are rare, we can assume that the non-COVID-19 risk of death should be similar to, or close to, zero if there is no residual confounding."...
"Since we do not expect the estimates to be primarily related to the effect of the vaccine, we refer to this as "reduction in risk of death" (compared with the unvaccinated population) rather than vaccine effectiveness."...
"However, even when including all adjustments for confounding factors, we observe a reduction in risk of non-COVID-19 death for vaccinated groups compared with the unvaccinated population. The risk of non-COVID-19 death is lower for all vaccinated groups compared with unvaccinated groups except the "over three months after the first dose" group.
This indicates the presence of residual confounding, despite taking into account recent socio-demographic factors and different sources of health data."
That last line is SOO understated 😉
Now that I understand how to read Figure 3 better, it strikes me as crazy how much residual confounding there appears to be, even after accounting for a long list of factors. Makes me wonder what else is really going on. For example, suppose the population of unvaccinated was significantly underestimated, would that explain this? I honestly don't know, need to think it through.
Prof Jeffrey Morris had calculated the adjusted VE here: https://twitter.com/jsm2334/status/1633498850924060674?s=20
(It does not reduce VE by as much as one might think)
Thanks. Something feels wrong to me about his method of adjustment, but I can't put my finger on it yet.... will give it some more thought. I feel like the hazard ratios need to be weighted given way more folks die of non-covid reasons than of covid but I will admit that this is not my 1st language in terms of statistical analysis. Fee free to email me if you have ideas or want to discuss.