ONS, have you no shame? Another lesson in losing the public trust
Yesterday’s post focused on the plausibility of the ONS’ stated estimates of vaccine effectiveness against death during the period of April 2021- March 2022. Naked Emperor, an astute reader, asked the logical question of why they chose to end their analysis in March 2022 when the dataset they were using went through December 2022.
Thank you Naked Emperor for prompting me to dig into this question a bit further. I will use the Deaths by vaccination status, England dataset upon which the ONS bases its analysis. Further, I will focus on the age groups above 60 (where almost all deaths occur) and focus on the three primary vaccination groups of interest (and where virtually all of the population resides): Unvaccinated, 2nd dose > 21 days, and 3rd dose >21 days. The charts that follow will show the ASMR (Age Standardized Mortality Rate) per 100,000 person-years by month and vaccination status (these will be circles, colored by vaccination status) with values on the left axis. In addition, a grey line will show reduction in ASMR for the vaccinated group vs. unvaccinated group for each month. These values are essentially an estimate of the raw, unadjusted, VE against death for a given month (note: the values are shown as negative, so -90% means the vaccinated group died 90% less frequently, i.e. VE ~ 90%). I also show a dotted line representing the average reduction in ASMR for the April 2021-March 2022 period and the April 2022-December 2022 period.
Double Dosed (Blue circle = Unvaccinated, Green circle = 2nd dose >21 days)
That’s so weird! While in the 1st period of time, we see significant reduction of deaths in the double vaxxed, especially in 60-69 year olds, in the 9 months that follow we actually see higher number of deaths among the double-dosed. Bad luck I guess!
Triple Dosed (Blue circle = Unvaccinated, Orange circle = 3rd dose >21 days)
Hmm, this is getting so odd. In the time period used for the ONS VE estimates, we saw reduced deaths anywhere from 89% up to 96%, but in the 9 months after that, it seems the VE reduced by ~1/2… going from 44% to 53%.
Sorry everyone, I try to keep this as a family friendly substack, but if anyone at the ONS is out there, this is for you:
Update: I don’t want to leave anyone the impression I’m hiding anything, so posting the same charts for the single dose >21 days (purple) vs. unvaccinated. See comment section for explanation on not including when I 1st posted.
If the ONS were trying to discredit their other report with this one, they could scarcely have done a better job. It's nice to see what the deaths data says without a lot of pointless adjustments, so thanks for that. However, I think there is a difference in the definition for "Covid death" in that the efficacy report merely requires mention of Covid-19 on death certificate while the deaths report requires primary cause. This may or may not influence results and the changes after March (i.e., maybe it would look even worse).
Using the deaths report (primary cause), unvaxxed seemingly aren't Covid-dying very much in 2022. At a certain point you would expect that the vaxxed should actually die more, simply because the control group is "all died out" while the vaxxed are still catching up with actually getting infected and either dying or gaining natural immunity. This is a known problem in long term vaccine efficacy appraisal https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7003633/
Maybe the ONS authors behind this new report wanted to "correct" for that problem. *edit: Actually, this problem might not have had any impact on their outputs had they used the months after March, 2022, anyway, because they are measuring time-to-event (HR), so a leveling off at the end shouldn't change anything - however, again, the results might be worse using the efficacy standard of "mentions" rather than "primary" cause of death. Really it just seems like a troll.
Aghh.. comment window just ate my comment..
I just wanted to point out that starting January 2022 was actually the perfect time to ascertain effectivenss as the cohort groups had finally stabilised with the vaccination campaign winding down. (Exception being the over 75's 2nd booster campaign in April-May).
Some may argue there were much fewer cases in 2022 but the test positivity figures suggest otherwise. The argument that the new omicron variants BA.2 and BA.5 brought much fewer severe cases is probably a valid.