Dear ONS, I don't believe you 🤨

The Office of National Statistics (ONS) in the UK has recently published data on mortality by vaccination status. When 1st published, they warned that the data could not be used un-adjusted to estimate vaccine effectiveness (VE) because it did not account for various confounders in the data. Now, the ONS has come back to make their estimates on VE, supposedly controlling for confounders. The topline figures for their estimates on VE against COVID deaths is summarized by Sarah Caul of the ONS in twitter thread:

Ok, so 58.7% for 1st dose, 88.6% for second, and 93.2% for 3rd dose. I will admit to not going through the details (I doubt enough detail is provided) as to how they arrived at those figures. However, I believe a valuable sanity check is show what those numbers would imply in an alternate reality where the vaccine never existed, given what actually happened.

Here’s how I propose to look at this alternate reality. Quite simply, the Projected Deaths (no vaccine) should be easily calculated from the Actual Deaths (with vaccine) and the assumed VE. Namely:

To walk through an example. Suppose we had a population people who all took the vaccine and 10 died. Suppose further that the vaccine is known to be 90% effective against death. This would imply that deaths were reduced by 90%, meaning we would have had 100 deaths if that population did not take the vaccine. I.e. Projected deaths = 10/(1-.9) = 100.

Ok, so now we can use the data linked to above to look at the period Sarah mentions (end of March 2021- end of March 2022) to see what the “no vaccine” alternative reality would have looked like. The ONS data classifies vaccination status into many categories. I will use conservative VE’s for each category:

To explain, anyone who is less than 21 days after 1st dose is treated as unvaccinated. People more than 21 days post 1st shot, and less than 21 days after 2nd shot are treated as “first dose” for VE. People more than 21 days post 2nd shot and less than 21 days post 3rd shot are treated as “second dose” for VE. Only folks more than 21 days post 3rd shot are treated as “third dose” for VE.

In the charts that follow, I show each of the 8 vaccination status levels, and their actual deaths April 2021- Mar 2022 (grey) vs. their projected deaths(orange) given the presumed VE. There are different scales on y-axis, so I have labeled the maximum values (most deadly month) to highlight the scale.

Unvaccinated and 1st Dose < 21 days

More of a sanity check. Vaccine had no impact in this group, so no orange is seen.

1st Dose >21 days & 2nd Dose <21 days

Given a 58.7% VE, Projected Deaths are ~2.4x higher than Actual Deaths.

2nd Dose between 21 days and 6 months & 2nd Dose more than 6 months

Given a 88.6% VE, Projected Deaths are ~8.8x higher than Actual Deaths.

3rd Dose <21 days & 3rd Dose > 21 days

For 3rd dose <21 days, with a given a 88.6% VE, Projected Deaths are ~8.8x higher than Actual Deaths. For 3rd dose >21 days, with a given 93.2% VE, Projected Deaths are ~14.7x higher than Actual Deaths.

All together now

This shows the proposed aggregate impact of vaccines in the April 2021-March 2022 time frame. The aggregate VE varies across the months as the population of each vaccination level changes.

So, if you were one to create models of estimated lives saved by the vaccines. With the VE’s presumed from above, the difference between the orange and grey areas would be the estimated lives saved.

How believable is this? To gain some perspective, let’s look back at the COVID deaths before April 2021. Using data from the ONS’ Deaths registered weekly in England and Wales, provisional, we can bring in the COVID deaths prior to April 2021. We will assume, despite vaccinations beginning in December 2020, that they had no effect until April of 2021. Here is the expanded timeline:

So, based on the ONS’ VE estimated, they believe that January, February and March 2022 would have all been more deadly than the previous most deadly month of the pandemic (January 2021). This despite Omicron being far less deadly. 🤔

For anyone who believes this, I might have a real estate deal you’d be interested in…