We don't know if COVID vaccinations prevent COVID severe disease and deaths

Too many confounders muddy the water

The COVID vaccines received emergency authorization approval based on the criteria of reducing symptomatic COVID. Testing reduction of severe cases (e.g. hospitalizations) and deaths was not part of the trial endpoints. Now that the vaccines have been used widely for ~18 months, it has become pretty clear to all but the most zealous of the vaccine supporters that cases are not being reduced by the vaccines. In fact, I have had several posts (as have many others), showing support that the vaccines may actually be increasing cases now. For example, here:

Vaccines and Cases: A look at US States

Since it should be clear that cases are not being prevented, the argument for taking the vaccine has now landed on the premise that it reduces COVID hospitalizations and deaths. Of course, this premise also assumes the vaccines have no effect on non-COVID hospitalizations and deaths. This should not be assumed, but I won’t address that in this post. I still see many of the doctors on Twitter who have been far more rational and balanced than most throughout the pandemic treat it as a given that the vaccines are very effective against COVID hospitalization and deaths. The data to support this is based on observational data whereby rates of COVID hospitalization and deaths among the vaccinated vs. unvaccinated populations are compared.

The purpose of this post is not to show that COVID hospitalizations and deaths are not being prevented with the vaccines, but rather to show that observational data alone cannot tell us this because there are too many confounders in the data.

Data Sources:

• Health characteristics of populations at the US county level. This dataset is provided by the University of Wisconsin Population of Health Institute.

• CDC COVID Vaccinations by US County

In the below charts I show the correlation between vaccination rates for adults within a US County and the following measures which can impact general health:

• Household Median Income

• % of Smokers

• % Adults with Obesity

• Life Expectancy

• Age Adjusted Death Rate

• % Physically Inactive

The charts exclude HI & AK, are broken down by Regions and filter out counties with less than 25,000 people (I think the data on very small counties are likely less reliable, removing them still leaves us with 94% of US population covered). Below each circle represents a US county, they are sized by population. If a given correlation is statistically significant (p-value < 0.05), I have placed a blue star in upper right corner. If the p-value is <0.01, I have made the star green. No star means the correlation is not considered statistically significant.

Here are how the regions are defined:

Starting in the West and moving across the country:

• Southwest: Strong evidence that higher vaxxed counties are less obese, have longer life expectancy, and are more physically active.

• Northwest: Very Strong evidence that higher vaxxed counties are richer, less smokers, have longer life expectancy, lower death rates, and are more physically active.

• West: Very Strong evidence that higher vaxxed counties are richer, less smokers, less obesity, have longer life expectancy, lower death rates, and are more physically active.

• South: Very Strong evidence that higher vaxxed counties are richer, less smokers, less obesity, have longer life expectancy, lower death rates, and are more physically active.

• Upper Midwest: Very Strong evidence that higher vaxxed counties are richer, less smokers, less obesity, have longer life expectancy, lower death rates, and are more physically active.

• Northern Rockies and Plains: Strong evidence that higher vaxxed counties are richer.

• South Atlantic: Very Strong evidence that higher vaxxed counties are richer, less smokers, less obesity, have longer life expectancy, lower death rates, and are more physically active.

• Southeast: Very Strong evidence that higher vaxxed counties are richer, less smokers, less obesity, have longer life expectancy, lower death rates, and are more physically active.

• Ohio Valley: Very Strong evidence that higher vaxxed counties are richer, less smokers, less obesity, have longer life expectancy, lower death rates, and are more physically active.

• New England: Strong evidence that higher vaxxed counties are richer, , have longer life expectancy, and lower death rates. Note: I am skeptical of the CDC’s data on some of the very low vaccinated counties (small dots between 40-60% vaccinated). I have encountered before some strange ones in New England. If I removed these, it’s likely we would see stronger correlations.

• Middle Atlantic: Very Strong evidence that higher vaxxed counties are richer, less smokers, less obesity, have longer life expectancy, lower death rates, and are more physically active.

In summary, all of the regions show evidence that the highly vaccinated counties are different in important ways from the less vaccinated counties (implying it’s likely that the vaccinated population is different in important ways than the unvaccinated population). In some regions, the differences are overwhelming. So, if we believe it’s likely the vaccinated population is richer, less likely to smoke, less likely to be obese, have longer life expectancies, lower death rates, and are more physically active, then how can we trust observational data on the VE against hospitalization and death? That’s why clinical trials (or observational trials that attempt to control for these confounders) are necessary.

*** Update 8/8/22. I’m adding a graph of just California. This is aimed to support the point of confounders in the data in San Diego in Mathew Crawford’s recent post.