One of the most notable things about the COVID-19 pandemic has been how fast two science denialist movements made common cause and, in essence, fused to become one movement. I’m referring, of course, to the antivaccine cranks/grifters and COVID-19 deniers/minimizers/antimaskers/cranks/grifters, who rapidly formed an unholy alliance that ultimately added QAnon conspiracy theorists to the mix to form one of the most toxic groups of conspiracy theorists ever seen. I note that by “toxic,” I don’t just mean toxic personalities, but I also mean toxic to science, medicine, public health, and the politics of trying to respond to the pandemic, with antivaxxers frequently augmenting COVID-19 deniers at various rallies and events, all the while crying “censorship,” infusing the COVID-19 denialist movement with antivaccine pseudoscience and conspiracy theories, and even launching a pre-emptive disinformation war against COVID-19 vaccines. Just two examples, antivax leader and propagandist Robert F. Kennedy, Jr. has gone all-in on COVID-19 minimization/denial, and Del Bigtree, who made the antivaccine propaganda film VAXXED with Andrew Wakefield, has urged his listeners to “catch this cold” in order to build up herd immunity among the “healthy,” because, to him, COVID-19 is not dangerous except to those who deserve to be endangered, specifically those with chronic conditions due to overeating, lack of exercise, excess drink, and the like. It was blaming the victim at its most blatant, very typical of antivaccine activists, typical of antivaccine nonsense. That’s not the only antivaccine tactic that COVID-19 denialist/minimizers have adopted, however; they’ve also started abusing the ecological fallacy, the same way antivaxxers have been doing for at least 12 years, which is the first time I noticed it, this time in the context of claiming that lockdowns don’t work.
I came to this realization when I saw this Tweet about a new study claiming that “lockdowns don’t work” to slow the spread of COVID-19. Of course, a lot of this claim depends on what you mean by “lockdown,” as anti-“lockdown” advocates love to portray almost any public health restriction designed to spread the slow of COVID-19, such as the closure of churches, gyms, and the like, as well as the banning of indoor dining at restaurants and the requirement that people not leave their houses except for “essential” business and that people who can work at home do so as “lockdowns” akin to what the authoritarian Chinese government did in Wuhan in January, in which people were not allowed to leave, with police arresting people who did. In any event, I saw this Tweet about one such study claiming that “lockdowns don’t work” because, if you believe the study he cites, lockdown stringency doesn’t correlate with COVID-19 mortality at the level of the nation:
Let’s just say that Peter Hitchens does not share his late brother Christopher’s critical thinking faculties, as of late he’s been very much in the COVID-19 minimization camp. Be that as it may, his Tweet rapidly led to this takedown of the study concluding that “lockdowns don’t work” and don’t impact COVID-19 mortality:
I’ve quoted him before in the context of taking on claims about a study that antimaskers have represented as concluding that “masks don’t work.” (Hint: They do, albeit imperfectly.) In this case, I fully recommend reading his entire Twitter thread, which explains why the study in question is such crap. When someone takes down a bad study as well as was done in this case, I sometimes wonder if there’s anything else for me to add. In this case, there is, because, as I read this Twitter thread, memories came back to me of studies I’ve deconstructed before, and it occurred to me: This antilockdown study is very much like an antivaccine study that I deconstructed nine years ago, so much so that I knew what I had to write. I had to provide context, a little history lesson, to show how anti-lockdown COVID-19 pseudoscientists are now using the same misapplication of scientific methods that antivaxxers have been using for decades, in this case, falling prey to the ecological fallacy, either through incompetence, ideology, grift, or any combination of these. How better to do this than to compare the study in question about “lockdowns” with the antivaccine study I discussed so long ago?
Before I dig in, let’s see if, from this description in the next couple of Tweets, any longtime readers recognize which antivaccine study this anti-lockdown study reminded me of:
Yes, I’m referring to a study from 2011 by the not-so-dynamic antivax duo of Neil Z. Miller and Gary S. Goldman that, as I described in my rather Insolent takedown, used the ecological fallacy (plus cherry picking of data sources) to find a correlation between the number of vaccines in a country’s recommended childhood vaccination schedule and the infant mortality rate of that country. At the time, I pointed out that the investigators had significant ties with the antivaccine movement. For example, National Vaccine Information Center (NVIC) donated $2,500 and Michael Belkin donated $500 (in memory of his daughter, Lyla) for open access to the journal article. The NVIC, as you recall, was founded by Barbara Loe Fisher and is one of the oldest and most influential anti-vaccine groups in the U.S., reinvigorated over the last decade by a torrent of cash from über-quack Joe Mercola. Michael Belkin, longtime readers might also recall, is the man responsible for The Refusers, a one man anti-vaccine rock act best known for execrably bad songs with risible titles like Vaccine Gestapo and Get Your Mandates Out of My Body. (Rage Against the Machine or The Clash, he ain’t.) At the time, Miller had a long history of anti-vaccine activism, having previously written books with titles like Vaccine Roulette: Gambling With Your Child’s Life, Immunization Theory vs Reality: Expose on Vaccinations, and Vaccines: Are They Really Safe and Effective?, among others. He was also the director of the ThinkTwice Global Vaccine Institute. His partner in pseudoscience Gary S. Goldman was the President and Founder of Medical Veritas, a rabidly anti-vaccine “journal” that was (and is) into HIV/AIDS denialism. In other words, these were ideologues and activists, not scientists.
What about the investigators who published this anti-lockdown study? I had never heard of them before seeing this study, but their credentials appear reasonable—superficially, at least—with institutions including the University of Toulouse, the Institute for Research in bioMedicine and Epidemiology of Sport, and Hôtel-Dieu, Assistance Publique—Hôpitaux de Paris. The departments are off, though, including the Centre de Recherche sur la Cognition Animale (Research Center on Animal Cognition) at the University of Toulouse and the Institute for Research in bioMedicine and Epidemiology of Sport. OK, that latter has epidemiology in the title, but clearly pandemics are not going to be a strength of this institute, which is dedicated to sports medicine. So, basically, they’re not as bad as Miller and Goldman and, as far as I can tell, but at least some of them are ideologues. For instance, the corresponding author, Jean-François Toussaint, has been known to play fast and loose with COVID-19 statistics to “prove” that COVID-19 is becoming less deadly and to deny that public health science-based restrictions can slow the spread of the virus.
It occurred to me as I wrote this that using the Health Nerd’s Tweets pointing out the study’s flaws would be an excellent way for me to guide you through the similarities between Miller and Goldman’s study and this antimask study. So let’s do it! Here we go, first:
I immediately took notice of that part about the CIA Factbook, because that’s exactly where Goldman and Miller got their data from regarding infant mortality, and, as was the case with this anti-lockdown study, the data used didn’t really match up very well with the years examined. In fact, Larochelambert et al should be really embarrassed, because arguably Goldman and Miller did a better job matching time periods than they did! No, seriously. Goldman and Miller used data from The World Factbook that were as recent as a couple of years before their paper, while most of the data used by Larochelambert et al were six years old or older. I guess they age of the data matches more closely for obesity rates, given that the World Health Organization’s (WHO) data repository was last updated in 2018.
What else? Basically, both papers seek to demonstrate a “dose-response” correlation between the “independent” variable being studied as a predictor of outcomes. (I use the “scare quotes” because these variables in these papers are not truly “independent” in that they are correlated with a number of other factors.) In the case of Goldman and Miller’s crapfest of an article, the “dose” variable was the number of vaccines in each country’s recommended childhood vaccine schedule, which was then correlated with each country’s infant mortality rate. In Larochelambert et al, it’s the “stringency index” of the lockdowns in each country that is being correlated to each country’s COVID-19 mortality rate. So let’s look at what the Health Nerd says about the way Toussaint and his fellow anti-lockdown authors cover “stringency”:
You can see from this succinct summary why the ecological fallacy is a problem in epidemiology: You can’t infer the properties of individuals from the properties of the group, which means that looking for correlations between various risk factors and groups is inherently perilous from a scientific standpoint. As the Health Nerd puts it, the average of a group isn’t always representative of the individuals, or, as the EpiWonk (whose blog is missed) defined the ecological fallacy: Thinking that relationships observed for groups necessarily hold for individuals.
Although I’m not an epidemiologist, I’ve noted that relationships observed for group-level data are also not always representative of much smaller groups within the larger groups. Indeed, antivaxxers fall prey to the ecological fallacy all the time in a less formalized way than publishing bad epidemiological studies when they point to high overall vaccine uptake in a large area (say, the State of California) to try to argue that vaccines don’t work when outbreaks of vaccine-preventable diseases occur. And, yes, in the case of outbreaks of vaccine-preventable disease it’s often true that the average overall vaccine rate for a large state like California is high. However, that argument ignores the fact that, even in the middle of an area with a high vaccine uptake, smaller areas with low vaccine uptake will be vulnerable to outbreaks of vaccine-preventable diseases due to diminished community immunity (formerly—and sometimes still—known as herd immunity). As these authors count countries with small areas with high stringency lockdowns and much larger areas with looser restrictions, the average of which ends up being high stringency index lockdown, “high stringency index” countries, antivaxxers use states or countries with high overall vaccine uptake but small areas of low vaccine uptake to represent the state or country as a “high vaccine uptake.” As is the case with this lockdown paper, averaging over a large area political unit (be it country or state) pulls the average in the direction the authors want, thanks to small areas within the unit analyzed that are very different from the overall unit.
I will say that Miller and Goldman did go one beyond in that they were a bit—shall we say?—creative in how they assigned their independent variable to countries. What I mean is that the way they counted the number of vaccines in the recommended schedules for each country was designed to maximize the apparent number of vaccines, as described here (remember, though, that this study is over nine years old when considering the specific numbers discussed and that vaccine schedules change):
There are a number of things wrong with this procedure – first of all, the way Miller and Goldman are counting vaccines is completely arbitrary and riddled with mistakes.
Arbitrary: they count number of vaccines in US bins (DTaP is one, hib is separate) and non-specific designations (some “polio” is still given as OPV in Singapore), rather than antigens. If they did that, Japan, still giving the live bacterial vaccine BCG, would immediately go to the top of the list. That wouldn’t fit the agenda, of course. But if you go by “shot” rather than by antigen, why are DTaP, IPV, hepB and hib counted as 4 shots for example in Austria, when they are given as Infanrix hexa, in one syringe?
Mistakes: The German childhood vaccination schedule recommends DTaP, hib, IPV AND hepB, as well as PCV at 2, 3 and 4 months, putting them squarely into the 21 – 23 bin. The fourth round of shots is recommended at 11 to 14 months, and MenC, MMR and Varicella are recommended with a lower age limit of 11 months, too, which means that a number of German kids will fall into the highest bin, at least as long as you count the Miller/Goldman way.
There was less opportunity for the authors of this anti-lockdown paper to game the independent variable than there were for Miller and Goldstein to game their independent variable of number of vaccines.
Another similarity leapt out at me. Miller and Goldman graphed infant mortality as a function of the number of vaccines received, while Toussaint’s group graphed COVID-19 mortality as a function of the “lockdown stringency index.” Miller and Goldman used linear regression to analyze for a correlation while Toussaint’s group used the more sophisticated logistic regression. In both cases, the researchers (if you can call them that) assumed that there was a specific mathematical relationship between the independent and dependent variable without a plausible mechanism to explain why they chose that mathematical relationship. True, it was more risible that Miller and Goldman assumed that there had to be a linear relationship between the number of vaccines and infant mortality, but, really, when you come right down to it, it wasn’t a whole lot less risible that Larochelambert et al assumed that a non-symmetrical logistic regression would find a curve “fit.” Sure, it sounds a lot more statistic-y and science-y than a linear regression.
Then there’s this:
What is endogeneity? Briefly, endogeneity occurs when variable, observed or unobserved, that is not included in a model, is related to a variable that is incorporated in our model. For example, in the case of Miller and Goldman, as I pointed out, so many more variables impact infant mortality than just potentially vaccines, and some of these variables could well be related to vaccine uptake. Moreover, Miller and Goldman also cherry picked the countries they examined, carefully excluding potential outliers that would interfere with their preconceived results. As for Larochelambert et al, there are indeed other factors that were not considered:
In any event, so many more variables might correlate with COVID-19 deaths, some of which are almost certainly also going to correlate with the “stringency index.”
Then, of course, there’s timing:
I could go on and on and on and on, obviously. Larochelambert et al have published a truly crappy and uninformative epidemiological study of lockdowns as a predictor of nationwide COVID-19 mortality using methods very similar to, albeit more statistically sophisticated than, those used by antivaxxers. However, the higher degree of mathematical sophistication doesn’t change the fact that what we are looking at here is ecological fallacy, nothing more, a research technique “pioneered” by antivaxxers years and years and years ago by antivaxxers as incompetent as Mark and David Geier (whose 2008 paper provided me much fodder). The bottom line is that COVID-19 pandemic deniers/minimizers are now using the same sort of bad science that antivaxxers have been using for a very long time.