It’s odd that I’ve written so little about Toby Rogers, given what I now realize to be his longtime antivaccine prolificacy. Maybe it’s because he started out in Australia and until recently appears not to have had a lot of impact here in the US on antivaccine messaging (although he does have his own page on Robert F. Kennedy Jr.’s antivax website, where he’s described as doing “grassroots political organizing with medical freedom groups across the country working to stop the autism epidemic” in the US). Maybe it’s some other reason.
I don’t know for sure, but I do know that the one and only time I’ve mentioned Toby Rogers on this blog was last year, when he watched the Advisory Committee on Immunization Practices (ACIP) November meeting regarding whether an emergency use authorization (EUA) should be granted for Pfizer‘s COVID-19 vaccine for children 5-11 years old. As you can probably guess, Rogers was not thrilled, characterizing ACIP meeting as “not a scientific review” but rather “banal bureaucrats announcing plans for a Blitzkrieg and the bought white coats were cheering them on.” I barely mentioned him in my post, didn’t look into who he was, and promptly forgot about him.
Months rolled by, and I didn’t think about Rogers again; that is, at least until I somehow had what has to be one of the silliest arguments against the CDC and vaccine safety studies that I’ve ever encountered in my two decades of writing about such things. It turns out that Toby Rogers has a Substack (because of course he does), where he exults:
As censorship has increased on Twitter, FB, and IG, the conversation has moved to Substack. I post my long-form essays here. My intention is to always have a free option alongside opportunities to financially support my work. The beauty of Substack is that all posts go directly to your email inbox. The revolution has begun and I have a lot of say about it.
Here’s a hint. Whenever someone has a Substack whose existence he justifies by invoking “censorship” on other social media, it’s about a 99% likelihood that his Substack will be filled with contrarian nonsense, and so it is with Rogers, arguably the most risible of which is a post from yesterday entitled The CDC’s failure to use multivariate analysis shows the total depravity of the vaccine program. Let’s just say that accusing the CDC of a failure of which it’s not guilty and then accusing it of “total depravity” tell me all I need to know about Toby Rogers.
And yet I still went past the title to read the post. First, he cites Mathew Crawford as having done a “brilliant series of articles” arguing that vaccine efficacy “may be zero.” I might have to do a deeper dive into those articles at some point, but let’s just say that Mathew Crawford is a statistician who appears to misapply statistics to medical questions. For instance, early in the pandemic, he was a big supporter of hydroxychloroquine, even after it had become clear that the drug didn’t work against COVID-19. Of more interest to me is his bragging right at the outset:
As many of you know, I got a master of public policy degree from UC Berkeley in 2012. What you may not know is that UC Berkeley is usually the top rated quantitative public policy program in the country. So what that means in this case is that it is heavily focused on econometrics.
Econometrics is beautiful. It usually starts with a large data set for a population. Then one uses sophisticated statistical software (STATA, SPSS) to analyze the data. Econometrics involves massive equations that are looking for the effect of a particular variable, while controlling for a wide range of additional variables.
So Toby Rogers knows statistics. Great. Statistics, however, is a tool to be used to set up and analyze the results of research in specific fields in which specific hypotheses are tested and research questions addressed. Does it mean that he understands medical and epidemiological research design? I think you know the answer to that question. It’s also apparent from the introduction to Rogers’ article that he’s making one massive appeal to authority—his authority. In this article, he misapplies that authority in one area, econometrics, to another area, vaccine safety. The only similarity between the two seems to be that statistics are used in both.
It’s interesting to see where Rogers is coming from before launching into his application of his statistics background to epidemiology and pharmacovigilance:
What’s fascinating about econometrics is that if one really builds the model correctly, the largest effect size that one will ever see for a single variable is about 0.3. This means that X intervention explains 30% of the outcome — the rest of the variables explain the rest. We live in a multivariate world.
Even with all of that complexity, there are still those (such as the great economist Steve Keen) who argue that econometrics, with its 15, 50, or even 100 variables is still completely inadequate and that if one really wants to understand how the world works, one must utilize the tools of physics (and supercomputers) to build models with millions of variables and account for things like chaos theory (this approach is called econophysics).
I can see physicians and biomedical scientists out there facepalming away, and well you should! Just because in econometrics the largest effect size one will likely see is about one third does not imply that this is true in medicine. While it’s true that many, if not most, effects in medicine detected through epidemiological study are less than 30%, there are a number of examples that are much more striking than that. The effect that smoking has on lung cancer risk, for example, is famously around a ten-fold elevation of risk. Are the epidemiological studies dating back 70+ years that found this effect therefore too simple because the effect is so much more than a 30% increase in risk?
It’s almost as though Rogers approached vaccine safety science from personal incredulity; just because he could not believe that vaccines can have huge effects in preventing disease (e.g., the MMR vaccine series being 95% effective at preventing measles) with an incredible record of safety must mean that there’s something wrong with the science behind vaccines and vaccine safety. Also, he comes from a world without much in the way of constraints. He can add as many variables as he wants to his econometrics models, almost without limit. Medicine doesn’t work like that.
I’ll revisit that last point before this is over, but first let’s take a look at Toby Rogers’ “reasoning,” such as it is. It boils down to, well, I’ll tell you what it boils down to after letting you read it first without my spin on it:
So that was my background when, in 2015, in the midst of a Ph.D. program in political economy, I started researching autism and decided to read a vaccine safety study for the first time. There are about 20 studies that the CDC points to as showing that there is no relationship between vaccines and autism. I assumed that I would not be able to read or understand these studies at all. Given what I knew about the complexity of econometrics, and knowing that the human body, biology, chemistry, and the immune system are even more complex than economics, I assumed that I would be looking at equations involving calculus, that used advanced statistical software to analyze hundreds or perhaps thousands of variables that impact health and disease.
So vaccine safety studies are inadequate because the statistics used in the epidemiology studies looking for adverse events are insufficiently complex compared to econometrics models? I note that Rogers doesn’t actually list the 20 studies or so that the CDC relies on for showing that there is no relationship between vaccines and autism. Instead, he refers readers to his PhD thesis, where, presumably, they’ll have to find them. As for his actual thesis, I have to ask: Holy hell, how did this get him a PhD? I thought that the University of Sydney was a reputable university, but its Department of Political Economy in its School of Social and Political Sciences somehow granted him a PhD based on it. If you wonder why I’m asking these questions, just read Chapter 9, which is riddled with antivaccine tropes and misinformation that regular readers will recognize as purest antivaccine pseudoscience and conspiracy mongering, and you’ll understand my puzzlement. (The rest of the thesis is full of anti-pharma tropes and conspiracy theories, too, as far as I can tell from a brief perusal.)
In any event, even without such a list, I’ve written about a few of them myself, which makes me laugh at the next passage:
The reality is quite different. Vaccine safety studies tend to be bivariate — they only look at two variables — the vaccine (independent variable) and whether someone suffered an adverse event (dependent variable).
I did a little searching on PubMed and easily found studies using multivariate methodology (e.g., this one from 2004) addressing the question of whether vaccines are a risk factor for autism. Looking at the general question of vaccine safety studies, searching PubMed, it’s not hard to find a number of studies using multivariate methodology. No doubt Rogers will dismiss them as not “multivariate enough,” but his argument at its core is silly, as someone with actual knowledge of how epidemiology is done pointed out:
Also:
He appears to be correct, too. A bivariate analysis does look at only two variables, and arguably there is no vaccine safety study that does that. It is true that such studies do look at whatever condition whose risk factor due to vaccines the investigators want to analyze as a dependent variable and vaccine uptake as the independent variable. (In fact, bivariate analyses are the simplest special case of multivariate analyses in which multiple relations between multiple variables are examined simultaneously.) All vaccine safety studies, though, look at lots of variables, including not just vaccination status with the vaccine(s) in question but anything that might confound the analysis, such as age, timing of vaccination, socioeconomic status, race/ethnicity, and a lot more.
Another thing to consider is that multivariate models in medicine are more often hypothesis-generating, rather than hypothesis-testing. Epidemiologists look for correlations, and then test them in more focused studies and experiments. What Rogers appears to want to do is to add more variables in the service of going on what we call a “fishing expedition,” which doesn’t involve actually testing a hypothesis but rather looking for hypotheses to test—or in Rogers’ case more likely, factors to blame on vaccines or to mask estimated vaccine effectiveness.
It can get quite complicated:
In the above Tweet, Gideon is, of course, noting a very simple adage in biostatistics: Too many variables are actually often a bad thing in a study. Indeed, in smaller, preliminary studies (say, under 50 subjects) there’s an old adage that if the number of outcomes you’re looking for starts to approach the number of subjects in the study, you have a big problem and need to narrow your net.
Joking aside, the reason for not needing that many variables is, of course, that if you are looking at risk factors for a respecified diagnosis (e.g., autism) for which you hypothesize that a vaccine is a risk factor then you need to control for as many confounding factors as you can that also affect the risk of that outcome. It’s basic epidemiology that confounders matter. Indeed, confounders matter even in simpler designs, and controlling for them can be hideously difficult, regardless of the specific study design.
Of course, antivaxxers frequently like to weaponize uncorrected data; i.e., data that haven’t been adjusted for relevant confounders. We’ve seen this so many times in the past. For instance, one prominent epidemiological CDC study looking at whether the thimerosal used as a preservative in some childhood vaccines was a risk factor for autism started out with unadjusted results that showed a high odds ratio for autism risk based on exposure to thimerosal-containing vaccines, but when appropriate adjustments were made to the data the apparent increased risk of autism went away. Naturally, antivaxxers portrayed adjusting for risk factors as a “coverup” designed to hide the link between mercury-containing thimerosal and autism; indeed, that became the basis of Robert F. Kennedy, Jr.’s Simpsonwood conspiracy theory in 2005. A similar thing happened with data from an MMR study, in which the unadjusted data showed a correlation between autism risk and MMR, but the adjusted data did not. The result? The CDC whistleblower conspiracy theory immortalized by Andrew Wakefield and Del Bigtree in VAXXED: From Cover-up to Catastrophe, a film so over-the-top in trying to make its claims that, as I wrote at the time, it would have made Leni Riefenstahl cringe.
None of this stops Toby Rogers from ranting:
Here’s the point I want to make: The CDC’s failure to use proper statistical tools (multivariate analysis and beyond) is a threat to national security. But the reason why the CDC relies upon crude bivariate analysis (that is categorically rejected by all fields of scholarly endeavor except vaccine safety studies) is because rigging these studies is the only way to hide the fact that these shots do not work and cause catastrophic harms. If the CDC ever used proper statistical methods, the national vaccine program would come to a screeching halt because the entire program is based on fraud.
This just embarrassing:
When I read a vaccine safety study for the first time, a sickening panic swept over me — “no, no, no, this cannot possibly be. THIS is what the CDC is relying upon!? THIS is what the CDC is using to claim that vaccines are safe!?” Far from being too advanced, these studies are so crude they would fail any Statistics 101 class in any college in America. Tears streamed down my face.
Tears! Did you hear me, tears! Toby Rogers wept (just like Jesus) because he was so upset at the CDC’s chicanery! It’s all fraud to him:
So I read another, and another, and another “vaccine safety” study (eventually reading all 20). And they were all the same. These studies are preposterous because they cannot possibly answer the question they are trying to tackle because their methods are too basic. Vaccine safety studies contain no biology, no chemistry, and vanishingly little statistics. These people are doing arithmetic, badly. In fact, they resemble a card trick more than anything having to do with science. I realized that the CDC has no idea what it is doing — the entire field of pediatrics and public health is a Potemkin Village, a Ponzi scheme, the worst intellectual fraud that I have ever seen.
So a political economist thinks that his statistical methods are far superior because…complexity. As I like to stay, statistical methods are tools, and you need to pick the right tool for the right job. I can’t comment on whether the super-complex models that Rogers brags about in his chosen field of econometrics are the right tools for the right job. I’m insufficiently knowledgeable in the field; so, imbued with what Rogers says about the supposed “humility” of the econometrics models that he touts, I won’t hazard a guess on that front.
I do, however, know biostatistics reasonably well for a non-statistician. I also know epidemiology reasonably well for a non-epidemiologist. Because of that, I know that in epidemiological studies, more variables are not necessarily better. What Rogers seems to be upset about is that, instead of thinking up hundreds of variables to look at, vaccine scientists looking for a link between vaccines and autism (for example) look only at potential confounders known to affect autism risk and, as controls, maybe a few known not to be. Why would they look at others? Another factor is that biomedical science exists in the real world, not in silicon, where one can add whatever variables and factors one wishes. Epidemiology datasets usually don’t have hundreds of variables associated with each case that can be mined.
Indeed, I have experience with one such dataset, a quality improvement database that I helped manage for a few years. Everyone always wanted to add more variables to it, but the consequence was that over the years before I became involved with the project the database had become quite unwieldy and the number of variables that needed to be entered and updated were becoming extremely burdensome to the hospitals who were part of our quality consortium. Toby Rogers is living in a world where he can construct models based on as many variables as as he pleases. Medical science doesn’t work like that because, even though computing power is still increasing rapidly while decreasing in cost, in the real world datasets are not infinitely expandable and infinitely updatable.
The other thing that Rogers apparently doesn’t understand is that adding a bunch of unnecessary variables to a vaccine safety study in the name of doing a “multivariate” analysis won’t just risk falsely decreasing the apparent vaccine efficacy (which he seems to be OK with) but will also ultimately dilute out any safety signals that he wants to find for vaccine injuries and adverse outcomes. Sure, the more variables you compare, the more likely you are to find associations between them, which is no doubt what Rogers, like all antivaxxers, likes, but proper statistical correction almost always eliminates their statistical significance.
Toby Rogers also goes straight to an antivax fallacy:
Vaccine safety studies almost always look at children who have received the full vaccine schedule as compared with children who have received the full schedule +1 more vaccine. And on that basis, they decide whether the 1 additional vaccine is safe. There is no unvaccinated control group. And they do not control for hundreds of other variables that influence health and disease.
Where have I heard this particular argument before about there being “no true unvaccinated control group”? I wonder…
Where have I heard this one before, other than pretty much everywhere in antivaccine conspiracy theories? Indeed, I remember writing about this one going back at least 15 years. It’s an article of faith among antivaxxers that the reason that neither the CDC nor any other reputable scientific organization has found a link between vaccines and autism is because there was never a true “unvaccinated control group.” Indeed, we even call this antivax trope a demand for a true “vaxxed/unvaxxed study,” and antivax doctors and scientists (not to mention quacks like homeopaths) have been serving up crappy studies based on this premise for years now.
Hilariously, Rogers laments how no one but him seems able to see the errors that he considers so obvious about “bivariate analysis” and characterizes as “categorically rejected by all fields of scholarly endeavor except vaccine safety studies”:
But here’s what I cannot figure out — why do scholars in these other fields, who actually know their stuff, fail to call the CDC out on their chicanery? Even worse, why do scholars in these other fields — who could spot the errors in these “vaccine safety studies” in about 10 minutes if they did any due diligence — poison their own kids and themselves? We live in the dumbest era in human history — we have the tools to know and do better and yet mainstream society still participates in self-inflicted genocide because they just want to fit in.
Obviously, Rogers is so brilliant that only he can see the gaping flaw in every vaccine safety study ever done or cited by the CDC! Only him and no one else, because he’s so brilliant with his multivariate models involving hundreds of economic variables! It never occurs to Rogers that maybe—just maybe—the reason that scholars in other fields don’t call the CDC out for using incorrect statistical models is not because they are cowardly, paid off, or “just want to fit in.” Maybe—just maybe—the reason that these scholars don’t call out the CDC for its vaccine safety studies (or call out the FDA and reputable non-CDC and non-FDA scientists) is because these studies use the right statistical tools for the job, testing hypotheses involving vaccine safety involving whether vaccines work as intended and/or are associated with increased risks of autism or various diseases and adverse outcomes and we don’t need Toby Rogers’ super-fancy big brain econometrics-like analyses of vaccine safety data to test these hypotheses.
I agree that we might live in the dumbest era in human history, but not for the reasons that Toby Rogers thinks that we do. From my perspective, articles like the one Rogers wrote are one indication that we might be living in the dumbest era in human history.
49 replies on “Toby Rogers knows statistics but not vaccines”
One uses a multi-variate analysis to develop hypotheses, not support them. Multivariate analyses are often just fishing expeditions for something, and when one has a bias going into it then you’re more likely to buy into noise as a real signal. It’s a shotgun, not a laser pointer.
Not sure what you mean here — MANOVA, multiple regression (linear, logistic, etc.), and other procedures do address both the form of a relationship and variable significance. Which procedures were you thinking of?
This explains it better than I can:
DOI: 10.1177/0016986219887200
Until you know that a MANOVA is absolutely necessary (multiple dependent variables) then it’s not as good as other techniques. Even with multiple dependent variables there are usually better techniques.
Nice! As I said, the right tool for the right research question. Toby Rogers seems to value multivariate analyses simply because it’s more complex and he can he can make it as complex as he wants. It’s almost as though he’s showing off, rather than seriously addressing a research question.
” .. simply because it’s more complex,,,”
Agreed. Don’t alties like to toss in arcane references and curiously named concepts as well as Really Big Words to impress their audiences of their unrivalled brilliance and secret knowledge?
So sure, say that it’s multivariate even when that is not especially relevant because it’s MORE and more is better. If they are talking mostly to people who haven’t studied stat and research design, they may be affected and feel that they have been educated.
So one altie might give minute details about the intricacies of his lab instruments and another may beef up individual words in his spiels by adding unwarranted suffices ( see ” iatrogeneticism”, ” athleticism” and ” parliamentarian” – the last signifying an MP!) or throwing in nearly random concepts from philosophy or physics- all useful for cargo cult cosplay.
Maybe he just likes using the “sophisticated statistical software”. Is there any actual use of this, or any other, statistical analysis, to support his hypothesis?
One of the first tenets of modeling is that you never [ok, possibly some work in physics, chemistry, or engineering have known models, but those are far from what is discussed here] know the true form of the model, your best hope is to devise one that provides a good approximation to the relationship you’re investigating.
In addition to his attempts to use his history in an appeal to authority, items like this
tell me he’s also trying to appeal to mathematics: “real work uses complicated stuff”. I will also point out that in his comment about “bivariate” work much of the work behind the scenes to obtain the results needed for testing and estimation relies on calculus, so he’s being intentionally misleading there.
And, unless the sample size is incredibly large, having thousands or hundreds of thousands of variables poses serious problems: huge amounts of noise and spurious correlations to name two. Unimportant variables can mask contributions of meaningful variables. Variable selection and dimension reduction become as important as the rest of the work.
For designed experiments the proper tool is (are) those designed to work with that design. I’m not sure what guideline he’s using to address appropriateness in general.
Pure crap there.
Yeah, he’s just an anti-vacc crank. Sadly, because he apparently understands some statistics, the other cranks in the world [as well as the clowns who post here] will take him seriously.
Same point about behind the scenes work applies to biology and virology, both needed to figure out what to test and what to control for.
The calculus thing just kills me. “My branch of science is better than yours because my branch uses calculus and you only use statistics!”
Uh huh. And to quote XKCD #1520 “The heroes of my field [biology] have slain on of the four horsemen of the apocalypse, while the heroes of your field [physics] gathered in the desert to create a new one.”
Or: your field is so simple that it can be modeled with calculus and the current level of computing power. Biology is a lot more complex that that. (Also, it can and will eat you.)
Assume a spherical economist?
Assume spherical reasoning.
Also: “All models are wrong but some are useful” – George EP Box, famous statistician.
If your model perfectly captures your system then either 1) it is your system and not a model or 2) you’ve missed something.
“The revolution has begun…a sickening panic swept over me…the entire field of pediatrics and public health is a Potemkin Village, a Ponzi scheme, the worst intellectual fraud that I have ever seen.”
This is the sort of hysterical ranting that we’d expect from Alex Jones and Mike Adams, not from a PhD economist who wants to be taken seriously.
Tears of laughter are appropriate.
Vaxopedia has an interesting take on the goings-on in Toby Rogers’ neighborhood.
I forget where I first saw it, but someone once said “with five variables, you can draw an elephant.”
Perhaps there’s an elephant hiding out in Dr. Rogers’s world, and needing to be found; but I doubt it.
Multiplication of variables is almost always not a good thing.
Maybe Toby should find some topic in science or medicine which he does accept and then show how his multivariate analysis gives the correct results there, but not for vaccine efficacy or safety. What’s good for the goose is good for the gander, right, Toby? No massaging variables to get the results you want, K?
I am NOT an expert on statistics, but the fact that introduction of vaccines drastically cut the death rate of covid in the USA and the strong correlation between least vaccinated states and worst covid consequences is compelling. I have friends who are paranoid of vaccination (and modernity in general) and all had a hard time psychologically admitting the correlation. Fortunately none spent more than a week in the hospital (so far). It doesn’t bode well for the other challenges of civilization, including coping with climate chaos.
“Lies, damned lies and statistics.”
“The vaccine turned me into a newt!”
“The vaccine turned me into a newt!”
My theory is that vaccines are turning newts into humanoids, and those ex-newts go on to become anti-vaxxers. It seems to explain a lot of what I see.
Thanks to all the idiots who occupied hospital beds for a week, my husband has to wait six weeks, instead of two, to have his cancerous prostate removed, so count me out of being grateful for their unncessary medical attention.
Certainly Rogers seems to have a truly dizzying intellect and has applied his statistical “spherical cow” in a tremendously powerful way, such that VAERS abuse cannot be far behind.
I’d be willing to bet that he’s already abused VAERS. I just didn’t feel like looking for it last night as I was writing this.
“Imagine a spherical cow of uniform density on a frictionless plane.”
Sure, but that doesn’t tell me anything about the behavior of a herd of cattle of mixed ages who are now stampeding off a cliff.
Honestly it’s the kind of thing I expect out of a physicist, not an economist.
You may have missed the joke about assuming a can opener, that even has a Wikipedia entry. https://en.wikipedia.org/wiki/Assume_a_can_opener
Years ago I told a person I met at a party about an article I’d read about the known and suspected oil reserves on the planet, and how quickly they would be depleted at current consumption rates. He started talking about economics, and we had a few back-and-forths before he said “oh, you mean the actual substance!” Yes, yes I did.
R is pointing and laughing, Mr. “I have a page at CHD.”
I’m continually shocked anyone is still using SPSS. I guess they just like to spend money uselessly.
Be nice. There are those of us who have neither the coding skill to use R nor the time and inclination to climb the steep learning curve to get good at it.
It’s really holding statistics hostage. The libraries and availability of R are vastly superior. It’s really not that hard to learn.
I always thought SPSS was hiding the mechanics of things.
“Really not that hard to learn”? If you’re a coder, I’m sure that’s true.
I was more poking at his apparent need to be cutting-edge. The attempted Borging that is “econophysics” primed the pump.
For higher level work I agree, but it is fairly easy to teach with, with learning curves between Minitab and R.
I have zero experience with Stata, the other softare rogers referenced.
I liked STATA when I learned it in grad school; it’s good for epi stuff.
Personally I mostly use JMP (a subset of SAS). I used MiniTab and Kalidagraph back in the day (and Origen because a boss insisted it was the best for making pretty graphs). If I never see MatLab again it will be too soon.
I’ve done some R, but getting IT approval to install it was a pain in the tuchas, and not enough of my coworkers know any programming to be comfortable with it. (There’s a lot to be said for the accessibility of a graphical interface for people who won’t use the software every week.)
I’ve found the different statistical software programs have different focuses, and tend to be better for one type of data/analysis over another. As long as you’re not doing your analysis in Excel, use what works.
R does have several GUIs, but I’m out of my depth and will shut up.
The ASA has a statement that Excel should not be used for teaching statistics or for any serious analysis, so there’s agreement there as well.
I haven’t used SPSS since 1985. I was surprised to hear it’s still going!
I saw something about SPSS last year and had the same reaction. It still exists? SAS, well okay, but SPSS? I remember detesting it back in the 1990’s.
I think he’s no longer based in Australia. I think he’s in Colorado now. Just his Ph.D. is from Australia.
See, I was so amused at the stupidity behind his post that I didn’t even bother to look where he’s based. He did, however, definitely get his PhD from the University of Sydney, which is depressing.
It is a bit discouraging but I wonder if is a case where his committee really did not understand the topic area and let a lot of things slip by? I know I have seen a couple of papaers seem to slip through peer review and thought that the topic was far enough from the journal’s main focus that the editors probably & reviewers lacked the expertise to realise how bad the papers were.
My standing google search for “Paul Thomas” and “Vaccine-Friendly Plan” popped up Rogers’
defecationdisseration early last year–because Rogers cites Thomas heavily in it. How in the heck his disseration committee let him use a non-peer-reviewed anti-vaccine paperback book as a legit citation for a PhD is inexplicable and unacceptable. Rogers’ unwarranted receipt of a PhD for his steaming pile of feces rivals that of Judy Wilyman’s anti-vaccine phd. Ugh.A few other thoughts about Dr. Rogers and his work:
A. In his dissertation, his chapter about the history of autism mostly cites anti-vaccine activists’ Blaxill and Olmstead’s book, which is unlikely to be an acceptable academic reference for most.
B. As a reminder, in the 2019 Hviid piece there was an unvaccinated group. You talked about this before: https://www.respectfulinsolence.com/2019/03/06/mmr-does-not-cause-autism/ It doesn’t matter to these people.
C. If Dr. Rogers’ MPP is from 2012 and he hasn’t done anything in the area since – and his LinkedIn has no updated information (it does wrongly still place him in CA – if you look it up, you can find his activism in Colorado online) – I don’t know that he can rely on those skills. They do need use and honing.
D. Rogers has been very aggressive at targeting members of the CDC and FDA’s advisory committees – both as a group, several times calling for them to be personally emailed, and by writing direct attack on members of the committees. He is a very angry man.
Oh wow! A is really bad and would have been decades ago.
I first ran across Toby Rogers back in 2019. My comment about his thesis is here: https://www.respectfulinsolence.com/2019/08/29/v-is-for-vaccine/#comment-414966
I pointed out then that his thesis was from the Department of Political Economy, so I imagine his examiners knew nothing about vaccines and didn’t bother to educate themselves about the sources he was citing. It remains an embarrassment for Sydney University.
It is clear from the sources Rogers used in his thesis he was a fully formed anti-vaxxer when he started his doctoral studies.
Having read his screed referenced above, I suggest he is not that expert in statistical methods either.
I refuse to give non-science departments a pass when they accept a misinformation-laden doctoral thesis like this. If someone is going to do a social or political science thesis on vaccines and autism, then the thesis committee needs to include one or two faculty with the appropriate expertise in those areas. The University of Sydney failed.
I might have to do a blog post on Rogers’ thesis; it might even be something that I should do at my not-so-super-secret other blog.
“he is not that expert in statistical methods”
I recall from decades ago, when I was a teenager, in my first job doing programming, the government project I worked for employed a bunch of sociologists. They were collecting reams of data from people who were the subjects of their studies. The data was analyzed as part of the project, exactly how and to what ends I didn’t understand or care about. But I was already pretty well versed in statistical theory.
They used SPSS on a mainframe. As they became more expert with SPSS the sociologists were convinced that they had become experts in statistics. They really did. These were not cranks, just sufferers of self-delusion. It got them in all kinds of trouble, as I discovered when they came to me for help from time to time. Well, I didn’t know much about SPSS but I could sometimes suggest more sensible methods to use.
As the years passed this theme has come up many times, and not just in statistics. Point a non-expert at a sophisticated and (sometimes) user friendly software package and before long they come to believe they’re experts in that area (statistics, mechanics, electromagnetics, etc.), and they get into trouble, producing nonsensical results.
It’s become more common since there is now so much excellent and open source modelling and analysis software available, easy to download and use on a home computer. The number of inexpert experts has ballooned. Some are cranks but most are well intended.
I’ve run into this with self-declared experts (hobbyists?) in AI, physics, astronomy and more. It is also common for them to identify the field with the software. For example, I know several people who use publicly available AI packages and think that’s the totality of AI. Scary. Without at least a modest understanding of the field, software tools are regularly misused and misapplied. Many refuse to accept that and forge onward regardless.
I’ve taken a lot of biostatistics courses (undergrad and up) and I still go ask our biostatistician when I run across an analysis that is new to me.
You’re right about the danger of statistical software in the hands of people who don’t actually understand it. I was reading an internal report (by a former coworker I don’t super respect) and I came across a standard error value. For a single data point.
I was so utterly confused by how anyone could think that you could calculate a standard error of a single point that I had to ask a coworker to double check what I was reading. Nope, this guy had generated (somehow) a standard error for a single point. (That’s not how standard errors work.)
So then I dug around and discovered that while real statistical software (JMP, MiniTab, Prism) won’t let you do that, Excel will!
So now we have an informal rule in my group: if it’s a statistical calculation and it’s going in a report, it can’t be done in Excel. Excel is where you organize your data, sure. But no stats, ever.
Does it? I just tried and got an error.
I agree and I extend that to graphs. Excel does some of the crappiest graphs ever. I automatically penalise students who submit graphs fitted with spline curves from Excel.
On graphs, one of my favourite graphs published ever contained a curve line fitting two data points.
So, basically, Rogers has his knickers in a twist because biomedical research is not the same as econometrics. Got it.
[…] at an antivaxxer to whom I hadn’t paid much attention before, Toby Rogers, specifically his hilarious assumption that all vaccine safety research cited by the CDC is corrupt because it didn’t use the sorts […]
Biomedical research is simple because it doesn’t use calculus. It ain’t Rocket Science!