Brian Hooker criticizes a vaccine safety study; hilarity ensues

Last week, the Journal of Pediatrics published a study that did a pretty good job of demolishing a favorite antivaccine trope used to frighten parents. In fact, it’s one of the most effective of antivaccine tropes, as evidenced by a large number of parents who are generally pro-vaccine expressing doubts when asked about this particular antivaccine slogan. I’m referring, of course, to the “too many too soon” slogan, in which antivaccinationists try to imply that the current vaccine schedule somehow “overwhelms” an infant’s immune system and leads to autism by some unknown and undemonstrated mechanism. There is no compelling science behind this particular idea, but that doesn’t stop it from being used as an argument against the current vaccine schedule by antivaccinationists and antivaccine-sympathetic doctors like “Dr. Bob” Sears (who created an “alternate” vaccine schedule based on the idea that the current schedule gives babies too many shots too soon) and Dr. Jay Gordon. “Too many too soon” even became the rallying cry of a march on Washington, DC led by the queen of the antivaccine movement herself (at least at the time, anyway), Jenny McCarthy.

So it was that when I wrote about this particular study and what it showed the first time, I couldn’t resist mentioning a few of the antivaccinationists who, predictably, attacked the study, including the aforementioned “Dr. Bob” Sears, whose tantrum on Facebook over the study was burning stupid distilled down to the most powerful stupid known, much as 16 Million Reserve distills down pepper extract to a mixture so hot that it’s painful. “Dr. Bob’s” screed has the same effect on the brain of anyone who has a science-based understanding of vaccines as 16 Million Reserve has on the tongue. So, for that matter, did homeopath Heidi Stevenson’s “scientific critique” of the study.

And so does Brian Hooker’s critique of the study.

The things I do for you! After reading Hooker’s “debunking” of DeStefano et al, I feel the need for many, many gallons of reason and science to wash the pain from my brain, much as a hapless person who ingests 16 Million Reserve will crave gallons and gallons of water to wash the pain from his mouth. Before I get into his “scientific critique,” let’s just remind everyone of who Brian Hooker is. Brian Hooker is a biochemical engineer who works as a consultant in the biotech industry who also, unfortunately, has turned to antivaccine quackery. Specifically, he belongs to the wing of the antivaccine movement that believes against all evidence that the mercury-containing preservative thimerosal that used to be in childhood vaccines until around 2001 to 2002 is a major cause of autism. He’s also somehow managed to beguile a certain budding antivaccine crank by the name of Jake Crosby to the point that Jake turned on his former allies and dished dirt in the form of their private e-mails, publishing them on the even crankier Patrick “Tim” Bolen’s website. Hooker’s claim, channeled through Crosby’s “reporting,” was that he actually had arranged the Congressional hearing on autism in November and that the antivaccine crank group SafeMinds had, through its lobbyist Beth Clay, stepped in and taken it over by misrepresenting themselves as representing Hooker.

I will give Hooker one thing. He did have a mildly funny quip to attack DeStefano et al when he said, “The Destefano et al. 2013 study is to science what the movie Ishtar was to cinema.” Of course, antivaccine quackery is more akin to what Plan 9 From Outer Space was to cinema and has about as much relationship to reality as Plan 9 did in that his complaints against the study nearly all derive from either antivaccine talking points or a complete misunderstanding what a case control study is.

First, however, I can’t resist but insert a brief quote from the editor of Health Impact News, which seems to be quite woo-friendly, introducing Hooker:

Why did we ask Dr. Hooker to comment? There are probably very few people in the world who have spent as much time looking at CDC studies related to vaccines and autism as Dr. Hooker. Dr. Brian Hooker, a PhD scientist, has been fighting the CDC since 2004 in trying to get them to comply with Freedom of Information Acts to see the CDC research that supposedly shows there is no link between mercury in vaccines and autism.

I couldn’t help but laugh at this. A better description is that few cranks have wasted as much of the CDC’s time and effort in pursuit of a scientifically bankrupt idea. Let’s take a look at Hooker’s complaints. First, he complains that the study “is not new,” as this were some sort of revelation. Well, duh. It’s right in the methods (and was discussed in detail by me when analyzing the study) that DeStefano et al uses data from a study by Price et al in 2010, which I also discussed. It’s not as though it were secret or anything. Although I’ve seen a lot of the fallacious antivaccine attacks on Price et al before, Hooker brought out a new one, in which he claims:

Not only was this original study fatally flawed due to a statistical error called “overmatching” (which I’ll discuss further below) but also the study authors hid data regarding the only valid part of the study (i.e., prenatal thimerosal exposure) which showed that children exposed to just 16 microgram mercury in thimerosal in utero were up to 8 times more likely to receive a diagnosis of regressive autism (Price C, et al. Thimerosal and Autism. Technical report. Vol I. Bethesda, MD: Abt Associates Inc; 2009). The study authors instead falsely reported no risk of autism associated with prenatal thimerosal exposure.

This technical report can be found here. It’s 235 pages long, and I was trying to figure out just what the hell he was talking about, given that Price et all was actually quite well controlled, and it was well explained how various risks were calculated. This smells like a bit of cherry picking of results from multiple comparisons. I pored over as much of the technical report as I had time to, and I couldn’t figure out what Hooker was talking about, but it’s certainly possible that I missed it. Every section that I read pointed out things like,” and the odds ratios in the relevant tables were all near 1. Googling for previous articles in which Hooker made this charge was unrevealing, too. He’s going to have to elaborate on this before I see any reason to take it the least bit seriously.

Next up is a complaint that seems to be a major talking point agreed upon by the antivaccine movement, namely that he thinks there are no “true controls in the study,” based on this passage from DeStefano et al:

Of the remaining 752 controls included in the analysis, 186 had an SCQ (Social Communication Questionaire) score <16 but had indications of speech delay or language delay, learning disability, attention deficit hyperactivity disorder or attention deficit disorder, or tics, or had an individual education plan.

Which leads Hooker to conclude:

This clearly shows that the 186 aforementioned controls (25% of the control group) were not controls at all but instead had some underlying developmental deficit (all of which are features of autism or autism spectrum disorder). Unlike the study design described (i.e., where autism cases were matched to neurotypical controls), autism cases were matched with “cases” of other, similar neurodevelopmental maladies. Thus, you would expect to see no difference between the two groups.

I discussed this complaint when I discussed Price et al the first time around and again just this week. But think of how silly Hooker’s complaint is. He once again does not understand what a case control study is. The whole point was to match cases, which means children with the condition under study, namely autism or autism spectrum disorder, or controls, which means children without autism or ASD. The whole point of administering the SCQ was to identify children in the control group who might have an undiagnosed ASD and in fact eliminated 7 children from the control group that way. Again, remember: The hypothesis being tested in Price et al was whether thimerosal-containing vaccines were associated with autism risk, and the hypothesis behind DeStefano et al tested was whether total antigen load due to vaccines is associated with autism.. Here’s another hint for Mr. Hooker: Speech delay is not autism, and, even if his complaint were valid, at worst such matching could only be expected to decrease any apparent difference between the groups, not eliminate it. Basically, Hooker is criticizing the investigators for not picking cases based on more than just autism. Of course, doing that would add heterogeneity to the case group.

Next up, Hooker complains that using the number of antigens as an estimate of vaccine exposure is meaningless. One reason he gives for this is the standard goalpost-moving antivaccine Gish gallop of claiming that it’s not the antigens but the “toxins” (yes, the “toxins gambit” is liberally used, in which Hooker complains about adjuvants, formaldehyde, and polysorbate 80 as the real culprits. As for the rest, the issue of using antigens as a surrogate for vaccines is discussed in this review article, along with a discussion of how children are exposed to far fewer antigens from vaccines today than they were in the past. Yes, it’s a crude estimate, but the argument that not all antigens are equal in evoking an immune response cuts both ways in that using, for example, the number of vaccines as the measure is arguably worse. Not that it probably matters, because autism risk doesn’t correlate with on time administration of vaccines compared to late administration anyway. Of course, the unspoken assumption behind Hooker’s objection is that the immune reaction caused by vaccines is an etiological agent that causes autism. There is no compelling evidence to support that assumption.

Another complaint Hooker has is that a high participation refusal rate led to selection bias. That’s an old complaint leveled against Price et al three years ago by SafeMinds, and it’s as without validity as SafeMind’s complaint was back then. Not that that will stop antivaccinationists from resurrecting it again and again. To them, repetition of a trope makes it true, rather like an incantation.

In fact, the rest of Hooker’s criticisms are basically warmed over SafeMinds criticisms about Price et al. He’s unhappy that there are no unvaccinated children in the study, to which I say: So what? That’s not what the study was studying, and it’s possible to get at the question of whether vaccines are a risk factor for autism without examining an unexposed cohort. The reason, of course, is that if vaccines do something to increase autism risk, there should be a dose-response relationship that can be gleaned from epidemiology. So far, no reputable researcher who’s looked has been able to detect such a relationship.

Even Hooker’s complaint about “overmatching” is warmed over SafeMinds:

The point made by Dr. DeSoto and Dr. Hitlan is that the cases and the controls in this study are too closely matched to each other. Cases were matched with controls of the same age, sex, within the same HMO and essentially the same vaccination schedule using the same vaccine manufacturers. This may be seen in Figures 1 and 2 of the Destefano et al. 2013 paper which indicated that there are almost no differences between the exposure to antigens between the case (autism) and control groups in every exposure group tested. This holds for cumulative antigen levels (Figure 1) as well as single day antigen exposure levels (Figure 2).

This type of error of course precludes “finding a difference” between cases and controls because all differences were matched out case-by-case.

This would be akin to analyzing radiation workers that got the same dosage of gamma radiation within cases and control groups to determine the relationship between gamma radiation and cancer incidence. Of course, since cases and controls got the same dosage, no effect would be seen. However, this is an unfair study. To see the true effect, cases would need to be matched with controls with variable levels of gamma radiation exposure and perhaps a “no exposure” group would be included as a baseline comparison to cancer rates within higher exposure groups.

As I said at the time about SafeMinds, Hooker seems to be criticizing a case control study for not being a cohort study, which is what he is proposing. So I’ll explain again, as it’s been three years since I’ve discussed this specific issue in depth. Here’s how case-control studies work. (Hint: They don’t work the way Hooker seems to think they work.) You take a population and identify the cases. Take a random subset of cases if you can’t examine all the cases. Then you look at the rest of the population and randomly select people who do not have the condition that you are studying. You match them for as many relevant demographic parameters as you can that could potentially confound any correlations that might be observed. Then you look for differences between the groups. If the cases, for instance, have a higher exposure to the substance under study, then the conclusion is that exposure to the substance is associated with the condition and therefore the substance might cause or contribute to the studied condition. If the exposure to the substance under study is lower in the case group than in controls, then the conclusion is that that substance might be protective. If the exposure is the same between the groups, then the conclusion is that that substance probably has no relationship to the condition under study, which is what this case-control study more or less concluded. Hooker looks at an observation that is a conclusion of the study and appears to misunderstand that a failure to find a difference does not mean that the groups were chosen that way, which is what was implied.

Basically, what Hooker is proposing is to match cases to controls on the variable being studied, which is a violation of Epi 101.If Hooker wants to say that a case control study is an inappropriate experimental methodology for the hypothesis being studied, let him make that case. But he’s doing nothing other than betraying ignorance and obfuscating the issue. Basically, what Hooker is proposing is to match cases to controls on the variable being studied, which is a violation of Epi 101. Of course, Hooker seems profoundly confused by the difference between case control studies and cohort studies. He seems to think that somehow the case groups and the control groups were intentionally matched to have equal exposure to vaccine antigens, rather than what really happened: Autism cases were matched with non-autistic controls, and no difference in vaccine antigen exposure was found between the groups. Indeed, he doesn’t even appear to know what overmatching is, of which there are three types. One type refers to matching that harms statistical efficiency, such as case-control matching on a variable associated with exposure but not the condition being studied. The second refers to matching that harms validity, such as matching on an intermediate between exposure and disease. The first form basically just adds noise, which seems to be the form to which Hooker is referring. Of course, the silliest aspect of this criticism is the complaint that the cases and controls were matched by birth year. Here’s a hint: That is not overmatching! For more details, go back to my original deconstruction of SafeMinds’ attack on Price et al.

Perhaps the silliest part of Hooker’s entire screed came when he writes:

The CDC is simply afraid of what they already know – vaccines cause chronic disease and an unvaccinated population will be much healthier, period (as evidenced in the Glanz et al. 2013 study within the Journal of the American Medical Association which stated that unvaccinated children were seen at a lower rate of frequency in emergency room and outpatient visits).

The study to which he refers is this one, and Hooker’s misinterpretation of the study is a perfect example of how he misunderstands case controls. He believes that the study shows that undervaccinated children are healthier. He fails to note that the study didn’t specifically look at unvaccinated children and, indeed, used the same sort of division that he castigated in DeStefano et al, namely between children who received all their vaccines on time and those who didn’t or who were undervaccinated. Apparently doing that is pure pseudoscience when CDC investigators do it but completely hunky dory when the authors of a study Hooker likes do it. (Do I need to repeat again how blatant the double standard is?) In any case, apparently it doesn’t occur to Hooker that perhaps there is a confounder here in that parents who tend not to vaccinate their children also tend to be parents who don’t take their kids to doctors or emergency rooms as much. Another glaring omission from Hooker’s discussion of the study is this inconvenient observation from the study that’s right in the abstract:

In a matched cohort analysis, undervaccinated children had lower outpatient visit rates compared with children who were age-appropriately vaccinated (incidence rate ratio [IRR], 0.89; 95% CI, 0.89- 0.90). In contrast, undervaccinated children had increased inpatient admission rates compared with age-appropriately vaccinated children (IRR, 1.21; 95% CI, 1.18-1.23). In a second matched cohort analysis, children who were undervaccinated because of parental choice had lower rates of outpatient visits (IRR, 0.94; 95% CI, 0.93-0.95) and emergency department encounters (IRR, 0.91; 95% CI, 0.88-0.94) than age-appropriately vaccinated children.

Yes, children who were undervaccinated not because of parental choice had a higher utilization of emergency room and inpatient services than those who were undervaccinated by specific parental choice. The authors even comment on this:

Children who were undervaccinated because of parental choice had lower rates of outpatient visits, lower rates of ED encounters, and no significant difference in inpatient admission rates compared with age-appropriately vaccinated children. These results suggest inherent health care–seeking behavioral differences between the 2 groups of parents. For example, published survey data31- 32 have shown that parents who choose not to have their children vaccinated are less likely to trust health care professionals and more likely to use complementary/alternative medicine providers than are parents who have their children fully vaccinated. It is therefore possible that parents who delay or refuse immunizations are less likely to use the traditional health care system when their children contract minor acute illnesses but will seek medical care when their children become seriously ill. Such differences could create a selection bias in studies that attempt to examine the risk of potential adverse events following vaccination. Future survey work with parents across a range of vaccination concerns could help explain these differences in health care utilization rates.

Emphasis mine.

In other words, this study says nothing about whether undervaccinated children are more healthy and a lot about the health-seeking behaviors of antivaccine parents. Hooker interprets it to mean that undervaccinated children are healthier and have less chronic disease when the study was not designed to show that and in fact says little or nothing about the health of such children relative to vaccinated children. A more willful misinterpretation of a study is hard to imagine.

Finally, Hooker opines:

Finally, this type of study misses the point entirely that children with autism are physiologically different than neurotypical children. Numerous studies have shown genetic (e.g., James et al. 2006), morphological (e.g., Herbert et al. 2005) and biochemical differences (e.g., Waly et al. 2004) between these two populations. To perform a case-control study such as that presented in the Destefano et al. 2013 paper assumes a genetically, morphologically and physiologically homogeneous population, which is simply not the case.

No one is claiming that children with autism or ASD got higher doses of vaccine antigens, thimerosal, MMR or whatever. What we know instead is that when our children received the same vaccines within the ACIP recommended schedule, they reacted differently. The scientists at the CDC are trained in managing infectious diseases with progressions that may be predicted with reasonable certainty. However, these neurological sequelae to vaccines are chronic, multifactorial conditions that cannot be put into the same tiny box as the common cold, influenza or chicken pox.

Ah, yes. Special pleading. You can’t study my hypothesis using your usual methods! Take that, skeptics! It’s also a bit of the old moving of the goalposts as well. Think about it. Back in the 1990s and early 2000s, vaccines were claimed to be The One True Cause of Autism. Indeed, back in the day, Generation Rescue used to say that autism was a misdiagnosis for mercury poisoning. That’s the sort of claim that implies a very high correlation between, in this case, thimerosal-containing vaccines and autism, and, given that mercury seems to be Hooker’s particular bogeyman, I felt obligated to mention Generation Rescue’s hyperbole, which was long ago scrubbed from its website. Indeed, we see this a lot among antivaccine cranks. As scientific evidence fails to find such correlations, it is amusing to me to watch antivaccinationists like Hooker retreat to less and less convincing excuses. In this case, he attacks a straw man, namely that a case control study must assume a morphologically and physiologically homogeneous population. In other words, the hypothesis has shifted from “vaccines cause autism” to “vaccines cause autism only in children with special undefined genetic susceptibilities.”

Same as it ever was. Hooker is, after all, an antivaccine activist. Of course, he misinterprets science, Gish gallops all over the data, confuses case control studies and cohort studies, insinuates dire conflicts of interest, and in general cherry picks the data to support his hypothesis. If he did otherwise, there might be hope that he might realize that he’s devoted his life to pseudoscience and quackery. I see little hope of that.