Better late than never: The Dunning-Kruger effect meets an incompetent anti-vaccine “analysis”

It never ceases to amaze me just how ignorant of very basic principles of science anti-vaccine activists often are. I mean, seriously. Every time they try to post something, whether they know it or not, they end up making themselves look so very, very stupid–or at least ignorant. The Dunning-Kruger effect takes over, and people who may actually be very successful–intelligent, even–in other fields of knowledge make newbie mistakes and draw egregiously misinterpreted conclusions from existing data. Worse, they have no clue that they don’t know what they’re doing. In the arrogance of ignorance, they believe themselves the equals or superiors or real scientists who have spent many years studying the discipline in question and doing real research in it. So it is, when anti-vaccine “scientists” (and I use the term very, very loosely) decide to reanalyze existing data.

Take, for instance, Mark Blaxill (please). Mr. Blaxill is associated with not just one but two anti-vaccine groups, first as a board member of SafeMinds and then as sometime blogger for the propaganda blog (Age of Autism) of the anti-vaccine group founded by J.B. Handley whose spokesmodel is Jenny McCarthy (Generation Rescue). Mark Blaxill, who has no advanced training in science and shows it day in and day out as he tries to promote his anti-vaccine views, so much so that my nickname for him is Mark “not a doctor, not a scientist” Blaxill. True to his nickname, Blaxill posted a typical anti-Gardasil screed disguised as a political attack on candidate for the Republican Presidential nomination Rick Perry for his ties to Merck entitled, Rick Perry and The Conflicts of Government Licensed Corporate Profit: Merck & Gardasil. What it is is a repost of an old two-parter that Blaxill wrote over a year ago, gussied up with an introduction complaining about ties between Rick Perry and Merck. I didn’t bother with it when it first came out, but for some reason it caught my eye this time, and I figured, what the heck? I feel the need for a chew toy last night.

I’m not going to bother with the vast majority of the political stuff in his post, because, quite frankly I don’t care that much about it, and I don’t like Rick Perry any more than I like Michelle Bachmann. For all I know, he really was in bed with Merck to mandate Gardasil in Texas, as he’s been accused of. If that’s true, it’s just one more strike against him, as far as I’m concerned.

Rick Perry aside, the first part of Blaxill’s article would be perfect for Prison Planet or Infowars.com, two of the premier sites favored by the tinfoil hat brigade. Basically, it’s one big conspiracy theory. Besides, weighing in at nearly 16,000 words, Blaxill’s “treatise” makes Orac look like the soul of brevity in comparison. Deconstructing it all would take multiple installments, and I doubt that even the connoisseurs of the longer Orac-ian screeds would be able to tolerate that. Certainly, I couldn’t tolerate the level of burning stupid contained in that many Blaxill-scribed words. Although my neurons wear miniature asbestos suits to protect them from damage from the napalm-grade burning stupid that washes over them any time I have the poor judgment to wander over to AoA, that doesn’t mean it doesn’t cause them extreme pain.

In any case, about a quarter of the way in, Blaxill asks, How stringent was FDA’s safety review for Gardasil? Not surprisingly, his answer is, “Not at all.” Even less surprisingly, Blaxill’s reasons for coming to that conclusion reveal just how clueless he is. As an example, I’m going to cite two of his paragraphs in full and let you savor the Dunning-Kruger goodness for yourself without actually having to click over to AoA and figure out what passages I’m talking about. If you are a physician or a scientist involved in clinical research, you won’t know whether to laugh or cry. If you are in any way an immunologist or do immunology research, I suspect you’ll be leaning towards crying. You have been warned.

As Kirk said in Star Trek II: The Wrath of Khan, “Here it comes”:

When the FDA issued its approval of Merck’s BLA for Gardasil on June 8, 2006, its decision was based on a review of Merck’s data from five separate clinical trials, each of which included efficacy and safety assessments for Gardasil. Four of the five trials approached their efficacy and safety studies in similar fashion, comparing Gardasil against a “placebo” that contained an active ingredient, with one trial comparing Gardasil against what the CBER reviewers described as a “saline placebo.” All together, these five trials examined a total of close 12,000 subjects who received at least one dose of Gardasil and compared their outcomes to roughly 10,000 subjects who received up to three injections of what Merck and CBER officials agreed to describe as a “placebo.”

But what is a placebo, really? One definition describes a placebo as “an innocuous or inert medication; given as a pacifier or to the control group in experiments on the efficacy of a drug.” The operative term here is the word inert. But in four of the five trials, Gardasil placebos contained a substance called an adjuvant, “a substance which enhances the body’s immune response to an antigen.” According to one of the trial publications, most of the Gardasil trial placebos actually contained an “amorphous aluminium hydroxyphosphate sulfate adjuvant… and was visually indistinguishable from vaccine.” So although the majority of the placebo treatments in the Gardasil trials did not include Gardasil VLPs, they were by no means inert. In control populations representing nearly 95% of all “placebo” recipients, the study subjects received a formulation that actually included an immunologically active (and potentially harmful) aluminum adjuvant.

Yes, indeed. Here you have someone who is neither a scientist nor a doctor perseverating over a single word in one definition of “placebo” that he found. In actuality, Blaxill completely misunderstands what a valid placebo is. Heck, even Wikipedia provides a pretty reasonable definition of what a placebo is, specifically “a substance or procedure… that is objectively without specific activity for the condition being treated.” Note the phrase “without specific activity for the condition being treated.” Under this definition of placebo, an injection that contains everything except the HPV antigens used in the vaccine to provoke an immune response against the virus is, in actuality, the most appropriate placebo to use for a clinical trial of such a vaccine. The scientists who did these trials chose exactly the most appropriate placebo. Of course, it’s obvious that Blaxill is trying to call these trials into doubt because he thinks the aluminum adjuvant is harmful when in fact it is not. Even more amusing, Blaxill appears to me to have–shall we say?–appropriated this argument wholesale from a three year old article by anti-vaccine loon Cynthia Janak. (Judge for yourself if you don’t believe me; each and every argument that Janak uses, Blaxill also uses. He just turns them into graphs.), In any case, we’ve met Janak before, and her understanding of science is, at best, challenged.. So it’s not even original crankery, and Blaxill doesn’t give any credit to Janak for having come up with this fallacious argument first.

But Blaxill did put it on steroids and pump it up to 15,000+ words.

In any case, Blaxill amplifies the display of his ignorance even further when he complains about one of the trials in particular, as though he had found a “Gotcha!” moment, in which, rather than a “saline control,” he found that actually a “carrier control” was used. he then tried to find out exactly what was in the carrier and was disappointed that he couldn’t do so easily. However, he did infer it from the Gardasil package insert:

It is possible, however, to infer the composition of the carrier solution from Merck’s Gardasil package insert, which lists the vaccine’s immunologically inactive ingredients. These include: “yeast protein, sodium chloride [table salt], L-histidine [an amino acid], polysorbate 80 [an emulsifier], sodium borate, and water for injection.” At least one of these chemicals, sodium borate, is a chemically reactive toxin, one that has many industrial uses as an active ingredient. These include applications as: a replacement for mercury in gold mining; an insecticide and fungicide; and a food additive that is now banned in the United States.

Oh, noes! It’s teh toxins! I’ve actually written about anti-vaccine loons trying to label L-histidine, polysorbate 80, and sodium borate as dangerous and why their arguments are a load of fetid dingo’s kidneys. After all, histidine is an amino acid used in small amounts. Polysorbate 80 is only harmful in rats at high levels; there’s no evidence that in the doses given in vaccines that it has any harmful effects. Sodium borate is often used as a buffer. In any case, once again, Blaxill is, as anti-vaccine activists are wont to do, completely ignoring the principle of “the dose makes the poison.” Sometimes I wonder if Blaxill is a homeopath. He seems to think that diluting ingredients like Polysorbate 80 makes them more toxic.

In all fairness, I’ll point out that Blaxill did admit that, for determining efficacy, using a carrier control or an aluminum adjuvant-containing control was the most appropriate. The evidence was strong enough that even Blaxill couldn’t spin it to say that the HPV vaccine is ineffective. So instead, he tries to claim that the failure to include a saline-only control calls into doubt the safety of the vaccine:

When it comes to the accurate measurement of adverse effects of Gardasil, there is little justification for reliance on a placebo with ingredients that are not inert. There is some limited value, perhaps, in comparing adverse events that are introduced solely by the addition of VLPs to the vaccine solution. But a truly rigorous safety assessment would investigate the full safety profile of the VLPs in combination with the aluminum adjuvant and compare that profile to the profile of an inert solution. After all, the adjuvant is present precisely because it is not inert.

This is the sort of argument that sounds superficially reasonable on the surface–but only if you don’t know anything about aluminum adjuvants or carrier solutions, which are used in large numbers of vaccines and have a long history of safe use, particularly. Even so, Blaxill undertakes a large number of contortions to try to demonstrate that somehow the overall results of the clinical trials used to demonstrate the safety and efficacy of the HPV vaccine are hopelessly tainted and actually show that Gardasil is dangerous. He fails. (Big surprise.) For one thing, testing the safety of aluminum adjuvant and the carrier solution was not the intent of the study. We already know the aluminum adjuvant and carrier solution are safe. Testing them again would be rather pointless and add unnecessary complexity and expense to the clinical trial.

The first intellectually dishonest thing that Blaxil does is that he doesn’t provide ready references to the five clinical trials whose data he “reanalyzed.” The second intellectually dishonest thing he does is that, with one exception, he doesn’t provide any sort of meaningful statistical analysis to tell an informed reader whether differences in groups were actually statistically significant. More importantly, Blaxill mixes and matches data from five different studies, comparing them in ways that are questionable at best and then not documenting what methodology he used, for example, this graph, which doesn’t conform to the way graphs are done in scientific studies. It is inappropriate to do a line graph for data of this type; a bar graph would have been far more appropriate. Also, there aren’t any data points; the data appear to be represented as continuous when they are not. And, of course, there are no statistical measures. Indeed, at another point in this “analysis,” we are assured that “on a retrospective basis” (whatever that means in the context of these data) “all but one of the reduced risks for the carrier solution group were statistically significant.” Unfortunately, Blaxill does not tell us what statistical test he used. My guess is that he didn’t use the appropriate test (ANOVA with a post-test to correct for multiple comparisons) and instead used a series of t-tests between each complication and the control, the latter of which is virtually guaranteed to produce false positives.

In any case, basically, Blaxill tries to argue that because the rate of local adverse events (pruritis, hemorrhage, etc.) is close to the same in the Gardasil groups and the aluminum adjuvant groups but lower in the “carrier control” group (which was only one clinical trial, contained several-fold fewer subjects, and was much younger than the other trials and therefore not really appropriately comparable to the other trials), that means the aluminum adjuvant is not an “inert placebo” and that it causes harm, meaning that the clinical trials comparing the two don’t demonstrate the safety of the vaccine. Blaxill’s logic is tortured at best, intellectually dishonest at worst, and at the very least the result of an utterly incompetent analysis.

Of course, thus far, Blaxill has only been discussing relatively minor reactions, such as local injection site reactions, and trying to use them to imply that, if the aluminum adjuvant causes the same frequency of local injection site reactions, it must also cause the same frequency of serious adverse reactions, like death. So he harps upon what he sees as an excessively high number of deaths in the Gardasil clinical trials, even though there were no differences between the control and the Gardasil groups. Even though the majority of deaths in both groups were due to trauma, and the others did not have a history that was in any way suggestive of being caused by the vaccine. Then, like all good anti-vaxers, Blaxill mines the VAERS database for more deaths, none of which appear on the surface to be related to the vaccine.

All of this “analysis” leads Blaxill to proclaim that Gardasil was not safe. He also claims that the clinical trials were done at a lowered standard of safety, which is simply not true; it’s a claim based on Blaxill’s ignorance of how clinical trials are done and what a valid placebo control is.

When I started this post, I originally planned for it to be a quickie, a little post mentioning Blaxill’s misunderstanding of what a valid placebo is and, quite frankly, making fun of it, especially given that Blaxill’s post is basically a rehash of posts from a year ago. After all, there’s the big Newsweek cover story out there about overdiagnosis and a truly annoying article elsewhere about how “everything you know about cancer is wrong,” one of which I had planned on dealing with. But then I noticed that Blaxill’s arguments were virtually identical to those of one of the crankiest anti-vaccine cranks out there, who posted the same sorts of arguments three years ago. Then I delved into the rest of the article itself and realized it to be an updated treasure trove of anti-vaccine crankery that I had somehow missed when it first appeared last year. As I wrote, I found myself being sucked in by its bad science, bad data presentation, and bad statistics, and you know how Orac can’t resist such a “target-rich” environment.

Oh, well. There’s always tomorrow or Friday to deal with those articles, or maybe I’ll do it at my other blogging location. In the meantime, I want to thank Mr. Blaxill for the yucks and the “teachable moment.” It takes a rare talent to be able to combine a massive demonstration of the Dunning-Kruger effect with such a badly done “analysis.” Truly it is not for nothing that Blaxill is the “scientist” at AoA.