Generation Rescue and “Fourteen Studies”

About a week and a half ago, something happened that makes me realize that the Jenny and Jim antivaccine propaganda tour that I mentioned a couple of weeks ago was clearly only phase I of Generation Rescue’s April public relations offensive. About ten days ago, courtesy of J.B. Handley, the founder of Generation Rescue, who in order to have a couple of famous faces fronting his organization has allowed himself to be displaced, so that Generation Rescue has now been “reborn” as Jenny McCarthy and Jim Carrey’s Autism Organization (the better to capitalize on her D-list celebrity yoked to Jim Carrey’s formerly A-list (but rapidly plunging) celebrity), announced Generation Rescue’s latest initiative in a post on its antivaccine blog Age of Autism entitled Fourteen Studies? Only if you never read them.:

For me, it all started when Amanda Peet said the following in her “apology” to calling parents parasites:

“Fourteen studies have been conducted (both here in the US and abroad), and these tests are reproducible; no matter where they are administered, or who is funding them, the conclusion is the same: there is no association between autism and vaccines.”

And, don’t think Amanda Peet is alone, the mantra comes fast and furious from all sides…

These comments were driving me nuts. I’d read a majority of the studies they were referring to, I knew how bad they were, and I also knew that most journalists couldn’t even find the studies being referred to, because most weren’t even on the web!!

Several hundred hours of work later, Generation Rescue is pleased to introduce a website with a very simple purpose: to tell the truth. You will find every study in its entirety and a whole lot more right here:

www.fourteenstudies.org

Before I dive in, let me just point out right here and right now that J.B. Handley wouldn’t be able to recognize good science if it bit him on the posterior. The same is true of bad science, because Mr. Handley simply does not have an understanding of the scientific method or the methodology involved that would allow him to distinguish good from bad science, anymore than I have an understanding of investment banking that would allow me to differentiate between various financial instruments. No, on second thought, strike that. I’m quite sure that I know more about investment banking than J.B. Handley knows about science.

Indeed, Mr. Handley’s well-known arrogance of ignorance (shared with his new spokesmodel Jenny McCarthy) leads him to be supremely confident that he knows as much or more than us pointy-headed scientists, whom he clearly despises and and regards with contempt. Well, actually, he only regards scientists who disagree with his fervent belief that mercury in vaccines cause autism with contempt. Wait, strike that, since all the studies rolling in over the last five years that have failed to find a link between mercury in vaccines and autism, coupled with the fact that the amount of mercury exposure American children receive through vaccines is lower than it’s been in at least 25 years, with no signs of the “autism epidemic” abating, Mr. Handley has turned on a dime and shifted the goalposts to change his belief to claiming that it’s the big, bad vaccines themselves, that it’s “too many too soon,” and that the U.S. is the most “overvaccinated” countrie in the world. So now Mr. Handley only views with contempt scientists who look at all the studies in their totality and don’t see even a whiff of a correlation between vaaccines and autism. Any “scientist,” even utterly discredited incompetent “scientists” like Andrew Wakefield, are “heroes” to him.

In any case, thanks to its arrogance of ignorance, Generation Rescue thinks it can judge the quality of complicated epidemiology and basic science, but such pronouncements are about as valid as Joe the Plumber holding forth on quantum physics. That’s why Mr. Handley’s claim that he recognized these studies to be bad science by reading them led me to chuckle heartily. After all, Mr. Handley’s proven time and time again that he doesn’t understand science, the scientific method, or epidemiology, and I find the picture that pops into my mind of Mr. Handley poring over epidemiological and scientific studies, carefully making note of the methodology and picking out the flaws almost as outlandish as Jenny McCarthy’s transformation from an “Indigo Mom” to a “warrior mom” for autistic children. That’s because J.B. also understands nothing about clinical trials and especially the ethics of clinical trials, one particularly egregious example of which I should someday post about. Be that as it may, right off the bat, I knew that this “Fourteen Studies” initiative was likely to be more of the same. The claim that it took “several hundred hours of work” is probably true, but, given the results, but it clearly wasn’t time and money well spent, at least on an intellectual level. Whether it was time and money well spent in terms of producing effective propaganda remains to be seen.

Taking a general view of the Fourteen Studies, I note that it’s divided broadly into what I can’t help but refer to “studies antivaccinationists hate” (because they are for the most part large, well-designed epidemiological studies that found no association between thimerosal and autism or vaccines and autism) and “studies antivaccinationists like.” Of this latter category, the studies listed that I recognize include some small preliminary studies (which are more prone to false positives), more than a fair amount of cluelessness (the Generation Rescue “vaxed versus antivaxed” phone survey, which I’ll discuss more later), the incompetent (pretty much anything by Andrew Wakefield or Mark and David Geier), or execrable (anything published in the vanity Medical Hypotheses, which discourages the publication of data and will publish virtually article as long as it presents a provocative and speculative hypothesis, making it a “what if?” speculative journal and not a serious research journal , or the crank journal, the Journal of American Physicians and Surgeons (JPANDS), whose penchant for antivaccine pseudoscience I’ve discussed in depth before). Heavy hitters in the world of vaccine or autism science, these are not. (Hint: If you want to be taken seriously in the world of medical science, don’t cite JPANDS. Really. Don’t. Doing so is a deal breaker. Also, citing articles from Medical Hypotheses as though they were evidence of anything other than the author’s speculative flights of fancy or wishful thinking shows more cluelessness than anything else.)

I also recognize that a lot of these criticisms are the same old dubious, exaggerated, and/or fallacious criticisms of these studies that have been floating around in antivaccine circles for a long time. I recognize them because I have read nearly all of these articles myself, both the ones Generation Rescue likes and the ones it doesn’t like, over the years, and I have seen these “criticism” True, I’m more familiar with some of these studies than others, but in fact I have even blogged about some of them, here and elsewhere. (I will link to such posts where appropriate.) In essence, what I plan to do is to set the stage by discussing the overall intellectual dishonesty that underpins Fourteen Studies and then take a few (I hope) well-chosen examples to illustrate this intellectual dishonesty. Over the coming days, it is my hope that some of my fellow SBM bloggers will take on other examples, explaining the fallacies behind the “critiques” and why the “studies antivaccinationists like” are, by and large, not very good and/or do not show what antivaccinationists think they show.

I do this because I realize that the Fourteen Studies website is going to metastasize all over the web. That is clearly Mr. Handley’s intent; he saw what he feared to be an effective public relations gambit from scientists and physicians, and he couldn’t let that go unanswered. To him, it’s not about the science, but winning the P.R. war. That’s why I know that antivaccinists will be referring journalists to it, if they have not already begun to do so. I also predict that antivaccine parents will be planting printouts of it in front of their pediatricians in order to try to persuade them that the evidence base supporting the consensus that vaccines do not cause autism is hopelessly corrupt and weak. This is exactly the purpose of Generation Rescue’s new website, which is why I believe that there should be as many rebuttals on the web to which people can be referred as possible. In fact, it is my hope that my readers will help us out by deconstructing some of the studies in the comments, and that readers who happen to be bloggers will join in this effort by applying some evidence-based criticism to the site.

First, let’s take a look at how Generation Rescue “ranks” these 14 studies. This alone should show you more than anything else why this site is propaganda, not science. Whoever put it together first ranked the studies on a scale of 40 points, made up of four 10-point subscales. The first of these subscales is described thusly:

Asked the Right Question (0-10 points):
Did the study actually contemplate the real world example of a parent vaccinating their child with 5 or more vaccines and then seeing a regression into autism afterward? Did it contemplate something close to that which could be helpful and generalized? A perfect question received a 10, a study that didn’t even contemplate the question at hand received a zero.

See what I mean? This is, of course, not the “right question” at all, other than in the minds of antivaccinationists who have already made up their minds about the answer. This is, at its heart antiscientific, an appeal to anecdotal evidence (“contemplate the real world example”) framed in the most “dramatic” way possible, full of inflammatory language about “five or more vaccines.” In other words, it’s typical antivaccinationist appeal to emotion, rather than science. Unfortunately, such appeals to emotion are often more effective than any appeal based on science. In any case, the right question is whether or not there is an association or correlation between autism and vaccination.

Also, note how vague and subjective this scale is. What determines how close to the “right” question a hypothesis will be ranked, apparently, is whether it sounds good to the Generation Rescue antivaccinationist who ranked it. Certainly there do not appear to be anything resembling even semi-objective criteria for determining the score, which are essential for any sort of scale like this. I also note with some amusement that one of the studies included as “Our Studies” on this website, namely a 2006 study by Raymond Palmer looking at whether proximity to coal-fueled power plants is associated with increased prevalence of autism, would get a zero on this scale, as it didn’t even look at vaccines at all! It was also a really crappy study, as I discussed extensively a few months ago during a deconstruction of Palmer’s followup study. At least Generation Rescue could have cited Palmer’s 2008 followup study instead, which, while still not very good at all (an understatement), was at least not as spectacularly awful as his 2006 study, which didn’t even bother to try to control for some very obvious confounders. Apparently asking the “right” question doesn’t really matter to Generation Rescue if J.B. Handley likes the answer. Nor does crappy science. As long as the answer, no matter how unrelated or tangential to a study and no matter how bad the study, is one that Mr. Handley likes, then it’s not “crappy science.”

Next, Generation Rescue lists:

Conflict of Interest (0-10 points):
We considered a scientist employed by a vaccine maker or a study sponsored by a vaccine maker to have the highest degree of conflict, with a public health organization (like the CDC) to be the second-worst. A conflict free study would receive a 10, a study rife with conflicts as low as zero.

This one really annoyed me, because I’ve discussed on multiple occasions how rife with conflicts of interest are many of the “studies” used by antivaccine advocates to support their point of view. For example, the grossly incompetent “study” by Generation Rescue that I discussed last week and that is featured on Fourteen Studies was commissioned and performed by an organization whose raison d’être is to convince lawmakers and the public that vaccines cause autism and that “biomedical treatments” targeted at “vaccine injury” can cure autism. If that’s not a conflict of interest, I don’t know what is. The same is true of an awful study of vaccination in monkeys by Laura Hewitson (which I blogged about several months ago and that Generation Rescue widely promoted, but apparently decided to leave off of its latest propaganda initiative even though it would get a 10 out of 10 on “asking the right question” by Generation Rescue’s definition. (Maybe the criticisms stung.) After all, Laura Hewitson is not only married to antivaccine hero Andrew Wakefield’s IT director but she and her husband are complainants in the Autism Omnibus proceedings, which led me and others to wonder at the time if their “monkey study” was in fact done explicitly to provide more ammunition for the test cases under consideration by the Vaccine Court at the time.

Finally, let’s not forget Mark and David Geier, both of whom have much invested in the concept that mercury in vaccines causes autism and advocate, in essence, chemical castration with Lupron as a “treatment” for autism to help “free” the mercury from testosterone and allow it to be chelated more effectively. In fact, they took conflicts of interest to whole new levels undreamt of by even the most cynical, greedy, and amoral big pharma executive when they specially created an IRB and stacked it with their cronies in order to approve “studies” of their Lupron therapy. Any of their studies would get a zero on this the conflict of interest scale. Heck, they’d get negative scores! And all of this doesn’t even count Andrew Wakefield, who was shown by Brian Deer not only to have been an incompetent scientist, but in the pocket of trial lawyers suing vaccine manufacturers for “vaccine injury.” Worse, these are undisclosed conflicts of interest, whereas any conflict of interest listed in the “fourteen studies” was disclosed right there on the manuscript. In marked contrast, it took Brian Deer years to dig up Andrew Wakefield’s blatant conflicts of interest; Kathleen Seidel’s digging to unearth the Geiers’ abuse of the IRB process; and the digging of several bloggers to figure out Laura Hewitson’s undisclosed conflict of interest in being a plaintiff in the Autism Omnibus. If there’s one rule of thumb in medical research, it’s that undisclosed conflicts of interest correctly raise far more suspicion of bias than disclosed ones do. Indeed, I’ve written about this myself in the context of criticizing JAMA and other biomedical research that has nothing to do with vaccines and autism.

Cleverly, Generation Rescue disingenuously inoculates itself from charges of picking studies with huge conflicts of interest by narrowly defining a conflict of interest the way that it wants to: As an investigator’s only being affiliated with a vaccine maker or the CDC. (Because, you know, antivaccine warriors can’t possibly ever have a conflict of interest.) Then, to guarantee that every study that it doesn’t like has an irretrievable “conflict of interest,” even if there is no pharmaceutical company connection that it could dig up, Generation Rescue appears to consider the mere fact of the lead investigator working for the NIH to be a severe conflict of interest. That is what is listed as the sole conflict of interest for Dr. Karen Nelson, who wrote a review article on thimerosal and autism. If that’s not enough to guarantee a “conflict of interest” for all of these “fourteen studies,” Generation Rescue also appears to define the mere fact of being funded through grants from the CDC, NIH, American Academy of Pediatrics, or the Canadian Institutes for Health Research. Scientists who have had NIH grants (such as myself) or grants from any of these other organizations know just how ridiculous considering that funding source to be a horrific conflict of interest is. After all, such grants are in general competitive grants awarded after rigorous peer review, and the NIH, for example, exercises almost no control over how researchers funded through it do their work, other than enforcing federal regulations on animal and human subjects welfare by refusing to disburse funds until the appropriate approvals are in place.

Next up, Generation Rescue ranks the ability to generalize and post-publication criticism:

Ability to Generalize (0-10 points):
This was a measure of the robustness, replicability, and usefulness of the study. Where possible, we looked to experts to help us gauge this ranking. The more robust, replicable, and broadly applicable, the higher the score.

Post-Publication Criticism (0-10 points):
Was the study widely accepted in the scientific community, or was it the subject of extreme criticisms from many sources? The closer to widely accepted, the closer to 10 points.

One wonders why Generation Rescue didn’t always look to experts to gauge this ranking. After all, neither J.B. Handley, Jenny McCarthy, nor Jim Carrey are scientists. In fact, they have shown time and time again that they are clueless about science. Any conclusion by them or the other antivaccine warriors at Generation Rescue on the generalizability of a study would be highly suspect even giving them every benefit of the doubt. As part of this propaganda effort, forget it. Also, one can’t help but note that the “experts” Generation Rescue does find happen to be antivaccinationists like Boyd Haley, the former chairman of the Department of Chemistry at the University of Kentucky whose academic career imploded as a result of his having delved too deeply for too long into mercury and vaccines pseudoscience (as well as “toxic teeth” amalgam pseudoscience), or Mark Geier, whose protocol that uses Lupron to treat autism is, in my considered opinion, dangerous quackery. In other cases, the “experts” are nothing more than some of J.B. Handley’s fellow antivaccine activists with no special scientific or medical training: Sallie Bernard of SafeMinds or Deirdre Imus–or even J.B. Handley himself. Such are Generation Rescue’s “experts”! Clearly, the talent pool of “experts” at Generation Rescue is not particularly deep. One wonders why Generation Rescue didn’t draft Jim Carrey or Jenny McCarthy to do a critique of one of these articles.

Also, it’s extremely disingenuous of Generation Rescue to rank these studies on “post-publication criticism” because it is the antivaccine movement, including Generation Rescue, that accounted for the vast majority of criticisms of these articles (most were not particularly controversial among scientists). Indeed, the antivaccine movement encouraged criticism at every turn. In most cases, Generation Rescue is conflating criticisms based on ideology with scientific criticisms, which is how they keep the vaccine/autism manufactroversy alive. One can’t help but note again that, if post publication criticism were a major criterion by which Generation Rescue’s favored studies should be judged, Andrew Wakefield’s 1998 Lancet study was disavowed by 10 of its original 13 authors and ultimately utterly repudiated by the scientific community–and deservedly so! Yet there it is on the “Our Studies” page of Fourteen Studies. Apparently only criticism by the antivaccine movement counts as a negative to Generation Rescue. Criticism by scientists? Apparently not so much.

So let’s move on to some of the studies. I’ll start with Mady Hornig’s recent study. Even though it was obvious from the start that this website is every bit as much propaganda as the Green Our Vaccines rally in Washington, DC last year, still I wanted to see what Generation Rescue said about some of the studies. So I picked a few that I am quite familiar with. First, off, I noticed that Mady Hornig’s study from last fall that tried to replicate Andrew Wakefield’s original Lancet study that implicated the MMR vaccine in the pathogenesis of “regressive” autism and enterocolitis. You might remember this one too. You might also recognize that all Generation Rescue did was to regurgitate old attacks by Andrew Wakefield, SafeMinds, and the National Autism Association on it. You may also recall that I blogged about this study shortly after it was published and showed you why these criticisms were, scientifically speaking, a load of horse hockey. Suffice it to say, they have not aged well.

Another study included on the list is an Italian study that came out this year by Tozzi et al, which was published in the February issue of Pediatrics and entitled Neuropsychological Performance 10 Years After Immunization in Infancy With Thimerosal-Containing Vaccines. It’s a study that I also blogged about in great detail when it first came out. The study was done in Italy, and one of its great advantages is that the amount of thimerosal to which the infants were exposed was actually known, unlike many epidemiological studies, where sometimes the dose of thimerosal (and therefore mercury) has to be estimated or inferred from the vaccine schedule at the time. The results were unsurprising and very much like the results of a study of thimerosal-containing vaccines as a risk factor for neurodevelopmental disorders other than autism that was published a year and a half ago. Most outcome measures showed no difference between the low and high thimerosal group, and the ones that did were small and entirely compatible with random chance due to multiple comparisons.

I knew at the time exactly what antivaccine activists would probably say when and if they get around to attacking this study, and, in fact, that’s basically the criticism on Fourteen Studies. (Most depressingly, Generation Rescue even reprints J.B. Handley’s scientifically illiterate criticism of the study.) In any case, the main criticism is that there was not a control group receiving no thimerosal. True enough. The authors themselves make that very point. However, if thimerosal in vaccines were associated with autism, one would not expect that it would be different than any other toxin associated with an abnormality, disease, or condition in that it would be expected that the chance of autism or neurodevelopmental disorders would increase with increasing dose.

The second argument that I thought advocates would try to make but that Generation Rescue actually did not make is that the dose-response curve for mercury and autism has a plateau, and that plateau is below 62.5 μg, hence the lack of difference between the two groups. There’s just one problem with that argument. An exposure to 62.5 μg, to which the low exposure group was exposed, corresponds to roughly the total dose of mercury in thimerosal to which American infants were exposed in 1989—before the alleged “autism epidemic.” Even if mercury does indeed cause autism and there is indeed a plateau in the dose-response below a dose of 62.5 μg, that would not be consistent with the antivaxers’ other pet claim, that an autism epidemic started in the 1990s because of the increasing amount of thimerosal exposure due to vaccines. That couldn’t have happened if a dose of thimerosal less than 62.5 μg maxed out the risk of autism, because 62.5 μg below the baseline exposure before the alleged “autism epidemic” started. The two claims (that of an autism epidemic in the 1990s due to increasing amounts of thimerosal in vaccines versus that of an effect that maxes out before a 62.5 μg cumulative dose of mercury from thimerosal) are mutually contradictory. I suppose antivaccinationists could postulate a threshold effect that doesn’t occur until a dose above 137.5 μg. Unfortunately for them, then they would have the problem of how long they ranted that any mercury was toxic and any mercury was unacceptable, not to mention the–shall we say?–inconvenient epidemiological evidence that autism rates did not plummet back to 1980s levels after 2001, which was when thimerosal was removed from most childhood vaccines and mercury exposure from vaccines plummeted to well under 62.5 μg.

Finally, the most ironic thing about this study is the question of why autism prevalence was so low among the Italian population studied. Of course, as Generation Rescue points out, all of the subjects received at least 62.5 μg mercury from their vaccines; so by antivaccinationist logic there should have been a much larger prevalence of autism, particularly among the Italian children who received 137.5 μg mercury. There wasn’t. The prevalence was much lower than in the U.S., which also rather undermines the claims Generation Rescue made in its recent “vaccines by nation” nonsense. So much for “mercury poisoning,” at least in this case.

Antivaccinationists really, really hate this next study (Schechter et al) because it looked at a very specific prediction that naturally follows from the hypothesis that mercury in the thimerosal preservative in vaccines was a major cause of the “autism epidemic.” I say “was” because by early 2002, thimerosal had been removed from the vast majority of vaccines, and this allows the testing of the hypothesis. Specifically, the prediction is that, if thimerosal in vaccines was indeed a major cause of autism, autism incidence should have plummeted within a few years after it was removed. This study showed that there is no evidence that the apparent rise in autism is even plateauing yet in California. In a truly fun bit of additional irony, this study even uses the California Department of Developmental Services (CDDS) database, to which David Kirby has often referred as a good way of looking at whether autism rates have started falling since thimerosal was removed from most vaccines. He has also since been moving the goalposts as to when he would expect autism rates to begin falling if the thimerosal hypthesis is true, going first from 2005 to 2007 and, most recently, to 2011.

In order to ask the question of whether autism rates had declined, Schechter and Grether examined data for clients with active status reported from January 1, 1995 to March 31, 2007. Using careful statistical analyses, they used two approaches to measure the occurrence of ASD during this period. The second approach, in which ASD prevalence was determined in the 3 to 5 year old cohort, is perhaps the most informative. It shows a continuing increase in autism prevalence without even a blip or decrease in the rate of increase after 2002. Indeed, showing the skill of some bloggers to analyze the same data, the money figure in the paper (Figure 3) looks almost exactly the same as the graph prepared in early 2007, a continually increasing curve since 1995. This result is not only consistent with multiple other published and unpublished studies, including the famous (or, if you’re an antivaccinationist, infamous) Danish and Canadian studies, but it is about as unambiguous evidence as can be obtained from a database like the CDDS database. Indeed, despite the known limitations of the use of this database, it is an excellent example of proponents of a “mercury injury” hypothesis of autism being “hoisted by their own petard,” so to speak. Even more amusingly, one of the criticisms of the study was written by Deirdre Imus, who has been known to regularly lay down some of the more–shall we say?–easily refuted antivaccinationist canards, all cloaked in “concern for the children,” is one of the “experts” refuting this study, while Boyd Haley wrote a long rant that is mostly a non sequitur.

The complaints about this study from antivaccinationists can be boiled down to one: That mercury hasn’t been completely eliminated from childhood vaccines. In other words, even though mercury has been removed from all mandated childhood vaccines other than the flu vaccine (and there are thimerosal-free versions of the flu vaccine available), leaving only trace thimerosal in the current set of vaccines, according to antivaccinationists that’s not enough. To them, mercury is just that toxic, and even the trace levels left in the vaccines are enough to keep the “autism epidemic” moving in a steadily upward direction. This is an utterly fallacious criticism for exactly the same reasons the criticisms of the Italian study listed above are similarly fallacious. Once again, remember that the “mercury hypothesis” states that an “autism epidemic” began in the late 1980s/early 1990s after the government expanded the number of mandated vaccines containing thimerosal. A consequence of that claim is that we should expect late 1980s/early 1990s levels of autism to be associated with a certain dose of mercury in vaccines, which, as you remember, was approximately 60-75 μg in 1989–by antivaccinationist Mark Blaxill’s own estimates! With now thimerosal removed from all vaccines save the flu vaccine (which many children still don’t get anyway), all that is present in the total vaccine dose received by children is a level of mercury less than what children received in 1989. So, in the world of antivaccinationists, lowering the level of total mercury received in vaccines by children to a level lower than what children received 20 years ago (before the “autism epidemic”) is still too toxic. (Consistency has never been the antivaccine movement’s strong suit.) Unfortunately, by the antivaccinationists’ own arguments, if the graph that Mark Blaxill likes to show represents not just correlation but causation, the dose-response curve for mercury must be such that the level in 1989 resulted in a relatively low level of autism; therefore, lowering it to levels below those seen in 1989 should result in a dramatic decrease in autism rates–again, using the mercury militia’s own numbers.

It didn’t. Stick a fork in the mercury-thimerosal-autism hypothesis. It’s quite done. The hypothesis that mercury in vaccines is strongly (or even weakly) associated with adverse developmental outcomes is also done as the next study (Thompson et al) shows. This study did not specifically look at autism, but was the first publication of a larger study looking at neurodevelopmental outcomes after vaccination using data from the Vaccine Safety Datalink (VSD) database, a massive database overseen by the CDC and maintained through several large HMOs that collects data on vaccination outcomes. The first publication, Thompson et al, looked at whether there were adverse neurodevelopmental outcomes other than autism that could be identified as being correlated with thimerosal-containing vaccines. A second study looking specifically at autism using the VSD is in the works.

The authors found weak associations between thimerosal and developmental outcomes that were most consistent with being due to chance. It is true that there were some negative correlations found that achieved statistical significance. However, when running 42 tests, it would be shocking if there were not a few anomalous associations. What makes the study authors confident that the findings are anomalous is that they were divided roughly equally in both directions, good and bad. It is true that there were some negative correlations found that achieved statistical significance. When running 42 tests, it would be shocking if there were not a few anomalous findings. What makes the study authors fairly confident that the findings are anomalous is that they were divided roughly equally in both directions, good and bad. Consequently, if antivaccinationists are going to insist that the correlation, for example, with increasing mercury exposure and poorer performance on the GFTA-2 measure of speech articulation test is real and due to likely causation, then it must also accept the findings of a beneficial association between mercury and identification of letters and numbers on the WJ-III test, as there is no scientific reason to reject it. Even Autism Speaks, one of the “guest commentators” in Fourteen Studies, concedes this but tries to latch onto these few findings as being somehow significant.

Another important aspect of this study is that Sallie Bernard of SafeMinds, was a panelist on the consulting board that helped design this study. She pulled out when the results weren’t what she had been hoping for and has been criticizing the study ever since, up to and including being one of the “guest commenters” on this article for Generation Rescue. Her complaint is the most ridiculous of all, specifically that the study

comprised children who were least likely to exhibit neuropsychological impairments. Specifically, children with congenital problems, those from multiple births, those of low birth weight, and those not living with their biological mother were excluded. The sample was skewed toward higher socioeconomic status and maternal education — factors that are associated with lower rates of neurobehavioral problems and higher intervention rates and that were not measured.

Ms. Bernard is too scientifically illiterate to realize that that’s because the hypothesis being studied was that mercury in vaccines is associated with neurodevelopmental problems in normal children. Remember, in virtually all of the testimonials about regressions after vaccines, parents insist that their child was absolutely, positively normal before those horrible pediatricians pumped them full of those nasty, toxic, autism-causing vaccines. In fact, this study was looking at exactly the “right question.” Also (and arguably more importantly for the study design) children with congenital problems and of multiple births or of low birth weight have a higher baseline level of neurodevelopmental problems. Including such children would have (1) made it difficult to determine whether thimerosal actually was associated with neurodevelopmental difficulties because the effect size could well be dwarfed by the baseline levels of neurodevelopmental disorders in this population and (2) introduced confounders that would have been very difficult to control for in the final statistical analysis.

In any case, Ms. Bernard is being intellectually dishonest to an extreme. She participated in the design of the study but never criticized its design until after its results contradicted her expectations. Does anyone think she would have had a problem with this study if it had come back less resoundingly negative, with even a bit of wiggle room? Such is the intellectual dishonesty of the antivaccine movement and a major reason why it was always a bad idea for the government to include antivaccinationists in the the decision-making process about anything in the name of “inclusion” and the hopes that if they had a stake in the outcome they would be more constructive. The government hoped to shut up antivaccinationists by including them on such panels; it should have realized that they would turn on the government as soon as it didn’t get its way or as soon as scientific studies in whose design they participated failed to yield the results craved by the antivaccine movement.

Finally, before I leave the discussion of the fourteen “studies Generation Rescue hates” to move on to discussing a couple of the studies they like, I point you to Steve Novella’s excellent deconstruction of the intellectual dishonesty of Generation Rescue’s critique of the Danish study, which failed to find any association between mercury in vaccines and autism in Denmark. Truly, it’s hard to take anything Generation Rescue writes about these studies seriously when it can’t even accurately describe the studies and what they concluded. The intellectual dishonesty is staggering.

I was curious to note what Generation Rescue apparently considers to be “good” studies. I already mentioned some of the awful studies, such as those by Andrew Wakefield or Mark and David Geier, that Generation Rescue likes. One study from 2009 included on the list is one I haven’t seen before, and I may well write about it on its own, as it makes a provocative but, in my opinion, ultimately highly doubtful conclusion.

One constant refrain of antivaccinationists is that there should be a study of “unvaccinated” versus “vaccinated” children. Never mind that they never quite state the hypothesis that such a study would study and sometimes even suggest utterly unethical studies, such as a randomized double-blind, placebo-controlled study. Such a study, although methodologically rigorous, would be completely unethical because it would leave the control group unprotected against vaccine-preventable diseases. Nonetheless, that doesn’t stop Generation Rescue from listing as number two its utterly risible and badly executed telephone study from 2007. As you may recall, I discussed this “study” in my usual gory detail back when it was first released and publicized. More amusingly, so did Kevin Leitch. Go back and read them both if you want to see how execrable this “study”was. So bad was it that Kev drolly observed:

There’s no getting away from this. This is a disaster for Generation Rescue and the whole ‘vaccines cause autism’ debacle. Generation Rescue’s data indicates that you are ‘safer’ from autism if you fully vaccinate than partially vaccinate. It also indicates that across the spectrum of autism, you are only 1% more likely to be autistic if you have had any sort of vaccination as oppose to no vaccinations at all – and thats only if you are male. If you are a girl you chances of being on the spectrum are less if you have been vaccinated! Across both boys and girls, your chances of being on the spectrum are less if you have received all vaccinations.

Prometheus has done an excellent job discussing how, if we really wanted to undertake a “vaccinated versus unvaccinated” study with both a reasonable degree of scientific rigor and ethically as well. Unfortunately, it would take a minimum of 20,000 children:

A study of autistic children in the 3 – 6 year age range would need over 683,000 children in each group to detect a 10% difference in vaccination level. It would need nearly 22,000 in each group to detect a 50% difference. With a predicted number of 110,000 autistic children in that age range, that is a sizeable fraction of all autistic children.

A more manageable study – one with 10,000 children in each arm (which is still a HUGE study!) – would only be able to tell the difference between the national average of 0.3% unvaccinated in the non-autistic group and 0.1% unvaccinated in the autistic group (at the specified levels of confidence). If the difference is smaller than that, the results would be considered negative (i.e. that there is no effect of vaccination). For reference, a study with 1,000 children in each arm would show statistical significance (at our specified level of confidence) only when the autistic group was below 0.01% unvaccinated or above 1.7% unvaccinated.

Of course, we wouldn’t have to just look at unvaccinated vs fully vaccinated with this study, which is a large part of its superiority. We could look at a dose response of vaccination - to see if it really is “too many” – as well as the age at youngest vaccination – to see if it really is “too soon”. In fact, a few studies have already looked at those issues and found that there is no difference between the autistic and non-autistic groups. I suspect this is the reason the folks pushing to “put on a study” want to look at vaccinated vs unvaccinated – they hope that the numbers will be different (or, at least, not as definitive) the other way round.

This would, of course, be a massively expensive study to do as well and would funnel more research money to dubious studies of highly unlikely hypotheses.

Finally, if you want an idea of how risibly low Generation Rescue’s standards are for studies that seem to support its belief that vaccines cause autism, take a look at its most recent atrocity (before the “Fourteen Studies” website, of course), namely a “study” unveiled by Jenny McCarthy and Jim Carrey on Larry King Live three weeks ago:

MCCARTHY: The vaccine schedule is too bloated right now. Thirty six shots right now. Back in 1989, the shot schedule was 10 shots given.

CARREY: Ten shots given.

KING: When I was a kid, what did we get, three?

CARREY: It’s twice as many as anywhere else in 30 countries in the Western World. We give twice as many shots as any of those countries. Why is that?

This point is made in more detail in J.B. Handley’s guest post:

Few parents appreciate that American kids are the most vaccinated on the planet. Generation Rescue just released a study called “Autism and Vaccines Around the World” which will surprise many. We looked at the vaccine schedules of 30 other first world countries to compare how many doses of vaccines children receive. What did we find? Compared to our 36, the average for the rest of the first world is 18, or half of the U.S. schedule. Perhaps more shocking, we looked at countries with the lowest rates of mortality for children under 5 (the U.S. ranks a disappointing 34th, behind Cuba and Slovenia). How many vaccines do the 5 countries with the lowest under 5 mortality rates give? Well, Iceland, Sweden, Singapore, Japan, and Norway give 11, 11, 13, 11, and 13 vaccines respectively – all less than 1/3 the number of vaccines the U.S. mandates!

How do autism rates compare in some of these other countries? Iceland’s rate is 1 in 1,000, Finland’s 1 in 700, and Sweden’s 1 in 800. These countries give 1/3 the vaccines we do and have autism rates that are as little as one-tenth of ours? Something isn’t right.

Indeed, this is the very theme of a “study” posted on the Age of Autism blog entitled Autism and Vaccines Around The World: Vaccine Schedules, Autism Rates and Under 5 Mortality. If you look at the actual PDF containing the report, you’ll see that it’s obviously intentionally designed to look very much like an actual research report. The type, the format, and the style very closely mimic that of a real research paper. Of course, it is not anything of the sort, and if it were submitted to any halfway decent peer-reviewed journal of psychiatry or–even more so–of epidemiology it would find its way into the cylindrical file, possibly even before being sent out for peer review. It’s nothing more than the logical fallacy of confusing correlation with causation taken to a ridiculous extreme and then prettied up with all sorts of “science-y”-sounding jargon in an attempt to put the proverbial lipstick on the proverbial pig. Whoever put this “study” together at Generation Rescue clearly has no clue how to write such a study up, much less how to do one.

The first thing that struck me about this “study” was how poorly written it was. It looked slapped together, and, although it states that a “full publication and literature review was completed to determine vaccine schedules and under 5 mortality rates for 30 countries. What, exactly, does that mean? If you read any meta-analysis or serious review, you’ll see a careful and precise description of the criteria used to choose studies to include. No such description is given here. One is left with the distinct impression that a bit of cherry picking is going on here.

The second thing that leapt out at me is that there was absolutely no attempt to control for confounders. Indeed, one commenter on Age of Autism, responding to another commenter who pointed out that zero controlling for confounders was even attempted said:

Confounding factors? Since when is that ever a consideration? Perhaps you watch too much television and, based on dispropotionately biased coverage, think every parent in America whacks their kids on the head then stuffs them in the trunk.

These are babies under five.

Against such ignorance and stupidity, the gods themselves contend in vain. The very reason it’s so easy to confuse correlation with causation, even in epidemiological studies that are testing a hypothesis rather than trying to support a preordained conclusion, is because of confounders, and, yes, it is completely irrelevant that these children are under five.

The reasons why the U.S. ranks 34th in childhood mortality is a topic of some controversy and considerable study and discussion. Of course, one wonders why Generation Rescue used the UN World Population Prospects The 2006 Revision when the 2008 Revision is easily available. (Apparently Generation Rescue can’t Google). (It couldn’t be because the under-five mortality rate in the U.S. fell from 7.8/1,000 live births to 7.3 between 2006 and 2008, could it?) Be that as it may, while the Generation Rescue “study” appears to be claiming that vaccines don’t have any relationship to overall childhood mortality, that’s completely the wrong question to be asking because vaccines do not prevent anything other than infectious disease. If we take a look, for example, the 2008 State of the World’s Mothers Report issued by Save the Children, we find some very interesting things, among which is the effect of disparities in health care among the poor, mostly black and Native American, and the better off sustaining our high child mortality compared to that of other developed nations. That’s one huge confounder. Another huge confounder is access to health care; many of the nations mentioned in the Generation Rescue report are nations with universal health care. Whether that drives the difference is a matter of controversy (given the politically charged nature of the discussions of universal health care, how could it be otherwise?) but it is yet another huge potential confounder. Of course, the biggest confounder of all is that childhood mortality has been falling in the U.S. for the last two decades. Add to that gun violence and other causes of childhood mortality that are elevated in the U.S. compared to other industrialized nations, and there are so many confounders as to make any inference from these raw numbers utterly meaningless. Again, vaccines only prevent death from infectious disease. Ironically, one of Generation Rescue’s heroes, a woman who’s been flirting with the antivaccine movement big-time, former NIH director Bernadine Healy, pointed out a lot of reasons why comparing childhood mortality in the U.S. with that in various European countries is fraught with confounders, making head-to-head comparisons of dubious value in many cases.

There is also at least one curious omission in the last table (Table 3), which was pointed out, surprisingly enough, by a commenter on Age of Autism. (And when a commenter in the antivaccine echo chamber that it Age of Autism points out a glaring hole in a Generation Rescue propaganda release, you know it’s a really huge hole! This commenter also pointed out that there was not even the most rudimentary attempt to control for counfounders. Truly, this propaganda stinkpile of a “study” is too stinky for any but the most brainwashed of Generation Rescue’s adherents.) Specifically, the U.K. was left out, which is listed as requiring 20 vaccines. That is close to France, which is listed as requiring 17 vaccines. One wonders if this omission is because the U.K. has an autism prevalence comparable to that of the U.S., possibly even higher. I wonder why Generation Rescue left the U.K. out of its last table, purporting to show the relationship between the number of vaccines a nation mandates and its autism rate. After all, the paper claims that it picked countries based on “materially lower levels of mandated vaccines and published autism prevalence data.” I could see where it might be hard to find publications with estimates of autism prevalence in, for example, Cuba, but it’s not as though recent estimates of autism prevalence in the U.K. aren’t readily available. Some “complete” literature search! Given the blatant “forgetting” of an example that most definitely does not support the “hypothesis” of the Generation Rescue paper, I also now wonder why only eight out of the 30 countries examined were chosen for inclusion in Table 3. You don’t think that Generation Rescue left the U.K. out of Table 3 because it didn’t fit with its hypothesis, do you? Unfortunately, I didn’t have time to go searching for all the prevalence data for the countries left out, but it wouldn’t surprise me at all if there were a few more U.K.’s in there.

Even curiouser (well, not really) are the sources used for all the prevalence data. Specifically, the dates of the studies are what stood out to me. Many of them are from the 1990s, and most are from before 2001. Given the increases in awareness, the broadening of the diagnostic criteria that occurred in the 1900s, using these prevalence estimates is meaningless. I could also point out that in the 1990s, the U.S. had a vaccination schedules with considerably fewer vaccines. Table 1 shows that! It would appear that Generation Rescue can’t even keep its story straight in its own study. In any case, comparing a vaccine schedule in 2006 (which is what Generation Rescue did in all cases) with prevalence rates from 5 to 10 years earlier taken from a period of time shortly after the diagnostic criteria had been changed is clueless at best and intellectually dishonest at worst, particularly when it’s done in the context of no effort to control for confounders, no statistical analysis, in essence nothing that any scientist or epidemiologist taking a serious look at the question would consider a minimum effort. In essence, because whoever at Generation Rescue passionately believes that (1) vaccines aren’t nearly as protective as we know them to be and (2) vaccines cause autism, this report ignores all other possible explanations for the numbers and, in fact, doesn’t even pick the right numbers to answer the question. I suppose that if these numbers had been properly picked for valid comparisons one might be able to view them as hypothesis-generating, but so many other studies have specifically looked at the question of whether vaccines cause autism using sufficient numbers to have the power to detect even a weak association and the correct epidemiologial and statistical techniques to analyze the data that throwing this sort of whole population data out as though it were “proof” of anything shows nothing more than how completely ignorant of science whoever wrote it is.

Of course, Generation Rescue isn’t really interested in accurate estimates of autism prevalence, as Sullivan has pointed out:

One of the problems with the vaccines-cause-autism groups is that they really don’t advocate for people with autism. They have abandoned entirely people of low income and minorities (except where they can be used for political gain).

It isn’t just that groups like SafeMinds, Generation Rescue and the rest can’t be bothered to spend the time worrying about minorities or adults. It’s the fact that the data those groups use to support the “epidemic” makes ZERO sense when you consider minorities.

Consider this: the “rate” of autism is 0.3 per 1,000 for Hispanics in Wisconsin, but 10.6 for Whites in New Jersey.

Why isn’t Generation Rescue calling for an investigating the Hispanics of Wisconsin? Shouldn’t they want to know what is “protecting” that subgroup from autism?

They don’t care, they don’t want to bring attention to the Hispanics in Wisconsin (or the under represented minorities across the nation), because it blows a big hole in the “epidemic”. Obviously we still aren’t counting all the people with autism in our prevalence estimates. How can we rely on the historical data that shows an “epidemic” if we aren’t doing a good job even now?

The Generation Rescue “study” is another instance of this very phenomenon, except that it even ignored a wealthy nation (the U.K.) because its autism prevalence data didn’t fit in with its conclusion that vaccines cause autism and that nations with lower numbers of mandated vaccines should therefore have lower autism prevalence. The study also completely ignored the far more likely reason why the United States has such an embarrassing level of childhood mortality: Poverty and disparities in care. That’s because, as Sullivan points out, groups like Generation Rescue are not interested in the poor or minorities, except when they are useful to their cause (Somali immigrants in Minnesota, for instance). All they are interested in is proving that vaccines cause autism. Whenever an antivaccinationist finds differences in autism prevalence, he ignores any other possible cause aside from the vaccines. That’s because, to J.B. Handley, Jenny McCarthy, Jim Carrey, and the rest of the antivaccine movement, first and foremost, it’s still all about the vaccines. It’s always been all about the vaccines. It always will be all about the vaccines. Always.

Unfortunately, thanks to Jenny McCarthy, who’s been hitting the pro wrestling circuit, doing celebrity poker events with Lance Armstrong, holding fundraisers with Hugh Hefner and Britney Spears and making numerous appearances to raise money for Generation Rescue, Generation Rescue appears to be flush with cash, and only J.B. knows how much money was spent on the Fourteen Studies website. I’m guessing it was quite a bit. The slickness of the website, coupled with all the TV appearances, and celebrity fundraisers, all of which coopt the message of “autism awareness” and conflate it with the antivaccine activist agenda, bespeak large amounts of money flowing into the antivaccine propaganda effort. Wouldn’t it be wonderful if Generation Rescue actually spent its money on real research, instead of propaganda like the Fourteen Studies website?

Don’t hold your breath waiting for that to happen.

That’s because, unfortunately, it’s not about science, evidence, or reason to Generation Rescue. It never has been; it is not now; and it never will be. No, it’s all about winning, which means that the evidence showing that vaccines are not associated with autism must be attacked, because the AAP, CDC, and Paul Offit are the “bad guys.” That’s why this Fourteen Studies website is nothing more than pure propaganda; indeed, it’s every bit as much propaganda as Jenny and Jim’s media tour to promote McCarthy’s latest paean to antivaccine pseudoscience and autism quackery or the “Green Our Vaccines” march last spring. I also note that there are far more than “14 studies” showing no association between vaccines and autism. Indeed, Mady Hornig cited twenty just looking at MMR and autism alone. That defenders of vaccines have chosen “fourteen studies” as a talking point was not a good idea; it was a strategy that made a website like Fourteen Studies possible. That, alas, was a failing of scientists as communicators.