It’s so cute when anti-vaxers try to discuss epidemiology…

The anti-vaccine movement is nothing if not plastic. It “evolves” very rapidly in response to selective pressures applied to it in the form of science refuting its key beliefs. For instance, when multiple studies looking at the MMR vaccine and autism failed to confirm the myth that the MMR causes autism or “autistic enterocolitis,” most recently late last year, it was not a problem to the anti-vaccine movement. Neither was it a major problem to the movement when multiple studies similarly failed to find a link between mercury in the preservative thimerosal that used to be in most childhood vaccines and is no more (except the flu vaccine) and autism. No problem! The anti-vaccine movement simply pivoted neatly, de-emphasized points that the evidence was so clearly against that even they couldn’t spin it to a positive anymore, and found new bogeymen. These days, it’s the “toxins” (such as formaldehyde and the latest antivax bogeyman, squalene), and “too many too soon” (a gambit given seeming respectability by Dr. Bob Sears, who’s lately dropped his pretense of not being anti-vaccine).

Evolution in action, indeed.

However, there is one trait of the anti-vaccine movement that, however its camouflaging plumage may evolve, never, ever changes. It is as immutable as creationists say that God is. That is that, whatever other claims, the anti-vaccine movement makes, at its core it is always about the vaccines. Always. No matter how much science fails to find a link between vaccines and autism or vaccines and whatever other horreur du jour the anti-vaccine movement tries to pin on vaccines, no matter how much it tries to deny and obfuscate by saying that it is not “anti-vaccine” but “pro-safe vaccine,” at its core the anti-vaccine movement is about fear and loathing of vaccines. Always. When inconvenient science doesn’t support their views, anti-vaccine activists either ignore it, distort it, or launch ad hominems against the people doing it or supporting it. And, as I said before, they evolve. Never again will the anti-vaccine movement yoke one of its major claims to a hypothesis that is as easily testable as the hypothesis that mercury in vaccines causes autism. The claim that mercury in vaccines causes autism predicted that, if thimerosal were removed from vaccines or reduced to pre-“epidemic levels” of the early 1990s, then autism rates should plummet. Thimerosal was removed from nearly all childhood vaccines (the sole exception being some flu vaccines), reducing infant mercury exposure from vaccines to levels not seen since the 1980s; yet autism rates continue to rise. This is about as resounding a refutation of the hypothesis that mercury in vaccines is a major cause or contributor to autism that even the anti-vaccine movement has backed away from the pure claim, which has now evolved to unnamed “environmental toxins,” either in concert with mercury or with other nasty things, as being the Real Cause of autism.

Evolution in action: These new claims are much “fitter” because they are much harder to falsify.

The latest favored demand of the anti-vaccine movement puzzles me, though. This demand is that the government conduct a large study of the “unvaccinated” versus the “vaccinated,” to compare them for health outcomes and, especially, the prevalence of autism. The reason I’m puzzled is because such a study has a very high risk of torpedoing virtually everything the anti-vaccine movement has been working toward in terms of promoting their message of fear about vaccines as being somehow credible (or at least not unreasonable) and based on science (more on that later). In the meantime, the drones over at the happy home for anti-vaccine propagandists, Age of Autism, are busily promoting the idea of a “vaxed versus unvaxed” study and disparaging anyone who dares to point out the difficulties, the latest target being Tom Insel of the Interagency Autism Coordinating Committee (IACC), who recently testified in front of Senator Tom Harkin’s comittee (yes, that Tom Harkin).

The misunderstanding and misrepresentation of epidemiology being trotted out to justify this study are astounding. For example, earlier this week, on AoA, a mechanical engineer named Catherine Tamaro tried to lecture scientists on statistics and the difference between observational versus experimental studies using quotes from a statistics textbook. She began:

After reading Katie Wright’s summary of the August 3 Senate Committee on Appropriations’ hearing on autism (HERE), I would like to comment on Dr. Insel’s testimony to Senator Harkin on why a study comparing vaccinated vs. unvaccinated children has not been done to date. Dr. Insel claimed that such a study would be “unethical” because, as he apparently envisions it, the study would entail dividing a large cohort of newborns into two groups, vaccinating one cohort but not the other through age two, and then comparing outcomes. This is just a bit disingenuous and I would like to explain why.

Of course, Ms. Tamaro is either ignorant or disingenuous herself in that some anti-vaccine advocates do indeed call for just such a study, even going so far as to demand a randomized, double-blinded study. J.B. Handley himself has attacked people who correctly call demands for such a study “unethical.” Apparently Ms. Tamaro isn’t aware of the strain running through the anti-vaccine movement that does want just such a study, or she herself is being even more disingenuous than she accuses critics fo being. Tamaro then continues:

Research studies are divided into two categories, observational studies and experimental studies. An observational study observes individuals and measures variables of interest but does not attempt to influence the responses. (The “epidemiological” studies to which Dr. Insel refers are actually observational studies.) An experimental study, on the other hand, deliberately imposes some treatment on individuals in order to observe their responses; the purpose of an experiment is to study whether the treatment causes a change in the response.

This paragraph just goes to show how a little knowledge is a dangerous thing. There are many flavors of “observational” studies, with varying degrees of power to detect differences and varying degrees of ease with which confounders are accounted for. These include cohort studies (both retrospective and prospective), cross sectional studies, longitudinal studies, and case series. All have their strengths and weaknesses, and the studies of vaccines and autism have used several kinds of methodology. Not that that sways Tamaro:

All studies done to date investigating a correlation between vaccinations and autism have been observational studies, but no observational study has been done comparing the prevalence of autism diagnoses in a vaccinated human population compared to an unvaccinated human population. When Dan Olmsted points out that he has identified large populations of unvaccinated children in the United States and asks why a study has not been done on them, he is actually asking why an observational study has not been done. When Senator Harkin asks Dr. Insel why a study has not been done on vaccinated vs. unvaccinated American children, he too is actually asking why an observational study has not been done to date. Dr. Insel, however, chooses to respond by saying that an experimental study would be required in order to resolve the issue.

Tamaro may have a bit of a point about Insel choosing the easiest to attack, but she is doing the same thing by ignoring the fact that there have been calls from the anti-vaccine movement for experimental studies, which, of course, would be highly unethical because they would leave large numbers of children completely unvaccinated and thus vulnerable to vaccine-preventable diseases. In any case, here’s where Tamara goes right off the deep end:

I would like to point out the epidemiological similarity between smoking/lung cancer and vaccines/autism. Smoking has been proven to cause lung cancer, yet not a single experimental study on humans was ever done – all of the human studies proving that smoking causes lung cancer were observational. The experimental studies were performed on research animals only. Attached at the end of this letter is a lesson taken verbatim from an introductory course in college statistics describing how the connection between smoking and lung cancer was made.

Point one: There have been lots of experimental studies on research animals of vaccines trying to show a link between vaccines and autism. I’ve written about some of them right here, and other bloggers have discussed them in detail as well. For example, there was the infamous Mady Hornig “rain mouse” study, in which she claimed that thimerosal at the doses infants received, adjusted for size, caused autistic symptoms. Both Prometheus and Autism Diva enumerated the numerous flaws and ethical lapses in that experiment. Then there was the more recent (and even more unethical) Laura Hewitson experiment looking at vaccinated and unvaccinated Macaque monkey infants. I was appalled at how badly designed and grossly unethical that experiment was, not to mention at the enormous undisclosed conflicts of interest of the investigators. The problem, of course, is that there is not yet a good animal model of autism. If there were, you can bet that there would be a huge amount of research using animal models. It’s not because the NIH is unshakably opposed to animal research on vaccines and autism. Rather it’s because such research is inherently not particularly useful or feasible absent a good animal model or two. Moreover, the history of such research (i.e., Hornig and Hewitson) is not exactly cause for optimism, given how badly done these studies were.

Point two: The tobacco comparison is a favorite canard of anti-vaccinationists. Certainly, it was primarily epidemiological studies that showed that smoking causes cancer. While she is correct to say that an experimental (i.e., randomized, blinded) study is not always necessary to provide sufficient evidence of causation, she’s picked the wrong example for a number of reasons. For smoking and cancer, the association is very strong, stronger than almost any environmental toxin and cancer that I can think of. Smoking results in a ten-fold increased risk of lung cancer, which is not hard to detect with an epidemiological study of even fairly weak design. But more importantly what Ms. Tamara either doesn’t know or fails to acknowledge is that the very same sorts of studies that identified smoking as a huge risk factor for lung cancer and cardiovascular disease are the same sorts of studies that have failed to find an association between either thimerosal and autism or vaccines and autism. For example, it was a case control study by Sir Richard Doll that was credited with first detecting the link between smoking and lung cancer (although actually the Nazis discovered the link with a case control study before Sir Doll did).

In any case, Ms. Tamara is also wrong when she says that a study of the vaccinated and unvaccinated has never been undertaken. In fact, Generation Rescue itself tried to do it. It was a very bad study, really no more than a phone survey, and its results were amusing in that they found that the highest rates of autism were in the so-called “partially vaccinated” children. The clear implication of the study was drolly described by Kevin Leitch:

There’s no getting away from this. This is a disaster for Generation Rescue and the whole ‘vaccines cause autism’ debacle. Generation Rescue’s data indicates that you are ‘safer’ from autism if you fully vaccinate than partially vaccinate. It also indicates that across the spectrum of autism, you are only 1% more likely to be autistic if you have had any sort of vaccination as oppose to no vaccinations at all – and thats only if you are male. If you are a girl you chances of being on the spectrum are less if you have been vaccinated! Across both boys and girls, your chances of being on the spectrum are less if you have received all vaccinations.

Ms. Tamara is also quite naive in that she clearly has no clue just how much money and how many children an observational study of the vaccinated versus unvaccinated would require to do properly, much less how tricky it would be to control for confounders, given that the unvaccinated vary in significant ways from the vaccinated. Again, Prometheus tells the tale. First, there is the issue of how few completely unvaccinated children there are, perhaps around 50,000 in the entire U.S. in the 3-6 year old age cohort. He then explains:

Let’s say – for the sake of argument – that we decide that a 10% difference in autism prevalence is enough to convince the skeptics that vaccines might cause autism and that a less than 10% difference will convince the believers that vaccines don’t cause autism. [I know, the latter assumption is pure fantasy.]

Well, plugging those numbers in – along with the current 1 in 150 autism prevalence – we find that we need over 360,000 children in each group to detect a 10% difference (you can try it yourself here). Unfortunately, that is more than the total number of unvaccinated children in the US, so that’s not going to happen.

What can we get with our “sample” of 49,652 unvaccinated children? If we manage to include each and every unvaccinated child in the US in the study, we could detect a 26% or more difference in autism prevalence.

Of course, it’s not even remotely practical to expect to get 100% of the unvaccinated children in the country into a study. How more about a practical number – say, 10% of them? That would allow us to detect a 70% or greater difference – about a three-fold difference in autism prevalence between the fully vaccinated and unvaccinated groups.

Does anyone labor under the illusion that parents who fervently believe that vaccines cause autism would ever accept negative results from a study that’s only powered to detect a three-fold difference in autism rates between vaccinated and unvaccinated children as sufficiently reassuring to accept the current vaccination as safe? Prometheus was right when he said that a study powered to find a 10% difference would probably not sufficiently reassure them. Given the religious fervor with which the anti-vaccine movement clings to the myth that vaccines cause autism, I doubt that it would accept a negative result from a study powered to detect a 1% difference in autism rates as sufficiently reassuring to abandon its fear. Moreover, as Prometheus tells us, even the study described above would be inordinately expensive and difficult to do. So he did a “back of the napkin” calculation of what a more feasible study could accomplish:

Finally, let’s “run the numbers” on a more practical study – one where we are able to enroll 500 unvaccinated children and 5000 fully vaccinated controls matched for age, sex, socioeconomic group, geographic location, urban vs rural vs suburban setting and race.

This study – which would still be very difficult and expensive to do – would only be able to detect a more than 15-fold difference in autism prevalence between the two groups. It could detect as little as a 7-fold difference, but only if we were willing to accept a beta error (chance of erroneously saying there is no difference when there is a difference) of over 50%.

I doubt this would “satisfy” the vaccines-cause-autism believers if the results were negative.

I can’t help but note that the study described by Prometheus would probably fail to find the well-known increased risk of lung cancer and heart disease due to smoking, the more so since the incidence of lung cancer in nonsmokers is considerably lower than 1 in 150, which is how many children are estimated to be autistic. Remember, the relative risk of lung cancer due to smoking is on the order of ten-fold.

The only way to get around the problems inherent in designing a study with sufficient numbers of unvaccinated children of sufficient power to detect a difference in autism prevalence between the unvaccinated and vaccinated children small enough to reassure most parents that vaccines do not cause autism would be to expand the study to multiple nations. Of course, doing such a study would be even more enormously expensive, take several years, and, because funding for autism research is pretty much a zero sum game, would divert huge amounts of money from more promising research to chasing down a highly implausible hypothesis that has virtually no credible empirical support behind it, either from basic science, epidemiology, or other evidence, certainly nowhere near enough evidence to justify such a huge expenditure and effort. Alternatively, as Prometheus points out, we could look at dose-response models:

Of course, we wouldn’t have to just look at unvaccinated vs fully vaccinated with this study, which is a large part of its superiority. We could look at a dose response of vaccination - to see if it really is “too many” – as well as the age at youngest vaccination – to see if it really is “too soon”. In fact, a few studies have already looked at those issues and found that there is no difference between the autistic and non-autistic groups. I suspect this is the reason the folks pushing to “put on a study” want to look at vaccinated vs unvaccinated – they hope that the numbers will be different (or, at least, not as definitive) the other way round.

In fact, I rather suspect that the smarter among the anti-vaccinationists know all the problems inherent in doing a study of vaccinated versus unvaccinated children. Certainly the government does, hence its reluctance to spend all sorts of money chasing a highly improbable hypothesis. (If only it would apply that reasoning to NCCAM!) In reality, the “vaccinated versus unvaccinated” gambit is just that–a gambit. The leaders of the anti-vaccine movement probably know that doing a study with sufficient power and numbers to exclude even a modest risk of autism due to the current vaccine schedule is so expensive and impractical that it would probably never be done and that smaller studies that are feasible will have too little power to reassure those who believe that vaccines cause autism that vaccines are in fact safe. Why do it then? It keeps the troops fired up thinking that there is some huge conspiracy to prevent such a study because of the fear of its results or that the government just doesn’t care enough about autistic children to do such a study.

Be that as it may, as cynical as the leaders of the anti-vaccine movement are, it’s so cute when AoA’ers try to do epidemiology and science. At least, it would be if their misinformation, pseudoscience, and utter nonsense were not such a profound threat to public health.