A survey administered by a German anti-vaccine homeopath backfires spectacularly

Sometimes I feel a little bit guilty when I’m writing a post deconstructing anti-vaccine nonsense, “alternative medicine” quackery, or some other form of pseudoscience. This guilt usually derives when I end up picking a target that’s just too easy, a study that’s just so mind-numbingly, brain-meltingly awful that it’s not much of a challenge, even though at the time I perceive that it needs to be done. I suppose it’s like the feeling that a professional sports team might feel if it were ever paired with a high school team–or even a junior high–team for a game. In fact, I was half-tempted just to post the link to the “study” (which it really isn’t, not really) and let you, my readers, have some fun. I’d consider it an exercise in seeing just how much regular readers have learned, or even how much newer readers have picked up. At the very least, it’d be a nice new chew toy for you all.

But what fun would that be for me, other than sitting back like a proud papa and chuckling as I watch you guys rip into the study in a manner that makes a starving cheetah ripping into its prey look downright restrained by comparison? If I get the first chomps in, I can still sit back and watch you all have at it, as long as I leave just a bit left over for you. But, before I do, a wee bit of history.

Ever since the anti-vaccine movement started backing away the now thoroughly discredited claim that the thimerosal in preservatives cause autism and the even more thoroughly discredited claim that the MMR vaccine causes autism and pivoted to readjust its reality to embrace the idea that it’s all about “too many too soon” and “toxins” in vaccines (even though that’s scientifically unsupported too), they’ve been clamoring for what they like to call a “vaxed-unvaxed study.” Basically, this is a study of vaccinated children versus unvaccinated children. At first, having no concept of medical ethics, anti-vaccine activists demanded a prospective, randomized, double-blind, placebo-controlled trial, apparently not realizing how utterly unethical such a study would be, given that it would leave the placebo control group unprotected against potentially dangerous childhood illnesses. Eventually, it started to sink in that such a study is neither feasible nor ethical (although sadly this wasn’t the case for all anti-vaccine loons). When that finally happened, it was so cute to see anti-vaccine activists try to propose epidemiological studies. Basically, it’s painfully obvious that anti-vaccine activists don’t understand the issues involved, particularly the size of the study that would be required, the difficulty in controlling for confounding factors in the sorts of designs that would be required (such as case-control, for example), and how expensive such a study would be. Also, to meet ethical standards, such a study would have to have a decent amount of preliminary data to support its hypothesis that vaccines cause autism (or whatever), and there is none, at least none not coming from anti-vaccine loons or investigators somehow associated with anti-vaccine loons.

Not that that’s totally stopped anti-vaxers from trying to do such a study.

For example, four years ago, J.B. Handley’s (now Jenny McCarthy’s) anti-vaccine propaganda group Generation Rescue did what was billed as a “study” of vaccinated versus unvaccinated children. It was nothing of the sort. Rather, it was a poorly designed phone survey whose results in some groups suggested that vaccines protected against autism, although Generation Rescue spun it as supporting the vaccine-autism hypothesis. Of course, the whole survey was so ridiculously badly designed that you really couldn’t tell anything from it at all, given its selection bias and failure to control for confounders, but that doesn’t stop it from periodically rising from the grave and shambling off to feast on the brains of antivaxers, who then cite it as though it’s evidence of anything other than the incompetence of Generation Rescue at any sort of research.

Now they’re at it again, although it’s not Generation Rescue who did this new “study.” Even so, not surprisingly, the anti-vaccine propaganda blog Age of Autism is nonetheless promoting it under the title Vax UnVax Study Results, as is the one anti-vaccine website that can challenge NaturalNews.com for the sheer intensity of its burning stupid, Child Health Safety, which is promoting the study/survey breathlessly as New Survey Shows Unvaccinated Children Vastly Healthier – Far Lower Rates of Chronic Conditions and Autism.

It does nothing of the sort.

In fact, looking at the actual survey used, although it pains me to say so, Generation Rescue comes out looking more competent than VaccineInjury.info, the English-language version of Impfschaden.info, a German anti-vaccine website run by a homeopath named Andreas Bachmair, who conducted the survey. You’ll see what I mean in a minute. The survey begins with this introduction:

For statistical evaluation of the state of health of entirely unvaccinated children we request you to fill out the following form. The data will be published anonymously and handled with utmost confidentiality. The results help us to acquire accurate information about the health of unvaccinated children.

Does anyone see a problem here? Well, actually, does anyone see several problems, but one glaring problem besides the problem of this being an anonymous Internet survey that anyone can fill out? Let’s just put it this way. Even Generation Rescue tried to have an actual control group, namely vaccinated children. Indeed, although Generation Rescue did a crappy and arbitrary job of it, its survey company at least tried to stratify respondents into different dose levels of vaccines, to produce three groups: unvaccinated, partially vaccinated, and fully vaccinated. What does Bachmair do? Nothing of the sort! He only collects data on “entirely unvaccinated children.” He even puts it in bold! Let’s just say that the construction of this survey demonstrates in this survey all the scientific understanding and rigor that I would expect from a homeopath, given that homeopaths believe that magic water cures people.

Before I come back to the horrendously bad methodology, why don’t we just summarize some of the results of this survey and then look at some of the reactions? There were a total of 7,762 children whose information was provided to the survey, and the general results are summarized as the state of health of unvaccinated children. Before we get to the “money results,” let’s take a peak at some anomalies that suggest that this particular group might not be–shall we say–strictly comparable to children in the population at large. For example:

The parents stated that their preferred treatment was naturopathic and homeopathic. Less than 10% said they preferred conventional medicine. Treatment in the “other” column was mainly chiropractic and supplemental.

So, right away, this survey demonstrates that the parents who filled it out were a self-selected, biased sample, the vast majority of whom favor alternative medicine and are hostile to scientific medicine. Indeed, 99.69% of the respondents report being happy that they did not vaccinate their children. One thing that this love of woo and hostility towards scientific medicine can mean is that a lot of these children could have subclinical or mildly clinical disease that goes undiagnosed because they never take their children to a real doctor, preferring instead homeopaths, naturopaths, and chiropractors.

To get a flavor of the health results, let’s look at the part of the report that asks about asthma and atopic diseases:

Asthma, hayfever and neurodermatitis are seen very frequently today. A recent German study with 17461 children between 0-17 years of age (KIGGS) showed that 4.7% of these children suffer from asthma, 10.7% of these children from hayfever and 13.2% from neurodermatitis. These numbers differ in western countries, i.e. the prevalence of asthma among children in the US is 6% whereas it is 14-16% in Australia (Australia’s Health 2004, AIHW)
The prevalence of asthma among unvaccinated children in our study is around 2.5%, hayfever 2.5% and neurodermatitis 7%.
According to the KIGGS study more than 40% of children between the ages of 3 and 17 years were sensitized against at least one allergen tested (20 common allergens were tested) and 22.9% had an allergic disease. Although we did not perform a bloodtest, around 10% stated that their children had an allergy.

One wonders how, if these parents chose homeopaths, naturopaths, and chiropractors over real doctors, they had any idea whatsoever whether or not their children actually had asthma. One of the most common presentations of asthma is cough alone. In fact, milder cases of asthma can be difficult to diagnose in children; so once again, what the parents report probably doesn’t tell us much. Neither does the claim that far fewer of these children had allergies.

Which brings us to autism.

If you scroll down to the graph looking at autism and various other problems, such as ADHD, you’ll find that the overall prevalence reported in these children was 0.57%. In terms of raw numbers, that’s 44 children, which makes this statement rather puzzling:

There are also autism cases in unvaccinated children. However over 80% stated, that it is only a mild form or a high functioning form of autism. Among all participants there were 4 severe autism cases.

Apparently, basic math isn’t a homeopath’s strong suit, which probably explains why they can’t understand the concept of Avagadro’s number. Be that as it may, if 20% of autistic children equals four, then there could only be 20 autistic children, but the survey suggests that there were twice that many in unvaccinated children. (One wonders what Tony Bateson would say.) In any case, a prevalence of 0.57%, even if this survey were accurate, would be within the range of estimated prevalences found in various studies. Perhaps Bachmair knows that, which is why he tried to emphasize “severe” autism and then came up with those additional factoids about some of these four to suggest that they had been exposed to mercury or heavy metals. Even worse for Bachmair, if you look at the graph of autism by age range in these children, depending on the age range it ranges from 0.37% to a whopping 2.36%, the latter of which is almost as high as a recent study in Korea found. In fact, if you look at the age range of the responses, nearly half of the responses (3,075) were for children under two years old, which is young enough that autism might very well have not been diagnosed yet, and in this group the reported prevalence was 0.37%, while in the 11-12 year range the prevalence was highest, at 2.36%. In fact, autism prevalence is so obviously not appreciably different in the unvaccinated in this survey compared to reported prevalence numbers that even a commenter at Age of Autism wrote:

If you look at all the age groups >2 years old the autism incidence ranges from 0.63 to 2.36%, with most being in the range 1-2%, the population size was about 3500 replies. My main criticism is a self selected population with potentially varying diagnostic criteria, but on these data the incidence of autism in unvaccinated children seems to match vaccinated children.

There’s no “seems” about it. The prevalence of autism in unvaccinated children in this survey does closely match reported numbers for overall population prevalence in populations where the vast majority of children are vaccinated. This result is an unmitigated disaster for Bachmair and his groupies, which is why I couldn’t stop laughing when I read this from ChildHealthSafety:

It is interesting neither the US National Institutes of Health [US$30.5 billion annual budget on medical research] nor the US Centers for Disease Control [US$11 billion budget annually] could find the time or money to fund this kind of research but instead waste US tax dollars on a great deal of pointless medical research and promotion of iatrogenic [man made] disease causing agents [modern drug company “treatments”].

No, it’s not really that interesting. Say what you will about the NIH, it does have a pretty rigorous peer review process, which means that it doesn’t (usually) fund crap. In fact, this survey was so poorly designed and analyzed that I doubt even the National Center for Complementary and Alternative Medicine (NCCAM) would fund it. Unfortunately, now this “study” will no doubt join the Generation Rescue “study” in the annals of crap vaccine/autism science, to circulate around Whale.to (where it belongs) and be dredged up as “evidence” periodically. Old, refuted anti-vaccine studies never die, alas.

In any case, I take some comfort in the hilarious result of this survey that demonstrates that autism prevalence in the unvaccinated is similar to autism prevalence among the vaccinated, no matter how much anti-vaccine activists try to spin it otherwise. I realize that this survey is in fact so poorly designed that it really doesn’t tell us much of anything, but it is fun watching anti-vaxer brains explode trying to spin this result as supporting the vaccine/autism hypothesis.

The enjoyment I get watching that assuages my guilt for picking on homeopaths so.

NOTE: I notice that the total number of children is increasing. It’s now up to 7,799 at this moment, suggesting that 30 people have filled it out since last night. Given that Child Health Safety lists it as 7,724 five days ago that suggests that the surveys still open and is automatically updating totals.

Hmmmmm.