A horrendously bad “vaxed/unvaxed” study rises from the dead yet again

Some posts I really enjoy doing. I’m so fired up by the topic that the words flow, and I finish a post in record time. Other posts are more of a chore, written not so much because I’m excited by the topic, but because I feel duty bound to address it. I feel the need to write such posts when, for example, a bit of pseudoscience has gained traction in mainstream groups and readers keep writing me about it, to the point where I finally give in. This is one of the latter posts. None of this is to say that I don’t still do my best with these posts to explain and argue my points. Fear not, I’ll get some good Orac snark in. It’s just that duty tends to be less fun than passion.

One of the most frequent topics for posts like this is a pseudoscientific or just plain bad study that, despite being retracted, keeps rising from the grave, like the proverbial zombie. I call them, appropriately enough, zombie studies. Depending on my mood when I write posts like this, I often add imagery featuring zombies (or, if you’re into The Walking Dead, walkers). Other times, I’ll include imagery featuring Jason Vorhees or Michael Myers, two supernatural slashers who would routinely through misbehaving teens for a whole movie, die (or appear to die) at the end of the movie, only to come back in the next installment in the series to kill again. Antivaccine pseudoscience (for example) is a lot like these monsters. In actuality, they’re probably more like Jason or Michael Myers than walkers because you can actually kill walkers dead for good. Be that as it may, whenever a truly awful study that should never have been accepted in the first place for publication in a peer-reviewed journal is retracted, you can be sure that it won’t be too long before it is magically resurrected and rears its ugly head again in some form or another, to be wielded not just as a weapon to frighten parents with but as a bogus example of how the peer-reviewed medical literature “suppresses” science that doesn’t support vaccines, to be used to feed the conspiracy theories behind the antivaccine movement. Same as it ever was.

This time around, the zombie study is one that I’ve been checking in with and covering periodically ever since its inception in 2012, when antivaxers were fundraising for it. The principal investigator was Anthony R. Mawson, M.A., DrPH. Indeed, J.B. Handley himself spearheaded the fundraising effort. It is, unsurprisingly, the Holy Grail of antivaccine studies, the mythical “vaccinated/unvaccinated” study. Antivaxers, at least the ones who retain a bit of reason with respect to medical ethics, have come to realize that a randomized, double-blind, placebo-controlled trial of vaccinated versus unvaccinated children is considered utterly unethical because it would leave half the children unprotected against vaccine-preventable diseases. They might not accept how unethical such a study would be, but they do realize that scientists do consider such a study unethical.

So they fall back on comparing health outcomes in children who are vaccinated to those who are unvaccinated (or undervaccinated). They’re pretty much all crap, because those carrying the studies out are biased and/or incompetent. Examples include a telephone survey disguised as a “study” done ten years ago and a survey disguised as a “study” performed by a German homeopath. This study is different in that it isn’t an antivaccine activist parent with no background in science or a homeopath but an actual academic. He is, however, clearly biased towards antivaccine views, as he has defended Andrew Wakefield’s 1998 Lancet case series and is a vocal supporter of his.

I’ve written about this study before. Hilariously, when it was published in its first form, the full study wasn’t published, only the abstract. Then the abstract was, in essence, retracted. Even more hilarious, it was a Frontiers journal, which is an even bigger dis because Frontiers journals are known for tending to be pay-to-publish predatory open access journals. If a Frontiers journal retracts your paper, it’s plenty bad indeed. It turns out that the manuscript had been reviewed by a chiropractor and a peer reviewer without expertise,

Then, back in February, the Mawson zombie study rose from the dead again, as antivaxers spread around copies of the retracted article and crowed that it had been accepted for publication elsewhere, and indeed it has. It’s fallen even farther down the food chain than a Frontiers journal, having been published by Mawson et al in the Journal of Translational Science, a journal published by Open Access Text, as Pilot comparative study on the health of vaccinated and unvaccinated 6- to 12- year old U.S. children.

There’s nothing new in this study that makes it any better than it was in its previous incarnations. Indeed, its introduction alone contains a boatload of fail that gives away the antivaccine leanings of Mawson et al. For example, there’s the implication of “too many too soon”:

Under the currently recommended pediatric vaccination schedule [7], U.S. children receive up to 48 doses of vaccines for 14 diseases from birth to age six years, a figure that has steadily increased since the 1950s, most notably since the Vaccines for Children program was created in 1994. The Vaccines for Children program began with vaccines targeting nine diseases: diphtheria, tetanus, pertussis, polio, Haemophilus influenzae type b disease, hepatitis B, measles, mumps, and rubella. Between 1995 and 2013, new vaccines against five other diseases were added for children age 6 and under: varicella, hepatitis A, pneumococcal disease, influenza, and rotavirus vaccine.

The implication is, of course, the common antivaccine trope that as a result of the gradual expansion of the recommended vaccine schedule children are getting, yes, “too many too soon,” with adverse health effects. Then there’s this:

A complicating factor in evaluating the vaccination program is that vaccines against infectious diseases have complex nonspecific effects on morbidity and mortality that extend beyond prevention of the targeted disease. The existence of such effects poses a challenge to the assumption that individual vaccines affect the immune system independently of each other and have no physiological effect other than protection against the targeted pathogen [21]. The nonspecific effects of some vaccines appear to be beneficial, while in others they appear to increase morbidity and mortality [22,23]. For instance, both the measles and Bacillus Calmette–Guérin vaccine reportedly reduce overall morbidity and mortality [24], whereas the diphtheria-tetanus-pertussis [25] and hepatitis B vaccines [26] have the opposite effect. The mechanisms responsible for these nonspecific effects are unknown but may involve inter alia: interactions between vaccines and their ingredients, e.g., whether the vaccines are live or inactivated; the most recently administered vaccine; micronutrient supplements such as vitamin A; the sequence in which vaccines are given; and their possible combined and cumulative effects [21].

The wag in me can’t help but provide Mawson with an example of a “complex nonspecific effect on morbidity and mortality” due to a vaccine. He’s not going to like it, though, because it shows that the benefits of the measles vaccine go beyond just preventing measles. Basically, there is a prolonged period of immunosuppression after the measles that lasts up to three years. Vaccinating against the measles prevents that immunosuppression and therefore lowers the death rate due to other infectious diseases to which children are more vulnerable after having had the measles.

The bias is also apparent in the statement of purpose for the study:

The aims of this study were 1) to compare vaccinated and unvaccinated children on a broad range of health outcomes, including acute and chronic conditions, medication and health service utilization, and 2) to determine whether an association found between vaccination and NDDs, if any, remained significant after adjustment for other measured factors.

This is serious bias, as the authors assume that vaccines cause harm. It’s not quite explicitly stated, but certainly implied. They clearly expected to find an association between vaccination and neurodevelopmental conditions, despite all the copious evidence that there is no such association.

I also can’t help but turn a frequent antivaccine trope back on itself. Antivaxers and promoters of alternative medicine often criticize studies of drugs and vaccines because the drug and vaccine manufacturers are frequently the funding source. That is not an entirely unreasonable objection—to a point. I myself look more skeptically at studies funded by drug companies, but with this caveat. If the study is well-designed, executed, and analyzed, I take its results seriously, regardless of funding. However, since antivaxers seem to think that even a whiff of pharma funding of a study invalidates it, I can’t help pointing out the funding of Mawson’s study:

This study was supported by grants from Generation Rescue, Inc., and the Children’s Medical Safety Research Institute, charitable organizations that support research on children’s health and safety. The funders had no role or influence on the design and conduct of the research or the preparation of reports.

Generation Rescue is Jenny McCarthy’s antivaccine organization, although it was originally founded by J.B. Handley, and the CMSRI is one of the looniest of the loony antivaccine groups. Sure, it’s probably true that Generation Rescue and the CMSRI didn’t directly influence design or execution of the study, but ask yourself this: Would these groups have funded an investigator if they weren’t pretty sure how his study would turn out? I think you know the answer to that question.

Of course a study this flawed is close to guaranteed to find a positive result. The flaws begin with the selection of study population:

The study was designed as a cross-sectional survey of homeschooling mothers on their vaccinated and unvaccinated biological children ages 6 to 12. As contact information on homeschool families was unavailable, there was no defined population or sampling frame from which a randomized study could be carried out, and from which response rates could be determined. However, the object of our pilot study was not to obtain a representative sample of homeschool children but a convenience sample of unvaccinated children of sufficient size to test for significant differences in outcomes between the groups.

We proceeded by selecting 4 states (Florida, Louisiana, Mississippi, and Oregon) for the survey (Stage 1). NHERI compiled a list of statewide and local homeschool organizations, totaling 84 in Florida, 18 in Louisiana, 12 in Mississippi and 17 in Oregon. Initial contacts were made in June 2012. NHERI contacted the leaders of each statewide organization by email to request their support. A second email was then sent, explaining the study purpose and background, which the leaders were asked to forward to their members (Stage 2). A link was provided to an online questionnaire in which no personally identifying information was requested. With funding limited to 12 months, we sought to obtain as many responses as possible, contacting families only indirectly through homeschool organizations. Biological mothers of children ages 6-12 years were asked to serve as respondents in order to standardize data collection and to include data on pregnancy-related factors and birth history that might relate to the children’s current health. The age-range of 6 to 12 years was selected because most recommended vaccinations would have been received by then.

Notice how Mawson claims that this is a cross-sectional study, when in reality it’s a survey targeting parents who homeschool. Of course, parents who choose to home school are not like your average parents. There are a lot of confounding factors that go along with home schooling, including the association between home schooling and antivaccine views. This association is very clear in the data, which show that 261 of the 666 subjects were unvaccinated. Of these 405 who were vaccinated, only 197 were “fully vaccinated.” Thus, less than 1/3 of the children in the study were fully vaccinated according to the CDC’s recommended schedule, and well over 1/3 were completely unvaccinated. This is not in any way representative of the population at large. Add to that the likelihood of selective memory and reporting, and the likelihood of this survey providing useful information is vanishingly small. Also, surveys are not the best means of gathering health data, and in this case it was a particularly bad situation. Mothers were asked whether their children were vaccinated, unvaccinated, or “partially vaccinated,” and what conditions or diseases their children had had. There was no effort to make any independent assessments of the children’s health, nor was there any attempt to account for bias, and there almost certainly was a lot of bias here:

A number of homeschool mothers volunteered to assist NHERI promote the study to their wide circles of homeschool contacts. A number of nationwide organizations also agreed to promote the study in the designated states. The online survey remained open for three months in the summer of 2012. Financial incentives to complete the survey were neither available nor offered.

Even more telling, consider how the subjects were recruited. The authors admit that the “object of our pilot study was not to obtain a representative sample of homeschool children but a convenience sample of unvaccinated children of sufficient size to test for significant differences in outcomes between the groups.” In other words, no effort was made to construct a representative sample.

So what are we to make of the results of this study, which show:

The vaccinated were less likely than the unvaccinated to have been diagnosed with chickenpox and pertussis, but more likely to have been diagnosed with pneumonia, otitis media, allergies and NDD. After adjustment, vaccination, male gender, and preterm birth remained significantly associated with NDD. However, in a final adjusted model with interaction, vaccination but not preterm birth remained associated with NDD, while the interaction of preterm birth and vaccination was associated with a 6.6-fold increased odds of NDD (95% CI: 2.8, 15.5). In conclusion, vaccinated homeschool children were found to have a higher rate of allergies and NDD than unvaccinated homeschool children. While vaccination remained significantly associated with NDD after controlling for other factors, preterm birth coupled with vaccination was associated with an apparent synergistic increase in the odds of NDD.

Nothing. The bias and flaws in this study guaranteed no other result, particularly when you consider another confounding factor, namely that the parents of children who are fully vaccinated are very different in their health-seeking behavior than those whose children are unvaccinated. They tend to take their children to visit the doctor more regularly, which means that health disorders their children have are more likely to be diagnosed and treated. They’re also less likely to be seeing naturopaths and other alternative practitioners.

I’ll conclude by pointing out yet again that it is a myth that there are no studies comparing the health of vaccinated children compared to unvaccinated children. In fact, there have been several. It turns out that they don’t show what antivaxers think a vaxed/unvaxed study will show. Basically, all of the vaxed/unvaxed studies not done by antivaccine-friendly scientists or quacks have shown either no differences in the prevalence of neurodevelopmental or chronic diseases between vaccinated children and unvaccinated children or have actually found better health outcomes in the vaccinated population. Mawson concludes by arguing that further “research involving larger, independent samples and stronger research designs is needed to verify and understand these unexpected findings in order to optimize the impact of vaccines on children’s health.” Mawson’s study is so biased, flawed, and incompetently carried out and analyzed that its results can be discounted as almost certainly worthless. It doesn’t provide the rationale for “more studies.” Quite the contrary.

Yet, that’s how antivaxers are spinning it, as they always do.

Same as it ever was.