False balance on vaccines in Portland, courtesy of KATU and Genevieve Reaume

Yesterday, I dealt with a false narrative about vaccines. Today I deal with false balance about vaccines. Given the ongoing measles outbreaks in the US that have produced more cases of measles than we’ve seen in a generation, I haven’t been able to get away from the one form of pseudoscience contributing to the decline in vaccine uptake and herd immunity responsible for the outbreaks, as much as I’d like to mix my topics up. (C’est la vie, at least for the moment.) Meet Genevieve Reaume, a reporter for KATU, the Portland, OR ABC affiliate, and her story The Vaccination Debate: Oregon doctors debate the efficacy and safety of vaccines. It’s an example of false balance to a level that I haven’t seen in a long time. Whether Genevieve Reaume or her producer chose the frame used to present the “vaccine debate” story and is thus primarily responsible for this travesty, I don’t know, but it’s bad.

Before I discuss Genevieve Reaume’s story, let’s just go back to the origin of this blog. Back in the day, particularly around 10-13 years ago. A common topic of my blogging back then was false balance in reporting about autism and/or vaccines. It seemed back then that every story about autism had to include one or more quotes by parents who thought that vaccines had caused their child’s autism. Similarly, it seemed that every story about vaccines also had to include one or more quotes from antivaxers or parents who thought that vaccines had caused their child’s autism. Many of these stories were quite risible, with antivaccine “luminaries” such as Andrew Wakefield, Jenny McCarthy, J. B. Handley, and other familiar people showing up all over the media, particularly during April, which is Autism Awareness Month. Indeed, I used to joke ruefully about how I dreaded the last couple of days of March because I could count on the beginning of an onslaught of credulous stories about antivaccine misinformation and the claim that vaccines cause autism to begin. Then, Andrew Wakefield was disgraced. He had his medical license taken away by the UK. Brian Deer showed very convincingly that Wakefield had committed scientific fraud. Suddenly (or so it seemed), the media started to get it, and the frequency of false balance declined markedly. True, this could be confirmation bias on my part, as I’m unaware of any quantitative studies to support my impression, but I’m not the only science advocate who got that impression.

Now, it seems that we might be going back to those days. Of course, it might just be that, with all the ongoing measles outbreaks going on, courtesy of the antivaccine movement, there are a lot more stories about measles and vaccines out there being done by more journalists, many of whom have never done such a story before and are more prone to false balance. Either way, it seems as though I’m seeing more false balance. This brings me back to Genevieve Reaume’s story, broadcast on Monday:

You know this story is going to be the very epitome of false balance when while introducing the segment the anchor states, “Genevieve, this Spotlight on America piece focuses on both sides of the vaccine debate.” Reaume then responds by bragging about the hours of interviews she did and how this “debate” is all about the data. Then, behind her, is a large graphic with the words “The Vaccine Debate.” Here’s a hint: If Reaume had really learned from the data, she would know that, as far as the science and medicine go, there really isn’t a “debate” about whether vaccines cause autism and all the other health issues attributed to vaccines by antivaxers. She then launches straight into the introduction of Michelle Herman, who thinks that vaccines nearly killed her son:

Her son is fine now, but that doesn’t stop Reaume from letting Herman parrot an oft-parroted antivaccine trope about the Vaccine Court asking how, if vaccines are safe and effective, the Vaccine Court has paid out $4 billion in compensation for “vaccine injury.” This is a trope that pro-vaccine advocates have debunked more times than I care to remember. For instance, Liz Ditz debunked it. Our reptilian friend Skeptical Raptor debunked it. I’ve discussed it a number of times. I’ve discussed how the Vaccine Court, by design, makes it easier to win and obtain compensation than regular federal courts, with so called “table injuries” being automatically compensated, and how it has a tendency to allow complainants to use even scientifically dubious “theories” of how vaccines caused the injury for which they are claiming compensation, all while paying complainants’ legal fees, win or lose.

I like how Liz Ditz illustrated how, as large as $4 billion sounds, the amount of compensation paid through the Vaccine Court is really not that much:

The table on pages 8-9 lists awards by fiscal year. However, for the purposes of a bird’s eye view, it’s acceptable to aggregating these together.
  • Total number of vaccines distributed, 1/1/2006-12/31/2014 = 2,532,428,541
  • Total number of vaccine injury awards, FY 2006 – FY 2014 = 2,140
  • Total amount of vaccine injury awards, FY 2006-FY 2014 = $1,306,003,930.68
  • Vaccine injury awards per million doses of vaccines distributed, FY 2006-FY 2014 = 0.85
  • Total amount of vaccine injury awards divided by total number of vaccines distributed, FY 2006-FY 2014 = $0.52
  • Total amount of vaccine injury awards divided by the total number of vaccine injury awards, FY 2006 – FY 2014 = $610,282.21
At this scale, it looks like the NVICP, although not perfect, is working well enough. Vaccine injuries are very, very rare: less than one per million doses of vaccines distributed. The $0.75/vaccine surcharge is easily more than the average cost of awards per vaccine; the average payout is not huge, given how serious many of the conditions are, but is not stingy either.

Now, I will give Genevieve Reaume credit for noting that in 70% of the cases compensated by the Vaccine Court for “injury” by MMR, for 70% it could not be definitively demonstrated that the MMR vaccine caused the injury. It’s also good that she included a real expert, Dr. Anna Meyers. It’s not so great how she phrased it:

Sadly, for whatever reason, the producers and Genevieve Reaume decided to follow this up with an interview with Dr. Paul Thomas:

So basically, only three people appear in on-camera interviews in this piece, each for roughly the same total amount of screen time, if you count Reaume’s paraphrasing Mrs. Herman’s objections to the MMR vaccine and Vaccine Court. Two of them are antivaccine. Indeed, there’s a reason I like to refer to Dr. Paul Thomas as a rising star in the antivaccine movement.

Dr. Thomas, of course, claims to be running a study based on his own practice and “trying to get his data published,” something he’s been saying for quite a while now and something I’ve been calling BS on for as long as I’ve known about it, especially after I found out that his “research” partner is almost certainly antivaccine crank James Lyon-Weiler. His claim is:

We have real-world data showing that the unvaxed population is having a lot less autism, and we’ll get the data to the world. We’re gonna get this published.” Dr. Thomas says.

Here are the statistics Dr. Thomas gave KATU. He says of the 3,344 patients who were born into Integrative Pediatrics between June 2008 and February 2019, 715 were fully unvaccinated. He says of those, only one child had autism. Based on those numbers he says his rate of autism in unvaccinated children is 1/715. He adds, of the other 2,629 kids who were partially vaccinated, only six had autism. With that, he concludes his autism rate in partially vaccinated kids is 1/438. He says this is much better than the CDC’s rate of autism, which is 1/59.

Basically, he has basically no patients who are fully vaccinated according to the CDC schedule, hence his comparison to the CDC’s published estimate of autism prevalence. Basically, his numbers are meaningless without a whole lot of other information, such as the age distribution of his patient population and the inclusion of a number of confounding factors. There are many problems that have been the subject of discussion in the comments by my readers and me before. For instance, pediatrician Dr. Chris Hickle noted:

This current attempt by Thomas is a rather twisted extension of a completely non-peer-reviewed “study” Thomas put into Appendix E of his book “The Vaccine-Friendly Plan: Dr. Paul’s Safe and Effective Approach to Immunity and Health-from Pregnancy Through Your Child’s Teen Years”. He describes three groups of children:

Group 1: Following Dr. Paul’s Vaccine-Friendly Plan: These children were either born into my practice or establishedcare by the two-month well baby visit. They did not get the birth or infant hepatitis B vaccine. If they got the hepatitis A vaccine, it was after age two. If they got the MMR and chicken pox vaccines, it was at or after age three. There were 1,098 children in Group 1.

Group 2: Unvaccinated: Because of parental choice, these children had no vaccines. There were 238 children in Group 2.

Group 3: Other/ Most Vaccinated: Though there was some individual variability, many children in Group 3 were vaccinated according to the CDC guidelines. There were 894 children in Group 3.

Thomas then reports no cases of autism in Groups 1 and 2 and 15 cases in Group 3 and states:

We would have expected to have approximately seventeen cases of autism in Group 3, and there were fifteen autism/ ASD diagnoses, which mirrors the national average. This data demonstrates with a high level of statistical significance that the Vaccine-Friendly Plan— as well as not vaccinating at all— was associated with less autism/ ASD.

So many holes in the “study” left open it’s not even funny. He does say this “study” is based on 2230 children in his practice from June 2008 (when they first opened) through Feb 2015 and that he was in the process of “compiling the data for a retrospective study approved by the Western Institute Review Board.”

It’s even worse than this. As has been pointed out, Epi Ren, an epidemiologist who is a Doctor of Public Health (DrPH), having graduated from Johns Hopkins University Bloomberg School of Public Health, has looked at Dr. Thomas’ most recent numbers reported in the antivaccine crankosphere in March and found them…wanting:

Okay, so back to the 1 in 715 compared to 6 in 2,640. Are those two proportions (0.0014) and (0.0023) different enough to show that there is an association between vaccines and autism/ASD, or are they just different out of pure chance? What is the p-value of the comparison of these two proportions?…

To answer those questions, we do a statistical test called “Fisher’s Exact Test” because the proportions are so small (“the values of the expected cells are too small”). (PLEASE NOTE: If I had access to the entire dataset, I could do a logistic regression, a more “robust” statistical analysis that takes into consideration all of the other variables presented by the patients, like age, gender, socioeconomics, number of vaccines given, etc.)

First, the p-value is 1, meaning that it is certain that vaccination status is not associated with autism/ASD status. At the bottom is the Odds Ratio of 1.63. This tells us that, based on this sample alone, those who are vaccinated have 63% higher odds of being diagnosed with autism/ASD. But then look at the 95% confidence interval above that: 0.197 to 74.895.

The 95% confidence interval says: “We are 95% confident that the true odds ratio in the population from where this sample was drawn is between 0.197 and 74.895.” In other words, there is a good chance that those who are vaccinated have between 80% less odds of autism/ASD and 7489% more odds of autism/ASD.” As you can see, it’s impossible to know based on these numbers what the true association is. There is even a good chance that the odds are the same between the two groups, not less nor more.

So we conclude that there is no association.

Same as it ever was with dubious studies by clueless antivaxers. I also predict that, even if Epi Ren had the whole dataset and could do more sophisticated analyses, the answer would be the same: No association.

Stepping back to broader questions of overall research design, I haven’t been able to find where Thomas checks for potential confounding factors or describes how his autism cases were diagnosed and whether they were screened for autism. For instance, if there’s a large component of children who are home-schooled or who go to unconventional schools like Waldorf schools in his practice, it’s quite possible that less severe cases of autism in these children could be missed because they probably wouldn’t undergo the same routine screening that children in public school do.

Similarly, if you don’t look you won’t find. If Dr. Thomas himself in his unconventional woo-infused practice doesn’t do the standard screening for autism that conventional pediatricians do, of course he’s going to miss cases. Indeed, I can imagine him doing “selective evaluation” in which he looks more closely for autism in his vaccinated patients than he does in his unvaccinated patients. Most likely, if he’s doing this he isn’t it intentionally, but he is human and his biases could easily affect how closely he looks for autism in his unvaccinated patients, whether he realizes it or not. (This sort of selective observation and analysis is why blinding is so important in randomized, double-blind clinical trials.) It’s human nature to see what one wants to see and disregard the rest. In other words, his study is an unreliable retrospective study that could easily have unaccounted-for bias. Did Genevieve Reaume consider these issues? It doesn’t really look like it.

As for denying that he is antivaccine (as nearly all antivaxers do), Dr. Thomas did say this in response to Genevieve Reaume’s questioning:

When KATU asked Dr. Paul if he thought doctors who follow the CDC vaccination schedule were putting patients at risk he answered by saying the U.S. wasn’t seeing this chronic disease a few decades ago: “It came out of nowhere. Fifty-four percent of kids are graduating high school on chronic medication. Something is different, and I’m not saying it’s just the vaccines. It’s toxins, toxins, toxins.”

That’s what we in the biz call the “toxins” gambit. It’s nonsense, and Thomas’ “it’s not just the vaccines” is unconvincing. He’s an antivaxer, his denials notwithstanding. It’s primarily the vaccines to him. At the doses in vaccines, none of the rest of the ingredients are “toxic. Indeed, Thomas is well known in the Portland area for dissuading parents from vaccinating:

Dr. Jay Rosenbloom, a pediatrician with Pediatric Associates of the Northwest, says he regularly sees parents who previously went to Thomas. “They want to protect their kids from these diseases, and he tries to pressure them out of it,” Rosenbloom says. “The problem comes with people who close their minds to true cause and effect and create their own reality.”

It also really didn’t help when Reaume asked Dr. Myers if she views Dr. Thomas as a threat, to which Dr. Myers answered yes. Basically, by providing him that sound bite, Reaume couldn’t have provided better advertising for Dr. Thomas if he’d paid for it. Antivaccine doctors like Dr. Thomas love to view themselves as a threat to the status quo, and Reaume gave him validation as such. You have to be careful answering questions like this, and it was clear that Reaume’s question caught Dr. Myers by surprise, as you can see if you watch the segment.

This story by Genevieve Reaume was one of the worst examples of false balance that I’ve seen in a while. (One can’t help but wonder if the fact that KATU is owned by Sinclair Broadcast Group, a station known for “must-run” segments promoting right wing propaganda and for Fox News-like reporting, has anything to do with it.) Unfortunately, after being criticized extensively on Twitter, she still seems not to understand:

I know it’s difficult for some journalists to grasp, but there are issues where there are not two sides to the story. This is one. Worse, the traditional structure of antivax claim followed by scientist or doctor refuting claim can actually reinforce the claim. When it comes to conspiracy theory-based movements like the antivax movement, facts don’t matter, and presenting antivaccine pseudoscience side-by-side with science does falsely elevate the pseudoscience, no matter how much refutation to antivax pseudoscience is included. This is information warfare, and it’s asymmetric warfare. Antivaxers don’t have the credibility of mainstream medicine; so they exploit the tendency of journalists to “tell both sides” even when there are not, scientifically speaking, two sides to tell. In this, Genevieve Reaume and her producer facilitated their efforts, serving unintentionally as a useful idiots.