Bad science about GMOs: It reminds me of the antivaccine movement (Thanksgiving edition)

Well, Thanksgiving’s over, and the orgy of consumerism known as Black Friday is in full swing. Personally, I have to work, at least part of the day, and I don’t go anywhere near the stores on Black Friday anyway. I haven’t for years. So we might as well briefly discuss a bit of science today. It won’t be long (by Orac standards), but there was a tidbit of news that hit the blogosphere on Thanksgiving Day that caught my interest. Apparently the execrable study by Gilles Séralini on the effect of using feed made from genetically modified organisms (GMOs) on rats is heading for retraction. You remember that study, don’t you? It’s one that I dissected in one of my usual 3,000+ word posts. If you want the details of why the study was such horrifically awful science, go there. In the meantime, let’s find out what’s happening, which was originally reported in the French press. Basically, the editor of the journal, A. Wallace Hayes, sent Séralini a letter on November 19 stating that the article would be retracted if Séralini didn’t agree to withdraw it.

Séralini, as you may recall, also tried to do an outrageous end run around the embargo system for pre-publication access of reporters to scientific papers to be published. Basically, he only showed the results to a carefully chosen group of reporters and insisted on a non-disclosure/confidentiality agreement that prevented them from discussing the findings with experts in the field or to get outside comments for use in their stories. From my perspective, this was about as unethical as it gets, a blatant attempt to control the initial press coverage and, as EmbargoWatch put it, “turn reporters into stenographers.” Fortunately, this effect was short-lived, and as soon as the study’s results claiming that long term feeding with GMO-feed led to large tumors growing on these rats were published, critical analysis began. Unfortunately, there was enough of a delay that all the usual suspects in the form of crank websites had free rein to use the study as “proof” of all their imagined evils of GMOs as pure poison.

Here’s the letter:

As you know, several months ago, we began a thorough examination of the data you provided to the journal. The panel looking at the data and its analysis sought to address the questions that had been raised in the Letters to the Editor received in response to your article.

The panel had many concerns about the quality of the data, and ultimately recommended that the article should be withdrawn. I have been trying to get in touch with you to discuss the specific reasons behind this recommendation. If you do not agree to withdraw the article, it will be retracted, and the following statement will be published it its place:

The journal Food and Chemical Toxicology retracts the article “Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize,”1 which was published in this journal in November 2012. This retraction comes after a thorough and time-consuming analysis of the published article and the data it reports, along with an investigation into the peer-review behind the article. The Editor in-Chief deferred making any public statements regarding this article until this investigation was complete, and the authors were notified of the findings.

Very shortly after the publication of this article, the journal received Letters to the Editor expressing concerns about the validity of the findings it described, the proper use of animals, and even allegations of fraud. Many of these letters called upon the editors of the journal to retract the paper. According to the journal’s standard practice, these letters, as well as the letters in support of the findings, were published along with a response from the authors.2 Due to the nature of the concerns raised about this paper, the Editor-in-Chief examined all aspects of the peer review process and requested permission from the corresponding author to review the raw data. The request to view raw data is not often made; however, it is in accordance with the journal’s policy that authors of submitted manuscripts must be willing to provide the original data if so requested.3 The corresponding author agreed and supplied all material that was requested by the Editor-in-Chief. The Editor-in-Chief wishes to acknowledge the co-operation of the corresponding author in this matter, and commends him for his commitment to the scientific process.

Unequivocally, the Editor-in-Chief found no evidence of fraud or intentional misrepresentation of the data. However, there is legitimate cause for concern regarding both the number of animals in each study group and the particular strain selected. The low number of animals had been identified as a cause for concern during the initial review process, but the peer review decision ultimately weighed that the work still had merit despite this limitation. A more in-depth look at the raw data revealed that no definitive conclusions can be reached with this small sample size regarding the role of either NK603 or glyphosate in regards to overall mortality or tumor incidence. Given the known high incidence of tumors in the Sprague-Dawley rat, normal variability cannot be excluded as the cause of the higher mortality and incidence observed in the treated groups.

Ultimately, the results presented (while not incorrect) are inconclusive, and therefore do not reach the threshold of publication for Food and Chemical Toxicology. The peer review process is not perfect, but it does work. The journal is committed to a fair, thorough, and timely peerreview process; sometimes expediency might be sacrificed in order to be as thorough as possible. The time-consuming nature is, at times, required in fairness to both the authors and readers. Likewise, the Letters to the Editor, both pro and con, serve as a post-publication peerreview. The back and forth between the readers and the author has a useful and valuable place in our scientific dialog.

The Editor-in-Chief again commends the corresponding author for his willingness and openness in participating in this dialog. The retraction is only on the inconclusiveness of this one paper. The journal’s editorial policy will continue to review all manuscripts no matter how controversial they may be. The editorial board will continue to use this case as a reminder to be as diligent as possible in the peer review process.

Please contact me as soon as possible to discuss the details of the analysis, and the procedures for withdrawing or retracting this article.

There are a few interesting and potentially controversial parts of this decision as described in the letter above. First, I tend to accept the editor’s assertion that there was no fraud here. What I found in my analysis looked far more like incompetence than fraud. It was basically an incompetence that led to the design of experiments virtually guaranteed to find “positive” results, using rats that are prone to developing tumors, inadequate control groups, large numbers of experimental groups that appeared not to have controls for multiple comparisons, and a very confusing and bizarrely counterintuitive way of presenting the evidence in the form of graphs whose results were nigh uninterpretable. In essence, the paper showed nothing, but the way the data were presented was with maximum spin to make it seem as though GMO feed led to horrific tumors in these rats.

I also agree that this paper should never have been published. Its scientific quality was so riddled with obvious problems and poor design that at the time I wondered how on earth such awful science was published, when it appeared to have been nothing more than a study designed to demonize GMOs. At that, it was quite successful. However, the question of what to do with it after it’s been published is a much less obvious question. Bad science is published all the time. So is science whose authors and their allies abuse it for political purposes, as Séralini’s allies (not to mention Séralini himself) did with gusto. Should all such science be retracted? This is not the same as the infamous case of Andrew Wakefield, where fraud was convincingly demonstrated through reports by investigative journalist Brian Deer. As Séralini himself cites, the Committee on Publication Ethics (COPE) states that the only grounds for retraction of a paper are:

  • Clear evidence that the findings are unreliable due to misconduct (eg data fabrication) or honest error.
  • Plagiarism or redundant publication.
  • Unethical research.

I think that a reasonable argument can be made for #1 in that Séralini’s research was clearly completely unreliable. Whether it is due to fraud or honest mistake doesn’t really matter; let’s assume it was due to honest error, given that the editor couldn’t find any evidence of fraud. It’s still a valid reason to retract Séralini’s paper. If Séralini had a whiff of scientific integrity, he would have withdrawn the paper himself, but he doesn’t, at least not on the issue of GMOs. Similarly, it could also be argued that #3 applies, that the research was unethical because of how the rats were treated, but this is a weaker argument because the use of an animal ethics committee like the ones used in the US gives Séralini cover. Even so, I can see an argument being made that this case isn’t clear enough. Moreover, the editor doesn’t help his case by stating that the data are “inconclusive.” So what? Inconclusive data are published all the time, as well they should be. Perhaps a better approach would have been to leave Séralini’s paper published, but with a disclaimer on the online version that reads much like the letter above, without the statement that the paper was being retracted. It could conclude with an admission and/or apology on the part of the editors that such a bad paper got through peer review.

Unfortunately (and predictably), in addition to his same tired responses to criticism of the paper, Séralini couldn’t resist invoking pharma shill gambit (or, in this case, Monsanto shill gambit) in his response to the retraction notice:

Hayes’ decision to retract the paper follows FCT’s [Food and Chemical Toxicity, the journal that published Séralini’s paper] appointment of Richard E. Goodman, a former Monsanto scientist and an affiliate of the GMO industry-funded group, the International Life Sciences Institute, to the specially created post of associate editor for biotechnology at the journal, early this year.[10]

Goodman’s appointment in turn followed an orchestrated campaign by GMO supporters to persuade FCT to retract the study. Some critics even accused Prof Séralini of fraud, without presenting any evidence. Many of the critics had undeclared conflicts of interest with the GMO industry.[11]

After Goodman was installed, FCT withdrew a separate study by Brazilian researchers that also raised questions about GM crop safety. The study showed that Bt insecticidal toxins similar to those engineered into GM Bt crops were not broken down in digestion, as is claimed by the industry and regulators, but had toxic effects on the blood of mice. The Brazilian paper, like Prof Séralini’s, had been peer-reviewed and published by FCT prior to Goodman’s arrival. After Goodman’s arrival, the paper was withdrawn without explanation from FCT[12] – only to be immediately published in another journal.[13]

There is no proof that Goodman was responsible for the retraction of Prof Séralini’s study. But his appointment, coming so soon after the “Séralini affair”, along with FCT’s failure to list the interests of its editors, raises questions about corporate influence on the editorial board at the journal.

I love that last paragraph, which basically is a “just sayin’, ya know” kind of non-disclaimer. Basically, Séralini is insinuating that somehow Dr. Goodman had something to do with his paper’s threatened retraction because he used to work for Monsanto and is now affiliated with International Life Sciences Institute. Actually, ILSI appears to have a pretty comprehensive policy on conflicts of interest and transparency. Séralini needs more than just his insinuations to demonstrate a connection between Goodman’s appointment and his paper’s retraction, but he clearly doesn’t have it. It’s also questionable that one newly appointed associate editor would have the clout right out of the box to influence the editorial board to decide on such a high visibility retraction.

In the end, I remain a bit conflicted about FCT’s decision to retract this paper. There’s no doubt that the Séralini paper is a truly awful piece of agenda-driven science, every bit as bad as any number of papers I could cite from the antivaccine movement, which is why I explicitly compared anti-GMO fear mongering to the antivaccine movement. More recent anti-GMO “research” has been, if anything, even worse than last year’s paper by Séralini. It should never have been published. However, there are a lot of papers that should never have been published out there. Should they all be retracted? I agree with the folks over at Retraction Watch that post-publication peer review is very important, but the devil, as always, is in the details.