Politics versus scientific peer review

In the United States, the federal government has long had a prominent role in funding science research. Be it the $30 billion a year or so that funds the National Institutes of Health or the $5 or $6 billion a year allotted to the National Science Foundation, the government funds a lot of basic and applied research, and this is in general a good thing. The number of discoveries and advances that have come out of this funding have well repaid the investment and much more. There is, however, a downside to this funding that’s a pretty obvious consequence any time the government funds anything, and that’s the possibility for lawmakers to meddle with science and try to dictate what science is funded. In other words, politically controlled science is a risk any time the government funds science. Fortunately, we have been very fortunate in the U.S. in that the lawmakers who founded the NIH, the NSF, and other agencies funding science to a large degree made sure that rigorous peer review controlled by scientists with the relevant expertise score and rank grant applications. Sure, there are programmatic considerations as to what sorts of projects are priorities for each agency, but by and large those considerations are devoid of direct interference by politicians.

I’m not saying that there aren’t exceptions. Perhaps the biggest and most blatant exception is one that is a frequent topic of this blog: The National Center for Complementary and Alternative Medicine (NCCAM). There was no compelling scientific rationale to carve out a center in the NIH devoted to the study of, let’s face it, mostly pseudoscience, nor were scientists and physicians clamoring for such a center. However, one very powerful Senator, Tom Harkin (D-IA), started the ball rolling by funneling some money to form the Office of Unconventional Medicine, which was later renamed the Office of Alternative Medicine. When then-NIH director Harold Varmus tried to rein in the OAM in 1998, Harkin responded by introducing legislation to make the OAM a full center. Thus was NCCAM born, and its current budget rapidly climbed to where it is now, fluctuating between around $120 and $130 million a year.

And don’t even get me started on Senator William Proxmire’s Golden Fleece Awards. They were the most ridiculously anti-intellectual and anti-science publicity stunt carried out by a politician for a sustained period of time that I can recall.

More recently, there have been other examples. Representative Darrell Issa tried to defund NIH grants already awarded through the regular peer review process because the research topics ran counter to his politics. Indeed, politicians try to insert their politics into government science policy all the time, particularly into climate science, reproductive medicine, and any other science that butts heads with politics or religious fundamentalism. Fortunately, such blatant direct meddling in the peer review process by a politician in the U.S. as what Issa attempted four years ago is, thankfully, relatively uncommon. Over the decades, there has been a fairly strong taboo against directly interfering in the peer review workings of the NIH and NSF, although periodically politicians try to make hay by attacking the NSF for funny-sounding science projects funded by the NSF or NIH.

It would appear to be a taboo that Meet Lamar Smith, a Republican Congressman representing Texas’s 21st district, wants to break, at least with respect to the NSF:

The new chair of the House of Representatives science committee has drafted a bill that, in effect, would replace peer review at the National Science Foundation (NSF) with a set of funding criteria chosen by Congress. For good measure, it would also set in motion a process to determine whether the same criteria should be adopted by every other federal science agency.

The legislation, being worked up by Representative Lamar Smith (R-TX), represents the latest—and bluntest—attack on NSF by congressional Republicans seeking to halt what they believe is frivolous and wasteful research being funded in the social sciences. Last month, Senator Tom Coburn (R-OK) successfully attached language to a 2013 spending bill that prohibits NSF from funding any political science research for the rest of the fiscal year unless its director certifies that it pertains to economic development or national security. Smith’s draft bill, called the “High Quality Research Act,” would apply similar language to NSF’s entire research portfolio across all the disciplines that it supports.

A draft of the legislation can be found here. The draft bill calls upon the director of the NSF to post a notice before funding a new grant a notice certifying that the the project funded by the grant:

1. is in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense by promoting the progress of science;
2. is the finest quality, is groundbreaking, and answers questions or solves problems that are of utmost importance to society at large; and
3. is not duplicative of other research projects being funded by the Foundation or other Federal science agencies.

As Steve Novella points out, stripped of context, these criteria sound relatively reasonable, although I do disagree with him on one point. Specifically, I’m not that worried about criterion #3, the ban on duplicative research. Steve thinks it is a bad idea because replication of research is key to science and that there is a problem with defining “duplicative.” True enough, but it’s also hard to define “innovative’ and “significant,” but that doesn’t stop the NIH from requiring applicants to demonstrate why their proposals are innovative and significant. Steve has a point, but personally I don’t have a huge objection to #3. Besides, this draft proposal says “is not duplicative of other research projects being funded by the Foundation or other Federal science agencies,” not “is not duplicative of other research projects that have been funded by the Foundation or other Federal science agencies.” (Emphasis mine.) There’s nothing in the third criterion to stop the funding of confirmatory research projects, which is what Steve was concerned about. Indeed, nearly all research grants tend to build on research that has gone before, which means they are at least in part confirmatory. The intent of that criterion is more along the lines of preventing research projects that studying basically the same thing in the same (or a very similar) way from being funded at the same time by the government. “Duplicative research” can be a good thing in concept, but in the real world there’s nothing that says that, in these days of highly constrained research funding (and, believe me, I know how constrained it really is—something like the 6th percentile from the NIH, worse from many other agencies), the government should be funding duplicative studies when doing so would tie up money that could go to other promising studies that aren’t duplicative. It really is a zero sum game right now.

Besides, the real damage comes from criteria #1 and #2, which are a radical change from the NSF’s current guidelines, which ask reviewers to consider the “intellectual merit” of a proposed research project as well as its “broader impacts” on the scientific community and society. Again, on the surface, these two criteria don’t sound all that unreasonable. After all, does anyone seriously want to fund research that is not of the finest quality and doesn’t serve the interests of the U.S.? The devil is in the details of how you define the “finest quality” and what is in the interests of the U.S. Who would determine that? According to the bill, the NSF director would, but to whom does the NSF director report? Congress.

Trying to counter criticism that this bill is going to replace peer review at the NSF with the above three criteria, Smith has responded by trying to say that he’s really, really for basic science and just trying to add “accountability.” Accusing critics of “playing politics” (what else would they do, given that this is politics, and what is Smith himself doing but trying to inject more politics into science funding?), Smith fired back:

It is the job of Congress and the NSF to make sure that taxpayer dollars are spent responsibly. I support basic research, which can lead to discoveries that change our world, expand our horizons and save lives. For example, we should prioritize research projects like the brain mapping initiative that may help cure Alzheimer’s, autism, epilepsy, and brain injuries.

“The draft bill maintains the current peer review process and improves on it by adding a layer of accountability. The intent of the draft legislation is to ensure that taxpayer dollars are spent on the highest-quality research possible.

Which is rather curious. If he’s just adding three more criteria for funding NSF grants, “another layer of oversight,” as he puts it, then it’s hard to see how this will actually achieve the stated aim. Remember, if this law passes, the NSF would have to insert these criteria into its existing criteria and might even have to change its grant application structure to require applicants to insert sections addressing each of these issues. In other words, it’s not hard to visualize applicants in the not-so-distant future being forced to justify how their science “is in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense.”

Suspicion is also warranted given that Smith recently tried to pull an Issa and directly meddle in the peer review process, demanding the peer review reports and program officer’s synthesis of the peer review discussions of five NSF-funded grants. Not surprisingly, given Smith’s history, all of them are in the social sciences. Smith’s action inspired Rep. Eddie Bernice Johnson to write to Lamar Smith to complain. Her letter is worth reading in full, particularly her sarcastic rejoinder to Smith that no Chair of the Committee on Science, Space, and Technology has ever in her memory directly inserted himself into the peer review of individual grants, thus in effect claiming expertise in that area of science, pointing out:

Interventions in grant awards by political figures with agendas, biases, and no expertise is the antithesis of peer review processes. By making this request, you are sending a chilling message to the entire scientific community that peer review may always be trumped by political review.

Which is exactly the purpose of Smith’s bill. By forcing the director of the NSF to certify that each grant funded is in the interests of the United States, Smith is creating an opening by which politicians can justify engaging in interference with peer review by simply asking if a project meets criterion #1. Moreover, criterion #1 itself will ahve a chilling effect on whoever is NSF director because that director will not want to certify funded projects as meeting that criteria if he or she knows that Smith or one of his successors might object to the project. Worst of all, Smith’s bill would require the NSF’s oversight body, the National Science Board, to monitor the director’s actions and issue a report in a year, with the intent of ultimately applying the principles in the bill to other federal science agencies.

Finally, there would be an effect on research that would be more insidious than political censorship, and, as before, criterion #1 is the key, where it has to be certified that each funded project must be “in the interests of the United States to advance the national health, prosperity, or welfare.” A clearer demand for only research that promises rapid, short-term translation into useful products I can hardly imagine.

This is a huge issue in medical research that scientists have complained about for years, if not decades, namely the NIH emphasis on “translational” research; i.e., research that leads directly to treatments or diagnostic tests. Here’s the problem. Translational research requires a robust pipeline of basic science findings to “translate” into treatments. Most of those basic science findings will lead to nothing, and it’s very difficult to know ahead of time which ones will bear fruit in the form of improvements in treatment and which ones will not. Hindsight is 20-20, but trying to guess which lines of research will lead to cures in advance is almost impossible. Such guesses tend to be more a matter of luck than science, and the complaints of scientists who think their work is so innovative are often sour grapes.

I’d like to finish with an example of the sorts of problems opening the door to direct political meddling in the peer review process could lead to. Imagine, if you will, an antivaccine politician. Oh, wait. You don’t have to. There was Dan Burton before he retired, and there still is Darrell Issa, who threw an antivaccine last hurrah for his good buddy Burton last November in the form of a Congressional hearing on autism that gave full voice to antivaccine loons. Then, yesterday I received an e-mail pointing me to this “call for action“:

Please click on the Take Action Link above to send a message to your member of the House of Representatives asking him or her to co-sponsor House Resolution H.R.1757, The Vaccine Safety Study Act. This bill directs the National Institutes of Health to conduct a retrospective study of health outcomes, including autism, of vaccinated versus unvaccinated children. The NIH adamantly refuses to do any study that compares health outcomes in these two groups. You have to wonder why.

The bill was introduced several days ago by Rep. Bill Posey (R-FL) and is co-sponsored by Rep. Carolyn Maloney (D-NY), who has long championed this cause. As a bi-partisan effort, we are optimistic that H.R. 1757 can move the House in a positive direction, if we make a strong effort to support this bill.

Now many of you are probably thinking, “Why would we expect the NIH to do an honest study?” That is an excellent question. But at the very least supporting this bill will help educate the house on the shoddy methods that are standard operating procedures in the vaccine industry.

The introduction of the H.R. itself is accompanied by a statement by Posey that’s chock full of antivaccine tropes and broadsides against the CDC. Amusingly, the bill itself demands that the each investigator of this study:

1. is objective;
2. is qualified to carry out such study, as evidenced by training experiences and demonstrated skill;
3. is not currently employed by any Federal, State, or local public health agency;
4. is not currently a member of a board, committee, or other entity responsible for formulating immunization policy on behalf of any Federal, State, or local public health agency or any component thereof;
5. has no history of a strong position on the thimerosal or vaccine safety controversy; and
6. is not currently an employee of, or otherwise directly or indirectly receiving funds from, a pharmaceutical company or the Centers for Disease Control.

One notes that requirements #3 through #6 pretty much destroy any chance the NIH would be able to find an investigator who meets requirement #2.

The whole study is, of course, utterly pointless, and I’ve explained in my usual inimitably nauseating detail why such a study would be very expensive, potentially unethical depending on design, not justified by science, and highly unlikely even to show what antivaccinationists want it so show, given the numbers of unvaccinated children that could be enrolled. It would be a complete waste of money.

Of course, this sort of law could be passed regardless of whether Smith’s draft bill ever becomes law, but it requires mustering all the usual votes necessary to pass any law. Anything short of a law would not compel the NIH to do such a study, and such a law would be very difficult to pass. Now imagine if powerful committee chairs could start meddling in the peer review process of NIH grant applications. If it is widely known that powerful lawmakers want a “vaxed-versus-unvaxed” study, then the process becomes easier. All that is needed is for an investigator or two to submit NIH grant applications, have them rejected, and then let antivaccine lawmakers start mucking around in the peer review process, making it known that they consider such a study to be in the national interest. Why not? Remember Representative Dan Burton, who retired in January after 30 years in the House? A supporter of quackery and a rabid antivaccinationist who thought nothing of abusing his power in the cause of promoting quackery and the myth that vaccines cause autism? Or Dave Weldon, a physician and Representative Dave Weldon of Florida, who was in the House of Representatives for fourteen years and, unfortunately, believed in the “autism epidemic.” He also apparently believed in the thimerosal-autism link, to the point of promoting highly dubious studies and attacking decent studies on the subject. Then there’s Representative Bill Posey, who, of course, is the sponsor of the H.R.1757. He’s also been invited, along with Dave Weldon and Dan Burton, to appear as part of a Congressional panel on autism, at that quackiest of autism quackfests, Autism One.

So is it so far-fetched to imagine politicians using Smith’s opening to meddle more openly in peer review? Maybe. But I don’t consider it as far-fetched as it should be.

Over the decades, through both Republican and Democratic administrations, Republican and Democratic control of Congress, a taboo against lawmakers directly interfering in the scientific peer review process as practiced at the NIH, NSF, and other federal science agencies has existed. The system has served us well. No one is arguing that the system can’t be improved, least of all me, but Smith’s way is not the way. It is a path to direct political interference in scientific peer review.