It looks as though I’ve been tagged by Drug Monkey, who apparently thinks that I might have something worth saying about the state of the NIH and its peer review system, about which the NIH is presently soliciting comments, as pointed out to me by Medical Writing, Editing, & Grantsmanship. Why Drugmonkey might think this to be the case, I have no idea, but presumably it has something to do with some previous posts that I’ve made about the NIH, how biomedical research is funded in this country, and the disconnect between vision and reality at the highest level of the NIH. Although I used to do lots of blog memes earlier on in my blogging career, these days I tend to ignore the vast majority of them that are sent my way. They rarely interest me these days, and now that I’ve become an old and established blogger, I’ve developed, reasonably or unreasonably, a rather contrarian streak in that I don’t like being told what to blog. This meme, however, is about the most science wonky meme I’ve ever seen; so I think I’ll participate. I also hope that you, my readers, will chime in as well.
So what is the NIH asking for comments about? This, with the comments page here:
NIH and the Working Group welcome your comments on these CSR’s current activities; however, we would particularly like your opinion, as a reviewer, applicant, or member of the public, on how to enhance the system employed by NIH to support biomedical and behavioral research, including the peer review process. The NIH is especially interested in creative, concrete suggestions to the following questions, for strengthening over the long term any and all aspects of our system for identifying the most meritorious and innovative research for support:
- Challenges of NIH System of Research Support. Please describe any specific challenges presented by NIH’s support of biomedical and behavioral research such as the current array of grant mechanisms, number of grants awarded per investigator, and the duration of grants.
- Challenges of NIH Peer Review Process. Please describe any specific challenges presented by the current peer review process at NIH.
- Solutions to Challenges. Please concisely describe specific approaches or concepts that would address any of the above challenges, even if it involves a radical change to the current approach.
- Core Values of NIH Peer Review Process. Please describe the core values of NIH peer review that must be maintained or enhanced.
- Peer Review Criteria and Scoring. Are the appropriate criteria (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-05-002.html) and scoring procedures (http://cms.csr.nih.gov/NR/rdonlyres/B2CFE17E-AA1C-46E5-BADB-FDBF2FBBEE80/11892/CSRScoringProcedure090706.pdf) being used by NIH to evaluate applications during peer review? If not, are there changes in either that you would recommend?
- Career Pathways. Is the current peer review process for investigators at specific stages in their career appropriate? If not, what changes would you recommend?
The ground rules are:
I. Include the links to the NIH RFI and the comments page.
II. Post the following RFI queries
III. Address at least one of the NIH queries in a post.
IV. Reach out and tag 7 someones, be they biomedical scientists, physical scientists, taxpayers or medical consumers.
Got it? So let’s go.
I’m not going to address all of these points. Rather, I’ll concentrate on maybe half of them. As far as challenges go, the biggest challenge at the moment to the NIH is its budget. However, that is an issue that is up to Congress and the President; the NIH can only live with the appropriations that it is given. Still, many of the problems of the NIH would be much less acute were it not subject to a boom-and-bust cycle, of which we are presently in a rather nasty bust. This leads to unwise spending when times are good and excessive conservatism and the endangerment of young careers when times are not. Even so, there is one thing that the NIH could do that would make its tight grant dollars go farther. It could stop providing salary support to principal investigators and co-investigators.
Really.
The NIH grant funding system is a gravy train in some ways for universities. I once discussed how we as biomedical investigators are rather like free lance used car salesmen. Medical schools in essence rent basic researchers laboratory space and infrastructure support, and then expect them to bring in grant support sufficient (1) to pay a significant proportion of their salaries and (2) to bring in indirect costs sufficient to fund the support of their labs. (Typically, NIH grants pay an 50-60% extra on the dollar for each grant dollar brought in, and that money goes to the institution to support infrastructure.) Although I can understand that it might be desirable to provide universities indirect costs to help support research, there’s no good reason that I can think of that universities should be given such a free ride when it comes to the salaries of their faculty who are principal investigators, and that money would be better spent funding the actual research, in the form of supplies and the salaries or stipends of postdocs, technicians, and graduate students doing the work.
A second reform that the NIH could make on its own would be to cap the amount of NIH grant funding per year that any single investigator can have. In this view, I’m with Drugmonkey. The only reason a few principal investigators can have multiple R01 grants at 10% effort is because they have less senior people working for them. Cap the amount of RO1 money, but don’t cap the amount of P01 money that a researcher can have. (PO1s are large collaborative awards in which multiple investigators working on related projects are funded for individual R01-level projects that make up the PO1, the concept being that they can utilize shared resources for more efficiency and to develop synergy.) More senior researchers could be the PO1 project leaders, while less senior researchers do the R01-level projects (and get the R01-level funding). More senior researchers would then also be almost forced to mentor younger faculty, because if they wanted their PO1s funded, they would have to make sure that each component R01 is up to snuff, because just one bad component can sink the whole project. That means they’d have to roll up their sleeves and help the other investigators in the P01 hone their individual applications to a fine edge of scientific excellence. Moreover, even for those investigators who are not project leaders, having multiple grants that are components of PO1s will encourage collaboration and synergy between multiple research groups.
Aside from funding issues, however well the system worked in the past, when money gets tight (as it is now), it becomes onerous not only to the investigators applying for grants but to those reviewing them. The reason is that, when money gets tight, the paylines similarly get tight, and this leads to investigators who might have been funded on the first or second try having to submit the same grant application three times (which is the maximum number of times an application can be submitted) before being funded or rejected for good. With few exceptions, even excellent grant proposals are not funded on the first try. Rather, they are put in a “holding pattern” of what are in essence niggling, nit-picky reviews to which they must respond, even though their proposals are sound and deserving of funding. Investigators thus have to revise their applications more, but, worse, from the NIH perspective, peer reviewers have to review all those additional resubmissions again and again until the investigator “pays his dues” and is funded. It’s a huge waste of time and effort. Whatever solutions to the NIH peer review problem has to somehow decrease the load on the peer reviewers. It’s already getting difficult enough to retain them. This is perhaps the most significant challenge facing the NIH peer review system. Moreover, the tight paylines make it very difficult for a new investigator to obtain funding. Consequently, the two biggest challenges to the NIH peer review process are to provide high quality peer review in as short a time period as possible, given that almost all investigators will require multiple resubmissions for the foreseeable future, without resulting in reviewer burnout and to find a way to fund more new investigators before they give up and disappear from academia and research.
Despite its problems, there is much that is good about the present NIH peer review system. Despite its flaws and its tendency towards conservatism when money gets tight, it is still arguably the most egalitarian system in the world. The quality of the science really is what winds out in the end the vast majority of the time. Any solutions to the current problems must maintain this aspect of the system.To this end, as much as I’ve expressed skepticism over its value, reviewing grants via online study sections, I think they’re probably the wave of the future. Although for a younger investigator like me, they take away one key advantage of serving on a study section (face time with senior colleagues from other institutions and a chance to network and show them how smart you are) and more detailed discussion of the proposals being reviewed, they have a huge advantage in time savings, cost savings (no more flights, hotels, or honorarium), and thus are attractive to many investigators. What needs to be done is to make them more interactive. Right now it’s just text on a screen, an online discussion forum. There’s no reason teleconferencing or video chat couldn’t be used to make these sessions less sterile and to allow the participants to put a face to the names on the computer screens.
Finally, I’ll finish with a word about career paths and grant mechanisms. The investigator-initiated award must retain primacy, because it allows investigators to propose whatever project that they can thingk of and to have them judged on the scientific quality. However, some tinkering might be in order. Specifically, there needs to be a reemphasis on preliminary grants for riskier ideas, much as the Army does. Presently, R21 grants, the primary mechanism for such projects, are almost as difficult and time-consuming to write as an R01, so much so that most senior faculty advise junior faculty not even to bother with them, given that they are also funded at an even worse payline than R01s. Such mechanisms should be beefed up, but particularly the application should be streamlined and shortened so that it doesn’t take as long to write an exploratory proposal for a one or two year preliminary project as it does to write a well-developed proposal spanning five years.
You know, this one’s so involved that I think I’ll end here and consider doing another post on the topic to look at other issues.
I now hereby tag:
- Steve Novella at Neurologica
- Abel Pharmboy at Terra Sigilatta
- Tara Smith at Aetiology
- Shelley at Retrospectacle
- Revere at Effect Measure
- Evil Monkey at Neurotopia
- Larry Moran at Sandwalk
No, no, really. Don’t thank me. I’m sure you’d all do the same for me.
