So, I finally got the reviews for the K99/R00 grant application I submitted to the NIH in June. Since the K99 application guide that I wrote back then has turned out to be one of the most-read articles on my site, I wanted to write a follow-up post about my reviews for those who are interested.

How the K99 is scored

First, let’s discuss how the applications are scored. Your application will be read by three reviewers on the study section, each of which will score your grant in five categories. In each category the scores can range from 1 to 9. 1 is the best and 9 is the worst. Here are the categories:

1) Candidate

• This score reflects what they generally think of you and your background, publication record, etc. I think it’s mainly based on what you write in the “Candidate” section and your biosketch, as well as your reference letters.

2) Career Development Plan / Career Goals / Plan to Provide Mentoring

• This score is based on your training plan, career goals and objectives, the mentor plan, letters from your co-mentors and collaborators, and possibly the description of institutional environment. The reviewers are evaluating whether the training you’ve proposed is appropriate given your background, career goals, and timeframe.

3) Research Plan

• This score is based on your research proposal, ranging from its conceptual merits and potential impact on the field to its technical feasibility and experimental design.

4) Mentors and Co-mentors, Consultants, and Collaborator(s)

• This score reflects not only the letters that your collaborators provide but also how well you convince the reviewers that these people are amazingly qualified and will greatly contribute to your project.

5) Environment and Institutional Commitment to the Candidate

• Here’s where the reviewers look the statements regarding equipment, facilities and resources, and the letter of institutional commitment. They will judge whether your institution, department, and lab are a good place for doing the kind of science you’ve proposed and training you to be a PI.

In addition, the reviewers will look at your other statements regarding vertebrate animals, training in the responsible conduct of research, resource sharing plan, budget, etc. But for these things the reviewers will simply indicate whether the plan is “acceptable” or not. If something is not acceptable you need to fix it before they’ll give you the grant, but it may not impact your score (unless it pisses them off).

What’s a good score?

So you’ll get the 5 category scores from 3 reviewers, and you’re hoping for mostly 1s and 2s. But none of these numbers really matter. All that matters is the final impact score, and this isn’t simply an average of the individual scores.

Here’s how it works in the study section. First, the three reviewers who actually read your application will present their opinions to the entire section and lead a discussion about it. Then all the members get to vote. They each come up with a single score from 1 to 9 (again, 1 is the best), those numbers are averaged, and that average is multiplied by 10 to get your impact score, ranging between 10 and 90.

The successful K99 applications that I’ve seen received scores of 20 or lower. But the scores don’t always relate directly to funding. Which applications get funded will be decided at a separate council meeting at least a month after the initial review.

Most of the decisions are made based on the impact scores, but I’ve heard that sometimes there can be exceptions for extenuating circumstances. And the distribution of scores varies for each batch of applications—if there were a lot of great scores in your batch then you’ll need a better score to get funded, and vice versa.

So, how did I do?

I received my impact score about four months after I submitted the grant. You don’t get the reviews (“summary statement”) till a month later.

My score was 42. While 42 may be the answer to life, the universe, and everything, it is not a good K99 score. As mentioned above, anything in the 10s is great, 20s are good, above that not so much.

So for a month I wondered what the reviewers hated about my application and whether I had given everyone terrible advice in my K99 application guide and would have to crawl into a hole and hide.

Then I got the reviews. They were actually pretty good.

The reviewers generally liked my research proposal. Two of them mentioned some specific weaknesses, but nothing that seemed that major. The scores I got for the Research Plan category were 3, 3, and 2.

The reviewers loved my training plan and all the other stuff I wrote in the “Candidate” section. I also got high marks for the Mentors/Collaborators and Environment/Institutional Commitment sections, which I expected since my advisor, collaborators, department, and university are all very highly regarded in the field. For these three categories (#2, 4, and 5 listed above) my scores were mostly 1s with a couple of 2s. Yay.

So what didn’t they like?

The fact that I haven’t published any papers during my postdoc.

Seriously, that’s about it. [Update: see below for new information from my PO.]

(Aside: It’s not like I didn’t anticipate this. This was the very reason I started putting together a paper last year and gave my advisor a complete manuscript in February. Unfortunately my advisor is a busy guy, and long story short, he still hasn’t signed off on submitting my paper.)

My lack of publications was reflected in the scores for the Candidate category, which were 4, 4, and 2. But even those scores aren’t terrible. If you average all the individual scores I got for the five categories, it comes out to 1.9, which (multiplied by 10) would be an impact score of 19. That’s a far cry from 42.

So it seems like that one thing – a lack of publications – completely overshadowed everything else. It also seems like maybe the rest of the study section didn’t like my application nearly as much as the three main reviewers did, possibly because they had differing opinions on the importance of one’s publication record.

My summary statement mentioned this specifically: “While all reviewers did not have the same level of concern about this issue, some on the panel expressed strong reservations.” Looks like those people won.

I’ve heard that the Program Officer (PO) can give you additional feedback and guidance on your reviews. My PO is currently out of the country so I have to wait till December to speak with him, but I’ll update this post if his feedback changes anything.

Update on 12/6/2016: I finally got to speak with my PO and he told me that yes, the main issue was my lack of papers, but in addition, some members of the study section had concerns about my research plan. Some specific issues he mentioned (which were alluded to in the reviews) were that the models I proposed were too simple, the preliminary data wasn’t strong enough, and the data analysis methods were limited. So for those of you worried that my story means your application will be rejected purely based on a lack of papers, this information should give you hope that maybe a really really strong research plan can overcome that limitation.

Will it get funded?

I highly doubt my application will get funded. I’ve been told that you never know, even with a score that seems unfundable, because we don’t know what kind of scores they gave to the rest of the applications. There’s also the possibility of writing some kind of appeal if you felt you were unfairly reviewed or have an extenuating circumstance.

IF I can improve my publication record in the next 6 weeks before the council meeting by getting my paper submitted or even accepted (ha), then that might be grounds for an appeal—but it’s a longshot.

Update on 12/6/16: My PO told me that for this study section, there’s no way that a score of 42 is fundable. Since I’m not eligible to resubmit my application, he recommended that I instead apply for a K01 award, which has a longer eligibility window. I haven’t looked into the K01 very much and I’m not sure I have the energy to apply for any more NIH grants as a postdoc, but I thought I’d mention it for others in a similar situation.

Final takeaways

The good news for me is that most of what I wrote was received pretty well. The main thing that brought me down was something entirely out of my control. The comments I got on my research proposal were fairly constructive and could help me on future grant applications.

The good news for you is that this probably means my K99 application guide does contain some solid advice. In particular, I was praised for assembling a diverse team of collaborators and a “clear and excellent” training plan. I also think my statements addressing the institutional environment, facilities and resources, equipment, etc. were successful in convincing the reviewers that I’m working in a great environment.

In addition, all the work I put into the writing did not go unnoticed (though it may go unfunded). The reviewers said my application was “well-organized” and “beautifully written”. This didn’t magically happen on its own; it took lots of editing and rewriting. So don’t put yours together at the last minute—leave plenty of time to rework it! Even if they’re not explicitly scoring you on your writing, good writing and organization makes it much easier to understand the content.

That’s all for now. I hope hearing about my experience is helpful for those of you putting together your own applications or navigating the review process. And if anyone else wants to share their own experience, feel free to comment below!