NIH Extramural Nexus

Publications are one of the important products of NIH research grants, and authors are required to cite their NIH support in their publications. But as many of us have experienced, the format of grant numbers varies greatly. Some authors use institutional grant tracking numbers, others abbreviate the NIH number, and many other permeations arise. It almost becomes a “Where’s Waldo?” of the grant number world! All this makes it very difficult to directly link NIH grants to their publications, but the combination of public access policies and an NIH software development effort have recently improved this problem for those publications available via the NIH National Library of Medicine.

In 2001, NIH developers created the first version of a database known as SPIRES (Scientific Publication Information Retrieval and Evaluation System). In a nutshell, SPIRES maps publications to NIH grants. In practice, creating the means to do this was not simple at all. SPIRES uses automated text manipulation methods to extract and reformat grant numbers cited in publications. The reformatted numbers are then compared to NIH grant data, and the “goodness” of the match is rated by the SPIRES system. 

A decade later, SPIRES is now a mature database that maps 30 years of publications from PubMed to NIH grants. The results display through a number of internal NIH systems and to the public on the RePORT website. Because we often demonstrate outcomes of NIH support through publications arising from NIH grants, SPIRES has proven to be a critical component to accurately measure impact. More details about the SPIRES system and an example of what can be learned from publication data are available in the paper “Metrics associated with NIH funding: a high-level view” in the Journal of the American Medical Informatics Association.

It came to my attention that the analysis we posted last month on the numbers of investigators with multiple awards couldn’t be recreated. A reader took ExPORTER data from 2009 and came up with different numbers.

We have been attempting to replicate your data on the number of grants for the top 20% of PIs and cannot. Here is what we have come up with. Our total, using each of two assumptions about how you counted grants, falls short of your total number of PIs by more than one quarter. Can you explain how PIs were counted in your data in FY2009?

So, we asked them for their data and took a second look at our own. And, unfortunately, we posted the wrong data. These are complicated analyses, and we often go through multiple iterations of each analysis. We at the NIH are, sometimes, human, and we chose the wrong file from the folder. I apologize for this error, and I will provide you with the correct data here. In addition to the updated data for the investigators who receive the top 20 percent in total funding, we’ve added the data for all PIs.

Compared to what we posted previously for the top 20 percent, the absolute number of investigators has been corrected downward, but the percentage of investigators falling into each category remains about the same.

Many of you requested similar data for all PIs. If you crunch the numbers, you will see that in each of the four years presented more than 90 percent of our investigators hold one or two research project grants. I hope this helps clear up any confusion with the previous data.

Last month, I blogged about the number of investigators having multiple awards, with a focus on relatively well-funded investigators (the top 20 percent in total funding, or “Top 20”). We got lots of great questions and comments, but a number of commenters weren’t convinced we busted the myth. For example,

“I don’t know how relevant it is to lump R01s in with R21s and R03s (at the smaller end) and big P and U grants (at the higher end). How do all these different types of grants correlate, and how do the actual DOLLARS per investigator map out to different grant types?”

So I would like to share some more information on the sources of support for the Top 20 and help answer at least part of the question. The figure below shows the 11 most frequently used research project grants used to support these investigators. 

Within the Top 20, those investigators having a single award (about 20 percent of the total) are relatively unlikely to have an R01 award and more likely to have one of the large multi-project grants (U01s, P01s, and U19s). The distribution of grant activities among those Top 20 investigators who have multiple awards is more similar to the overall NIH distribution of these activities. There is a higher frequency of some of the smaller grant mechanisms (R21s and R03s) among investigators not in the Top 20, as you might expect. 

I’ll continue to post additional analyses like this one as we work to better understand patterns of NIH support and the pool of investigators we support.