Report of the NIH Working Group on New Investigators

June 4, 1997

• Executive Summary

• Introduction

• Part 1: How Have New Investigators Competed in the NIH Funding System?

• Part 2: Are the Mechanisms That Have Been Used to Target New Investigators Effective?

• Part 3: Conclusions and Recommendations

• Appendix 1: List of Members of the Working Group of New Investigators

• Appendix 2: Description of Programs Discussed in the Report

• Appendix 3: Tabular Data Associated with Figures in Text

Executive Summary

Introduction

Our national capacity to advance biomedical research and to improve the health of the American people in no small measure resides in the nurturing, education, training, recruitment, and employment of cadres of new investigators. After a long period of rapid growth in the biomedical research enterprise the past decade has seen a significant reduction in the rate of increase. In this new environment the NIH along with many other federal and non-federal agencies and organizations has become deeply concerned about the prospects for survival of the next few generations of biomedical scientists.

A major consequence of slower growth can be a reduction in the rate at which new people enter the system. This was clearly demonstrated in the course of our analysis, when it was observed that the entry rate of new investigators into the research system (i.e. the yearly percentage of the R01 and R29 principal investigator pool that are new investigators) has decreased from 11-12 percent in the early 1980's to about 9 percent currently. In order to insure that such decreases do not continue, it was necessary to examine how new applicants to the research grant system have fared in the past, what has been the success of the various mechanisms designed to bring new investigators into the research system, and what is needed to insure that in the future NIH has a stable number of new investigators that can continue to maintain the high quality of research which characterizes today's biomedical research community.

Part 1: How Have New Investigators Competed in the NIH Funding System?

Q. Are new investigators applying in lower numbers?

No. Between the early 1980's and the early 1990's, the number of applicants applying for their first R01 or R23/29 awards each year has stayed essentially constant. However, the median age of new investigators has increased from about 35 in 1981 to 39 in 1994.

Q. Have new investigators been selectively disadvantaged as success rates have decreased?

No. New applicants for an R01 who have a previous history of research grant support have had a success rate only slightly better than that of investigators without prior NIH support. This differential has stayed constant over the time period examined. This combines the outcomes for new R01 applicants and R23/29 applicants.

Q. Do new awardees have a significantly lower probability of success on competitive renewal than investigators with a previous research record?

No. When funded applications were matched for peer review outcome, i.e. the top 10 percent of new awardees matched against the top 10 percent of those with a research history, etc., there was overall only a small advantage to investigators with a previous research record.

Q. Is it true that virtually no new investigator who receives an R01 award gets it on the first try (i.e. unamended)?

No. In the period 1993-1995, 58.6 percent of all new R01 and 53.2 percent of all R29 awardees received their award based on the original (unamended) application. However, this represents a significant decrease from 1980-1983, when 85-90 percent of the awards were based on unamended applications.

Q. Is there any difference in the success rates for new M.D. investigators compared to Ph.D.?

No. The success rates are virtually identical.

Part 2: Are the Mechanisms That Have Been Used to Target New Investigators Effective?

Q. Do applicants for an R29 have an advantage over new applicants for an R01?

Yes. Those applying for an R29 have consistently had a significantly higher success rate relative to new applicants for an R01.

Q. Do R29 awardees do differently on renewal than new R01 grantees?

Yes. The R29 recipients in the top 10 percent of peer review outcome do just as well as their peers with R01s. However, awardees in the second and third decile are significantly less successful in receiving subsequent R01 awards than their new R01 grantee counterparts.

Q. Is it true that receipt of an R23 or R29 award has either increased the number of first-time applicants to NIH or has promoted their retention in the system?

No. The number of applicants has stayed roughly constant since 1980. Further, although a higher percentage of R29 applicants get awards than do new R01 applicants, they have a lower success rate on subsequent resubmission. The result is no net increase in yield.

Q. Is it true that the K awards have increased the number of M.D. applicants or those awarded independent research grants?

No. In fact, the number of M.D. applicants and awardees, as well as the fraction of the applicant pool, has decreased since 1980.

Q. Is it true that receiving a K award provides a useful entry point for M.D. applicants with limited research experience?

Yes. Approximately 20 percent of the applicant pool for K08/K11 awards subsequently receive an R01.

Q. Is it true that the R03 mechanism is effective in bringing investigators into the research system of NIH?

No. The number of awards has always been small, and only about 9 percent of the applicant pool subsequently receive an R01.

Part 3: Conclusions and Recommendations

Because of the importance of new investigators to the health and long-term viability of research, over the past two decades the NIH has initiated several mechanisms with the intent of facilitating the entry and retention of new investigators in the sponsored research system. These have met with varying success.

1) The K08/K11 mechanisms appear to have been successful. These mentored scientist awards are primarily targeted toward M.D.s who are interested in research careers and have not had significant research training. A yield of 20 percent of the initial applicant pool seems quite significant, given that the initial pool represents M.D. applicants with little prior research experience and considering the importance of attracting M.D. researchers. However, the majority of M.D.s who receive an R29 or R01 did not apply for or receive a K award. For example, of 1,025 M.D.s receiving either a R29 or R01 in 1987-89, only 178 (17 percent) had previously held a K award. Further, the number of M.D. applicants or awardees has not been increased over the period 1980-1995. This mechanism may thus be valuable to target specific populations of M.D. researchers where a specific need is seen, but it will not significantly contribute to the total number of M.D.s entering the research system.

Recommendation

The K08 mentored scientist award should be retained, since it appears to generate a high yield of R01 awardees. However, it is unlikely to make a significant contribution to the total number of M.D. investigators receiving independent research awards.

2) The R03 award is not exclusively targeted at new investigators and is used for different purposes by the different Institutes, Centers, and Divisions at NIH. The value of this mechanism in contributing to these varied goals is not considered here. There is little evidence, however, that the R03 contributes significantly to the numbers of new investigators introduced into or retained within the NIH research system. Further, the yield of R01 awardees from the initial population of applicants is very small, less than 9 percent.

Recommendation

The R03 award should be evaluated based on its utility for specific goals of the Institutes and the NIH. However, it does not contribute to the general goal of stimulating the entry and retention of new investigators.

3) The NIH has created a mechanism, the R29, which has resulted in the creation of two populations of new investigator applicants. One of these, the R01 applicant pool, has a higher probability of future success (as defined by subsequent receipt of an R01 award) than the other. However, a selective advantage in initial award rates is provided to the less successful group, the R29 applicants, while simultaneously giving them significantly less resources (which may contribute to their lack of success in later grant submissions). As a result, the trade-off of higher initial success rates for lower costs appears to the Working Group to be in the end not justifiable, as is the creation of two separate pools of new investigator applicants.

Recommendation

The R29 mechanisms should be abolished and replaced by an R01 application that clearly identifies on the face page (or in some other standard manner) that the applicant is a new investigator, i.e. someone who has never had previous research grant support. The default award should be for five years, and the peer review groups should be instructed to require a lesser level of preliminary information than would normally be expected of an established investigator. The amount awarded should be whatever is deemed appropriate by peer review and Institute judgments.

4) Given that the average cost of an award to a new investigator will inevitably increase with the removal of the R29 award, there is clearly a concern that success rates will drop. It is essential that NIH define for itself what the entry rate for new investigators should be, based on an assessment of the needs of the research system.

Recommendation

The Working Group recommends that the number of awards to new investigators be made at levels no less than the numbers required to replace investigators leaving the research system. This has historically been at about 9 percent. Increases beyond the replacement rate will be dependent on real growth in the NIH budget and judgments about the appropriate distribution of such growth.

A consequence of this recommendation will be to insure that the steady decline in the entry rate of new investigators into the NIH research system does not continue. Together with the previous recommendation, this should provide new investigators with some assurance of stability at levels of support and lengths of awards that will optimize the probability of eventual success.

Finally, it has become clear to the Working Group, from interviews and other reports, that the NIH communicates poorly the content and purpose of its programs to new investigators. This is due in part to the multiplicity of mechanisms, the different uses to which they are frequently put in the various Institutes, Centers, and Divisions of NIH, and the unnecessarily obtuse bureaucratic language in which they are described. It is essential that this be corrected, and that new approaches be developed to help communicate clearly to the new investigator, who is feeling sufficient stress without having to wade through the morass of NIH official language.

Our national capacity to advance biomedical research and to improve the health of the American people in no small measure resides in the nurturing, education, training, recruitment, and employment of cadres of new investigators. After a long period of rapid growth in the biomedical research enterprise the past decade has seen a significant reduction in the rate of increase. In this new environment the NIH along with many other federal and non-federal agencies and organizations has become deeply concerned about the prospects for survival of the next few generations of biomedical scientists.

A major consequence of slower growth can be a reduction in the rate at which new people enter the system. This was clearly demonstrated in the course of our analysis, when it was observed that the entry rate of new investigators into the research system (i.e. the yearly percentage of the R01 and R29 principal investigator pool that are new investigators) has decreased from 11-12 percent in the early 1980's to about 9 percent currently (Figure 1). (Appendix 3 contains the tabular data corresponding to the figures.) In order to insure that such decreases do not continue, it was necessary to examine how new applicants to the research grant system have fared in the past, what has been the success of the various mechanisms designed to bring new investigators into the research system, and what is needed to insure that in the future NIH has a stable number of new investigators that can continue to maintain the high quality of research which characterizes today's biomedical research community. The Working Group on New Investigators was established to examine these issues. The first meeting of the Working Group was held on October 17, 1996, and discussions concluded in May 1997. The membership of the Working Group (given in Appendix 1), included both NIH staff and extramural investigators with recent experience as first-time applicants.

In order to evaluate whether new investigators are successful, it was first necessary to determine what success means. We did not address the question of what represents a successful career for Ph.D.s and M.D.s trained in research, but rather limited our analysis to those individuals who chose to apply for NIH research grants. A successful outcome was defined as the receipt of an R01 award. The R01, the individual investigator award, is generally considered to be the hallmark of an independent investigator who is recognized by his or her peers as conducting, or having the potential to conduct, highly meritorious research. We were not able to include in our analysis new investigators participating in centers or program projects, due to the fact that the necessary information is not contained in the NIH database. Also, the former ADAMHA Institutes are excluded in order to make the historical comparisons with the period prior to their incorporation into NIH (1992) consistent. We do not believe that our results or conclusions are vitiated by the absence of these data.

The report is divided into three sections. The first provides an overall analysis of new investigators competing in the NIH grant system in the period 1980-1995. The second section looks at the efficacy of various mechanisms for bringing new investigators into the system. New investigators tend to enter the system either through applications for an R01, an R29 ("FIRST") award, a K08/K11 mentored career development award (primarily directed toward M.D.s), or an R03 small grant award that provides an opportunity for generating preliminary results. (See Appendix 2 for details of these mechanisms.) After the initial award several routes are possible for those mechanisms that are specifically designed to enhance entry into the funding system. For example, some K08/K11 awardees (albeit a small number) move on to FIRST awards before applying for an R01. The analysis in Section 2 tracks the relative successes of the various mechanisms in progressing through each step in arriving at the award of an R01. The third section presents the conclusions and recommendations of the Working Group.

All of the results and conclusions that follow were possible only through the collection and assembly of large amounts of data. The Working Group is particularly grateful to Dr. James Onken, Chief of the Office of Program Analysis and Evaluation, National Institute of General Medical Sciences, for help in developing and interpreting the data on which the first part of the study was based, and to Dr. Onken and one of the Working Group members, Dr. Ronald Geller, Director of the Division of Extramural Affairs, National Heart, Lung, and Blood Institute, for the material on which the second part of the study was based. Analytic support was provided to the Working Group by Quantum Research Corporation under a task order contract to the Office of Science Policy and Technology Transfer, Office of the Director, NIH.

Part 1: How Have New Investigators Competed in the NIH Funding System?

As the rate of growth of the NIH budget has slowed, there has been special concern regarding the fate of new investigators. With the decline of success rates, it was thought that new investigators are selectively disadvantaged because, it is claimed, "it is nearly impossible for young scientists to compete in the same applicant pool against seasoned investigators¹." Further, a National Research Council study indicated that applications to NIH from investigators 36 years old and younger dropped by 54 percent between 1985 and 1993, raising "serious questions about the future of life-science research²."

¹"Trends in US Funding for Biomedical Research," Jennifer Ruzek, Edward O'Neil, Renee Willard, Rebecca W. Rimel, San Francisco, CA: UCSF Center for the Health Professions, May 1996, pp. 14-15.

²"The Funding of Young Investigators in the Biological and Biomedical Sciences," Committee on the Funding of Young Investigators in the Biological and Biomedical Sciences: Board on Biology, Commission on Life Sciences, National Research Council, National Academy Press, Washington, D.C. 1994.

It is clear that the climate for funding has become more competitive for all members of the research community. If, as suggested above, new investigators are particularly affected and perhaps even particularly discouraged from applying, it is important to know the reasons in order to identify the remedies. We have therefore conducted an examination of accessible NIH records to look at how the events of the past decade and longer have specifically affected the new investigator. A "new investigator" can be defined in two different ways, depending on the information that is desired. In one case, a new investigator is defined as someone who has not previously received NIH research support as a principal investigator, but who may have previously applied for support. Other analyses require the more restrictive definition whereby the "new investigator" is defined as someone who has never previously applied for support. It should be noted as well that new investigators, as defined in this report, are not necessarily individuals who are beginning an independent career in science, (although the great majority do fall in this category), since they may include those who have been supported from other sources or have moved from industry or from abroad, where NIH support is not the norm.

We looked at data for the period 1980-1995. In most cases, these analyses compare the outcomes for new investigators to the outcomes for "experienced" investigators. These are individuals who either currently have an R01 or have had one in the past. The comparison is always between new applications. Competing renewals are not considered since they reflect a pre-selected population that has already been judged of high enough quality to be funded.

Has there been a decline in the number of new investigators applying?

Perhaps the first issue that needs to be addressed is the disturbing report from the National Research Council describing a drastic decline in the number of applications to NIH from investigators under 36 years of age². In fact, this does not reflect a decline in the number of new investigators applying for funding. Figure 2 shows that the number of investigators between 1980-1995 that had not previously applied to NIH for an R01 or R23/29 has remained roughly constant. In fact, changes in the number of applicants for one mechanism are almost exactly offset by opposite changes in the other. The events in 1987, when the R29 was introduced, is a particularly striking example. At the same time, however, the median age of first-time applicants has increased from about 36 years to about 39 years (Figure 3). The decline in applications noted in the National Research Council report is the result of this increase in median age and does not indicate that newly independent investigators are being discouraged from applying. Rather, it appears to reflect an increasing length of time before an individual is in a position to apply as an independent investigator.

Have new investigators been selectively disadvantaged as success rates declined?

If we examine success rates over time, it is clear that the success rate for new investigators has decreased from about 26 percent in the early 80's to about 20 percent in the early 90's. (Figure 4). (When success rates are examined, the definition of a new investigator is one who has not previously received an award, but may previously have submitted an application.) This is also reflected by the decrease in the total number of R01 and R23/29 awards made to new investigators (Figure 5). However, the success rate of new applications from experienced investigators has gone down in parallel, from about 30 percent to about 23 percent in the same time period (Figure 4). Thus, although there has been a small but consistent advantage to new applications from previously or currently funded investigators, there is no indication that new applicants have been selectively disadvantaged as award rates overall have declined. Furthermore, there is no evidence that applicants with M.D. degrees are treated differently than those with Ph.D. degrees. The success rates for new M.D. applicants have been equal to or only slightly less than for Ph.D. applicants (Figure 6).

What is the probability of retention in the funding system after the first award?

To examine the ability of new investigators to remain funded after receipt of their initial awards, we analyzed applications submitted by two populations of funded investigators. The first group of investigators were those who submitted their "index" applications in 1978-1980, and the second group in 1989-1991. We then divided the index applications into those that were new applications from experienced investigators and applications from new investigators. Each group was ranked and divided into deciles, (i.e. applications were identified as falling in the best 10 percent of applicants, in the second best 10 percent, etc.). We then looked at how well those who reapplied fared on submission of their competing continuation 3-5 years later. We assumed that a "good" outcome was a resubmission which scored within the first three deciles of all applications submitted that year (e.g., in the range of scores with the highest likelihood of funding).

Figures 7 and 8 show the results for investigators initially submitting in 1978-80 and 1989-91. New applicants are only a little less likely to be successful on recompetition compared to experienced applicants who were given new awards. For example, new principal investigators who were scored in the second decile in 1989-91 had a 40 percent likelihood of scoring in the top three deciles on competitive renewal, compared to a 42 percent probability for an experienced investigator who had received a new award. Furthermore, if one compares the outcomes for new investigators in each decile compared to either competing renewals or new applications from current or previously funded investigators, it can be seen that the relative risk for new investigators on competitive renewal is no worse in 1989-91 than it was in 1978-80.

What is the likelihood of an RO1 or R23/29 application from a new investigator being funded without amendment ?

The percentage of funded applications awarded without amendment has in recent years been between 50-60 percent for both R01 and R29 applicants (see Table 9A in Appendix 3). Although this may be higher than expected, the probability of getting an award without amendment and resubmission has declined drastically for both R01 and R23/29 applicants. Between 1980-1995 the percentage of R01s awarded as amended applications increased between 3-4 fold, and the percentage for R23/29s increased 6-7 fold (Figure 9) . These very large changes may color the perception by applicants of the difficulties in arriving at a successful outcome more than the drop in success rate. (The marked decline in the percentage of awards made as amended to R29 applicants in 1987 is an artifact. Since that was the year that the R29 was introduced, virtually all the awards made that year to the R23/29 applicant pool would be to the original application.)

These problems are not unique to new investigators. New applications from current or previously funded investigators show comparable changes in the probability of getting an award at the initial submission (Figure 9).

What is the probability of success, at any time after first application, for a new investigator?

Since many applications require revision before an award is made, the success rate in any given year is not a good reflection of the probability that a new investigator will ultimately be supported. Table 1 gives the percentage of new applicants receiving an award at various intervals after initial application. A new applicant is here defined as one who has not previously submitted a research grant application. It is clear that even in the last years for which data were available, the probability of success in the first three years following initial submission was in the range of 25-35 percent. Although this is lower than the likelihood of funding in the 1980's, it is still higher than is generally reported.

Summary

1. The numbers of new investigators applying has stayed essentially constant between 1980-1995.

2. Although success rates have decreased for applications from both new investigators and those with a previous record of receiving awards, there is no evidence that new investigators have been selectively disadvantaged. This is true for both Ph.D.s and M.D.s.

3. New investigators overall do about as well in peer review in competitive renewals following the initial award as do their counterparts with a previous history of NIH awards.

4. The probability of funded applicants receiving an award on first submission has dropped dramatically between 1980-1995. This is equally true for applicants for the R29, new applicants for the R01, and experienced applicants applying for a new R01.

5. The probability of a new investigator being funded in the first three years following initial submission was between 25-35 percent in the most recent years for which data were available. Although this is lower than the success rates in the early 1980's, it is still higher than is generally reported.

Part 2: Are the Mechanisms That Have Been Used to Target New Investigators Effective?

The three types of mechanisms that have been used to target new investigators are the R23/R29 (research grants), the K08/K11 (mentored career development awards), and the R03 (small grant awards). (The R03 mechanism is not used exclusively for this purpose, and is applied by different Institutes for different needs. See Appendix 2.) The purpose of all these efforts has been to stimulate the entry of new investigators into the research system. This includes increasing the numbers of applicants, the number of awards, and the retention of awardees as independent investigators. For our purposes, retention is defined as receipt of an R01, the standard for success in the NIH research support system. In order to demonstrate that these approaches have been successful it is necessary to provide evidence either that they have generated increased numbers of new investigators holding R01 awards or that they have conferred some advantage so that the yield of R01 awardees is increased for those who have previously held an award such as a K08, for example.

Has there been a noticeable change in the number of new investigators applying for research grants?

As noted earlier, between 1980 and 1995, the number of first-time applicants has stayed essentially constant, fluctuating between 3000-3500/year (Figure 2). Despite efforts to provide facilitated access to the research system, primarily through the R29, (and earlier through the R23), there appears to have been no increase in the numbers applying. The curb to the size of the applicant pool lies elsewhere. Furthermore, it is not a simple function of the number of Ph.D.s produced, since between the middle 1980s and the middle 1990s the average annual production of Ph.D.s in the biomedical sciences went up almost 25 percent. In any event, to the extent that the mechanisms developed were intended to increase the numbers applying, they have not been successful.

Do the various mechanisms facilitate progression to an R01?

Although the intent of all the mechanisms was to assist new investigators, their goals were distinctly different. The R23 and R29 were intended to provide a facilitated entry into the research system by identifying a separate award for which new investigators alone could compete. Although there were no funds set aside for this award, it was assumed that identification of the applicant as a new investigator would provide an incentive for both reviewers and those responsible for funding to provide some preferential treatment for this group. The R23/R29, with its reduced funding levels compared to the average R01, was considered a "starter grant," with the ultimate goal a transition to an R01. If this mechanism has been successful, there should be evidence that receipt of the R23/R29 stimulates that transition to an R01. In contrast to the R23/R29, the K08/K11 and R03 mechanisms are not "mini-R01s" but were designed to provide either additional training or interim resources (respectively) to enable investigators to better compete for the R01. If these mechanisms have been successful, it should be possible to demonstrate that new investigators that had received such an award would then be able to compete effectively for the next step.

The tables presented below look at the successes of new applicants for each of the mechanisms in the years 1987-1989. (These are the first three years that the R29 was offered.) These examine cadres of first-time applicants in each year. The outcome is counted as successful if an award is made at any time after initial application. Since the time interval examined is not infinite, but has 1995 as the last point queried, there is some distortion due to time-censoring of the data. For example, more than 99 percent of all the awards made to the cadre of R29 applicants entering in 1989 will have been made by 1995, compared to 94 percent of the R01 applicants in that year. (This estimate is extrapolated from the data in Table 1.) Although this difference is marginal, there will be a somewhat larger effect when the successes in obtaining subsequent R01s of the awarded groups is examined. For example, about 9 percent of all R29 applicants who get an award receive it two years after initial application. For the cadre applying in 1989, a five-year award would be made to about 9 percent of the awardees in 1991, and this group would not have had time to seek competitive renewal by 1995. All of the outcomes measuring subsequent R01 awards will be somewhat undervalued because of this effect. Further, the different lengths of awards between the various mechanisms will also have a differential effect on the outcome. However, all of these conditions represent small differences, and do not affect the conclusions of the study.

R29

The R29 mechanism was initiated in 1987 to replace the R23. Both the average length of award and the total direct costs available were increased. Although, as noted previously, the total number of new applicants did not change, a higher percentage of the new applicant pool chose to submit an R29 than had previously chosen to submit an R23 (Figure 2). The appeal presumably came from the increases in award length and costs, and from the perception that for new investigators there was a higher probability of obtaining an R29 than an R01. Nevertheless, even with these additions and advantages, only about 30 percent of the new applicant pool has chosen to apply for the R29 rather than the R01.

Earlier examination (Figure 4) has shown that new investigators do about as well as experienced investigators when applying for a new research grant. However, when applicants for R29 awards are segregated from the applicants for RO1 awards, the data show (Figure 10) that R29 award applicants enjoy a significant advantage in success rates compared to applicants who also had no prior NIH experience but had applied for RO1s. R29 applicants also have an advantage compared to applicants for new grants who had a previous record of research grant support, although the overall decrease in success rates for the three groups over time remains comparable.

It is not clear why this differential exists. Some Institutes at NIH give a selective advantage in paying R29 awards compared to new investigator RO1s. Similarly, there is reason to believe that peer review groups are also more tolerant of the deficiencies of R29 applications compared to other applications. What is not clear is why there should be an advantage in applying for an R29 compared to a new investigator applying for an RO1. After all, they are both new investigators. There are several possibilities: the R29 more clearly identifies the new investigator, requires letters of recommendation, focuses less on preliminary data, and costs less than half of the average RO1. Whatever the reason, the results are clear--R29s have a substantial advantage over their competitors in initial success rate.

However, this advantage to the R29 applicant is reversed when the awardees subsequently submit a competing renewal. We matched the pools with respect to the initial peer review outcomes as illustrated previously in Figures 7 and 8. Figure 11 shows that in 1988-90 R29 awardees who score in the first decile initially did essentially as well as new investigators holding RO1 grants on competitive renewal, but grantees with R29 awards did noticeably worse than their counterparts when the lower deciles are compared.

This pattern can be seen clearly in Table 2, which shows the progression of first-time cadres of applicants in the years 1987-1989. It follows the applicant pool through the first award to a subsequent application and award of an R01. (The R29 was first introduced in 1987. Since the average R29 is awarded for 5 years, and the last data examined is that of 1995, a significant number of awards of competing renewals would not be captured if we followed R29 awards made after 1989. Although a competing renewal is not the only next step to an R01, it is the most common. Consequently, award dates after 1989 were not considered.) Although the R29 applicant is more likely to receive an initial award than the R01 applicant, poorer outcomes with subsequent R01 applications neutralize this advantage and result in a yield that is identical for the two groups of initial applicants. (This is consistent with the data shown above). Thus, in the period 1987-89, 17.7 percent of the initial R29 applicant pool eventually progressed to an R01. This compares to 17.4 percent of the initial new R01 applicant pool that subsequently were awarded an R01 and successfully recompeted to remain in the system. Tables 3 and 4 also demonstrate that this conclusion is equally true for M.D. and Ph.D. applicant pools.

K08/K11

The K08/K11 awards were targeted primarily to M.D.s to provide an opportunity for mentored research. Between 1980-1994, 8,623 first-time M.D. applicants attempted to obtain R01 support, 3,280 applied for an R23/29, and 4,279 applied for a K08/K11. (Table 5) These numbers for the K08/K11 are 26 percent of the total applicant pool for M.D.s. Over this period, the numbers of M.D. applicants for an R01 or R23/29 has changed little, if at all (Table 5), while the fraction of total R01 awards made to M.D.s or M.D./Ph.D.s has actually declined slightly (Table 6). Thus, there is no evidence that the existence of the K08/K11 mechanism has affected the number of M.D. investigators awarded independent research awards.

The success rate for K08/K11 applicants is quite high, close to 60 percent (Table 7). Furthermore, of the awardees who chose to subsequently apply for an R01 or an R23/R29 (about 65 percent of the awardees), the probability of success is also quite good, averaging 50 percent. (Of the awardees who subsequently submitted applications, about 25 percent opted for the R23/R29 award.) The net result is that about 19.3 percent of the K08/K11 applicant pool in the years 1987-1989 progressed to an R01 or R23/R29. This appears quite significant, given that the initial pool represents M.D. applicants with little prior research experience.

R03

The R03 award, although not designed exclusively for the new applicant population, was intended to provide an entry point by allowing time and opportunity for individuals to obtain preliminary information. The numbers of new investigators applying for the R03 have always been small. Between 1982-1995 there were a total of 4,096 first-time R03 applicants, compared to 33,174 R01 and 15,041 R23/29 applicants (Table 8). Looking again at the period 1987-89 for comparison, the numbers of applicants were low, the percentage receiving an award was only 31.5 percent, and of those awarded only about 55 percent subsequently submitted an application for an R01 (Table 7). Because the numbers are so small, it is difficult to evaluate whether there was any advantage gained by those who received an award and later competed for an R01. However, the net outcome is that the probability of an applicant in these years ultimately receiving an R01 was only 8.8 percent.

Summary

1. The introduction of specific mechanisms to facilitate the entry of new investigators has not increased the number of applicants.

2. Applicants awarded an R29 are less likely to proceed to a subsequent R01 than are their counterparts who initially were awarded an R01. Consequently, the higher initial award rates for the R29 are offset, resulting in no net increase in yield of applicants progressing in the research funding system compared to the R01 awardees.

3. The existence of the K08/K11 awards has not increased either the number of M.D. applicants either applying for or awarded independent research grants.

4. The pool of applicants applying for a K08/K11 award progress to a subsequent R01 with a yield of 20 percent. This appears quite significant, given that these are initially M.D. applicants with little research experience.

5. The numbers of new investigators applying for an R03 have always been small. Further, the yield from the initial applicant pool is only about 8.8 percent.

Part 3: Conclusions and Recommendations

The stresses on the NIH research funding system over the past several decades have been reflected by increasing concerns over the fate of new investigators. Given the increased competition resulting from diminishing success rates for research support it has been assumed that new investigators, with neither a track record nor existing resources to provide preliminary data, would be particularly vulnerable. There have even been claims that new investigators are submitting applications in lower numbers, as a result of discouragement from the system due to low success rates.

Careful examination of the data indicates that although new investigators have certainly suffered as resources grew more slowly, they have not been selectively disadvantaged. When both R23/29 applicant or new R01 applicant success rates are compared with new R01 applications from experienced investigators, a comparable change in success rate can be seen for all the groups. This does not mean, of course, that the level of support is appropriate. It does however suggest that no hidden biases against new investigators have emerged that require correction.

It should be noted that the decrease in success rate is not the only contributor to malaise among new investigators. An informal survey of new applicants and awardees generated a spontaneous concern that too much time was being spent writing applications. It is quite clear that the likelihood of funding an initial application has decreased even faster than the success rate, and has almost certainly contributed to the increased insecurity felt by many applicants. In 1980, only 12.6 percent of all new R01 awards were funded after amendment and resubmission. In 1995, that percentage was 42.7 percent. The changes are even larger for R23/29 awards, from 8.7 percent to 51.5 percent. This is of course not unique to new investigators, but it is an aspect of the funding system that has generated major stresses. In part, this is due to the increased time required to write and submit applications, as noted above. Additionally, the rejection of an application, even if the application is supported through a later resubmission, contributes to the perception that research support is even more restrictive than is actually the case.

It is difficult to see how to reverse this trend toward delayed support of applications since the reduction in funding levels is itself one cause of this phenomenon. As award rates drop the quality of the unfunded group improves and the likelihood of an unsuccessful applicant choosing to reapply and to be funded increases significantly. As a result, the pool of applicants never having received an award becomes much larger than the number entering for the first time in that year. For example, in 1994, the total number of applicants who had never previously received an award was 7,083, while the number applying for the first time was 3,313. This existence of a large, highly competitive pool of unfunded investigators necessarily results in a increased level of amended applications being funded.

Because of the importance of new investigators to the health and long-term viability of research, over the past two decades the NIH has initiated several mechanisms with the intent of facilitating the entry and retention of new investigators in the sponsored research system. These have met with varying degrees of success.

The K08/K11 mechanisms appear to have been successful. These mentored scientist awards are primarily targeted to M.D.s who are interested in research careers and have not had significant research training. Given the importance of attracting M.D. researchers the yield of 20 percent of the initial applicant pool progressing to an R01 seems quite significant. However, the majority of M.D.s who receive an R29 or R01 did not apply for or receive a K award. For example, of 1,025 M.D.s receiving either an R29 or R01 in 1987-89, only 178 (17 percent) had previously held a K award. Further, the number of M.D. applicants or awardees has not been increased over the period 1980-1995. This mechanism may thus be valuable to target specific populations of M.D. researchers where a specific need is seen, but does not appear to significantly contribute to the total numbers of M.D.s entering the research system. (Other K awards, described in Appendix 2, are not evaluated in this document either because they are not aimed at the new investigator or because they have been initiated too recently for data to be available.)

Recommendation

The K08 mentored scientist award should be retained, since it appears to generate a high yield of R01 awardees. However, it is unlikely to make a significant contribution to the total number of M.D. investigators receiving independent research awards.

The R03 award is not exclusively targeted at new investigators and is used for different purposes by the different Institutes and Centers at NIH. The value of this mechanism in contributing to these varied goals is not considered here. There is little evidence, however, that the R03 contributes significantly to the numbers of new investigators introduced into or retained within the NIH research system. Further, the yield of R01 awardees from the initial population of applicants is very small, less than 8 percent.

Recommendation

The R03 award should be evaluated based on its utility for specific goals of the Institutes and the NIH. It does not contribute, however, to the general goal of stimulating the entry and retention of new investigators.

The R29 award was the most difficult issue to resolve. It is clear that it has provided real benefits to both the applicants and to NIH. For the awardees it provided a greater degree of stability through a five-year period of award than was normally available to R01 awardees. Further, the probability of success was consistently higher than for new applicants for an R01. From the point of view of NIH, it was highly cost-effective, since each R29 award cost about one-half that of an award to a new R01 recipient. Indeed, one could say that the applicants for the R29 accepted the trade-off between higher success rates and lower costs, and that this was beneficial to all involved.

There are however some significant problems with the R29 award. The constraints on the size of the award have become increasingly onerous. Informal surveys of investigators who have currently or had previously obtained R29 awards confirm that in most cases these awards had to be supplemented by outside support, either from the home institution or through successful competition for other private or public funds. When queried, many of these awardees felt that the level of funding was too low for successful completion of a research project, and that the level of support should be raised even at the expense of fewer awards being made. Additionally, it is hard to justify making an award at roughly one-half the level provided to their counterparts and expecting equivalent productivity, when great efforts are made to provide as close to full funding as possible to more established investigators. It seems curious that we accept so easily much smaller awards to new investigators who are likely to be less well equipped to deal with such constraints. Finally, the requirement for 50 percent effort has made the R29 an unacceptable choice for many who need to recover their salary based on the percent effort requested. The limitations on direct cost for these investigators coupled to the 50 percent effort results in little funds being left available for conducting the research. However, if the level of direct cost support for the R29 were to be increased, as has been suggested, the distinction between the R29 and the R01 would begin to disappear.

Perhaps even more important, R29 awardees are consistently less successful on recompetition than are new investigators holding an R01. Analysis shows that although the most highly rated R29s, by peer review, do as well as the comparable new R01 awardees, the R29 awardees falling in the second and third deciles of peer reviewed applications are significantly less successful in subsequently receiving R01s. If the goal of NIH is to place resources where the yield, in terms of future success, is greatest, than we should be giving a selective advantage to new R01 applicants over R29 applicants. Exactly the opposite is true, resulting from a combination of preferential treatment by peer review groups and funding decisions by the Institutes. If this were to be reversed, the primary benefit of the R29 would disappear.

In summary, the NIH has created a mechanism, the R29, which has resulted in the creation of two populations of new investigator applicants. One of these, the R01 applicant pool, has a higher probability of future success than the other. However, a selective advantage in initial award rates is provided to the less successful group, the R29 applicants, while simultaneously giving them significantly less resources (which may contribute to their lack of success in later grant submissions). As a result, the trade-off of higher initial success rates for lower costs appears to the Working Group to be in the end not justifiable, as is the creation of two separate pools of new investigator applicants.

Recommendation

The R29 mechanism should be abolished and replaced by an R01 application that clearly identifies on the face page (or in some other standard manner) that the applicant is a new investigator, i.e. someone who has not previously had research grant support. The default award should be for five years, and the peer review groups should be instructed to require less preliminary data than would normally be expected of an established investigator. The amount awarded should be whatever is deemed appropriate by peer review and Institute judgments.

Given that the average cost of an award to a new investigator will inevitably increase with the abolition of the R29 award, there is clearly a concern that success rates will drop. It is essential that NIH define for itself what the entry rate for new investigators should be, based on an assessment of the needs of the research system. There has always been an assumption that new blood is required to keep biomedical research flexible and innovative, and as productive in the 21st century as it is in the concluding decade of the 20th century.

Over the years, there has been a continued decrease in the entry rate of new investigators, defined as a percentage of the total principal investigator pool (Figure 1). There is no obvious way to determine what is the optimum level of new investigators entering the research system. However, a minimum level can be defined. This is equal to the percentage of the total R01 principal investigator pool that leaves the system. This number, defined as the "retirement rate," reflects those investigators that leave the research support system (R01 or R29 support) and do not reappear. Since it is not uncommon for investigators to lose their grant support for some period and reenter the system later, this is a number that is dependent on the time interval examined. Examination shows that investigators who lose their awards and do not reenter within four years have markedly diminished probability of subsequently reappearing on the NIH rosters (Figure 12). Since the latest date examined was 1995, retirement rates determined for the period up to 1991 are unlikely to show significant instability on later examination. For the period 1980-1991, the retirement rate has remained remarkably constant at an average near 9 percent per year (Figure 13). The entry rate for new investigators should be maintained at a level no less than this. In 1995, for example, with 15,607 funded investigators, this translates to 1,405 awards to new applicants. (Rather remarkably, the actual number of R01 and R29 awards to new investigators in 1995 was actually 1,403.) In 1995 this would result in a success rate of 22 percent, compared with the actual rates of 18 percent for new R01 applicants and 28 percent for R29 applicants.

Resources will have to be found to maintain this award level at the expected higher average costs necessitated by conversion of R29 awards to R01s. Given the average total costs for an R01 to a new investigator and the costs for an R29, this would have amounted in 1995 to an additional $55 million.

An entry rate for new investigators that equals the retirement rate only reflects the minimum number of new entrants required to maintain a steady-state population of investigators. If there is real growth in the NIH budget, judgments must be made as to how to distribute these between growth in the total pool of investigators, (i.e. an increase over replacement rate), and growth in the opportunities available to funded investigators, (i.e. multiple awards or growth in budgets above inflation).

Recommendation

The Working Group recommends that the number of awards to new investigators be made at levels no less than the numbers required to replace investigators leaving the research system. This has historically been at about 9 percent per year. Increases beyond the replacement rate will depend on real growth in the NIH budget and judgments about the appropriate distribution of such growth.

Finally, it has become clear to the Working Group, from interviews and other reports, that the NIH communicates poorly the content and purpose of its programs to new investigators. This is due in part to the multiplicity of mechanisms, the different uses to which they are put in the various Institutes and Centers of NIH, and the unnecessarily obtuse bureaucratic language in which they are described. It is essential that this be corrected, and that new approaches be developed to help communicate clearly to the new investigator, who is feeling sufficient stress without having to wade through the morass of NIH official language.

Appendix 1

List of Members of the Working Group on New Investigators

Co-Chair
Marvin Cassman, Ph.D.
Director
National Institute of
General Medical Sciences
NIH

Co-Chair
Ellie Ehrenfeld, Ph.D.
Director
Division of Research Grants
NIH

Ronald G. Geller, Ph.D.
Director
Division of Extramural Affairs
National Heart, Lung, and Blood Institute
NIH

Stephen I. Katz, M.D., Ph.D.
Director
National Institute of Arthritis and
Musculoskeletal and Skin Diseases
NIH

John H. Krystal, M.D.
Associate Professor of Psychiatry
Director, Division of Cognitive and
Clinical Neuroscience
Department of Psychiatry
Yale University School of Medicine

Alan I. Leshner, Ph.D.
Director
National Institute on Drug Abuse
NIH

Trina Schroer, Ph.D.
Associate Professor of Biology
Department of Biology
Johns Hopkins University

Harold C. Slavkin, D.D.S.
Director
National Institute of Dental Research
NIH

Appendix 2

Description of Programs Discussed in the Report

Research Projects

Research Career Programs

Over the last 25 years the NIH and the Institutes have used a large number of career development grant (K series) mechanisms. Many of these K mechanisms duplicated or closely emulated the purpose of other K mechanisms and in some cases, several distinctly different career development mechanisms were grouped together under one K series program. For the purposes of the New Investigator Working Group, two K series programs were selected for analyses because of their overall uniformity as a mentored award, length of use, and consideration as a mechanism of support for new physician investigators prior to the submission of a research grant application.

Appendix 3

Tabular Data Associated With Figures in Text