February 20, 2003, Summary
Peer Review of Stem Cell Science Working Group
Members Present
James Battey (NIDCD, Chair), Robert Hammond (NIDDK), Della Hann (OER), James Kushner (University of Utah School of Medicine), Brent Stanfield (CSR).
Other Attendees
Arlene Chiu (NINDS), Greg Downing (NCI), Charles Goldthwaite (science writer), John Thomas (NHLBI), Anne White-Olson (NIDCD), Baldwin Wong (NIDCD).
Introduction and Charge to the Group
Dr. James Battey welcomed participants and thanked them for their time and expertise. He noted that this meeting was called to address the peer review process, in particular as it applies to grants involving human embryonic stem cell (hESC) research. He asked participants to reflect on available data as well as their collective wisdom to analyze, discuss, and provide recommendations to optimize peer review for stem cell research. Noting that peer review provides a unique and successful approach to evaluate research proposals, Dr. Battey posed the following questions to the group:
Is the current grant review process satisfactory with regard to applications that involve hESC research?
How can the review process be improved?
Before beginning general discussions, Dr. Battey noted that NIH has funded five of eleven R01 applications (46%) that involve hESC research since May 2002. Five Study Sections within the NIH Center for Scientific Review (CSR) reviewed the majority of these applications: Hematology (HEM-1), Brain Disorders and Clinical Neuroscience (BDCN-2); Metabolism (MET); Cell Development and Function (CDF-5); and Molecular, Developmental and Cellular Neurosciences (MDCN-6). Dr. Battey also reviewed statistics on concomitant funding of R43, R21, R24, T15, and R15 mechanisms.
Participants then engaged in an open discussion of relevant topics, from which the main recommendations of the group were distilled. Highlights of the discussion are captured below, followed by specific recommendations and their related details.
Open Discussion
One participant noted that the hESC research field is relatively young; once the infrastructure has been established, the R01 grant will become the most common avenue for funded research. Thus, discussion about the future of hESC research should center on R01 grant mechanisms and how this research will fit into disease- or organ-specific study sections.
How should hESC research be funded? It was noted that, although R01 applications are able to provide only limited preliminary data at present, there appears to be no reviewer bias against funding of hESC-related proposals. In the present time, some hESC research grants have been funded by "high program priority" or "special emphasis" programs that certain Institutes have available for special research.
Discussants agreed that two issues must be addressed regarding the review process:
Reviewers are accustomed to seeing preliminary data, but we must now ask them to focus primarily on feasibility and future potential. This reflects a tension between basic science and clinical application; the proposed hESC research is basic, yet its promised value is clinical.
Reviewers may perceive hESC experiments as variations on a theme, arguing that the only unique aspect of hESC experiments are the cells themselves and that the experiments can be (or, in some instances, have already been) carried out using stem cells from other species. It was noted, however, that development of comparative tools is necessary for the field and must be carried out early to provide the groundwork necessary for future experiments.
One participant commented that NIH institutes have some flexibility with regard to the development of tools and other fundamentally descriptive research necessary for hESC endeavors. Research and development contracts may be used to fund specific, well-defined tasks.
One discussant noted that there may be misconceptions about hESC research by investigators who view it as little more than the human analog of experiments already conducted in other species (e.g., mouse). However, it was also noted that the field is in flux; even central concepts such as "plasticity" move in and out of vogue in applications over a several year period.
One participant inquired whether a specialized study section was needed for review of hESC-related research proposals. Is sufficient scientific expertise available to populate such sections? Is the expertise of current reviewers sufficient to provide excellent review of hESC research? Would the members of a hESC study section be reviewers for each other's applications due to the relatively small number of eligible participants? It was noted that current reviewers may possess "stem cell expertise," although this is not necessarily synonymous with "hESC expertise" because the hESC field is relatively small. To define a basic population, it was suggested that a list of investigators currently funded for hESC research (through investigator-initiated grants and administrative supplements; not infrastructure-based awards) could be compiled and provided. However, it was observed that a trade-off exists to such a scenario: a group of experts in hESC research may not possess expertise in other key areas (e.g., mouse ES cells, neuronal differentiation, etc.) that will inform the review process and the selection of the most promising applications.
An alternate approach would be to create a Special Emphasis Panel (SEP) consisting of approximately a dozen investigators who could be used as needed to evaluate specific proposals. It was argued that an SEP could provide some continuity while avoiding conflicts of interest. All multi-institute/center announcements regarding hESC research could be the responsibility of the SEP. It was noted that funds are available for hESC initiatives that could be used to solicit applications that would be referred to the SEP.
The NIH currently has several existing study sections with experience in evaluating hESC research proposals, and this critical mass of experts could be the basis of the SEP. Alternatively, applications that involve hESC research could be steered toward these study groups.
It was also suggested that a "temporary" SEP could be assembled for a two- to three-year period with the hope that this time frame would be sufficient to allow the field to mature. However, it was noted that long-term segregation of hESC research from the scientific mainstream would impact the field negatively. Any separation between hESC research and other categories of NIH-funded research must strike a balance between providing an opportunity for the gathering of preliminary data and promoting the long-term robustness of the discipline. Protracted segregation of hESC research may ultimately lower the quality of stem cell research, as opportunists will eventually compete for what is perceived as "easier" funding.
It was noted that few of the hESC applications reflect the efforts of first-time R01 applicants; most R01 applicants who propose hESC research have previous experience with animal SC research. New investigators have been known to be reluctant to attempt an hESC-related grant application on their first R01 submission.
One participant commented that the hESC expertise at the NIH is currently concentrated within a few institutes. Moreover, there is heterogeneity in the approaches of the various institutes to hESC research, especially at the second level of review. Even though many awardees of administrative supplement applications have provided promising feedback, there is still a two- to three-year period before the data from these supplements will be sufficient for incorporation into an R01 application.
Due to substantial differences in the budgets and mechanisms available to the various NIH Institutes and Centers, assignment of a grant application to multiple institutes was proposed. In this way, an application that fails to meet the payline of one institute may be selected by another institute for funding, although it was noted that this strategy happens rarely in practice. Also, options are available to allow additions to existing applications; perhaps these options should be publicized through a program announcement. Moreover, some institutes have discretionary funds that can be used to fund meritorious applications that are outside of the payline. Some institutes have caps on the amount of funds available for competitive renewals, and it was suggested that such caps should be removed in the case of hESC research to promote the field's continued growth.
Dr. Battey then inquired whether the NIH provides sufficient guidance to applicants whose research incorporates hESCs and to reviewers on study sections that review hESC grant applications. Respondents noted that there are currently no detailed instructions for hESC grant applicants. It was suggested that appropriate Scientific Review Administrators (SRAs) should be trained to understand thoroughly, considerations unique to research grants involving hESC. To assist the SRAs, CSR has developed a fact sheet to provide the reviewers some guidance when considering hESC applications.
Recommendations
The group provided three central recommendations designed to "mainstream" the NIH peer review process for hESC research. These suggestions and related discussion are summarized below:
Direct hESC applications to a limited number of existing study sections (currently around five) that possess the experience and expertise to address the quality of the proposed research.
Enhance the orientation of the SRAs and of the study section members with respect to issues unique to hESC grant applications (e.g., the state of the science and the difficulty in providing preliminary data at the time of initial application).
Solicit grant applications for descriptive, less hypothesis-driven research relevant for advancing knowledge in stem cell biology, and evaluate these applications through a Special Emphasis Panel.
Discussion
It was suggested that applicants could be encouraged to designate the appropriate study sections, while CSR would determine which reviewers have the appropriate expertise to sit on the study sections. CSR can also provide the community with a list of the standing study sections that are the most knowledgeable regarding stem cell-related research. Moreover, NIH can encourage researchers with hESC experience to join these standing study sections. These study sections will need to be trained about the rapidly changing science of hESC research; it was suggested that a "point person" on hESCs within CSR could disperse information to appropriate SRAs. The number of such study sections should be limited, and the Scientific Review Administrators who manage these sections must be acquainted with all issues associated with hESC research, including policy concerns and the potential shortage of preliminary data to support most proposals.
There was also a short discussion about the potential benefit of allowing NIH intramural scientists to participate in scientific review panels related to hESC research. Although this practice has been limited in other areas, it will be re-examined to determine whether the use of such expertise will facilitate the review process for hESC-related research proposals.
The group also discussed the option of creating a standing study section in the Center for Scientific Review dedicated to peer review of grant applications that involve hESC research. Although the establishment of such a study section was not deemed necessary at this point, the standing study section approach was favored over creation of a Special Emphasis Panel (SEP) due to the emerging nature of the hESC field. Because the field is not heavily populated, there was concern that a SEP approach might allow reviewer applications to be considered in a situation that would be viewed as a conflict in a standing study section.
The group also discussed the creation of a Special Emphasis Panel (SEP) to evaluate proposals for fundamentally descriptive, less hypothesis-driven research (e.g., interspecies comparison studies of stem cells, development of standardized techniques and reagents, etc.) that is not likely to be funded through an R01 mechanism. It was agreed that proposals for such basic research should be solicited through a specific program announcement (PA) or request for applications (RFA). However, it was noted that the locus of review (either through the CSR or an individual institute) must be carefully designed. It was suggested that once a strategy has been agreed upon, information about the review process (e.g., where to send grant application, who will review the application) must be clear and easily accessible to investigators. One participant suggested posting an online flow chart, a virtual "you-are-here" map that explains the fit and flow of the available grant mechanisms.
It was suggested that such an initiative would be trans-NIH and could be developed using a "planning grant" mechanism, similar to that currently used to encourage descriptive animal SC research. Because the nature of such research does not fit the paradigms of traditional R01 proposals, participants agreed that a pathway consisting of an R21 (exploratory/developmental grant) followed by an R33 (phase II developmental grant) could set the necessary groundwork for a successful R01 submission. The information obtained during the R21 and R33 stages would be accumulated toward an eventual R01 submission. The R21/R33 proposal would require a different infrastructure from that of the R01.
Several approaches were suggested to the address logistical issues of such a proposal. First, R21 mechanisms have fixed time and budget constraints (e.g., $275,000 allotted for direct costs over a two-year period), and it was suggested that special announcements could be made for exceptions, citing that research is experimental and exploratory.
Alternatively, the R21 could be awarded for two years, followed by the transition to an R33, which would require demonstration of certain milestones in the research. Such an approach would save the applicant approximately nine months of re-application time. Current R21/R33 combined grant programs ("Phased Innovation Awards") at the NIDDK and the NCI were cited as possible models.
One participant noted that there is a paucity of data in these basic research areas; thus, there is no convenient place for a researcher to "jump into" the field. All protocols should therefore also be coordinated with extramural initiatives. Some of this descriptive research may be performed by the private sector, and it was suggested that NIH institutes could use their Small Business Innovation Research (SBIR) funds to facilitate partnerships. Because small businesses have demonstrated an interest in stem cell research, the SBIR proposals would be solicited through "companion" initiatives to the R21/R33 proposals, perhaps employing the same review team. The two initiatives would reference each other and would be hyperlinked within the Purpose section of the two documents. To encourage the interest of the private sector, an emphasis should be placed on addressing intellectual property (IP) and financial concerns, as well as developing useful platforms for enabling technologies.
It was proposed that these initiatives be posted on the NIH Web site.
Final Remarks
Dr. Battey thanked participants for their time and enthusiasm in addressing such important issues. The meeting was then adjourned.
If you have questions about the Task Force, please contact:
Science Policy and Planning Branch
National Institute on Deafness
and Other Communication Disorders, NIH
Bethesda, MD 20892
Phone: (301) 402-2313
Fax: (301) 402-2265
E-mail: stemcell@mail.nih.gov