Peer Review Process
Peer Review News
Enhancing Peer Review at NIH: On February 28, 2008, the Final Draft of the NIH 2007-2008 Peer Review Self-Study was submitted to Dr. Elias Zerhouni, Director of NIH, marking the end of the diagnostic phase of the peer review enhancement effort.
Side-by-Side Comparison of Enhanced and Former Review Criteria
- (MS Word - 49 KB)
NOT-OD-09-023: Enhancing Peer Review: The NIH Announces Updated Implementation Timeline
NOT-OD-09-024: Enhancing Peer Review: Enhancing Peer Review: The NIH Announces New Scoring Procedures for Evaluation of Research Applications Received for Potential FY2010 Funding (http://grants.nih.gov/grants/guide/notice-files/not-od-09-024.html)
NOT-09-025: Enhancing Peer Review: The NIH Announces Enhanced Review Criteria for Evaluation of Research Applications Received for Potential FY2010 Funding (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-025.html)
NOT-OD-09-033: The NIH Implements New Registration Process for Reviewer Reimbursement for Participation in NIH Peer Review Meetings (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-033.html)
Registration Instructions to Receive Reimbursement and Honoraria for Participation in NIH Peer Review -
(12/23/2008) - (MS Word - 1.8 mb)
NIH policy is intended to ensure that grant applications submitted to the NIH are evaluated on the basis of a process that is fair, equitable, timely, and conducted in a manner free of bias. The NIH dual peer review system is mandated by statute in accordance with section 492 of the Public Health Service Act and federal regulations governing "Scientific Peer Review of Research Grant Applications and Research and Development Contract Proposals" (42 CFR Part 52h-).
The first level of review is carried out by a Scientific Review Group (SRG) composed primarily of non-federal scientists who have expertise in relevant scientific disciplines and current research areas. The second level of review is performed by Institute and Center (IC) National Advisory Councils or Boards. Councils are composed of both scientific and lay members chosen for their expertise, interest, or activity in matters related to health and disease. Only applications that are favorably recommended by both the SRG and the Advisory Council may be recommended for funding.
Back to Top
Depending on the grant assignment, initial peer review meetings are administered by either the Center for Scientific Review (CSR) or an individual NIH IC. Peer review meetings are announced in the Federal Register. The meetings are closed to the public, although some meetings may have an open session; the Federal Register provides the details of each meeting.
A. Peer Review Roles
Scientific Review Officer:
Each SRG is led by a Scientific Review Officer [(SRO), formerly Scientific Review Administrator (SRA)]. The SRO is an extramural staff scientist and the Designated Federal Official responsible for ensuring that each application receives an objective and fair initial peer review, and that all applicable laws, regulations, and policies are followed.
- Analyze the content of each application, and check for completeness.
- Document and manage conflicts of interest.
- Recruit qualified reviewers based on scientific and technical qualifications specifically related to each grant application, including:
- Authority in their scientific field (42 CFR 52h.4)
- Dedication to high quality, fair, and objective reviews
- Ability to work collegially in a group setting
- Experience in research grant review
- Assign applications to reviewers for critique preparation.
- Attend and oversee administrative and regulatory aspects of peer review meetings.
- Prepare summary statements for all applications reviewed.
NOTE: SRG rosters are posted on the NIH website thirty days in advance of each meeting.
- Serves as moderator of the discussion of scientific and technical merit of the applications under review.
- Is also a peer reviewer for the meeting.
- Receive copies of the grant applications approximately six weeks prior to the peer review meeting.
- Prepare a written critique for each application assigned per the SRO, based on review criteria and judgment of merit.
- Make recommendations concerning the scientific and technical merit of applications under review, in the form of final written comments and numerical scores.
- Make recommendations concerning appropriateness of budget requests.
- Make recommendations concerning protections for human subjects; inclusion of women, minorities, and children in clinical research; welfare of vertebrate animals; and other areas as applicable for the application.
Other NIH Staff
- Federal officials who have need-to-know or pertinent related responsibilities are permitted to attend closed review meetings.
- NIH IC or other federal staff members wishing to attend an SRG meeting must have advance approval from the responsible SRO. These individuals may provide programmatic or grants management input at the SRO's discretion.
B. Initial Peer Review Meeting
- Most SRGs convene for one-two days.
- Applications are reviewed based on established review criteria (see below).
- Assigned reviewers present their prepared critiques to the group.
- An open discussion follows.
- Final scoring is conducted by private ballot.
The initial scientific peer review of most research applications also will include a process in which only those applications deemed by the reviewers to have the highest scientific merit, generally the top half of the applications under review, will be discussed at the SRG meeting, assigned an overall impact score, and receive a second level review. Applications in the lower half are not discussed or scored at the SRG meeting. This process allows the reviewers to focus their discussion on the most meritorious applications.
Summary statements for streamlined applications contain the written critiques submitted by the assigned reviewers but do not contain a resume and summary of discussion. Streamlined applications are not barred from potential funding and may be revised and re-submitted.
NOTE: December 2, 2008 - See Notice (NOT-OD-09-025) Enhanced review criteria (below) have been issued for the evaluation of research applications received for potential FY2010 funding and thereafter. Research applications received for potential FY2009 funding will be evaluated according to criteria adopted on October 12, 2004 (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-05-002.html) and modified May 11, 2006 (see http://grants.nih.gov/grants/guide/notice-files/NOT-OD-06-069.html).
Side-by-Side Comparison of Enhanced and Former Review Criteria (12/01/2008) - (MS Word - 49 KB)
Current Review Criteria (research applications received for potential FY2009 funding)
NIH applications are evaluated using established criteria (42 CFR 52h). Specific initiatives or programs may indicate review criteria in addition to the following required criteria:
Significance: Does this study address an important problem? If the aims of the application are achieved, how will this advance scientific knowledge? What will be the effect of this study on the concepts or methods that drive this field?
Approach: Are the conceptual framework, design, methods, and analyses adequately developed, well integrated, and appropriate to the aims of the project? Does the applicant acknowledge potential problem areas and consider alternative tactics? For applications designating multiple Project Directors/Principal Investigators (PDs/PIs), is the leadership approach, including the designated roles and responsibilities, governance and organizational structure consistent with and justified by the aims of the project and the expertise of each of the PDs/PIs?
Innovation: Does the project employ novel concepts, approaches or methods? Are the aims original and innovative? Does the project challenge existing paradigms or develop new methodologies or technologies?
Investigators: Are the PD/PI(s) and other key personnel appropriately trained and well suited to carry out this work? Is the work proposed appropriate to the experience level of the PD/PI(s) and other researchers? Do the PD/PI(s) and investigative team bring complementary and integrated expertise to the project (if applicable)?
Environment: Do(es) the scientific environment(s) in which the work will be conducted contribute to the probability of success? Does the proposed study benefit from unique features of the scientific environment or subject populations, of employ useful collaborative arrangements? Is there evidence of institutional support?
Additional criteria. In addition to the above criteria, the following items will be considered in the determination of scientific merit and the priority score:
- Recombinant DNA research
- Protection of human subjects from research risks
- Inclusion of women, minorities, and children
- Vertebrate animal research
- Select agents
Additional considerations. The following considerations do not contribute to the priority score:
- Resource sharing *(May be included as a review criterion that may affect the score for certain programs and initiatives.)
Enhanced Review Criteria (research applications received for potential FY2010 funding)
The mission of the NIH is to support science in pursuit of knowledge about the biology and behavior of living systems and to apply that knowledge to extend healthy life and reduce the burdens of illness and disability. As part of this mission, applications submitted to the NIH for grants or cooperative agreements to support biomedical and behavioral research are evaluated for scientific and technical merit through the NIH peer review system.
Overall Impact. Reviewers will provide an overall impact score to reflect their assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the following five core review criteria, and additional review criteria (as applicable for the project proposed).
Core Review Criteria. Reviewers will consider each of the five review criteria below in the determination of scientific and technical merit, and give a separate score for each. An application does not need to be strong in all categories to be judged likely to have major scientific impact. For example, a project that by its nature is not innovative may be essential to advance a field.
Significance. Does the project address an important problem or a critical barrier to progress in the field? If the aims of the project are achieved, how will scientific knowledge, technical capability, and/or clinical practice be improved? How will successful completion of the aims change the concepts, methods, technologies, treatments, services, or preventative interventions that drive this field?
Investigator(s). Are the PD/PIs, collaborators, and other researchers well suited to the project? If Early Stage Investigators or New Investigators, do they have appropriate experience and training? If established, have they demonstrated an ongoing record of accomplishments that have advanced their field(s)? If the project is collaborative or multi-PD/PI, do the investigators have complementary and integrated expertise; are their leadership approach, governance and organizational structure appropriate for the project?
Innovation. Does the application challenge and seek to shift current research or clinical practice paradigms by utilizing novel theoretical concepts, approaches or methodologies, instrumentation, or interventions? Are the concepts, approaches or methodologies, instrumentation, or interventions novel to one field of research or novel in a broad sense? Is a refinement, improvement, or new application of theoretical concepts, approaches or methodologies, instrumentation, or interventions proposed?
Approach. Are the overall strategy, methodology, and analyses well-reasoned and appropriate to accomplish the specific aims of the project? Are potential problems, alternative strategies, and benchmarks for success presented? If the project is in the early stages of development, will the strategy establish feasibility and will particularly risky aspects be managed?
If the project involves clinical research, are the plans for 1) protection of human subjects from research risks, and 2) inclusion of minorities and members of both sexes/genders, as well as the inclusion of children, justified in terms of the scientific goals and research strategy proposed?
Environment. Will the scientific environment in which the work will be done contribute to the probability of success? Are the institutional support, equipment and other physical resources available to the investigators adequate for the project proposed? Will the project benefit from unique features of the scientific environment, subject populations, or collaborative arrangements?
Additional Review Criteria. As applicable for the project proposed, reviewers will consider the following additional items in the determination of scientific and technical merit, but will not give separate scores for these items.
Protections for Human Subjects
Inclusion of Women, Minorities, and Children
Additional Review Considerations. As applicable for the project proposed, reviewers will address each of the following items, but will not give scores for these items and should not consider them in providing an overall impact score.
Budget and Period Support
Select Agent Research
Applications from Foreign Organizations
Resource Sharing Plans
NOTE: The NIH will implement the new scoring system described below, effective for all applications for research grants and cooperative agreements submitted for funding consideration for FY2010 and thereafter. See NOT-OD--09-024 (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-09-024.html)
New Scoring System (for applications submitted for potential FY2010 funding)
Before the SRG meeting, each reviewer and discussant assigned to an application will give a separate score for each of five core review criteria (Significance, Investigator(s), Innovation, Approach, and Environment; see above). For all applications, even those not discussed by the full committee, the scores of the assigned reviewers and discussant(s) for these criteria will be reported individually on the summary statement.
Before the review meeting, each reviewer and discussant assigned to an application will give a preliminary impact score for that application. The preliminary impact scores will be used to determine which applications will be discussed. For each application that is discussed, a final impact score will be given by each eligible committee member (without conflicts of interest). Each member's impaft score will reflect his/her evaluation of the overall impact that the project is likely to have on the research field(s) involved, rather than a weighted average applied to the reviewer's scores given to each criterion.
The new scoring system will utilize a 9-point rating scale (1 = exceptional; 9 = poor). The overall impact score for each discussed application will be determined by calculating the mean score from all the eligible members' impact scores, and multiplying the average by 10; the overall impact score will be reported on the summary statement. Thus, the overall impact scores will range from 10 (high impact) to 90 (low impact). (Overall impact scores will not be reported for applications that are not discussed.)
Current Scoring System (for applications submitted for FY2009 funding)
Priority scores reflect the relative strengths and weaknesses of an application, with the lowest scores indicating the highest level of merit:
- 100-150: Outstanding
- 151-200: Excellent
- 210-250: Very Good
- 251-350: Good
- 351-500: Acceptable
Applicants should contact the program official for the application to seek additional feedback on the score and summary statement.
Rarely, an application will receive an 'NR' score, indicating that it is not recommended for further consideration because it lacks significant scientific merit, or because it presents serious research risks and protections against risks are inadequate. An application with an 'NR' cannot be moved to the second level of review and should not be resubmitted until the problems are resolved.
D. Summary Statement
Within one or two months of the SRG meeting, a summary statement will be available to the Principal Investigator via his/her NIH Commons account. The summary statement contains information about the application's review including:
- Contact information for the Program Officer handling the application
- Overall impact score or priority score (depending on the fiscal year)
- Percentile (if applicable)
- Resume and summary of the discussion (only for applications that are discussed)
- Reviewer critiques and individual criterion scores (if under the new scoring system)
- Committee recommendations concerning the budget
- Human subject and vertebrate animal concerns (if applicable)
- Additional administrative comments (if applicable)
- Official meeting roster
Understanding the Percentile
The overall impact score or priority score is used to determine an application's rank relative to other applications reviewed by the same SRG. This is referred to as the percentile ranking of an application. Some application types are not given percentile rankings.
Back to Top
Second Level Of Review - Advisory Council or Board
Who Reviews the Application?
The Advisory Council/Board of the potential awarding IC performs the second level of review. Advisory Councils/Boards are composed of scientists from the extramural research community and public representatives ( NIH Federal Advisory Committee Information ). Members are chosen by the respective IC and are approved by the Department of Health and Human Services.
- NIH program staff members examine applications, their overall impact scores or priority scores, percentile rankings and their summary statements and consider these against the IC's needs.
- Program staff provide a grant-funding plan to the Advisory Board/Council.
- The Advisory Board/Council also considers the IC’s goals and needs and advises the IC director.
- The IC director makes final funding decisions based on staff and Advisory Council/Board advice.
Back to Top
- Not Funded – What Next?
The NIH receives thousands of applications for each application receipt round. Funding on the first attempt is difficult, but not impossible. If an application does not result in funding, NIH has resources available to help applicants prepare a possible application revision and resubmission. Applications in response to a specific initiative with set aside money typically cannot be resubmitted, but you the program officer should be consulted about next steps.
- Fundable Score – What Next?
If an application results in an award, the applicant will be working closely with the IC program officer on scientific and programmatic matters and a grants management officer on budgetary or administrative issues.