The CSREES Peer Review Process for Competitive Grant Applications

CSREES reviews all proposals accepted in the individual competitive programs through the peer review process. The following description of that process portrays general concepts that are shared among CSREES competitive grants programs. However, specific details on the panel meeting, review format, and evaluation criteria will vary among programs. Processes and procedures specific for the National Research Initiative (NRI) are noted.

The RFA. The review process begins with the publication of the Request for Applications (RFA) for the CSREES competitive program of interest. The RFA is published on the agency Web site and accessible through funding opportunity Web pages. The RFA can also be accessed through Grants.gov, the web site for federal government grants. Occasionally RFAs are also published in the Federal Register. The RFA includes all of the pertinent information for the current funding cycle, including program purpose, legislative mandates, award types, eligibility requirements, evaluation criteria, submission instructions, program goals and funding priorities, proposal submission deadlines, and application submission instructions. CSREES also conducts two grantsmanship workshops each year, covering the NRI Integrated, Education and other Programs, to educate applicants regarding CSREES funding opportunities and provide a forum for potential applicants to talk directly with program leaders about their proposed projects.

Often, after reading the RFA, applicants will contact the program staff to discuss the applicability of a topic to the program goals and suitability for prospective submission as a proposal.  Applicants are encouraged to submit only those proposals responsive to the funding priorities outlined in the RFA. Proposals that do not respond to priorities in the RFA are returned without review. Some individual NRI program areas also now require submission of letters of intent prior to proposal submission. For these program areas, proposals submitted without prior approval of the letter of intent by the program leader are returned without review. The letter of intent contains a descriptive title of the proposed project; names and roles of the project directors and other key personnel, along with their institutions; and a brief statement of approaches and objectives, including the program priority to which the project is responding. CSREES program staff evaluates these letters for the suitability of the project to program goals and priorities and in relation to program scope and needs. Invitations to submit a full application are then issued by the program leader for letters describing proposed projects best fitting these criteria.

Selection of a panel manager. Many CSREES competitive programs utilize a panel manager who is selected by the program leader to assist with administration of the program. For example, all the NRI programs select a panel manager. The panel manager is an active, established scientist possessing broad-based knowledge in the program area. The professional stature of the panel managers within their respective scientific communities brings additional visibility and recognition to the program.  Panel managers become part-time, temporary (1–2 years) USDA employees. Duties of the panel manager include assisting program leaders in selecting panel members and ad hoc reviewers, assigning proposals to reviewers, chairing the panel meeting, and assisting program leaders with funding decisions. Panel managers (or their family members) cannot submit an application to the panel that they head, as either project director (PD) or co-PD.

Selection of panelists and proposal review. The program leader and panel manager aim to assemble a diverse panel active in research, education, and/or extension (as appropriate for the program) related to the subject matter in question.  The goal is to create a balanced panel with the necessary expertise to cover the range of the proposals, while also maintaining diversity in geographical location, institution size and type, professional rank, gender, and ethnicity.  Programs also strive to have continuity on the panel from previous years by inviting at least 30 percent of the panelists to return for a subsequent year.  Potential panelists must be dedicated to high quality, fair reviews, and be able to devote sufficient time to the review process.  No more than one individual, including the panel manager, can serve from a single institution or, with a few exceptions, from a single state.  As with the panel manager, panelists cannot submit an application to the panel on which they’ve agreed to serve. The integrity of competitive programs depends upon the stature and qualifications of the individual panelists and the fairness and scientific skill with which they administer their scientific review  responsibilities. All these qualities are necessary for the careful review and evaluation of the submitted applications.

The program leader and panel manager study the proposals carefully and assign them for review to panel reviewers and, when additional expertise is needed, to ad hoc reviewers. Typically, three to four panelists review each proposal.  If needed for additional expertise, up to three ad hoc reviewers may also evaluate a proposal. Each panelist is assigned 12 to 20 proposals, for which they provide written reviews.  During the review panel meeting, each panelist also provides an oral evaluation of the assigned proposals.  

Reviewers prepare reviews using the evaluation criteria, published in the RFA and available on the CSREES Web site, to assess the strengths and weaknesses of each proposal.  Review criteria are specific for the CSREES competitive program. For the NRI, proposals are evaluated for scientific merit; qualifications of project personnel and adequacy of facilities; and relevance to program priorities including importance of the topic for agriculture. 

Confidentiality and conflicts of interest. Confidentiality is critical to ensuring the integrity of the peer review process.  Names of submitting institutions and individuals, as well as application content and peer evaluations, are kept confidential, except to those involved in the review process, to the extent permitted by law.  Identities of peer reviewers remain confidential throughout the entire review process, and the names of the reviewers are not released. Reviewer comments and discussion during the review panel meeting also remain confidential. This issue is emphasized repeatedly from the time a panelist is invited to serve on the panel to completion of the panel meeting. The panel manager, program leader, panelists, and program staff are permitted access to the written reviews immediately before and during the panel meeting.  Otherwise, written reviews and evaluations of each proposal are shared only with the respective applicant.

During the review process, special care is taken to avoid conflicts of interest. Individuals involved in the review process may not participate in any aspect of the proposal evaluation if they have: (1) served as an adviser or advisee to the applicant(s); (2) collaborated or served as a coauthor with the applicant(s) during the past 3 years; (3) are currently affiliated with, were previously employed within the past 12 months by, or are being considered for employment at the institution(s) of the applicant(s); (4) participated in a consulting/financial arrangement with the applicant in the past 3 years; or (5) are the spouse, child, sibling, parent, partner, or close friend, or otherwise have a relationship that might affect judgment, or could be seen as doing so by a reasonable person familiar with the relationship. These rules apply to everyone involved in the review—the program staff, the panel manager, panelists, and ad hoc reviewers. When a proposal comes up for discussion during panel, any panelist with a conflict of interest leaves the panel room and does not participate in review, discussion, or ranking of that proposal. Similarly, if the panel manager or program leader has a conflict of interest with a proposal, they do not participate in any aspect of the review for the proposal, including assigning reviewers or being present during panel discussion.

The review panel meeting.For many CSREES panels, such as the NRI, panelists often are not assigned the same set of proposals for review. At the panel meeting, panelists are seated around a single table to allow the discussion among the various panelists assigned to a proposal. This arrangement also allows the entire panel to participate in the discussion of any proposal reviewed in the panel for which they do not have a conflict of interest. Some CSREES competitive programs, such as those administered by the Higher Education unit, assign the same set of proposals to three to four panelists for review. At the panel meeting, panelists are seated at separate tables with panelists reviewing the same set of proposals assigned to the same table. In this case, the panelists discuss only their assigned proposals and come to a consensus on a final score in an intimate environment. They do not participate in discussion or scoring of other proposals reviewed at other tables in the panel meeting.

Prior to the panel meeting, the panel manager and program leader read the applications to identify special issues affecting panel discussion. Throughout the meeting, the panel manager and program leader enforce rules about conflict of interest. They ensure that panelists leave the room during review and discussion of applications submitted from their own institutions or from individuals with whom they have a conflict of interest. They also emphasize confidentiality regarding all matters concerning submission, review, recommendations, ranking, and panel composition, and that confidentiality must be maintained outside the panel meeting room and after the panel meeting as well.

The panel manager and program leader serve as chairs of the panel meeting and are responsible for assuring that every application receives a thorough and objective review. They do not provide an opinion or review of the proposal; the rating and ranking of the proposal is entirely the consensus opinion of the panel. The panel manager and program leader also ensure assure that different types of applications, such as research, integrated, and strengthening proposals, are discussed and ranked separately.

During the meeting, the panelists discuss the proposal and reviews and arrive at a consensus ranking. For the NRI, ranking categories are "outstanding," "high priority," "medium priority," "low priority," and "do not fund." Only proposals ranked in the first three categories may be considered for funding; those ranked in the latter two are ineligible for an award. Applicants should refer to the Funding Opportunity web page for programs of interest to see the previous year’s success rate and gauge the level of competition for that particular program.

Following the evaluation and initial ranking of each proposal, a "panel summary" document is written by a panel member reflecting the panel consensus. It details the salient points of the panel’s assessment of the strengths and weaknesses of the proposal.  The panel summary also contains a section with synthesis comments, describing areas, and potentially providing suggestions, for improvement.  The synthesis also provides comments either to encourage or discourage resubmission of a revised application. For panels such as those in the NRI, on the final day of panel meeting, the panelists reassess the initial rankings of the proposals and re-rank proposals, as needed, to ensure that they accurately categorize and order the proposals. 

After the completion of the panel, the program leader and panel manager use the panel ranking to determine the proposals that will be recommended for funding. The program leader and panel manager also review both the budgets of these top proposals to make sure the request is appropriate and the funding the applicant has, or may receive, from other grant agencies to ensure that the project is not already funded. Generally proposals are funded according to the panel ranking until program funds are used up.  In the NRI, lower ranked proposals that fall below this funding line may be supported with “strengthening” funds, a percentage of the NRI budget set aside to support proposals from eligible small to mid-sized institutions or those in EPSCoR states (see the RFA for definitions and eligibility requirements). Following the funding decisions, applicants in most CSREES competitive programs receive copies of the written reviews of their proposal (with reviewer name removed to maintain reviewer confidentiality), the panel summary, and information on the relative ranking of their proposal. This information is commonly sent to the applicant through e-mail correspondence.  Some CSREES programs only send applicants the panel summary through either U.S. postal service or e-mail correspondence.