As Chair of a study section, you play a key role in its function and success. The leadership that you and the Scientific Review Officer (SRO) provide is essential to ensuring that the highest quality science reviewed by your Study Section receives the best scores. The Study Section should operate in a fashion that yields high quality and fair reviews.

To help you in this endeavor, the guidelines below were developed by several former study section chairs, in collaborations with SROs in the Center for Scientific Review (CSR). These are not "rules of conduct" but rather suggestions and guidelines that you can incorporate in a way that best fits your own style and the needs of your study section.

Chairing the meeting

Preparations before the meeting. Many chairs find it helps them maintain the proper focus, balance and fairness of discussions if they become familiar with most or all of the applications under review. To compensate for the time this involves, you may ask the SRO to assign fewer applications to you for explicit review. However, it is important that you continue to be an assigned reviewer of proposals, since it enables you to establish a role model for other panel members.

Soon after proposals have been sent to reviewers, the SRO will check to make sure that reviewers are comfortable with their assignments. It is important that panel members be encouraged to call the SRO, if they have reservations about their expertise for any of their assigned applications. If possible, members should call soon after receiving the applications, but calling even a few days before the meeting is preferable to arriving at the meeting with a misassigned application.

Begin with synopses. At the meeting, the review of an application should start with each assigned Reviewer and Discussant briefly indicating their overall level of enthusiasm for the application. This synopsis can include a rough numerical score or range of scores. If score spreading needs to be encouraged, one strategy is to have each reviewer indicate how the application compared with the others they evaluated. This helps to reveal discrepancies between the score and the rating of the application and/or over-use of scores in the upper (better) range.

Spend time wisely. Meetings should move at a pace that ensures reviews that are fair and of high quality. Discussions of reviews should focus on key issues and on the applications whose fate is least clear.

1. Discourage panel members from reading reviews in their entirety. Rather, encourage reviewers to state those issues that most determine their level of enthusiasm. This helps to keep the whole committee engaged and the discussion focused.
2. Invest time where it is most needed. Spend the most time on applications where there is greatest disagreement, especially if the application is likely to be in the best 20 to 30% of those reviewed. Less time should be spent on those applications where there is uniform high or low enthusiasm.
3. Be aware that the discussion of the first few applications on the agenda often uses a disproportionate amount of time. Allowing extra time for these applications is frequently a necessary part of establishing the committee's process, especially for the new and temporary members participating. Otherwise, the chair needs to make sure that the discussion moves along.
4. Remember that discussion does not always lead to consensus. It is essential that all major issues are aired, and the reasons for differences of opinion are clear to all. However, once this has been accomplished and further progress toward consensus is not being made, you should terminate the discussion, recapitulate the key arguments, have the reviewers and discussants state their "post discussion" enthusiasm and then ask panel members to vote as they see fit.

Applications in which discussion does not lead to consensus need special attention. Remember that the only substantial information conveyed to an applicant (other than score) are the written critiques of the reviewers and a summary of the discussion, prepared by the SRO. Applicants need to receive as clear a picture of their application's assessment as possible. In that spirit, remind reviewers that they can modify their reviews in light of the discussion, if they wish. In addition, you or the SRO may ask discussants who raised particularly important points to write a brief comment. The SRO's written evaluation of the discussion is particularly critical. If the SRO asks you to review a summary, be willing to do so.

Promote balanced discussion.

1. Help the panel concentrate on the most important issues. Reign in wandering discussions. Interrupt if necessary.

2. Don't allow one person to monopolize the discussion. Create an open atmosphere and encourage reticent reviewers to speak up.

Be a guardian of fairness.

1. Watch for evidence that a reviewer may be influenced by inappropriate personal interests (competition, scientific bias, personal antagonism etc.). If you sense that this might be happening during the review, determine a diplomatic way to handle it, such as inviting the opinion of other members of the committee. After the meeting, speak in private to the SRO, if you have serious concerns about the fairness of any of the reviews.

2. Beware of your own biases: Although you shouldn't hesitate to state your scientific opinion when appropriate, be cognizant of your role as chair, and don't champion your favorite areas of science over others.

Promote consistent scoring. To maintain fairness and provide the best input to the Institutes for funding decisions, the score range used should be broad. This should be 1.0-3.0 if 50% of the applications are streamlined, and even broader if less than 50% are streamlined. Maintaining a consistent spread from the beginning to the end of the meeting, from meeting to meeting, and from reviewer to reviewer, requires constant diligence by both the Chair and the SRO.

Set an example by your own scoring behavior, and feel free to challenge a reviewer who appears to be causing grade inflation. In addition, remind committee members that if they plan to vote outside the range of scores discussed by the reviewers, they should make sure that their opinion has been clearly stated before the committee.

Ensure that criteria-based scoring is used properly. During the discussion, each criterion should be explicitly assessed and the reviewers should state how they impacted the overall evaluation. Although it is up to the reviewers to determine the appropriate weight of each criterion in determining the score, serious consideration should be given to each criterion.

Speak up. Evaluate whether the discussion corresponds to the score assigned. If not, say so; very commonly, other panel members will agree with you but have remained silent.

Clarify differences in the review of different categories of applications. Different criteria are used in evaluating various categories of applications (e.g., post-doctoral fellowships vs. RO1 applications). As you begin to evaluate a new category of application, the SRO will explain the process and criteria to be used. Remind the reviewers of these criteria, if you find them straying, as they present their reviews.

Make sure other issues are discussed. Make sure that other important issues are adequately aired. These include: comments or concerns about vertebrate animals, human's subjects or biohazards; the inclusion of both genders, minorities and children; and the unique resources of the foreign sites for foreign applications.

Budget: Make sure that budget issues are properly addressed at the end of the discussion, after the final score recommendations are made. Work as a team with the SRO to make sure that there is not inappropriate consideration of the budget in the scoring decisions.

Take an active interest in the composition of the panel

The single most important determinant of a Study Section's excellence and credibility are its members. Please work with the SRO to identify and attract outstanding scientists to your Study Section. The ultimate responsibility for identifying and inviting the best possible panel members rests with the SRO. However, the chair should play an active role in this process. It is essential that the members of the panel be well respected in the scientific community. This ensures the credibility of the evaluations, and makes it easier to recruit other members of comparable reputation.

The criteria to be used in selecting panel members are:

1. scientific excellence (as demonstrated by grant and publication record);
2. respect in the scientific communit
3. breadth of expertise
4. fairness and evenhandedness in review
5. willingness to do the work required
6. clarity of presentations and quality of participation at meetings

Be aware that the SRO also needs to consider:

1. that the membership covers the full range of expertise needed
2. diversity in gender, ethnicity and geographic distribution

Helping identify and assess candidate reviewers. By suggesting names for potential panel members and providing feedback about temporary members who are invited, the chair can help ensure that the panel has all the important areas of expertise covered and that the best possible reviewers are brought on to the panel. In addition, be available to offer advice and input about individuals that the SRO has identified from other sources.

Senior scientists, with high stature in the community and past experience with peer review, can be especially valuable contributors, and should be seriously considered when appropriate. They can help provide balance and insights from their experience with other panels. Such reviewers can rotate among panels (perhaps participating in one panel a year).

Making initial contacts with candidates. Some prospective members find excuses more difficult when a colleague who already is committing time and effort to peer review approaches them for service on the study section. When you and the SRO agree that it may be helpful, be willing to make an informal contact with a prospective member.

Help train panel members

Work with the SRO to provide constructive comments to the regular members of the panel, particularly those who are there for only the first or second time. Be sensitive to the difficulties that being a first-time reviewer presents and provide constructive guidance and encouragement.

Involve the next chair whenever possible

Once the potential next chair has been identified, encourage his/her participation. This would include chairing when you are in conflict, and providing feedback and suggestions about reviewers. However, take care not to make public statements to the members of the study section or others about the likely future chair. No choice is official until approved by the NIH administration.

Welcome evaluations of performance, including your own

Panel members should be explicitly invited to contact the SRO or you with confidential comments regarding each study section meeting. Those comments may include evaluation of your own performance, or suggestions for ways you or the SRO might improve Study Section function. You may want to send an email to panel members after a meeting, inviting their suggestions and input. Welcome these comments and take them seriously. The SRO and Chair may choose to have a meeting annually, prior to the formal study section session, to stimulate a wide-ranging discussion of Study Section performance.

After each meeting, speak with the SRO to discuss the performance of the panel and its individual members. In particular, consider whether particular temporary members at the meeting would make good permanent members of the panel.