In January 2004, the Center for Scientific Review Advisory Committee (CSRAC) recommended that the CSR Director implement the proposed guidelines for the last of the Integrated Review Groups (IRGs) that will be created in the Panel on Scientific Boundaries for Review reorganization. Although much work has gone into planning for reorganization, successful implementation is the real challenge. While CSR is working to implement the new IRGs and study sections, we are also developing a systematic and continuous evaluation process for reorganized IRGs and study sections. We would like to assure the scientific community that we recognize the importance of examining the review process and will take additional steps where necessary to create the best possible review environment.

Ensuring Quality

In September 1998, the CSRAC recommended the formation of ad hoc Working Groups to provide advice on the organization, management, and leadership of the IRGs and their component study sections. The goal was to ensure that all applications assigned to the IRGs consistently receive high quality reviews. The Working Groups were to include active, widely respected researchers in disciplines related to those reviewed by the IRGs. Over the subsequent years, the first complete round of Working Groups was completed. A total of nineteen Working Groups were constituted, conducted assessments of their assigned IRGs, and reported their findings to the CSR Advisory Committee.

A concern expressed by several Working Groups was that unless measures are taken during the reorganization process to ensure quality reviews, the success of the huge reorganization would be in doubt. The objective of ensuring fair and quality reviews of all applications, regardless of study section, is an important one that CSR will carefully monitor. The reassignment and recruitment of reviewers necessary for implementation of the new study sections will be done to ensure the presence of high quality reviewers on all review committees.

Role of Reviewers

Several Working Groups expressed interest in populating all study sections with a diverse mixture of reviewers, including "senior reviewers" (i.e., very experienced scientists with prior study section experience). CSR is piloting new types of study section membership for "senior reviewers". In the end, fair and expert review comes down to the reviewers, their breadth of expertise, and their willingness to embrace the new structure of study sections.

Future Assessments

The boundaries among study sections and IRGs created in the reorganization are intended to be dynamic and will change to keep pace with the ever-changing landscape of science. Inherent in this type of dynamic structure is a need to assess whether reviews are appropriate and high quality. CSR is committed to assessments of study sections at regular intervals by distinguished site visitors using the Working Group process that we have developed (described above) and by “customer” surveys (targeting program staff and applicants). CSR gained valuable experience performing review structure evaluation from its comprehensive evaluation of its reorganized peer review of the neurosciences (see http://www.csr.nih.gov/events/Neuroscience_Evaluation_7_03_03.pdf). As we plan additional assessments, CSR will refine its tools and methods to yield the most useful information about the functioning of our peer review committees.

Your Role

CSR is grateful for the tremendous interest and support the scientific community has demonstrated throughout our reorganization. Approximately 400 scientific community members lent their expertise on study section design teams and we carefully considered 1,800 of your comments when developing implementation recommendations. While CSR is exerting substantial effort to establish the best review organization possible, we continue to rely upon your participation and input.

While the reorganization process is now in the implementation phase, we still require input and assistance from the scientific community. For example, CSR understands that you may well be the best judge of the most appropriate review locus for your application, and therefore we encourage suggestions of appropriate study sections. When submitting an application, please include a cover letter indicating your preference for a particular study section, or the expertise necessary to review your application.

CSR also requires participation of the scientific community on its peer review panels--these panels can only be as good as the reviewers who will agree to serve on them. Especially important is the participation of leading scientists willing to serve as reviewers on study sections and as participants on our Working Groups. If you are interested in participating in any of these activities, please contact Terra Vinson at vinsont@csr.nih.gov.

CSR is committed to building a progressive and responsive peer review process that will identify the best science for NIH to consider for funding. This is CSR's contribution to the larger NIH goal of understanding and curing human health problems. We look forward to hearing from you.