Dr. Theodore Kotchen, Special Advisor on Clinical Research for the Center for Scientific Review (CSR) and his colleagues at the National Institutes of Health (NIH), have completed new studies of data on the peer review of clinical research at NIH. An article describing this work—“Outcomes of NIH Peer Review of Clinical Grant Applications”—appears in the January 2006 issue of the Journal of Investigative Medicine.

These studies follow up earlier CSR studies which showed that NIH peer review outcomes in 2002 were slightly but significantly less favorable for grant applications for clinical research than for laboratory research. ("NIH Peer Review of Grant Applications for Clinical Research," Journal of the American Medical Association, February 18, 2004).

Studies of Three Factors that Might Affect Clinical Research Reviews

The latest studies assess the possible influence of three factors that could contribute to this difference: (1) the percent of clinical applications assigned for review to a study section, (2) the greater cost of clinical research, and (3) the clinical research experience of the reviewers.

None of the data collected and analyzed support the hypothesis that the less favorable review outcomes for clinical applications are related to these review factors:

·         In contrast to 1994, in 2004, clinical applications reviewed in study sections reviewing relatively low percentages of clinical applications  were not found to be any more disadvantaged than those reviewed in greater numbers by other study sections.

·         The requested direct costs were greater for clinical than for non-clinical R01 applications, but those requesting greater funds in both categories received more favorable scores.

·         Priority scores assigned by reviewers with or without clinical research experience were not found to differ. 

Dr. Kotchen and his colleagues suggested that peer review outcomes for clinical research will benefit from the recent refinement of NIH review criteria that emphasize the unique contributions of clinical investigation, and from increased training opportunities for clinical investigators.

A related “commentary” in the same issue of the Journal of Investigative Medicine (Byrns PJ, Orringer EP.  Is the crisis in clinical research being effectively addressed?) suggests “a number of additional factors that may contribute to the less favorable outcomes of clinical applications:  e.g., the experience and competence of the principal investigators; the number of human subject concerns identified by the reviewers; the number of new versus revised submissions contained within the group; and perhaps even the complexity and limitations of conducting clinical research.”

Ongoing Efforts

Dr. Kotchen began his work with CSR in August 2002. Since that time, he has been involved in a number of activities, including gathering and analyzing data concerning the peer review of clinical research. He has been assisted in these efforts by colleagues at NIH: Teresa Lindquist, M.S., Program Analyst, CSR; Anita Miller Sostek, Ph.D., Director, CSR Division of Clinical and Population-Based Studies; Raymond Hoffmann, Ph.D., Professor of Biostatistics, Medical College of Wisconsin; Brent Stanfield, Acting CSR Director;  Ellie Ehrenfeld, Ph.D., former CSR Director; and Karl Malik, Ph.D., Director, CSR Office of Planning, Evaluation and Analysis.

CSR will continue to study and monitor clinical review outcomes to assess factors that may affect them; however, CSR has taken steps to address concerns related to both the preparation and review of clinical research applications:

·         A strong statement of commitment was made when CSR appointed Dr. Kotchen to be its Special Advisor on Clinical Research. He is also Professor of Medicine and Epidemiology, and Associate Dean for Clinical Research at the Medical College of Wisconsin. At CSR, he has worked to advance activities to ensure that clinical applications are reviewed appropriately. In this capacity, he served as the lead author of the JAMA article. In addition, Dr. Kotchen has worked as CSR's liaison with the external clinical research community.

·         A systematic reorganization of CSR's review groups has been completed, with guidance from external experts, including representatives from the clinical research community. One of the major goals of this reorganization is to ensure that clinical applications are clustered so that they make up 25-30% of the applications in the study sections where they are reviewed.

o        Increased numbers of clinical investigators will be recruited to fully implement CSR's reorganization.

o        Additional clinical study sections and clusters of clinical applications will be created as they are needed.

·         Supplementary guidelines for clinical research applicants and reviewers were developed and posted on CSR's Web site to remind them of the factors that make a strong clinical research application.

·         Clinical professional societies have been invited to submit names of qualified clinical investigators for consideration as possible members of CSR study sections.

·         New flexible terms of service are being explored to allow more clinical investigators to be reviewers.

·         The priority scores CSR's study sections give to clinical and laboratory-oriented research applications are tracked and compared on an ongoing basis.

·         Additional studies of clinical review outcomes are ongoing and planned to better assess how well clinical applications fare in NIH peer review and to identify and address factors that may adversely effect the preparation or review of clinical applications. 

For additional information, please contact Don Luckett, CSR’s Communications Director, at 301 435-1111 or luckettd@nih.gov.

The Center for Scientific Review organizes the peer review groups that evaluate the majority of grant applications submitted to the National Institutes of Health. We also receive all NIH and many Public Health Service grant applications and assign them to the appropriate NIH Institutes and Centers and PHS agencies. Our primary goal is to see that NIH applications receive fair, independent, expert, and timely reviews that are free from inappropriate influences so NIH can fund the most promising research. Additional information on CSR is available on our Web site (http://www.csr.nih.gov) or by calling 301 435-1111.

####