NIH director Dr. Elias Zerhouni addresses ACD meeting on Dec. 7, 2007. At left is NIH deputy director Dr. Raynard Kington.

The ACD heard the interim report of a working group on peer review, cochaired by NIDCR director Dr. Lawrence Tabak and Dr. Keith Yamamoto of the University of California, San Francisco. Since last July, the group has gotten more than 2,600 responses, held two teleconferences with medical school deans that attracted more than 100 participants, conducted five regional meetings and consulted a host of national advisory councils, scientific liaisons and NIH staff.

“We’re sort of at the end of the beginning,” said Tabak. The data-collection phase is largely over and the diagnostic phase has begun, with pilot programs forecast for February 2008.

Zerhouni’s charge to the group has been to assure that NIH “funds the best science, by the best scientists, with the least administrative burden.” But, as various ACD members pointed out, who gets to define “best?”

And while almost everyone agrees that easing the amount of time NIH’s volunteer reviewers (there were 18,000 of them in 2007) spend poring over paperwork, there is a danger to doing everything online, too. As Dr. Annelise Barron of Stanford University pointed out, face-to-face peer review discussions heighten the keenness of scientific debate and have the virtue of assuring that participants “know their stuff.”

Speaking out on various topics at the Dec. 7 meeting of the advisory committee to the NIH director were (from l) Dr. Thomas Kelly of the Sloan-Kettering Institute, Dr. David Botstein of Princeton University and Dr. Annelise Barron of Stanford University.

Virtually all ACD members were impressed by the depth of the Tabak-Yamamoto analysis, particularly in such a short period. The work group “reaffirms and reemphasizes the core values of peer review,” Yamamoto said, focusing on such solutions as adopting an “editorial board model” for initial review of applications, limiting proposals to 7 pages and emphasizing economy, explicitness and impact. “We want to draw the best people back into the enterprise, so they can feel their service has an impact.”

Tabak ventured into the myriad criticisms of peer review, including charges that there are too many applications in the system, too many funding mechanisms, too much time spent writing applications, too much emphasis on methods and preliminary data, but not enough on impact and innovation.

JAMA editor-in-chief Dr. Catherine DeAngelis (r) listens to discussion, along with Dr. Barbara L. Wolfe of the University of Wisconsin and Dr. Ralph I. Horwitz of Stanford University.

But Princeton University’s Dr. David Botstein warned that even the seemingly laudable goal of focusing on innovative or transformative science might ultimately be unproductive. He argued that there is an important role for merely “very essential” science that may not shift paradigms but is nonetheless indispensable.

NIGMS director Dr. Jeremy Berg pointed out that many “traffic jams” in peer review are the result of nitpicking amendments to applications that really deserve the more honest evaluation, “Your baby happens to be ugly.”

Zerhouni acknowledged that peer review “is one of the most challenging tasks you have…it involves a very complex set of inputs. But any organization, at the end of the day, must understand how its resources are allocated.”

He conceded that issues of control, filtering and priority-setting come into play, and that the culture of peer review must change, especially with respect to supporting new investigators. But piecemeal solutions will fail, he warned. “You cannot reach [a height of] 100 feet with 10 separate 10-foot ladders.”

As with the peer-review discussion, most ACD members were impressed with Zerhouni’s slide show on the gradual graying of the grantee workforce. “It’s really dramatic—the rapidity with which the NIH-supported workforce is aging,” said Dr. Thomas Kelly, director of the

Sloan-Kettering Institute. Added Dr. Catherine DeAngelis, editor- in-chief of the Journal of the American Medical Association, “It’s not fair for the old to feed on the young.”

UCSF’s Dr. Keith Yamamoto headed the extramural side of the working group on peer review.

Zerhouni’s analysis—compiled with input from the Office of Extramural Research, the budget office and the advice of “top-line demographers and actuaries”—surveyed the period 1980 to the present, and even looked out to 2020, when, if current trends prevail, there will be far more grantees age 65 and older than in 1980, when the median age for grantees was 36.

The inherent lengthening of the career path for scientists is due to many complex factors, Zerhouni explained, but there is no denying an explosion in applications for grants: There were about 19,600 in 1998, and the success rate was 31 percent. In 2007, the applications jumped to 33,800, but the success rate plunged to 21 percent.

The director called for “an adaptive strategy for tough times.” His goal is to stabilize the number of new and competing grants at about 10,000 per year, strengthen support for at-risk populations (new investigators, first-grant renewals and well-established investigators with little or no additional support, who would be eligible for so-called Bridge grants) and offer no inflationary adjustments for noncompeting renewal awards in FY 2008.

Zerhouni says 25 years of preparation and training are lost every time NIH loses a grantee and emphasized, “You don’t want to cut the gas while the plane is taking off.”