Last month, Representative Lamar Smith, chairman of the House Committee on Science, Space, and Technology, introduced a divisive new bill, the High Quality Research Act, that would change the criteria by which the National Science Foundation evaluates research projects and awards funding. (The N.S.F., with a budget of seven billion dollars, funds roughly twenty per cent of federally supported basic research in American universities.) Currently, proposals are evaluated through a traditional peer-review process, in which scientists and experts with knowledge of the relevant fields evaluate the projects’ ”intellectual merits” and “broader impacts.” Peer review is a central tenet of modern academic science, and, according to critics, the new bill threatens to supersede it with politics.
John Holdren, the director of the White House Office of Science and Technology Policy, said last week that “adding Congress as reviewers is a mistake.” Representative Eddie Bernice Johnson warned more forcefully that Representative Smith was “sending a chilling message to the entire scientific community that peer review may always be trumped by political review.” But in a statement, Representative Smith said the draft bill “improves on [the peer-review process] by adding a layer of accountability.” The bill’s new three-point criteria for funding require that a project be “in the interests of the United States to advance the national health, prosperity, or welfare, and to secure the national defense”; solve “problems that are of the utmost importance to society at large”; and not be “duplicative of other research projects being funded by the Foundation or other Federal science agencies.”
Implicit in the proposal’s language is a desire for oversight built into the process of determining which areas of study are significant. (Representative Smith cited five N.S.F.-funded social science projects, with concerns as to whether they “adhere to NSF’s ‘intellectual merit’ guideline.”) To Smith’s point, despite sizable public investment in research and development (nearly $150 billion this year), relatively scant attention is devoted to investigating whether the process of science, in its current form, is well-designed for generating knowledge.
As it turns out, the N.S.F. recently awarded a grant to Kevin Zollman, an assistant professor of philosophy at Carnegie Mellon University, to investigate what drives researchers to pursue particular projects, and how they secure funding for them. In an ideal world, all science would proceed from the simple and lofty social goal of expanding human knowledge, but researchers are of course subject to the constraints of economic reality. To disentangle these knotted incentives, Zollman is applying a branch of game theory known as “mechanism design,” which uses simple models to understand how individual players within a system balance competing motivations to arrive at different ends.
Zollman’s first object of study is the “priority rule,” which states that all of the credit earned by scientific discovery—the money, the praise, the promotion and adulation—is directed to the first researcher across the finish line, whether by weeks, days, or hours. The priority rule was first elaborated in the nineteen-fifties by the sociologist Robert K. Merton, who observed that “In the institution of science, originality is at a premium.” He described its role over centuries of scientific progress, with fierce and stubborn quarrels punctuating the time line of discovery: Newton and Leibniz fought over the invention of calculus, Galileo and Simon Mayr over the discovery of Jupiter’s moons.
As researchers scramble to carve out intellectual real estate, however, overly aggressive competition and singular focus on originality can elicit a host of negative behaviors: bias toward reporting positive or rushed results, withholding or fabricating data, and counterproductive levels of output. Of fifty million scholarly articles published since 1665, more than half of them appeared in the last twenty-five years. While a number of factors contribute to this glut, one of them is the pressure to publish results even when they’re not immediately relevant. “Much of the recent scientific literature is repetitive, unimportant, poorly conceived or executed, and oversold; perhaps deservingly, much of it is ignored,” wrote the microbiologist Ferric Fang in a 2012 editorial. The problem is becoming particularly acute as a growing pool of scientists face a shrinking pot of money, both from drying stimulus funds scheduled to disappear this September and 2.9-percent budget cuts forced on the N.S.F. through sequestration. The currently overcharged pull of priority, by spurring unnecessary questions and hairs to be split, can thus dilute or stifle the advancement of science.
In light of these problems, Representative Smith proposed his bill. Yet the economic straits in science already encourage research that targets specific, funder-driven priorities over riskier, more open-ended questions. As the Nobel laureate Roger Kornberg lamented in 2007, “If the work that you propose to do isn’t virtually certain of success, then it won’t be funded.” Elevated political scrutiny would likely only reduce the willingness of agencies like the N.S.F. to fund projects without clearly defined, or even expected, outcomes. In tension with this reality is the fact that revolution in science is often indebted to prolonged exploration of basic research projects, which may at their outset fail to meet Representative Smith’s criteria. Forecasting how well research will “advance the national health, prosperity, or welfare,” or predetermining the degree to which a project is or is not “groundbreaking,” demands an unlikely prescience. Further embedding specific requirements in grant allocation could make improvements at the margins by defunding egregiously conspicuous research. But it also threatens to close off a large landscape of research questions with unforeseen potential.
Zollman is looking to tweak the internal reward system of science to promote better outcomes. He notes that the priority rule is generally effective “because it encourages scientists to diversify among different projects” and pursue novel approaches. The work of determining whether these projects or approaches have merit is then left to experts like those at the N.S.F. But understanding conditions under which the benefit of priority breaks down—when, for instance, are scientists pushed to pursue personal over social gain, through excessive duplication or shoddy science?—is essential to a more efficient institution.
“There aren’t many alternatives to the way credit is given out,” Zollman said. “I’m hoping to come up with some so there can be a real discussion about all the ways that we might assign credit, and why it is that the priority rule may or may not be best.” This insight could lighten the pressure on scientists to be recognized as “the first,” a demand that can sometimes bend the long trajectory of inquiry toward shorter-term incentives. In the end, like Smith, Zollman seeks a better science. It is perhaps no small irony, then, that his application would have likely been rejected under Smith’s proposed criteria.
Dylan Walsh is a freelance science journalist and an editor at The Solutions Journal.
Photograph, of Representative Lamar Smith, by Bill Clark/Roll Call/Getty.
