On 14 January, the National Science Foundation (NSF) revised its Grant Proposal Guide to reflect the changing realities of what counts as scientific bona fides, and also to help rectify long-standing confusion over its "broader impacts" review criteria. (A summary of these changes can be found here, while the complete, updated guide is here.) Probably the most significant change is in the wording in the biosketch section, which now will direct proposers to list five research "products" directly related to the proposal, and up to five additional products, in place of a list of publications. "It's just a recognition of a broadening of what could be put into the biographical sketch," NSF senior policy specialist Beth Strausser tells Science Careers in an interview. "Products of research are not just publications."
The proposal guide's new language states that "[a]cceptable products must be citable and accessible including but not limited to publications, data sets, software, patents, and copyrights." Explicitly excluded are "unpublished documents not yet submitted for publication, invited lectures, and additional lists of products."
NSF's list is suggestive, but it leaves much to the imagination. So we asked Strausser for some clarification—and learned that NSF does have a pretty specific idea of what it's looking for. NSF grant applicants, Strausser says, should list only "formal products of research." If it's been published somewhere, it needs to be something "formally vetted by an editorial board or something of that nature."
We further proposed a few specific examples, asking Strausser whether they should be included:
• A blog post outlining the details of one's study? No.
• Even a really detailed, scholarly one? Still no.
• A scholarly communiqué with colleagues via an online forum? No again.
Of course, data sets and software programs—both listed as acceptable products—may not be vetted in the traditional way, so some ambiguity still surrounds the definition of what NSF deems acceptable under the new classification. Maybe NSF will release a clarification soon.
Project summary, three ways
Another major change is the modification of the project summary text box in NSF's online submission system, FastLane. Applicants previously entered, in a single, page-length text box, a brief overview of their proposal, its "intellectual merit" (which the guide describes as its likelihood to “[a]dvance knowledge and understanding within its own field or across different fields”) and its "broader impacts"—a nebulous constellation of considerations that include promoting teaching, training, and learning; increasing participation of women and underrepresented minorities in science; and benefiting society in other ways.
In the new system, that single text box has been replaced by three text boxes: the first for entering an overview, the second for describing the project's intellectual merit, and the third for citing broader impacts. The overall length of the project summary remains unchanged: Applicants still have 4600 characters, but now they must divvy those characters up among the three boxes. None may be left blank. The intent is to spotlight the broader impacts criterion, making it impossible for applicants to bury or ignore.
Need to enter equations, symbols, or other information that a text box won't accommodate? Don't worry: If a summary includes equations, applicants may upload an attachment instead of filling the text boxes—but the summary should still be formatted into three separate sections.
A related change requires that, in the "Results from Prior NSF Support" section, researchers should list intellectual merit and broader impact results under separate headings. NSF is also ditching its list of broader impacts examples because, NSF says, some scientists interpreted the list as all-inclusive and prescriptive rather than as a list of examples as NSF intended.
Merit review revisited
Although the National Science Board (NSB) Task Force on Merit Review—the committee that proposed the new guide's revisions—toyed with the idea of altering NSF's two merit review criteria (intellectual merit and broader impacts), they ended up keeping them intact. They did, however, expand the explanations of and motivations for the criteria to give applicants a better understanding of what NSF reviewers and grant-review panels are—or should be—looking for. New language in the proposal guide exhorts applicants to:
• Do research of the highest quality and ensure that it has "the potential to advance, if not transform, the frontiers of knowledge."
• Consider that NSF explicitly wants to fund projects that contribute to the betterment of society, either directly through the proposed project's aims or indirectly through complementary activities.
• Consider that NSF will assess the success of any project in relation to the scale of the resources devoted to it, including the size of the grant. If a project is smaller in scope, NSF will view it as one of many such projects and assess the impact cumulatively.
Over the years, the relative importance of broader impacts has waxed and waned as NSF directors have come and gone. The current thinking at NSF, Strausser says, is that broader impacts and intellectual merit "really are on equal footing" and that "neither by itself is sufficient." Evidence of this can be found in the new, separate text box for broader impacts in the project summary, she says, as well as its emphasis in the new merit review principles.
Interestingly, comments by the NSB task force subtly contradict the claim that broader impacts and intellectual merit are held in equal esteem. The report notes that: "All stakeholders gave more weight to the Intellectual Merit review criterion than to the Broader Impacts review criterion, including NSF program directors and division directors." Still, "NSF staff felt that reviewers should be giving more consideration to the Broader Impacts than they are currently doing."
The little stuff
A number of smaller changes and clarifications have also been made to the guide:
• The instructions indicate that principal investigators (PIs) should be included in the budget section under "Personnel" only if they'll be getting paid from the grant.
• The "Facilities, Equipment and Other Resources" section has been overhauled to make it clearer to reviewers what resources, including physical resources and personnel, the PI can draw on. Resource costs and dates of acquisition should not be listed here.
• Proposals that involve the usage of wild vertebrate animals must now comply with handling and welfare guidelines provided by the American Society of Mammalogists.
