best practices in broader impacts
Top-down approaches
- The US National Science Foundation & Broader Impacts
- Merit Review Broader Impacts Criterion: Representative Activities
- Response to Recommendations from the Committee of Visitors for the Division of Biological Infrastructure
- NSF's Broader Impacts
- Capturing the Outcomes and Impacts of Publicly Funded Research
- Example of Broader Impact Highlight from Chemistry
- Measuring Success in Relation to Federal Investments in University Research: AUCC Discussion Paper
- "NSF's Broader Impacts Criterion"
- Broader Impacts Showcase
Fall 2005 a poster session at the ACS Fall 2005 National Meeting & Exposition, Washington, D.C. Put on by chemists at the NSF Mathematical and Physical Sciences Advisory Committee (MPSAC) in an attempt to provide more education related to the broader impacts criterion.
- Response to the Report on the Committee of Visitors on Information Technology Research (2001-2003)
2005 This is a response from the Office of Integrative Activities to a Committee of Visitors (COV) report. COV’s are asked to evaluate and comment upon NSF program performance and results. Note particularly that the COV suggested that there might be confusion about what “broader impacts” mean and how they could be evaluated in review. To this the response was, “The National Science Board and NSF management purposefully articulated the ‘broader impacts’ criterion to be broad. This criterion can be addressed in many ways (e.g., through education, training, societal impacts) depending on the scope of proposed projects, recognizing the range of project types supported by the agency. This flexibility—in appropriately aligning science and engineering research and education with its broader impacts—is important in encouraging innovation from all science and engineering communities.”
- Grant Proposal Guide
- Important Notice No. 127
- NSF's "An Impressive Return on Investment"
- Working Group on Implementation on Review Criterion #2
- Division of Materials Research Dear Colleague Letter on the Broader Impacts Review Criterion
- Division of Chemistry Dear Colleague Letter on the Broader Impacts Review Criterion
- NSF Call for Highlights of Best Practices
- "Broader Impacts Review Criterion"
March 2009 A presentation by Kathie Olsen, Jean Feldman, Pat Tsuchitani, and Ann Carlson of NSF given at the March 2009 workshop "Research Funding and the Good Life." Provides a general overview of the peer review process at NSF.
July 2007 In this NSF publication entitled Merit Review Broader Impacts Criterion: Representative Activities, there is an attempt to list appropriate practices/activities for fulfilling BIC. Following each of the components of BIC, examples of specific activities that would fulfill that component are given. The authors warn, however, that “this list is not intended to be exhaustive, nor is any particular example relevant to all proposals.” These are provided more as samples, and the authors encourage new and creative ways to fulfill the criterion.
June 2007 The first recommendation from the committee of visitors pertains to the broader impacts criterion; the committee advises continued education of the scientific community on the meaning and importance of the criterion. Recommendation 1 (c) recommends the establishment of a website where examples of best practices could be posted. The response from the Biological Sciences Directorate (BIO) and the Division of Biological Infrastructure (DBI) ensures that this will be done.
February 2007 NSF's Nancy Pelaez's workshop powerpoint presentation on the broader impacts criterion.
2007 A call from the Social Sciences and Humanities Research Council of Canada for a kind of metrics to measure the "social, cultural and economic impacts of [publicly funded] research."
2006-2007 In this “nugget” from Chemistry (a nugget is essentially a summary of a highlight, often with visual support) examples are given representing “technical highlights,” “non-technical highlights,” and “broader impacts highlights.” The broader impact highlight described is a professor who contributed lectures and experiments to the California State Summer School for Mathematics and Science at the University of California, Irvine. The broader impact lies in the fact that, “These lectures introduced high school students to the interdisciplinary area of the physical chemistry of macromolecules.” And furthermore that, “Impact of her work is further broadened through international collaborations in Italy and the Netherlands.”
November 2006 Out of Canada, this report is from the Association of Universities and Colleges of Canada. Some indicators they use include--First Order Indicators: funding, enrollment in graduate studies and international enrollment in graduate studies, attraction and retention of researchers, co-authorship, private sector investments in university research, international research funding; Second Order Indicators: Graduate degrees awarded, publications and citations, commercialization, cross-sectoral partnerships, economic impact.
June 2006 A PowerPoint presentation on the broader impacts criterion, delivered by two members of NSF at the 2006 Annual ASEE Conference.
September 2004 In the grant proposal guide, NSF provides information on what they will consider regarding broader impacts when evaluating proposals in determining funding decisions.
July 2002 A notice from Director Rita R. Colwell on the Implementation of new Grant Proposal Guide Requirements Related to the Broader Impacts Criterion.
2001 As part of their strategic plan, on page 14, NSF lists as a performance goal to continue to stress the importance of both criteria, and claims it will "collect and make available examples of broader impacts and develop a plan to disseminate them."
February 2001 Office of Polar Programs (OPP). Provides "specific examples of activities appropriate for review under Criterion #2." They note however, that these are not meant to be exhaustive or directive, but rather, will hopefully inspire creativity on the part of the scientific community to think of new ways to address the criterion.
NSF Dear Colleagues Letters
2008 Dear Colleague letter from the director of the Division of Materials Research attempting to clarify the meaning of the broader impacts criterion and how it factors into funding decisions.
April 2008 A Dear Colleague letter from the Division of Chemistry stating that a simple listing of outreach activites, or reference to inclusion of research personnel who are members of underrepresented groups "falls short of the rigor required to satisfactorily address [the broader impacts criterion]." It further calls attention to the need to address broader impacts in all proposals in the project summary, project description, and the results of prior support sections.
January 2008 A Dear Colleague Letter from the Division of Chemistry requesting highlights that will advance the mission of the division. More than just research accomplishments, a request for information about broader impacts is specifically requested.
2007 Dear Colleague Letter from the Division of Materials Research reiterating the existence and importance of the broader impacts criterion. Examples of appropriate activities to satisfy the criteiron are provided.
Bottom-Up Approaches
- ScienceOnline2010: The Importance of Blogging
Interview with Amy Freitag, a PhD student at Duke Marine Lab, who talks about the influence of broader impacts in her research and the import of having a blog up, running, and respected when the time comes to write broader impacts statements. - COSEE Costal Trends -- For Scientists: Creating Broader Impacts
- UC Davis Sports Biomechanics Lab
- COSEE NOW Blog
- Broader Impacts Toolbox
- Advising Researchers on Broader Impacts: Approaches at UW-Madison
April 2009 Jane Harris Cramer, Associate Director of the Center for Biology Education, delivered this lecture at the University of North Texas on her work at UW-Madison addressing the broader impacts criterion.
- Stanford University Office of Science Outreach
The Office of Science Outreach (OSO) assists Stanford faculty in incorporating outreach ideas into their research proposals. They provide a "Menu of Outreach Ideas for Faculty" to help researchers fulfill the broader impacts criterion.
- "So you want a piece of the NSF pie? Broader Impacts."
December 2008 NSF Funded Researcher's Blog entry on broader impacts.
- Proceedings of the Conference on K-12 Outreach from University Science Departments: 2008
2008 A conference hosted by The Science House, North Carolina State University.
- "Making the Broader Impacts in K-12 Science Education"
David Haase, Emeritus Director of The Science House at North Carolina State University speaks on fulfilling broader impacts through improving K-12 science education.
- Integrating Broader Impacts into your Research Proposal
A presentation from a 2008 workshop from the Delta Program in Research, Teaching, and Learning.
- From bench to blackboard
- Characterizing Broader Impacts: An Exploratory Study of Metrics and Measures for the Research on Gender in Science and Engineering Program
- Building Capacity to Achieve Broader Impact through the Center for Integration of Research, Teaching, and Learning
- NSF Broader Impact: A Lever for Institutional Change
- Broader Impacts and the Museum
- "Constructing a Broader and More Inclusive Value System in Science"
- The Broader Impacts Toolbox Workshop: Helping Researchers Effectively Meet the National Science Foundation Broader Impacts Criterion
2006 A presenentation by Diandra L. Leslie-Pelecky, part of the Broader Impacts Toolbox. She helps provide information on the broader impacts criterion, how to address it, and provides some case studies.
- Communicating Science Broadly
- Broader Impacts: What, Why, and How
- PSA Education Workshop Report: Algae and the Broader Impacts of Science
- Center for the Integration of Research, Teaching, and Learning (CIRTL) Appendices for Annual Report (April 2005 - March 2006)
- Broader Impacts Posters
- Making a Broader Impact: Geoscience Education, Public Outreach, and Criterion 2 Report
- Using a team approach that provides direct involvement in the planning and implementation of the EPO activity by all constituents: the researcher, the educator, and members of the intended audience.
- Aligning EPO activities with the needs of the audience, e.g., if the target audience is formal K-12 education, activities should reference relevant science standards.
- Involving graduate students and young scientists to promote an awareness of the importance of EPO efforts by scientists and the need for culture change.
- Providing a menu of opportunities for researcher involvement, recognizing that not all scientists or research programs can or should be engaged in EPO at the same level or in the same role.
- A venue and opportunity for dialogue between scientists and educators to focus on the benefits of bridging the two cultures of science research and education
- Professional development opportunities for both scientists and educators to gain fluency and confidence in areas of mutual interest and professional intersection.
- Appropriate personnel and financial support to complete the proposed activities, including strong project management.
- Institutional commitment and incentives, including the development of a system of rewards for advancement and tenure of science faculty that recognizes the importance of public communication of science.
- Realistic consideration of sustainability and continuity in activities. Ongoing evaluation coupled with feedback to all partners in a timely fashion.
- Broader Impact: What Busy Scientists Need to Know I
- Broader Impact: What Busy Scientists Need to Know II
- Teaching Biocomplexity in the Geosciences
- Much Ado About Criterion Two: Does Anyone Care?
- Integrating Research and Education
- Merit Review Broader Impacts Criterion
- "Meet Your NSF Broader Impacts Goals"
University of California, Irvine's discussions on how to meet the broader impacts criterion.
- Broader Impacts
- "Ethics & Values of Data: Coping with Complexity & Uncertainty"
A presentation by Joan E. Sieber addressing the complexities of data organization and the broader social implications of how data is organized and shared.
Acknowledging that while many scientists are adept at communicating their research to their peers, this research often does not fully connect with public understanding of scientific concepts. To enhance the broader impacts of scientific research COSEE introduces and explains the Scientist-Educator Partnership program and how it works.
A site dedicated to providing resources to assist graduate students and postdocs in addressing the broader impacts criterion of grant proposals, and the broader impacts of their research more generally.
One of the bloggers from this site has set out to foster a discussion on writing effective Broader Impacts statements, as well as collecting examples of good projects.
The result of an NSF funded workshop that set out to discuss the broader impacts criterion and the community response to it. This website aims to provide the tools to help researchers address the broader impacts criterion. BIT collects examples of successful case studies, as well as other ideas and resources on how to address and implement broader impacts of research.
October 2007 An article from the Yale School of Medicine describing the broader impacts criterion and Yale's response of creating a dedicated position to support education and outreach efforts.
August 2007-June 2008 A summary of a study that claims to take "a first step toward characterizing measures of the broader impacts for the Research on Gender in Science and Engineering Program (GSE) at the National Science Foundation (NSF)."
April 2007 A paper by Robert Mathieu, Christine Pfund, and Don Gillian-Daniel, delivered at the 2007 R & T Colluquium on international policies and practices for acadmeic inquiry.
April 2007 A presentation by Robert Mathieu and Christine Pfund at the 2007 R & T Colloquium on international policies and academic inquiry.
A presentation by Bruce MacFadden about the broader impacts activities he engages in as curator of the Florida Museum of Natural History. A related article by MacFadden can also be found here.
January 2007 A BioScience journal article on broader impacts related issues such as diversity, and the science/society relationship, arguing ultimately for a broader value system in science.
November 2006 A PEW/INTERNET & AMERICAN LIFE PROJECT. This study funded by NSF exmamines how science is being communicated through the media while specifically treating the broader impacts criterion's question "Will the results be disseminated broadly to enhance scientific and technological understanding?"
October 2006 A brief description of the broader impacts criterion and issues surrounding it. This comes from Richard E. Ladner, University of Washington, SIGACT Chair, and member of NSF Committee on Equal Opportunity in Science and Engineering (CEOSE).
July 2006 Phycological Society of America (PSA) Education Workshop Report: Algae and the Broader Impacts of Science - Juneau, Alaska by Gisèle Muller-Parker and Rich Triemer lists “four areas of Broader Impacts activities”:
a. Broader Impacts in General
b. Teacher Professional Development
c. Public Outreach
d. Outreach to members of underrepresented groups
Aside from activities and practices, they also emphasize the role their website can play in serving as a “portal for the education and outreach activities undertaken by members of the society and as a means to bring ‘algae to the forefront’ of peoples minds.”
May 2006 Appendix H (pages 178-186) address broader impacts. "Achieving Broader Impact: The Delta Program's Approach at UW-Madison to Effective and Sustainable Broader Impact" -- Christine Pfund, Don Gillian-Daniel, and Robert Mathieu.
August 2005 An interesting collection of poster presentations on broader impacts presented at the 230th ACS National Meeting. The posters are organized according to components of the broader impacts criterion--Advancing Discovery and Understanding While Promoting Teaching, Training, and Learning; Benefits to Society; Broad Dissemination to Enhance Scientific and Technological Understanding; Broadening Participation of Underrepresented Groups; Enhance Infrastructure for Research and Education.
May 2005 Making a Broader Impact: Geoscience Education, Public Outreach, and Criterion 2 Report. Report of a workshop held at the University of California Museum of Paleontology, Berkeley California. May 11-13, 2005. There is a section on best practices where they list the following:
They add that important structural components include:
Recommendations for next steps are also provided.
December 2004 Abstracts from an Ocean Sciences session of AGU's 2004 Fall meeting addressing the broader impacts criterion.
December 2004 Abstracts from an Ocean Sciences session of AGU's 2004 Fall meeting addressing the broader impacts criterion.
April 2003 This website hosts resources from a 2003 workshop of the same name. There are tips for educators on how to communicate and teach the emerging field of geoscience, as well as other resources on how to develop sucessful partnerships and conscious attempts to address the broader impacts criterion.
Spring 2002 A list of presenters and a panel abstract are provided about a session with panelists from NSF, CGD/NCAR, and CIRES. The panel was held to address the history and current status of the braoder impacts criterion in the review and funding decision processes.
A site on integrating research and education. They have a section on their site devoted to broader impacts, and provide examples of "successful partnerships, as well as tips on assessment and dissemination."
Hosted by Texas A&M University's Office of Proposal Development, these pages contain general information, guidance, examples and resources dealing with the broader impacts criterion.
Maintained by the Science House, a resource to North Carolina State University, this site provides information on their 2005 workshop addresseing broader impacts, as well as resources on addressing the broader impacts criterion.