![[logo] US EPA](https://webarchive.library.unt.edu/eot2008/20081109122904im_/http://www.epa.gov/epafiles/images/logo_epaseal.gif){kind=logo}
Basic Information
Topics
Research Performance
To ensure EPA’s Office of Research and Development (ORD) remains a high-performing organization with demonstrated results that contribute to EPA’s mission, ORD measures and manages performance at the organizational level as well as for each research program. Research accountability efforts are implemented in accordance with the Government Performance and Results Act of 1993 (GPRA), the President's Management Agenda (PDF) (64 pp, 635 KB, About PDF) -- including the Research and Development Investment Criteria, and the Office of Management and Budget's (OMB’s) Program Assessment Rating Tool.
EPA‘s Board of Scientific Counselors (BOSC), an independent body comprised of expert scientists, conducts regular peer reviews of ORD research. The BOSC evaluates the outcomes and impacts of ORD science and engineering research, programs and plans, laboratories, and research-management practices, and recommends actions to improve the quality of research and its relevance to EPA's mission. These BOSC evaluations provide ORD with advice concerning strategic planning, program implementation, utilization of peer review within ORD, human resources development, and other organizational challenges.
The products developed in ORD's research programs are peer reviewed in accordance with the highest standards, and a variety of metrics are used to demonstrate the quality, relevance and performance of each program. Progress toward achieving strategic, long-term goals is measured, in part, by the timely completion of key research milestones. Bibliometric analyses are used to assess the extent to which research products are cited and published in the most prestigious scientific journals, as well as the extent to which products are cited and used in regulatory and policy decisions. Together, these accountability efforts help to ensure that EPA research has a critical impact on decisions and actions that improve human health and the environment.
Research Outputs
ORD’s Annual Performance Measures (APMs) are the outputs (e.g., publications, tools, reports, or models) that research programs plan to complete in a given fiscal year. The completion of APMs is an indication of progress toward accomplishing Annual Performance Goals (APGs) and allows program management, stakeholders, and evaluators to determine that ORD's research programs are achieving annual results.
Research Quality and Impact
To determine if EPA is producing science that impacts decisions, ORD conducts partner surveys for all of its major research programs. These surveys enhance ORD’s understanding of how federal and state decision makers use ORD's research to address environmental problems.
As another tool to assess the impact of ORD’s science, bibliometric analyses are used to measure the extent to which ORD’s research is on the cutting-edge of science. Common bibliometric methods determine the impact and popularity of a publication by assessing how frequently the publication is cited in scientific journals. ORD’s bibliometric analyses demonstrate that its publications are consistently cited more frequently than average when compared to publications in the same fields, thus indicating that ORD’s science is substantially impacting the scientific community.
ORD is finalizing a process for measuring the extent to which specific ORD partners cite ORD research in their documents and decision-making. Results from this type of decision-document analysis will help demonstrate how ORD research contributes to improvements in human health and the environment.
Since the inception of the Program Assessment Rating Tool (PART) process in 2002, the White House’s Office of Management and Budget has reviewed ORD’s research programs to determine if programs are effective and are achieving results.
The results of ORD’s performance are included in EPA’s annual Performance and Accountability Report.
Research Efficiency
In 2007, after examining best practices at other agencies, EPA adopted efficiency measures for all of its research programs that helped the Agency achieve a “green” rating on its President’s Management Agenda (PMA) Scorecard for the Performance Improvement Initiative. EPA also engaged the National Academy of Sciences (NAS), the Office of Management and Budget (OMB), and other agencies in a study to determine how best to measure the efficiency of research in federal agencies.
The NAS report Evaluating Research Efficiency in the U.S. Environmental Protection Agency provides EPA and other federal research agencies with suggestions on implementing and assessing efficiency. ORD continues to lead discussions with OMB and other research agencies concerning how best to apply the NAS recommendations, especially in the context of OMB’s Program Assessment Rating Tool (PART) reviews.
In April 2008, ORD and OMB jointly led a government-wide panel discussion on efficiency measurement that included panelists from the Department of Energy, National Science Foundation, National Institutes of Health, National Aeronautics and Space Administration, National Institute for Occupational Safety and Health, as well as the National Academy of Sciences.
Research Advisory Groups
Board of Scientific Counselors — The Board of Scientific Counselors (BOSC) provides advice, information, and recommendations about the Office of Research and Development’s (ORD) research programs. Members of the BOSC Executive Committee and Subcommittees constitute a distinguished body of scientists and engineers who are recognized, non-governmental experts. In addition to evaluating the quality and relevance of ORD’s research, the BOSC reviews progress toward long-term goals and provides recommendations for ORD’s research programs.
Science Policy Council — The Science Policy Council (SPC) serves as a mechanism for addressing EPA's many significant science policy issues that go beyond regional and program boundaries.
Office of the Science Advisor — The Office of the Science Advisor (OSA) provides leadership in establishing specific mechanisms for ensuring that scientific results and hypotheses, with technical evaluation and peer-review, play a prominent role in all regulatory decisions.
Contact Information
Housed in ORD’s Office of Resources Management and Administration (ORMA), the Accountability Team leads ORD's effort to be a high-performance organization with demonstrated results. Inquires related to ORD performance and accountability can be directed to Phillip Juengst, Accountability Team Leader, at juengst.phillip@epa.gov.