Summary
The mission of the Office of Science & Technology (OST), within the Department of Justice's National Institute of Justice (NIJ), is to improve the safety and effectiveness of technology used by federal, state, and local law enforcement and other public safety agencies. Through NIJ, OST funds programs in forensic sciences, crime prevention, and standards and testing. To support these programs, Congress increased funding for OST from $13.2 million in 1995 to $204.2 million in 2003 (in constant 2002 dollars). GAO reviewed (1) the growth in OST's budgetary resources and the changes in OST's program responsibilities, (2) the types of products OST delivers and the methods used for delivering them; and (3) how well OST's efforts to measure the success of its programs in achieving intended results meet applicable requirements.
OST's budgetary resources grew significantly in recent years, along with the range of its program responsibilities. From fiscal year 1995 through fiscal year 2003, OST received over $1 billion through Department of Justice appropriations and the reimbursement of funds from other federal agencies in exchange for OST's agreement to administer these agencies' projects. Of the over $1 billion that OST received, approximately $749 million, or 72 percent, was either directed to specific recipients or projects by public law, subject to guidance in congressional committee reports, or directed though reimbursable agreements. At the same time that spending expanded, OST's program responsibilities have changed--from primarily law enforcement and corrections to broader public safety technology. OST delivers three groups of products through various methods. The three groups include (1) information dissemination and technical assistance; (2) the application, evaluation, and demonstration of existing and new technologies for field users; and (3) technology research and development. According to OST, as of April 2003, it has delivered 945 products since its inception. Furthermore, OST identified an additional 500 products associated with ongoing awards. OST makes its products available through a variety of methods, such as posting information on its Web site and providing research prototypes to field users for testing and evaluation. OST has been unable to fully assess its performance in achieving its goals as required by applicable criteria because it does not use outcome measures to assess the extent to which it achieves the intended results of its programs. OST's current measures primarily track outputs, the goods and services produced, or in some cases OST uses intermediate measures, which is a step toward developing outcome measures. The Government Performance and Results Act of 1993 provides that federal agencies measure or assess the results of each program activity. While developing outcome measures for the types of activities undertaken by OST is difficult, we have previously reported on various strategies that can be used to develop outcome measures, or, at least intermediate measures, for similar types of activities.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
Laurie E. Ekstrand
Government Accountability Office: Homeland Security and Justice
(202) 512-2758
Recommendations for Executive Action
Recommendation: To help ensure that OST does all that is possible to measure its progress in achieving goals through outcome-oriented measures, the Attorney General should instruct the Director of NIJ to reassess the measures OST uses to evaluate its progress toward achieving its goals and to better focus on outcome measures to assess results where possible. In those cases where measuring outcome is, after careful consideration, deemed infeasible, appropriate intermediate measures that will help to discern program effectiveness should be developed.
Agency Affected: Department of Justice
Status: Implemented
Comments: In November 2003 we reported on our review of performance measures that the National Institute of Justice's (NIJ) Office of Science & Technology (OST) developed to evaluate progress toward achieving program goals and found that none were outcome-oriented. Instead the measures focused on outputs (the count of goods ands services produced) or intermediate measures (progress to achieving intended results). We recommended NIJ to reassess the measures, focus on outcomes (actual results compared with intended results) where possible, and develop intermediate measures in those cases where it deemed measuring outcomes as infeasible. Acting on our recommendation in 2004, NIJ stated that it concentrated its reassessment efforts on 8 measures that it considered best reflected program outcomes or were, at a minimum, intermediate measures that provided meaningful information about program effectiveness. We agree that one of those new measurements is outcome-oriented: the percent reduction in DNA backlog. According to NIJ, reassessment efforts continued through 2006, such as updating its strategic plan and providing staff with performance measurement training. In addition, NIJ noted that the White House Office of Management and Budget initiated a Performance Assessment Rating Tool (PART) review of all NIJ programs in 2005. We encourage NIJ to continue its efforts to improve the measurement of OST program effectiveness, as such efforts will assist Congress and NIJ's management and customers to better assess whether investment in OST programs was paying off with improved law enforcement and public safety technology.
