Summary
The Weed and Seed program, within the Department of Justice's Office of Justice Programs (OJP), aims to prevent and reduce violent crime in targeted neighborhoods, but it cannot optimize its effectiveness without sound management practices. In 1999, GAO made four recommendations to the Executive Office for Weed and Seed (EOWS) to improve the program's management, including (1) developing adequate internal controls to fully document decisions, (2) improving program monitoring, (3) developing criteria for determining when sites have become self-sustaining and when to reduce or withdraw program funding, and (4) developing additional performance measures. GAO did this study to assess progress in implementing these recommendations.
Despite some progress toward addressing GAO's recommendations aimed at improving program management, GAO's review shows that EOWS has not fully implemented the management improvement recommendations GAO made in 1999. First, although EOWS has revised its internal controls to require that significant qualification and funding decisions be documented and readily available in the central grant files for review, EOWS has not always ensured that its policies and procedures were followed, for the grant files GAO reviewed. Second, EOWS reported taking a number of actions intended to improve program monitoring, such as mandating the timely submission of progress reports and adequate recording of site visits as GAO recommended. Nonetheless, GAO found that while EOWS was able to provide such documentation before its review ended, documentation was not available in some of the central grant files GAO reviewed. Thus, the documentation was not readily available for external reviewers, as required by OJP policies and GAO's internal control standards. Third, GAO found that EOWS still lacks fully developed criteria to determine when sites become self-sustaining and when to reduce or withdraw Weed and Seed funds because of the level of sustainability, even though sustainability is a central goal of the program. At the time of GAO's review, no site's funding had been reduced or withdrawn because of sustainability during the 13 years of the program's existence. Fourth, EOWS has not developed outcome performance measures that can be used to adequately track progress toward program outcomes of the Weed and Seed program. While EOWS has initiated studies on how to develop performance measures, at the time of GAO's review, none of these studies had been completed. Without requirements to monitor improvements and assign accountability, progress will be difficult to achieve.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
Eileen R. Larence
Government Accountability Office: Homeland Security and Justice
(202) 512-2758
Recommendations for Executive Action
Recommendation: The Attorney General of the United States should require the Assistant Attorney General for OJP to ensure that the Executive Office for Weed and Seed fully implement the intent of our previous recommendations by maintaining the documentation of the basis and rationale for qualification and funding decisions in appropriate grant files.
Agency Affected: Department of Justice
Status: Implemented
Comments: In our 2004 report, we found that the Executive Office for Weed and Seed (EOWS) had developed policies and procedures to help ensure the documentation of significant decisions; however, challenges remained in making the documentation readily available. Specifically, we found that official recognition files and official grant files were missing key qualification and funding documentation, such as ratings information, rejection letters, and applications. We concluded that without having official recognition and official grant files complete and readily available, it may delay and complicate EOWS officials' oversight of the documentation of qualification and funding decisions. As a result, GAO recommended that the Attorney General require the Assistant Attorney General for the Office of Justice Programs to ensure that the Executive Office for Weed and Seed (now called the Community Capacity Development Office-CCDO) maintains the documentation of the basis and rationale for qualification and funding decisions in appropriate grant files. Since then, the Office of Justice Programs (which houses the Community Capacity Development Office) strengthened its internal controls by updating its Grant Manager's Manual in March 2008 to require that qualification and funding documentation, such as ratings information, reasons for rejection/denial, and applications be retained electronically in its Grants Management System (GMS). Award letters, with electronic signatures are also stored in GMS, while hard copies of denial letters are retained in grant files. Further, officials at the Community Capacity Development Office stated that since all required grant documents are submitted and retained in GMS, there is an audit trail in the system. Officials also stated that there are multiple levels of review for approving and awarding funding, which act as checks to determine whether all documentation in the file is complete. For example, both new and continuation applications are reviewed using checklists to ensure that their content, which includes the basis and reasons for grant awards, is complete. Specifically, officials use the CCDO Competitive Application File Contents checklist and the FY 2008 Continuation Grant Application Review Checklist.
Recommendation: The Attorney General of the United States should require the Assistant Attorney General for OJP to ensure that the Executive Office for Weed and Seed fully implement the intent of our previous recommendations by retaining progress reports and site visit reports in official grant files.
Agency Affected: Department of Justice
Status: Implemented
Comments: In our 2004 report, we found that the Executive Office for Weed and Seed (EOWS) had taken steps to improve program monitoring; however, challenges remained in making the monitoring documentation readily available. Specifically, we found that progress reports and site visit reports were missing from some official grant files. We concluded that not having complete and readily available official grant files may delay and complicate EOWS officials' and external reviewers' assessment of whether the Executive Office for Weed and Seed's monitoring requirements are being followed. As a result, GAO recommended that the Attorney General require the Assistant Attorney General for the Office of Justice Programs to ensure that the Executive Office for Weed and Seed (now called Community Capacity Development Office) retain progress reports and site visit reports in official grant files. Since then, the Office of Justice Programs (which houses the Community Capacity Development Office) updated its Grant Manager's Manual in March 2008 to require that site visit reports be retained electronically in its Grants Management System (GMS). The Grant Manager's Manual also states that progress reports are to be maintained electronically in GMS. Further, officials at the Community Capacity Development Office stated that the Grant Management Tool in GMS provides an accountability system for ensuring that progress reports and site visit reports are complete and maintained in GMS. Specifically, officials stated that the Grant Management Tool/GMS produces weekly reports on progress reports and can run reports to determine if progress reports are delinquent.
Recommendation: The Attorney General of the United States should require the Assistant Attorney General for OJP to ensure that the Executive Office for Weed and Seed fully implement the intent of our previous recommendations by clearly defining criteria to assess when sites are self-sustaining and apply the criteria to sites when making further funding decisions.
Agency Affected: Department of Justice
Status: Implemented
Comments: In our 2004 report, we found that the Executive Office for Weed and Seed (EOWS) had taken steps to encourage sites to become self-sustaining but had not fully developed criteria to make such determinations. We also found that the Executive Office for Weed and Seed had not reduced or withdrawn funds from any Weed and Seed sites for reasons related to becoming self-sustaining. We concluded that without criteria, there was no basis for determining when sites are self-sustaining and when to reduce or withdraw Weed and Seed funds. As a result, GAO recommended that the Attorney General require the Assistant Attorney General for the Office of Justice Programs to ensure that the Executive Office for Weed and Seed clearly defines criteria to assess when sites are self-sustaining and apply these criteria to sites when making further funding decisions. In our follow up work, we determined that the Community Capacity Development Office (CCDO) (formerly the Executive Office for Weed and Seed) had taken numerous steps to meet the spirit of the recommendation. In 2006, the CCDO updated its Policies and Procedures Guide and it now identifies community resources as an essential part of long term Weed and Seed program sustainability, and states that Weed and Seed funding applicants are required to identify, coordinate, and leverage public or private funds at a level of five times the grant contribution. In other words, sites should leverage at least $175,000 per year in other, non grant funds, for a total of at least $875,000 by the end of the 5-year funding eligibility period. The CCDO has also implemented a 5-year bell curve funding structure that peaks in year 3 and then decreases and ends in year 5, so grantees know they must sustain their initiative with other funding. This guide also states that successful and sustainable Weed and Seed sites should achieve certain goals by the end of each year and document these benchmarks, or criteria, to show sites the level at which they should be performing on an annual basis. Compliance with these benchmarks is incorporated into each grant award and is the basis of future funding decisions. Similarly, the CCDO has also incorporated sustainability assessment into its grant application review process, which is tied to funding decisions, through use of its Peer Review Rating Instrument; which is used to assess and score applicant sustainability and funding leveraging planning/coordination. The CCDO also uses the fiscal year 2008 Continuation Grant Application Review Checklist to ensure that the continuation applicant's program narrative addresses sustainability. To assist applicants, the CCDO published a guide for developing a sustainability plan, that provides Weed and Seed sustainability dimensions, which are accompanied by potential indicators, as well as requirements for achieving sustainability. Furthermore, in order to promote and encourage Weed and Seed Site sustainability, the CCDO instituted Graduated Site status which allows Weed and Seed sites that completed their 5-year grant in good standing and no longer receive Weed and Seed grant funds to continue to implement their Weed and Seed strategy, retain the Weed and Seed name, and receive technical assistance from the CCDO. The local U.S. Attorney's Office must certify annually that five criteria have been met in order for the site to retain its Graduated Site status. Specifically, the U.S. Attorney's Office must certify its own continued participation; at least quarterly Steering Committee meetings; updated contact information for the Site Coordinator; continued implementation of a Weed and Seed strategy; and submission of a current Government Performance Results Act Report-Part I. Continued implementation of the Weed and Seed strategy must be evidenced by a list of active Steering Committee members and a list of accomplishments covering the four Weed and Seed elements.
Recommendation: The Attorney General of the United States should require the Assistant Attorney General for OJP to ensure that the Executive Office for Weed and Seed fully implement the intent of our previous recommendations by developing outcome performance measures--or, where measuring outcome is, after careful consideration, deemed infeasible, intermediate measures--that can be used to adequately track progress toward program outcomes of the Weed and Seed program.
Agency Affected: Department of Justice
Status: Implemented
Comments: In our 2004 report, we found that the Executive Office for Weed and Seed (EOWS) mostly collected data about site activities rather than performance outcomes. We concluded that the Executive Office for Weed and Seed needs performance measures that focus on program outcomes, and/or achievement of intermediate goals, so that it can assess to what extent the program is achieving its goals. As a result, GAO recommended that the Attorney General require the Assistant Attorney General for the Office of Justice Programs to ensure that the Executive Office for Weed and Seed develop outcome performance measures or intermediate measures (if outcome measures are infeasible) that can be used to adequately track progress towards program outcomes. The Executive Office for Weed and Seed (now called the Community Capacity Development Office) recognized GAO's concerns related to measuring outcomes and awarded a grant to the Justice Research and Statistics Association (JRSA) to expand the available information for assessing the performance of individual local sites as well as provide indicators for the overall National Weed and Seed Initiative. JRSA's May 2006 Weed and Seed Performance Measures report includes suggested performance measures, which correspond to specific Weed and Seed activities and are accompanied by information regarding the measure's use. While the JRSA did not identify the measures as intermediate or outcome, in our assessment, the performance measures suggested in the report are intermediate measures, as they represent conditions that would likely precede or contribute to achieving ultimate outcomes, and therefore they respond to our recommendation. For example, awareness of increased police presence in the community is an intermediate performance measure used to assess increased police patrols. We believe this recommended performance measure is an intermediate measure, because it contributes to achieving the ultimate outcome of crime reduction. In addition, the Weed and Seed Performance Measures report suggested another intermediate measure for Weed and Seed job training programs that assesses the extent to which those who attend job training obtain employment. Specifically, this measure states that job training activities be evaluated by using the change in unemployment rate (within the Weed and Seed area) or the number of program participants employed at given points or for given periods of time after program completion. This measure closely resembles the intermediate measure GAO provided as an example for Weed and Seed job training activities in our report, and its classification as an intermediate measure rests on the assumption that individuals who are employed are less likely to commit crimes.
