Detailed Information on the
Weed and Seed Assessment

Improving the automation of performance data collection and handling to better track how the program is performing.

Conducting a rigorous national evaluation to assess the impact of the Weed and Seed program, or its component strategies, at sites across the nation.

Revising goals and collecting new performance data to evaluate how the program is performing.

Working with grantees to improve the sustainability of the program--even in the absence of continued funding for each site.

Measure: Percent reduction in homicides per site funded under the Weed and Seed Program.

Explanation:Because sites operate at different levels of maturity and activity, reporting will be for sites that have been in operation for at least three years. This means that the reporting cohort of sites will change each year, as sites mature or achieve sustainability without the need for future funding. Note: Data for this measure is collected on an annual calendar year basis and will not be available until Spring 2006.

Measure: Application processing time (in days) in program office to process an application.

Explanation:The measure is calculated from GMS data as the average number of days it takes to process an application based on the date an application is submitted up to and including the date of award determination. Note: Target figures reflect corrections to erroneous figures entered into the PART instrument

Measure: Crime rate for Weed and Seed sites in the OWS/Urban Institute indicator study

Explanation:Reducing the crime rate is a positive outcome. Based on incident specific data collected and analyzed by Urban Institute, sites will be grouped into cohorts each year, after which results following 3 years of program operation will be reported. Currently, there are 10 sites in the study, however, OWS/Urban Institute are working toward adding additional sites. Discontinued: No useful data obtained from the study. W&S found the recommendations proposed to be not feasible.

Measure: Percentage of sites including a multi-jurisdictional task force.

Explanation:The measure reflects activity within a community to help reduce crime, leading to reduction in crime. Note: Data for this measure is collected on an annual calendar year basis and will not be available until Spring 2006.

Measure: Percentage of sites that have a prosecutor dedicated to trying firearms cases

Explanation:The measure reflects activity within a community to help reduce crime, leading to reduction in crime. Note: Data for this measure is collected on an annual calendar year basis and will not be available until Spring 2006.

Measure: Percentage of sites using 3 or more community policing activities

Explanation:The measure reflects activity within a community to help reduce crime, leading to reduction in crime. Note: Data for this measure is collected on an annual calendar year basis and will not be available until Spring 2006.

Measure: Number of homicides per site (average for sites reporting)

Explanation:This measure ultimately will be replaced by the measure under development that attempts to assess homicide activity for a cohort of sites that has had at least 3 years of program activity, and which represents a better, longer-term outcome. Note: Data for this measure is collected on an annual calendar year basis and will not be available until Spring 2006.

Measure: Number of homicides per site (average for sites reporting)

Explanation:Reduction of violence, especially homicides, is a significant measure of the outcome associated with Weed and Seed. Note: Data for this measure is collected on an annual calendar year basis and will not be available until Spring 2006.

Is the program purpose clear?

Explanation: The purpose of Weed and Seed is to prevent and reduce violent crime and drug crime in high-crime areas.

Evidence: This purpose is clearly stated in the Weed and Seed Implementation Manual (see www.ojp.gov/ccdo/impmanl.htm) and Funding Application Kits (see www.ojp.usdoj.gov/ccdo/pdftxt/2004_OR_application.pdf).

Does the program address a specific and existing problem, interest or need?

Explanation: Weed and Seed addresses serious federal, state, and local crime problems which reduce the quality of life. Specifically, the program brings coordination to federal/state/local resources to target reduction in violent and drug crime in blighted communities.

Evidence: U.S. Attorneys, local law enforcement officials and leaders, and community residents come together to identify the area and crimes to be addressed in a Weed and Seed site strategy, which is described in the 2004 Executive Office for Weed and Seed Program Guide and Application Kit. (See www.ojp.usdoj.gov/ccdo/pdftxt/2004_OR_application.pdf). Additionally, strategies for potential sites are located in the Office for Weed and Seed.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: There is no similar program that provides for targeting of federal/state/local resources and coordinated leveraging of resources. The goal is to provide complementary but not duplicative services to the residents of Weed and Seed sites and others affected by crime related to those sites.

Evidence: A 1999 National Institute of Justice (NIJ) National Evaluation of the Weed and Seed Cross-site Analysis states that this is the only program that takes a holistic approach across disciplines to reduce crime and rebuild neighborhoods. Other Federal programs may only support one aspect of these areas.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Program design includes checks and balances by U.S. Attorneys, scrutiny by Weed and Seed Office staff and Office of Justice Programs (OJP) Office of the Comptroller (OC). For 2004, sustainability criteria have been developed for graduating sites from the program.

Evidence: GAO found that the program lacked fully developed criteria for transitioning sites off of program funding, which inhibits funding for new sites. GAO-04-245 Efforts to Improve Weed and Seed Program New sustainability criteria have been outlined in the program's application materials. (See www.ojp.usdoj.gov/ccdo/pdftxt/2004_OR_application.pdf)

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The program is effectively targeted to areas suffering from high rates of violence and drug crime that are also amenable to coordinated federal/state/local efforts by working through the U.S. Attorney Offices and requiring data on the key crime problems of the target area.

Evidence: Applicants for Official Recognition of a Weed and Seed strategy must report key crime data and obtain concurrence from their U.S. Attorney and local law enforcement to ensure that the site is willing to work to improve their community (see www.ojp.usdoj.gov/ccdo/pdftxt/2004_OR_application.pdf). These data are derived from local reported crime statistics and cross-checked with Uniform Crime Report (UCR) data and further verified through on-site visits. Additionally, when choosing a site, an applicant's crime rate must surpass the neighboring jurisidictions to be selected.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Specific outcomes measures under development are reduction of homicides and reduction of the crime rate for Weed and Seed sites. The measures will be based on sites that have at least 3-years of experience with the program to allow time for Weed & Seed strategies to take effect and to ensure comparability among reporting sites. An existing homicide measure, based on existing reporting for selected sites, continues to be used as a proxy measure for outcome purposes pending implementation of the new measures.

Evidence: The 2004 Weed and Seed Competitive Application Kit requires Weed and Seed sites to report homicide data annually on Government Performance and Results Act (GPRA) reports as well as Letters of Intent, which requests Part I crime data, and are retained in-house. Individual site data are accessible online (see www.weedandseeddatacenter.org), however, aggregate data are maintained and analyzed by Justice Research and Statistics Association (JRSA). In a prospective site's letter of intent to become Officially Recognized, they must submit Part 1 violent crime stats and their Drug Offense Rate. Once this information is received, it will be reviewed by JRSA relative to National, regional, and local crime rates.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: For the new homicide measure, the target is an annual reduction of 5% in homicides--revised upward from the 5% reduction over three years identified in the Strategic Plan. This appears reasonable, especially given that urban homicides continue to increase in recent crime statistics. For the existing homicide measure, the program targets have been focused on non-specific decreases -- and not always successfully, as demonstrated by performance. However, the program is adopting more specific, numeric targets.

Evidence: The 2003-2008 DOJ Strategic Plan (see www.usdoj.gov/jmd/mps/strategic2003-2008/index.html) has added an improved measure of the effectiveness of Weed and Seed in the form of setting as a goal the reduction of homicides by 5% over a 3-year period of operation in sites.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program tracks homicides at Weed and See sites, which can be considered an outcome focused measure. Generally, in recent years, homicides have fallen at Weed and Seed sites -- although 2003 experienced an increase. This upsurge is not necessarily an indication that the program is failing to make progress toward long term goals, but may be only a temporary phenomenon or a reflection of the experience mix of the programs reporting that year.Annual reporting of a number of activities (or outputs) associated with the program includes: the percentage of Weed and Seed sites using multi-jurisdictional task forces, prosecutors dedicated to trying firearms cases, and 3 or more community policing activities. These measures are important indicators of the implementation of Weed and Seed strategies, which are intended to lead to reduced homicide and violent crime--the longer term outcomes.

Evidence: For these and other annual reporting requirements, see Grantee Site Characteristics and Activity Data Report Forms Guidance at: www.weedandseeddatacenter.org/downloads/EOWS2.4.pdf

Does the program have baselines and ambitious targets for its annual measures?

Explanation: For the homicide measure, the program has set a goal of a 5% reduction annually. The baseline reflects prior year performance. This appears reasonable, especially given that urban homicides continue to increase in recent crime statistics. For the program activity measures (e.g., percentage of sites using 3 or more community policing activities, percentage using multi-jurisdictional task forces, etc.) Weed and Seed sets targets for 2004 and 2005 based on a baseline of combined 2001-2002 performance. The targets stress gradual though continuous improvement, but have not been well justified. Beginning in 2004, the program sets goals based on the percentage of sites employing these elements of the Weed and Seed strategy. Prior to 2004, targets were numeric (e.g., targeting numbers of sites) and less meaningful.For the program efficiency measure, application processing time, a baseline has been established and targets for improvement have been set. While the targets do not appear ambitious, OJP promises to revisit these targets within the year, once grants management systems improvements are in place.

Evidence: The 2003-2008 DOJ Strategic Plan (see www.usdoj.gov/jmd/mps/strategic2003-2008/index.html) sets as a goal the reduction of homicides by 5% over a 3-year period of operation in sites. The basic goal of a 5% reduction over three years has been revised to an annual goal.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The Weed and Seed grant application requires the grantee to abide by the Weed and Seed coordination strategy, which involves all parties committing to work together toward the annual and long-term goals of the program, including signing a Memoranda of Understanding (MOU).

Evidence: OWS program managers perform phone and on-site monitoring to check and encourage commitment by Weed and Seed site partners. See the Weed and Seed Program Guide and Application Kit (see www.ojp.usdoj.gov/ccdo/pdftxt/2004_OR_application.pdf), and progress reports, which are maintained in hard copy in the OJP/OC's Official Files.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: JRSA has performed two studies on Weed and Seed site performance: 1) An analysis of homicides reported and 2) a Crime Pattern analysis of the crimes reported at Weed and Seed sites. These analyses compare and contrast performance data across sites. A more comprehensive and rigorous evaluation is needed that attempts to compare outcomes for Weed & Seed sites to comparable sites not receiving such assistance.

Evidence: JRSA found that during 1996-2001, of the 220 sites with sufficient data, 77 percent showed positive results: In 122 sites, homicides decreased by about 50 percent; in 31 sites, homicides remained stable; and in 17 sites, homicides increased at a slower rate than their host jurisdictions (see JRSA Report, A Comparison of Homicide Trends in Local Weed and Seed Sites Relative to Their Host Jurisdictions, 1996 to 2001, November 2003; www.jrsa.org/weedandseeddata/studies_other/jrsa_comparison_homicide.pdf, and JRSA Memo to Bob Samuels, Three-Year Homicide Trend Analysis, April 13, 2004; www.jrsa.org/weedandseeddata/studies_other/jrsa_homicide_trend.pdf).JRSA's analysis of crime patterns finds generally positive results in Weed and Seed sites (see JRSA Report, Weed and Seed Crime Pattern Data Analysis, May 3, 2004; www.jrsa.org/weedandseeddata/studies_other/jrsa_crime_pattern.pdf).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: OJP's budget process aligns budget and performance information for major programs, although the effects of funding on results are somewhat unclear. While OJP's budget process aligns budget and performance information, the formal budget submission (both OMB & congressional) only do so at a level of aggregation well above the program level. OJP will revise it's 2006 submission to make more explicit the linkage between budget and performance for the Weed and Seed program.

Evidence: In the 2005 Congressional Budget submission, the Weed and Seed program budget request is supplemented by summaries of evaluation results. The budget for the larger decision unit (Improving the Criminal Justice System) includes some performance information for the Weed and Seed program.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Weed and Seed guidance has required improved strategic planning from sites and the Weed and Seed program office has provided training and technical assistance to sites in strategic planning and sustainability. Weed and Seed sites must begin by going through a strategic planning process through which they develop an application for Official Recognition of their site's Weed and Seed strategy. Funding criteria have been revised in 2003 and 2004 to reward sites making progress toward sustainability.

Evidence: The Official Recognition strategic planning guidelines have been modified to require more crime data and analysis earlier in the process and training in strategic planning has been bolstered by the addition of benchmarks for site strategies outlined in Weed and Seed Program Guidance, see (www.ojp.usdoj.gov/ccdo/pdftxt/2004_OR_application.pdf)

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: OWS requires that GPRA reports be submitted with every application for funding. Sites which show an increase in homicides receive technical assistance from OWS to improve their anti-crime strategies. The new performance target for homicides is a reduction of 5% over 3 years of program implementation.

Evidence: Annual GPRA filings are used to identify and follow up on key program areas such as the number of homicides. OWS staff contacts sites where homicides have increased significantly to analyze why and to offer assistance in dealing with the problem. GPRA data can be found on-line (see www.weedandseeddatacenter.org).

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Grantees are subject to annual desk reviews to assess how well they have been managing Weed and Seed funds and implementing the program. Where there are performance problems or where there is a high carryover of unobligated balances, a site will be excluded from the funding solicitation.

Evidence: In 2004, 8 Weed and Seed sites were excluded from the 2004 Community Capacity Development Office (CCDO) Program Guide and Application Kit because OWS program manager desk reviews identified high carryover balances or implementation problems at those sites. Data can be found in OJP/OC's Financial Capability system, (FinCap).

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: OWS indicated that Weed and Seed funds are obligated and expended in a timely manner, as set forth in the grant processing timeline manitained by the OJP/OC, and spent for intended purposes through voluntary compliance and enforcement of OJP rules by OWS guidance and monitoring and OC guidance and monitoring. However, OWS provided no evidence of the extent to which this actually is the case. Data provided by OJP to GAO recently suggests that, at least for recent years, 7+% ($4M or more) of program funds generally remains unobligated at the end of each fiscal year.

Evidence: Grantees are required to submit quarterly financial status reports, so that program managers can monitor whether they are spending their grant funds in a timely manner. OWS program staff ensures that funds are spent for the intended purpose though review of proposed budget modifications, phone monitoring, on-site monitoring, and referrals to the OC for in-depth financial monitoring. Additional information can be found on-line (see www.ojp.usdoj.gov/FinGuide/).Data on recent unobligated balances can be foundon page 8 of GAO's recent review of Weed and Seed program management practices (GAO-04-245, see www.gao.gov/new.items/d04245.pdf)

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: OJP is developing a competitive sourcing proposal that will include grants management functions. In addition, average grant awards during the life of the program have gradually declined, requiring Weed and Seed sites to "do more with less." Also, web-based training has been developed to enable Weed and Seed sites to do low-cost coordinator training. OWS also helps sites upgrade their IT equipment for computer learning labs and improved site management.

Evidence: The per-site dollar amount provided by OJP to Weed and Seed grantees has been reduced since the program began. In 1992-3, site awards averaged about $750,000 over a 12-month period. In 1997, they were reduced to $175,000 for the first year and $225,000 for subsequent years. In addition, sites have been required to absorb costs previously borne by OJP, e.g., the cost of the Drug Education For Youth program; and travel costs training for training. Additional information can be in the 2004 Executive Office for Weed and Seed Program Guide and Application kit.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Weed and Seed is essentially a coordination strategy, and reinforces coordination with related programs through training; awarding points on competitive funding reviews for site coordination with related programs; and stressing leveraging of resources with other programs.

Evidence: Weed and Seed guidance to sites in the Implementation Manual and Official Recognition Guidelines calls for the completion of Memoranda of Agreement (MOA) with other agencies and partners to pursue the Weed and Seed strategy. In addition to site-level MOA, the national Office of Weed and Seed has ongoing Interagency Agreements and Cost Reimbursement Agreements with HHS/Center for Substance Abuse Prevention, HUD, Executive Office for U.S. Attorneys, DEA, FBI, and the COPS Office. Also, in the review of 2003 Weed and Seed Competitive applications, 41 out of 70 sites received extra points for coordination with other programs such as Empowerment Zone, HUD Hope VI, Brownfields, Project Safe Neighborhoods, or Drug-Free Communities. Additional information can be found in the 2004 Executive Office for Weed and Seed Program Guide and Application kit.

Does the program use strong financial management practices?

Explanation: Weed and Seed staff provides financial management training to applicants for grants. OJP/OC's provides extensive web-based and in-person training for Weed and Seed and other OJP grantees. and Weed and Seed staff performs thorough budget reviews in cooperation with the OJP/OC, and conducts on-site reviews of financial management and refers grantees for OC on-site monitoring when questions arise. OJP received an unqualified audit opinion in 2003. Additionally, Weed and Seed follows OJP/OC's Financial Guide to ensure that the program, 1) has procedures in place to ensure payments are made properly for the intended purpose to minize erroneous payments, 2) financial managemenet systems meet statutory requirements, 3) financial information is accurate and timely, 4) integrated financial and performance systems support day-to-day operations, and 5) financial statements receive clean audit opinious and no material weaknesses.

Evidence: OWS referred over 20 sites to the OC in 2003--most for monitoring visits, but several cases involved freezing funds or deobligation of grant awards. The unqualified audit opinion was stated in the 2003 DOJ/OIG/OJP Annual Financial Statement.For program financial procedures, see the OJP/OC Financial Guide (www.ojp.usdoj.gov/FinGuide/finguide.pdf).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: On balance, the program has resolved many of its management deficiencies. One of the most significant management deficiencies has been the lack of site sustainability critieria, and the inability to plan for sites achieving independence from the limited Weed and Seed program funding available. For 2004 Weed and Seed has published sustainability criteria with the application package. Sites applying for grant funding now understand the emphasis on achieving sustainability over time, as well as what steps will need to be taken to achieve it. In 2004, Weed and Seed has added a new funding criterion that requires grant applicants to identify other sources of funding at a level that is at least five times the Weed and Seed contribution, which also will help sites achieve sustainability independent of Weed and Seed grant resources. Competitive criteria have been adopted that provide extra points for applications that promise to fund a full-time coordinator from other than Weed and Seed grant resources. GAO has expressed concern about the completeness of many grant files. Although many of the files are now housed on-line, OWS is reviewing all files for completeness. Weed and Seed undertakes annual customer assessments, including reviewing the customer service provided by OWS. OWS still needs to undertake work to establish meaningful and ambitious targets for program performance, however.

Evidence: GAO-04-245: Based on the findings in GAO's recent report (GAO-04-245), OWS has taken the following corrective actions: 1) CCDO has implemented policies requiring the completion of documentation to support major decisions and recorded monitoring information to ensure files are are appropriately maintained and readily available. All files are undergoing a completeness review to concluded by July 1. This review will be followed up by a newly established CCDO internal control requiring a quarterly file review of ten percent of randomly selected working files. 2) To encourage self-sustainability, the 2004 Executive Office for Weed and Seed Program Guide and Application Kit request applicants identify all monetary resources in support of an activity or task to reflect leveraging of existing resources. In addition, newly identified sites are encouraged to support personnel through resources other than Weed and Seed by awarding applicants who support a full time site coordinator an additional point in the competitive ranking review.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Only sites which have received official recognition of their strategic planning efforts can become Weed and Seed grantees. Further, sponsorship of the US Attorney for the district in which the site is located is needed for sites to compete for grant funding. All sites are ranked by specific factors in the application and assigned points based on the information submitted. The sites that rank highest in points achieve consideration for the funding that is available in any given year. Although Weed and Seed works with US Attorneys and the Executive Office for US Attorneys to facilitate US Attorneys' support for implementing the Weed and Seed stratgey in all US Attorney Districts, it is not clear why all US Attorney Districts do not participate in the program. Conceivably, without greater efforts to ensure the participation of all US Attorneys, some sites may be unable to compete for funding.

Evidence: Only 50% of applicants for Weed and Seed competitive grants in 2003 were successful in receiving Weed and Seed grants. Information supporting these totals are kept in-house in the Office for Weed and Seed.The competitive ranking criteria are described in the program application material. (See www.ojp.usdoj.gov/docs/WSCompetitiveSol.pdf)

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Weed and Seed provides for program manager site visits even before the Official Recognition designation is given as well as periodically after grant awards are made. Additional site assessment visits by technical assistance providers, GPRA report validation visits by JRSA, and OC monitoring visits are conducted. While oversight practices are generally sufficient, the program does not have sufficient resources to visit all funded sites even within a single year. OWS needs to document or develop criteria to target these limited resources effectively, to ensure that those sites needing visits (because of performance or other problems) are visited relatively frequently in comparison to sites needing less attention.

Evidence: In FY 2003, there were 127 site visits. Most site visits are performed by OJP employees, while others were performed by technical assistance providers, FBI Fellows, and JRSA GPRA verification visits. The on-site visits are designed to verify reports submitted to OJP, in the form of applications, or required quarterly financial status reports, semi-annual progress reports, and required GPRA reports. Data supporting this response is housed in OJP/OC's financial system, Fincap.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Submission of GPRA reports has been made a required part of annual filings by grantees. GPRA reports are posted on a publicly accessible website hyperlinked to the official OWS website at www.ojp.usdoj.gov/eows

Evidence: GPRA reports for over 240 sites are posted on the Weed and Seed Data Center website, which is hyperlinked to the OWS website (see htt://www.weedandseeddatacenter.org).

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Currently, Weed and Seed is reporting average homicides across sites on an annual basis. The data generally show that homicides have trended downward in recent years. An uptick during 2003 has unknown significance, and may only be a temporary perturbation. Also, most sites experienced lower homicide rates compared to their host jurisdictions. One of the problems has been that the existing measure reflects a mix of programs, both mature and new. However, the measure is being revised and should lead to a more stable, meaningful set of outcome data.

Evidence: Seventy-seven percent of the Weed and Seed sites had positive results in their homicide trends when compared to host jurisdictions. In most of the sites (122) homicides decreased by about 50 percent (from an average of about 10 homicides per site in 1996 to 5 per site in 2001). In 31 sites (14%), the number of homicides remained stable while the homicide statistics for their respective jurisdictions increased and in 17 sites (8%), homicides increased at a slower rate than their jurisdictions. These trends in the most serious of crimes were a significant accomplishment for Weed and Seed sites considering that homicides in the host jurisdictions (and the nation) increased in 2000 and 2001. Annual data for Weed and Seed peformance can be found on-line (see www.weedandseeddatacenter.org).

Does the program (including program partners) achieve its annual performance goals?

Explanation: There was an increase in average homicides per site during 2003, of which the significance is unknown. The measure is being redesigned to provide more meaningful information about the extent to which progress is actually reached in achieving homicide reduction. The Weed and Seed program has exceeded other performance goals for 2003. No performance data is available yet for the program's annual efficiency measure, which aims to measure the time between application and grant award, with targets set to decrease the "application to grant" interval.

Evidence: Homicides were targeted for a decrease, but increased from 3.8 per reporting site in 2002 to 5.0 in 2003. The percentage of sites using a multi-jurisdictional task force was targeted for 77.6% in 2003, but actual was 80%. The number of sites with a dedicated prosecutor was targeted for 28.1% in 2003, but actual was 62.6%. The number of sites using 3 or more community policing activities was targeted for 80% in 2003, but actual was 91.3%.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Average site awards have fallen dramatically during the life of the program, requiring sites to be more inventive and to reach out to other partners to continue achieving the program's goals. Sites have been asked to absorb other costs over time in order to continue providing funding to as many sites as possible. However, continuous progress in obtaining increased cost effeciency is not well documented.

Evidence: In 1992-3, site awards averaged about $750,000 over a 12-month period. In 1997, they were reduced to $175,000 for the first year and $225,000 for subsequent years. For 2004, first year awards continue to be $175,000 (See www.ojp.usdoj.gov/ccdo/pdftxt/2004_OR_application.pdf).

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: While no known programs share the Weed and Seed program's unique focus on crime reduction and community revitalization, numerous programs share elements of mostly one or the other of these goals. In terms of actual outcomes, the performance of many of these programs is unknown. For reduction of homicides and violent crimes, which Weed and Seed attempts to track for performance purposes, there is little data available to suggest which specific program or approach improves outcomes. Also, Weed and Seed is designed to work in tandem with the other programs, encouraging sites to see assistance and resources from those programs, when available. That Weed and Seed relies in part on other programs for its suggest makes comparison with those programs difficult.

Evidence: Among the Federal State/local assistance law enforcement programs that also share crime reduction as goals are: .DOJ Edward Byrne Memorial Grant Program .DOJ Local Law Enforcement Block Grants .DOJ Project Safe Neighborhoods .DOJ The Juvenile Justice Grant Programs (several) .DOJ Community Oriented Policing Services Among the Federal Economic Revitalization programs are: .HUD Empowerment Zone/Enterprise Communities .HUD HOPE IV .HUD Community Development Block Grant .HUD National Community Development Initiative

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations suggest that the program may be effective and achieving results. However, an updated national and comprehensive evaluation is needed along with additional work to explore the extent to which stronger evaluation methods (e.g., randomized controlled trials or quasi-experimental methods) could shed more light on the effectiveness of the Weed and Seed program.In addition to the JRSA cross-site studies described above, there have been 34 well-documented evaluations of Weed and Seed sites: 8 in the 1999 national impact evaluation; 4 by Justice Research and Statistics Association; and 22 by local evaluators whose method and data adequacy has been reviewed by JRSA.

Evidence: Of the 34 site evaluations, 8 have focused on issues other than crime or had insufficient data, 5 showed inconclusive results (results essentially similar to the jurisdiction as a whole), and 21 showed positive results (reductions in crime greater than in the overall jurisdiction, reductions in crime greater than in a control area, or statistically significant reductions in the Weed and Seed area), see NIJ's 1999 National Evaluation of Weed and Seed Cross-Site Analysis (at www.ncjrs.org/pdffiles1/176358.pdf, and summarized at www.ncjrs.org/pdffiles1/175685.pdf).