Evaluating a Weed and Seed Strategy Jan Roehl, Ph.D. Justice Research Center In consultation with the Executive Office for Weed and Seed May 2002 Table of Contents Introduction . . . . . . . . . . . . . . . . . . . . . . . . . .1 Why Evaluate Your Weed and Seed Strategy?. . . . . . . . . . . .1 Evaluation in Five Easy Steps . . . . . . . . . . . . . . .1 Challenges of Evaluating a Weed and Seed Strategy . . . . .2 Planning an Evaluation of Weed and Seed. . . . . . . . . . . . .2 Goals and objectives, logic models, design, working with an evaluator, resources Figure 1: Simplified Conceptual Framework for Evaluation. .5 Conducting the Process Evaluation. . . . . . . . . . . . . . . .8 Table 1: Blueprint for Program Documentation . . . . . . .9 Table 2: Blueprint for Program Monitoring. . . . . . . . 10 Assessing Program Impact . . . . . . . . . . . . . . . . . . . 12 Table 3: Blueprint for Assessing Program Impact. . . . . 13 Putting it all Together. . . . . . . . . . . . . . . . . . . . 15 Additional Evaluation Resources. . . . . . . . . . . . . . . . 16 Sample Observation Form. . . . . . . . . . . . . . . . . . . . 17 Sample Community Survey. . . . . . . . . . . . . . . . . . . . 21 Evaluating a Weed and Seed Strategy Introduction This brief report presents an abbreviated but step-by-step approach to a comprehensive local Weed and Seed evaluation. It has been written primarily for program administrators who are planning for a local evaluation, working with a local evaluator, or launching a self-evaluation effort. Why Evaluate Your Weed and Seed Strategy? Everyone with a stake in a local Weed and Seed strategy has an interest in knowing whether the program is operating as intended and whether it is effective and worthwhile for designated neighborhoods and individuals. Accurate information is vital for understanding the success or failure of a Weed and Seed strategy, and is essential for anyone wanting to expand or replicate activities that work, or fix ones that don't. Program administrators responsible for managing Weed and Seed's many technical aspects also must provide information to the Executive Office for Weed and Seed (EOWS) and other supporting agencies to enable them to judge whether their resources are being well spent. These interests can be met through comprehensive local Weed and Seed evaluations. A comprehensive evaluation includes a process evaluation that describes how the program operates and whether it is operating as you and other stakeholders intended it to operate, as well as an impact evaluation, which describes how well your program operates and whether it is effective in reaching stated goals. Evaluation in Five Easy Steps 1. Plan -- Decide what you want to know and how to get the answers. Advance planning for information gathering is critical to the success of local (and all!) evaluation efforts. 2. Identify an evaluator -- Recruit an experienced evaluator from a local university, private firm, or agency, or launch a self-evaluation from within the Weed and Seed organization. 3. Describe and assess how your Weed and Seed strategy works, vis-a-vis how you (collectively with other stakeholders) intended it to work. 4. Assess how well the Weed and Seed strategy works in reaching long-term objectives such as reducing violent crime in a designated neighborhood, increasing citizen perceptions of safety, and improving neighborhoods physically and economically. 5. Communicate your evaluation results and strive to strengthen any weak points in your strategy. Keep record keeping systems current. And always, always, celebrate your successes. Challenges of Evaluating a Weed and Seed Strategy The "kitchen sink" program -- Weed and Seed guidelines have changed only slightly since the basic approach of implementing weeding, community policing, and seeding interventions in designated neighborhoods through problem-solving partnerships was first tested in two dozen cities a decade ago. A local evaluation of Weed and Seed is in part a compilation of many focused evaluations of complex strategies launched by law enforcement and justice system officials, community organizations, city agencies, non-profits, and citizens themselves. Advance planning is crucial to the multi-faceted evaluation needed, to build data requirements into routine record keeping efforts. Sharing sensitive information -- Because of its many facets, evaluating -- in fact, running -- a Weed and Seed initiative requires sharing sensitive information among key partners. Practical and legal barriers to sharing information among law enforcement and non-law enforcement partners should be fully aired during early meetings of the steering committee and resolved via memoranda of understanding among participating agencies covering information sharing, storage, access, and dissemination. Comparison areas, displacement and other methodological issues -- Comparison areas are needed in a local evaluation, to help assess whether any observed differences in the designated area were due to Weed and Seed efforts or some other factor(s), yet they nearly double the evaluation effort. They do little to answer the oft-asked question of displacement, "Did you reduce violent crime or just move it across town?," although city-wide, non-designated area data can help. Community surveys are increasingly difficult to launch technically, and the best ways are the most expensive. New record keeping systems may be needed to accurately track Weed and Seed data. Costs and other obstacles -- All of these methodological issues often add up to costs, gaps, and delays in data collection and analysis. Again, these may be reduced by coordinated advance planning and will be helped by the addition of dedicated evaluation staff. Planning an Evaluation of Weed and Seed Most Weed and Seed sites began planning for a local evaluation, methodically or not, during the process of applying for Weed and Seed funds or Official Recognition. The Official Recognition Application, for example, requires organization and strategy descriptions, a community needs assessment (which may require the collection of crime data or conducting a neighborhood survey), and the development of a strategic plan (which includes identifying goals, objectives, tasks, and outcome measures). Ideally, evaluation has been a topic of discussion since the first meeting of your Steering Committee -- when mention was first made of applying for Weed and Seed status -- as it is much easier to design an evaluation as you are developing program activities than to try to tack one on after the fact. In particular, it is much easier to build in data collection steps from the start rather than recreating data long after the intervention is underway. Yet regardless of when an evaluation is designed, the basic planning steps are the same -- you must identify program goals and objectives (which point naturally to evaluation objectives), decide exactly which questions you want answers to and with what rigor, design the evaluation methodology, and figure out how to get the information desired. Identify your goals and objectives. The first step in most evaluations is the specification of goals and objectives, fine-tuned into measurable terms. Goals are broad statements of the Weed and Seed program's principal aims, such as "to reduce juvenile crime in the designated neighborhood" and "restore the community by encouraging home ownership and rehabilitating dilapidated property." Some objectives which may be listed under the goal of reducing juvenile crime: Increase the number of nonviolent juvenile offenders referred to Youth Court, to 125 per year. Expand the number and type of community service opportunities for nonviolent juvenile offenders. Identify the top 100 high risk juveniles in our jurisdiction whose prior offenses have involved alcohol, drug, and/or weapons. Add two probation officers to provide intensive supervision to as many as these 100 high risk juveniles as possible, plus other juveniles arrested on alcohol, drug, and/or weapons charges. Add 20 outpatient substance abuse slots for juveniles. Increase by 20% the number of youth who enter substance abuse, anger management, or other appropriate programs within two years. Reduce the number of alcohol, drug, and/or weapons violations committed by juveniles by 30% within 3 years. Increase job and recreational opportunities for high risk youth. Objectives are narrower, measurable, operational specifications of goals, and your strategy undoubtedly has many of them. The length and content of a list of objectives to match the goal of reducing juvenile crime depends on local strategies and may be as long as your stakeholders have ideas and dreams. Who should be involved in specifying the local Weed and Seed goals and objectives? The general answer is that it should be the stakeholders, who are individuals representing organizations, agencies, or groups affected by the implementation or results of the Weed and Seed strategy. The more specific answer is clear in the mandated local organizational structure: the stakeholders are found in the Weed and Seed Steering Committee and significant subcommittees. These committee members have a vested interest in seeing the Weed and Seed strategy succeed, and, among other responsibilities, are charged with the responsibility to develop a vision or mission statement, conduct a community needs assessment, set goals, and commit resources to the selected activities. Weed and Seed objectives are always multiple, and different stakeholders will have varying, even competing, views of appropriate objectives. Substance abuse specialists and neighborhood residents, for example, may see the road to reductions in juvenile crime paved with treatment programs, alternatives for youth, and mentoring, while the District Attorney is concerned with increasing arrests, decreasing time to disposition, and extending the purview of juvenile probation officers. These differences need not be detrimental to an evaluation effort. Rather, your stakeholders should discuss and reach a consensus on the short- and long-term goals of Weed and Seed, then identify specific -- multiple -- objectives for achieving them. This process may occur over the course of several meetings during the implementation phase. Developing a conceptual framework or logic model. Developing a conceptual framework of your program, or a logic model, is another way to specify how the various components of the strategy lead to desired outcomes and goes hand-in-hand with the specification of goals and objectives. A logic model is a graphic version of the key elements of a given program which identifies how the desired outcomes result from the interventions applied to a given problem in an identified setting. A simplified version of a logic model can be used to outline the types of information needed for the evaluation. Figure 1 presents a conceptual framework -- a kind of a bird's eye view -- of a comprehensive local Weed and Seed evaluation. The Weed and Seed conceptual framework begins with the identification and selection of a target problem, which occurs in a specific setting, a neighborhood with known characteristics and resources. [Column A] A Weed and Seed strategy is created to address the target problem. It begins with the creation of a multi-agency partnership which closely examines the target problem, designated neighborhood, and resources available, and launches appropriate intervention strategies [Column B] to achieve certain immediate proximal outcomes [Column D]. But wait! The outcomes of these intervention strategies may also be affected by historical and current events outside of the program [Column C]. These "contextual conditions" are often pictured in a long horizontal box that stretches below the entire conceptual framework to indicate that local conditions and events may affect all aspects of the strategy. As the intervention strategies continue, it is hoped that positive changes in longer-term outcomes will occur [Column E]. A full understanding of how and how well a Weed and Seed strategy works requires expanding this bird's eye view into a more detailed road map for the evaluation. After discussing who might conduct your evaluation and what evaluation design is most suitable, a brief guidebook is offered for that journey. Figure 1: Simplified Conceptual Framework for Evaluation (A) The Problem: Setting and Program Characteristics (B) Implementation and Processes: Program Management and Activities (C) Intervening/Mediating Variables: Other Programs and Happenings (D) Proximal/Process Outcomes (E) Short- and long-term Outcomes Targeted neighborhood problems and characteristics Resources available W&S goals and objectives W&S organization, structure, and management Weeding activities by law enforcement, prosecution, probation/parole, and corrections Community policing activities Seeding activities for Prevention, Intervention, and Treatment Neighborhood Restoration activities Other anti-crime and anti- drug strategies Jurisdiction's history of partnerships and interagency collaboration Jurisdiction's problems, infrastructure, characteristics of note Number of events, people involved, jobs created, houses rehabbed, other relevant things... Arrest statistics Person-hours devoted to W&S Case tracking through CJS Prosecution statistics Probation/parole statistics Part I crime statistics, target crime statistics Resident views on fear of crime, victimization, quality of life, neighborhood satisfaction Observations of physical changes in neighborhood Recidivism Reductions in substance abuse Changes in key indicators Where can you find a good evaluator? 1. Your local college or university. Partner with professors in psychology, sociology, criminal justice, public administration, or other relevant research field. Call the chairs of these departments or the central administration to inquire about affiliated policy or research institutes. 2. Local agencies, within or outside Weed and Seed. Nearly 30% of the members of the American Evaluation Association work in government agencies or nonprofit organizations. Perhaps there are researchers in the police department's Research and Planning Unit, or statisticians in the city's Community Development Department who would be willing and able to help. A self-evaluation -- "not that there's anything wrong with that" -- will be the result if these agencies are closely tied to Weed and Seed. 3. Independent research firms and consultants. If you are not aware of any local firms, begin asking other program directors (the drug court coordinator, or Safe Schools, Healthy Students program director, for example) about their experience with local researchers. 4. Find out about who is working with similar programs, such as Arrestee Drug Abuse Monitoring (ADAM -- operating in over 30 cities) and the Strategic Approaches to Community Safety Initiative (SACSI -- in 10 cities). Choosing and working with a local evaluator. In this report both the spirit of self-evaluation and the rigor of neutral, objective examination are embraced. Much of what is called process evaluation is simply good program monitoring. While there are definite advantages to an evaluation conducted by outside experts, having program practitioners engage in self-evaluation has its benefits as well. The approach taken here is pragmatic and eclectic, aiming to provide local program practitioners with the basic knowledge and tools to meet evaluation needs. Find a local evaluator with a broad range of relevant experience and expertise, and time to devote substantial attention to the many facets of a Weed and Seed evaluation. With the input of the steering committee, specify the role of the evaluator. Consider the SACSI model, in which a local researcher is an integral partner with a dual role of analyzing target crimes to help direct problem-solving efforts and monitoring program activities and outcomes over time. Qualities to search for in your evaluator: A solid understanding of social science and criminal justice research techniques coupled with relevant experience. A broad knowledge of W&S problem areas -- crime, drugs, quality of neighborhood life -- and how to measure change in these areas. Patience, comfort working with multiple agencies and committees, good humor, cultural competence, discipline, organization. Resources to support the evaluation. In FY 2001, EOWS-funded Weed and Seed Continuation sites were given the option of using up to $50,000 in Special Emphasis Initiative funds to cover the costs of a local evaluation. Funding may also be available from other local, state, and federal government agencies, or private foundations, for all or part of the local evaluation. Weed and Seed initiatives lend themselves to expensive evaluations, for all the reasons previously discussed -- they have many different facets and components, neighborhood surveys can be expensive, comparison areas drive up the costs, criminal records are difficult to obtain, etc. But evaluations do not have to be expensive. A solid evaluation may be pieced together by paying a small amount of funds to an experienced evaluator to design and oversee the evaluation, with data collection conducted by existing staff or students who are paid at something over minimum wage (but not much) or who do the work as part of their education, in class, as a special project, or as part of community service requirements. Evaluation design. Evaluation, or research, design refers to how impact evaluations are conducted. The strategic aim is to control factors so that any observed differences in key outcomes (such as a decrease in gun violence) may be accurately attributed to the Weed and Seed intervention and not some other event or activity (e.g., a new state law with increased penalties for weapon possession by probationers, a city-wide educational campaign, or cold rainy weather). A solid design for evaluating a Weed and Seed strategy is a quasi-experimental design in the form of a pre/post case study with a comparison area. Anything less rigorous will leave you open to criticisms -- that you haven't described the overall strategy thoroughly, or that outcomes weren't assessed, or that you haven't shown whether Weed and Seed or some other program or set of factors led to changes observed in the designated neighborhood. More rigorous evaluation designs will be prohibitively expensive. In practical terms, a pre/post case study with a comparison area means: 1. Developing a detailed description and assessment of how your Weed and Seed site operates (a.k.a. the case study). This case study will be both qualitative and quantitative -- it will include a written description of the designated neighborhood, how the Weed and Seed Steering Committee was formed, how the partners work together, the nature of the intervention activities, etc., coupled with information as to how many people were arrested on what charges, how many youth attend Boys and Girls Club activities, etc.. 2. Gathering "baseline" (before the program begins in earnest) information on key impact indicators, such as the number of arrests in the designated area, number of youth participating in afterschool programs at the Boys and Girls Club, resident satisfaction with the neighborhood, number of calls for service for reported gang activity, Part I crimes, etc., and then repeating the same data collection effort when program impact is expected to be measurable, usually one to two years after implementation (this is the pre/post part). A simple example: if reducing violent crime in the designated neighborhood is a goal, the case study might include figures on the number of arrests for crimes involving the use of a weapon for a period immediately preceding Weed and Seed interventions, during the interventions, and afterwards. If these figures are gathered monthly, a simple graph over a timeline will show whether this measure of violent crime is going up or down over time. (For the statistically sophisticated, 50 months or more of these data may be used for time series analysis.) Alternatively, you can compare pre-intervention to post-interventions periods, such as the number of arrests for crimes involving weapons from April to June 2000 compared to the number of arrests for crimes involving weapons from April to June 2002. 3. Doing #2 and an abbreviated version of #1 for (1) a neighborhood which matches your Weed and Seed neighborhood in all relevant aspects (size, population characteristics, Part I crime rate, etc.) or (2) your city (or other jurisdictional boundary) as a whole (this is the comparison area part). In either case, you must gather the same pre/post figures on your relevant outcomes in these areas as you do in the Weed and Seed designated neighborhood. A matched comparison area lends confidence that any changes observed in the designated neighborhood are due to Weed and Seed rather than other potential causal factors (such as new state laws, local campaigns, or weather), but does not address the displacement problem. Using jurisdiction-wide statistics (more accurately, non-designated area jurisdiction-wide statistics) provide some -- but not complete -- information relative to displacement. Conducting the Process Evaluation Step 1: Collect information. What? A local Weed and Seed evaluation should begin with a basic description and subjective assessment of the site's history, implementation, central characteristics (including its goals, organization and management, designated neighborhood characteristics, and resources), competing and complementary efforts, and current operations. This will encompass the information contained in Columns A-D of the conceptual framework. This information is fleshed out in the "blueprints" for documenting program characteristics (Table 1) and monitoring progress (Table 2). Yet as detailed as these blueprints may appear, they are only suggestive of the types of information you will want to collect -- while local Weed and Seed sites have much in common, each also has a unique strategy and local evaluations must be tailored to meet individual site needs. How? There are two primary methods for gathering the data needed to assess program operations: (1) interviewing and/or surveying the full range of stakeholders and partners and (2) gathering and reviewing program documents. Gathering and reviewing program documents. Historical and current program materials are a rich source of information for studying program operations. These should be pulled together and culled for information for the case study. Interviewing and/or surveying stakeholders and partners. Key stakeholders in the Weed and Seed strategy and primary partners and staff (especially those who have been involved since the beginning) hold special knowledge of the program's early history, development and implementation, and current day-to-day activities. Informal interviews should be conducted with a few key individuals to produce an objective summary of the program's history and key events, combined with information gleaned from program documents. If this summary is compiled by essentially one person, as may be the case in a self-evaluation, others should review it for completeness and accuracy. More structured interviews should be conducted with all stakeholders or a good cross-section of them, to gather systematic information about the functioning and dynamics of the Weed and Seed partnership. This step may be best conducted by an independent evaluator in person, as interviews enable one to gather detailed information and ask clarification questions as needed. A written survey mailed to stakeholders enables the information to be gathered anonymously, although the group of respondents is rather small and their identities may be evident from their responses. The information to be gathered is subjective and the nature of the questions is that they require mostly open-ended answers -- another obstacle to written surveys. Table 1 Blue print for Program Documentation Program Characteristics Suggested Details to Capture Goals and Objectives Goals: Broad statements of the program's principal aims or purposes. Examples: (1) To decrease juvenile crime in the designated neighborhood. (2) To improve the quality of neighborhood life. Objectives: Measurable, operational specifications of goals. Examples: (1) To increase the number of nonviolent juveniles referred to Youth Court for alternative sentencing and disposition. (2) To remove abandoned vehicles and trash from neighborhood streets. Designated neighborhood and comparison area characteristics Population, Area in square miles, city blocks. Racial/ethnic, gender, age, household composition. Median family income, % of families below poverty line. Housing: % renter-occupied, % public, vacancy rate, number abandoned. Part I crime rate, assessment of special problems (drug markets, violent crime, etc.). Unemployment rate. School dropout rate, achievement scores. Resources, organizations, and institutions. Program organization and management Organizational Chart, by-laws. Membership and role/responsibilities of Steering Committee and all other committees and subcommittees (W&S committees; law enforcement, community policing, PIT, and restoration committees; planning committee, etc.). Meetings: Date, attendance, minutes. Staff: Position, employing agency, supervisory agency, time devoted to W&S. Training and technical assistance received in building partnerships. Resources available Application of Weed and Seed grant funds, by program area and activities/purpose. From each participating agency: Total funds and in-kind support given to program areas, activities. Competing and complementary anti- crime or anti-drug efforts Descriptions of other programs underway, such as Neighborhood Watch or an anti- drug public service campaign. Changes in law enforcement/criminal justice practices, procedures, or laws. Significant events (e.g., industry shut-down, four teenagers killed in drunk driving accident, etc.). Table 2 Blue print for Program Monitoring Program Areas and Central Activities Process Measures (Data to be collected) General Sources of Data Program Management: Steering Committee and subcommittee activities. Partnership dynamics -- communication, coordination, leadership, etc. Agency resources/contributions to W&S Survey of Steering Committee members. Weeding Law enforcement (examples): High visibility/saturation patrols. Buy-bust operations, controlled buys. Enforcement of nuisance ordinances. Search/arrest warrants. Identification and targeting of serious/ violent offenders, trouble spots. Weapons tracing, seizure. Prosecution (examples): Gun crime prosecutions, enhancements. Federal and state case review. Special case handling. Community prosecution. Drug courts. Probation and parole: Identification and intensified supervision of high risk probationers and parolees. Police-probation home visits, sweeps. Increased services. "Counts" of the number of events (e.g., sweeps), things (e.g., guns seized), people (e.g., serious chronic offenders), etc. Monthly arrest statistics from designated and comparison areas for drug-related crimes, violent crimes, other target crimes. Person-hours spent on spent on specific W&S activities. Case tracking from arrest to disposition. Prosecution statistics for W&S cases: number of defendants charged, released pre-trial; case disposition; average length of sentence; total asset forfeitures, etc. Monthly statistics for W&S cases: probation violations, home visits made, sweeps, jobs secured, education, etc. Routinized record-keeping implemented for W&S. Existing statistics from police and sheriff's departments. Existing or new case management/tracking systems. Existing statistics from prosecutor's offices (county, state, Federal). Existing or new case management/tracking systems. Existing statistics from probation and parole departments (county, state, Federal). Community policing (examples): Problem-solving. Foot, bicycle, alternative patrols. Attending community meetings, working in partnership with residents. Youth activities and programs. Organizational changes to support CP. Crime prevention programs. Person-hours spent on community policing activities in the designated neighborhood. Number of people regularly involved in community anti- crime/drug activities. Number and quality of crime tips received. CBO membership and participation. Routinized record-keeping implemented for W&S. Community survey and/or interviews with key informants. Seeding: Prevention, intervention, and treatment (examples): Safe Havens. Boys and Girls Clubs. Substance abuse services and programs. Alternative activities for youth. School-based prevention programs. Neighborhood restoration (examples): Job programs. Business recruitment and revitalization. Housing improvements. Public works improvements. Clean-up, graffiti eradication. Code enforcement, nuisance abatement. Hours open, staffing, activities of all programs Number and age of people participating in services, frequency, duration, outcome where appropriate. Number and nature of jobs created No. and characteristics of people employed. No. of new businesses opened, small business loans made, units rehabbed, etc. Nature and cost of neighborhood improvements. Number of clean-ups, evictions, etc. Routinized record-keeping implemented for W&S. Existing statistics from prevention, intervention, and treatment programs. Routinized record-keeping implemented for W&S. Examples of Documents to Gather Minutes of early Steering Committee and subcommittee meetings, with attendance lists, positions and agencies of representatives. Documents that put in writing the site's purpose and objectives, such as missions statements and formal goals and objectives. Descriptive accounts of the site's history, such as funding proposals, grant applications, media coverage (e.g., local newspaper articles), annual reports, etc. Program documents that relate to how the site operates. These may include such diverse things as the eligibility criteria for Safe Havens' or housing loan programs; interdepartmental police memos describing the purpose, staffing and operations of a sweep or probation home visits; and the interagency agreement which specifies the role and responsibility of a cross-designated gun prosecutor. Existing statistics on crime, drug use, quality of life issues, social indicators (truancy rates, dropout rates, unemployment, etc.), and city and designated neighborhood demographics. Strive to obtain objective information which enables you to compare pre-Weed and Seed conditions to post-Weed and Seed conditions. Step 2: Analysis and Reporting At this point, you will have collected the data that will form much of your pre/post case study, the data which encompasses a process evaluation of your Weed and Seed activities. It is a worthwhile exercise, at this point, to write the case study (see Putting It All Together) and discuss the process evaluation results among the steering committee and other stakeholders as appropriate. The process information and analysis of it should tell the steering committee (1) whether the Weed and Seed partnership is functioning the best way possible, (2) whether Weed and Seed activities have been implemented as planned, (3) whether Weed and Seed activities are producing the desired immediate effects or not, and (4) where improvements or changes are needed. Areas to Cover in Stakeholder Interviews 1. The respondent's role in W&S, particularly in policy setting, implementation, and day-to-day operations. 2. Views of the membership of the Steering Committee and other key subcommittees -- who is essential, who is missing? 3. Partnership dynamics -- who leads, how are decisions made, how is conflict addressed, etc. 4. Views of key components and policies, including law enforcement tactics, community policing activities, and seeding programs. 5. Views of citizen mobilization and participation. 6. Program strengths and weaknesses. 7. Weed and Seed's impact on the respondent's agency. 8. Local conditions, such as budget constraints and prison overcrowding, that impact the W&S strategy. 9. Suggestions for changes and improvement in the strategy. As an example, see the National Process Evaluation of Operation Weed and Seed (Roehl et al., 1996), which describes the characteristics, activities, and immediate outcomes of the first 19 Weed and Seed sites funded. This report covers partnership characteristics, key activities, and immediate outcomes, but stops short of presenting any impact evaluation results. These may be found in the final report of the national impact evaluation conducted by Dunworth et al. (1999), the National Evaluation of Weed and Seed: Cross-Site Analysis. Assessing Program Impact Step 1: Collect data What? Gathering accurate and unbiased data related to program outcomes is arguably the most difficult part of evaluating a Weed and Seed strategy. Again, outcomes are multiple and diverse, the information sources are many, and specific variables and indicators listed herein -- while significant and common -- are suggestions only. A list of potential outcome measures appears in Table 3, which is an expanded version of Column E in the Conceptual Framework. Outcome measures and variables specific to your program will flow from the identification of goals and objectives and the creation of your own conceptual framework. How? Four major methods are suggested for gathering outcome data. The first two -- gathering criminal justice statistics and conducting resident surveys -- are the most necessary for assessing program objectives. The second two, gathering neighborhood indicators and tracking/testing individuals, provide important information, but may be found only in the most well-funded and sophisticated of evaluations. Gathering criminal justice system statistics. Given the nature of the weeding activities, law enforcement, prosecution, and probation departments will be your main source of outcome data. From law enforcement sources, these data include calls for service (for both serious Part I crimes and quality of life crimes) and reported crime statistics (Part I crimes and additional target crimes, if any). Prosecution sources can provide data on the number of serious offenders sentenced to prison terms. Probation and parole data include the number of probation/parole violations and returns to incarceration. Table 3 Blueprint for Assessing Program Impact Program Areas and Central Activities Outcome Measures (Data to be collected) General Sources of Data Program Management: Steering Committee and subcommittee activities. Partnership institutionalization. Expansion to new designated area or crimes. Partnership satisfaction. Survey of Steering Committee members. Weeding Law enforcement (examples): High visibility/saturation patrols. Buy-bust operations, controlled buys. Enforcement of nuisance ordinances. Search/arrest warrants. Identification and targeting of serious/ violent offenders, trouble spots. Weapons tracing, seizure. Prosecution (examples): Gun crime prosecutions, enhancements. Federal and state case review. Special case handling. Community prosecution. Drug courts. Probation and parole: Identification and intensified supervision of high risk probationers and parolees. Police-probation home visits, sweeps. Increased services. Part I, Part II, drug-related crimes, weapons-related, other target crimes (pre/post). Resident perceptions of fear of crimes, neighborhood satisfaction, quality of life, police-community relations, police and city agency performance, etc. (pre/post changes). Victimization (self-reported). Physical changes in designated neighborhoods. Length of sentences received and served by W&S offenders. Citizen perceptions of safety, "revolving door" of the justice system. Recidivism. Probation/parole violations, returns to incarceration. Long-term productivity, risk resiliency, etc. Existing statistics from police and sheriff's departments. Community surveys, key informant interviews, focus groups. Existing victimization surveys. Systematic observations of physical changes made for W&S. Existing or new case management/tracking systems. Community surveys, key informant interviews, focus groups. Existing or new case management/tracking systems. Community surveys, key informant interviews, focus groups. Community policing (examples): Problem-solving. Foot, bicycle, alternative patrols. Attending community meetings, working in partnership with residents. Youth activities and programs. Organizational changes to support CP. Crime prevention programs. Ongoing citizen involvement. Resident perceptions of police, citizen- police relations, priority of neighborhood problems. Resident perceptions of safety, increase in satisfaction with the neighborhood. Positive use of public places. Community surveys, key informant interviews, focus groups. Systematic observations of physical changes made for W&S. Seeding: Prevention, intervention, and treatment (examples): Safe Havens. Boys and Girls Clubs. Substance abuse services and programs. Alternative activities for youth. School-based prevention programs. Neighborhood restoration (examples): Job programs. Business recruitment and revitalization. Housing improvements. Public works improvements. Clean-up, graffiti eradication. Code enforcement, nuisance abatement. Changes in individuals' self-esteem, risk and protective factors, attitudes. Reduced substance abuse in community. Changes in truancy and dropout rates. Changes in vacancy and abandonment. Permanent jobs, new business. Improved physical and social quality of life. New construction, affordable housing. Increase in property values. Routinized testing by prevention, intervention, and treatment programs. Existing statistics from emergency rooms, treatment programs. Existing statistics from schools. Existing statistics from housing departments, business associations, etc. Remember that plans and agreements have to be made up front, not two or three years into strategy implementation. Law enforcement agencies will have to track arrests, calls for services, reported crimes, etc., for the specific designated neighborhoods and comparison areas, which may or may not fit the department's beat or other reporting area. Monthly figures are best, as they can be combined into quarterly or annual figures as needed. Two other caveats must be mentioned. Law enforcement data are far from perfect; they are affected by changes in departmental policy and susceptible to reporting foibles. Monthly figures are suggested because they also enable you to examine crime data at different points in time -- and sometimes an increase is a mark of success rather than failure. If your efforts begin by concentrating on mobilizing citizens and encouraging them to report crimes of all types, calls for service should increase over the next several months if citizens are responding. Gradually, those calls for service should decrease and level off if the number of actual incidents decreases. Neighborhood surveys and systematic observations. Many of the desired outcomes of a Weed and Seed strategy are to change neighborhoods in positive ways. Some of these changes, such as a reduction in abandoned vehicles and graffiti, can be assessed by making regular observations using rating forms. Changes in other outcomes, particularly significant ones such as fear of crime and perceptions of the quality of neighborhood life, must be assessed by asking the views of those who live and work in the designated neighborhoods (to the extent that fear of crime can be measured in residents' use of public spaces, systematic observations may be used as well). Resident opinions may be gathered in many ways -- through telephone or door-to-door surveys, mailed questionnaires, focus groups, interviews with key neighborhood leaders, community meetings or town halls, brief questionnaires distributed and collected in public places, etc. The most objective methods are to recruit independent interviewers to survey a large (several hundred), representative sample of residents, randomly chosen, by telephone or in person. Most of the other methods are likely to produce more biased answers, as they will be drawn from a smaller group of people, many of them motivated to respond or participate for specific reasons. You must be prepared to conduct your survey in both the designated and comparison areas. The survey instrument should be kept brief and simple, but should cover the topics of fear of crime, victimization, quality of life measures, satisfaction with the neighborhood, changes over time, quality of city services, police-community relations, and perceptions of the police. Participation in the survey should be voluntary, anonymous (except for limited information gathered for verification purposes, if done), and fully informed. A sample survey instrument and observation form are included at the end of this document. Neighborhood-wide indicators. As shown in Table 3, your Weed and Seed strategy may be evaluated by its impact on neighborhood life outside of crime -- by improving school attendance and performance of neighborhood youth, for example, or creating new jobs or housing. In these cases, you will want to gather data about these outcomes which indicate changes attributable to Weed and Seed. These indicators (truancy rates, standardized achievement test scores, number of jobs created, etc.) should be gathered for periods prior to and after implementation of your Weed and Seed strategy. It is often difficult to obtain these indicator data at the neighborhood level. In regard to school data, for example, neighborhood youth certainly attend a variety of schools, and any one school will have summary data only which groups neighborhood youth with others. Individual tracking, monitoring, and testing. Weeding and seeding at the individual level calls for the worst offenders to be removed from the neighborhood, and if they return, to return to a crime- and drug-free, productive life. Thus, a complete comprehensive evaluation might track individuals from arrest, through prosecution and sentencing, to release after incarceration, to an assessment of their lives after prison. Other Weed and Seed activities aim to improve individual behaviors and/or attitudes. Evaluating such activities might require assessing target youth's self-esteem or a high risk juvenile's job readiness pre and post their involvement in Weed and Seed. Step 2: Analysis and Reporting Statistical analyses and presentations of outcome data should be conducted by an experienced researcher. Simple statistical tests of differences between means (such as the average number of arrests during a time period or the average satisfaction rating given to city services by a survey respondent) or observed outcomes (such as the number of people attending neighborhood meetings) should be conducted to assess difference before and after Weed and Seed and between the designated neighborhoods and comparison areas. Continuous data -- such as monthly arrest data over a period of years -- should be graphed to show changes in the designated neighborhoods and comparison areas and perhaps tested by time series analysis. Neighborhood surveys will produce both quantitative information (e.g., a score from 1 to 5 on how safe the respondent feels alone in the neighborhood after dark) and qualitative information (e.g., an answer to an open-ended question about how the neighborhood has changed over the past year). As content analysis of open-ended questions can become laborious, stick to closed-ended questions as much as possible. Putting It All Together Now the tough part (and this is where you will wish you had an independent evaluator if you didn't take that route in the beginning) is writing all this up. The results of your evaluation are important information for your stakeholders, policy makers, staff, and funding agencies. Begin with a concise written narrative of the site's history, implementation, and operations -- in short, the case study. Add to it the immediate and long-term outcome data gathered, highlighting differences before and after major Weed and Seed activities and differences between the designated neighborhood(s) and comparison areas. Present quantitative results in simple tables, x-y graphs, bar graphs, and pie charts, with accompanying statistical tests where appropriate. Through interviews with residents and stakeholders, individual anecdotes may provide a human touch your evaluation findings. Stories of how the neighborhood has changed -- people on the street at night for the first time in years, for example, or a 19-year-old's description of his first non-criminal job -- while not "scientific," may be some of your evaluation's most powerful findings. Your evaluation report should end with a summary of progress, strengths and weaknesses, and activities yet to be accomplished. It may also include recommendations for changes in management, organization, or activities. Your evaluation report should fulfill multiple purposes. A summary of it will serve as the preface for funding proposals, annual reports to sponsors and funders, and informational materials given to the media and other interested parties. This narrative summary should be updated periodically -- at least annually -- as policies, procedures, activities, and outcome data change. Second, the report should be used to strengthen the Weed and Seed strategy. The factual information compiled should be critically examined by the Steering Committee to see if revisions and mid-course corrections are needed. The Steering Committee should ask itself whether the Weed and Seed activities have been provided as intended, what activities are missing and needed, whether the desired target population is being reached, etc. Your evaluation journey should be an enlightening and enjoyable experience. Celebrate your successes! Additional Evaluation Resources EOWS recommends the following resources for additional information/assistance with evaluation: The Justice Research and Statistics Association's (JRSA), Training and Technical Assistance Center provides assistance on a variety of topics, including evaluation and research. The JRSA site also contains the Weed and Seed Data Center. Visit the JRSA website at www.jrsa.org Statistical Analysis Centers (SACs) are state agencies that contribute to effective state policies through statistical services, research, evaluation, and policy analysis. To find contact information for the SAC in your state, visit www.jrsa.org/sac/index.html The following websites provide information on conducting evaluations as well as links to publications and other evaluation resources and websites: Juvenile Justice Evaluation Center Online: www.jrsa.org/jjec/ Bureau of Justice Assistance Evaluation Website: www.bja.evaluationwebsite.org Observation Form for An Evaluation of Safe Streets Now! For pre- and post-observations Justice Research Center, November 1999 Observation Form Project at [problem location address]:__________________________________ ID No:_______ Date:_____________ Start time:___________ End time:____________ Observation No.____ 1. Below, draw a street map of the block you are observing, noting residential and commercial properties, schools (etc.), and significant street places (bus stops, phone booths). [Map needed only on first observation] 2. Is there a "center of activity," where people are congregating (e.g., in front of the problem location, around the phone booth, behind the liquor store)? If so, describe below: Physical observations of the general area of the face block: 1. Is there garbage or broken glass in the street or on the sidewalks? ____ Almost none ____ Yes, but not a lot ____ Yes, quite a bit ____ Yes, almost everywhere 2. Is there paper trash, litter, or junk (e.g., old shoe) in the street or on the sidewalks? ____ Almost none ____ Yes, but not a lot ____ Yes, quite a bit ____ Yes, almost everywhere 3. Are there cigarette or cigar butts or discarded cigarette packages on the sidewalk or in the gutters? ____ Almost none ____ Yes, but not a lot ____ Yes, quite a bit ____ Yes, almost everywhere 4. Are there any needles, syringes, condoms, or drug-related paraphernalia on the sidewalk, in gutters, or on the street? ____ Almost none ____ Yes, but not a lot ____ Yes, quite a bit ____ Yes, almost everywhere 5. Are there empty beer or liquor bottles visible in streets, yards, or alleys? ____ Almost none ____ Yes, but not a lot ____ Yes, quite a bit ____ Yes, almost everywhere 6. Is there much graffiti? ____ Almost none ____ Yes, but not a lot ____ Yes, quite a bit ____ Yes, almost everywhere Traffic: 7. On the face block (both sides), record the number of: parked cars 0 1 2 3 4 5 6-10 11+ mobile cars 0 1 2 3 4 5 6-10 11+ abandoned cars 0 1 2 3 4 5 6-10 11+ delivery vans 0 1 2 3 4 5 6-10 11+ 8. Volume of traffic: ____ No traffic ____ Very light traffic ____ Light ____ Moderate ____ Heavy ____ Very heavy Security and Safety: 9. Are any public police visible (check all that apply)? ____ Police mobile patrol car visible ____ Police foot patrol visible ____ Mounted police patrol visible ____ No police visible 10. Are any private security personnel visible? ____ Yes ____ No 11. Are any school crossing guards present? ____ Yes ____ No Social Observations on Face Block: Tally the number of people engaging in the following behaviors, which are not mutually exclusive: 12 13 14 15 Pre-school children playing in: a private yard the street a school yard public park or playground Supervised Not supervised Males Females 16 People sitting, working, or playing in their yards, or on the porch, etc., on their own property. 17 Elderly people stopping to talk, greeting one another 18 Adults stopping to talk, greeting one another 19 Youth stopping to talk, greeting one another 20 Mixed ages stopping to talk, greeting one another 21 Adults loitering, congregating by (enter place): 22 Youth loitering, congregating by (enter place): 23 Youth carrying or playing "boom boxes" 24 Homeless people or panhandlers 25 People selling drugs 26 People drinking, or obviously intoxicated 27 People smoking 28 People on bus stop, waiting to get on or just getting off 29 People talking on pay phones 30 31 32 Pedestrian traffic: Adults Youth Children 33 34 35 People on bicycles: Adults Youth Children 36 People going in and out of commercial establishments 37 Other behaviors: Draft Community Survey for the Weed and Seed Baseline Data Collection Project Institute for Social Analysis, January 1995 Weed and Seed Community Survey Basic Interview Data ISA ID No._______________ Street Address:________________________________ Apt. No.________ Site: __________ ___________ Day and time of contacts: __________ ___________ __________ ___________ 1.__________________________ __________ ___________ 2.__________________________ 3.__________________________ Neighborhood:______________________ 4.__________________________ (name) 5.__________________________ Designated Neighborhood Comparison Area(s) (circle date of completion) Interviewer:____________________ Perceptions of the Neighborhood 1. How long have you lived in the neighborhood? a. Years_________ b. Months_______ c. Since_________ (translate later to years and/ months) d. Don't Know. . . . 99 (circle) 2. In general, in the past two years, would you say this neighborhood has become a better place to live, a worse place to live, or stayed the same? (circle number response) Better . . . . . . . .3 Worse. . . . . . . . .1 About the same . . . .2 Did not live here two years ago8 Don't know . . . . . .9 3. In some neighborhoods, people do things together and help each other. In other neighborhoods, people mostly go their own way. In general, what kind of neighborhood would you say this is? People help each other2 People go their own way1 Don't know . . . . . .9 4. Has this changed in the past two years, meaning do people now help each other more or less than they did two years ago? More . . . . . . . . .3 Less . . . . . . . . .1 No change. . . . . . .2 Don't know . . . . . .9 5. How many people in this neighborhood do you know and trust well enough to ask them to watch your house when you are gone? No one . . . . . . . .1 A few. . . . . . . . .2 Many . . . . . . . . .3 Don't know . . . . . .9 6. Compared to two years ago, do you know and trust more, fewer, or about the same number of neighbors as today? More . . . . . . . . .3 Fewer. . . . . . . . .1 About the same . . . .2 Did not live here two years ago8 Don't know . . . . . .9 Now I am going to read a list of things that you may think are problems in this neighborhood. After I read each one, please tell me whether you think it is a serious problem, moderate problem, minor problem, or almost no problem in this neighborhood. Serious Moderate Minor Almost no Don't problem problem problem problem know 7. The first one is, police not making enough contact with residents . . . . . . . . . .1 2 3 4 9 8. Police stopping too many people on the streets without good reason . . . . . . . . . 1 2 3 4 9 9. Litter and trash on the streets and sidewalks . . . . . . . . . . . . . . . . . . . . . . . . . 1 2 3 4 9 10. Public drinking or gambling . . . . . . . . . . .1 2 3 4 9 11. Youth disruption -- young people hanging out, vandalizing, making noise . . 1 2 3 4 9 Serious Moderate Minor Almost no Don't problem problem problem problem know 12. Drug dealers on streets, street corners, or in other public places . . . . . . . . . . . . . . . . 1 2 3 4 9 12. Drug sales in commercial establishments (stores, etc.) . . . . . . . . . . . . . . . . . . . . . . . 1 2 3 4 9 13. Drug sales out of homes or apartments . . .1 2 3 4 9 14. Burglary and other property crime . . . . . .1 2 3 4 9 15. Robbery and other street crime . . . . . . . . .1 2 3 4 9 16. Gang activity . . . . . . . . . . . . . . . . . . . . . .1 2 3 4 9 17. Drug use . . . . . . . . . . . . . . . . . . . . . . . . . .1 2 3 4 9 Compared to two years ago, would you say the following problems have gotten better, worse, or stayed the same? [Did not live here two years ago . . . . . . . . . 8 (skip to 31)] About Don't Better Worse the same know 18. The first one is, police not making enough contact with residents . . . . . . . . . . . . . . 3 1 2 9 19. Police stopping too many people on the streets without good reason . . . . . . . . . . . . . 3 1 2 9 20. Litter and trash on the streets and sidewalks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 2 9 21. Public drinking or gambling . . . . . . . . . . . . . . . 3 1 2 9 22. Youth disruption- young people hanging out, vandalizing, making noise . . . . . . . . . . . . . 3 1 2 9 24. Drug dealers on streets, street corners, or in other public places . . . . . . . . . . . . . . . . . . . . 3 1 2 9 25. Drug sales in commercial establishments (stores, etc.) . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 2 9 26. Drug sales out of homes or apartments . . . . . . . 3 1 2 9 About Don't Better Worse the same know 27. Burglary and other property crime . . . . . . . . . . . 3 1 2 9 28. Robbery and other street crime . . . . . . . . . . . . . . 3 1 2 9 29. Gang activity . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 2 9 30. Drug use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1 2 9 Neighborhood Empowerment and Informal Social Control 31. Today, if some 12 year old youth were spray painting a wall in this neighborhood, how likely is it that a resident would tell them to stop? Very likely. . . . . .4 Somewhat likely. . . .3 Somewhat likely, or. .2 Not likely at all? . .1 Don't know . . . . . .9 32. Today, if there was a problem needing some services from a city agency, how likely is it that residents in this neighborhood would take steps to get the problem solved? Very likely. . . . . .4 Somewhat likely. . . .3 Somewhat likely, or. .2 Not likely at all? . .1 Don't know . . . . . .9 33. Today, how do you feel about this neighborhood as a place to live? Are you . . . Satisfied. . . . . . .4 Somewhat satisfied . .3 Somewhat dissatisfied, or2 Dissatisfied?. . . . .1 Don't know . . . . . .9 34. Compared to two years ago, do you feel it is more or less likely that a resident would tell some 12 year old youth spray painting a wall in this neighborhood to stop? More likely. . . . . .3 Less likely. . . . . .1 No change. . . . . . .2 Did not live here two years ago8 Don't know . . . . . .9 35. Compared to two years ago, do you feel it is more or less likely that residents in this neighborhood would take steps to get a problem needing some services from a city agency solved? More likely. . . . . .3 Less likely. . . . . .1 No change. . . . . . .2 Did not live here two years ago8 Don't know . . . . . .9 36. Compared to two years ago, are you more or less satisfied with this neighborhood as a place to live? Are you . . . More satisfied . . . .4 Less satisfied . . . .3 No change. . . . . . .2 Did not live here two years ago8 Don't know . . . . . .9 Fear of Crime 37. Today, how safe do you feel out alone in this neighborhood during the day? Very safe. . . . . . .4 Somewhat safe. . . . .2 Somewhat unsafe, or. .3 Very unsafe? . . . . .1 Don't know . . . . . .9 38. Today, how safe do you feel out alone in this neighborhood after dark? Very safe. . . . . . .4 Somewhat safe. . . . .2 Somewhat unsafe, or. .3 Very unsafe? . . . . .1 Don't know . . . . . .9 39. Today, how often does worry about crime prevent you from visiting public parks or other neighborhood places? Would you say . . . Very often . . . . . .4 Somewhat often . . . .3 Rarely, or . . . . . .2 Never at all?. . . . .1 Don't know . . . . . .9 40. Compared to two years ago, do you now feel more or less safe out alone in this neighborhood during the day? More safe. . . . . . .3 Less safe. . . . . . .1 No change. . . . . . .2 Did not live here two years ago8 Don't know . . . . . .9 41. Compared to two years ago, do you now feel more or less safe out alone in this neighborhood after dark? More safe. . . . . . .3 Less safe. . . . . . .1 No change. . . . . . .2 Did not live here two years ago8 Don't know . . . . . .9 42. Compared to two years ago, does worry about crime prevent you from visiting public parks or neighborhood places more or less often now? More often . . . . . .3 Less often . . . . . .1 No difference. . . . .2 Did not live here two years ago8 Don't know . . . . . .9 Victimization Next, I would like to ask you about some things which may have happened to you or your family in the past two years. As I read each one, please think carefully and tell me if it happened since the end of 1992. 43. Since the beginning of 1992, has anyone broken into your home, garage, or another building on your property to steal something? Yes. . . . . . . . . .1 No . . . . . . . . . .2 Don't know . . . . . .9 44. Other than that, have you found any sign that someone tried to break into your home, garage, or another building on your property to steal something? Yes. . . . . . . . . .1 No . . . . . . . . . .2 Don't know . . . . . .9 45. In the past two years, has anyone stolen something from you or a member of your family by force or threatening you with harm in this neighborhood? Yes. . . . . . . . . .1 No . . . . . . . . . .2 Don't know . . . . . .9 46. In the past two years, has anyone tried to steal something from you or a member of your family by using force or threatening you with harm in this neighborhood? Yes. . . . . . . . . .1 No . . . . . . . . . .2 Don't know . . . . . .9 47. In the past two years, has anyone beaten you (or a member of your family) up, attacked you, or hit you with something, such as a rock or a bottle? (other than the incidents already mentioned) Yes. . . . . . . . . .1 No . . . . . . . . . .2 Don't know . . . . . .9 48. In the past two years, have you or a member of your family been knifed, shot at, or attacked with some other weapon by anyone at all? (other than incidents already mentioned) Yes. . . . . . . . . .1 No . . . . . . . . . .2 Don't know . . . . . .9 Police Response 49. In the past two years, have you called the police to report a crime that happened to you or a member of your family? (include all calls to police concerning the incidents you have already told me about) Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 50. (If yes to #49). How many times? _____________ 51. (If yes to #49). How satisfied were you with the police response? Very satisfied . . . .4 Somewhat satisfied . .3 Somewhat dissatisfied, or2 Very dissatisfied. . .1 Don't know . . . . . .9 52. In the past two years, have you called the police to report a crime that did not involve you or a member of your family? Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 53. (If yes to #52). How many times? __________ 54. (If yes to #52). How satisfied were you with the police response? Very satisfied . . . .4 Somewhat satisfied . .3 Somewhat dissatisfied, or2 Very dissatisfied. . .1 Don't know . . . . . .9 55. In the past two years, have you called the police to report a problem or disturbance (not a crime) in your neighborhood? Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 56. (If yes to #55). How many times? __________ 57. (If yes to #55). How satisfied were you with the police response? Very satisfied . . . .4 Somewhat satisfied . .3 Somewhat dissatisfied, or2 Very dissatisfied. . .1 Don't know . . . . . .9 58. Today, how responsive are the police in this neighborhood to community concerns? Are they . . . Very responsive. . . .4 Somewhat responsive. .3 Somewhat unresponsive, or2 Very unresponsive. . .1 Don't know . . . . . .9 59. Today, how good a job are the police doing to keep order on the streets and sidewalks in this neighborhood? Would you say they are doing a . . . Very good job. . . . .4 Good job . . . . . . .3 Fair job, or . . . . .2 Poor job?. . . . . . .1 Don't know . . . . . .9 60. Today, how good a job are the police doing in controlling the street sale and use of illegal drugs in this neighborhood? Would you say they are doing a.... Very good job. . . . .4 Good job . . . . . . .3 Fair job, or . . . . .2 Poor job?. . . . . . .1 Don't know . . . . . .9 61. Compared to two years ago, are the police in this neighborhood today more or less responsive to community concerns? Are they . . . More responsive. . . .3 Less responsive. . . .1 No change. . . . . . .2 Did not live here two years ago8 Don't know . . . . . .9 62. Compared to two years ago, are the police doing a better or worse job in keeping order on the streets and sidewalks in this neighborhood? Would you say they are doing a . . . A better job . . . . .3 A worse job. . . . . .1 No change. . . . . . .2 Did not live here two years ago8 Don't know . . . . . .9 63. Compared to two years ago, are the police doing a better or worse job in controlling the street sale and use of illegal drugs in this neighborhood? Would you say they are doing a . . . A better job . . . . .3 A worse job. . . . . .1 No change. . . . . . .2 Did not live here two years ago8 Don't know . . . . . .9 64. During the past six months, have you talked to a police officer in this neighborhood about issues or concerns? Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 65. (If yes to #64) Over the past six months, how often would you estimate you have talked to an officer about neighborhood concerns? Several times a week .6 Once a week. . . . . .5 Every other week . . .4 Once a month . . . . .3 Two or three times . .2 Once . . . . . . . . .1 Don't know . . . . . .9 Here are a few specific situations in which you might have seen the police. During the past month, have you seen... Don't Yes No know 66. A police car driving around your neighborhood? . . . . . . . 2 1 9 67. A police officer walking around or standing on patrol in the neighborhood? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 9 68. Police officers patrolling in the alley, or checking garages in the back of buildings? . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 9 69. A police officer chatting/ having friendly conversation with people in the neighborhood? . . . . . . . . . . . . . . . . . . . . . . 2 1 9 70. Police officers arresting someone for buying or selling drugs in the neighborhood? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 9 Community Involvement 71. During the past two years, have you heard about any community meeting newly organized to deal with local problems? Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 72. Have you attended any of these meetings? Yes. . . . . . . . . .2 No . . . . . . . . . .1 73. Was anyone from the police department at any of these meetings? Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 74. Other than these meetings we just covered, have you attended meetings of established community groups that deal with local problems in the past two years? Yes. . . . . . . . . .2 No . . . . . . . . . .1 75. Was anyone from the police department at any of these meetings? Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 76. During the past two years, have there been any social get-togethers, like block parties, or other large social events in this neighborhood? Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 77. Have you attended any of those events? Yes. . . . . . . . . .2 No . . . . . . . . . .1 78. Was anyone from the police department at any of those events? Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 79. During the past two years, have you attended or participated in any of the following events in this neighborhood? Don't Yes No know a. Community fair . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 9 b. Anti-drug rally, vigil, or march . . . . . . . . . . . . . 2 1 9 c. Neighborhood clean-up project . . . . . . . . . . . . . 2 1 9 d. Citizen patrol . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 9 e. Organized observations of drug activity . . . . . . 2 1 9 f. Perceptions of City Services and Other Programs 80. Today, how satisfied are you with city service such as street cleaning and garbage pick-up in this neighborhood? Very satisfied . . . .4 Somewhat satisfied . .3 Somewhat dissatisfied, or2 Very dissatisfied. . .1 Don't know . . . . . .9 81. Today, how satisfied are you with the availability of sports and recreation programs for youth in this neighborhood? Very satisfied . . . .4 Somewhat satisfied . .3 Somewhat dissatisfied, or2 Very dissatisfied. . .1 Don't know . . . . . .9 82. Compared to two years ago, are you more or less satisfied with city services such as street cleaning and garbage pick-up in this neighborhood? More satisfied . . . .3 Less satisfied . . . .2 No change. . . . . . .1 Did not live here two years ago8 Don't know . . . . . .9 83. Compared to two years ago, are you more or less satisfied with the availability of sports and recreation programs for youth in this neighborhood? More satisfied . . . .3 Less satisfied . . . .2 No change. . . . . . .1 Did not live here two years ago8 Don't know . . . . . .9 Perceptions of the Weed and Seed Program 84. Have you heard of a programs called the Weed and Seed Program? Yes. . . . . . . . . .2 No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 [skip to #86] Don't know . . . . . .9 85. Can you tell me what the Weed and Seed Program is all about? [Interviewer to code references to any of the following]. Anything else? Mentioned Not mentioned Increased drug enforcement . . . . . . . . . . . . . . . . . . . . . . 1 9 Increase in arrests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 9 More vigorous policing . . . . . . . . . . . . . . . . . . . . . . . . . . 1 9 Community policing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 9 Prevention, intervention, or treatment services . . . . . . . . 1 9 Neighborhood revitalization . . . . . . . . . . . . . . . . . . . . . . 1 9 86. In the past two years, in your opinion, have the police gotten tougher (i.e., increased arrests) about drug dealing in this neighborhood? Yes. . . . . . . . . .2 No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 [skip to #88] Don't know . . . . . .9 87. Has this increased police activity reduced drug dealing in this neighborhood? Yes. . . . . . . . . .2 No . . . . . . . . . .1 Don't know . . . . . .9 88. If someone in this neighborhood is arrested for a crime, are they more apt to serve time in jail or prison or more apt to be released and returned to the neighborhood? Serve time . . . . . .2 Be released. . . . . .1 Don't know . . . . . .9 89. Compared to two years ago, are there more, less, or no change in the number of programs and activities for youth in this neighborhood? More . . . . . . . . .3 Less . . . . . . . . .1 No change. . . . . . .2 Don't know . . . . . .9 90. Compared to two years ago, is there more, less, or no change in the availability of drug treatment services in this neighborhood? More . . . . . . . . .3 Less . . . . . . . . .1 No change. . . . . . .2 Don't know . . . . . .9 91. Compared to two years ago, are there more, less, or no change in the number of jobs in this neighborhood? More . . . . . . . . .3 Less . . . . . . . . .1 No change. . . . . . .2 Don't know . . . . . .9 Respondent Information That is all the survey question I have. I would like to ask several questions about you, for statistical purposes only. 92. In what year were you born? Year___________ Refused. . . . . . . .8 93. Are you presently employed ful-time, part-time, a student, a homemaker, or unemployed? [Circle one or two categories as needed.] Working full-time. . .1 Working part-time. . .2 Homemaker. . . . . . .3 Unemployed . . . . . .4 Retired. . . . . . . .5 Disabled . . . . . . .6 Full-time student. . .7 Part-time student. . .8 Other. . . . . . . . .9 Refused. . . . . . . .88 Don't know . . . . . .99 94. How many people under 18 years old live here? Number of children_______________ Refused. . . . . . . .8 Don't know . . . . . .9 [Answer #95 and #96 by observation only if obvious] 95. What is your racial or ethnic background? Are you . . . Black. . . . . . . . .1 White. . . . . . . . .2 Hispanic . . . . . . .3 Asian/Pacific Islander4 American Indian. . . .5 Something else?. . . .6 Refused. . . . . . . .8 Don't know . . . . . .9 96. Respondent sex: Male . . . . . . . . .1 Female . . . . . . . .2 97. We would also like to have an idea about the total income of all the people living here. Here is a card (give card to respondent) with some general categories on it. Please tell me which category includes your total household income (what everyone here made together last year). You don't have to give me the actual total- just tell me the correct number. 1. (Under $2,000). .1 2. ($2,000-$3,999) .2 3. ($4,000-$4,999) .3 4. ($5,000-$5,999) .4 5. ($6,000-$7,999) .5 6. ($8,000-$9,999) .6 7. ($10,000-$14,999)7 8. ($15,000-$19,999)8 9. ($20,000-$29,999)9 10. ($30,000-$39,999)10 11. ($40,000-$59,999)11 12. ($60,000 and over)12 Refused. . . . . . . .88 Don't know . . . . . .99 98. [If refused or don't know] Would you just indicate if it was under $XX,XXX or over $XX,XXX (poverty level)? Under $XX,XXX. . . . .1 Over $XX,XXX . . . . .2 Refused. . . . . . . .8 Don't know . . . . . .9 99. Now, in case my supervisor wants to call and verify this interview could I please have your telephone number? Number________________________ Refused. . . . . . . .8 No phone . . . . . . .2 That is all. Thank you very much. You have been very helpful. Interviewer Observations While standing at the front door of the building unit, did you see any of the following in the immediate area (within 50 feet) where you did this interview? 100. Litter or trash on the street, sidewalk, or grounds: No litter. . . . . . .4 A few pieces . . . . .3 Several pieces . . . .2 Small piles. . . . . .1 101. Graffiti on buildings, fences, etc.: No graffiti. . . . . .4 A few words. . . . . .3 A substantial amount .2 Complete wall coverage1 102. A vacant building, but not boarded up: Yes. . . . . . . . . .1 No . . . . . . . . . .2 103. A boarded up building: Yes. . . . . . . . . .1 No . . . . . . . . . .2 104. A seemingly abandoned car: Yes. . . . . . . . . .1 No . . . . . . . . . .2 105. People congregating/loitering for several minutes or more: Yes. . . . . . . . . .1 No . . . . . . . . . .2 If yes, number of people______________ Are they mostly male or mostly female? Mostly male. . . . . .1 Mostly female. . . . .2