EXAMPLES OF QUESTIONS TO ASK,  EVENTS TO OBSERVE, AND WHO OR WHAT TO COUNT

How These Examples Are Organized

Questions for Personal Interviews

Questions for Focus Groups

Events to Observe During Participant-Observation

Who or What to Count During Process Evaluation

Closed-Ended Questions and Statements

6. Page Number for Examples of Questions to Ask, Events to Observe, and Who or What to Count During Evaluation of Programs to Prevent Unintentional Injury

EXAMPLES OF QUESTIONS TO ASK,  EVENTS TO OBSERVE, AND WHO OR WHAT TO COUNT

HOW THESE EXAMPLES ARE RE ORGANIZED

The examples are divided into four categories: "Questions for Personal Interviews," page 76, "Questions for Focus Groups," page 81, "Events to Observe During Participant-Observation," page 89, and "Who or What to Count During Process Evaluation," page 98.

Each of these categories is further divided into items suitable for each of the 12 types of programs to prevent unintentional injury that CDC currently funds (see below). The programs range from those that are large-scale and reach many people to those that are small-scale and reach individual people.

1. Programs to Build Infrastructure (e.g., a state injurycontrol program):

• Getting Budgetary Support

• Institutionalizing a Program (i.e., getting an injuryprevention program included on the state’s health agenda)

2. Programs to Build Coalitions (e.g., advisory committees of government, community, or business organizations)

3. Programs to Develop or Improve Injury-Prevention Legislation (e.g., programs to support bicycle-helmet legislation)

4. Public Information or Advertising Campaigns (e.g., programs to increase awareness of the necessity for compliance with seatbelt laws)

5. Programs for Environmental Change (e.g., to increase the number of homes with smoke detectors or to re-engineer roadways)

6. Programs to Develop or Modify Products (e.g., programs to make bicycle helmets lighter and cooler)

7. Programs to Establish Data Bases or Record Systems (e.g., programs to set up a uniform data gathering system in all local hospitals)

8. Formal Education Programs (e.g., safety classes in elementary or high schools)

9. Training and Development Programs for Service Personnel (e.g., programs to train law-enforcement officials, emergency medical services [EMS] personnel, hospital medical staff, injury-control program personnel, and beverage servers)

10. Programs to Distribute Safety Products (e.g., programs to lend child safety seats to low-income families)

11. Programs to Enforce Safety Laws (e.g., programs to enforce DWI laws):

• Checkpoints for DWI (driving while intoxicated)

• Assessment and Treatment for DWI

• Periodic Increases in Enforcing Selected Laws (e.g., increases in speed-limit enforcement or checkpoints to inspect seatbelt use)

12. Programs to Change Individual Behavior (e.g., programs to encourage children to wear bicycle helmets):

• Counseling and Therapy

• Incentive Programs (e.g., giving prizes to children who wear bicycle helmets)

• Skills Training (e.g., teaching children to stop, drop, and roll if their clothes catch fire)

Table 6, which is essentially a table of contents, will help readers find suitable examples quickly. To use this table, first look through the list of CDC-funded programs for the program that most closely fits yours. Then refer to the program in Table 6 to find the page number with examples in whichever category is appropriate for your evaluation: personal-interview questions, focus-group questions, events to observe during participant-observation, or what to count during process evaluation.

In the examples, we occasionally refer to injury-prevention products (e.g., bicycle helmets, car occupant restraints, and smoke detectors). However, the questions can be adapted for use in evaluations of programs involving any type of injury prevention device.

QUESTIONS FOR PERSONAL INTERVIEWS

The following are examples of questions that an evaluator might use during personal interviews. The most appropriate people to answer the suggested questions are listed in italics under the type of program.

1. Programs to Build Infrastructure: Getting Budgetary Support
Interviewees: State government personnel in budget or injury areas

• Who were the main people involved in obtaining budgetary support for injury control?

• How did these people first come together?

• What steps did they take to start getting a budget?

• What were some of the competing budgetary demands that had to be addressed?

Programs to Build Infrastructure: Institutionalizing a Program
Interviewees: Health department staff, injury-prevention staff

• How was it decided that a bicycle-helmet program should become a part of the Health Department?

• How was the idea initially received by your colleagues? Other staff members? Other department personnel?

• What were the barriers to creating a standing program?

• What concerned people about establishing such a program?

2. Programs to Build Coalitions
Interviewees: Coalition members

• Tell me about the earliest days of the coalition. How did it get started?

• What would you change about the start-up of the coalition? (If appropriate) Have those changes occurred now?

• What do you remember about your first contact with the coalition?

• Were people supportive of the idea from the beginning? How did their support change? Probe: Why?

3. Programs to Develop or Improve Injury-Prevention Legislation
Interviewees: Members of the general public

• Tell me about the state's new bicycle-helmet or seatbelt law.

• Do you think it is a good idea? Probe: Why? Why not?

• What do people you know say about the law?

4. Public Information or Advertising Campaigns
Interviewees: Members of the general public

• Have you noticed the new television advertisements to get people to use [bicycle helmets, smoke detectors, occupant restraints]?

• What do you think of the new advertisements?

• What have you heard about the advertisements from other people?

5. Programs for Environmental Change 
Interviewees: Consumers, members of the general public

• Is this a good location for the smoke-detector sales display? Probe: Why? Why not?

• How do these bicycle helmets feel compared with others you have worn? Probe: Why?

• Is this seatbelt any different from others you have used? Probe: How is it different? Is it preferable?

6. Programs to Develop or Modify Products
Interviewees: Consumers of the products

• Which of the following bicycle helmets do you like best? Probe: Why? Why not?

• What do you think is unattractive about bicycle helmets? Probe: Why?

• What makes smoke detectors difficult to install?

7. Programs to Establish Data Bases or Record Systems
Interviewees: Potential users

•  If you had an ideal record system, what type of information would you want from it?

• What information should this data base contain (at a minimum)?

• What other record systems would you want this ideal record system to link with?

• What information should be collected so the new system can be linked with other record systems?

• Have you tried to use the records or forms developed for the new system? Probe: What was your experience? What can be improved?

• Have you tried to access the data base? Probe: What was your experience? Why did you need the data? What can be improved?

8. Formal Education Programs
Interviewees: Students

• If I were in your health classes on bicycle safety, what would I see you doing?

• What did you like best about the bicycle-safety program that the police came and did at your school? Probe: Why?

• Was the Fire Department's fire-safety class fun? Probe: Why? Why not?

• How well did the module on using seatbelts work?

• What difficulties did you encounter?

• Was anything missing from the curriculum that you expected to cover? Probe: If so, what?

9. Training and Development Programs for Service Personnel 
Interviewees: Law-enforcement officials

• Have you attempted to enforce the new legislation? Probe: What was your experience? What problems did you have?

• What additional support do you need to enforce the legislation effectively?

• What information do you need in order to educate drivers about the importance of using seatbelts?

• What do you think of the training materials?

• What was your experience when you were stopped by the police for failure to wear your seatbelt? Probe: Did the officer talk to you about safety?

• Describe a typical run to pick up a burn victim. Probe: Did you take any actions to prevent an inappropriate incident from re-occurring?

• If you were to talk to children about the importance of bicycle helmets, what would you tell them?

• What information might we provide that would help you do your job better?

• What resources do you need to be most effective in your work?

• What does your EMS team do least well? Probe: Why?

• When children are brought in with bicycle-related injuries other than to the head, how might you talk to them about the importance of helmet use?

• When injury victims are released from the hospital, how might you provide information to prevent a future incident?

• With regard to emergency response, what is this hospital’s greatest shortcoming?

• What would you like to know about occupant restraints and occupant-restraint programs?

• What skills will be the most helpful to you in creating and running the injury-control program?

• How will you be able to use the information provided so far in this course?

• What have you learned about the relationship between drinking and seatbelt use?

• Will the information and skills we have provided influence the way you do your job? Probe: How?

• Is there information we have not provided that you would like to receive?

10. Programs to Distribute Safety Products
Interviewees: Members of the program’s target population

• What have you heard about the state's new smoke detector distribution program?

• Do you think people will take advantage of the program? Probe: Why? Why not?

• How might the program be changed to make it better?

11. Programs to Enforce Safety Laws: Checkpoints for DWI
Interviewees: People stopped at the checkpoint

• What was your experience when you were stopped for a breath test? Probe: Was any part of it bad?

• What did you hear about the police setting up roadblocks to give breath tests?

• Do you believe that roadblocks will reduce driving under the influence? Probe: Why? Why not?

Programs to Enforce Safety Laws: Assessment and Treatment for DWI
Interviewees: People sent for assessment or treatment

• How were you referred to the DWI treatment program? What did you know about it before you came?

• What was your experience when you first contacted the DWI assessment program?

• Was any part of your experience difficult or uncomfortable?
  Probe: What was difficult or uncomfortable?

• Do you believe that attending a program like this will stop people from driving while intoxicated? Probe: Why? Why not?

• If you had designed this DWI program, how would it be different?

Programs to Enforce Safety Laws: Periodic Increases in Enforcing Selected Laws
Interviewees: People stopped for speeding

• Tell me about being stopped for speeding. Probe: How were you treated?

• What have you heard about the crackdown on speeding in the area? Probe: Do people seem to think it is working?

12. Programs to Change Individual Behavior: Counseling and Therapy
Interviewees: People counseled

• Do you think your bicycle crash could have been prevented? Probe: Why? Why not?

• What has prevented you from getting or using a child safety seat for your child?

• Did the emergency-department nurse talk to you about how to avoid this type of injury? Probe: What were you told?

Programs to Change Individual Behavior: Incentive Programs
Interviewees: Members of the program’s target population

• What factors influenced you to buy the smoke detector?

• What rewards or prizes would make it worthwhile for you to wear your bicycle helmet?

• How did you hear about the reduction in homeowner's insurance for people with properly installed smoke detectors?

Programs to Change Individual Behavior: Skills Training
Interviewees: Trainees

• What did you learn about installing the child safety seat that you did not know before?

• What were you least comfortable with before the training?

• How has that changed since the training?

QUESTIONS FOR FOCUS GROUPS

The following are examples of questions that an evaluator might use during focus groups. The most appropriate people to invite as group members are in italics under the type of program.

1. Programs to Build Infrastructure: Getting Budgetary Support
Group members: Government personnel and injury program personnel

• How did the idea of obtaining budgetary support for an injury-prevention program come about?

• How did you begin the process of obtaining funding? If you were to do it over, what would you do differently? Probe: Why?

• What actions have you taken to obtain funding? Probe: Tell me about [an action that interviewee described].

• What have you learned so far about how to obtain budgetary support for injury programs?

• Who would you say are the key people involved in the effort to obtain budgetary support for injury control? Probe: How did they become involved in this effort?

• How would you describe the legislative view on injury control when your efforts began? Has this view changed during the course of your efforts? Probe: Describe how it has changed.

• What have you done to get people involved in your efforts to obtain funding for injury-prevention activities?

• What strategies have you found to be most effective in obtaining funding and other types of support?

• What strategies to obtain support and funding have had problems? Probe: Tell me about the problems.

Programs to Build Infrastructure: Institutionalizing a Program
Group members: Government personnel

• Of all the steps you have taken, which would you say were the most valuable in moving you toward institutionalizing the injury program?

• What would you tell other programs to be sure to do when they attempt to institutionalize an injury program?

• What would you tell them to avoid?

• Who have you found to be most important in helping to make injury prevention a permanent program?

• What barriers have you encountered to incorporating injury control into the organizational structure?

• What have you done well to make this a permanent program? What would you do differently?

2. Programs to Build Coalitions 
Group members: Coalition members

• How did you become involved in the coalition?

• Has your work with the coalition had any particularly positive moments? Probe: Tell me about one.

• Has your work with the coalition had any unpleasant moments? Probe: Tell me about one.

• Describe some of the barriers the coalition faced in coming together.

• What insights did you gain from overcoming the barriers?

• What about the coalition has worked well? Probe: What do you think made it work well?

3. Programs to Develop or Improve Injury-Prevention Legislation
Group members: Members of the general public

• When did you first hear about the state's new bicycle helmet law?

• What was your reaction to the law?

• Why do you think the law was created?

• Do you think the law is making a difference? Probe: Why? Why not?

4. Public Information or Advertising Campaigns
Group members: Members of the general public

• What do you think of the new advertisements to get people to use occupant restraints?

• Do you think more people will use [seatbelts, bicycle helmets, smoke detectors] because of the campaign?

Probe: Why? Why not?

• Who do you think is most likely to notice the advertisements?

• What have you heard about the campaign from children? Other adults?

5. Programs for Environmental Change
Group members: Engineers

• What was the most difficult part of re-engineering the road?

• Describe your experience of being involved in an environmental change. Probe: What was least satisfying about your involvement? Most satisfying?

• Is there anything you dislike about the new highway design?

• Since the changes, what are people saying about the road?

• What do you notice most about the new road?

6. Programs to Develop or Modify Products
Group members: Injury-control personnel

• What barriers did you encounter in distributing the free smoke detectors?

• What would you do differently in a similar situation in the future?

• Did you experience any false starts when you were developing or modifying the child safety seats? Probe: Describe what went wrong.

• What did you do particularly well when developing your bicycle helmets? Can some aspects be further improved?

• Have you had any problems with the new child safety seats? Probe: Describe them.

• What might the engineers do to improve these seats?

• What do you think of the new bicycle helmets?

• Is there anything you would like to see changed about the new helmets? Probe: What?

• Is there anything about the helmets that you liked better before the change? Probe: What?

7. Programs to Establish Data Bases or Record Systems
Group members: Data collectors

• What resources does your county need in order to participate in the state injury record system?

• What problems has participation in the state record system caused for you?

• Are the monthly reports helpful to you? Probe: How have you used them?

• What information would you like to be able to get from an injury-data system?

8. Formal Education Programs
Group members: Students

• Why do you think some people your age don't wear their seatbelts?

• Which of these bicycle helmets do you like best?
  Probe: Why?

• Was any part of the fire-safety program boring?
  Probe: What made it boring?

• Which activities of the fire-safety class did the children seem to enjoy most? Why do you think that is?

• What feedback have you gotten from the children about the bicycle-safety unit?

9. Training and Development Programs for Service Personnel
Group members: Law-enforcement personnel

• What problems do you see with the bicycle-helmet legislation?

• What do you think are the main reasons that people do not obey the new occupant-restraint law?

• What would you like to know about bicycle helmets and the bicycle-helmet law that the course did not cover?

• What would help you enforce the occupant-restraint law?

• What prevents you from educating parents and children about the importance of bicycle helmets?

• What role can EMS professionals play in educating people about injury prevention?

• What did your emergency medical technician (EMT) training teach you about injury prevention?

• What information provided by this training program will be most useful to you on your job?

• What did you hope to learn that was not covered?

• How can emergency-department personnel help prevent injuries?

• What have you learned from this course that will change the way you treat patients with head injuries?

• What resources would you need in order to educate patients about preventing future injuries? Probe: Who might be able to provide these resources?

• Of all the material covered in this training program, what do you think will be the most valuable to you in treating injuries? The least valuable?

• What knowledge is the most important for injury-control personnel?

• What skills are the most important for injury-control personnel?

• What do you see as the primary responsibilities of an injury-control specialist?

• What are some aspects of bicycle safety about which most injury-control personnel still need training?

• How does an injury-control professional benefit from the information presented in this course?

• Do you think beverage servers have a role in preventing motor-vehicle crashes? Probe: Why? Why not?

• What skills do beverage servers need to help prevent persons from driving while intoxicated?

• What do you think of the proposed training program?

• If you had created this training program, how would you have done it differently?

10. Programs to Distribute Safety Products
Group members: Program participants

• How did you find out about the distribution program?

• Tell me about your experience in getting the smoke detector?

• Have you talked with other people about the program?
  Probe: What do they have to say about it?

• Is your smoke detector installed? Was installing it difficult? Where did you put it?

• Has anyone you know taken advantage of the smoke detector distribution program? Probe: Why? Why not?

• How do you think people find out about programs like this?

• What would increase participation in the program?

11. Programs to Enforce Safety Laws: Checkpoints for DWI
Group members: People stopped at the checkpoint

• How were you treated by the roadblock personnel when you were stopped? Probe: Was it clear why you were stopped?

• How did you feel about being stopped at the roadblock? Probe: Was there anything about the personnel or the setting that caused you to feel that way? Probe: What was it?

• How could your experience have been improved?

• What do you think of the location(s) chosen for the roadblock(s)?

• At any time while you were conducting the checks did you feel at personal risk? Probe: Describe what happened.

• How might the roadblocks be improved?

• What have you heard about roadblocks being set up to catch DWI offenders?

• Do you think roadblocks are a good way to catch DWI offenders? Probe: Why? Why not?

• What are some of the good points of setting up roadblocks to catch DWI offenders?

• What are some of the problems?

Programs to Enforce Safety Laws: Assessment and Treatment for DWI
Group members: People assessed or treated

• What did you expect this program to be like? Probe: Did it meet your expectations?

• Was there anything you wanted from this program that you did not get? Probe: What did you not get?

• How might this program help change your driving behavior?

• What circumstances tempt you to drive while intoxicated? Probe: Has this program helped you know how to deal with these circumstances?

• Would you recommend this assessment program to other people? Probe: Why? Why not?

• What is most difficult about performing a DWI assessment? Probe: What might make this less difficult?

• What additional resources might assist you in assessing or treating DWI offenders?

• Are there any changes you would make in the program materials you were given? Probe: What would you change?

• What are the strengths of this assessment or treatment program? What are its limitations?

• What do you think is the purpose of a DWI assessment program?

• Do you think DWI treatment programs are effective? Probe: Why? Why not?

• Do you agree with spending public funds for a DWI treatment program? Probe: Why? Why not?

Programs to Enforce Safety Laws: Periodic Increases in Enforcing Selected Laws
Group members: Members of the general public

• Have you heard about the child safety seat enforcement program? Probe: How did you hear about it?

• What keeps people from using child safety seats?

• When and where are people least likely to use child safety seats? Probe: Why do you think that is?

• Is there anything that you think should be changed about the child safety seat enforcement program?

• Will the enforcement program make a difference?
  Probe: Why? Why not?

• ä Do you think it works to enforce child safety laws by stopping cars in some areas and not in others? Probe: Why? Why not?

• How did you react to being cited for failure to use a child safety seat?

• What problems are you likely to encounter in enforcing the use of child safety seats by stopping cars at selected checkpoints?

• How was the enforcement program received by drivers? By the public?

• Would you make any changes in the locations or times that were selected for enforcement? Probe: What would you change?

• What, if anything, do you think is working particularly well about the program? What needs improvement?

• What effect do you think this selective enforcement program will have on the use of child safety seats?

12. Programs to Change Individual Behavior: Counseling and Therapy
Group members: Members of the general public

• Who do you think people trust most for information about child safety seats? Probe: Why?

• What information do physicians usually provide about child safety seats?

• What did you learn in this program?

Programs to Change Individual Behavior: Incentive Programs
Group members: Members of the general public

• What incentives would influence people to put a smoke detector in their home?

• Do you think [name the incentives] are effective in getting people to install smoke detectors? Probe: Why? Why not?

• What is the best way to let people know about the bicycle helmet incentive program?

Programs to Change Individual Behavior: Skills Training
Group members: Trainees

• Do you wear a bicycle helmet? Probe: Why? Why not?

• What kind of people do you think wear bicycle helmets?

• When is it hardest to wear a bicycle helmet?
  Probe: What makes it hard?

• Do other people ever say anything about your wearing a helmet? Probe: What do they say?

• Which items covered in the program do you think will help kids to start, or to keep, wearing helmets?

EVENTS TO OBSERVE DURING PARTICIPANT -OBSERVATION

Participant-observers both participate in a program and observe it. As observers, they watch for nonverbal—as well as verbal—reactions to situations and notice behaviors of which participants are unaware. Listed below (under each type of injury-prevention program) are suggested events, behaviors, or procedures to observe, and under those are lists of questions that observers can answer.

1. Programs to Build Infrastructure: Getting Budgetary Support
Observe: Meetings to identify potential resources

• Who participates in meetings?

• How are potential sources of support identified?

• How do participants make decisions about following up on any plans?

• What alternatives are considered?

• How does the group decide whether to accept something (e.g., a procedure, a plan, or distribution materials)?

• Who provides documentation?

• How is the documentation verified?

• To whom is documentation distributed?

• What role does documentation play in monitoring the planning process?

• How is contact made?

• Who makes the contact?

• What is the format of the meeting?

• What is the tone of the meeting? Does it change during the course of the meeting?

• How are the supporters’ concerns elicited?

• How is the meeting documented?

• What topics are discussed?

• How frequently does correspondence occur?

• How are commitments of support documented?

Programs to Build Infrastructure: Institutionalizing a Program
Observe: Marketing the idea for a program

• How are potential contacts identified?

• Who is contacted?

• How is the concept marketed? Orally? Through print media? Through other news media?

• What is the response of those approached?

• Who attends the meetings?

• What are the salient issues?

• On what issues is there consensus?

2. Programs to Build Coalitions
Observe: Meetings of coalition members

• How do staff from different agencies interact with one another?

• Is there a central agency to whom all the other agency representatives speak or defer?

• Are there subgroups of agencies that link with other subgroups, or do all agencies interact directly?

• How is leadership negotiated?

• Do members volunteer or are they nominated?

• Do agencies representing certain constituencies work in concert and exclude other agencies?

• How is the agenda established or disrupted?

• Is the leadership supported by the other members of the group?

• Which members are disruptive?

• How are decisions made?

• Are there specific accomplishments by the end of the meeting?

• How are workloads divided?

• Are certain agencies asked more often to perform tasks? If so, does the agency staff seem to object to, or to accept willingly, these requests?

• Are the roles of new group members negotiated or assigned?

3. Programs to Develop or Improve Injury-Prevention Legislation
Observe: The legislative process

• Who supports the bill?

• What barriers are raised against the bill?

• How does support change during the life of the bill?

• How is the bill modified?

• How do police enforce the law? Suggestion: Ride with police to see firsthand the police and public reaction to enforcing laws.

• How do the courts react to enforcement of the laws?

Suggestion: Attend court when cases pertaining to the law of interest are on the docket.

• What are the legal consequences?

• What are offenders’ reactions?

• What is public’s reaction to the law? Suggestion: Attend public meetings that discuss the laws of interest.

• What views are presented at public meetings?

• What concerns are raised?

• Does public opinion change with time? Suggestion: Review print and electronic media coverage of the law.

• Is media coverage positive or negative?

• How frequently does coverage occur?

• How extensive is coverage?

• Does coverage wane? How quickly?

• When (day and time) is coverage presented?

• Who provides the most frequent coverage?

4. Public Information or Advertising Campaigns
Observe: Public reaction

• Are people paying attention to the billboard or advertisement?

• Do people have verbal or nonverbal reactions?

• What are the characteristics of people who react negatively?

• What are the characteristics of people who react positively?

Suggestion: Observe television watchers in airports, restaurants, and dormitory lobbies; radio listeners on street corners and in building lobbies; newspaper readers in hotel lobbies and on public transportation. To watch people’s reaction to posters and billboards, stand as near as possible without being intrusive.

5. Programs for Environmental Change
Observe: Public’s behavior related to an environmental change (e.g., a new ramp to a highway)

•  Do people appear to notice the change?

•  Are their reactions positive or negative?

•  Does the design have obvious or visible problems?

•  Do users appear to be confused by the design?

Suggestion: Ride with users.

6. Programs to Develop or Modify Products
Observe: Users’ reaction to a new or modified product

• Do they struggle in any way when using the product (e.g., when they put on the helmet, install the child safety seat, or change the battery in the smoke detector)?

• Do some designs appear to be more cumbersome than others? Suggestion: Station observers where products are tested or used.

• Do people show concern? Irritation?

• What questions do they ask about the product?

• What is the general demeanor of those demonstrating or selling the new or modified items?

• Do they respond appropriately to questions?

Suggestion: Station observers where products are distributed.

• Did respondent understand questions clearly?

• How much thought did respondent give to each question?

• How long did respondent take to answer each question?

• Did recorder understand questions?

• Was recorder confused by any question or instruction?

• Did recorder follow the prescribed interview protocol?

• How long did it take to abstract data?

• Did data entry clerk have difficulty with coding scheme, layout, or computer program used to store data?

• Do particular sections of training program lose the students’ attention?

• Do they appear to find some sections silly? Embarrassing? Demeaning?

• What is the content of any discussion among students and between students and teachers?

• What did they talk about positively? Negatively?

• Do they engage in behaviors that risk injury?

• Do they wear bicycle helmets?

• Do they wear seatbelts?

• How do they behave as pedestrians?

• Do they follow the curriculum?

• Are there specific parts of the curriculum they do not use?

• Are they able to answer all questions that students ask?

• Do they incorporate injury prevention into other tasks (e.g., writing assignments)?

• Do they develop class exercises that focus on injury prevention?

• Do particular sections of the training program lose trainees’ attention?

• How much do trainees participate in the discussion?

• What is the demeanor of trainees? Bored? Interested? Enthusiastic?

• Do trainees discuss material learned during the course?

• What are their comments about the course?

• Do trainees refer to course materials?

• Did trainees incorporate what they learned into their work habits?

• What characteristics are similar among recipients? Are they mostly women? Mostly young?

• How are these characteristics similar to or different from those of the distributors?

• How do recipients react to, or interact with, distributors?

• How do recipients interact with each other?

• How do recipients react to the product?

• Do they immediately take it out of the box or bag? Try it on (if appropriate)?

• Do they ask questions about the product’s function or how to use it?

• How do they approach recipients?

• What (if any) information do they offer?

• Do they follow the established protocol (if one exists)?

• What are the characteristics of people in the distribution locality who are not participating in the distribution program?

• Do they have similar characteristics? Are they mostly poor? Mostly apartment dwellers?

• Are these characteristics similar to those of participants? Are they mostly the same age?

• How are they similar to or different from the distributors?

• Are there any obvious barriers that prevent nonparticipants from using the program?

11. Programs to Enforce Safety Laws: Checkpoints for DWI

• What are the characteristics of the location (e.g., traffic density, socioeconomics of area)?

• Are there similarities among checkpoint locations?

• How do checkpoints correlate with the frequency of crashes in the area?

• Does the checkpoint disrupt neighboring businesses or other activities in the area?

• What are the characteristics of the people checked?

• What are their reactions to being checked?

• What are their reactions to the checker? Friendly? Belligerent? Fearful?

• Do the people stopped follow the directions of the checker?

• What problems do the people stopped appear to experience?

• How do they interact with the people they stop? Are they friendly? Hostile?

• Do they give consistent instructions for all people they stop?

• Are they clear?

• How do they treat people who are not sober?

• Is their treatment consistent for everyone?

• Is their treatment consistent with protocol?

Programs to Enforce Safety Laws: Assessment and Treatment for DWI
Observe: Behavior of program staff

• How does staff interact with program participants?

• Are assessments standardized?

• Is staff attentive to participants’ responses?

• Is staff courteous?

• Do counselors allow enough time for participants to respond to questions?

• Observe: Behavior of program participants

• Do they appear to be involved in the program?

• Are their questions and responses insightful?

• Do they respond to counselors’ questions freely?

• What is their general demeanor?

• Do they interact more freely with each other than with program personnel?

Programs to Enforce Safety Laws: Periodic Increases in Enforcing Selected Laws

• Do they follow a standard protocol?

• How do they communicate with the drivers? Is their manner friendly? Hostile?

• What is their response? Accepting? Annoyed? Hostile? Frightened? Resigned?

• What questions do drivers ask about the enforcement?

• Is privacy ensured?

• Is the setting conducive to one-on-one or group interaction?

• How do they show clients that they are paying attention?

• Do they pause for questions?

• ä Do they verify that the client has understood them?

• Do they appear attentive?

• What questions do they ask?

• Do they demonstrate a change in behavior? (For example, when they leave the premises, do they buckle their own and their child’s seatbelts?)

• Are they courteous?

• Do they give all necessary instructions to recipients?

• How do they handle people who do not meet the requirements to receive product?

• Are they attentive to the trainees?

• Do they model the skill in an explicit way?

• Do they give feedback to the trainees? Positive feedback? Constructive criticism?

WHO OR WHAT TO COUNT DURING PROCESS EVALUATION

Next are examples of items and people to keep track of while the program is in operation. Keeping exact, detailed records of program activities is essential if the program is to be a success. The information tracked while your program is under way can tell you whether the program is reaching the target population and, if so, whether it is reaching as many as you expected in the target population. The information tracked becomes the data for process evaluation.

1. Programs to Build Infrastructure: Getting Budgetary Support or Institutionalizing a Program
Who or what to count

• People met.

• People favorable to budgetary support.

• Letters written.

• Supporting letters received.

• Meetings held.

• Calendars.

• Correspondence logs or files.

• Meeting minutes.

• Signatures on petitions.

• Contact forms (see Appendix B for an example).

• Groups or agencies participating in a coalition.

• Meetings held.

• Communities represented.

• People participating.

• People served by participating agencies (measures potential effect of coalition).

• Requests for information.

• Agencies contacted.

• New program dollars generated.

• Additional resources garnered (e.g., helmets or smoke detectors donated).

Where to collect information for the count

• Attendance sheets.

• Calendars.

• Correspondence logs or files.

• Signatures on petitions.

• Contact forms (see Appendix B for an example).

3. Programs to Develop or Improve Injury-Prevention Legislation
Who or what to count

• Letters sent to supporters.

• People contacted for opinions (by age group).

• Public information meetings held.

• Attendees of public information meetings.

• Signatures of support.

• News media contacted.

• Press interviews given.

• Press releases prepared.

• Newspaper subscribers that the program contacted.

• Radio-station listeners contacted.

• Television-station viewers contacted.

• Contact forms (see Appendix B for an example).

• Attendance sheets.

• Files or logs of letters and news releases.

4. Public Information or Advertising Campaigns
Who or what to count

• Advertisements developed.

• Advertisements tested.

• Types of media for which advertisements were prepared.

• Subscribers for each newspaper where advertisement was placed.

• Listeners to each radio station where advertisement was run.

• Viewers of each television station where advertisement was run.

• Times advertisement was run in each newspaper and on each television or radio station.

• Press releases distributed.

• Articles written.

• Interviews or talks requested.

• Interviews and talks given.

• Interviews and talks taped.

• Public-information sessions held.

• People attending public-information sessions.

• Churches and schools visited.

• Files or logs of letters, news releases, and interviews.

• Attendance sheets.

• Institution contact forms (see Appendix B for an example).

• Individual contact forms (see Appendix B for an example).

5. Programs for Environmental Change
Who or what to count

• Users of vehicles passing a particular location.

• Positive and negative comments by letter.

• Positive and negative comments in news media.

• Observers at particular locations.

• Correspondence files.

• Local publications.

6. Programs to Develop or Modify Products
Who or what to count

• Developers contacted.

• Developments or modifications tested.

• Advertisements run (new development or modification).

• Letters sent.

• Inquiries received.

• Inquiries answered.

• Units distributed or sold.

• Units returned.

• Correspondence files.

• Contact forms (see Appendix B for an example).

• Logs of media coverage.

• "Items Distributed" forms (see Appendix B for an example).

• "Items Received" forms (see Appendix B for an example).

7. Programs to Establish Data Bases or Record Systems
Who or what to count

• Counties and sites approached to participate in system.

• Counties and sites recruited.

• Counties and sites that adopted a standard form.

• Personnel trained on system.

• Lap-top computers purchased.

• Lap-top computers placed.

• Counties and sites that use new system.

• Records processed.

• Forms documenting contact with county.

• Correspondence files.

• Purchase orders.

• Attendance sheets.

• Record systems.

8. Formal Educational Programs
Who or what to count

• Teaching modules or curricula developed.

• School systems and teachers that requested information.

• School systems and teachers that request newly developed curricula.

• Classes that use newly developed curricula.

• Students taught.

• Telephone and correspondence logs.

• Attendance sheets.

• Student contact forms (see Appendix B for an example).

9. Training and Development Programs for Service Personnel
Who or what to count

• Teaching modules and curricula developed.

• Requests for information received.

• Training sessions held.

• Attendees.

• Telephone and correspondence logs.

• Attendance sheets.

• Participant contact forms (see Appendix B for an example).

10. Programs to Distribute Safety Products
Who or what to count

• Resources garnered (e.g., bicycle helmets or smoke detectors donated to program, use of vehicles offered).

• Dollar value of resources donated.

• Volunteers to distribute products.

• Offers of resources for future use.

• Items distributed (e.g., smoke detectors, bicycle helmets).

• People in households to which products were distributed.

• Donor lists (include resources offered and resources given).

• Distribution forms.

• Volunteer lists.

11. Programs to Enforce Safety Laws: Checkpoints for DWI
Who and what to count

• Days and shifts when sobriety checks were conducted.

• Checkpoint locations.

• Drivers stopped.

• Drivers checked.

• Drivers with blood-alcohol levels higher than the legal limit.

• Amount of media coverage.

• Time cards or time sheets.

• Maps showing checkpoint locations.

• Contact forms (see Appendix B for an example).

• Media files.

Programs to Enforce Safety Laws: Assessment and Treatment for DWI
Who or what to count

• Inquiries about the services.

• Assessments and treatments scheduled.

• Assessments and treatments completed.

• Sessions conducted.

• Fees collected.

• Professionals trained to assess or treat DWI offenders.

• Contact forms (see Appendix B for example).

• Appointment books.

• Billing records.

• Continuing-education records.

Programs to Enforce Safety Laws: Periodic Increases in Enforcing Selected Laws
Who or what to count

• Days and shifts on which enforcement was conducted.

• Personnel involved in enforcement.

• Sites involved.

• People stopped.

• Citations and warnings given.

• Time cards and time sheets.

• Maps showing checkpoints.

• Contact and encounter forms (see Appendix B for an example).

• Citations.

12. Programs to Change Individual Behavior: Counseling and Therapy
Who or what to count

• Professionals trained.

• Clients counseled.

• Sessions held.

• Continuing education records.

• Contact forms (see Appendix B for an example).

• Counselors’ process notes.

Programs to Change Individual Behavior: Incentive Programs
Who or what to count

• Inquiries received.

• Incentives given.

• Telephone contact forms (see Appendix B for an example).

• Record systems.

Programs to Change Individual Behavior: Skills Training
Who or what to count

• Teaching modules developed.

• Inquiries received.

• Clients served.

• Training sessions held.

• Contact forms (see Appendix B for an example).

• Logs of telephone inquiries.

CLOSED ENDED QUESTIONS AND STATEMENTS

Next are examples of closed-ended questions and statements that can be used for surveys or interviews to assess people’s knowledge, attitudes, beliefs, or behaviors with regard to your program’s injury intervention. Such questions and statements as these can be used to gather data before your program begins (baseline data), when the program is in operation, and after the program is complete.

Multiple Choice
When you are driving 15 or more miles per hour (mph) over a 45-mph speed limit, how many times more likely are you to die than if you were driving within 9 mph of the speed limit? (Select one answer.)*

Multiple choice is particularly well-suited to questions that seek to find a person’s level of knowledge. The answers are easier to process than the answers to open-ended or fill-inthe-blank questions. The question itself should be clear and specific and the choices should be appropriate for the question. For an example, see below.

The answers should be mutually exclusive (i.e., not more than one answer should be completely correct), and only one answer should be correct for each respondent. This means that, for demographic questions, the answers should be exhaustive (i.e., all possible answers should be included). See example below:

Scaled
How important do you believe it is to have a smoke detector in your home? (circle one)

This page last reviewed April 1, 2005.

Privacy Notice - Accessibility

Centers for Disease Control and Prevention
National Center for Injury Prevention and Control