Data Collection Methods at a Glance
| Method | Overall Purpose | Advantages | Challenges |
|---|---|---|---|
| Questionnaires, surveys, and checklists | When you need to quickly and/or easily get lots of information from people in a non-threatening way |
- Can be completed anonymously - Is inexpensive to administer - Is easy to compare and analyze - Can be administered to many people - Can produce a lot of data - Sample questionnaires already exist |
- You might not get careful feedback - Wording can bias a client's responses - Can be quite impersonal - Surveys may require a sampling expert - Do not tell the full story |
| Interviews | When you want to fully understand someone's impressions or experiences, or learn more about their answers to questionnaires |
- Collects full range and depth of information - Develops relationship with client - Can be flexible with client |
- Can take much time - Can be hard to analyze and compare - Can be costly - Interviewer can bias a client's responses |
| Documentation review | When you want an impression of how a program operates without interrupting the program; involves reviewing applications, finances, memos, minutes, etc. |
- Collects comprehensive and historical information - Does not interrupt program or client's routine in program - Information already exists - Involves few biases about information |
- Can take much time - Information may be incomplete - Need to be quite clear about what you are looking for - Inflexible; data are restricted to what already exists |
| Observation | To gather accurate information about how a program actually operates, particularly its processes |
- Views program operations as they occur - Can adapt to events as they occur |
- Can be difficult to interpret people's behaviors - Can be complex to categorize observations - Can influence behaviors of participants - Can be costly |
| Focus groups | To explore a topic in depth through group discussion, e.g., reactions to an experience or suggestion, understanding common complaints, etc.; useful in evaluation and marketing |
- Quickly and reliably collects common impressions - Can efficiently collect broad and deep information - Can convey key information about programs |
- Can be hard to analyze responses - Need a good facilitator - Can be difficult to schedule |
| Case studies | To fully understand or depict a client's experiences in a program, and conduct comprehensive examination through comparing cases |
- Fully depicts client's experience with program - Powerfully portrays program to outsiders |
- Can take much time - Collects deep, but not broad information |
Adapted from:
McNamara, C. Basic guide to program evaluation. Available on-line at: http://www.mapnp.org/library/evaluatn/fnl_eval.htm#anchor1581634.
Return to Day 4: Conducting Your Evaluation. |
 |
||||||||||||
|
TOC |
|
||||||||||
| |
||||||||||||
Last Modified: 06/12/2008
