The Evaluation Matrix: When & Where & Who &
How
Thomas Shuster, Ph.D.
Todd Braeger, M.S.
Spectrum Consulting / Safe Food Institute
1770 North Research Parkway
North Logan, Utah
(435-753-9333)
Some Definitions of Evaluation
- Research or measurement
- Extent that objectives are attained
- Evaluation as professional judgment
- Auditing or quality control
- Collecting information for decision-makers
Our Definition of Evaluation
Judging the worth or merit of something
Objectives for Presentation
- Give a you tool to help you in your work.
- Give you an appreciation of the complex
nature of effective evaluation and how it can
help your projects.
Research vs. Evaluation
- Both use the same tools
- Both use same methods
- Sometimes the questions are the same
The Difference between research and evaluation
is the purpose…
- The purpose of research is to produce
generalized knowledge based on inference from a
sample to a population
- The purpose of evaluation is to judge the
merit or worth of something
Evaluation Answers Questions Like…
- Does it work?
- Does it do what we want it to?
- How well does it work?
- Does it work for the reasons we think it
does?
- How much does it cost per benefit gained?
- Does it have side effects?
Evaluation requires good, clear thinking
because sometimes (often?) the real world fools
us!
Examples….
- Anti-bacterial soap, cutting boards….
- Surveys and behavior….
- After-school programs….tutoring…..
Evaluation is a complex task…
Evaluation is not just pre-test / post-test
comparisons of data.
Properly Used Evaluation Can…
- Focus a project at the beginning
- Keep the project on track
- Make mid-course corrections
- Control costs by focusing resources
- Answer/discover critical questions
- Document success/avoid failure
- Bridge perceptions with data analysis
- Lead to more funding!
Evaluative Questions
Our evaluations start with questions. These
questions guide and focus the evaluation
activities during the course of the project.
Evaluation Matrix…
- A simple tool used for planning and
conducting an evaluation.
- Simple yet Complex…..
Kitchen Observation Study
Overhead of Evaluation Matrix…
Grocery 101 Scenario….
You be the evaluator! How exciting!
Project Problems Will Arise…..
How do you handle this as an evaluator……
- Do nothing?
- Do something differently?
- Do something additionally?
Project Problem #1
How do you handle this as an evaluator…
Grocery staff is not learning the information
based on your data.
Project Problem #2
How do you handle this as an evaluator……
New staff enjoys and learns, but experienced
staff say they already know this but according
to data they are committing violations….
Project Problem #3
How do you handle this as an evaluator……
Grocery stores A and B are improving but C is
not.
Reporting of Evaluation Results
- Timely
- Clear and concise
- Useful
- Evaluative in nature….we make judgments and
recommendations based on data
- Practical
- Meet requirements of funding agency
Primetime Live Lessons…
- Focus your message
- Set the right tone
- Findings need to be scientifically defensible
and open to review
Other Issues in Evaluation
- When does evaluation team start?
- Qualitative vs. Quantitative?
- Cost of evaluation
#