Return-Path: <nifl-family@literacy.nifl.gov>
Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id g14N9gu02649; Mon, 4 Feb 2002 18:09:42 -0500 (EST)
Date: Mon, 4 Feb 2002 18:09:42 -0500 (EST)
Message-Id: <C50587E3B025D3118DDB00A0C9FC3C5E02F3CBFC@EXCHANGE>
Errors-To: listowner@literacy.nifl.gov
Reply-To: nifl-family@literacy.nifl.gov
Originator: nifl-family@literacy.nifl.gov
Sender: nifl-family@literacy.nifl.gov
Precedence: bulk
From: Virginia Tardaewether <tarv@exchange.chemeketa.edu>
To: Multiple recipients of list <nifl-family@literacy.nifl.gov>
Subject: [NIFL-FAMILY:712] Re:
X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas
Content-Transfer-Encoding: 8bit
Content-Type: text/plain;
X-Mailer: Internet Mail Service (5.5.2653.19)
Status: O
Content-Length: 5837
Lines: 140
Mary
I guess what I would add to what Meta said is that an evaluation process
can, and I think should, be about program improvement. If all the staff and
all the students view evaluation as a tool for getting better at what they
(we) do, it makes the process useful and interesting. Students get involved
as well as staff.
Va
Virginia Tardaewether
Chemeketa {Place of Peace}
Outreach Instructor
Dallas, OR 97338
tarv@chemeketa.edu
503-316-3242
-----Original Message-----
From: MWPotts2001@aol.com [mailto:MWPotts2001@aol.com]
Sent: Monday, February 04, 2002 7:27 AM
To: Multiple recipients of list
Subject: [NIFL-FAMILY:709] Re:
Mary,
Choosing an evaluator is so very important. I hope this helps. I believe
that most of this came from RMC Research Corp in New Hampshire.
What are the Characteristics of an Effective Evaluator?
Even Start programs are diverse in design. To evaluate unique programs,
dedicated evaluators must have:
*strong communication skills,
*knowledge and understanding of the evaluation process and analysis,
*experience with literacy programs
*and a willingness to spend time at the site of evaluation.
Awareness of environmental culture of the constituency is also important.
Strong Communication Skills. The ability to communicate with all
stakeholders in the evaluation process heads the list of important strengths
for evaluators. Evaluators meet with program staff and listen to what is
happening in the program. They discuss what the staff would like the
evaluation to show. The evaluator communicates the idea that evaluation can
highlight the good things programs accomplish---things which might otherwise
be overlooked. Grants without strong evaluation components do not receive
funding. To get accurate results evaluators must have test scores, surveys
and additional information from the staff. This helps the staff to see
themselves as part of the evaluation team.
Another important communication skill of the evaluator is to focus program
staff members on the different groups that will see and use the evaluation
information. To this end, an effective evaluator asks questions about how
evaluation information may be used, both positively and negatively, and
assists in examining the political climate.
Inexperienced program staff members often are surprised at how involved the
evaluation process can be. In addition to formulating a balanced approach,
the evaluator may answer questions about the type of report, who will
deliver
the report, time frames and report audiences.
Knowledge and Understanding of the Evaluation Process. Does the evaluator
have knowledge of both formative and summative approaches to evaluation?
Ideally, the program staff members and the evaluator will be flexible in the
evaluation process, tailoring the evaluation to meet the program’s needs.
Because program needs vary, it is wise to investigate the evaluator’s
background knowledge and beliefs early on.
Experience Working with Family Literacy Programs. A strong knowledge of
family literacy is an ideal skill for evaluators, but in reality it may be
difficult for programs to find evaluators with such a background. Since
this
situation is widespread, the program might consider including a person, or
organization, who understands family literacy along with the evaluator in
the
request for proposal (RFP) planning process. Lacking experience in family
literacy does not necessarily mean an evaluator will not be effective. Some
have strong backgrounds in adult or early childhood education or reading.
Combining evaluator strengths with the skills and expertise of the program
director and staff, consulting with colleagues, and conducting research to
broaden his/her understanding of family literacy will familiarize the
evaluator with family literacy issues.
When interviewing a potential program evaluator, the evaluation team may
want
to include these questions:
What is your prior knowledge of family literacy?
What is your interest in learning about literacy issues?
What do you perceive as issues in evaluation? For example,
“How can we measure adult progress?”
Expertise in Analysis. A number of the individuals surveyed stated that
skill in analysis is crucial. Once the program administrator and staff
determine evaluation needs, they will work with the evaluator to gather
data,
analyze it, interpret the results and report them. Administrators and staff
members may choose an evaluator who has particular strength and expertise
lacking in the program, however, the ability to analyze and interpret data
is
a must for every evaluator.
Willingness to Spend Time On Site. Directors and their staff members want
evaluators to establish a personal connection with the Even Start program.
To establish this connection, the evaluator must spend time becoming
familiar
with all facets of the program.
Other suggestions. One evaluator suggested that programs take the
initiative
in asking the evaluator for references. Programs need to ask key questions:
“Where has the evaluator done similar work? What were the outcomes and
successes of evaluations?” Program directors and staff that do not ask
these
questions may go into the evaluation process with high expectations and
emerge with the belief that they got poor service for their dollars. When
program staff members do not articulate their needs, they get evaluations
which do not meet their expectations and all parties become losers.
Effective evaluators are aware of the cultures they serve. Evaluation
should
reflect clear understanding of teaching and learning in multicultural
environments. In many evaluations multicultural aspects are not really
examined.
Meta Potts
Glendale, AZ
This archive was generated by hypermail 2b30 : Fri Jan 17 2003 - 14:40:54 EST