Outreach Evaluation Resource Center
Annual Report
Year Two May 2002-April 2003

Training and Continuing Education

The OERC staff conducted nine presentations and evaluation training sessions during Year 2 of the contract:

• Full-day evaluation workshop at the Medical Library Association Meeting, Dallas, TX by C. Burroughs and M. Blake (May 2002)
• Invited presentation titled "Program Evaluation: Tips for Planning" by invitation for the Northern Wisconsin Area Health Education Center Conference, Wasau, WI, by C. Burroughs (June 2002)
• Full-day evaluation workshop at Tribal College Librarians Institute, Bozeman, MT, by C. Burroughs and M. Blake (June 2002)
• Full-day workshop, at the Rochester Regional Library Council for health sciences and public library members, Rochester, New York, by C. Burroughs and M. Blake (September 2002)
• Full-day workshop at the Mid-Atlantic Chapter/Medical Library Association, Washington, DC, by C. Burroughs and M. Blake (October 2002)
• Presentation about the OERC at the Regional Advisory Council of the NN/LM, MCR, Kansas City, Kansas, by C. Burroughs (October 2002)
• Outreach evaluation workshop for invited PNR network members in Seattle, WA, by C. Burroughs and M. Blake (February 2003)
• Presentation about the Outreach Evaluation Resource Center to the National Library of Medicine (February 2003)

The following table shows that the workshops received very high ratings on the MLA Continuing Education evaluation forms. The workshop's contribution to participants' knowledge and skills received 100% positive ratings from participants. Most other aspects of the workshops (with the exception of "facility") received between 90% to 100% positive ratings. Three out of 4 workshops had 90-100% positive global ratings.

Participant Feedback for OERC Evaluation Workshops
	Percentage of Positive Ratings (4 Workshops Total)
	Number of Workshop with ...
	100% Positive Ratings	90-99% Positive Ratings	<90% Positive Ratings
[Participant] Acquired useful knowledge and skills*	4
Objectives*	3	1
Content*	2	2
Instructional materials*	2	2
Instructor(s)*	2	2
Facility*	2	2
Global rating**	2	1	1
* Positive rating="Agree" or "Somewhat Agree"** **Positive rating="Excellent" or "Above Average" or, in the case on of the Dallas workshop that had a different evaluation form, a "grade" of "A" or "B"

To increase her expertise in various areas of program evaluation, the assistant director of the OERC completed several professional development workshops:

• "How Libraries and Librarians Help" workshop sponsored by the Institute of Museum and Library Services, University of Washington (October 2002)
• Three continuing education courses at the American Evaluation Association annual conference in Washington, D.C.; workshop topics were collaborative evaluation, focus group interviewing, and questionnaire development. (November 2002)
• C. Burroughs made a presentation entitled: "Developing an Evaluation Guide for the National Network of Libraries of Medicine" at the American Evaluation Association annual conference, Washington, DC. (November 2002)

Web site development

• C. Burroughs continued maintaining and updating the OERC website.
• C. Burroughs and Dr. Zoe Stavri, consultant, developed a concept outline of a stand- alone web-based toolkit for the evaluation guide on ways to develop useful research objectives and methodology for decision-based evaluation. They also worked to clarify goals and expectations for an online learning resource regarding outreach planning and evaluation. By the end of the year, they decided to shift Z. Stavri's role to advisor.

Consultation with Network members

• C. Burroughs consulted with project staff of the Texas Lower Rio Grande Valley Health Information Hispanic Outreach project to develop project objectives for each outreach site for submission to NLM.
• At the RML director's meeting on May 17 in Dallas, Texas, C. Burroughs facilitated a full staff discussion about cross-regional evaluation to prioritize progress measures for NN/LM outreach to public libraries and public health departments. NN/LM staff reached consensus on 3 priority objectives for each outreach audience that the RMLs will track and evaluate based on a cohesive evaluation plan. Over the next several months, C. Burroughs worked with NNO to identify and convene two task forces (consisting of RML staff and stakeholders from each outreach audience) to address how these objectives will be reached and evaluated.
• C. Burroughs reviewed the projects currently funded by the NN/LM SCR to help provide suggestions on how the projects might evaluate their efforts. She also served as a reviewer for 3 proposals received by NN/LM SCR in response to their "Closing the Gap" RFP.
• C. Burroughs has served as a resource to the Tribal Connections Phase 3 evaluation consultant. Her participation included:
• • Assisting with development of TC3 pre-program interview questions
  • Traveling in October to one of the TC3 sites (the Confederated Tribe of the Umatilla Reservation in Oregon) and assisting with interviews of several key contacts at the tribe
  • Attending TC3 staff meetings and providing input on the development of the project's objectives, intended outcomes, and development of the memorandum of understanding template.
• C. Burroughs worked with Claire Hamasu and Roy Sahali on a 2-year project to capture the lessons learned by partners who are participating in tribal outreach in the Four Corners area of the United States C. Burroughs provided consultation to the proposal which was submitted by C. Hamusu and approved by NLM. She assisted with planning and initiating this project, and recruited the evaluation consultant who was subsequently hired to plan and facilitate a mapping workshop with TC4 collaboration participants.
• Betsy Kelly of the NN/LM, MCR asked C. Burroughs to review a network member survey that will collect baseline data about member's services, resources, and relationships.
• C. Burroughs worked with the health outreach evaluation task forces, conducting the following activities:
• • Drafting a memo to initiate and provide specific charges to the public libraries and public health department outreach evaluation task forces
  • Meeting by phone with task force chairs for the public libraries and public health outreach evaluation task forces (Neil Rambo and Debbie Sibley respectively), Angela Ruffin, Keith Cogdill, and Lalitha Kutty to discuss strategy and timeline for the work of their groups
  • Participating in task force conference calls that took place for the next two months and assisting the chairs in managing, documenting, and reporting their progress
  • Assisting the chairs in preparing final reports submitted to NNO in January.
• C. Burroughs assisted with the development of a needs assessment survey of Indian Health Services staff in a tribal community.
• C. Burroughs provided feedback about a working draft of the community partnership RFP.

Assistance with analysis, synthesis, and reporting of evaluation results

C. Burroughs began working on an evaluation methods for the public libraries and public health department outreach initiatives by:

• Drafting the evaluation plan for the public libraries and public health department outreach evaluation task forces
• Testing ways to measure outputs listed in the public libraries and public health logic models.

Communications and collaboration

C. Burroughs participated in the following communication and collaboration activities:

• She worked with members of the MLA/PNC chapter to plan and develop a survey to assess need and use of electronic document delivery methods.
• She participated as a resource for the TC4 Collaboration project.
• As an invited guest, she attended meetings, as invited member, of the NIEHS Superfund Basic Research Program External Advisory Group.
• She launched the outreach_eval listserv.

Utilization and promotion of the evaluation guide

• NN/LM PNR continued to distribute the guide based on request.

Administrative

C. Burroughs completed the following administrative projects:

• Facilitation of a site meeting with staff from the National Outreach Mapping Center.
• Submission of documentation to the University of Washington Libraries for consideration for continuing status as an Associate Librarian.
• Development of a Year 3 budget and narrative for the OERC and submitted it to NLM.