Outreach Evaluation Resource Center
Annual Report
Year Five May 2005-April 2006

Training and continuing education

Listed below are the workshops and training sessions offered by the OERC offered nine workshops and training sessions Annual Year 5.

• Overview of the PubMed/MedlinePlus Followup Survey and the survey results browser for RML via Macromedia Breeze at an all-RML teleconference staffed by Susan Barnes and Aron Beal (September 14, 2005). The survey will be administered for the duration of Year 5 of the 2001-2006 NN/LM Contract.
• "Measuring Your Impact" workshop at the Midwest Chapter, MLA autumn meeting, Fargo, North Dakota, by S. Barnes and Maryanne Blake (September 16, 2005).
• Overview of the SurveyMonkey web-based questionnaire software via Macromedia Breeze at an all-RML teleconference by M. Blake (November 11, 2005).
• "Planning and Evaluating Health Information Outreach Projects," a new workshop to staff from six regions plus network members from the Pacific Northwest Region, presented in Seattle by S. Barnes and Cindy Olney (January 20, 2006).
• Two sessions of the "Planning and Evaluating Health Information Outreach Projects" workshop conducted by S. Barnes and C. Olney: one in Houston to RML staff from three regions plus network members and others from the South Central Region (February 3, 2006); and one in Oakland to RML staff from two regions plus network members from the Pacific Southwest Region (February 28, 2006).
• "Measuring Your Impact: Using Evaluation to Demonstrate Value" workshop presented to the Ohio Health Science Library Association in Columbus, OH by S. Barnes and M. Blake (April 25, 2006)
• "Measuring Your Impact: Using Evaluation to Demonstrate Value" workshop presented to Iowa State Library Association's Health Sciences Subdivision meeting, Des Moines, by M. Blake and Betsy Kelly of the Mid-Continental RML (April 27, 2006).

In order to develop and improve the one-day workshops Measuring Your Impact and Planning and Evaluating Health Information Outreach Projects, thorough evaluation information was collected at the workshops. Measuring Your Impact is a workshop designed to help hospital libraries identify the value they add to their hospitals and develop evaluation methods to help them demonstrate that value. Findings from evaluation data collected for Measuring Your Impact showed that participants responded very well to the workshop. Ninety-five percent or more of participants gave positive ratings to instructors on all criteria (knowledge, preparation/ organization, delivery skills, and responsiveness to questions) and 89% said the instructors were a major strength of the workshop. Table 1 shows the high percentage of positive ratings given by participants on key workshop features (88% of participants said they would give the workshop an "A" grade). In written comments, many participants gave examples of how they planned to apply the workshop information.

Table 1: Participant Feedback for "Measuring Your Impact" Workshops
	Percentage of Positive Ratings (88 Participants; 4 Workshops)
Instructional materials were used effectively	99%
Instructional materials were relevant/useful	99%
Hands-on sessions were useful	98%
Session objectives met my expectations	98%
Session content was well organized	99%
Length was appropriate for course content	99%
I acquired knowledge and skills I can use	99%
Facility was conducive to learning	98%
Global rating (i.e., grade participant gave workshop)	95%
* Positive rating="Agree" or "Somewhat Agree" ** Positive rating=a "grade" of "A" or "B"

• Findings from the evaluation data for Planning and Evaluating Health Information Outreach Projects showed that it was well-received overall. Ninety-seven percent or more of participants gave the instructors positive ratings on knowledge, preparation/organization, delivery skills, and responsiveness to questions. Table 2 shows the percentage of respondents who gave positive ratings to certain features of the workshop (58% gave it an "A" grade and 39% gave it a "B" grade). The participants did give fairly consistent feedback that the workshop has too much content to cover in 6 hours, so the instructors are working on creating several workshops from this one.

Table 2: Participant Feedback for "Planning and Evaluating Health Information Outreach Projects" Workshops
	Percentage of Positive Ratings (3 Workshops)
Instructional materials were used effectively	97%
Instructional materials were relevant/useful	97%
Hands-on sessions were useful	94%
Session objectives met my expectations	94%
Session content was well organized	95%
Length was appropriate for course content	89%
I acquired knowledge and skills I can use	95%
Facility was conducive to learning	91%
Global rating (i.e., grade participant gave workshop)	98%
* Positive rating="Agree" or "Somewhat Agree" ** Positive rating=a "grade" of "A" or "B"

Cindy Olney compiled and analyzed data from the evaluations received for the two OERC classes that were taught during Year 5 and then created a report for each class: Measuring Your Impact and Planning and Evaluating Health Information Outreach.

To increase her expertise in various areas of program evaluation, S. Barnes, attended the following workshops and classes:

• "Strategies for Teaching Ins and Outs of Evaluation," (full-day class) by Hallie Preskill, at the Claremont Graduate University's Professional Development Workshop Series (July 14)
• "Health Behavior Program Development & Evaluation" (full day class) by Eusebio Alvaro (July 15).
• Two workshops (a two-day qualitative workshop and half-day empowerment evaluation methods workshop) at the American Evaluation Association national conference in Toronto, Ontario, Canada (October, 2005). (S. Barnes also attended the conference).

Consultation with Network members and NNO

• Cathy Burroughs, C. Olney, and S. Barnes presented reports about the Public Library and Public Health Outreach Initiatives, and the PubMed/MedlinePlus Followup Survey, at the May 2005 RML Directors' Meeting in San Antonio.
• The OERC participated in three evaluation projects in preparation for the May 20 Directors Meeting In Phoenix. First, the Public Health and Public Libraries outreach initiatives survey was updated after collecting input from the 8 NN/LM Associate Directors through a conference call. The updated survey was administered to all eight regions in March and data were analyzed by C. Olney. A second project involved a final compilation and analysis of results from the PubMed/MedlinePlus followup survey (conducted by C. Olney). The third activity involved a collaboration between S. Barnes and Karen Comer of the NOMC to produce maps and tables of outreach activities. Results from all three projects were presented by S. Barnes at the RML Director's Meeting.
• S. Barnes collected feedback regarding the PubMed/MedlinePlus followup survey from NN/LM Consumer Health Coordinators and the NN/LM Outreach/Education Coordinators (via conference calls) and NN/LM ADs (via email discussion). She submitted a recommendation to suspend the followup survey as of 5/15/06, which was accepted by the NNO. (Details about this recommendation are available as part of the OAWG final report).

Assistance with analysis, synthesis, and reporting of evaluation results

• A. Beal of Web-STOC completed the PubMed/MedlinePlus Followup Survey results browser and created extensive documentation on the NN/LM StaffWiki to support its use.
• S. Barnes provided further consultation regarding Tribal Connections Four Corners collaboration reporting and an effective practices entry for the Corporation for National and Community Service web site.
• S. Barnes began the migration of the OERC web site into the new templates that resulted from Web-STOC's Usability Evaluation and Redesign effort.

Communications and collaboration

• S. Barnes participated as a member of the Effective Practices workgroup of the Tribal Connections Four Corners collaboration.
• S. Barnes contributed an article about the Public Health and Public Library outreach initiatives to the NN/LM Mid-Continental Region's newsletter, "Plains to Peaks Post."
• S. Barnes completed sections of the NN/LM StaffWiki that provide history and overviews of the Public Health and Public Library Outreach Initiatives and of the PubMed/MedlinePlus Followup Survey.
• S. Barnes served on the planning committee for the Medical Library Association Institute on Designing Web-Based Continuing Education, which is funded by NN/LM Greater Midwest Region.
• S. Barnes began service as a participant in the Web-STOC-NN/LM Usability Evaluation and Review project.
• S. Barnes provided feedback to Claire Hamasu regarding a survey for participants in the Tribal Connections Four Corners collaboration.
• As a member of the NOMC TAC, S. Barnes facilitated a discussion in which NN/LM PNR staff provided feedback regarding proposed enhancements to the OARF and reported the results to the TAC.
• S. Barnes and Keith Cogdill co-chaired the Outreach Activities Working Group (OOAWG) formed by the NNO in early March. The OAWG charge was to assist NOMC with enhancements to the Outreach Activities Report Form, to address issues related to outreach activity reporting, and to discuss the PubMed/MedlinePlus followup survey. Discussions took place via email and in two conference calls. Recommendations were submitted to NNO and the OAWG final report was accepted by the end of April, 2006. Details about the OAWG are available in the Outreach Activities Working Group section of the NN/LM StaffWiki (access restricted to NN/LM and NLM staff).

Utilization and promotion of the evaluation guide

• C. Olney and S. Barnes completed three booklets which are the "Planning and Evaluating Health Information Outreach Projects Series." These booklets were designed to condense and update the "Measuring the Difference" guide. The workshop listed in the first section of this report, Planning and Evaluating Health Information Outreach Projects, was designed to use these booklets. The beta version of the booklets was distributed in all three workshop. (Remaining printed copies of the beta version were provided to the Texas Library Association upon request from a participant from the Houston class.) Feedback was received and revisions were made with a target of publication of the final versions in June, 2006. (A no-cost budget extension through July 31, 2006 made it possible to extend this target date past the original date of April 30, 2006.) The booklets are available on the OERC web site in pdf form at http://nnlm.gov/evaluation/booklets/.
• Aaron Kuhns, student assistant, upgraded the online version of "Measuring the Difference" to provide improved navigability.

Administrative

• S. Barnes became Acting Assistant Director of the OERC, following C.Burroughs' promotion to Associate Director, NN/LM Pacific Northwest Region.
• A proposal for the 2006-2011 OERC contract was completed and submitted to the National Library of Medicine.
• S. Barnes attended orientation at the National Library of Medicine, since she recently began a new position.
• With M. Blake's guidance, the OERC initiated a subscription to the SurveyMonkey web-based questionnaire software for use by RML staff in all 8 regions.
• S. Barnes compiled and submitted responses to questions from the National Library of Medicine regarding the proposal for the 2006-2011 OERC contract.
• S. Barnes compiled responses to questions from the National Library of Medicine regarding the proposal for the 2006-2011 OERC contract, and submitted the Final Revised Proposal to NLM in late April. At the end of April, news was received that the contract for the NN/LM Outreach Evaluation Resource Center at the NN/LM PNR in Seattle had been renewed.