Outreach Evaluation Resource Center

Quarterly Report: August - October, 2006

Executive summary

In support of dissemination of effective practices, the OERC published the new 3-booklet "Planning and Evaluating Health Information Outreach" series, made the full text available on the web, and distributed 257 sets. The OERC joined the group of Macromedia Breeze early adopters within NN/LM by using Breeze in its August 1 all-NN/LM teleconference, "What the OERC Can Do for You," and in its August 30 presentation about evaluation to 28 hospital librarians in the Southeastern/Atlantic Region of the NN/LM. To build evaluation capacity among NN/LM staff, the OERC helped staff from the NN/LM Greater Midwestern Region prepare for and teach the "Measuring Your Impact" workshop about using evaluation for library advocacy, and began to create curricula for a new workshop series based on what was learned in piloting a day-long outreach evaluation class during the previous contract. The OERC Blog, with sections on News, Liaisons, and Collaboration, was made public; two Evaluation Liaisons Teleconferences were held; participation in WebSTOC's Usability Evaluation and Review continued; Cindy Olney created a method to track OERC activities and measurable objectives to be used in tracking the OERC's progress during the contract; and the OERC helped the eight regions and NLM progress toward selecting an area of national collaboration for the 2006-2011 contract. OERC staff provided consultation for four regions (GMR, MCR, SCR, PSR), the NTCC, Web-STOC, and MLA (see table 1, below, for details).

Classes and consultations

Date	Responsible Staff (Last Name)	Class/Consultation Topic	Location (city, state) of Class or Last Name of Consult Recipient
8/1/2006	Barnes	Train: All RML teleconference. "What the OERC Can Do for You"	NN/LM staff, NNO, national
8/2/2006	Barnes, Olney	Consult: Subcontract proposal	Erinn Aspinall, UNM HSLIC, Albuquerque, NM
8/4/2006	Barnes	Consult: TC4C effective practices	Claire Hamasu, MCR, Salt Lake City, UT
8/10/2006	Barnes	Consult: TC4C 2006 proposal	Claire Hamasu, MCR, Salt Lake City, UT
8/14/2006	Barnes, Olney	Consult: Cross-regional Collaboration Breeze teleconference	RML ADs and Directors, NNO, national
8/17/2006	Barnes (with Holst and Leskovec)	Train: Measuring Your Impact	Univ of Minn BioMed Lib, Minneapolis, MN
8/22/2006	Barnes, Olney	Consult: Eval liaison teleconference	Evaluation Liaisons, NNO, national
8/25/2006	Barnes	Consult: PSR LSTA grant	Heidi Sandstrom, PSR, Los Angeles, CA
8/29/2006	Barnes	Consult: Electronic reporting system	Michael Boer, Web-STOC, Seattle, WA
8/30/2006	Barnes	Train: Evaluation Overview	NN/LM Southeastern Atlantic Region, from Seattle via Breeze
8/30/2006	Barnes	Consult: Staff recruitment	Renee Bougard, SCR, Houston, TX
9/6/2006	Barnes	Consult: Staff recruitment	Renee Bougard, SCR, Houston, TX
9/7/2006	Barnes	Consult: Staff recruitment	Renee Bougard, SCR, Houston, TX
9/8/2006	Barnes	Consult: Training assessment	Sharon Brown, NTCC, New York, NY
9/18/2006	Barnes	Consult: TC4C final report	Claire Hamasu, MCR, Salt Lake City, UT
9/25/2006	Barnes	Consult: Presentation quality maps	Angela Ruffin, NNO, Bethesda, MD
9/26/2006	Barnes, Olney	Consult: Eval liaison teleconference	Evaluation Liaisons, NNO, national
10/2/2006	Barnes	Consult: Review of 8/14 teleconference about Cross-regional collaboration and process so far	RML ADs and Directors, NNO, national
10/5/2006	Barnes	Consult: Exhibit evaluation	Erinn Aspinall, UNM HSLIC, Albuquerque, NM
10/6/2006	Barnes	Consult: Review of 8/14 teleconference about Cross-regional collaboration and process so far	RML ADs and Directors, NNO, national
10/12/2006	Olney	Train: OERC workshop series overview	NN/LM staff, NNO, national
10/16/2006	Barnes	Train: OERC services and publications overview at RML update for four western MLA chapters	Heidi Sandstrom, PSR; Cathy Burroughs, PNR; Seattle, WA
10/17/2006	Barnes	Consult: Electronic reporting system	Angela Ruffin, NNO, Bethesda, MD
10/26/2006	Barnes	Consult: MLA/NLM Health Information Literacy Research Project	Jean Shipman, MLA, Chicago, IL

Web site

Susan Barnes participated in WebSTOC's ongoing Usability Evaluation and Review (UER), including teleconferences and final completion of new web templates and navigation based on previous UER findings.

Dissemination of effective practices

The OERC distributed sets of "Planning and Evaluating Health Information Outreach" booklets:

Number of Sets		Region
45		MAR
65		SAR
40		GMR
42		MCR
25		SCR
10		PNR
0		PSR
30		NER
Total:
257

In addition, the OERC sent 74 sets to requestors in 19 states, DC, two Canadian provinces, and the UK. 26 copies of "Measuring the Difference" were sent out.

Susan Barnes participated as a member of the TC4C Effective Practices group, with specific responsibility for compiling a bibliography of publications about effective practices in health information outreach.

Subcontractor update

Not applicable. The OERC has no subcontracts at present.

Other staff activities-this section is organized according to OERC's goals for the 2006-2011 contract

1. Develop awareness, skill, and capacity in use of evaluation methods.

   Presentation and Training: The OERC was the subject of the August 1, 2006 All-NN/LM teleconference: "What the OERC Can Do for You" on 8/1, via Macromedia Breeze. Susan Barnes presented information about the "Planning and Evaluating Health Information Outreach" series of 3 booklets (their content and upcoming availability to the regions); the planned series of 5 evaluation workshops based on the 1-day class piloted in Seattle, Houston, and Oakland this past winter and using the booklet series (development process and ideas about teaching); the "Measuring Your Impact" workshop developed by Betsy Kelly and Maryanne Blake, also taught by Susan Barnes, with Ruth Holtz and Jacqueline Leskovec from NN/LM GMR taking on responsibility for teaching this class in its region (first opportunity will be at University of Minnesota BioMedical Library on August 17); an updated on activities related to NNO's Priorities for OERC as Presented on 5/20/06 (Dissemination of best practices in outreach, Identify/encourage opportunities for cross-regional collaboration, Presentation quality mapping); consulting services from OERC; and national collaboration planning activities (primarily the conference call with RML ADs, Directors, and their representatives on August 14).

   Susan Barnes gave a presentation to 28 hospital librarians in the Southeastern/Atlantic Region via Breeze on August 30, "Evaluating Outreach Activities: Creating Tiny Ripples of Hope or Great Waves of Success" (the title was assigned by Toni Yancey). The 45-minute talk included these topics: Why Should I Evaluate What I Do? How Do I Evaluate? And Where Can I Go for Help?

   Susan Barnes assisted Ruth Holst and Jacqueline Leskovec in their co-teaching of "Measuring Your Impact" for the first time, on August 17 at the University of Minnesota Biomedical Library in Minneapolis, MN.

   As part of developing the new OERC workshop series, Cindy Olney and Susan Barnes held a Macromedia Breeze meeting with the regions' two most experienced Evaluation Liaisons, Maryanne Blake and Betsy Kelly. We obtained a lot of useful feedback from Maryanne and Betsy. Later, in October, Cindy Olney gave a presentation about these new workshops to an all-NN/LM Breeze presentation which led to subsequent discussions with NNO and clarification about the use of the word "survey."

2. Develop mechanisms to disseminate and use evaluation findings to help NLM and the NN/LM document and demonstrate their accomplishments.

   Mapping: Susan Barnes participated in a meeting with Angela Ruffin, Aron Beal, Michael Boer, Cathy Burroughs, and Sherri Fuller to discuss ideas and issues regarding the new reporting system and mapping. Susan also participated in a Breeze meeting about how NLM's OCCS might be involved in NN/LM GIS activities.

   Publications: The OERC Blog went public this quarter, with three sections for postings: News, Liaisons, and Collaboration.

3. Strengthen and build an evaluation culture throughout NN/LM.

   Evaluation liaisons: Susan Barnes and Cindy Olney led two Evaluation Liaison teleconferences this quarter via Breeze (the October teleconference was cancelled due to numerous schedule conflicts). The third teleconference on August 22 featured a presentation from Mellanye Lackey about her NLM Associate Fellowship project in which she analyzed the types of evaluation being used in NNLM subcontract projects described in the CHOS Outreach Project Database conducted between 2001-2006 with funding levels of $2500 - $70,000; the OERC Blog; a reminder about the booklet series; and updates from the regions. In the fourth, we discussed using the liaisons to pilot a "dot vote" on areas of possible national collaboration; SurveyMonkey tips; and a round robin of regional activities.

   Evaluation of the OERC: Cindy Olney created an "Events Data" spreadsheet for tracking and counting OERC activities. She also began planning the evaluation of OERC based on goals in the approved contract proposal.

4. Provide consultation and direction in implementing evaluation systems, structures, and resources to help the NN/LM's efficacy and efficiency.

   Consultations: Consultations are listed in Table 1, above.

5. Enhance and refine use of community-based approaches to planning.

   Cross-regional collaboration: The OERC organized and led an August 14 teleconference with RML Directors and Associate Directors to begin the process for selection of an area for national collaboration. Subsequently, ongoing work included development of a vision statement for the collaboration and dot voting about possible collaboration areas. Several followup Cindy Olney designed and implemented an "Events Data" spreadsheet for tracking and counting OERC activities. She also began planning evaluation of OERC based on goals in the approved contract proposal.

   Conference calls were then held for those who had been unable to join the first discussion: Kathel Dunn, Karen Brewer, and Angela Ruffin on October 2 and Wayne Peay, Sherri Fuller, Cathy Burroughs, and Angela Ruffin on October 6. Later, the OERC led a discussion with the ADs and NLM about top 5 vote-getting areas of possible collaboration and collected readiness ratings from the ADs and participating NLM staff of these top 5 topics.

   Medical Library Association: Susan Barnes participated in an initial meeting with Cathy Burroughs, Carla Funk, and Jean Shipman about the Medical Library Association's Health Information Literacy project.