Community Day Evaluation Plan
| Topics on this page: |
|---|
The Outreach Evaluation Resource Center (OERC) OERC will conduct process evaluations of the three Community Day pilot projects, and provide information to the National Network of Libraries of Medicine (NN/LM) National Network Office (NNO) at the National Library of Medicine (NLM) and the three participating Regional Medical Libraries (RMLs) about what was planned, what worked, what was learned, what might have been done differently, and what recommendations should be provided to future Community Day project sites. The OERC will collect this qualitative information through pre-pilot and post-pilot interviews with individuals at the pilot sites managing the Community Day events, and through post-pilot focus group interviews (one for each pilot site) with representatives from Community Day partner organizations (such as first responders, public health agencies, and community-based organizations) that have exhibits at the event.
Evaluation Methods
Pre-pilot Interviews. A first set of interviews will be conducted after pilot sites have planned their activities based on the project objectives identified by the NNO and RML contacts, but before Community Day occurs. In these first interviews, we will ask the individuals managing the pilot sites to expand on their written plans (which will be provided to the OERC by NNO) and talk about how they think their projects are shaping up so far. They will also provide background information such as:- their ideas of what would characterize a successful project
- their expectations for the projects (ie, what they hope to accomplish)
- their previous experience
- existing partnerships and plans for new ones
- other available resources.
Post-pilot Interviews. The OERC will conduct post-Community Day interviews with the individuals at each site who managed Community Day and focus group interviews with participants from up to five Community Day partner organizations. The interviews after Community Day will address these evaluation questions:
- Were the goals for Community Day met?
- What worked about the Community Day activities?
- What could have been done differently and if they had it to do over again, how would they proceed?
- What recommendations do they have for future Community Day projects?
Quantitative data. The pilot sites will be expected to collect data that the RMLs can submit to the Outreach Activities Reporting Form (OARF) system and the Outreach Project Database. The NNO will work with the regions to identify how this data will be collected; methods parallel to the regions usual ways of collecting activity and participant data about outreach projects for the OARF should suffice.
Responsibilities
The OERC will be responsible for:
- Entering the project evaluation plan on the Community Day page of the NN/LM StaffWiki.
- Conducting pre-Community Day interviews with individuals at each pilot site managing the events.
- Working with the pilot site project managers to organize focus groups to be conducted remotely via Adobe Connect.
- Conducting the focus groups. (Focus group participants will be able to participate from their own computers and/or telephone.)
- Completing a written summary of the focus group data within six weeks after all the post-pilot interviews have been completed.
The pilot sites will be responsible for:
- Collecting data to be reported in the Outreach Activities Reporting Form (OARF) system.
- Participating in the pre-pilot and post-pilot interviews.
- Working with the OERC to identify focus group participants (up to 5) among their Community Day partners, then securing an initial commitment from those partners.
The NNO will be responsible for:
- Updating and maintaining the Community Day page of the NN/LM StaffWiki.
- Sending the written plans from the pilot sites to the OERC prior to the pre-Community Day interviews.
- Coordinating with the RML contacts the details of collecting OARF and OPD data with the pilot sites.
The RML contacts will be responsible for:
- Obtaining a written agreement from the pilot sites to participate in the evaluation activities described in this proposal. This agreement could be included in an MOU. Suggested wording for this MOU has been provided below.
- Coordinating (with NNO and the pilot sites) the collection of OARF and OPD data and entering data as needed.
Evaluation Memorandum of Understanding
The pilot sites will be expected to agree to participate in the pre- and post-pilot interviews and to recruit focus group participants from their Community Day partners. The three participating regions will collect the sites agreement via the MOU for the pilots, which could read something like this:
Suggested MOU Wording
Pilot sites will participate in project evaluation through:
- Collecting data to be reported in the Outreach Activities Reporting Form (OARF) system.
-
Participating in pre- and post-pilot interviews conducted by the NN/LM Outreach
Evaluation Resource Center (OERC) about activities planned in relation to NLM's
five objectives for Community Day, which are:
- Strengthen and expand health sciences and public library partnerships.
- Encourage, strengthen and/or expand library partnerships with first responders.
- Provide access to emergency management information tools and resources.
- Integrate and involve libraries into their communities' emergency preparedness, response and recovery planning.
- Promote and publicize the role libraries can play in emergency preparedness.
- Recruiting representatives from up to five Community Day partner organizations to participate in a post-pilot focus group conducted remotely by the NN/LM Outreach Evaluation Resource Center. Participants will be able to participate using their own computer and/or a phone connection.
- Collecting activity and participant data needed to complete the OARF and submitting it to their RML contact to be entered into the system.
- The OERC will be responsible for the following evaluation activities:
- Conducting pre-and post-project interviews and post-project focus groups.
- Coordinating the focus group (e.g., negotiating time and date; sending out links to the Web site where the focus groups are conducted, facilitating the discussions, taking notes, and recording the discussions using Adobe Connect).
- Providing a written summary of the results to the pilot sites, the RMLS, and the National Network Office within six weeks after all the post-pilot interviews have been completed.
