Public Health Outreach Evaluation Task Force
- Final Report
- Program Logic Model ( 18kb)
- Appendix ( 16kb)
RML Public Health Outreach Evaluation Task Force Final Report -- June, 2003
Excerpted from Who Will Keep the Public Healthy? Educating Public Health Professionals for the 21st Century, Institute of Medicine (2002):
A public health professional is a person educated in
public health or a related discipline who is employed to
improve health through a population focus. The committee
believes that well-educated public health professionals,
who have a real understanding of the multiple
determinants of health and their interactions (the
ecological model), are critical to shaping new knowledge,
programs and policies relevant to both individual health
and health care, and to population health in the coming
century. These professionals must have a broad range of
skills and information. They must be able to understand
and apply new advances in science (e.g., genetics),
information, and computer science technology to public
health practice and learning (i.e., public health
informatics). They must be proficient in communication in
order to interact effectively with multiple audiences, to
understand and incorporate the needs and perspectives of
culturally diverse communities in public health
interventions and research, and to inform
policy.
Task Force Members
Neil Rambo, Chair
Associate Director
NN/LM Pacific Northwest Region
University of Washington
Seattle, WA
David Carney, MPH
Maryland Department of Health and Mental Hygiene
Baltimore, MD
Betsy Kelly
Assessment and Evaluation Liaison
NN/LM Midcontinental Region
University of Utah
Office Location: Washington University
St. Louis, MO
Julie Kwan
Library Network Coordinator
NN/LM Pacific Southwest Region
University of California, Los Angeles
Los Angeles, CA
Kay McCloskey
Public Health Liaison
NN/LM Midcontinental Region
University of Utah
Salt Lake City, UT
Jocelyn Rankin, Ph.D.
Centers for Disease Control and Prevention
CDC Information Center
Atlanta, GA
Caroline Sparks, Ph.D.
Associate Professor, Department of Prevention and
Community Health
Deputy Director, Prevention Research Center
School of Public Health and Health Services
George Washington University
Washington, D.C.
Virginia (Ginny) Tanji, MEd
Medical Librarian
Library Resource Center
John A. Burns School of Medicine
Honolulu, HI
Resource Staff
Cathy Burroughs
Assistant Director
Outreach Evaluation Resource Center
NN/LM Pacific Northwest Region
Seattle, WA
Keith Cogdill, Ph.D.
Outreach Librarian
NN/LM National Network Office
National Library of Medicine
Bethesda, MD
Angela Ruffin, Ph.D.
Head
NN/LM National Network Office
National Library of Medicine
Bethesda, MD
Task Force Charge
I. Recommend plans for achieving the desired outcomes for
outreach to public health departments identified at the
May 17, 2002 RML Directors Meeting, which include
increases in the numbers of: 1) links to online health
information resources by health departments; 2) public
health department workforce trained; and 3) outreach
collaborations by NN/LM with other agencies.
a. Refine the outcomes that will be measured.
b. Determine how the outcomes will be measured or
counted
c. Determine how the data will be collected and
analyzed.
d. Determine a timeline for data collection, including
defining baseline and/or continuous and/or follow-up
measures.
e. Determine how the data will be synthesized, reported
and used.
II. Recommend ways to increase effectiveness in reaching the outcomes being measured. That is, how can the NN/LM regions overcome barriers and make the most effective progress toward the outcomes of interest? Knowledge or suggestions about best practices might be drawn from our collective RML experience, the audience or institutions we are trying to reach, our contacts/partners, and the published literature.
a. Determine what barriers or problems that currently
exist in reaching the outcomes.
b. Determine what can be done to address these
issues.
c. Determine ways to reasonably and efficiently identify
and collect suggested strategies and "best
practices".
d. Determine ways to disseminate findings to the RMLs in
a timely way that will facilitate their use in the areas
of outreach being assessed.
Background
The task force split into two subgroups, each to address
one of the charges. The first group, to address the first
charge (Recommend plans for achieving the desired
outcomes for outreach to public health departments…)
included Julie Kwan, Betsy Kelly, Kay McCloskey, and
Caroline Sparks. The second group addressed the second
charge (Recommend ways to increase effectiveness in
reaching the outcomes being measured) and included David
Carney, Jocelyn Rankin, and Ginny Tanji. Neil Rambo, as
task force chair, participated in both groups. The
resource staff participated in both groups as they were
able to. The whole task force participated in two
conference calls, Group 1 worked through three additional
conference calls, and Group 2 through an additional two,
for a total of seven conference calls.
Group 1, with the urging of Caroline Sparks, quickly adopted the approach of developing a logic model as a coherent framework to explicate what we mean by outreach to the public health workforce. The work of this group focused on developing and refining a logic model for the three goals or objectives selected at the May 2002 RML Directors Meeting.
Group 2 considered the environment of public health practice and how to target NN/LM outreach to improve the effectiveness of interventions. The group considered such questions as the role of workforce readiness in outreach effectiveness, the need to ensure compatibility between the intervention and the need, and the need for a service network or networks to be available before people can be referred.
Recommendations
The task force recommendations are intended to guide
outreach to the public health practice community by
establishing uniform expectations of what is to be
accomplished, resulting in a baseline of knowledge about
workforce information needs and how to most effectively
meet them. The recommendations are a starting point only.
They delineate a minimum to be met in Year 3. Some RMLs
may have the resources and opportunities to go beyond
these recommendations.
Plans for achieving the desired outcomes for outreach to the public health workforce (task force charge #1) are detailed in the program logic model and appendix developed by the task force. For each of the three outreach objectives that received the most votes at the May 2002 RML Directors meeting, the logic model delineates the resources, activities, project management milestones, outputs, and the desired short- and long-term outcomes. These recommended actions apply specifically to Year 3. Follow on actions for Years 4 and 5 will need to be developed later.
Refer to the logic model for details. The summary recommendations are:
Objective 1. Increase links to NLM resources on
public health agency web sites.
Each RML will develop a plan to promote adding links to
NLM resources to public health agency web sites, at least
at the state level. This endeavor can be based on an
analysis of agency websites for links to NLM resources,
but this is optional and not required. All agencies found
to lack links to NLM will be contacted and encouraged to
add them.
The logic model appendix explains what is meant by "public health agency". Recommended points of contact at state health departments include: Distance Learning Coordinators, Health Alert Network Coordinators, deputy secretaries, division directors (of epidemiology or nursing or environmental health), or training or workforce development coordinators. Network member libraries may have easier entrée to local public health agencies, which can be sponsored by RMLs.
In the process of accomplishing Obj. 1, contacts will
be made and there will be opportunities for raising
awareness and establishing relationships.
Objective 2. Improve skills in use of NLM resources
by public health professionals.
Those public health workforce subgroups that are most
likely to benefit from being trained to use NLM resources
to meet their professional information needs will be
targeted. A public health training workgroup will be
formed to develop training modules to be used by each RML
to train the public health workforce. The workgroup will
include librarians who are knowledgeable of public health
workforce information needs and uses, and experienced in
training public health workers. Public health
professionals will also be represented on the workgroup.
NNO staff will provide project management leadership for
development of the training modules and will be the
communication link between the workgroup and Outreach
Coordinators. Each RML will offer the training at least
twice.
Some RMLs will be able to accomplish more, and learn
more, than is suggested for this objective. This is to be
encouraged. An attempt will be made to share the
knowledge and expertise of the training workgroup about
workforce information needs with RML staff. This will be
explored as part of the interface between the workgroup
and the Outreach Coordinators..
Objective 3. Increase relationships and
collaboration among public health agencies and health
science libraries.
The recommended activity is actually preliminary to
increasing relationships and collaboration and thought to
be a necessary foundation to build on. Each RML will
assess what library services are available to the state
level public health workforce. In doing so, each RML will
gain an understanding of service network relationships
and gaps that exist in access to services. Each RML will
recommend how the NN/LM can work with other networks to
fill identified gaps.
A task force member pointed out that in order to refer
public health professionals to library services there
have to be services to refer them to. It is not well
understood what library services are available to public
health workers in most states, or through what networks.
Nor do we know much about the nature or extent of the
current or potential use of library service.
The assessment outlined for this objective will be the
basis for increasing relationships among public health
agencies and libraries. As with Objective #1, the process
will open opportunities for raising awareness and
establishing relationships.
----------------------------------------------------------------------------------------------------
Left to be done for all three objectives, is
consideration of the collection, analysis, synthesis, and
reporting of data resulting from these activities, as
well as how to evaluate the desired outcomes.
-----------------------------------------------------------------------------------------------------
Recommendations of ways to increase effectiveness in reaching the outcomes (task force charge #2) are woven into the activities and project management milestones of the logic model. Ways to decrease barriers and increase effectiveness are based on the synthesis of lessons learned from the NLM-funded public health outreach projects as presented at the Apr. 2001 forum at NLM. These include:
1. Not all of the public health workforce has a need
for knowledge-based information resources (e.g., journal
literature), in their work. Most NN/LM outreach staff are
trained as medical librarians and are knowledgeable in
using the literature and related tools (e.g., PubMed).
Identifying segments of the workforce that have a need
for this science-based information is an approach that is
likely to increase the chances of success. For example,
those engaged in research, program design, and policy
analysis are likely users. Others with educational
backgrounds and professional training that included use
of the literature may have developed a pattern of
continuing to use the literature of their field as a
normal course of doing their work.
2. Assessing information needs may have to be an
iterative process leading to a refinement of objectives.
Taking the time to understand information needs and
developing a response that addresses those needs is worth
the investment.
3. Short-term outcomes may be 1) increased awareness
of information resources and their application; 2)
increased access and decreased barriers to use of
information resources; and, 3) increased skills in using
information resources.
4. Long-term outcomes may be 1) a change in
information seeking behavior, and use of information
resources, leading to improved job performance; 2) the
establishment of library networks to serve the public
health workforce; and 3) other collaborations between
agencies and libraries.
5. The support of leaders is important in underscoring the importance of using information resources as an instance of implementing evidence-based practice. Reaching and getting the support of leaders should be part of outreach intervention planning.
References
1. Rambo, N. Zenan, JS. Alpi KM. Burroughs CM. Cahn MA.
Public Health Outreach Forum: lessons learned. Bull Med
Libr Assoc. 2001 October; 89 (4): 403406.
Full text HTML and PDF formats can be found at http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=57972
2. O'Carroll, P. Rambo, N. Information Needs and Uses of the Public Health Workforce -- Washington, 1997-1998. MMWR Weekly February 18, 2000; 49(06);118-120.
Full text HTML at http://www.cdc.gov/mmwr/preview/mmwrhtml/mm4906a3.htm
3. W.K. Kellog Foundation Logic Model Development Guide , October 2000.
4. Who Will Keep the Public Healthy? Educating Public Health Professionals for the 21st Century. K. Gebbie, L. Rosenstock, LM. Hernandez, Editors, Committee on Educating Public Health Professionals for the 21st Century. Institute of Medicine. 2003.