This
is the overview of an initiative funded
by the Bureau of Primary Health Care (BPHC)
to measure homeless health care outcomes.
Twenty Health Care for the Homeless (HCH)
grantees participated in a study of outcomes
related to the provision of health care
services to homeless people. The first
part of this document contains information
regarding the history of this initiative
and general observations made through
the course of the initiative. Following
this general overview are individual summaries
of the 20 participants final project reports.
Background
Since its inception in 1987, the Federal
HCH Program has supported the delivery
of high-quality, comprehensive primary
care and substance abuse services to hundreds
of thousands of homeless people nationwide
each year. Even though individual project
sites differ from one another with regard
to their organizational structure and
the administration of their programs,
there are certain hallmarks that are typically
an integral part of any HCH program--aggressive
outreach, intensive case management, comprehensive
services, and extreme flexibility in the
delivery of services to patients. These
characteristics are a direct response
to the unique needs of homeless people.
In the Spring of 1996, the BPHC assembled
a work group to consider appropriate strategies
to assess homeless health outcomes. This
group reached a consensus that the importance
of data collection and data analysis derives
first and foremost from the need to be
certain that patients are indeed receiving
the quality health care that they deserve.
It is also important, however, that HCH
programs be able to document and articulate
how and why the services they provide
and the interventions they perform are
so effective. Information about HCH program
performance is needed by officials at
many levels (e.g., Federal, State, and
local policy makers; managed care administrators)
to allow for informed decisions regarding
policies that may dramatically affect
the lives of homeless people.
In the Fall
of 1996, 20 HCH grantees received supplemental funding from the BPHC to assist them
in completing outcome studies related
to homeless health care. A variety of
study areas were chosen ranging from childhood
immunization to patient satisfaction.
For approximately 18 months, these grantees
collected and analyzed data while receiving
research assistance from a research mentor.
Lessons Learned In addition to the study-specific
conclusions reached by individual programs
participating in this initiative, some
overarching conclusions were reached as
well. Some of these lessons have to do
more with the planning process and personal
dynamics that apply in all assessment
efforts, while other lessons have to do
with specific research design or other
technical details. Ultimately, what may
matter most is the application of the
information obtained through the assessment
process. Some of these lessons are discussed
below.
Advance Planning
- The amount of time and energy that needs
to be invested in the planning stages
cannot be overstated. Many of the programs
involved in this initiative found this
to be the most important part of their
projects. A comprehensive planning process
is essential for the successful completion
of any outcome assessment activity. Early
and thorough planning will allow the best
and most appropriate measures to be chosen.
Early planning also allows plenty of time
for adjustments to be made as needed.
Inclusiveness
- Just as everyone in an organization
is important to its proper functioning
day by day, they will all also be an important
part of the success of any study. From
day one, everyone needs to be included
and feel involved. Everyone--support staff,
clinicians, and administrators-- needs
to understand why the study is important
and how it will help them serve their
clients even better. As it seems appropriate,
patients should also be included in planning
and execution of some studies. Patients
are the ones who utilize the services
an organization provides and, therefore,
have the potential to provide valuable
input into study design. Clients should
also be informed of changes they may experience
in the delivery of their health care as
a result of any study.
Questions to Consider
- Once all the key players are poised
to participate, planners must think carefully
about what measure(s) would be most useful
to study. They should consider the services
the organization provides, organizational
strengths, and those services which are
most critical to clients. This analytical
thinking will help with the development
of precise research questions and hypotheses.
Research Question and Hypothesis
- To help with the formulation of research
questions and hypotheses, planning should
consider program objectives or goals.
These objectives should be clear, specific,
and quantifiable. Be specific about what
type(s) of service(s) for what type(s)
of client(s), as well as when and how
often and to how many.
- Although any program has multiple goals
and objectives, it is best to examine
only one or two program goals as part
of a given study.
- The clearer the program objectives are,
the clearer the research questions and
hypotheses will be. A review of program
objectives and goals may lead to implied
research questions or hypotheses. For
example, if one program goal is to provide
prenatal care to all pregnant patients
because there has been a high prevalence
of low birth weight among patients, a
general research question may simply be,
Is the prenatal care helping to ensure
that there are fewer low birth weight
babies?
Similarly, a hypothesis may be that there
is a direct relationship between client
participation in a prenatal care program
and the likelihood of a mother having
a low birth weight baby.
Study Design
- A hypothesis is essentially a statement
about a suspected relationship between
two variables in which one variable is
having some kind of effect on the other.
Therefore, as the hypothesis is refined
and the study design
planned, it is important that these two
variables be as tangible and measurable
as possible.
- It is important to begin with a literature
review or have 5 one done for you (e.g.,
through the Health Care for the Homeless
Information Resource Center, 1-888-439-3300).
A literature review cannot only help with
refining the research question and hypothesis,
it can also help with determining the
best way to obtain the data needed to
explore the research question.
- Although serving homeless clients is
different in many ways from serving other
individuals, including those with low
incomes who are not homeless, there is
no need to reinvent the wheel. Regardless
of the nature of the information that
needs to be collected, there is probably
a proven data collection tool available
somewhere. The literature review as well
as use of the Internet are two ways to
obtain information about these tools.
- Just as important as the tools used
to collect data is the consistency used
while doing data collection.
Whether data is gathered from interviews
or coding information from medical records,
the more consistent the methods used for
gathering the data, the more reliable
the data generated is likely to be.
- An experimental research design is generally
accepted as the type of design that will
yield the most reliable and valid results.
Although there are different types of
reliability and validity, reliability
essentially refers to the repeatability
of results and validity to the generalizability
of the same. That is, the more a study
can be repeated and yield the same results,
the more reliable it is. The more a study
measures or quantifies what it is intended
to measure, the more valid it is. Therefore,
the greater the reliability and validity
of a studys results, the more certain
others can be of the accuracy of the studys
conclusion.
- Performing a study that utilizes an
experimental research design involves
creating two groups which are chosen in
a systematically randomized manner. One
group receives a treatment/intervention,
while the other does not. The purpose
of the randomization process and the use
of a control group is to minimize bias
of any kind and avoid any other potential
threats to reliability and validity. In
so doing, it increases the level of certainty
that it is in fact the intervention that
has made the difference.
- Although the ideal scientific research
model often involves the use of a control
group for comparison
purposes, this is not always practical,
or even ethical. There are other ways
to create a group to allow for some comparison.
A comparison group may be created by finding
a group (retrospectively or prospectively)
with similar characteristics to your primary
group that will be receiving the actual
intervention.
Sampling
- Using a simple random sample may be
a little easier to do than true randomization.
Obtaining a simple random sample (SRS)
essentially involves taking a list of
program users and randomly selecting a
percentage of people from the list to
track. However, even though it may be
easier to obtain a SRS than to randomize
subjects as part of an experimental design,
use of a SRS may still be problematic
from an ethical perspective. Unless a
study is done retrospectively, looking
back in time, ethical issues are likely
to surface because there will always be
the potential for some clients to be denied
care if they are not chosen as part of
the random sample.
- For example, if there are concerns about
depriving clients access to a service,
one option is to designate a period in
the past during which individuals may
have received the particular service that
are being researched, taking a random
sample from this possible pool of subjects.
- However, doing a study in the above-mentioned
manner will mean that threats to reliability
and validity are more likely to occur.
That is, the information obtained from
chart reviews will be limited and will
not allow for as much control over the
study as compared to a prospective study.
- Sometimes it may only be possible to
monitor one group of people. In this instance,
the cohort should be observed both before
and after an intervention, providing some
kind of a baseline measurement. In situations
such as this, it may be best to obtain
multiple measures before and after the
intervention to allow for increased reliability
and validity.
- No matter how good a study design is,
there will always be factors that may
influence the study that will be beyond
anyones control. These influences, or
possible confounding factors, often relate
to the study participants and changes
that they make to their own lives intentionally
or unintentionally.
- At other times, these confounding factors
may have more to do with changes in the
external environment (e.g., a similar
program in the area, a media campaign
on a topic similar to that which is being
studied, as well as the persistent pressures
of poverty and homelessness themselves).
In some instances, there may be a way
to do some statistical adjusting for such
factors during data analysis.
Collaboration
- As planning for the study progresses,
one should consider accessing analytical
resources that have previously not been
utilized. Take advantage of any resources
that may already be in place and seek
out even more. Resources can be books
or articles, but frequently they are people
working for organizations or institutions
(e.g., research organizations, universities,
and state agencies) that possess expertise
needed to complete the study.
- Not surprisingly, many HCH grantees
have collaborated with graduate students
or faculty at area colleges and universities.
This can often be a win-win situation--
students frequently need to find projects
to work on and can provide research expertise
that may otherwise not be available. This
type of collaborative effort can be helpful
during all stages of a project, but it
can be particularly helpful during data
collection and analysis.
Mentors
- Another special type of resource is
a mentor. When identifying potential resources,
look for an individual
with some experience in research and evaluation
to help guide the project.
- The literature review may also be helpful.
If an article(s) is especially helpful,
consider calling the
author(s). In addition, consider contacting
people who have recently completed a similar
study and may be willing to provide some
guidance about their experiences.
Conclusion
While this document is intended to provide
a starting point to help with planning
and completing a study, there is much
additional information that could be obtained
from other sources. To begin, contact
the Health Care for the Homeless Information
Resource Center (1-888-439-3300) for annotated
bibliographies, clinical guidelines, and
other references.
Acknowledgments
The BPHC gratefully acknowledges the hard
work and dedication of the 20 HCH grantees
who participated in this study. In addition,
BPHC thanks Dr. Lillian Gelberg, University
of California at Los Angeles, School of
Medicine, and Dr. Steven Asch, West Los
Angeles Veterans Administration Hospital,
for the many hours of mentoring they provided
to the HCH grantees throughout the study.
Finally, special thanks goes to Ms. Kim
Y. Evans, M.P.H., Health Care for the
Homeless Branch, BPHC, for her tireless
work with the grantees and mentors in
bringing this study to its conclusion.
Summaries
The second portion of this document consists
of summaries of the 20 studies completed
by the HCH grantees that participated
in this initiative. Grantees chose their
topics based on program needs and interests,
therefore the study topics are diverse
and include childhood immunization, patient
satisfaction, case management, tuberculosis,
and HIV/AIDS, just to name a few.
Not surprisingly, the 20 HCH programs
began at different starting points with
different levels of expertise, and therefore
experienced varied levels of success in
their studies. However, all 20 grantees
took a vital step forward in understanding
and appreciating the need for, and usefulness
of such research.
The summaries are organized alphabetically
by state. For additional information about
any of these studies, please write or
telephone the individual listed as the
contact. Please note also that some grantees
are accessible via electronic mail.
