This guide provides definitions and instructions for
constructing and using the logic model worksheets to help Child
Support Enforcement agencies specify—clearly and
concretely—what their projects are trying to accomplish, for
whom, with what resources, in what context, facing what
barriers. It will also help agencies to specify the data
needed to show whether the project has been effective in achieving
its goals.
The logic model worksheets are meant as a guide for project
staff and evaluators. You may adapt them to fit your project.
The boxes on the forms may be too small to include all the
information you want to provide. You may use a blank page and label
it with titles corresponding to appropriate boxes on the logic
model worksheets. In addition to the detailed worksheets, we
recommend developing a one-page logic model that highlights key
features of your project in addition to any more
detailed/descriptive information you may produce. The
one-page format is a helpful tool to give those who will review and
rate your proposal a good overview of your project. It is also a
helpful tool to educate your staff and partners of the thrust of
your proposal; equivalent to an outline upon which you would
describe the details during a presentation.
A. Overview: Logic Model
Framework
A logic model is a plausible and sensible model for how a
program is supposed to work.[1] It is a road map—a visual representation of
a program or project, and what it is supposed to accomplish.
A logic model is a useful tool for linking what a program does to
what it hopes to achieve and how to measure that achievement.
A logic model can be useful in program planning, program
implementation, project management, and program evaluation.
It is a highly useful tool in developing and organizing a grant
application, and when strategizing any program improvement.
A logic model can be used for program planning; it can also be
used for performance monitoring and project refinement. For program
planning, the logic model should be developed during the project
design phase, beginning first by specifying the ultimate goals
desired and the underlying assumptions regarding how this goal can
be achieved. It identifies where you are going and how you
plan to get there. It is important that project partners
collaborate in (see Exhibit 1):
- Defining program goals. What is the overarching
goal you seek to achieve? Why are you thinking about
intervening?
- Specifying underlying assumptions. What are your
underlying assumptions about the nature of the problem,
clients’ needs, and what will help? Why does the problem
exist? Why do you think it is amenable to intervention? Do clients
need the intervention? Want it?
- Identifying key contextual factors so
that project partners make explicit and agree upon—before
they decide on a program approach—why they are collaborating,
what they are trying to achieve and for whom. What is the
environment in which your program operates? This can be state and
local issues (e.g., funding, leadership) or organizational (e.g.,
capacity).
- Identifying necessary inputs. What resources are
being used (or are needed) to operate the program? For example,
these can include funding, staff, time, materials, space.
- Specifying project interventions. What are
you doing (or do you plan to do), for whom, with the funding you
request? An intervention is a specific cluster of activities
targeted to a specific group of people. An intervention is
not usually a single activity but rather a multi-pronged
approach. Each “prong” should be
listed.
- Describing the outputs expected to result from
project activities. What will the program produce immediately with
respect to participation and service delivery? Each intervention
activity has a corresponding output. Outputs have relevant
time frames. They are short-term and quantifiable.
- Detailing the immediate and subsequent
outcomes—or benefits—expected for program
participants. What are you trying to change with the intervention?
What are the expected results? How will participants benefit?
A logic model can also be used after a project is developed to
assess performance and refine the project. For example:
- Are you doing what you proposed to do? If not, why not?
What changes are necessary?
- Are you producing what you said you would produce? If
not, why not? What changes are necessary?
- Are you achieving desired results?
Exhibit 1: Logic Model Framework
B. Logic Model Worksheets
The following instructions—and accompanying
worksheets—are designed to help you develop a logic model for
your project. These worksheets are presented in the order in
which you should proceed with developing your logic model.
Worksheet #1: ULTIMATE PROJECT GOALS
The first step in creating a logic model is defining the
ultimate project goal. In other words, where are you going
with the intervention?
This worksheet will assist project partners in clearly defining
what they ultimately hope to achieve with the project. It has
three boxes. The first is overall project goals. Why
are you even thinking about intervening? Who needs your assistance?
What’s the problem you’re trying to address?
Project goals are broad, long-term, ultimate outcomes sought
for the target population—for example, “economic
self-sufficiency,” “improved material well-being of
children,” and “increased father involvement in
children’s lives.”
As the first two boxes indicate, project goals should align with
the legislative goals of the funding authority and with child
support and collaborating agency goals. For example, the
national CSE strategic plan (which may be found on the OCSE web
site - http://www.acf.dhhs.gov/programs/cse/) lists the
following strategic planning goals for 2005-2009:
1. All children have parentage established.
2. All children in IV-D cases have support orders.
3. All children in IV-D cases have medical coverage.
4. All children in IV-D cases receive financial support from
parents as ordered.
5. The IV-D program will be efficient and responsive in
its operations.
If all of the expected immediate and intermediate outcomes are
achieved, then program goals are likely to be met. Depending
on the resources available for a program evaluation, program goals
may or may not be translated into measurable outcomes to be
assessed in the program evaluation.
Worksheet #2: UNDERLYING ASSUMPTIONS
The purpose of this worksheet is to make explicit what is often
left implicit in program development—namely the beliefs or
assumptions upon which the project is built. As the worksheet
indicates, there are four areas of assumptions to
consider:
- What are assumptions surrounding the problem being
addressed? Why does the problem exist? Why do you think it is
ameliorable to intervention?
- What are the assumptions about the needs of the clients you
expect to serve? Do clients want this intervention? What
other resources or avenues for amelioration of the problem do
clients have?
- What are the assumptions surrounding the characteristics of the
clients expected to participate?
- What assumptions are you making about why the approach will
work? Do you have evidence from research or other program
evaluations that it should work?
A key purpose in clarifying and making explicit underlying
assumptions is to make sure project partners are “on the same
page” in terms of why they are collaborating and what they
are trying to achieve, for whom. Toward this end, this worksheet
allows project partners to document key assumptions and any
principles underlying and guiding the development and
implementation of the proposed project.
Worksheet #3: CONTEXT
This worksheet describes the major factors in the environment
that are likely to affect the project, and to clarify the
characteristics and relationships between the organizations most
closely involved in the project. Looking back at the overall
logic model framework (Exhibit 1), note that arrows from the
context box to all other boxes in the logic model illustrate how
every aspect of a project is embedded in a larger context. As
this worksheet indicates, context is captured by six boxes:
(1) characteristics of the environment; (2) site characteristics;
(3) agency characteristics; (4) other related interventions; (5)
other targeted resources; and (6) other organizational
arrangements. Each is described below.
- Characteristics of the Environment.
What are the key characteristics of the general environment
that you believe might affect the project in some way? These could
include changes in agency funding or leadership, local community
program expansions or closures, the economy, and population
demographics.
- Site Characteristics. What are the
characteristics of the site(s) in which interventions will be
provided that have implications for the program’s success? If
a child support initiative is targeting unmarried parents in local
hospitals, for example, you may want to note how many births occur
in the hospital each year, how hospital services are staffed, and
staff turnover. Other characteristics relevant to a site
include accessibility (e.g., on public transportation routes) and
other on-site services (e.g., genetic testing). If multiple
sites are involved, and if project activities will differ across
sites, you may want to use a different worksheet for each site.
- Agency Characteristics. What are
the characteristics of the sponsoring organization? These include
the type of organization, its size, the services it offers, its
service area, and any number of other characteristics that you
think are important to understand the role the project will have
on, and the resources it will be able to draw upon from, the
sponsoring agency. Collaborative arrangements between the
agency and other programs (e.g., state or local government
programs, community-based organizations) should be noted.
- Other Related Interventions. What
are the characteristics of other programs in the
environment—whether sponsored by your organization, other
organizations, or supported by grants? Do these interventions
duplicate or complement the activities proposed by the
agency?
- Other Targeted Resources. What are
other programs operated by the agency (whether by its own funding
or by special grants) that represent joint ventures or
collaborative efforts with other agencies?
- Other Organizational Relationships.
What are other community programs or organizations from which you
could expect to get referrals? What are community organizations to
which you could/would refer clients for services not provided by
the program/project? Identify the extent to which personal
and/or contractual relationships already exist between your agency
and these other organizations, or whether informal and/or formal
relationships need to be forged.
Worksheet #4: INPUTS TO INTERVENTIONS
This worksheet identifies inputs (e.g., child support staff,
hospital social workers, marriage education programs) necessary to
provide the intervention. In designing a program, monitoring
program performance and refining program design, it is critical to
understand which inputs are required for the intervention so that
resources can be allocated—or reallocated—as
needed. The worksheet includes three boxes.
- Agency Inputs. What are inputs (such as
staff and major equipment) available from the agency?
- Inputs from Partners. What are
important resources that are contributed to the project by your
partners? These inputs can be represented by a financial
contribution or formal assignments of staff (includes in-kind
contributions). Inputs from partners may include such things
as space, equipment, and staff involvement (whether paid or
unpaid).
- Interventions. What are your
interventions? An “intervention” is defined as a
specific set of activities aimed at a specific group of people
(e.g., unmarried fathers, unmarried couples) to accomplish specific
objectives (e.g., educate the non-custodial parent on the
importance of fathers in children’s lives; paternity
establishment; inform of child support procedures). An
intervention is not usually a single activity, but a multi-pronged
approach. For example, an intervention might be “offer
paternity establishment in hospitals,” which involves many
activities (such as training in-hospital paternity establishment
staff, creating paternity establishment informational materials,
and talking with unmarried fathers about paternity
establishment). Defining the intervention requires that you
fully describe what specific activities you will do; with what
intensity/frequency/dosage; for whom; and over what period of
time.
You should not find yourself with a long list of interventions;
if you have such a list, you should see if you can group them based
on overlapping objectives. You may want to number the
interventions and use the number on subsequent worksheets to
minimize repeating the detailed information. It is important
to specify a target population for each intervention, and to
specify an intervention (and corresponding set of activities) for
each target population.
Check to make sure that all inputs necessary to
provide each intervention are listed.
Worksheet #5: INTERVENTIONS TO EXPECTED OUTPUTS
What will the interventions produce that will ultimately benefit
participants? For example, how many workshops will you hold?
How many brochures will you distribute? How many referrals will be
made? How many participants will you serve with each intervention
activity? Participation and service delivery outputs can
include the number of staff trained, the number of workshops held,
the number of couples served.
This worksheet has two boxes.
- Interventions. This repeats the
information from Worksheet #4. Numbers or other shorthand
labels for the interventions described in Worksheet #4 can be used
to minimize copying.
- Expected Outputs. What is each
intervention expected to produce? Outputs refer both to
building capacity for providing services (e.g., number of
staff trained, number of interagency informational presentations
made) and the actual products of service delivery (e.g.,
number of clients served, number of brochures distributed,
etc.). Outputs are short term—immediately following an
intervention (e.g., a marriage education class). Because
these numbers are only meaningful in relation to a specific time
frame, the relevant time frame should be noted. Expected
outputs should be linked directly to each specific
intervention. There can be multiple outputs from a single
intervention; similarly, more than one intervention could
contribute to a single output. Expected outputs reflect estimates
only; they will help you and the program staff understand the level
of activity expected in relation to specific intervention
activities. Expected outputs may change over time, as your
program learns about the needs of the target population and
interventions are adapted and refined.
If your project includes both a capacity-building intervention
and a direct service delivery intervention, you may want to develop
separate logic models for each, given that the outputs and outcomes
of capacity-building interventions (e.g., number of staff
trained) usually translate into antecedent “inputs”
required for service delivery interventions.
Note that outputs differ from
outcomes. The former is a direct product of the
intervention. The latter, as the next worksheet demonstrates,
tracks expected results and benefits to the participants.
Check to make sure that each expected output is
associated with at least one intervention, and that
each intervention has at least one expected
output associated with it.
Worksheet #6: EXPECTED OUTPUTS TO EXPECTED OUTCOMES
Whereas outputs are what the program intervention will produce
with respect to participation and service delivery, outcomes are
actual benefits to the participant. The purpose of this
worksheet is to identify the outcomes you expect, for whom, and by
when, as a result of the intervention’s outputs. In
other words, what will show that you’ve made a difference in
your participants’ lives?
There are three boxes on this worksheet. Two boxes are
labeled “Expected Outcomes”—one delineates
immediate outcomes, and one delineates subsequent
(longer-term) outcomes. This is because outcomes occur over
time and, oftentimes, an earlier outcome has to have occurred
before a subsequent outcome can occur. To reflect this
element of time, we’ve divided Expected
Outcomes into Immediate and
Subsequent. More than one intervention may
contribute to a single outcome, and multiple outcomes may result
from a single intervention.
This worksheet is extremely important. Your agency has the best
chance of achieving its ultimate project goals the more closely
aligned the program interventions are with the expected outcomes.
Use one Worksheet #6 for each intervention, and list the
intervention being addressed at the top, using the name of the
intervention or its number consistent with those shown on
Worksheets 4 and 5.
Start by transferring expected outputs from Worksheet #5.
Then, in each of the boxes labeled Expected Outcomes,
indicate the specific outcome expected—for whom, and by
when—as a result of implementing the intervention with the
target population.
- Expected Immediate Outcomes. What
is expected to occur immediately following exposure to the
intervention, as a direct result of participation in project
activities? For whom will it occur? By when? For
example, if the intervention is providing information on paternity
establishment (its importance and/or how to establish paternity) in
the hospital, then outcomes expected immediately could include an
increase in knowledge about the importance of fathers in
children’s lives, and an increase in paternity
acknowledgments in the hospital. If the intervention is a marriage
education class, outcomes you would expect immediately (upon
completion of the program) include greater knowledge and skills
relating to communication and conflict-resolution, greater
marital/relationship satisfaction, more positive attitudes toward
marriage, and an intent to apply the knowledge and skills learned.
Expected immediate outcomes typically must occur before
subsequent outcomes of ultimate interest can occur. For
example, if paternity is not established, then a child support
order cannot be established for that child (subsequent outcome) and
this could compromise the child’s future financial well-being
(ultimate project goal). Or, if couples do not gain improved
knowledge and skills from a marriage education program, then there
may be little hope of witnessing changes in conflict-resolution
behavior—or ultimately, in child well-being—as a result
of the program.
- Expected Subsequent Outcomes. What
intermediate and longer-term outcomes that require either more time
to elapse or changes in preceding outcomes before change in these
subsequent outcomes can be expected? For example, if an
intervention is designed to increase in-hospital paternity
establishment, one would expect a subsequent increase in child
support orders (an intermediate outcome), which should translate
into increased child support payments by the non-custodial father
to his child(ren) (another intermediate outcome) and, thus,
improved financial well-being of his child(ren) (the ultimate
goal). Or if an intervention is designed to improve
communication skills (the immediate outcome) of non-married
parents, one would expect a subsequent improvement in (reduction
of) marital/interparental conflict (an intermediate outcome), which
may then lead to greater financial and emotional contributions by
the non-custodial father to his child(ren) (another intermediate
outcome) and, thus, improved child well-being (the ultimate
goal).
The distinction between outcomes expected immediately and
outcomes expected subsequently is not always clear, but it does
require you to think about the causal sequence of what must happen
in order to achieve the ultimate goal. Distinguishing immediate
from subsequent outcomes should also lead you to ask, “How
long do we think it will take before we would observe a
change—in attitudes, knowledge, skills, behaviors, child
well-being—as a result of this intervention?” and
“How long do we think these changes will last?”
Your answers to the “How long before change is
observed?” question may have implications for when you
should collect data to document program performance and for
outcomes evaluation. And your answers to the “How long
will it last?” question may have implications for program
design and refinement—for example, instituting booster
sessions and other follow-up services to sustain positive outcomes
for clients.
Check to make sure that each expected immediate
outcome is associated with at least one expected
output, and that each expected output has at
least one expected immediate outcome associated with
it.
Check also to make sure that each expected subsequent
outcome is associated with at least one immediate
outcome, and that each expected immediate
outcome has at least one expected subsequent
outcome associated with it.
Worksheet #7: MEASURING OUTCOMES
This worksheet will help you think about how to measure the
outcomes you have identified. There are three columns.
- Immediate outcomes. In the
first column of Worksheet #7, list the immediate outcomes you
identified in Worksheet #6.
- Indicators. In the second column,
list an indicator (or measure) for each outcome. An
‘Indicator’ is the specific information you will
collect or already have available to you to track this
outcome. For example, the number of in-hospital paternity
affidavits signed is an indicator.
- Evidence of Change. Finally, in the
third column, list what you would consider evidence of change on
this outcome—or how much change in this outcome (as measured
by this indicator) is necessary for you to conclude that your
project was a success. Using the example above, the number of
signed in-hospital paternity affidavits before and after the
intervention is evidence of change. You may have numerical
targets, based on experience and your assumptions about how much
your intervention will affect this outcome (as reflected by this
measure). Or you may be tracking an entirely new outcome,
requiring a new and different measure than what has been used in
the past; in this case, you may not know what target levels (or
amount of change over time) should be expected. Or you may
have an experimental “control” group (identical, on
average, to your program group, due to random assignment) or a
quasi-experimental “comparison” group (identified as
“similar enough” to the program group) whose outcomes
you are also tracking and can compare to the program group.
Note, however, that even if your evaluation design allows you to
detect statistically significant differences between the
program and control/comparison group and you use this as evidence
of change, you should still articulate numerical targets that you
would like your program group to attain, in order to speak to the
meaningfulness of the program outcome on an absolute level.
Therefore, it is especially important to articulate
what outcomes and/or changes you consider “a success,”
though you should be open to reassessing your outcome targets
and/or your measurement strategy as the project proceeds.
Once you design your interventions and specify what you
expect to happen, you can think about the challenges you may
face and plan accordingly. A logic model can help you monitor
program implementation and refine intervention activities as
necessary. In this way, it is a “living” document
that continues to be useful beyond the project planning stage.
The next set of worksheets is designed to help you think through
the practical issues you will face in recruiting clients into
program services and in implementing program services, and help you
devise a strategy for refining the program as necessary.
Worksheet #8: Barriers to Implementation
The purpose of this worksheet is to identify hurdles you might
face and how you will overcome them. That is, what factors
impede implementing the project and what are the possible solutions
to these barriers? Generally speaking, these include barriers
to recruiting target populations into program services, and
barriers to delivering program services. Use one
Worksheet #8 for each intervention, and list the intervention being
addressed at the top of the worksheet, using the name of the
intervention or its number consistent with those shown on other
worksheets.
There are four boxes on the worksheet. Two focus on
barriers, and two focus on possible solutions to the identified
barriers.
Potential Barriers are factors that may slow down,
or impede, implementation of a particular intervention
activity. These may or may not be factors that can be changed
by you agency’s efforts. It is important to identify
these factors regardless of whether the project can do anything
about them. But factors that can be addressed through
program modifications and refinement should certainly be
identified.
- Barriers to Recruiting Clients and Possible
Solutions. What hurdles might you face (or are you
facing) in recruiting the target population into services?
This includes marketing/advertising, recruitment/outreach,
enrollment (registration, intake), and retention/program
completion. For example, what helps or hinders the
recruitment of males into marriage education programs? What helps
or hinders parents’ on-going participation (lack of child
care? transportation? too little time in a day?). What
factors in the project site and outside environment interfere with
or facilitate reaching target populations? After identifying each
possible (or, during implementation, each actual)
barrier, you should identify a concrete action step (possible
solution) for addressing this barrier.
- Barriers to Service Delivery and Possible
Solutions. What hurdles might you face (or are
you facing) in delivering services to the target population?
This includes difficulties securing inputs (e.g., staff, materials,
meeting space), unforeseen challenges in working with the target
population (e.g., cultural/language barriers) and/or in working
with project partners (e.g., lack of support from key management or
community leaders). After identifying each possible
(or, during implementation, each actual) barrier, you should
identify a concrete action step (possible solution) for addressing
this barrier.
Information on barriers is descriptive and may be easiest to
record in a narrative form, but it needs to be linked to the
interventions and each target population. And again, barriers
and possible solutions may change as the project evolves. The
goal is that—through on-going project monitoring and
refinement—barriers are eliminated, strengths are built upon,
and the program finds ways to effectively recruit, enroll, retain,
and serve clients at the level proposed for the duration of the
program.
Worksheet #9: USING INFORMATION ON ACTUAL OUTPUTS TO REFINE INTERVENTION
The purpose of this worksheet is to help you anticipate (during
planning) and monitor (during actual implementation) potential
problems in program implementation as indicated by information on
project outputs. In other words, how will you stay on
track?
There are three boxes in this worksheet.
- Expected Outputs. This repeats the
information on expected outputs, for each intervention, from
Worksheets 5 and 6.
- Actual Outputs. This box allows you to
indicate the level of outputs you are currently actually achieving
(noting over what period time these outputs have been obtained).
Comparing numbers from the actual outputs box to numbers in
the expected output box, you can quickly see whether you are
achieving what you expected. In fact, you may want to bold or
otherwise highlight which outputs fell short of expectations, thus
indicating a possible need to refine the intervention (or to adjust
expectations/assumptions about when outputs would be achieved or
the level of outputs attainable).
- Refinement of Intervention. How will you
use information on expected versus actual outputs to identify how
you may need to refine your intervention activities? List options
for how to refine the program to increase the likelihood that you
will achieve expected outputs. In general, this involves
strengthening the inputs (e.g., increase staff time) and/or
improving the process by which inputs translate into outputs
(e.g., hold project team meetings). If your in-hospital paternity
establishment intervention is not yielding the number of trained
hospital-based paternity establishment staff initially proposed,
then you need to ask yourself why—is it how the training is
being conducted, or who is doing the training, or which hospital
staff are being targeted for the training—and refine the
intervention accordingly, perhaps by modifying where, when, how
in-hospital staff are trained or how fathers are approached in the
hospital.
Worksheet #10: USING INFORMATION ON ACTUAL OUTCOMES TO REFINE INTERVENTION
This worksheet will help you anticipate (during planning) and
monitor (during actual implementation) potential problems in
program implementation as indicated by information on project
outcomes.
There are three boxes.
- Expected Outcomes. This repeats the
information on expected outcomes, for each intervention, from
Worksheet #6.
- Actual Outcomes. This box allows you to
note the level of outcomes you are currently actually achieving
(noting over what period time these outcomes have been obtained).
Comparing numbers from the actual outcomes box to numbers in
the expected outcomes box, you can quickly see whether you
are achieving what you expected. Again, you may want to bold
or otherwise highlight which outcomes fell short of expectations,
indicating a possible need to refine the intervention (or to adjust
expectations/assumptions about when outcomes would be achieved or
the level of outcomes attainable).
- Refinement of Intervention. This
helps you explore how you can use information on expected versus
actual outcomes to identify if and how you may need to refine your
intervention activities, then list options for how to refine the
program to increase the likelihood that you will achieve expected
outcomes. This can involve:
1. Strengthening the intervention activities (e.g.,
improve paternity establishment informational materials; increase
the quality/content/duration of marriage education workshops).
2. Increasing outputs (e.g., increase the number of
fathers participating in the program).
3. Improving the process by which outputs translate into
outcomes.
4. Improving the process by which immediate outcomes lead
to subsequent outcomes (e.g., improve the process by which
paternity establishment results in establishment of a child support
order and child support payments).
Worksheet #11: BUILDING THE ONE-PAGE LOGIC MODEL
This worksheet requires you to pull together information from
the previous, detailed worksheets to come up with a one-page
picture—or road map—of your entire project.
As shown below and described earlier, there are eight boxes in
your final logic model:
(1) Project goals
(2) Underlying assumptions
(3) Context
(4) Inputs
(5) Interventions and activities
(6) Outputs
(7) Immediate outcomes
(8) Subsequent outcomes
You may find it a challenge to distill key information from
earlier worksheets and have it “fit” on this single
page. Do your best; it will be worth the effort. It
will be enormously helpful to you (to keep the big picture
in mind and see how all the pieces fit together), project
staff (they can see how their role fits into and is
indispensable to the overall project), current and potential
partners (who want to know what you plan to do, for whom, with
what resources, and what role is expected of them), and
potential funders (who need to know that you have a clear
program goal and a well thought-out, logically coherent set of
activities that are highly likely to result in the achievement of
that goal).
[1] Bickman, L.
(Ed.) (1987). Using program theory in evaluation. In New
Directions for Program Evaluation, Series: No. 33. San
Fransisco: Jossey-Bass, p. 4.