State Program Evaluation Guides:
Writing SMART Objectives

This document is also available in Portable Document Format (PDF–149K). Learn more about PDFs.

Guides:
	Writing SMART Objectives
	Developing and Using a Logic Model
	Developing an Evaluation Plan
	Fundamentals of Evaluating Partnerships

The evaluation guide Writing SMART Objectives is aimed at helping states develop realistic and measurable objectives. This guide describes the components of a SMART objective and provides examples. An exercise at the end offers an opportunity to work through the development of SMART objectives. We will appreciate your comments on the utility and applicability of this guide.

As part of the program development process, states propose objectives and develop a work plan and an evaluation plan. Both these plans are based on proposed objectives so it is important that they are developed using the SMART approach. Such objectives offer specific, relevant, and measurable benchmarks to measure achievement of your state’s HDSP program goals and will serve as the foundation for your program activities.

Goals and Objectives

In program planning, we often think of what we want to accomplish in terms of goals and objectives so that we can easily explain our expectations to others.

A goal is a statement that explains what the program wishes to accomplish. It sets the fundamental, long-range direction. Typically, goals are broad general statements.

Example: Improve control of high blood pressure in (state).

Objectives break the goal down into smaller parts that provide specific, measurable actions by which the goal can be accomplished. Objectives define for our stakeholders and partners the results we expect to achieve in our program or intervention. For our program expectations to be clear, we must write clear, concise objectives.

The two general types of objectives are process and outcome. Process objectives focus on the activities to be completed in a specific time period. They enable accountability by setting specific activities to be completed by specific dates. Process objectives explain what you are doing and when you will do it. They describe participants, interactions, and activities.

Example: By June 30, 2006, provide training for 20 community health center administrators in the use of electronic medical records.

Outcome objectives express the intended results or accomplishments of program or intervention activities. They most often focus on changes in policy, a system, the environment, knowledge, attitudes, or behavior.

Objectives can also be thought of as short-term, intermediate or long-term.

• Short-term objectives are generally expected immediately and can occur soon after the program or intervention is implemented, very often within a year.

• Intermediate objectives result from and follow short-term outcomes.

• Long-term objectives state the ultimate expected impact of the program or intervention.

Example: Let’s put all these concepts together to form a series of related objectives.

For the short-term objective listed above, activities or process objectives could include the following:

• By December 30, 2006, provide training for 20 community health center administrators on the impact of using guidelines and electronic medical records.
   
• By February 2006, form one collaborative of at least five health centers to examine issues related to blood pressure control.
   
• By May, 2006, provide one train-the-trainer program on the JNC7 guidelines for collaborative members.

Objectives are meant to be realistic targets for the program or project. They are written in the active voice and use action verbs such as plan, write, conduct, and produce (rather than more vague terms like learn, understand, feel). Well-written objectives will always answer the following question:

WHO is going to do WHAT, WHEN, and TO WHAT EXTENT?

States develop an HDSP work plan using short-term, intermediate, and long-term objectives for capacity building and interventions. Capacity building objectives will likely focus on the achievement of products or processes, such as developing a state plan, a partnership, or a burden document that will help the state reduce heart disease and stroke. Intervention objectives should focus on processes such as training, or on outcomes such as systems change or health status change. The state work plan will also include activities or processes needed to achieve proposed objectives.

Developing SMART Objectives

One way to develop well-written objectives is to use the SMART approach. Developing specific, measurable objectives requires time, orderly thinking, and a clear picture of the results expected from program activities. The more specific your objectives are, the easier it will be to demonstrate success.

SMART stands for
  Specific
  Measurable
  Attainable/Achievable
  Relevant
  Time bound

Specific—What exactly are we going to do for whom?

The “specific” part of an objective tells us what will change for whom in concrete terms. It identifies the population or setting, and specific actions that will result. In some cases it is appropriate to indicate how the change will be implemented (e.g., through training, or through implementation of the Chronic Care Model). Coordinate, partner, support, facilitate, and enhance are not good verbs to use in objectives because they are vague and difficult to measure. On the other hand, verbs such as provide, train, publish, increase, decrease, schedule, or purchase indicate clearly what will be done.

Measurable—Is it quantifiable and can WE measure it?

Measurable implies the ability to count or otherwise quantify an activity or its results. It also means that the source of and mechanism for collecting measurement data are identified, and that collection of these data is feasible for your program or partners.

A baseline measurement is required to document change (e.g., to measure percentage increase or decrease). If the baseline is unknown or will be measured as a first activity step, that should be indicated in the objective as “baseline to be determined using HRSA database, 2005.” The data source you are using and the year the baseline was obtained should always be specified in or adjacent to your objective statement. If a specific measurement instrument is used, you might want to incorporate its use into the objective. For example, “By June 2007, increase the proportion of physicians at the Green Clinic who are 100% compliant with the JNC7 Guidelines from 70% to 80% as measured by the Physician Guideline Self-Assessment Tool.” specifies not only the performance measure, but the data source as well.

Another important consideration is whether change can be measured in a meaningful and interpretable way given the accuracy of the measurement tool and method. For example, to estimate population awareness of the signs and symptoms of heart attack, we estimate awareness using a sample of the state population. Since this is an estimate, there is a chance of error associated with it—usually expressed by a confidence interval (the point estimate, plus or minus an estimate of variability). Projecting a very small change in population awareness, although measurable, might not be meaningful because the change projected falls within expected variability or within the bounds of the confidence interval for population awareness.

Attainable/Achievable—Can we get it done in the proposed time frame with the resources and support we have available?

The objective must be feasible with the available resources, appropriately limited in scope, and within the program’s control and influence.

Sometimes, specifying an expected level of change can be tricky. To help identify a target, talk with an epidemiologist, look at historical trends, read reports or articles published in the scientific or other literature, look at national expectations for change, and look at programs with similar objectives. Consult with partners or stakeholders about their experiences. Often, talking to colleagues in other states who have implemented similar programs or interventions can provide you with information about expected change.

In some situations, it is more important to consider the percentage of change as a number of people when discussing impact. Will the effort required to create the amount of change be a good use of your limited resources?

For example, our intervention might be intended to increase awareness of the symptoms of stroke and the need to call 9–1–1 among patients in a statewide health clinic system. If as a result of our intervention we measure a 5% increase in awareness among all clinic patients, but 5% of our population is a very small number, we might want to consider the cost of the intervention relative to the number of people affected. We could choose to enhance the intervention for a greater impact or not implement that intervention again.

Relevant—Will this objective have an effect on the desired goal or strategy?

Relevant relates to the relationship between the objective and the overall goals of the program or purpose of the intervention. Evidence of relevancy can come from a literature review, best practices, or your theory of change. For state HDSP programs, the objective should accomplish one of the following:

• Directly lead to achieving or enhancing one of the required recipient activities.
   
• Directly lead to a desired change in one of the CDC HDSP priority areas (controlling high blood pressure or high cholesterol, increasing knowledge of signs and symptoms, improving health care or emergency response, or eliminating disparity).
   
• Directly lead to a policy or system-level change in a priority setting.

For example, although it may be important to public health, assessing the height and weight of high school students does not directly lead to change in an HDSP priority area for an at-risk population. Also, the intervention focuses at an individual level rather than on a system change.

Time bound—When will this objective be accomplished?

A specified and reasonable time frame should be incorporated into the objective statement. This should take into consideration the environment in which the change must be achieved, the scope of the change expected, and how it fits into the overall work plan. It could be indicated as “By December 2010, the HDSP program will” or “Within 6 months of receiving the grant,...”

Using SMART Objectives

Writing SMART objectives also helps you to think about and identify elements of the evaluation plan and measurement, namely indicators and performance measures.

An indicator is what you will measure to obtain observable evidence of accomplishments, changes made, or progress achieved. Indicators describe the type of data you will need to answer your evaluation questions. A SMART objective often tells you what you will measure.

Consider the example “By February 15, 2006, increase by four the number of community health centers in [State] that have incorporated into the clinic system electronic medical records with reminders of treatment protocols.” The indicator is

the number of community health centers in [State] that have incorporated electronic medical records with reminders of treatment protocols into the clinic system.

A performance measure is the amount of change or progress achieved toward a specific goal or objective. SMART objectives can serve as your performance measures because they provide the specific information needed to identify expected results.

Consider the example, “By February 15, 2006, increase by four the number of community health centers in [State] that have incorporated into the clinic system electronic medical records with reminders of treatment protocols.” The performance measure is

increase by four the number of community health centers in [State] that have incorporated electronic medical records with reminders of treatment protocols into the clinic system.

Getting Started

To develop SMART objectives, use the template below and fill in the blanks:

By_____/_____/_____, ___________________________________________________
[WHEN—Time bound]   [WHO/WHAT—Specific]

from _____________________ to __________________________________________
[MEASURE (number, rate, percentage of change and baseline)—Measurable]

Next, review your objective to assure that it is achievable and relevant to your overall program goals.

Here are some examples of SMART objectives:

• By June 29, 2006 (time bound), increase the number of training sessions given for HDSP program partners on “Implementing and Evaluating System Change” (specific & relevant) from 10 to 14 (measurable & achievable).
   
• By December 31, 2009 (time bound), increase awareness of the signs and symptoms of stroke and the importance of calling 9–1–1 among African American men in [State] (specific & relevant) from 11% to 15% (Baseline: 2005 BRFSS) (measurable & achievable).
   
• By February 15, 2006 (time bound), increase by four (measurable & achievable) the number of community health centers in [State] that have incorporated into the clinic system electronic medical records with reminders of treatment protocols (specific & relevant).

Each of these objectives is either a capacity building activity or directly relates to one of the HDSP program areas and will be a policy and systems-level change.

Exercise

Take the following objectives and “make them SMART.”

1. Increase the number of HDSP partners.

______________________________________________________________
______________________________________________________________

2. Train physicians on clinical practice guidelines.

______________________________________________________________
______________________________________________________________

3. Enhance EMS policy to decrease stroke deaths in 2002.

______________________________________________________________
______________________________________________________________

4. By June 2007, increase by 10% the percentage of state residents that know the signs and symptoms of stroke and heart attack.

______________________________________________________________
______________________________________________________________

5. Increase the number of work sites that adopt heart-healthy insurance options.

______________________________________________________________
______________________________________________________________

Making them SMART

There are many ways that these objectives could be made SMARTer. For each example, several things to think about as you review your answer are provided below.

1. Increase the number of HDSP partners. The objective needs to specify how many and by when. Do you have a baseline? Is the proposed increase reasonable given the context and available resources? Does the program seek a specific kind of partner or partners from a specific setting? What action must be taken for someone to be considered a partner (e.g., a written agreement)?
   
   One SMART example might read: By June 29, 2006, increase from 12 to 14 the number of HDSP program partners that represent the health care setting and have signed an “HDSP Program Letter of Partner Commitment.”
    
2. Train physicians on clinical practice guidelines. How many physicians? By when? Which guidelines? Do you expect them to increase their knowledge? How will you measure that? Do you expect them to increase guideline use in their practice? Are you training a specific type of physician or physicians in a certain geographic region?
   
   One SMART example might read: By June 29, 2007, a minimum of 50 [State] primary care physicians who attend training will achieve a score of 90% or higher on the “HDSP Program Blood Pressure Standardization” post training test.
    
3. Enhance EMS policy to decrease stroke deaths in 2002. What does “enhance” mean? How will you measure “enhance”? By when? Is this realistic? Will you measure policy adoption or decreased stroke deaths?
   
   One SMART example might read: By December 31, 2008, [State] Emergency Medical Services will adopt a statewide policy to transport stroke patients to the nearest certified hospital.
    
4. By June 2007, increase by 10% the percentage of [state] residents who know the signs and symptoms of stroke and heart attack. Is this 10% of a number or 10 percentage points? Is this a meaningful increase? Do you have a baseline? What is the data source? Does “know” mean all the signs and symptoms? Of both heart attack and stroke? Does this apply to all residents or a specific group or region? Adults only or are teens included? Is it reasonable, given the level of activities, to see this amount of increase population wide? Is it achievable in the time frame stated? Is a data source available to measure change at the appropriate time, at the level we have targeted the intervention? (The signs and symptoms BRFSS module measures state-level change—typically not county-level data—and is planned for implementation in 2005 and 2009.)
   
   One SMART example might read: By December 30, 2009, increase by 10 percentage points from baseline, the percentage of adult males in [State] who know all five of the signs of stroke (as listed by the American Stroke Association) and who also know to call 9–1–1 if stroke is suspected. Baseline to be determined by 2005 BRFSS.
    
5. Increase the number of work sites that adopt heart-healthy insurance options. Increase by how much? By when? Where? All work sites or a subset? What does “adopt” mean (how will you know when it’s adopted)? What exactly are “heart-healthy insurance options” (is there a list)? What data source(s) are you using or how will you document an increase?
   
   One SMART example might read: By December 31, 2008, 50 work sites that participated in the Chamber of Commerce education day will offer employee health insurance benefit packages that include comprehensive rehabilitation services for heart attack and stroke survivors.

Acknowledgements

This guide was developed for the Division for Heart Disease and Stroke Prevention under the leadership of Susan Ladd and Jan Jernigan in collaboration with Nancy Watkins, Rosanne Farris, Belinda Minta, and Sherene Brown.

State Heart Disease and Stroke Prevention programs were invaluable in the development and fine-tuning of this guidance document. Their review contributed significantly to the clarity and utility of this guide. Special thanks are extended to the following:

Susan Mormann, North Dakota Department of Health,
Ghazala Perveen, Kansas Department of Health and Environment,
Ahba Varma, North Carolina Department of Health and Human Services, and
Namvar Zohoori, Arkansas Department of Health and Human Services.

We encourage readers to adapt and share the tools and resources in the document to meet program evaluation needs. For further information, contact the Division for Heart Disease and Stroke Prevention, Applied Research and Evaluation Branch at cdcinfo@cdc.gov or (990) 488–2424.

Bibliography

American Heart Association. Heart Disease and Stroke Statistics–2006 Update. Dallas, Tex: American Heart Association; 2006.

Centers for Disease Control and Prevention. Prevention Works: CDC Strategies for a Heart-Healthy and Stroke-Free America. Atlanta, GA: U.S. Department of Health and Human Services; 2003. Available at http://www.cdc.gov/DHDSP/library/prevention_works/index.htm.
 

Return to Top of Page

Page last reviewed: October 15, 2008
Page last modified: October 15, 2008
Content source: Division for Heart Disease and Stroke Prevention, National Center for Chronic Disease Prevention and Health Promotion