Evaluating Success

Formal evaluation of a tobacco-free campus initiative is an ongoing activity that is necessary to assess whether the initiative has achieved its objective and what tangible outcomes it has had. The evaluation can also identify ways to improve the initiative. Management input can be as valuable in this phase as it is in your initial assessment and planning phases.

Conducting the Evaluation

Begin planning early to ensure that the data you want will be collected during all stages of the initiative. Develop a plan for evaluating the implementation and impact of the policy initiative, including the impact of the expanded cessation services. Remember to keep it fairly simple to assure that the evaluation will be conducted.

Common Evaluation Plan Components

Project name
Sponsoring departments
Contact person
Problem statement
Project goals and objectives for each component of the initiative
Evaluation design
Evaluation logic model
Timeline and budget for evaluation

Below are some sample questions that will help you develop your evaluation design:

What information does management want? How does management define success?
Are employees aware of the new policy and available tobacco cessation services? Are employees who use these services satisfied with the support received?
Are employees satisfied with how the policy is being enforced?
What information is needed to determine whether you have achieved the project goals and objectives?
How will this information be gathered? Are data collection instruments needed (e.g., employee survey)?
Who needs to be involved? Who will collect the data? Who will analyze the data? Who will be responsible for tracking and compiling employee questions and comments?
When should key tasks be completed?
How long will the information be kept and how will the security of the data be ensured?
How will evaluation data be reported?

Consider conducting a pre- and post-implementation employee survey to assess awareness and knowledge of the policy and available tobacco use cessation services. Before collecting employee input, obtain guidance from appropriate agency experts to help determine what approvals are needed. For example, federal agencies are subject to Office of Management and Budget (OMB) regulations and will experience a long lead time if OMB approval is needed. Similarly, if your assessment is part of a research project, check with the appropriate institutional review board (IRB) to determine whether IRB approval is needed. Each agency or company should work with its own internal policy and legal staff to develop appropriate guidelines and procedures for gathering employee input.

To measure the extent and effectiveness of TFC policy enforcement, track the number of employees and visitors who are reported using tobacco on campus on a monthly basis. This can be determined by the number of "health tickets" distributed each month, if you choose to use that communication strategy.

In addition, consider evaluating quit rates after six months among those who have received tobacco use cessation services through the employee health services. For example, track how many employees have

Called or come in for assistance
Visited the TFC Web site
Tried to quit
Received medications
Received a second round of medications
Succeeded in quitting for at least three months

Report evaluation results to the planning team and top management.

Next Steps

After you have completed the initial evaluation of your TFC initiative, consider conducting periodic follow-up assessments to determine whether the initiative is continuing to be implemented as planned and to identify necessary modifications.