Primary Navigation for the CDC Website
CDC en Español

Case Studies

Comprehensive Treatment of Tobacco Dependence in Oregon

Return to Index
Previous
Next


Evaluation

Type(s) of Evaluation Planned or Conducted and Status

 

What is the status of your evaluation?

Completed

Do you address process evaluation?

Process measures are incorporated into the overall evaluation plan. In the OHP (Medicaid) program, process measurement is undertaken to determine the degree to which the recommended clinical guidelines regarding "the 5 A's" are incorporated into general practice, and the degree to which the plans are providing comprehensive assistance (pharmacotherapy and counseling).

In the early stages, plans were required to report their progress in developing their systems to assure advice and assistance. Next, measurement was three pronged, including chart auditing, member surveys and plan administrative data. Chart audits are conducted by the OHP agency's external quality arm to determine the percentage of enrollees to whom providers gave personalized advice to quit at least one time during the reporting year (Advise) and for those indicating an interest in quitting, the percentage whose medical records indicate a Quit Plan including the proposed timeframe for quitting, follow-up or the provider's referral to other staff for follow-up. (Assist & Arrange) Tobacco prevalence and advice to quit were next tracked through OHP member surveys.

Tracking of the participating health plans' actual volume of cessation services to their members, continues including the provision of pharmacological (NRT, Zyban) and counseling support. A one-time-only survey of providers and members by an outside researcher assessed awareness of the OHP benefit.

Operations data used for the process evaluation of the Quitline are collected by the program contractor and reported on a monthly basis. These process measures include:

On a statewide basis, the Tobacco Prevention & Education Program (TPEP) collects data and monitors the following measures:

Do you address outcome evaluation?

For the Quitline, annual satisfaction surveys are conducted to assess overall satisfaction with the cessation services. A second survey is conducted to assess quit rates among those who are six months beyond their initial call to the Quitline. Lastly, a 6-cell clinical trial is currently underway that evaluates the cost-effectiveness of three different levels of counseling service offered by the Quitline, each level offered with and without nicotine replacement therapy (NRT).

Briefly describe the evaluation design.

For satisfaction surveys, data are collected annually via telephone survey of all callers between two weeks to one month from the date of their call to the Quitline. The occasional surveys to determine 6-month quit rates are conducted using a sample of randomly selected callers that are six months beyond their initial call. The clinical trials looking at the cost-effectiveness of different service levels utilize a 6-cell randomized control trial, with and without NRT and with and without the "Free and Clear" or 4-call comprehensive approach to cessation support. All "result" surveys are conducted by independent data collection firms or independent researchers. Sample sizes and stratification are determined to address particular topics of interest and have varied over time. Augmentations of the state BRFSS survey measure statewide progress.

Data Collection Methods

Data Source

Range of Intended Outcomes

List key evaluation findings and/or conclusions for each intended outcome.

The Oregon cessation program has produced measurable, successful results. As indicated by the significant increase over 3 years in the percentage of smokers who have been advised to quit and offered assistance to do so by their physician, the program's efforts to change the health care system have been working. Of equal importance, the percentage of smokers who think that cessation assistance would help has increased from 56% to 62%. Other improvements include measurable declines in the percent of the Oregon population that smoke and in annual per capita cigarette sales. As of August, 2000, Oregon's annual rate of cigarette sales per capita has dropped more significantly since the program's inception than observed for the 46 states without cessation programs during the same timeframe.

The Quitline component of Oregon's program has also proven to be a success. Satisfaction with services is extremely high and 80% of those using the service have made at least one quit attempt after contacting the Quitline. Using the "intent to treat" analysis methodology at 6-months after the first call, 13% have been smoke- free (defined as going without "even a puff") for at least 7 days. This percentage is comparable to other statewide, real-world evaluations published in the literature. Additionally, this percentage should be compared to findings in the Morbidity & Mortality Weekly Report (MMWR, 7/26/02) which indicate that only 4.1% of the general population were found to be successful when quitting on their own. (It is very important to note that many quit lines are not using this rigorous "intent to treat" standard for quitting outcomes. Unless people who are not reached for follow-up evaluation are included in the denominator measure, programs cannot be compared. See evaluation notes)

Were evaluation findings and/or conclusions disseminated to policy and/or program intervention stakeholders?

Evaluation findings and /or conclusions have been disseminated widely to various relevant audiences. Evaluation findings have been presented to meetings of TOFCO, the health services task force and are incorporated among statistics included in the Medicaid measurements.

Briefly describe how evaluation findings and/or conclusions were used to inform program planning or development?

Evaluation findings and/or conclusions have been incorporated at the level of both process and outcome. The Oregon program has identified opportunities for improvement in the need to clearly explain the overall program strategy to stakeholders, particularly as it pertains to ensuring adequate funding levels. Other modifications have been made in provider training protocol and in determining the amount of media required for communication.

For the Quitline component, the process evaluations have been used to fine-tune program requirements such as the elimination of screening calls, modifications to the content of the "quit kits" and in understanding the need to more effectively educate health plan enrollees about the availability of cessation benefits. Process data were also used to determine which components of the overall media promotion were more or less effective. For example, the data show that print ads have been the least effective form of program promotion.

Evaluation Notes

It is important to note that the outcomes and evaluation findings presented here reflect performance in a "real-world" study of smoking cessation. In the context of clinical trial analyses, cessation rates may differ as a result of the characteristics of study subjects. Motivation levels among all Oregon smokers are likely to vary more widely than for those participating in clinical trials of cessation services.

Denominators used in the evaluation of Oregon's program are based on all Quitline callers who sought one-time counseling. Note that a sample replacement method was not used. Using the less rigorous standard of counting only the 67% of people who were reached at follow-up, the quit rate was 21%.

Return to Index
Previous
Next


Return to Top

 

Page last modified 07/25/2007