Collect Patient Data

Simply comparing outcomes to costs for an entire program does not provide enough information to allow for systematic improvement of a treatment program for several reasons:

• At any given time, patients will have been exposed to varying amounts of treatment.

• Patients respond differently to different treatment procedures.

• Patients with different backgrounds and drug abuse histories may respond differently to the same treatment procedure.

• Differences in backgrounds between therapists and patients may influence the response to treatment.

What is needed is a record, for each patient separately, of the patient's characteristics, types and amounts of outcomes achieved, treatment procedures used, and resources expended. These individual findings can then be combined to show the effectiveness of the program as a whole and of its individual parts.

To understand whether and how a treatment procedure is responsible for outcomes observed, it is also useful to collect data on psychological, social, and possibly biological processes that occur within individual patients. Although difficult to observe directly, these biopsychosocial processes translate what is done in treatment -the procedures enacted by counselors and others -into the end results of treatment -the outcomes.

Before Treatment Begins

Patient Characteristics

Most of the data on patient characteristics such as the following are collected during the intake process:

• Date of birth

• Gender

• Ethnic background

• Employment status

• Marital status

• Drugs abused

• Physical health

• Severity of the abuse problem -the Addiction Severity Index is commonly used to measure this.

• Contacts with the criminal justice system

• Date treatment begins

This information is kept in the patient's file. In a large program with many patients, especially if computerized, the data could be recorded on a Patient spreadsheet similar to the spreadsheets developed to track costs, with characteristics as rows and patients as columns.

Provider Characteristics

To be able to determine whether patient outcomes are related to characteristics of the program staff, record date of birth, gender, marital status, ethnic background, experience, and training for all therapists and staff members who interact with patients. These can be coded as provider data and entered on the Patient spreadsheet, if used.

Prescribed Treatment

Some measure of amount of treatment that the patient has received should be developed. This could be time in treatment, percentage of prescribed treatment completed, or some other measure, depending on the type of program. A patient who has finished only half the program cannot fairly be compared with patients who have completed all their treatment.

In order to determine the proportion of treatment received, you will need to know the amount of treatment prescribed. In most cases, this can be recorded as hours per procedure. In all cases, amount of treatment recommended should be expressed numerically, if possible.

Process Measures

Several well-tested instruments for assessing specific problem areas are available:

• General brain functioning
  • Trail Making Test

• Neurological impairment
  • Porteus Maze

• Depression
  • Beck Depression Inventory (BDI) (Beck and Steer 1988)

• Anxiety
  • Manifest Anxiety Scale
  • State-Trait Anger Expression Inventory (STAXI)

• Health functioning
  • SF-36

• Motivation
  • Addiction Severity Index (ASI) (McLellan et al. 1985a, b)
  • CMRS (Circumstances, Motivation, Readiness and Suitability) (DeLeon and Jainchill 1986)

One instrument, the Symptom Check List, 90-item version, revised (SCL-90-R) (Derogatis 1979), is designed for diagnosis of multiple mental health problems and collection of related demographic data. It can be used to measure the following processes that should change during substance abuse treatment:

• Somatization
• Obsessive-compulsive
• Interpersonal sensitivity
• Depression
• Anxiety
• Hostility
• Phobic anxiety
• Paranoid ideation
• Psychoticism
• Global Severity Index
• Positive Symptom Distress Index
• Positive Symptom Total

Some instruments have been developed that, although more time-consuming than those listed above, provide measures of almost any process that could be posited as active in substance abusers:

• Minnesota Multiphasic Personality Inventory, version 2 (MMPI-2)

• Millon Clinical Multiaxial Inventory-II (MCMI-II)

• Structured Clinical Interview for the Diagnostic and Statistical Manual of the American Psychiatric Association, version IV (SCID)

In addition, some comprehensive process assessment instruments have been developed specifically for substance abuse treatment. One such instrument is the Individual Assessment Profile (IAP) (Flynn et al., 1995).

Researchers have found that some personality factors measured by process assessment instruments are indeed related to drug preferences (Craig 1979; Flynn et al. 1995; Mirin et al. 1988); however, diagnoses are not reliably related to treatment outcomes.

Standardized tests yield scores or ratings that can be entered in the patient's file or on a Patient spreadsheet. These before-treatment scores provide a baseline for comparison later in treatment.

During and After Treatment

Most clinical researchers and program evaluators begin outcome measurement when a program has ended. Drug treatment is, however, different from many other programs: Patients typically begin dropping out soon after treatment begins. Other patients are excluded early from many drug treatments. If counseling sessions, medication, and other procedures are interim outcomes, then outcome measures and followup on treatment effects begin as soon as treatment begins.

Standardized Effectiveness Measures

There are many ways to record patient progress. Standardized measures give you the flexibility to look at each measure separately or to combine all measures for an overall cost-effectiveness analysis.

To standardize different outcome measures, find a common unit in which they can be measured or a common scale for all measures. This may not be as impossible as you might first think. A common unit for different measures, such as drug abstinence and employment, could be days, where drug-free days and days of employment are treated as equivalent, positive outcomes.

A common scale could be used for less observable measures, such as emotional maturity and quality of relationships with others, where "1" on the scale means "much less than desirable for recovery" and "10" on the same scale means "as much as is desirable for recovery."

Another way to standardize measures is to measure them at the beginning and end of treatment (or at least earlier and later in treatment). Calculate the percentage change in the measure, and the effectiveness of the program on all measures will be in the same units. For example, if days employed per month increased from 5 to 10, a 100-percent improvement occurred. If drug-free days per month increased from 5 to 20, a 300-percent change occurred.

Followup

How long does one have to follow a patient to determine whether the cessation of substance abuse is permanent? This is a very difficult question to answer conclusively. Relapses to drug use have been recorded 5 and even 10 years after the last use of the substance. One way to determine the length of followup is to specify an interval that is convenient, affordable, or typical. The typical interval is between 1 and 2 years, although longer periods are desirable.

Another way to answer the question "When will we know for sure that the patient has or has not succeeded in kicking the habit?" is to challenge the validity of the question. A number of therapists and researchers believe that addiction is a lifelong process that may never end completely. Instead, the interval between uses of the addictive substance may be increased markedly by treatment. The duration of the relapse and the dose of the substance may be decreased by treatment. The addiction, however, may continue forever.

If this is the approach to treatment outcome, followup becomes a potentially perpetual process; however, budget limits and evaluator interest typically limit duration of followup in these programs to a few years.

Should Dropouts Be Included?

It is tempting to exclude from evaluations of treatment outcomes data for patients who have dropped out of treatment. Counselors may rightly feel that behaviors exhibited by patients who quit treatment in the first month or two do not represent the real effectiveness of treatment. Indeed, these patients have not received the minimum necessary "dose" of treatment and probably do not show how effective it can be.

Nevertheless, patients who dropped out did consume resources during treatment. Because intake is an expensive procedure in most substance abuse programs, it would be inaccurate to distribute those intake costs across patients who stay in treatment. When costs are examined, that approach would penalize programs that have higher dropout rates.

Also, if dropouts are excluded from analyses of program outcomes, and if the costs of treating dropouts are excluded as well, treatments that exclude all patients except those who succeeded will appear more successful than treatments that persist in trying to help patients with more serious problems. This creaming (as it is called by program evaluators) can produce findings of apparent effectiveness, cost-effectiveness, and cost-benefit that cannot be generalized to most other programs. The problems of ignoring dropouts become severe if the early dropout rate is a significant percentage of the total number of patients who seek treatment from the program.

Of several resolutions possible for the issue of what one does with outcome and cost data for dropouts, the most satisfactory for most programs is to include the cost of treatment for dropouts and to think of staying in treatment as a crucial interim outcome.

The perspective from which outcomes are being assessed also suggests including dropouts in outcome measurement. The dominant perspective in most substance abuse treatment is that of the community or society. This is due to the widespread effects that substance abuse can have on the public as well as the public nature of much funding for substance abuse treatment.

From this perspective, the question that outcome and cost measures should answer is not, "How effective and inexpensive can treatment be?" It also is not, "How effective and inexpensive was treatment for those who finished the basic course of treatment procedures?" From the perspective of the community and of society at large, the question of outcome is, "How many of those who needed to cease their addictive behavior actually did so, did so permanently, and at what actual cost?"

Another way to look at this is to ask, "How much does it cost to operate our program and what do we get from those costs?" From this perspective, serving dropouts, for even a short time, is part of what the program does. Costs are associated with this service. Further, using the CPPOA model allows for associating costs and resources as well as procedures specific to dropouts so that ways to reduce these costs or change procedures to reduce dropout rates can be adopted.

[Manual Index] [Previous Section] [Next Section - Find Cost-Effectiveness]