Before you can analyze your program, you need to define the outcomes you are seeking and the program components that contribute to the patient outcomes. Before you can change your program based on your analysis, all interested parties must share a common understanding of what treatment is and what it is trying to do. To that end, it may be useful for the major interest groups to construct a model of the service system. The worksheets at the end of the chapter may help you in discussions with program staff and other interest groups.

Cost-Procedure-Process-Outcome Analysis Model

Table 2 and Table 3 outline a model for cost-procedure-process-outcome analysis (CPPOA) (Yates 1996). Table 2 shows the basic model in a simple flowchart with arrows representing which parts of the model influence which other parts in ongoing treatment. These arrows show the primary direction of action. Feedback from outcomes back to procedures is characteristic of good program management and could be represented by a loop from the box "Interim and Long-Term Outcomes" to "Program Procedures." There also is a feedback loop between outcomes and costs: If outcomes are positive, expenditure of additional resources is justified.

The CPPOA model in table 3 is more detailed. It lists several possible measures for each of the major parts of the model. For example, individual therapy, group therapy, and health education are listed under procedures. Measures such as employment and independent living appear under outcomes. Treatment programs and therapists have their own theories about what are and are not the important psychosocial processes to address in substance abusers; they also have their own treatment procedures for changing those processes. Because of this, a universal set of processes and procedures is difficult to establish for CPPOA. The following processes and procedures only illustrate how the ideas of processes and procedure-process relationships can be used to understand, evaluate, and improve a treatment program. You will want to select your own procedures and processes to describe your program.

Procedures

The major procedures used in providing substance abuse treatment to patients include -

• Individual counseling.
• Group counseling.
• Acupuncture.
• Pharmacotherapy.
• Education about human immunodeficiency virus and sexually transmitted diseases.
• Vocational counseling.

Dividing your treatment program into specific procedures is one of the most important steps in your cost-effectiveness analysis. These are the activities that you will later decide to retain, to enhance, to diminish, or to drop altogether. For this purpose, we suggest that you consider at least four different treatment procedures. For manageability, however, do not record more than 15 or 20 procedures.

It is especially important to consult with direct service staff when defining program procedures. As a first step, consider what sets of actions make separate contributions to patient outcomes. For instance, it seems likely that individual counseling and group counseling make separate contributions.

Definitions are needed for each procedure. The definitions should be clear enough that program staff can reliably agree when one procedure or another is being performed. Some procedures (such as individual counseling) will be easier to define reliably than other procedures (e.g., confrontational counseling). Specific program components should also be defined. For example, a confrontational group may be defined as a therapeutic intervention where 8 to 12 patients address another patient on inappropriate behavior in a group setting facilitated and monitored by one or more counselors.

Your cost-related analysis will be most useful if the procedure definitions used in your program also correspond closely to procedures used in similar programs. A publication by the Center for Substance Abuse (Crowe and Reeves 1995) defines treatment modalities and program components for substance abuse programs. For instance, pharmacotherapy is typically defined as a treatment using approved medications to reduce substance abuse.

Most procedures can be divided into more specific procedures. There are many types of individual counseling, for example. If reflective, analytic, behavioral, and other types of counseling are practiced in your program and contribute to program outcomes to different degrees, you probably should establish each form of counseling as a separate procedure. Some programs use confrontational as well as supportive and educational forms of group counseling, which also can be denoted as different procedures.

Also, if different staffing levels or qualifications are necessary for some procedures, those procedures should be considered separately (because their costs will probably differ). For instance, if a feelings group is facilitated by a paraprofessional while a psychotherapy group is conducted by a licensed psychologist, these two types of groups should be considered separately.

The defined procedures should be all-inclusive; when considered together, they should constitute the entire program. To do this, you may need to add to a list of specific therapeutic procedures a catch-all category such as "other treatment activities." Be sure, however, that this category does not become a dumping ground. It should include only procedures used one time or infrequently.

Processes

The same treatment procedures work for some people but not others because a moderating process either facilitates or inhibits the impact of treatment procedures on outcome. A process internal to the patient can be created or encouraged by treatment procedures. For example, some counselors believe that personal growth and responsibility are crucial processes that treatment must foster. Other processes internal to the patient may be targets of treatment procedures designed to blunt or even eliminate them. For instance, some treatment procedures attempt to reduce self-destructive impulses and highly selfish, manipulative processes in patients.

Therapists often believe that outcomes are the product of changes in patient processes that are themselves the product of treatment procedures. Unfortunately, we cannot assume that these processes are the ones that were at work. The outcomes of treatment could have been due to entirely different processes. Also, the procedure could have changed different processes than those intended. The processes that actually were changed may or may not have then produced the outcomes.

To get a clearer picture of which links are active between procedures and processes, and between processes and outcomes, you need to measure the procedures, the likely processes changed by those procedures, and the targeted outcomes.

The most important processes to measure are those that counselors and other treatment providers believe to be the crucial determinants of program outcome. In some cases, these will correspond to psychological or other processes for which reliable instruments have been developed. In other cases, the processes active in the patient that determine whether treatment succeeds or fails may have to be measured by instruments you develop.

The first step in selecting or developing instruments to measure processes is to ask therapists to explain their theories of what processes need to occur for treatment to succeed and what other processes can prevent this. The following sections discuss common processes involved in substance abuse treatment.

Psychological Disorders

Because persons who have psychological disorders may be more likely to abuse substances, psychological problems are addressed in most substance abuse treatments. Mental illness is more common in substance abusers than in nonsubstance abusers according to a number of studies (e.g., Ross et al. 1988).

The presence of psychological problems may moderate the impact of treatment. The problems may impede treatment or, if psychological processes are at a severe phase at the beginning of treatment, more rather than less improvement may result (Friedman and Glickman 1987).

Some treatment providers hope that reducing negative mental processes will subsequently reduce or stop substance abuse. Common psychological disorders in substance abusers that may be the focus of treatment procedures include -

• Antisocial personality disorder.
• Phobias.
• Psychosexual dysfunction.
• Major depression.
• Dysthymia (moderate depression).

Other Biopsychosocial Processes

Some substance abuse programs believe that one or more of the following processes within the patient must change to achieve outcome goals:

• Expectancies of reinforcement or punishment
• Certain attitudes and belief systems
• Destructive or self-centered interpersonal dynamics

These treatment programs believe that psychological, social, and perhaps biological processes must be changed before patients successfully and permanently cease substance abuse.

Readiness to Change

Yet another way to conceptualize the biopsychosocial processes involved in substance abuse cessation is to say that the processes that need to be addressed by treatment procedures are determined by how ready the patient is to change. This approach to understanding the process of change says that most patients are, at any moment in treatment, in one of several distinct stages of increasing readiness to change. To move the patient to the next stage, certain biopsychosocial processes need to occur. These crucial processes are evoked by specific treatment interventions or procedures. The different processes necessary for transition between stages may require different treatment procedures (DiClemente 1993; Prochaska and DiClemente 1986).

Procedure-Process Links

The next step is to define the specific relationships between the treatment procedures and the processes they are designed to either encourage or discourage. Asking for all the procedure-process links also provides a check on the completeness of the process list. If some procedures remain for which no processes are specified, either additional processes need to be described or the procedure may be unnecessary.

Table 4 gives a sample matrix of a program's processes and procedures. The cells of this matrix indicate each possible combination of procedures and processes in the treatment program. The cells in the table are filled in with numbers indicating the strength of the relationship between each procedure and each process. Working with the program staff to specify the procedure-process links helps build a better cost-procedure-process-outcome model that is easier to analyze later.

It is extremely important to be very tactful when you ask therapists for information or suggest changes. Most treatment providers develop their procedures for substance abuse treatment over long periods of intense training. They have accumulated considerable experience in and wisdom about what works for which patients, how well, and when. It is important to work with the staff to find the best way to describe the presence, strength, or absence of critical psychosocial processes. These descriptions usually lead to methods of measuring the processes.

Tailor Procedure-Process Links to Your Setting

Suppose, for example, that the majority of counselors at a clinic express a firm belief that substance abuse is caused by (a) a strong desire to escape a deplorable situation, (b) a reluctance to face adult responsibilities, and (c) a wish to harm or kill oneself. Moreover, suppose that these counselors believe that, for the patient to cease and maintain cessation of substance abuse, each of these causes must be moderated or worked around while abstinence is maintained. Your task, then, is to find or develop measures of processes (a), (b), and (c) above. The counselors also should be able to specify the treatment procedures that they use to address processes (a) through (c), so you can help them measure the occurrence of each procedure for each patient.

Process (b), a reluctance to face adult responsibilities, might produce the following list of results that counselors expect their procedures to yield:

• Patient keeps treatment and other appointments.
• Patient does not miss work.
• Patient pays bills on time.
• Patient resolves outstanding legal issues.

The counselors then would identify the procedures used to change the process: individual counseling, daily scheduling, monthly budget development, and role modeling. The final step is to measure the process. In this example, a monthly checklist can measure the number of days missed at work, the number of missed appointments, the number of bills paid, and so on -all signs that the desired process is occurring. That documentation charts the patient changes related to the process.

Outcomes

Measuring the impact of treatment procedures is key to analyzing and improving cost-effectiveness and cost benefits. While the primary outcome desired by all substance abuse treatment programs is total, permanent abstinence from illicit drugs, achieving that goal requires patients to make many major changes in lifestyle, attitudes, friends, skills, and so forth. A patient who has not made the necessary adjustments to a drug-free life has a high probability of relapse.

These changes thus become desirable outcomes in themselves. Some programs may consider them interim outcomes while others may see them as final outcomes. Choosing which outcomes represent success in a substance abuse treatment program depends greatly on the theoretical basis of the treatment approach.

Objective Effectiveness Measures

Outcome measurement has a relatively long history in substance abuse treatment. The focus has long been on "real results" rather than on measures that seem indirectly related to the problems that initiated treatment in the first place. Such objective measures of effectiveness include -

• Biological measures of drug use: analyses of urine, blood, breath, hair.
• Biological measures of infections related partially or fully to drug use: HIV tests (negative or positive, and immune system cell counts), hepatitis status, sexually transmitted disease infection status.
• Criminal convictions, arrests.

Objective measures are important because their validity is high, as is their acceptance by a broad range of interest groups. It is therefore crucial for treatment programs to collect objective measures, whether they are the ultimate goals of substance abuse treatment or the means to other ends.

Subjective Effectiveness Measures

Certain interest groups believe that subjective measures are unimportant. For example, a questionnaire designed to tap an individual's level of maturity or personal development strikes some funders as rather different from what "really counts" -the number of assaults and thefts committed during or after treatment. Nevertheless, many therapists, patients, patient associates, and researchers are concerned with such measures of treatment effectiveness as -

• Self-reports of illicit drug use.
• Self-reports of alcohol and tobacco use.
• Productivity on the job.
• Depression, anxiety.
• Patient functioning in different areas, such as family living, employment, education.
• Physical health.
• Psychological well-being.

Each program must define its goals for patients in ways that can be measured. Most of these will be improvements along a continuum. Some may be staff estimates of change. Whenever possible, objective or external measures should be used; staff reports may be perceived as biased in favor of the program.

Benefits

Objective monetary benefits of substance abuse treatment include the following:

• Financial records from accountants, funders, and tax agencies of legal employment during and after, versus before, substance abuse treatment.
• Records of welfare benefits paid during and after, versus before, substance abuse treatment.
• Records of public health services used during and after, versus before, substance abuse treatment.
• Records of funds spent on arrests, convictions, and other interactions of the patient with the criminal justice system during and after, versus before, substance abuse treatment.

Process-Outcome Links

Individual processes can also be linked to outcomes for a more refined analysis of your program effectiveness. Table 5 shows a sample matrix with the estimated contribution of each process to each outcome.

Processes Versus Interim Outcomes

As you work with program staff to define outcomes and processes, you may find that overall outcomes are easy to define (e.g., permanent cessation of all addictive behaviors), but that many intermediary outcomes are being proposed (e.g., recovery from a brief relapse). There is a point at which processes stop and outcomes begin. Events occurring inside the patient, whether psychological or biological, usually are processes. Events occurring outside the patient usually are outcomes.

For example, enhanced self-efficacy for substance abuse cessation may be the result of treatment procedures, but it is rarely an end in itself. Self-efficacy for substance abuse cessation is enhanced by treatment procedures as a means to an end -permanent cessation of substance abuse.

Treatment programs may put certain outcomes before others. These intermediary outcomes may include patient compliance with a regimen of weekly counseling sessions, daily methadone maintenance, legal employment, or a combination of these and other outcomes. These outcomes occur outside the patient and are themselves the result of changes in processes (e.g., expectations of rewards for compliance with the regimen). They are intermediate or interim outcomes, however, because they are not the outcome for which treatment is designed and funded. That ultimate or final outcome is cessation of drug use.

Interest Group Differences

Sometimes, one person's process measure is another person's final outcome measure. For certain researchers, therapists, and patients, the goal of treatment is to change the patients' internal state -to make them mentally and physically healthy. For other interest groups, including much of the tax-paying public, mental and physical health are intervening processes at best. For these interest groups, the goals of treatment are to get patients off drugs, to keep them off drugs, to stop them from committing criminal acts, and to help them become net benefits rather than net costs to society.

A potentially useful strategy for dealing with interest groups advocating different procedures, processes, and outcomes is to acknowledge the importance of each measure and to structure the analyses so that all measures are included. Some analyses can show how well different treatment procedures produce the different process measures. Additional analyses can find out how well the same procedures produce the various outcome measures. Further analyses can see whether there was a relationship between producing the process measures and attaining the outcomes.

For example, the extent to which different treatment procedures improved patient functioning, patient health, and patient depression and anxiety can be tested in one set of analyses. Another set of analyses can examine how different treatment procedures affected patient use of drugs, criminal acts, and job productivity.

Using the Worksheets

The following worksheets are provided to help you develop a model of your program -what it does, how it does it, and the outcomes it expects to produce. Working with all interested parties and staff members is important to assure that everyone has the same concept of the program and the same perspective on proposed changes.

Worksheets.pdf (Due to the size of these files, the worksheets are available as a pdf file - 43 Kb)

Resources

To make a complete list of the crucial resources invested in treatment (worksheet A) and to assess their value (their cost) accurately, ask different interested parties what they contribute to treatment and the value of those contributed resources.

After the list of procedures is available, you can check the completeness and accuracy of the resource list by mapping resources onto procedures. Make sure that there are sufficient amounts of each type of resource listed to put each of the procedures into effect. Worksheet B can facilitate this cost-procedure mapping and the review for completeness and accuracy.

Procedures

Procedures can be classified by theoretical perspectives or by the parties responsible for delivering the procedures. The latter generally provide a more concrete list of what was done to whom, by whom, and when.

Just as many interest groups may need to be consulted to obtain a complete and accurate list of resources that make treatment possible, a variety of parties may take part in the delivery of procedures that facilitate patient recovery and other outcomes (worksheet C).

Processes

This part of the service system model often is the most challenging to construct. Direct service providers, such as counselors, aim their treatment procedures at a variety of processes internal to the patient. Because these processes are difficult to observe or detect with psychological tests and other measures, serious disagreements may result about what is being changed by treatment procedures. Providers' strong beliefs in their own favorite treatment procedures may further complicate discussion of procedures and procedure-process-outcome linkages.

A minimal check on the completeness of the process list (worksheet D) is possible. Each procedure listed earlier should be targeted at one or more processes. If a procedure exists for which no process can be named, it is likely that another process needs to be made explicit. If a process exists for which no treatment procedure is identified, the process may involve community economics or politics. If, however, a crucial process is identified for which no procedure is present in the treatment program, introducing a procedure for this process could result in superior outcomes. Worksheet E can facilitate this procedure-process checking.

Another way to check on the completeness of the process listing is to make sure that there is at least one process mapped to each outcome (worksheet F).

Outcomes

You may wish to distinguish between interim and long-term outcomes. You also may find it useful to list separate outcomes that are and are not monetary. Of the outcomes that are not monetary, you also may want to distinguish between those that can and cannot be readily monetized (worksheet G).

CPPOA Model

To help describe your program, summaries of direct cost-outcome relationships may be useful. Worksheet H can be used to make this analysis easier.

The preceding analyses need to be integrated, so that the links between resources and procedures, procedures and processes, and processes and outcomes can reveal the most cost-effective and cost-beneficial path to outcomes. The steps and diagrams in worksheet I can help you settle on several ways to improve the outcomes and/or reduce the costs of your program.

Summary

You may not be able to collect detailed information on all the psychological, social, and neurological processes that can moderate relationships between procedures and outcomes. You may barely have room and time to record basic demographic characteristics of patients, such as gender, race, and age, that may influence procedure effects. Even if you cannot measure and analyze each cost, procedure, process, and outcome variable that might be important, thinking and talking about them sometimes can set the stage for systematic improvement of program outcomes within cost constraints. These discussions also help you identify what procedures and processes make up your program. A clear understanding of what your program really is, how it works, and what changes it can prompt is essential in making decisions about program changes.