Despite the interests of consumers, families, and professional groups, the actual rates of vocational assistance and of competitive employment for people with SMD are quite meager. People with SMD have consistently had the worst employment outcomes of the various disability groups served by state and federal rehabilitation programs ( Marshak, Bostick, & Turton, 1990 ). Many studies show that less than 15% of people with SMD are employed at any one time ( Anthony & Blanch, 1987 ). Even successful programs show low levels of competitive employment for people with the most severe disorders, such as schizophrenia and bipolar disorder (e.g., Jacobs, Kardashian, Kreinbring, Ponder, & Simpson, 1984 ).
The barriers to employment for people with SMD include societal stigma; attitudes of professionals, family members, consumers, and employers; economic incentives of social insurance programs (e.g., Supplemental Security Income, Social Security Disability Insurance, Medicaid, and Medicare); lack of access to vocational services; and services that emphasize assessment and prevocational goals rather than competitive employment and follow-along supports ( Bond & McDonel, 1991 ; Rutman, 1994 ). How the Americans with Disabilities Act will affect these barriers is not yet clear.
According to a recent review, the most promising development in the vocational rehabilitation field during the past decade has been the supported employment (SE) movement ( Clark & Bond, in press ). SE emphasizes competitive jobs in integrated work settings with follow-along supports ( Federal Register, 1987 , 1992 ; Wehman & Moon, 1988 ). Of the approaches to vocational rehabilitation currently available to people with SMD, SE has the strongest empirical support ( Bond, Dietzen, McGrew, & Miller, 1995 ; Drake, Becker, Biesanz, Torrey, McHugo, & Wyzik, 1994 ; Gervey & Bedell, 1994 ; Meisel, McGowen, Patotzka, Madison, & Chandler, 1993 ). Although originally developed for people with developmental disabilities, SE has also been widely advocated for people with SMD ( Anthony & Blanch, 1987 ; Fabian & Wiedefield, 1989 ; Nichols, 1989 ; Rosenberg & Cook, 1990 ) and is increasingly used in mental health programs ( Toms-Barker, Raffe, & Leff, 1993 ).
Little consensus exists, however, on how to extend SE to people with SMD. Specific approaches to SE have not been compared in experimental studies. One issue that particularly warrants study is the organizational linkage between mental health and vocational services ( National Institute of Mental Health, 1991 ). In the traditional service system (the brokered model), clients are linked with vocational agencies, or vendors, with the Department of Vocational Rehabilitation serving as a broker. Categorical funding, organizational differences, and differences in training perpetuate a philosophical and practical separation of mental health and vocational services. This structure of parallel organizations has the potential advantages of specialization, avoiding the stigma of mental illness treatment in vocational services and developing a different culture of higher expectations ( Cook, 1992 ). In practice, collaboration between clinical and vocational programs often proves problematic because the service systems become rigid at the interface and have difficulty cooperating to individualize services for a particular client ( Harding, Strauss, Hafez, & Liberman, 1987 ; Rutman, 1994 ). Some providers have, therefore, advocated for integration of clinical and vocational services so that the burden of combining clinical and rehabilitative perspectives in a coherent way rests entirely on the providers rather than on the clients. Most prominent among the integrated clinical and vocational models is the Assertive Community Treatment (ACT) model, which has continued to evolve to emphasize competitive employment ( Russert & Frey, 1991 ). Earlier controlled studies of ACT failed to demonstrate clear advantages in terms of competitive employment ( Hoult, Reynolds, Charbonneau-Powis, Weekes, & Briggs, 1983 ; Mulder, 1982 ; Stein & Test, 1980 ), and most ACT replications have dropped or deemphasized the vocational component ( Bond, 1992 ). Two recent unpublished studies do, however, show advantages for ACT programs over the usual brokered system in terms of competitive employment ( McFarlane et al., 1993 ; Test, 1992 ).
A second issue involves the role of preemployment skills training ( Bond, 1992 ). Many experts advocate prevocational skills training before vocational placement ( Mueser & Liberman, 1988 ). Others have argued that clients need job experiences rather than skills training before vocational placement ( Bond, 1992 ). Rapid placement has some empirical support. Bond and Dincin (1986) randomly assigned 107 clients in a psychiatric rehabilitation program to accelerated or gradual placement in transitional employment. After 15 months, 20% of those in the accelerated group were in competitive jobs, compared with 7% of those in gradual placement. In a second study, Bond et al. (1995) randomly assigned 86 psychiatric clients to accelerated versus gradual entry into supported employment. After 3 years, 59% of those who received accelerated placement, compared with only 6% who received gradual placement, were competitively employed.
The purpose of the present study was to compare two models of supported
employment for people with SMD. One program (Group Skills Training; GST)
was an exemplar of the brokered service model (i.e., vocational and mental
health services in separate agencies) and emphasized preemployment skills
training. The second program (Individual Placement and Support; IPS) used
an integrated service model (vocational and mental health services combined
within the same program) and did not include preemployment skills training.
Thus, the models offered a clear contrast on these two key dimensions.
In this article, vocational and nonvocational outcomes for 18 months are
examined.
Study Hypotheses
Method
One hundred forty-three adults with SMD from the two mental health centers in which the study was conducted served as voluntary participants in this experiment. For inclusion in this study, a participant was required to have (a) a major mental illness with major role dysfunction of at least 2 years duration, (b) clinical stability (i.e., out of the hospital) for at least 1 month, (c) local residence for at least 6 months, (d) an age between 20 and 65, (e) unemployment for at least 1 month but interest in competitive employment, (f) informed consent, and (g) absence of significant memory impairment, medical illness, or substance dependence that would preclude participating in a training program.
As Table 1 shows, clients in the sample were predominantly young, Caucasian, and single. They had relatively good employment histories but were not currently working. Their primary diagnoses were heterogeneous: schizophrenia and related psychotic disorders, 46.9%; bipolar disorder and other severe mood disorders, 42.7%; and other disorders (primarily severe personality disorders), 10.5%. Nearly all were on prescribed medications. Their current levels of psychiatric symptoms and of alcohol and drug use were low, but many had histories of hospitalization, homelessness, or incarceration during the previous year.
To examine representativeness, we compared the sample with two groups. The first comparison group involved a statewide survey of 1,536 patients who were certified as having chronic mental illness and were being served in community mental health programs in New Hampshire during the time of the study. Results of this comparison, shown in Table 1 , indicate that the clients in the study were demographically similar to clients in the community throughout the state, although those in the study were slightly younger and more educated. Clients in the study were also less likely to be working and less likely to be abusing alcohol and other drugs, which was consistent with inclusion criteria for the study. They were also slightly more likely to be in independent living situations.
The second comparison was with a sample of 29 nonstudy clients selected at random in the two mental health centers in which the study took place. This comparison group showed that study clients were younger, better educated, less likely to be in school or working, more likely to have a history of competitive employment, and more likely to desire employment. On the other hand, they had lower Global Assessment Scale scores. 1Few other differences were statistically significant, even before adjustment for the Bonferroni inequality. Thus, consistent with study criteria, clients had good vocational histories and motivation to work but were otherwise similar to other clients with SMD in the same community mental health centers.
Complete 18-month vocational outcome data were obtained on 140 of the
143 clients (97.9%). One client dropped out of the study after completing
the baseline interview, a second client died of cancer 7 months after beginning
the study, and a third dropped out after 12 months in the study.
Programs
The first program was identified as the outstanding exemplar of the brokered model in New Hampshire. GST was a supported employment program for individuals with SMD that began in Massachusetts in 1985 ( Harrison & Perelson, 1989 ; Trotter, Minkoff, Harrison, & Hoops, 1988 ). The program offered individualized intake, preemployment training in a group format, individualized placement and support on the job, liaison with mental health providers, and follow-along supports. The preemployment training was designed to develop awareness and skills in the three areas of choosing, getting, and keeping a job. In addition to discussing and practicing the skills needed for these tasks, clients were encouraged to explore work-related values and to understand realistically their strengths and weaknesses as workers. Following the initial skills training, clients met with staff in a group twice each week to continue building interview skills and to discuss potential job leads and interviews. Once employed, clients continued to receive individual support services from GST staff. Empirical support for GST came from an open clinical trial in Boston in which 35% of 156 clients with SMD completed the program and obtained competitive employment ( Harrison & Perelson, 1989 ).
The IPS model was also developed to provide supported employment services for people with SMD ( Becker & Drake, 1994 ). IPS used a team approach to integrate mental health and vocational services. Employment specialists were hired by mental health centers and attached directly to clinical teams to ensure coordinated services. Rather than providing preemployment assessment and training in job-related activities, IPS employment specialists began helping clients to find jobs immediately and, after securing employment, provided training and follow-along supports as needed. The IPS employment specialists assumed that clients would learn about the job world, and about their skills and preferences, on the job rather than through preemployment training. The IPS model was derived primarily from the ACT program. In contrast to the ACT model, however, employment specialists in IPS handle all of the vocational activities, link with more than one clinical team, and concentrate on clients who express interest in competitive employment rather than on all clients.
Empirical support for IPS came from a quasi-experimental study in New
Hampshire in which one of two comparable rehabilitative day treatment centers
was closed and replaced by the IPS program ( Drake, Becker, Biesanz, et
al., 1994 ; Torrey, Becker, & Drake, 1995 ). In the program that converted,
rates of competitive employment among 71 clients with SMD increased during
the first year in the IPS program from 25% to 39%, and among 27 regular
day treatment attenders from 33% to 56%, while employment remained constant
in the comparison day treatment program. No negative outcomes were associated
with the program conversion. A replication in the site that served as a
comparison for the initial study showed similar outcomes ( Drake, Becker,
Biesanz, & Wyzik, in press ). Competitive employment increased from
13% to 23% among 112 clients with SMD, and from 9% to 40% among 35 clients
who had been regular attenders of day treatment.
Measures
All clients were assessed at baseline, 6, 12, and 18 months using a
combined interview that incorporated sections of several standardized instruments:
the Employment and Income Review ( Center for Psychiatric Rehabilitation,
1989 ), the Global Assessment Scale (GAS; Endicott, Spitzer, Fleiss, &
Cohen, 1976 ), the expanded Brief Psychiatric Rating Scale ( Lukoff, Liberman,
& Nuechterlein, 1986 ), the Rosenberg Self-Esteem Scale ( Rosenberg,
1969 ), and the Quality of Life Interview (QOLI; Lehman, 1983 ). The interview
assessed demographics, vocational history, psychiatric history, entitlements,
financial status, several domains of functioning, symptoms, self-esteem,
global functioning, vocational goals, and quality of life. Subjects also
received a diagnostic interview using the Structured Clinical Interview
for the Diagnostic and Statistical Manual of Mental Disorders (3rd
ed., revised; Spitzer, Williams, Gibbon, & First, 1989 ) during the
course of the study. The primary outcome, competitive employment, was defined
as work in the competitive job market at prevailing wages supervised by
personnel employed by the business. Employment was assessed weekly by employment
specialists in both programs and by direct interviews with clients.
Design and Procedure
The basic study design was a two-site, controlled, clinical trial with random assignment to GST or IPS. Both the GST and IPS programs were implemented in two New Hampshire cities with populations of 166,000 and 119,000 respectively. The same private, nonprofit vocational agency implemented the GST model in both cities. This agency conducted the 8-week skills training classes in a single setting and used a common staff for job development in the two cities, but separate staff members provided the direct support in finding and maintaining jobs in the two cities. The IPS model was implemented in two separate community mental health centers that served their respective cities. Each mental health center had its own staff and preexisting vocational program. A team leader within each mental health center supervised the IPS employment specialists in that center.
Implementation was monitored through observations of team meetings in both programs, through site visits from individual members of the research team and through daily logs of use of services within both programs. The research project director observed 1-hr team meetings with GST staff involved in the project weekly throughout the 3 years of the project; she also attended a 2-hr staff meeting every other week with the combined IPS staff from the two mental health centers. The project director also reviewed computerized implementation data and gave feedback to program leaders whenever evidence of model drift was detected. Other members of the research team made periodic visits to both programs to interview staff and administrators regarding implementation.
Implementation data generally supported the fidelity of both interventions. The two programs had approximately equivalent personnel-three full-time staff members-throughout most of the project but deployed the staff differently. The three IPS staff worked directly with clients in all phases of supported employment, whereas the three GST staff divided functions into job training, job development, and job support roles. Clients in the two programs received approximately the same amount of direct contact hours, (61.6 ± 37.1 for IPS versus 74.1 ± 59.2 for GST, t (141) = 1.49, ns ), but the types of service units received were different and consistent with their respective models. Implementation of GST across the two cities was similar, although the program focused attention on the smaller city during the last year of the study. Consistent with the model, the program exclusively targeted competitive jobs. Implementation of IPS differed in the two cities. Both IPS programs assisted some clients in obtaining volunteer work and sheltered jobs. In one site, these jobs were used as a means of transitioning clients to competitive work, which was consistent with the IPS model. In the second site, however, employment specialists placed more emphasis on sheltered jobs and used them for assessment and long-term placement, contrary to the IPS model. Despite feedback to supervisors from the project director, this pattern persisted throughout the study, and this site was considered to have a weaker implementation of IPS.
Informational meetings with clients, families, and mental health providers were used to recruit clients with SMD from two community mental health centers in New Hampshire. Interested clients were required to attend at least four sessions of a weekly research induction group, in which the research project director and staff from the two vocational programs described all research and program procedures, answered questions, and screened clients for inclusion criteria ( Drake, Becker, & Anthony, 1994 ). The purpose of the prerandomization group experience was to ensure that potential participants met inclusion criteria, were fully informed, were motivated to participate in the project, and were able to give informed consent.
After giving informed consent for all research procedures, clients were stratified on the extent of previous employment and randomly assigned within site to GST or IPS. All clients completed baseline assessment procedures and were linked successfully with their assigned vocational program. They were assessed at baseline with a composite interview of approximately 1 hr duration by a research interviewer who was independent of the clinical or vocational programs. Clients were reevaluated using a similar interview at 6, 12, and 18 months. They were also interviewed about each job after 1 month of employment, after every 6 months of employment, and at the termination of the job. Employment specialists and clients were interviewed separately about all jobs to assess hours worked and wages earned. Discrepancies, which usually involved jobs that clients obtained on their own, were clarified by further investigation (e.g., reports from third parties).
Interviewers were trained using survey research interview techniques from the University of Michigan ( Guenzel, Berckmans, & Cannell, 1983 ) and checked throughout the study for reliability. Both inter-rater and test-retest reliability were maintained at high levels (kappas and intraclass correlations consistently above .7, with the exception of recent drug use, which was infrequent in this sample).
Standard statistics were used to test group differences throughout the
analyses. All reported effect sizes (ES) are variants of the d statistic.
Hedges' g, which divides the mean difference by the pooled standard
deviation, was used for the difference between independent group means
( Rosenthal, 1984 ). Cohen's w, which is equal to the phi coefficient
in a 2 × 2 contingency table, was converted to d ( Rosenthal,
1984 ). Within-group effect sizes were computed as g, which divides
the difference between the posttest and the pretest by the standard deviation
of the pretest ( Becker, 1988 ).
Results
The entire sample ( N = 143) at baseline was used to assess group equivalence after random assignment to treatment conditions. The IPS group had 74 clients, and the GST group had 69 clients. Univariate tests of significance were conducted for each variable, using chi-square tests of significance for discrete data and t tests of the difference between independent group means for continuous data. All statistical tests were two-tailed. In some cases, in which there was high skewness or underused categories, the data were recoded into fewer categories.
For the purpose of assessing group equivalence, 78 variables in six domains (demographic, psychiatric, employment history, current employment status, quality of life, and self-esteem) were examined. The groups differed at p < .05 on six of these variables (see Footnote 1). Since four differences would be expected on the basis of chance, the groups were largely equivalent at the outset. The groups did not differ on any measures of demographic characteristics, psychiatric diagnosis, employment history, education and training, global assessment scale, quality of life, alcohol and drug use, living situation, psychiatric history, or vocational supports. Of 24 variables relating to current work status, three self-report items regarding reasons for current unemployment were significant at p < .05. GST clients were more likely to report that they were not currently working due to mental disability, fear of losing benefits, and being a homemaker. After Bonferroni correction within this category, only one variable (not working because of mental disability) remained significant at p = .05. One subscale of the BPRS (anergia) showed a significant difference favoring IPS clients, but this difference was not significant after Bonferroni correction. A group difference on the Rosenberg Self-Esteem Scale also favored IPS clients: 24.0 versus 21.5; t (141) = 2.67, p = .008. Finally, IPS clients were more likely to take lithium as a medication, but this difference failed to attain significance after Bonferroni correction within the category of medications.
Vocational Outcomes for All Clients
For the analysis of vocational outcomes, the three clients with incomplete follow-up data (one in IPS and two in GST) were dropped. Because there is no consensus in the field about the most important vocational outcome variable, several measures of vocational attainment were analyzed. The analysis proceeded in two steps. First, overall group differences were examined. Second, because differences in the overall employment rate might account for further differences in quality or quantity of work, work variables were examined with the analysis restricted to clients who obtained employment.
IPS clients were approximately twice as likely to obtain a competitive
job during the study: 78.1% versus 40.3%;
To examine the temporal pattern of competitive employment in the two programs, we studied monthly employment rates throughout the 18-month follow-up (see Figure 1 ). As expected, IPS subjects began to become employed in the first month of the study. Their rate of competitive employment stabilized close to 40% by the fourth month of the study and was maintained at around that level through the 18 months. On the other hand, GST clients worked very little during the first 3 months of the study while they were engaged in skills training. Their rate of competitive employment increased gradually and stabilized at around 20% in the 10th month of the study. The monthly employment rates show a significant difference in favor of IPS in 14 of the 18 months.
Figure 1
No interactions of Program × Site were found, and this study was
not designed to support separate analyses in the two sites, because the
samples were small in each setting ( n = 91 in the larger city and
n
= 49 in the smaller city) and the sites were not independent (GST staff
overlapped in the two sites). The process analysis indicated, however,
that IPS was poorly implemented in the smaller city and that, conversely,
GST staff concentrated their efforts in the smaller city. These observations
led to exploratory analyses by site. The overall rates of competitive employment
strongly favored IPS over GST in the larger city
To examine the influence of other variables on the overall rate of employment, we conducted logistic regression analyses that included previous work, treatment group, site, gender, age, and psychiatric diagnosis as predictors. As shown in Table 2 , only previous work and group were significant predictors of competitive employment in the main-effects model; no interaction term was statistically significant.
Vocational Outcomes for Workers Only
Among clients who obtained jobs ( n = 57 for IPS, and n =
27 for GST), few treatment group differences emerged. IPS and GST workers
were similar in terms of proportion working at least 20 hr per week, weeks
in the longest job, and work status in the 18th month. However, IPS workers
averaged more total hours (777.4 ± 882.0 vs. 509.0 ± 495.7)
and earned more total wages ($4,346.71 ± $5,824.20 vs. $2,674.58
± $2,877.07) than GST clients, and these differences were statistically
significant when previous work was used as a covariate, F s(1, 81)
= 5.35 and 5.57, respectively, p s < .05; ES = .34 and .33, respectively.
Previous work did not interact with treatment group, and there were no
significant main effects or interactions for age, gender, diagnosis, or
site. Among those who were working at the 18-month interview, there were
also no group differences in satisfaction with job, F (1, 29) =
1.40, ns .
Nonvocational Outcomes for All Clients
Changes over time in global functioning, quality of life, self-esteem, and psychiatric symptoms were also examined. These analyses were considered exploratory, leading to downward adjustment of the per-comparison alpha level to account for multiple tests. The multivariate approach to repeated measures analysis of variance was used to examine the main effect for group (IPS vs. GST), the main effect for time (baseline and 6, 12, and 18 months), and the Group × Time interaction. Effect sizes for the main effect for time are reported as the standardized mean difference between the endpoint (18 months) and the pretreatment baseline point.
One cluster of variables was related closely to the employment outcomes
in this study: global functioning (GAS; anchored scale rated by interviewer;
range = 1-100), satisfaction with finances (QOLI; average of five 7-point
Likert-type items), and satisfaction with vocational services (QOLI; one
7-point Likert-type item). Each revealed a significant main effect for
time. The full sample (both groups combined) showed significant improvements
in global functioning
A second cluster of variables that were not directly related to the
employment outcomes in this study included measures of overall symptom
severity (BPRS total score; sum of 24 seven-point Likert-type symptom items;
range, 24-168), overall life satisfaction (QOLI; average of two 7-point
Likert-type items), satisfaction with both housing and town (QOLI; average
of five 7-point Likert-type items), satisfaction with mental health care
(QOLI; one 7-point Likert-type item), and self-esteem (Rosenberg 10-item
scale; range, 10-50). The results showed significant main effects for time
on overall symptom severity and self-esteem. For symptom severity, the
full sample showed a slight increase in symptoms (still in the nonclinical
range) over the course of the study
Discussion
The lack of interactions between program type and individual characteristics indicates that IPS was the preferred program for all of the groups that were considered: clients with good vocational histories versus those with poor histories, men versus women, older versus younger clients, and those with schizophrenia versus those with affective disorders. Employment history also predicted attaining competitive jobs but did not interact with the influence of vocational program assignment.
One study hypothesis was that skills training might lead to greater quality of employment: more satisfying, longer lasting, or higher paying jobs. Once clients became employed, however, job satisfaction and job tenure were quite similar in the two programs, whereas total earnings, which were strongly correlated with total hours of employment, actually favored IPS. In other words, the types of jobs that clients obtained were similar between the two programs. Consistent with studies of rapid placement ( Bond & Dincin, 1986 ; Bond, Dietzen, McGrew, & Miller, 1995 ), there was no evidence that preemployment skills training yielded any advantages in terms of quantity or quality of employment.
Because the brokered model in this study (GST) differed from the integrated model (IPS) on two dimensions-offering vocational services in a program that was separate from the mental health program and offering preemployment skills training-it is difficult to say whether one of these dimensions alone would have made a difference. The process analysis clearly indicated that interagency difficulties were common in the GST model. Clients often had difficulty transitioning between one agency and another, and miscommunications were normative. Explaining how skills training could have long-term adverse consequences rather than merely result in a delay in finding employment is difficult on the surface, but the explanation may be related to limited resources. In other words, devoting time and energy to skills training means not devoting resources to helping clients find and keep jobs.
Quality of implementation of the IPS model appeared to be related to differential effectiveness, and the outcome difference in the larger city, which had a better implementation of IPS, appeared to be carrying the overall program effect. However, these site-specific findings are difficult to interpret because of the small sample sizes, the nonindependence of sites, and the observations that many factors related to clients and work environments, in addition to implementation, differed between the two cities.
The vocational results reported here could have been due to other factors. One possibility that cannot be ruled out involves the confound between program and site. The IPS program was delivered only in mental health centers and the GST program only in the rehabilitation agency. Unmeasured influences, such as staff conflict, attitudes, skills, or demoralization related to these sites, could have influenced the results.
Another possibility is an overall poor implementation of GST. Process data showed, however, that the GST program was a faithful implementation of the intended model and that the GST staff were well educated, well trained, and well supervised. Moreover, the vocational outcomes in a previous implementation of the GST model in Boston found a 35% rate of competitive employment ( Harrison & Perelson, 1989 ), as compared with 39% in this study. Nevertheless, other external factors, such as local economic conditions, the nature of the New Hampshire mental health system, or the restrictions imposed on the program by participation in a research study, may have limited the effectiveness of GST. On the other side, independent studies of IPS and of other integrated models support the results of the present study ( Drake, Becker, Biesanz, Torrey, McHugo, & Wyzik, 1994 ; Drake, Becker, Biesanz, & Wyzik, 1994 ; McFarlane et al., 1993 ; Test, 1992 ). Preliminary findings from a study of IPS versus the brokered model that is ongoing in Washington, DC, with an urban, formerly homeless sample are also consistent with the results reported here.
Despite assumptions in the literature, there was no evidence in this study that program differences produced outcome differences in domains of functioning other than employment. Clients in both programs improved over time in areas that were proximally related to employment, such as global functioning and satisfaction with finances, but showed little change in other areas. The mild increase in symptom levels is difficult to interpret but probably means little because symptoms remained in the subclinical range. It seems likely that nonvocational domains of outcome are only weakly related to vocational function and that program effects are specific to the content of the program. Another possibility is that clients need more time in jobs before vocational gains generalize to other domains of functioning.
