Table of Contents | Previous | Next |
Executive Summary
This report presents interim results for the Minnesota site in the national Employment Retention and Advancement (ERA) project. Conceived and funded by the Administration for Children and Families (ACF) in the U.S. Department of Health and Human Services (HHS), the ERA project is testing 15 innovative programs across the country that aim to promote steady work and career advancement for current and former welfare recipients and other low-wage workers. MDRC, a nonprofit, nonpartisan research organization, is conducting the ERA project under contract to ACF, and is producing a similar interim report for each site in the project.1
The Minnesota ERA evaluation is a test of a program in Hennepin County (which includes the city of Minneapolis) that targeted a segment of the Temporary Assistance for Needy Families (TANF) caseload that had received cash assistance for a relatively long period. The program provided a range of services designed to address these individuals’ barriers to employment and to help them find and keep jobs. Funded by the Minnesota Department of Human Services (DHS), the program operated from January 2002 to June 2004.
Origin and Goals of the Minnesota ERA Program
The Minnesota ERA program builds on the state’s TANF program, known as the Minnesota Family Investment Program (MFIP).2 Like many TANF programs, MFIP requires recipients to work or participate in employment-related services or face financial penalties (sanctions), includes an earned income disregard that allows recipients to work and still remain eligible for some benefits, and places a lifetime limit of 60 months on cash benefits. MFIP in Hennepin County is referred to as the Tier 1 program.
While many individuals have moved off welfare and into work since the MFIP program was implemented, program administrators have found that a significant fraction of recipients have remained on the rolls for a long time without working. To address the needs of this group, who appeared most likely to reach their benefit-receipt time limit, the state distributed grants to several counties to design special services. The program implemented in January 2002 in Hennepin County was referred to as the Tier 2 program.
In Minnesota, the ERA evaluation is a test of the Tier 2 program in Hennepin County; it examines how employment and other outcomes for individuals involved in the Tier 2 program compare with those for individuals who remained in the Tier 1 program. The Tier 2 program used Tier 1 as its foundation, but built on it in three key ways. Tier 2 provided: (1) smaller caseloads for program staff to facilitate more attention to the unique circumstances of individuals and their families; (2) a more in-depth assessment of new cases, covering a wide range of issues facing clients and their families; and (3) a greater emphasis on assigning and referring individuals to a broader range of services, including supported employment positions and more specialized services, such as those available from programs that help people with mental health or substance-related problems. In short, the goals of the Tier 2 program were to better assess the employment barriers faced by a portion of the TANF population in Hennepin County who had been on welfare for a long time and had worked very little, and to address those barriers through referrals to appropriate services and close monitoring and follow-up.
The ERA Evaluation
As in the other ERA sites, MDRC is using a random assignment research design to assess the effectiveness of the Tier 2 program. Between January 2002 and April 2003, the Hennepin County Office of Training and Employment Assistance identified MFIP recipients who met the Tier 2 program eligibility criteria: They had been assigned to MFIP employment services for 12 months or longer, were currently unemployed and had not worked in the preceding three months, were not currently participating in an approved education or training program, and were not currently being sanctioned. (Individuals who were “exempt” from participating in MFIP because, for example, they had young children or were seriously ill or incapacitated were not included in the study.) Half of these recipients were assigned at random to the Tier 2 program (referred to as the Tier 2 group), and half were assigned to remain in the Tier 1 program (referred to as the Tier 1 group).3
MDRC is tracking both groups using data provided by the State of Minnesota that show each individual’s monthly welfare and food stamp benefits as well as quarterly earnings in jobs covered by the Minnesota unemployment insurance (UI) program. One and a half years of follow-up data are available for each person in the report’s analysis. In addition, a survey was administered to a subset of Tier 2 and Tier 1 group members about a year after they entered the study.
Because individuals were assigned to the Tier 2 and Tier 1 groups through a random process, the two groups were comparable at the start. Thus any differences in outcomes that emerge between the two groups during the follow-up period can be validly tested for the likelihood that they arose because of the program and not by chance variation. A total of 1,692 single parents are included in this report’s analysis, using administrative records. Analyses using the survey include 503 single parents, representing 76 percent of those who were selected to be interviewed. Additional analyses suggest that the survey sample is representative of the fuller evaluation sample.
The Tier 2 Target Population
The ERA study participants in Minnesota had a high prevalence of barriers to employment. In addition to long stays on welfare and low education levels, many reported health-related problems and responsibilities caring for ill or disabled children. Nearly one-third met diagnostic criteria for past-year major depression, for example. In addition, many individuals had multiple barriers to employment.
Key Findings on Program Implementation
- The Tier 2 program was generally well implemented. The low caseloads for program staff allowed them to conduct in-depth assessments and interact more with clients.
Because Tier 2 evolved from the basic structure of Tier 1, the two programs shared some important characteristics: a strong focus on obtaining employment, four weeks of job search followed by community service for those who did not find work, support services including child care and transportation, and some assistance with retention and advancement for working participants. The fact that the Tier 1 group had access to this range of services and supports may have set a high standard for Tier 2 to surpass and demonstrate impacts.
Nonetheless, the Tier 2 program did differ from Tier 1 in several ways. Case managers worked with small caseloads — 25 to 30 clients per Tier 2 worker versus 75 to 100 in Tier 1 — which allowed them to pay greater attention to the unique circumstances of recipients and their families. For example, the relatively small caseloads allowed Tier 2 case managers to conduct full-family assessments, which were far more in-depth than the basic screenings used in the Tier 1 program. In addition, individuals in Tier 2 met with their case managers more often than individuals in Tier 1, although both groups reported high levels of contact with their case managers.
- Despite greater attention to assessment, the Tier 2 program did not increase the use of services that help people with barriers to employment, such as problems with mental health, substance abuse, or domestic violence.
Because barriers to employment often go undetected, assessment was a key component of the Tier 2 program. Although the full-family assessments were both comprehensive and well implemented — and almost certainly led to increases in the identification of problems — survey data suggest that Tier 2 clients and their family members were no more likely than Tier 1 clients and their families to receive services to address critical barriers to employment. This was due in part to the fact that, while the case managers could encourage participants to receive recommended assistance or care, they could not require people to do so. Moreover, it was influenced by the loss — in July 2002 — of a waiver of the federal welfare reform law that had allowed Minnesota to count social service activities as meeting federal work participation requirements. Nonetheless, over 15 percent of the individuals in both Tier 1 and Tier 2 received mental health services, and at least 6 percent received services for substance-related issues and domestic violence.
| Outcome | Tier 2 Program | Tier 1 Program | Difference (Impact) | P-Value | |
|---|---|---|---|---|---|
| Quarters 2-7 | Ever employed (%) | 68.0 | 64.7 | 3.3 | 0.113 |
| Earnings ($) | 6,476 | 6,529 | -54 | 0.902 | |
| Ever received TANF (%) | 93.4 | 93.1 | 0.3 | 0.793 | |
| Amount of TANF received ($) | 5,162 | 4,991 | 171 | 0.222 | |
| Ever received food stamps (%) | 97.5 | 96.8 | 0.7 | 0.383 | |
| Amount of food stamps received ($) | 4,536 | 4,465 | 71 | 0.429 | |
| Quarter 7 | Ever employed (%) | 45.2 | 43.4 | 1.8 | 0.443 |
| Earnings ($) | 1,271 | 1,358 | -87 | 0.377 | |
| Ever received TANF (%) | 58.6 | 54.2 | 4.3 * | 0.067 | |
| Amount of TANF received ($) | 613 | 577 | 36 | 0.245 | |
| Ever received food stamps (%) | 80.3 | 74.7 | 5.6 *** | 0.005 | |
| Amount of food stamps received ($) | 694 | 667 | 27 | 0.215 | |
| Sample size (total = 1,692) | 845 | 847 | |||
|
SOURCES: MDRC calculations from UI, TANF, and food stamp administrative records from the State of Minnesota. NOTES: This table includes only employment and earnings in jobs covered by the Minnesota unemployment insurance (UI) program. It does not include employment outside Minnesota or in jobs not covered by UI (for example, "off-the-books" jobs, some agricultural jobs, and federal government jobs).
Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. A two-tailed t-test was applied to differences between outcomes for the program and control groups. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The p-value indicates the likelihood that the difference between the program and control group arose by chance.
Dollar averages include zero values for sample members who were not employed or were not receiving TANF or food stamps. |
- The Tier 2 program modestly increased job search activities and participation in supported employment, but participation levels in other types of activities (such as education or training) were similar under Tier 2 and Tier 1.
Tier 2 did not increase participation in education or training activities, but it did lead to small increases in job search activities. (Tier 2 sample members participated in job search activities an average of four weeks longer than Tier 1 sample members.) Tier 2 also led to a small increase (10 percent versus 4 percent) in supported employment, where participants worked for a wage in jobs supervised by program staff. Moreover, they worked in these kinds of positions for longer periods of time. However, it must be kept in mind that the overall rates of participation in activities were quite high for both Tier 1 and Tier 2 clients. For example, over 80 percent of both groups participated in job search.
Finally, it is important to note that, aside from the more in-depth assessment, the two groups had access to essentially the same range of services and supports, and this may have set a high standard for Tier 2 to surpass and demonstrate impacts.
Key Findings on Program Impacts
- The Tier 2 program, compared with Tier 1, had little effect on employment or earnings over the one and a half years of follow-up. An early increase in employment did not persist.
Table ES.1 summarizes the Tier 2 program’s effects on employment in UI-covered jobs and public assistance during the first year and a half of the study period. Differences in outcomes between the two groups that are marked with asterisks are statistically significant, meaning that it is very likely that the program led to these differences.
As shown in the top panel of the table, both the Tier 1 and Tier 2 groups had similar rates of employment over the follow-up period (68 percent versus 65 percent). The Tier 2 program did lead to a modest increase in employment in the quarter just after program entry (not shown in the table), but this difference did not persist beyond Quarter 2. The impacts appear to have faded because of job loss among those who went to work. The bottom panel of the table shows that effects on employment or earnings also did not materialize in the longer run, that is, by Quarter 7. These data also highlight the problems with employment stability faced by this population: although the majority of both groups worked at some point during the period, only about 45 percent were working at the end of the period.
- The Tier 2 program, compared with Tier 1, led to a modest increase in employment for those with recent work experience, although the effects faded after Year 1. A large fraction of the employment effect appears to be due to the increased placement of the Tier 2 group in supported employment positions.
Although the program was targeted to recipients who had not worked recently, employment data are, in fact, often reported with a lag to the TANF program. As a result, about a third of the sample had work experience in the quarter prior to study entry. For this group, the Tier 2 program led to notable increases in employment during the early quarters, effects that did not persist beyond Year 1. Another effect of the program for this group was a 12 percentage point increase in the rate of participation in supported employment. Additional analyses suggest that much of the early employment increases were associated with supported work.
In general, the effects of the Tier 2 program did not vary across subgroups defined by the presence of barriers to employment (such as depression, poor health, poor child health, low education level, and learning disabilities).
- The Tier 2 program, compared with Tier 1, had no effect on public assistance receipt over the follow-up period as a whole. Many families left welfare on their own over time.
Table ES.1 presents the effects of the Tier 2 program on TANF and food stamp receipt. Data for the Tier 1 group show that many of the Tier 1 recipients left welfare over time — by Quarter 7, only 54 percent were receiving benefits.
For the entire year and a half of follow-up, the Tier 1 and Tier 2 groups had similar rates of TANF and food stamp receipt. Receipt gradually fell over the follow-up period at roughly the same rate for both groups, with the exception of the last one or two quarters. In the last quarter of follow-up, rates of receipt were somewhat higher for the Tier 2 group, by 4 to 5 percentage points. Longer-term follow-up will show whether this late effect on public assistance receipt persists.
Policy Implications
Many states share an interest in finding strategies to more effectively assist individuals who receive cash assistance benefits for long periods. Three other sites — in two cities — in the national ERA project are similarly focused on populations that appear to be hard to employ (New York City and Portland, Oregon).4,5 Moreover, more results will eventually be available from the HHS-funded Enhanced Services for the Hard-to-Employ Demonstration and Evaluation Project, which is testing the effectiveness of various programmatic interventions for the hard-to-employ in four sites. Although the results presented in this report are important, and add to the knowledge base of what works — and doesn’t work — for hard-to-employ groups, they are not the final word on the Minnesota program. MDRC will ultimately track employment outcomes for three years. In the future, more definitive findings will be possible. However, some preliminary conclusions can be drawn from the results in this report.
-
Based on the Minnesota results and those of other studies, lower caseloads alone may not be sufficient for programs to succeed. For example, a special study in Riverside, California, as part of the GAIN evaluation, similarly found that significantly lower caseloads for program staff did not improve employment outcomes for welfare recipients.6 These results may indicate that lower caseloads, and the greater level of interaction they allow, may be effective only when clients also participate in a fuller range of services that address their needs.
- Similarly, the Tier 2 program devoted considerable resources to implementing in-depth, full-family assessments. This aspect of the program was of special interest, given the site’s desire to reach a hard-to-employ population. However, this study found that the full-family assessments — even though they were well implemented — did not increase clients’ use of services to address critical barriers to employment. This finding may indicate that future programs should consider ways to strengthen the linkages between assessments, referrals, and service engagement. It may also suggest that, in some instances, assessment processes can deter or prolong entry into key program services. Useful strategies for engaging and assessing clients have emerged from previous research on welfare-to-work programs.7
It is important to point out that the issue that prompted Minnesota to implement the Tier 2 program remains salient. The group of MFIP recipients in this study includes people who worked during the follow-up year and people who did not; earnings for those who did work, on average, remained very low. The relatively poor outcome levels for both the Tier 2 and Tier 1 groups highlight the importance of discovering effective supports and services for individuals with significant employment barriers.
| Table of Contents | Previous | Next |