Madam Chairman, members of the Subcommittee, I am pleased to be here today to discuss
the critical role of research and evaluation in the implementation of welfare reform. This
Subcommittee played a central role in ensuring that these activities would be an integral
part of the Temporary Assistance to Needy Families (TANF) program.
A central focus of the Department of Health and Human Services' (HHS') welfare reform
research and evaluation effort is to develop reliable, credible information about how
different strategies are working in order to inform federal and state policy makers and
the public about how welfare reform is progressing and how families are faring. Today I
will describe key elements of the Department’s welfare reform research agenda, what
we have learned so far and what we will learn in the future. Our efforts have two major
parts: working in partnership with states to develop in-depth information; and using
national data to understand the overall progress of welfare reform. (There are other
important areas, however, for brevity I'll focus on these two parts today.)
State Evaluation Activities
The broad flexibility provided by TANF creates the opportunity and the challenge to
evaluate a variety of different approaches to welfare reform to determine which strategies
are most effective in moving families to work and self-sufficiency. A state-federal
partnership is critical to any successful strategy to evaluate welfare reform for several
reasons.
- Studies of processes, programs and populations that are carried out at a local level can
have a depth that is not possible at the national level, especially given the great
flexibility in program design that TANF permits.
- Studies that employ random assignment methods, which are the most rigorous methods for
measuring the magnitude of the effects of alternative policies and program designs, must
be implemented at the state or local level.
- States and localities have access to a rich set of administrative data, including linked
administrative data sets, that can be used for tracking recipients and former recipients.
- Involving states as partners assures that the information produced will be relevant to
state policy makers and thereby greatly increases the likelihood that states will use the
information as they re-design their TANF programs.
We are grateful that Congress has appropriated funds for the Administration for
Children and Families (ACF) and the Office of the Assistant Secretary for Planning and
Evaluation (ASPE) to allow us to fund over three-quarters of the states for activities
related to the assessment of welfare reform. Some of the primary areas for these grants
include:
- Evaluations of Specific Employment Programs and Welfare Reform Policies Using Randomized
Field Experiments
--As authorized under TANF, we provided funds to nine states to
continue evaluations that began under waivers and use random assignment to address the
effects of alternative welfare reform programs and policies. In addition, we continued
previous multi-site experiments of welfare-to-work strategies and post-employment
services. Recently, we also funded two demonstrations to rigorously evaluate: (1) an
innovative strategy to identify and provide treatment linked to employment and training
for welfare recipients who are substance abusers, and (2) an innovative approach that
combines a strong commitment to work (requiring at least 20 hours of work a week) with a
strong commitment to post-secondary education.
- Welfare Leavers
--We are providing grants to 11 states and three counties (or
consortia of counties) to collect and analyze, using a combination of administrative and
survey data, information on families that have left welfare. In the next several months we
intend to fund additional studies with an emphasis on studies of families that have been
formally or informally diverted from coming on the rolls.
- Implementation
--We are funding numerous studies of the implementation of particular
policies and programs in order to identify problems and solutions. For example, we're
funding a study of local devolution across the counties of Maryland, a study of
implementation of state TANF in rural and tribal areas in Montana, and a study of
relocation of families from economically depressed areas in South Carolina.
Two weeks ago the strength of the state-federal partnership was demonstrated in ACF's
Second Annual Welfare Reform Evaluation Conference. The two and one-half day meeting was
attended by about 300 people from state and federal agencies, universities and policy
research and evaluation organizations. The attendees included research and program staff
from 49 states plus the District of Columbia who participated in a lively meeting both
presenting and hearing the latest findings from evaluations. The gathering also provided
the venue for two one-day meetings: an ASPE/ACF-sponsored meeting for states conducting
studies of families leaving assistance and a meeting of researchers and states working on
a book to improve the design of implementation studies.
Analysis of National Data
While state and local studies provide us with depth, only national studies can provide
nationally representative information about how families are faring overall. Another
important source of national data is, of course, the national TANF data reported to HHS by
the states. These data provide information on the characteristics and conditions of
families that continue to receive TANF assistance. There are also a number of surveys that
have long been central to the study of welfare issues. All capture information on income
from various sources including earnings and transfer payments, and several contain
detailed information on employment, childbearing, family structure, and child well-being.
While the strength of these surveys is their ability to capture very detailed and rich
information, they also generally suffer from some degree of underreporting, and there is
some evidence that underreporting of Aid to Families with Dependent Children (AFDC)/TANF
benefits is increasing over time. Specific examples of the most important national surveys
for assessing welfare reform follow.
- The Census Bureau's March Income Supplement to the Current Population Survey (CPS)
provides an annual snapshot of the economic and employment condition of families.
- The Census Bureau also conducts the Survey of Income and Program Participation (SIPP)
which collects in-depth information three times a year on families' income and program
receipt over four-year periods. This Committee was instrumental in providing an additional
$70 million to the Census Bureau to extend two panels of the SIPP and create the Survey of
Program Dynamics (SPD) which will track some families up to ten years.
- The Panel Study on Income Dynamics (PSID) has tracked a representative sample of
families for nearly 30 years, collecting in depth information on income, family formation,
and program participation.
- The National Longitudinal Survey of Youth (NLSY) has tracked a representative sample of
youth and young adults for nearly 20 years collecting detailed information on how these
individuals have moved into adulthood.
No one national survey will provide us with all the information needed. It is important
to link information across surveys, and also to take advantage of administrative data. For
example, we are funding a project to link SPD data to earnings records kept by the Social
Security Administration, both to get a better understanding of biases in that SPD sample
that may result for attrition, and to get a longer earnings history to complement the
survey data.
Except for the CPS and one quarter of TANF administrative data, post-TANF data from
these national sources are not yet available, but ultimately they will provide a critical
complement to data derived from state and local sources. An important part of what I'll
discuss today is based on some early analysis, by staff in the Administration, of CPS
data.
What We've Learned So Far
The employment of welfare recipients and former recipients has increased significantly.
A key measure of the success of welfare reform is its effect on employment. Analysis of
all available sources of information shows that the employment rate of current and former
TANF recipients has increased significantly. Each March the CPS, which is used to
calculate unemployment rates, collects information about households' income and program
participation in the previous calendar year in addition to employment and earnings data
reflecting individuals' March employment status. As a result we know whether adults who
received AFDC or TANF in the preceding calendar year were employed the following March.
Between 1992 and 1996, the employment rate increased from 20 percent (its approximate
level for the previous four years) to 27 percent. However, in the last two years it jumped
even more dramatically to 34 percent in 1998. Thus, whereas in 1992 one in five previous
year recipients was working the following spring, in 1998, the figure was one in three.
Large employment gains are also evident from rigorous waiver evaluations that measure
the effects of reform policies by comparing randomly assigned individuals who were subject
to either welfare reform or standard AFDC rules. Unlike the CPS analysis, which does not
separate out the effects of state welfare reform policies from those of the economy, other
policies which promote employment such as the enhancement of the Earned Income Tax Credit
(EITC) or the expansion of child care subsidies, the strength of experimental studies is
that they do precisely that. Several studies examined policies which are typical of state
TANF programs in that they increase participation in mandatory work activities and/or
increase the amount of assistance a family can receive when they go to work. The
persistent employment effects of these programs are in the five to 13 percentage point
range. I would also note that these are probably quite conservative estimates in that the
treatment groups are compared to control groups which received a substantial level of
mandatory employment services and also were not isolated from the atmosphere of welfare
reform, even though they did not directly experience welfare reform policies.
Preliminary findings from four of the ASPE-funded studies of
families leaving welfare indicate that between one-half and
three-fifths of former TANF recipients found work in jobs which were covered by their
state's Unemployment Insurance program.
Employment rates were even higher – 75 to 82 percent – when measured as the
percentage of those who were ever employed within the first 12 months. These
employment rates are consistent with findings in many other leavers’ studies,
although methodological differences cause rates to be slightly higher in some studies
(e.g., rates are sometimes higher in studies using survey data, or limiting study
population to leavers who do not return to welfare). While these
employment rates are not radically different from the patterns of AFDC leavers in earlier
studies, they indicate a dramatically large increase in the absolute number of
families leaving welfare with earnings, given the significant caseload decline in the past
few years.
Finally, there has also been a significant increase in employment of current welfare
recipients. Between 1992 and 1997, the percentage of adults on welfare in direct work
activities (including employment, work experience and community service) has tripled. All
states met the overall work participation rate requirement for 1997. And, the percent of
TANF adults who were employed rose from 14 percent in the first three quarters of 1997 to
18 percent in the last quarter of 1997. Thus, each of these sources of information
consistently points to higher levels of employment, among current and former welfare
recipients.
Earnings have also increased, but not uniformly.
A second important measure of success in welfare reform is whether welfare recipients
and former recipients are earning more. Although welfare reform is having a positive
effect on the earnings of some categories of recipients, the story here is somewhat more
complicated than the employment story. For example, an examination of welfare reform
waiver demonstrations suggests that those programs which were strongly oriented toward
increasing employment activities and mandatory participation (as measured by an increase
in participation and sanction rates) achieved annual earnings gains in the range of
$600-$700 for a least one primary target group of applicants or recipients. One employment
and training program in Portland which combined a strong employment focus, an emphasis on
moving recipients into higher paying jobs with benefits, and the provision of necessary
child care produced even larger effects with averaging earnings gains of over $900 per
year. Of state welfare reform approaches which relied primarily on incentives without work
requirements, earnings gains were not observed.
Along with the employment gains described above, the CPS data suggests average earnings
for all female-headed families with children have increased substantially between 1993 and
1997 from $14,668 to $17,646 (both in 1997 dollars). However, the early CPS suggest
preliminarily that the gains are not evenly distributed over the period with roughly
three-quarters of the gain occurring between 1993 and 1995, and only one-quarter between
1995 and 1997. In addition, while employment gains for the bottom fifth of female-headed
families with children were stronger from 1995 to 1997, the average earnings of this group
increased from 1993 to 1995 but did not increase from 1995 to 1997. Better understanding
of these trends will require both longer term follow up and analysis of other national
data sets as they become available.
Finally, TANF administrative data just for welfare recipients who remain on the rolls
indicate that average monthly earnings for those who are working increased substantially
from $506 in the first three quarters of FY 1997 to $592 in the last quarter.
Income has increased for some families, but there is also some preliminary evidence
that some families are experiencing losses.
Income is another central measure of how families are faring under welfare reform. Here
our results are even more preliminary than for employment and earnings, although we will
have much better data over time. Much of our current information relies on administrative
records which typically examine family income defined as the total of TANF, Food Stamps
and earnings. However, these analyses do not take into account other sources of income,
such as the EITC, child support and Supplemental Security Income (SSI); the income of
other household members; in-kind supports such as child care or Medicaid; nor, on the
other side of the ledger, the expenses that families incur when they are working. A few
current data sources such as the CPS and some early studies of families leaving welfare
are based on household surveys, and many more of our studies ultimately will have this
information.
Data from the four waiver evaluations in which the reform program succeeded in
increasing mandatory work activities, employment and earnings suggest mixed effects on
family income, depending on the generosity of benefit levels and earnings disregards. In
the two states with both generous benefits and earnings disregards, there were increases
in average annual income of $762 for applicants in Iowa, and $1,065 for long term
recipients in Minnesota. In Florida, a program that accomplished comparable earnings
gains, but had low benefits and generous earnings disregards raised family income by $289,
whereas a fourth program in Indiana that accomplished comparable earnings gains but had
low benefits and retained the standard AFDC earnings disregards had no effect on income.
Examination of the Florida findings also suggests that these effects are not uniform
across recipients and that higher-skilled recipients may gain income, whereas the income
of lower-skilled recipients may decline. In Florida, recipients who had both a high school
degree and recent work experience averaged $752 higher average annual income for the three
years following entry into a welfare reform program, while those with neither experienced
losses of about $485. This gain/decline pattern is consistent with patterns in some
earlier leaver studies. For example, a study in Iowa of families that lost their entire
benefit because they failed to establish a self-sufficiency plan showed that about 40
percent increased their income, about 50 percent suffered a decrease, and about 10 percent
had unchanged income.
For the period 1993 to 1997, CPS data indicate that the average annual income of all
female-headed families with children increased, as did employment and earnings as
described above. This measure of income includes both earnings and a broad range of
transfer programs. Again, the income increases were unevenly distributed over the period,
with larger gains in the 1993 - 1995 period, and across the income distribution. The
bottom quintile did not fare as well as the top four fifths, especially in the 1995-1997
period, suggesting preliminarily that we need to be alert to monitoring more disadvantaged
families.
What We Will Learn
As described above all the findings I've noted are preliminary. In almost all the
studies I've described above, additional administrative data are being collected and
surveys are being fielded or will be in the next several years. In the national surveys,
data for 1997 and 1998 have been collected and are being processed. Thus, time and
resources are necessary to produce more definitive results. I'll describe some of the more
important information that we'll be obtaining.
Longer Term Follow-Up
Our findings are currently based on follow-up periods ranging from 6 months to 3 years.
To understand the effects and outcomes of welfare reform will require longer term
follow-up on the order of four to six years. Particularly important is that in most
states, very few individuals have reached time limits, and in many larger states, none
have.
More Comprehensive Information
As indicated above most of our current measures of employment, earnings and income are
from administrative records. Over time we will obtain much more comprehensive information
through surveys, both in conjunction with waiver demonstrations and through studies of
families that have left TANF. Most importantly, we will be able to learn a great deal more
about the income families have to support themselves and in particular, what happens to
families who leave welfare and do not have earnings.
More Outcomes, Especially Related to Child Well-Being
A critical measure of the success of welfare reform is how it affects children. In 1996
ACF provided grants to 12 states to work with a team of researchers (funded by ASPE) to
develop measures of child well-being to examine how different welfare reform programs and
policies are affecting children. The partnership proved to be very successful, and
subsequently ACF has augmented the funding of five state welfare reform evaluations
(Connecticut, Florida, Indiana, Iowa, and Minnesota) to rigorously evaluate the effects of
welfare reform on family processes and child well-being. In addition, ASPE has provided
funds to 13 states to work with the Chapin Hall Center for Children at the University of
Chicago and other experts to develop indicators of child well-being. Some of the 13 states
are exploring using similar measures to those used in the five states as well as other
administrative and survey data. The importance of measuring child well-being is vividly
illustrated by the presentation you are hearing today on the New Hope demonstration. A
central finding of the interim effects of this program designed to support working
families was that it increased significantly boys' school performance along with
increasing their participation in extended day child care and other structured activities,
while having other positive family effects.
More Rigorous Information about What Works and What Doesn't
Because of caseload reduction, states now have significant financial resources that are
not required for immediate cash assistance, and thus are available for investments in
those families still on the caseload, including those with the greatest problems, and to
enable families who leave the rolls to keep their jobs and move up instead of returning to
welfare. As a result, many states are increasingly focused on strategies to increase job
retention and advancement for recipients and former recipients. ACF is working with
thirteen states to develop, pilot and ultimately rigorously evaluate the effects of
alternative strategies. Through this activity and others, we will be learning about the
role of supports for working families such as child care, child support and other services
in sustaining and advancing in employment. These kinds of evaluations are critical to
using the flexibility provided by TANF to maintain a learning environment in which federal
and state tax dollars are used to make investments that really work.
More Information about Sub-Populations, especially the Harder-to-Employ
As more in-depth information is developed it will be possible to understand better how
particular sub-populations are faring. We currently have projects underway to examine
issues related to: non-custodial parents, rural populations, families with disabled
members, Native Americans, victims of domestic violence, child-only cases and families
with mental health or substance abuse problems. In addition, we will learn more about the
impact of strategies to serve the hardest-to-employ through our Department's recently
begun evaluation of the Department of Labor's Welfare-to-Work grants program. The
evaluation, which is at an early stage, will include an examination of the impact of
Welfare-to-Work on participants' employment and wellbeing. We will share these results
with Congress as the project progresses.
More Nationally Representative Data
Over time as more post-TANF data become available from the national surveys, especially
longitudinal data, it will be possible to merge findings from in-depth studies in states
and localities with nationally representative data and use the strengths of each to
develop a comprehensive picture of how the nation's families are faring under welfare
reform.
Conclusion
The preliminary results I've described above illustrate the promise of how investments
in careful research and evaluation can produce information that can inform policy makers
at all levels. Perseverance in these investments can play a critical role in supporting
strategies that can realize the goals of welfare reform.
I will be happy to answer your questions.
