Assets for Independence Act Evaluation:
Phase I Implementation Final Report
October 5, 2001

• Non-experimental impact study: This research includes a multi-wave longitudinal survey of a randomly selected national sample of 600 AFIA program participants to assess the effects of program participation on low-income savings, asset accumulation, and other aspects of family well-being. The participant outcomes will be measured versus a comparison group of AFIA-eligible nonparticipants, using data from the Survey of Income and Program Participation (SIPP) conducted by the U.S. Census Bureau. During Phase I, clearance from the U.S. Office of Management and Budget (OMB) was obtained for the survey of AFIA program participants.
• Process study: This research includes site visits each year by Abt Associates staff to five or six selected AFIA programs. During these visits, interviews are conducted with program coordinators, program associates, and representatives of financial institutions to understand how programs have been implemented, how they operate, and how program features may affect participant outcomes. During Phase I, visits were conducted to five IDA programs that received AFIA funding through the initial (Fiscal Year 1999) program grants.

• For the purpose of testing the effectiveness of different approaches to providing incentive payments, the survey sample will be randomly divided into two subgroups. During Wave One, the members of one subgroup will receive their $35 incentive payment after completing the interview. The members of the other subgroup will receive a $10 incentive payment with their advance letter (mailed to them prior to the interview month) and $25 after completing the interview. Whichever method is found to yield a higher response rate will then be adopted for all sample members during Waves Two and Three.
• In addressing the issue of selection bias that arises inherently in non-experimental impact studies, alternative approaches to propensity score matching will be considered. These alternatives will go beyond the previously proposed use of earned income tax credit (EITC) participation as an indicator of the likelihood AFIA participation and may include, for example, the presence of a checking account or savings account as another basis for the propensity scoring.
• A separate subanalysis will be conducted of geographically matched samples of participants and nonparticipants in metropolitan statistical areas. This will enable a more rigorous test of program impacts among metropolitan residents, by accounting more explicitly for the effects of geographic location.

• The crucial challenge facing AFIA program grantees is to find the right fit between the program's basic requirements and the capacities of the operating organization, its partners, and its intended clientele, within the constraints of available funding.
• The five visited programs, selected to encompass a range of operational models, indeed differ greatly in their organizational structures and philosophies and in their operational approaches to major program activities: targeting, recruitment, and screening; case management; and financial education.
• These programs exemplify the tradeoffs that exist between (a) serving needier individuals within the program-eligible population - ones who might not otherwise succeed in saving and accumulating assets, but who require intensive support services to do so - versus (b) serving more members of the program-eligible population with fewer program services.

• Community Services Agency, Reno, Nevada (the "Reno site");
• Mercy Housing, Sacramento, California (the "Sacramento site");
• Mt. Hope Housing Company, Bronx, New York (the "Bronx site");
• Social Development Commission, Milwaukee, Wisconsin (the "Milwaukee site"); and
• YWCA of Greater Pittsburgh, Pittsburgh, Pennsylvania (the "Pittsburgh site").

(a) The one visited program operated by a financial institution was very different from those operated by social service organizations. Even though this program represents only one example, and the bank involved is not typical, it is an informative example of how an AFIA program with strong bank involvement (a "bank-oriented" program) might differ from others. We see the key differences as follows:

• In the bank-oriented AFIA program, IDAs are viewed more as a financial product rather than as a program with ongoing support services. Dollar Bank views IDAs as one financial instrument among many, as a product that has the potential to attract mortgage business from a previously untapped market. This is in contrast to the common view by the social service community of IDAs as a "program" for personal empowerment and transformation. The bank's view shapes its expectations of individuals; a "successful" IDA account is one that turns into an accepted mortgage application within a reasonable length of time. The bank's view also shapes the services provided - technical homeownership information and extensive credit repair services, but little (if any) case management.
• Bank-oriented programs are likely to attract a different population than those recruited by social service agencies. A bank-oriented program is likely to place more emphasis on "casting a wide net" to reach a broader clientele, but then screening applicants to ensure that only those likely to become mortgage-ready are accepted.
• Bank-oriented programs may be capable of operating at a larger scale—but for a different population. Because of the absence of case management and a less personalized approach, bank-oriented programs may well be "leaner" administratively. In contrast, AFIA programs operated by social service organizations may end up being accessible to fewer people, but may reach more individuals "at the margins"—those who can succeed at asset acquisition, but only with program support.
• Banks may be unlikely to want IDAs to be used for purposes that do not promote bank business. The original Dollar Bank program was designed only for home ownership, so this is the only option available to AFIA participants. To the extent that home purchase and business startup have greater loan potential than post-secondary education, banks may choose to restrict IDAs to these uses.

(b) Programs vary in their leniency or stringency, according to organizations' philosophies. A program can be lenient or stringent in several respects: (1) the minimum deposit required; (2) attitude toward participants' unrealistic aspirations; (3) tolerance for emergency withdrawals; (4) tolerance for inactive accounts, or participants who appear unlikely to attain their saving goal; and (5) how closely saving deposits are monitored. One consequence of a relatively "lenient" program design may be a higher number of inactive accounts (because more individuals are accepted who may not succeed). Another might be saving requirements that are unrealistically low for the asset in question, especially for home purchase in tight housing markets.

(c) "Piggybacking" IDA program components onto existing program components can have mixed results. For the most part, the sites developed the components of their AFIA programs specifically for these programs. One site stands in contrast. The Pittsburgh site integrated AFIA program elements into the pre-existing activities of the three partner organizations. Recruitment was to be conducted by the housing authority, case management by the YWCA, and financial education and credit counseling through Dollar Bank's pre-existing homeownership program. This strategy has met with mixed success. Although some elements appear to be working well, others are not. The result is a program that appears fragmented and incohesive.

In contrast, at the other organizations, developing the case management and financial education components specifically for the IDA program involved substantial startup costs, but resulted in centralized, more cohesive program elements.

(d) The administrative requirements of AFIA programs may make them difficult to implement for small organizations with a limited funding base. AFIA-specific administrative costs are not easily measured, but the anecdotal evidence suggests that such costs are extensive. Uniformly, program staff noted that organizations must find other sources besides AFIA administrative funding. Most of the visited sites possessed the resources to cover startup and administrative expenses. Some have been able to subsidize their AFIA program from other internal sources.

The challenges of covering administrative expenses can be particularly acute for organizations affiliated with multi-site grantees. In these cases, the AFIA administrative funds are divided among many organizations, but each one must independently incur the "front-end" administrative costs of implementing an IDA program.

(a) AFIA programs do not "sell themselves." Substantial effort and thought are required to conduct a successful recruitment initiative. This finding was common across sites. Program staff also agreed, however, that "once you get over that initial hump and get positive word of mouth, an IDA program can sell itself."

(b) A variety of recruitment approaches can be effective. The visited sites offer examples that both targeted recruitment and "mass marketing" can work well. Targeted recruitment aimed at a "known" population, such as the organization's client base, can be efficient because less effort is expended on those who turn out to be ineligible. However, this approach will reach fewer people. Mass outreach to the general public casts a wider net, but more individuals may be ineligible, requiring more effort to screen them out.

(c) Participant screening occurs at several levels and undoubtedly affects outcomes. The outcomes that will be observed in AFIA programs will depend in large part on the characteristics of the individuals enrolled. The participant profile is itself importantly determined by: AFIA-mandated eligibility criteria; applicant self-selection (as each program may tend to inherently attract some types of people more than others); and agency selection (screening criteria imposed by the administering organization). The latter two types of selection are subtle and complex.

Anecdotal evidence suggests very strongly that self-selection definitely occurs in AFIA programs, perhaps most notably with respect to motivation. Individuals who enroll in an AFIA program probably already possess a relatively high level of motivation to improve their situation in life. How programs are structured can influence the degree to which participants self-select. For example, imposing a burdensome application process, lack of support services, and stringent program rules will probably result in a program comprised of individuals who are very motivated, with low subsequent dropout. In contrast, providing extensive support and lenient program requirements may allow the less motivated to enter, but with expectedly higher dropout.

Selection of individuals who possess certain characteristics can also occur by the agency. Most of the organizations have an "open-door" policy in which any eligible applicant is accepted. Even so, agency selection can be implicit and very powerful - for example, in its focus on one target population (such as refugees) over another.

(a) Programs vary greatly in their emphasis on case management. Two of the visited sites, Milwaukee and Pittsburgh, offer minimal case management. Their example suggests that intensive case management may not be essential—if the program is composed of accountholders who have the wherewithal to succeed on their own. At the Milwaukee site, the non-intensive nature of the case management stems at least in part from the fact that the target population, refugees, is so motivated that little case management is needed. At the Pittsburgh site, the lack of case management reflects the operational approach of the financial institution (Dollar Bank). The interviewed participants in this program did not express a desire for more case management - but this site also has a fairly high number of inactive accounts. This may indicate that some participants are indeed "falling through the cracks."

(b) Intense, personalized case management can make a difference for individuals "at the margin." Case management can have a significant effect on the type of individuals who stay, and possibly succeed, in an AFIA program. Minimal case management can weed out individuals who need support, perhaps resulting in a participant population quite different than intended. Although a program that offers little case management can support a larger caseload, the program may end up serving clients who are 'self-starters' and might have done equally well without the program. In contrast, intensive case management can make the crucial difference for some individuals at the margins. Offering such support may limit the number of clients who can be served, but may permit the program to serve a needier segment of the community.

In short, both approaches can achieve results—but the affected populations are likely to differ in size and description. Whether a program aspires to serve a needier population is an important strategic decision that should be made after a thoughtful assessment of the fit between the target population's needs for case management and the support that a program can realistically offer.

(c) Program support is especially important at the beginning of an individual's participation. Because attrition can be very high in the initial stages of entering an AFIA program, program staff strongly felt that the need for intensive case management is greatest at the beginning of an individual's participation. Many programs have found that "hand-holding" at that stage is worth the effort - even if this means, for example, having a staff member accompany a participant in going to the bank to open an account.

(d) The tone and apparent effectiveness of case management hinges on interpersonal relationships. The personal style of program staff can matter greatly in motivating participants and sustaining their commitment to the program. With respect to case management specifically, the personality of the case managers appears important for programs that are structured to provide high levels of one-on-one support. In contrast, a relatively impersonal program may be easier to staff and to institutionalize, but it will tend to attract participants who can do well with minimal support.

Organizations should therefore be very careful about whom they assign to the critical "front-line" positions that involve direct contact with AFIA participants. Where the target population includes those of different cultures, cultural sensitivity is of paramount importance.

(a) Excellent financial education can take a number of forms. The type of financial education that is most appropriate depends on the specific needs of the participant population. The financial literacy component varied greatly in length, approach, and content. We saw examples of financial training that ranged from a highly technical approach ("How to buy a home") to a more holistic "life skills" approach ("What does money mean to you"). Both were well received and seemed to meet participants' needs. The important factor seems to be tailoring the financial literacy curriculum to the needs of the specific target population. For example, some populations may need a curriculum aimed at promoting a savings mindset, while others (e.g. refugees) already possess this and may need practical information about how the American financial system works.

(b) Credit counseling is a component of all visited AFIA programs, but the intensity varies widely and may be of varying usefulness. The credit counseling offered among these sites ranges from a one-hour examination of one's credit report with the IDA coordinator (and referral to more extensive credit repair services in the community) to a comprehensive credit counseling program with a bank credit counselor that can last as long as two years. It is too early to know whether these varying approaches are adequately addressing the needs for credit repair.

(c) The quality of the financial literacy training appears to vary. It is uncertain whether weak training will hinder participant success. At several of the visited sites, participants were satisfied with the quality of the financial training they had received. At other sites, participants criticized the financial education they had received. At this early stage, it is unclear whether weak financial education hinders such participants. As with case management, the answer appears to depend on the nature of a program's client population.