A wide variety of evaluation projects have been funded under Adult Education National Programs since beginning in FY 1988. These projects have been administered by the Office of Vocational and Adult Education (OVAE), the Office of the Under Secretary, Planning and Evaluation Service (OUS), or the Office of Educational Research and Improvement (OERI). The projects funded have ranged from small commissioned papers to large-scale nationally representative surveys and a random assignment experiment of a welfare reform program involving adult education. New studies are focusing on assessment of the impact of programs for special target populations and development of models of effective programs.

The Department is continuing to organize adult education studies around two broad categories--

• Assessing the impact of Federal programs and
• Improving local practice in adult education

The first category--assessing the impact of Federal programs--is a long-term, basic priority for Adult Education National Programs. Studies completed or almost done have assessed the basic State grant program for adult education as well as several smaller discretionary programs such as the Section 353 State set-aside for training and innovation. Prior to developing the reauthorization proposal for the Adult Education Act, we synthesized research and evaluation findings from these and other studies.

The second category--improving local practice--is another long-term priority for adult education. There have been few high-quality evaluations in the field of adult education that assess the effectiveness of local practice. The Department has recently begun major evaluations of Adult Basic Education and English-as-a-Second Language (ESL) programs to identify effective practices in those areas. The Department has also collaborated with the National ESL Clearinghouse on Literacy Education (NCLE) to develop products on effective strategies for working with adults on ESL and in providing technical support for disseminating findings from the National Adult Literacy Survey. We have also conducting a case study of family literacy programs in conjunction with the Even Start program.

In addition to these long-term categories for adult education national programs, the Department has continuing interest in the categories of technology, professional development, welfare reform, and State and local performance and evaluation data.

Among the Department's topical goals that respond to priorities set by the President and the Secretary or to specific adult education issues needing attention are --

• Using technology to improve instruction and administration,
• Examining adult education's role in welfare reform, and
• Improving State and local performance and evaluation data.

Using technology to improve adult education instruction and administration is consistent with both the Secretary's broad interest in supporting the use of educational technology and the growing demand of adult educators for greater Federal leadership and support for development and technical assistance. Technology can play a key role in adult education, particularly in supporting the individualization of instruction for adults. The Department has underway two projects with the National Center on Adult Literacy (NCAL) one for the development of interactive teleconferences for adult education and literacy service providers and the other to plan for the development of adult education software. In collaboration with NCAL, we are also supporting an evaluation of the Crossroads Cafe Distance Learning Project which is developing ESL instructional materials.

Welfare reform, a key priority for this Administration, has major implications for adult education. Policy makers have identified long-term welfare recipients' lack of basic skills as an important barrier to self-sufficiency. Adult educators need to improve their programs' holding power for welfare clients as well as collaborate with social services agencies in designing a coordinated, comprehensive set of services that will promote self-sufficiency and employment. The Department is continuing to support the assessment of literacy and math gains as part of the JOBS evaluation, in addition to special analyses of adult education services for welfare clients.

Improving State and local performance and evaluation data is a priority that builds on a number of studies. The data collection experiences reported by Development Associates, Inc., plus the discrepancy between their total number of participants and the State reports, has confirmed the need to improve both local and State data systems. Local projects do not often give priority to data collection and record-keeping, limiting their own, as well as State and Federal ability, to learn from their experiences. A new contract with the American Institutes for Research is developing a national performance measurement system in collaboration with State Directors of Adult Education.

Descriptions of projects in each of these areas follow.

Using Technology to Improve Instruction and Administration -- Crossroads Cafe Distance Learning Project

English as a Second Language (ESL) is the fastest growing instructional area in the adult education program. The Department considers addressing the instructional needs of this expanding ESL population and working to ensure the quality of available ESL instruction as priorities.

Development of high-quality ESL instructional materials is underway through the collaborative efforts of five States - New York, California, Florida, Illinois, and Arizona - and a private sector company, INTELECOM. PBS and Jones Intercable, two national networks, are committed to broadcasting the series. The content portion of the project will be developed and broadcast by INTELECOM. Instructional segments are to be 30-60 minute in length, with a cast of several characters who portray a variety of real life situations. Diversity in people and situations will be emphasized. Course content would include cultural themes, issue-oriented topics as well as life skill topics of interest to immigrant adult learners. Each instructional segment will give viewers practice in developing listening, observing, speaking, reading and writing skills. Multi-level work texts will also be written to enable the adult learners to build on the content of the video lessons. The work texts will also allow students to practice pronouncing the words and structures they hear in the video and to move from mechanical to meaningful communicative levels of English. Staff development would enter into the Crossroads Cafe project in that ESL instructors would receive training by the appropriate State or local adult education service providers in using the ESL series effectively as an instructional tool.

The role of the Department in Crossroads Cafe Distance Learning Project is to support an evaluation of the 26 ESL language instructional modules and the work texts that are being developed by this consortium. This evaluation endeavor will: 1) ensure development of a high-quality series of ESL instructional video tapes; 2) ensure that the distance learning processes used in the product are effective and efficient; and 3) permit the early identification of any needed refinements, corrections. The evaluation design will involve nationally-recognized distance learning experts to assess the project.

The first component, funded in FY 1994 through NCAL, will obtain information at various sites on the instructional content of the instructional series. Some questions to be answered include: How easily can the material be introduced to a group of ESL students? How appealing do learners find the characters used in the video? How well do the learners comprehend the action, story line, and lessons of the video?

An evaluation of the implementation of the Crossroads Cafe video series and accompanying material will be conducted through a competitive procurement with FY 1995 funding. It will describe the implementation process in several sites where Crossroads Cafe is used to augment classroom ESL instructional programs and/or provide ESL instruction to adult learners who view Crossroads Cafe solely in their homes. The major factors that need to be considered in initiating and maintaining an ESL distance learning project at the local level will be identified. The potential replicability of the Crossroads Cafe Project will also be examined.

Special Projects

"What Works" Evaluation for Adult ABE Students

The Department has awarded a contract to Abt Associates, Inc. to conduct a major evaluation of promising adult education basic skills programs for low-literate learners. This study will be conducted in three phases:

• Development of two models (year one).
  • a model or models of the key elements of effective programs for low-literate learners.
  • an evaluation model using experimental design to determine whether the program model is effective in a particular site.
• Pilot testing the models in a few sites (years two and three).
• Initiation of a national study to confirm the effectiveness of the models in a variety of sites throughout the country (year three in overlap with second year of pilot tests through year five).

The development phase of this study will be highly iterative, with initial models identified by the contractor undergoing review and testing against available data and knowledge. Activities will include secondary data analyses (especially of data from the National Evaluation of Adult Education Programs and from the Even Start Evaluation; widespread consultation and interviews with the adult education field and experts in evaluation methodology; and case studies of sites that have the characteristics identified in the models (both successful and unsuccessful sites). From these activities, the contractor will develop a final set of program and evaluation models for pilot testing in specific sites.

The pilot test phase will involve testing the program and evaluation models in about five sites that meet key elements in the effective programs model and are willing to participate in a rigorous evaluation. We will also extend the evaluations of the pilot sites a second year to attempt to validate the effectiveness of the programs in those sites.

The national study phase will begin in year three, after initial effects and feasibility of the models are tested in the pilot sites. This phase will involve either a stratified sample or a purposively selected sample of projects that represent a broad range of types of local programs. The end result should be both validation of evaluation methodology for assessing "what works," including under what circumstances the models work and don't work, and identification, description, and validation of effective practices that can be widely used by adult educators for low literate learners.