Reply to:   RE: [NIFL-PLI] Re: NIFL-PLI Digest, Vol 8, Issue 1
Sonia makes some interesting points that I'd like to turn back to the group for discussion. I've been doing this work for almost twenty years now, but it's hard for me to remember the days before program accountability was a primary focus for those of us who are leaders in our field. None-the-less, there was such a time and it wasn't as if we all sat around waiting for something to do or for someone to invent an adult education accountability system. 
Before the advent of the NRS, which data, if any, WERE programs actually collecting and analyzing data as a basis for program improvement? We often talk about data for decision-making as if it were a no-brainer, but in my experience, not every program buys into ongoing data collection and analysis as a basis for continuous improvement.

Last night, I was reading a dissertation by Priscilla Sissem, one of my PA colleagues, titled, "Leaders of Adult Basic and Literacy Education Programs in Pennsylvania: A Typology of Leadership Styles and Organizational Issues in Context."  Her premise is that leaders in our field can be broken into three typologies:  Founders, Mainstreamers, and Prospectors -- with each type approaching this work somewhat differently. She notes that whereas some program leaders believe that increased accountability has set a higher standard for our field, has brought us credibility and a new professionalism, others suggest that increased accountability has undermined one of our most deeply held commitments - the commitment to learners as individuals and to each learner's need for individualized instruction. This tension is fascinating and certainly underscores the diversity of the leadership in our field.

Larry's questions about how we use data for program improvement are based on the assumption that program leaders actually find value in collecting and analyzing data as a basis for decision-making. Clearly, some do and others do not.  I'm wondering if any discussion related to using the NRS for program improvement might need to start with an exploration of state and federal accountability systems in general and the impact they have had, and continue to have, on our field.

Any thoughts?


Sandy Strunk
Lancaster Lebanon Intermediate Unit 13
Adult Basic and Family Literacy Education
1110 Enterprise Road
East Petersburg, PA 17520
(717) 519-1006

Sonia Socha wrote:

>This overview from Larry was excellent information for a provider in the >field to pass on to staff who are trying to implement the NRS.

>

>The questions at the end will yake along time to answer and I prefer not to >write.  Would much rather have a personal conversation with Mr. Condelli.

>

>In many ways, our program was way ahead of NRS intitially with collecting >data and making program improvements based on our collections--but what we collect >and what we look for is slightly different than what the feds are looking for.  >And , of course, there are those items that are the same--but we anaylze them >differently.

>

>Do not have the time to answer these questions now--too much on the plate >in the month of May--especially completing grants for adult literacy funding!  We >are also preparing for a visit from USDE in June--I know our staff will have much >to say about data collection during that monitoring visit.

>

>Happy to talk sometime over the summer--I am sure our program will have >much to contribute to the questions outlined below.

>

>Thank you for the opportunity.   >

>>>> nifl-pli-request at nifl.gov 05/10/04 12:00PM >>>

>Send NIFL-PLI mailing list submissions to

>	nifl-pli at literacy.nifl.gov >

>To subscribe or unsubscribe via the World Wide Web, visit

>	http://www.nifl.gov/mailman/listinfo/nifl-pli >or, via email, send a message with subject or body 'help' to

>	nifl-pli-request at literacy.nifl.gov >

>You can reach the person managing the list at

>	nifl-pli-owner at literacy.nifl.gov >

>When replying, please edit your Subject line so it is more specific

>than "Re: Contents of NIFL-PLI digest..."

>

>

>Today's Topics:

>

>   1. Using NRS Data for Program Improvement (Sandy Strunk)

>

>

>----------------------------------------------------------------------

>

>Message: 1

>Date: 10 May 2004 07:59:03 -0700

>From: Sandy Strunk <sandy_strunk at iu13.org>

>Subject: [NIFL-PLI] Using NRS Data for Program Improvement

>To: "NIFL -PLI List" <nifl-pli at literacy.nifl.gov>

>Message-ID: <20040510115616.D0950470023 at relay.iu13.org>

>Content-Type: text/plain; charset="iso-8859-1"

>

>The following posting is from Larry Condelli, Managing Director in the >Education and Human Development Division of the American Institues for Research. >Larry leads the adult education team and is project director for the National >Reporting System. He has been involved in projects on ESL research, the professional >development of adult educators and the development of content standards for adult >education. He has worked in adult education since 1990.

>

>At the end of his posting, Larry poses several questions he hopes you'll >respond to. I, too, hope you will take this opportunity to discuss the National >Reporting System and how NRS data can be used for program improvement.

>

>Sandy Strunk, PLI List Moderator

>

>********************************************************

>

>

>	Several years ago, I started a new job and my first assignment was a very >small project to help the Division of Adult Education and Literacy (DAEL) assist >states to improve their ability to evaluate the quality of local adult education >programs.  Little did I suspect that this work would eventually evolve into >developing the national accountability system for the adult education program, and >what we now know as the National Reporting System (NRS).  Before this, I had little >knowledge of adult education and accountability, but life has a way of bringing >about the unexpected. >	Most people probably don't know that the NRS had its origin in 1996, when >a group of state directors of adult education were the first to move forward the >idea for a national accountability system.  At that time there were proposals in >Congress to roll adult education into a national work force development block >grant system.  It was widely feared that block grants, giving governors discretion >on how to set up the system in their states, would mean the end of the adult >education program.  At the same time, demands for accountability increased for all >programs, at both the federal and state levels.  To preserve meet these demands and >adult education, state directors and others realized we needed a way to collect >valid and reliable data to show what adult education is, the type of students who >enrolled and the outcomes students achieved * and that these outcomes went beyond >employment.  DAEL responded in 1997 with a two-year project, which I directed, >to develop an outcome-ba!

>sed accountability system in collaboration with state education directors >and other adult educators.  >	One goal in developing this system was to produce a set of valid and >reliable data that could demonstrate to legislators and other policymakers the value >of adult education.  But we also wanted the system to do more than that.  We >recognized the value of data as a program improvement tool and wanted the system to be >a source of information for states and local programs to use for program >management and improvement. Therefore, we tried to build state flexibility into the data >requirements.

>

>Based on the system we developed though this project in 1997-1998, the NRS >was born and eventually incorporated into the Workforce Investment Act (WIA) in >1998.  WIA made the system mandatory (we had considered it voluntary until then) >and added a requirement for states to set performance standards for outcome >measures and an incentive award program to motivate performance.  Our focus then >shifted from system development to implementation.

>

>Since 1998, we have engaged in a series of training and technical >assistance activities that we hoped would build on each other to meet our dual goals of >having an effective accountability system and a source of data for local and state >program improvement efforts.  We planned these activities in three phases.  >First, we needed to get the NRS understood and working in each state, which we >accomplished through national and regional training, development of policy and >procedural documents, web sites and individual technical assistance. >In the second phase we focused on improving the quality of data and >improving the uniformity of how states and programs collected the date.  In 2002, we >produced a data quality guide for local programs that described ways to collect data >in valid and reliable ways and reinforced this information with regional >trainings of state staff.  As these data quality improvement efforts continue we moved >into the third phase of assistance, promoting data use for program management and >improvement among states and local programs.  We again developed a "how-to" >guide, accompanied by regional state trainings in 2003.  Most recently, we are just >now completing a third series of workshops on using data for program monitoring to >assess local program performance and identify areas for change.

>

>Promoting NRS data use for program improvement continues to be one of our >top priorities.  We see using data as the key to improving the quality of local >programs and as a way to get state and local staff to understand the value of data >* to see data not as an administrative burden, but a valuable tool.  I think we >are beginning to see some successes in this area.  We used to have to spend some >time at our trainings to answer questions about the NRS and defend its purpose and >value.  Now we not only hear more acceptances, but a genuine desire to learn to >use data and really improve program qualityØour original goal all along.  For >example, programs are using data to make recruitment and enrollment decisions and >schedule class times and instructional approaches.  States are using data to >identify high performing and low performing programs, to target technical assistance >for program improvement and make funding decisions.   Many states NRS data systems >now have reports built !

>in that help local programs understand their student attendance, enrollment >and outcomes and make informed decisions on program improvement efforts.

>

>While this is heartening to know, we also know there is a lot more work to >do, which brings us to our topic, using NRS data for program improvement.   In >traveling the country and doing training and technical assistance on this topic I >recognize some of the issues involved in using data and have heard several of the >questions that can serve as a basis for our discussion here.

>

>What kinds of decisions do you make using data?  What kinds of data do you >use to steer local improvement?

>

>How accessible are these data to you?

>

>What successes have you had in using data?

>

>What data do you need that NRS does not collect?

>

>What are barriers to using data?  What other systems and structures do >states and local programs need to put into place to supplement NRS data?

>

>Larry Condelli

>American Institutes for Research

>

>

>

>------------------------------

>

>_______________________________________________

>NIFL-PLI mailing list

>NIFL-PLI at literacy.nifl.gov >To unsubscribe or change your subscription settings, please go to >http://www.nifl.gov/mailman/listinfo/nifl-pli >

>

>End of NIFL-PLI Digest, Vol 8, Issue 1

>**************************************

>_______________________________________________

>NIFL-PLI mailing list

>NIFL-PLI at literacy.nifl.gov

>To unsubscribe or change your subscription settings, please go to >http://www.nifl.gov/mailman/listinfo/nifl-pli

>

>RFC822 header

>-----------------------------------

>

> Return-Path: <nifl-pli-bounces at literacy.nifl.gov>

> Received: from relay.iu13.org ([unix socket])

> 	by mail.iu13.org (Cyrus v2.1.11-Lancaster-Lebanon-IU13-Mail-Services) >with LMTP; Mon, 10 May 2004 15:47:38 -0400

> X-Sieve: CMU Sieve 2.2

> Received: from gabbro.iu13.org (gabbro.iu13.org [172.21.1.73])

> 	by relay.iu13.org (Postfix) with ESMTP

> 	id BC6584701CF; Mon, 10 May 2004 15:47:37 -0400 (EDT)

> Received: from literacy.nifl.gov (unknown [192.188.111.2])

> 	by gabbro.iu13.org (Postfix) with ESMTP

> 	id 1A1672FEA9; Mon, 10 May 2004 15:47:35 -0400 (EDT)

> Received: from literacy.nifl.gov (localhost [127.0.0.1])

> 	by literacy.nifl.gov (8.10.2/8.10.2) with ESMTP id i4AJkLm12593;

> 	Mon, 10 May 2004 15:46:21 -0400 (EDT)

> Received: from NW51_1 (mail.southbaltimorelearns.org [141.157.37.194])

> 	by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id i4AJURm12187

> 	for <nifl-pli at nifl.gov>; Mon, 10 May 2004 15:30:27 -0400 (EDT)

> Received: from SBLC_DOM-Message_Server by NW51_1

> 	with Novell_GroupWise; Mon, 10 May 2004 15:27:24 -0400

> Message-Id: <s09f9f5c.024 at NW51_1>

> X-Mailer: Novell GroupWise 5.5.4

> Date: Mon, 10 May 2004 15:27:15 -0400

> From: "Sonia Socha" <Ssocha at southbaltimorelearns.org>

> To: <nifl-pli at nifl.gov>

> Mime-Version: 1.0

> Content-Type: text/plain; charset=ISO-8859-7

> Content-Disposition: inline

> Content-Transfer-Encoding: 8bit

> X-MIME-Autoconverted: from quoted-printable to 8bit by literacy.nifl.gov id

> 	i4AJURm12187

> X-Mailman-Approved-At: Mon, 10 May 2004 15:46:14 -0400

> Cc: > Subject: [NIFL-PLI] Re: NIFL-PLI Digest, Vol 8, Issue 1

> X-BeenThere: nifl-pli at literacy.nifl.gov

> X-Mailman-Version: 2.1.4

> Precedence: list

> Reply-To: Program Leadership and Improvement Discussion List ><nifl-pli at nifl.gov>

> List-Id: Program Leadership and Improvement Discussion List

> 	<nifl-pli.literacy.nifl.gov>

> List-Unsubscribe: <http://www.nifl.gov/mailman/listinfo/nifl-pli>,

> 	<mailto:nifl-pli-request at literacy.nifl.gov?subject=unsubscribe>

> List-Archive: <http://www.nifl.gov/pipermail/nifl-pli>

> List-Post: <mailto:nifl-pli at literacy.nifl.gov>

> List-Help: <mailto:nifl-pli-request at literacy.nifl.gov?subject=help>

> List-Subscribe: <http://www.nifl.gov/mailman/listinfo/nifl-pli>,

> 	<mailto:nifl-pli-request at literacy.nifl.gov?subject=subscribe>

> Sender: nifl-pli-bounces at nifl.gov

> Errors-To: nifl-pli-bounces at nifl.gov

> X-Content-Scanned: by IU 13 eMail Scanning

>