Hi Jennifer,
 
Thanks for your interest, and thanks to Marie for putting together this forum.  I'm an assessment Specialist for a large, part-time adult ESL program, but my answer might surprise you.  Of course we give tests; Massachusetts' DOE has worked hard on trying to find the best way to provide valid and reliable testing for Doe-funded ABE programs, and given the tests that are out there, they did a good job.
 
But we also report time spent in class, and goals set and goals met, all in an effort to try to show the wonderful things that happen in our classes.  Like you, we sit down and ask the student what are their reasons for wanting to learn with us, and we revisit them as often as we can.  Whenever a student meets a goal, we have a statewide database called SMARTT (don't ask me what it stands for), in which we record goals.
 
 
**PERSONAL STATEMENT ALERT** the following section gets long, (sorry) and may contain some opinion as well as fact- be warned ;-)
 
What we do goes far beyond test scores.  Of course literacy gains are what we're here for, but students aren't learning English because of its inventive and fascinating spelling- they are learning it to get better jobs, to enculturate in North American.  These are the real reason why students come to our programs.  We can measure a 78-point gain in the BEST Plus, and that's great, but for me, success is when a student is able to do something they couldn't before, at least in part because of our program's help.  
 
For me, success is when a student, who 5 years ago could NEVER imagine themselves speaking English, becomes the translator for a friend, or communicates in English with their doctor, their in-laws, their child's teacher, their new boyfriend or girlfriend.  It is a sense of restored self-agency; people who come to a new land with new language and culture lose so much of their autonomy, and success is when they regain some measure of it.
 
I believe that most ABE teachers feel this strongly, and the MA DOE gets this.  There is a prevailing national trend in WIA towards ABE as strictly a worker-training program.  The DOE in Massachusetts is hip to the fact that there is more going on, that there are other aspects to success that how to punch a clock or dress for an interview.  So, since the 90's, it has worked to keep student goals at the center of the accountability system.  Thankfully, the powers that be here in MA feel that goals are as important as test scores, but we needed to back this up with valid and reliable record-keeping to make it manifest up on the Hill, where positivism is the key, and no one believes anything that's not scientifically-verifiable.  So we need to make copies of documentation like library cards, letters of closing, voter registration cards, etc, in order to prove that these things, these "outcomes" are actually happening to our students.  Forgive my arch tone- I know that 39 million in state and federal funds is a lot of money, and people want oversight and bang for their buck, I just sometimes resent the time we take away from teaching in order to keep good records. 
 
I also fear that the only programs that are attractive to funders are those that pigeonhole learners into low-wage jobs with no career ladder, and that's not what our students deserve.  
 
At the turn of the last century, the immigration wave found an expanding industrial economy that readily absorbed them with only minor training.  The turn of this century finds us in a post-industrial period, what Andrew Hargreaves calls "the Knowledge Society"; we need to move beyond filling in an application and being on time.  Murnane and Levy's book, "Teaching the New Basic Skills", talks more about this, as does the MassINC report "New Skills for a New Economy"- there are "soft skills" that students need to succeed in today's economy- computer, teamwork, leadership, self-direction- these things don't show up on the REEP, the BEST Plus, the TABE, BUT I KNOW THEM WHEN I SEE THEM IN MY CLASS!
 
So, in sum, in Massachusetts we report test scores, time spent in class, goals set and goals met; all this in effort to try to show the wonderful things that happen in our classes.  Imagine you needed to scientifically verify your child's growth- it would be a list of stats (height and weight, etc); it would include narrative description of quotidian achievements (made and ashtray, played three innings in little league, made peace with sister, etc).  It could not. However describe the miracle of watching someone grow and develop.  I'm not comparing our adult students with children, mind you, just trying to show that the quality of human development loses a lot when quantified.

________________________________

From: assessment-bounces at dev.nifl.gov on behalf of Marie Cora
Sent: Mon 12/5/2005 5:44 PM
To: AssList
Subject: [Assessment] Measuring outcomes


Hi everyone,
 
I'm reposting part of a message that was sent by Jennifer Harper at the end of this past October.  She asks about measuring outcomes.  Several email replies noted sources of student goals and other resources for surveying students.  How do you measure success in your program?  Do you depend on test results to help you?  Do you use other methods?  Do you use a combination of tests and something else?
marie
 
 

>We are trying to find ways of measuring outcomes.  We 

decided 

>that it is best for literacy students to set their OWN 

>goals.  One way we have decided to measure outcomes is 

>through a goals sheet.  We would ask the students their 

goals 

>in the beginning of the program, and then interview them to 

>find out when they have achieved their goals.  All of this 

>information would be recorded and used as a way of measuring 

>success.

> 

>That is our main way of measuring outcomes.  Do any of you 

>have interesting methods of measuring outcomes in literacy 

>programs that you would be willing to share?

> 

>Thanks in advance,

>Sincerely,

>Jennifer Harper

>JLHarp21 at aol.com <mailto:JLHarp21 at aol.com?Subject=Re:%20%5BNIFL-ASSESSMENT:1303%5D%20RE:%20Working%20with%20Student%20Goals&In-Reply-To=%3C006f01c5db27$ec5bbdd0$0202a8c0 at frodo%3E>  

 
(nifl archives:  http://www.nifl.gov/nifl-assessment/2005/  Oct. 26 and 27, 2005)
 
 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/ms-tnef
Size: 11644 bytes
Desc: not available
Url : http://www.nifl.gov/pipermail/assessment/attachments/20051205/508dd2cd/attachment.bin