<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:st1="urn:schemas-microsoft-com:office:smarttags" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]--><o:SmartTagType
namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PlaceType"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="PlaceName"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="State"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="place"/>
<o:SmartTagType namespaceuri="urn:schemas-microsoft-com:office:smarttags"
name="PersonName"/>
<!--[if !mso]>
<style>
st1\:*{behavior:url(#default#ieooui) }
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";
color:black;}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:blue;
text-decoration:underline;}
pre
{margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.EmailStyle18
{mso-style-type:personal;
font-family:Arial;
color:navy;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:Arial;
color:navy;}
@page Section1
{size:8.5in 11.0in;
margin:1.0in 1.25in 1.0in 1.25in;}
div.Section1
{page:Section1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor=white lang=EN-US link=blue vlink=blue>
<div class=Section1>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>The Longitudinal Study of Adult Learning
has been following a target population of ABE learners over a long period of
time. It’s finding exactly the pattern that others have been
describing – many adults participate in programs over a series of “episodes”
which often span multiple years (NRS accounting periods). When we’ve
presented these data, we’ve suggested that NRS will not capture all of
the impact that programs have on learning in part because of its short-term
focus for measuring both participation and outcomes. I wonder if states
could get waivers on a pilot basis to experiment with longer reporting periods
as David Rosen suggested.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>-<st1:PersonName w:st="on">Steve Reder</st1:PersonName><o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<div>
<div class=MsoNormal align=center style='text-align:center'><font size=3
color=black face="Times New Roman"><span style='font-size:12.0pt;color:windowtext'>
<hr size=2 width="100%" align=center tabindex=-1>
</span></font></div>
<p class=MsoNormal><b><font size=2 color=black face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;color:windowtext;font-weight:bold'>From:</span></font></b><font
size=2 color=black face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma;
color:windowtext'> assessment-bounces@nifl.gov
[mailto:assessment-bounces@nifl.gov] <b><span style='font-weight:bold'>On
Behalf Of </span></b>Dan Wann<br>
<b><span style='font-weight:bold'>Sent:</span></b> Thursday, April 19, 2007
6:47 AM<br>
<b><span style='font-weight:bold'>To:</span></b> 'The Assessment Discussion
List'<br>
<b><span style='font-weight:bold'>Subject:</span></b> [Assessment 756] Re:
Using Data</span></font><font color=black><span style='color:windowtext'><o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>I wonder if there is enough data to even
show that adult basic and ESL students stay with a program in large enough
numbers to track over a longer period? The conventional wisdom of
those outside of the adult basic skills network is that basic skills programs
have little impact because students do not stay long to make a difference.
Do we have any evidence that shows we work with the same students more than one
year and that we work with a high enough number of students more than one year
to make a significant difference? <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><st1:PersonName w:st="on"><font size=2 color=navy
face=Arial><span style='font-size:10.0pt;font-family:Arial;color:navy'>Dan
Wann</span></font></st1:PersonName><font size=2 color=navy face=Arial><span
style='font-size:10.0pt;font-family:Arial;color:navy'><o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>Professional Development Consultant <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'>IN Adult Education Professional
Development Project <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><a href="mailto:dlwann@comcast.net">dlwann@comcast.net</a>
<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=navy face=Arial><span style='font-size:
10.0pt;font-family:Arial;color:navy'><o:p> </o:p></span></font></p>
<div>
<div class=MsoNormal align=center style='text-align:center'><font size=3
color=black face="Times New Roman"><span style='font-size:12.0pt;color:windowtext'>
<hr size=2 width="100%" align=center tabindex=-1>
</span></font></div>
<p class=MsoNormal><b><font size=2 color=black face=Tahoma><span
style='font-size:10.0pt;font-family:Tahoma;color:windowtext;font-weight:bold'>From:</span></font></b><font
size=2 color=black face=Tahoma><span style='font-size:10.0pt;font-family:Tahoma;
color:windowtext'> assessment-bounces@nifl.gov
[mailto:assessment-bounces@nifl.gov] <b><span style='font-weight:bold'>On
Behalf Of </span></b>David J. Rosen<br>
<b><span style='font-weight:bold'>Sent:</span></b> Thursday, April 19, 2007
2:05 AM<br>
<b><span style='font-weight:bold'>To:</span></b> <st1:PersonName w:st="on">The Assessment
Discussion List</st1:PersonName><br>
<b><span style='font-weight:bold'>Subject:</span></b> [Assessment 752] Re:
Using Data</span></font><font color=black><span style='color:windowtext'><o:p></o:p></span></font></p>
</div>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'>Larry, and others,<br>
<br>
Tina, and many other program administrators have observed patterns like this
that suggest that a one-year time frame, a funding year, may not be the best
unit of time in which to measure learner gains, except for those who are doing
basic skills brush-up or who have very short-term goals like preparing for a
drivers license test. I wonder if there is a possibility that the NRS might be
adjusted, perhaps in a pilot at first, so that a longer period of learning, say
three years, might be used to demonstrate learner gains. Of course, there
would need to be intermediate measures, but accountability -- for programs and
states -- might be based on a longer period of time. <br>
<br>
It seems to me that the one-year time frame within to measure learning gains or
goals accomplished comes not from K-12 or higher education, but rather
Congressional expectations for job skills training. Would you agree?<br>
<br>
Also I wonder if you or others have some examples of programs that track and
report learner outcomes over several years, and use the data for program
improvement.<br>
<br>
David J. Rosen<br>
<a href="mailto:djrosen@comcast.net">djrosen@comcast.net</a><br>
<br>
<br>
<a href="mailto:Tina_Luffman@yc.edu">Tina_Luffman@yc.edu</a> wrote: <o:p></o:p></span></font></p>
<div>
<p class=MsoNormal><font size=2 color=black face=Verdana><span
style='font-size:10.0pt;font-family:Verdana'>Hi Luanne,<o:p></o:p></span></font></p>
</div>
<div>
<p class=MsoNormal><font size=2 color=black face=Verdana><span
style='font-size:10.0pt;font-family:Verdana'> <o:p></o:p></span></font></p>
</div>
<div>
<p class=MsoNormal><font size=2 color=black face=Verdana><span
style='font-size:10.0pt;font-family:Verdana'>I find it interesting that what
you are finding in data seems to be consistent with what we see in our GED
classes here in <st1:place w:st="on"><st1:State w:st="on">Arizona</st1:State></st1:place>.
Often the last group who enter in March are the least likely to stay with the
program until posttesting, and the August group seem to have the highest
posttesting and retention rate.<o:p></o:p></span></font></p>
</div>
<div>
<p class=MsoNormal><font size=2 color=black face=Verdana><span
style='font-size:10.0pt;font-family:Verdana'> <o:p></o:p></span></font></p>
</div>
<div>
<p class=MsoNormal><font size=2 color=black face=Verdana><span
style='font-size:10.0pt;font-family:Verdana'>Tina<o:p></o:p></span></font></p>
</div>
<div>
<p class=MsoNormal><font size=2 color=black face=Verdana><span
style='font-size:10.0pt;font-family:Verdana'><br>
<br>
<br>
<br>
Tina Luffman<br>
Coordinator, Developmental Education<br>
<st1:place w:st="on"><st1:PlaceName w:st="on">Verde</st1:PlaceName> <st1:PlaceType
w:st="on">Valley</st1:PlaceType></st1:place> Campus<br>
928-634-6544<br>
<a href="mailto:tina_luffman@yc.edu" target=blank>tina_luffman@yc.edu</a><o:p></o:p></span></font></p>
<div>
<p class=MsoNormal><font size=2 color=black face=Verdana><span
style='font-size:10.0pt;font-family:Verdana'><o:p> </o:p></span></font></p>
</div>
<p class=MsoNormal style='margin-bottom:12.0pt'><font size=2 color="#990099"
face=Verdana><span style='font-size:10.0pt;font-family:Verdana;color:#990099'>-----assessment-bounces@nifl.gov
wrote: -----</span></font><font size=2 face=Verdana><span style='font-size:
10.0pt;font-family:Verdana'><o:p></o:p></span></font></p>
<p class=MsoNormal style='margin-bottom:12.0pt'><font size=2 color=black
face=Verdana><span style='font-size:10.0pt;font-family:Verdana'>To: <a
href="mailto:assessment@nifl.gov"><assessment@nifl.gov></a><br>
From: "Luanne Teller" <a href="mailto:lteller@massasoit.mass.edu"><lteller@massasoit.mass.edu></a><br>
Sent by: <a href="mailto:assessment-bounces@nifl.gov">assessment-bounces@nifl.gov</a><br>
Date: 04/18/2007 10:56AM<br>
Subject: [Assessment 746] Using Data<br>
<br>
<o:p></o:p></span></font></p>
<div><u1:SmartTagType name="State" namespaceuri="urn:schemas-microsoft-com:office:smarttags"></u1:SmartTagType><u1:SmartTagType name="place" namespaceuri="urn:schemas-microsoft-com:office:smarttags"></u1:SmartTagType>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>Hi
all: <u1:p></u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'><u1:p>
</u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>I
wanted to chime in about our program’s use of data since this is the
focus of our discussion. Coincidentally, I am in the process of writing
our proposal for next year, so I am knee-deep in data even as we speak! <u1:p></u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'><u1:p>
</u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>The
use of data takes many forms in our program. We look at what most people
consider the “hard data” -- the raw numbers with regard to
attendance, learner gains, retention, goal attainment, etc. We believe;
however, that the numbers alone provide an incomplete picture of what is
happening, so we use the numbers as a basis for discussion, not decision
making. After analyzing the numbers, we begin to look at additional
sources of data that we find essential in informing our planning---meetings
with staff, classes, our student advisory board, and focus groups. <u1:p></u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'><u1:p>
</u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>Here’s
an example we’re currently working on---we did a two year analysis of
learner retention, and began to document why students did not persist. We
found that the retention for students who enrolled after January 1 (our
programs runs on a school calendar year from September to June) was
significantly lower than the retention for students who began in
September. Even more compelling, we learned that the retention for
students who began after March 1 was 0%. <u1:p></u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'><u1:p>
</u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>We
met with staff and students, and did some research around student retention
issues. After a year-long process, we decided to pilot a “managed
enrollment” approach. In <st1:State u2_x003a_st="on" w:st="on"><st1:place
w:st="on"><st1:State w:st="on">Massachusetts</st1:State></st1:place> </st1:State>,
our grantor (MA DOE) allows us to “over-enroll” our classes by 20%,
so we enroll 20% more students in the fall. When students leave, we
“drop” the overenrolled students into funded slots. This allows
us to keep the seats filled even with the typical attrition that occurs. <u1:p></u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'><u1:p>
</u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>In
January, when we do our mid-point assessments; we move students to the higher
level who are ready to progress….that typically leaves several openings
in the beginner levels and we begin students in February as a cohort.
This year, we implemented new orientation programs including a requirement that
new students observe a class before enrolling. <u1:p></u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'><u1:p>
</u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>While
it is still too early to tell if these new procedures will have a positive
impact, we are hopeful and we know anecdotally that the transition seems to be
easier for some of these students. We are eager to look at the data at
the end of the year to analyze the effectiveness of this plan. <u1:p></u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'><u1:p>
</u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>As
we begin to look at our data, we are finding that there seem to be a unique set
of issues for our beginner ESOL students. We suspect that the lack of
effective English communication skills to advocate for themselves with
employers is influencing their attendance and persistence. This is an
issue that we are beginning to tackle in terms of policy. Do we need to
have a more flexible, lenient policy for beginner students? Is there a
way to support students in addressing these employment issues? How can we
empower students more quickly? Are there other issues for these beginner
level students that affect their participation? As we enter these
discussions, the numbers will provide a basis for developing strategies, but
the students themselves with be our greatest source of valuable data. <u1:p></u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'><u1:p>
</u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>Luanne
Teller <u1:p></u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'><u1:p>
</u1:p></span></font><o:p></o:p></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 color=black face=Arial><span style='font-size:10.0pt;font-family:Arial'>Luanne
Teller <u1:p></u1:p></span></font><o:p></o:p></p>
</div>
<p class=MsoNormal><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt;font-family:"Courier New"'>-------------------------------<br>
National Institute for Literacy<br>
Assessment mailing list<br>
<a href="mailto:Assessment@nifl.gov">Assessment@nifl.gov</a><br>
To unsubscribe or change your subscription settings, please go to <a
href="http://www.nifl.gov/mailman/listinfo/assessment" target=blank>http://www.nifl.gov/mailman/listinfo/assessment</a></span></font><font
size=2 face=Verdana><span style='font-size:10.0pt;font-family:Verdana'><o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color=black face=Verdana><span
style='font-size:10.0pt;font-family:Verdana'><o:p> </o:p></span></font></p>
</div>
<pre wrap=""><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre
style='text-align:center'><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre
style='text-align:center'><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre
style='text-align:center'><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'>
<hr size=4 width="90%" align=center>
</span></font></pre><pre style='text-align:center'><font size=2 color=black
face="Courier New"><span style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre
style='text-align:center'><font size=2 color=black face="Courier New"><span
style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'><o:p> </o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>-------------------------------<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>National Institute for Literacy<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Assessment mailing list<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'><a
href="mailto:Assessment@nifl.gov">Assessment@nifl.gov</a><o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>To unsubscribe or change your subscription settings, please go to <a
href="http://www.nifl.gov/mailman/listinfo/assessment">http://www.nifl.gov/mailman/listinfo/assessment</a><o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'><o:p> </o:p></span></font></p>
</div>
</body>
</html>