Larry and all,

That's true. Table 4 and Table 4b contain a great deal of information, which if analyzed by program, site and class, can help begin to identify strengths and areas needing additional support. For example, these two tables can help program administrators/teachers identify some of the following reasons behind low percentages of level gains:

1. Students aren't staying long enough to post-test. If the total contact hours divided by the total number of students is less than the number of hours required for post-testing, we know we have a retention problem. Now we can look at: the intake process and transition into class; whether instruction is meeting student needs/expectations; whether curriculum and materials meet the varied levels and learning styles/modalities of students; outside barriers that keep students from attending; etc.

2. Students are staying long enough to post-test, but are not being post-tested. (Compare the number of students on Table 4 to the number on Table 4b.) If this is the case, we can look into whether this is a staff problem - i.e. they have no sytem to know when a learner has enough hours to post-test; they need assessment training/support; etc. - or a student problem. In some programs we have found that when students learn they will be taking a post-test, they stop coming for a period of time. In this case, we need to help students understand that we are assessing our effectiveness, not their intelligence.

3. Students are being post-tested, but not making level gains. (Compare the number of students on Table 4b -i.e. those who have been post-tested - to the number who show a gain on that table.) If this is the case, we can begin to look at instructional strategies and curriculum/materials.

There is one other possibility that I can think of. Students may be post-tested and making gains, but in some cases there is a glitch in the paper flow and for some reason information is not being submitted for data entry or data entry is flawed. If you have a system for submitting class data back to each teacher for review, they can help you identify if this is the case and where the glitch may be taking place.

In short, I love the NRS reports because they tell you so much about your program if you take time to analyze them. I hope I haven't taken up your time telling all of you things you already knew. I'm looking forward to hearing some great new ideas this week.

Barbara Hofmeyer 
Coaching Consultant for 
Indiana DOE, Division of Adult Education

On Tue, 17 Apr 2007 12:07:10 -0400, Condelli, Larry wrote

> Barbara, 

>   

> That is a great idea and if you can break it down even further -- by site or class, for example -- you can get even better insights.

> 

>


-----------------------------------------------------------------------

From: assessment-bounces at nifl.gov [mailto:assessment-bounces at nifl.gov] On Behalf Of bhofmeyer

> Sent: Tuesday, April 17, 2007 9:51 AM

> To: The Assessment Discussion List

> Subject: [Assessment 727] Re: Using Data for Program Improvementbegins today!

> 

> 

> Barbara, 

> In addition to the other great ideas discussed yesterday, I'd like to offer an additional way to analyze NRS data on the local level. If you divide the contact hours by the number of enrolled students in each educational functioning level (EFL) on Table 4, you will get a rough idea of how long students in each EFL are staying in class. If you find that students in one of the levels - say beginning literacy - are leaving before they have enough hours to post-test or make a level gain, you may wish to examine further the instructional strategies, curriculum, and professional development needs of staff serving those learners. If the average contact hours are high, but level gains in that EFL are low, once again, a review of assesssment, instruction and curriculum may reveal specific professional development needs for your program. 

> Barbara Hofmeyer 

> Coaching Consultant for 

> Indiana DOE, Division of Adult Education 

> 

> On Mon, 16 Apr 2007 13:25:04 -0600, Barbara Arguedas wrote 

> > Hello Tina, 

> > I wonder if you could say more about what exact data you use to see what your teachers need.  Is it attendance or level gain, or something else?  

> >   

> > We use attendance data to track total hours students are attending and to determine if they are eligible to post-test (minimum of 40 hours in NM).  If students attend at least 75% of the potential hours, then the student is eligible to get a certificate at the end of the session (usually a 12-week session that meets 5 hours each week).  We also see what the overall retention rate is by teacher as well as the post-test rate and the level gain rate.  I agree that these could indicate a need for training. 

> >   

> > We are interested in other ways that programs use data to help with retention.  Thank you. 

> >   

> > Barbara Arguedas 

> > Santa Fe Community College 

> > Santa Fe, NM 

> >   

> > -----Original Message----- 

> > From: assessment-bounces at nifl.gov [mailto:assessment-bounces at nifl.gov] On Behalf Of Tina_Luffman at yc.edu 

> > Sent: Monday, April 16, 2007 12:41 PM 

> > To: The Assessment Discussion List 

> > Subject: [Assessment 715] Re: Using Data for Program Improvementbegins today! 

> >   

> > 

> > Hi there, 

> > 

> >   I like to use state database information to show me which teachers are needing assistance and which teachers are modeling good practices. The database is certainly not a final word, as we all have had groups of students that performed well or poorly regardless of instruction.  The data is a good place to show red flags, however. Student assessments and staff self-assessments are also great for predicting professional development needs. Data can also help us see what people groups we are reaching with advertising, and which people groups are not. Then we can create new means of recruitment for our program. 

> > 

> > Tina 

> > 

> > 

> > Tina Luffman 

> > Coordinator, Developmental Education 

> > Verde Valley Campus 

> > 928-634-6544 

> > tina_luffman at yc.edu 

> > 

> >



> > "Condelli, Larry" <LCondelli at air.org> 

> > Sent by: assessment-bounces at nifl.gov 

> > 04/16/2007 09:15 AM



> > Please respond to 

> > The Assessment Discussion List <assessment at nifl.gov>



> >



> > To  

> > "The Assessment Discussion List" <assessment at nifl.gov>



> > cc  

> >  



> > Subject  

> > [Assessment 713] Re: Using Data for Program Improvement begins         today!



> >  



> >    

> >  



> >



> > 

> > 

> > Hi Everyone, 

> >   

> > It is a pleasure to be a guest on the list this week and my thanks to Marie for asking me and organizing this. 

> >   

> > There is a strong federal initiative to promote use of data for program improvement at the state and level.  Through the National Reporting system project which I direct, we have conducted several training and technical assistance activities over the past 4 years on this topic, including two general training seminars on using data and more specific ones on promoting adult education programs, monitoring, developing state and local report cards. All of the training materials and other information on the topic, including sample work from states, is available on the NRSWeb website, which Marie has referenced. 

> >   

> > All of the other guests have done a great deal of interesting work and many of them attended our training (and Sandy Strunk served as a trainer for us a few years back). 

> >   

> > I will be interested to get your questions and learn of your experiences, as well as the responses from the other guests. 

> > 

-----------------------------------------------------------------------


> > 

> > From: assessment-bounces at nifl.gov [mailto:assessment-bounces at nifl.gov] On Behalf Of Marie Cora 

> > Sent: Monday, April 16, 2007 7:29 AM 

> > To: Assessment at nifl.gov 

> > Subject: [Assessment 710] Using Data for Program Improvement begins today! 

> > 

> > Good morning, afternoon and evening to you all.   

> >   

> > Today begins our discussion on Using Data for Program Improvement.  I have pasted the announcement below – please note that there have been some edits to Guest bios.  Also, I am trying to send 4 attachments (they are power points) but I’m having a tough time getting them through the server.  For now, you have the announcement below and as soon as I arrange access to the Power Points, I’ll let you know.  If you received the original announcement that I sent, you have one of the attachments already (“Using Data Effectively DCornellier”).  Thank you for your patience with this. 

> >   

> > Also, I would like to acknowledge that today is Patriot’s Day and is celebrated in many corners of the United States.  Some of our guests, as well as subscribers, may not be present on-line with us today and this is fine – they will catch up with us tomorrow.  For anyone ready to begin, please feel free to post your messages. 

> >   

> > I’ll start us off by asking our guests to briefly describe how they use data in their work to improve literacy services.  Subscribers, please post your questions and share your own experiences using data.  What type of data would you like to track and why? 

> >   

> > Thanks! 

> >   

> > Marie Cora 

> > Assessment Discussion List Moderator 

> >   ------------------------------- 

> > National Institute for Literacy 

> > Assessment mailing list 

> > Assessment at nifl.gov 

> > To unsubscribe or change your subscription settings, please go to http://www.nifl.gov/mailman/listinfo/assessment 

> 

> -- 

> Open WebMail Project (http://openwebmail.org)


-- 
Open WebMail Project (http://openwebmail.org)

 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.nifl.gov/pipermail/assessment/attachments/20070417/35f8013e/attachment.html