since my colleagues and I are in the throws of developing a computer-driven adult assessment tool in reading, I'd like to learn more about the MAPT to avoid redundancy. We have a prototype developed and are moving forward with item development, but we are a long way from having a tool ready for field testing. When we get that far, I trust there will be practitioners out there willing to assist us in standardization. Yes, standardization is essential even for tests used to measure specific progress. barb given

Barbara K. Given, Ph.D.
Director, Adolescent and Adult Learning Research Center
Krasnow Institute for Advanced Study, and
Director, Center for Honoring Individual Learning Diversity, an International Learning Styles Center 
George Mason University
Fairfax, VA 22030-4444
Fax: 703-993-4325
Ph:  703-993-4406


----- Original Message -----
From: Marie Cora <marie.cora at hotspurpartners.com>
Date: Friday, June 30, 2006 9:02 am
Subject: [Assessment 397]  FW: [AAACE-NLA] testing


> Dear Colleagues,

> 

> I thought you would find this email exchange of interest.  The 

> followingresponse to both Janet's and Hal's posts comes from Bob 

> Bickerton.(Read from the bottom up to get the thread in the 

> correct order!).

> 

> Thoughts?  Comments?  Is your state satisfied with the elements that

> make up its accountability system?  Is anyone else developing 

> their own

> assessments?  How do we hold onto the good purpose of standardization

> ("level the playing field") and improve how we (or test publishers)

> understand the scope of content as well as test design?  Do you find

> that our present landscape of tests and testing "adds value" or not?

> What do you think?

> 

> Marie Cora

> Assessment Discussion List Moderator

> ______________________________________

> 

> 

> Dear Colleagues,

> 

> Janet's question is key as are the observations from Hal Beder's 

> study.There is an aspect of this conversation that I hope will be 

> explored and

> vetted on this list and that is:  Under what circumstances does 

> testinghave/add value, to what extent, and what other measures 

> must accompany

> the

> value that testing (may) bring?

> 

> My concern is that the volume of criticism of standardized testing 

> leadssome (many?) to a conclusion that there is no such thing as 

> "addedvalue" by

> such testing.  I disagree.

> 

> Much of the criticism of standardized testing is right on target.  

> Manysuch

> tests are poorly designed.  Perhaps even worse, many standardized 

> tests,whether well or poorly designed, are used for purposes for 

> which they

> were

> not designed and are not appropriate for.  Complicating this second

> point is

> that many test marketers are all to ready to blur the line between 

> whatis

> and is not an appropriate use of the test -- for them, testing ethics

> come

> second after profits.

> 

> When I started as an adult educator (some 35 years ago) I was an 

> adamantopponent of standardized testing -- primarily based on what 

> I had heard

> in

> the media, re, test bias -- particularly when used on African American

> and

> other "minority" populations.  There was already some pressure 

> back then

> to

> use standardized tests so I decided the learn everything I could about

> the

> "enemy" so I could push back with "facts" rather than just my opinions

> and

> my emotional response.  While teaching (part time in 3 different

> programs) I

> completed four semesters of relatively advanced statistics and two

> courses

> in tests and measures.  What I learned is that it isn't as clear 

> cut as

> I

> had thought.  I have continued to read the research and study the pros

> and

> cons of standardized testing (I highly recommend reading the 

> "Standardsfor

> Educational and Psychological Testing," AERA, APA, NCME) and have

> concluded

> that it is possible to develop and conduct standardized testing that

> adds

> real value, but that it is extraordinarily difficult (and 

> expensive) to

> do

> so.  

> 

> In response to all the problems we were aware of with the TABE and 

> otherstandardized tests in Massachusetts, we entered into a 

> partnership with

> the

> REEP program in Arlington, VA to adapt their ESOL writing 

> assessment to

> our

> purposes, helped field test and ultimately adopted the BEST+ for ESOL

> oral

> assessment, and this July 1st we will be transitioning from using the

> TABE

> (we held our noses for 2 years) to a MAPT, a brand new ABE online test

> we

> developed in partnership with the U.Mass Center for Educational

> Assessment

> -- a portion of which is computer adaptive.  We've invested a lot of

> time

> and money (including hundreds of adult educators and thousands of

> students)

> to align this test with the learning standards in our curriculum

> frameworks

> (more than 5 years in the making and based on the work of hundreds of

> adult

> educators), to pilot the test getting reams of feedback from teachers

> and

> students including many face-to-face interviews and real time

> observations,

> and we believe the test we begin implementing in a few days WILL add

> value

> to the teaching and learning process as well as provide more valid,

> reliable, fair and legitimate data for our state's ABE performance

> accountability system.  I've copied Jane Schwerdtfeger because she's

> labored

> tirelessly in the office -- but even more in the field with our

> colleagues

> to help us reach this point.  Thank you Jane!

> 

> Each state will approach these issues differently -- and for the most

> part,

> this can be good.  What I believe we shouldn't do is:

> 1.  Accept the claims of test publishers.  It's the test users

> responsibility to determine how appropriate a test is for the intended

> use.

> 2.  Write off all standardized tests as bad and hope they'll just go

> away --

> or spend hours commiserating when they don't.  We can use our time

> better

> than that.

> 

> [NOTE:  the "Standards" referenced above took 10 years of debate among

> AERA,

> APA and NCME during the very time when criticisms of standardized

> testing

> were at their peak.  Many of those involved in the debate had very

> strong

> reservations about the quality of many standardized tests and how they

> were

> often misused.  The standards are designed to address these very

> issues.]

> 

> take care,

> bob bickerton, MA associate commissioner of education and come 7/1/06

> past

> chair of NCSDAE/NAEPDC (welcome back to the chair role to Israel 

> Mendozaof

> WA) and former MA state director of adult education (3 cheers for Anne

> Serino, MA SDAE!).  

> 

> -----Original Message-----

> From: aaace-nla-bounces at lists.literacytent.org

> [mailto:aaace-nla-bounces at lists.literacytent.org]On Behalf Of Janet

> Isserlis

> Sent: Wednesday, June 28, 2006 9:14 AM

> To: National Literacy Advocacy List sponsored by AAACE

> Subject: [AAACE-NLA] testing

> 

> 

> AND

> 

> what do the tests actually test?

> 

> Janet Isserlis

> 

> 

> > From: Jon Steinberg <jons at lacnyc.org>

> > Reply-To: National Literacy Advocacy List sponsored by AAACE

> >> Date: Tue, 27 Jun 2006 11:48:37 -0400

> > 

> > Re: Adult Literacy Assessment

> > Another NCSALL publication, "Lessons from NCSALL's Outcomes and

> Impacts

> > Study" by Hal Beder in FOB (http://www.ncsall.net/?id=386) 

> summarizesan

> > analysis of 17 studies of adult literacy programs. This article 

> notes> that although standardized tests showed little evidence of 

> progress,> most learners asserted that they had made significant 

> gains. Beder

> > offers various hypotheses that might explain this discrepancy. 

> At a

> > minimum, his analysis should make us wary of asserting that

> > disappointing test results accurately measure the effectiveness of

> adult

> > education programs even though the students in them are so convinced

> > they are learning that they attend class week after week, often

> despite

> > great obstacles. As Marx (Groucho) said, "Who are you going to

> believe,

> > me or your own eyes?"

> > 

> 

> _______________________________________________

> AAACE-NLA mailing list: AAACE-NLA at lists.literacytent.org

> http://lists.literacytent.org/mailman/listinfo/aaace-nla

> LiteracyTent: web hosting, news, community and goodies for literacy

> http://literacytent.org

> 

> -------------------------------

> National Institute for Literacy

> Assessment mailing list

> Assessment at nifl.gov

> To unsubscribe or change your subscription settings, please go to 

> http://www.nifl.gov/mailman/listinfo/assessment