A slightly nuanced way to express one of your possibilities I'd think, off
hand ...

People's lives are shaped by their personal and familial networks' literacy
so that only rarely do they encounter instances that challenge their skills.
Literacy is thus both an outcome and a determinant.

So these assessments, and those designing similar in the future, need to put
the responses of participants in the context of what the literacy loads in
participants' lives actually are in order to provide an honest expression of
the data, no?

This is always an issue with self-reporting in any context, so maybe we
should refrain from asking why people have this problem in their responses
and ask why the methodology of the assessment didn't account for it?

fwiw,

Andrew






On Tue, Feb 10, 2009 at 3:26 PM, John Benseman <
john.benseman at criticalinsight.co.nz> wrote:


>  Yes, some very interesting points here Tom. I think the discrepancy

> between people's self-assessments and assessed skills is a really important

> challenge for us. The debate has mainly been pitched in terms of the

> relative validity of each approach, but in many ways that is irrelevant. It

> seems to me that it is much more useful to think about why the discrepancy

> occurs:

>

>

>

> §         People's self-assessments are simply wrong (at least to some

> degree)

>

> §         Their everyday literacy needs only demand a low level of skill

> and so they rarely come across situations that 'show up' their skills – in

> terms of their lives, their skills are perfectly OK (as long as you don't go

> near things like computer manuals or tax forms)

>

> §         They know their skills are low, but they deliberately deny to

> themselves this reality

>

> §         They know their skills are low, but maintain a public front,

> making out them to be higher than they are (common with many people who

> approach literacy programmes)

>

>

>

> I look forward to reading the Murray report as there is a dearth of cost

> benefit literature to back the claims that we make for literacy. It is not

> the only justification, but an important one nonetheless – especially in the

> current world we live in. The workplace literacy project I am working on

> here in New Zealand is evaluating 15 companies' programmes and we hope to

> have ROIs done on about seven of these to complement the other evaluation

> material. We hope to have the first of the ROI studies by about the middle

> of this year.

>

>

>

> Regards, John

>

>

>

> John Benseman

>

>

>

> *   john.benseman at criticalinsight.co.nz

>

>

>

> *   64 9 627 4415  Cell  027 454 0683

>

>

>

> * 52a Bolton St, Blockhouse Bay, Auckland 0600, NZ

>

>

>

> Skype: boltonstbunch

>

>

>

>

>

> -----Original Message-----

> From: assessment-bounces at nifl.gov [mailto:assessment-bounces at nifl.gov] On

> Behalf Of tsticht at znet.com

> Sent: Wednesday, 11 February 2009 7:04 a.m.

> To: assessment at nifl.gov

> Subject: [Assessment 1775] How well do you read?

>

>

>

> February 10, 2009

>

>

>

> How Do You Know How Well You Read?

>

>

>

> Tom Sticht

>

> International Consultant in Adult Education

>

>

>

> In a recent report Murray et. al (2009) stated that based on their

>

> performance on the International  Adult Literacy Survey (IALS) and the

>

> Adult Literacy and Life Skills Survey (ALL), both standardized tests of

>

> literacy,  as many as 48 percent of Canadian adults lack the literacy

>

> skills  "thought to be needed to take full advantage of opportunities

>

> present in the global economy"(p. 3). After a discussion of a number of the

>

> positive relationships of literacy to economic and social advantages, the

>

> authors go on to say, "In view of these facts, one would expect that low

>

> skilled adults would be clamouring to raise their literacy levels…[but] The

>

> reality is not as expected. The overwhelming majority of Canadian adults

>

> with low literacy skills believe that their skills are adequate and see no

>

> need to invest in raising their literacy skill level"(p. 14).

>

>

>

> Looking for a way to explain the gap between the statistic that 48 percent

>

> of adults need literacy training according to their scores on standardized

>

> literacy tests, while the great majority (over 90 percent) think their

>

> literacy skills are adequate, the authors state,  "many adults have no way

>

> to judge the adequacy of their skill, a fact that causes them to

>

> under-invest [in literacy education]" (p. 3).

>

>

>

> This raises the question, how do any of us know how adequate our literacy

>

> skills are?

>

>

>

> The Cost/Benefit study of Murray et.al based its analysis on the idea that

>

> all 48 percent of Canadian adults should receive education to raise them up

>

> to an acceptable level of literacy as measured by the international adult

>

> literacy surveys (this is Level 3on these tests). This would bring 100

>

> percent of adults up to this standard. However, a 2006 poll of adults by

>

> the Canadian Council on Learning (CCL) found that Canadians thought that it

>

> would be acceptable if only 80% of adults could meet most everyday reading

>

> requirements. Further, the poll presented data indicating that Canadian

>

> adults thought the country had reached the 80% standard in most basic

>

> skills, including reading. The poll further indicated that 90% of Canadians

>

> thought that they had learned what they need to work and live well with

>

> others, 88% thought that they had learned what they need to think

>

> critically and formulate new ideas, 81% thought that they had learned what

>

> they need to enjoy their life, and 76% thought that they had learned what

>

> they need to succeed in their career.

>

>

>

> This raises the question of how we might judge ourselves and our fellow

>

> citizens with respect to the adequacy of our/their basic skills. What

>

> criteria would we consider? Would we compare ourselves to some standard,

>

> such as a TV news anchor reading the news? How do we know how literate we

>

> are, how well we think critically, etc.?

>

>

>

> If, as suggested by Murray et. al,  adults have no way to judge the

> adequacy

>

> of their skills, and if it is adults' self-perceived needs for education

>

> that determines whether or not they will enroll in literacy programs, then

>

> it might be useful to develop methods for assisting adults to judge the

>

> adequacy of their literacy skills, and perhaps their other skills as the

>

> CCL poll results suggest. This might involve the conduct of national

>

> assessments of adult's self-perceptions, what explains their

>

> self-perceptions, what information would help them better assess their

>

> literacy skills, and to then provide respondents who assess themselves as

>

> poor in literacy with information about how they might locate adult

>

> literacy programs. It might also be useful to determine other education

>

> needs or desires so that adult literacy education providers might better

>

> align their delivery system and offerings with adult educational needs.

>

> References

>

>

>

> Murray, T. et. al (2009). Addressing Canada's Literacy Challenge: A

>

> Cost/Benefit Analysis. DataAngel Policy Research Incorporated.

>

> http://www.library.nald.ca/research/cost_benefit/cost_benefit.pdf

>

>

>

> [Note: For more on comparisons of standardized test and self-report methods

>

> of assessing adult literacy skills use Google searches to find a review of

>

> adult skills assessment in Europe by Jens Henrik Haahr and Martin Eggert

>

> Hansen, November 2006. Also see Sticht  (2005) The New International Adult

>

> Literacy Survey (IALS): Does it meet the Challenges of Validity to the Old

>

> IALS? The Canadian Council on Learning poll can also be found using Google

>

> searches.]

>

>

>

> Tom Sticht  tsticht at aznet.net

>

>

>

> -------------------------------

>

> National Institute for Literacy

>

> Assessment mailing list

>

> Assessment at nifl.gov

>

> To unsubscribe or change your subscription settings, please go to

> http://www.nifl.gov/mailman/listinfo/assessment

>

> Email delivered to john.benseman at criticalinsight.co.nz

>

> -------------------------------

> National Institute for Literacy

> Assessment mailing list

> Assessment at nifl.gov

> To unsubscribe or change your subscription settings, please go to

> http://www.nifl.gov/mailman/listinfo/assessment

> Email delivered to andrew.pleasant at gmail.com

>

>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.nifl.gov/pipermail/assessment/attachments/20090210/cf53361f/attachment.html