Yes, qualitative is time and labor intensive. However, it seems to me to be to best method for acquiring reliable data of depth in this case. Here is why I believe this. There are many factors that must be involved in following through with PD, not the least of which are interest and personal vision. Why were they in attendance in the first place is a factor. Did they need the hours for recertification? Were they truly interested in the subject matter. Was it a conference and the session they really wanted was filled and this was a second or third choice.? Some deal with the participant's attitude toward the PD, the delivery method of the PD, the subject matter of the PD, how much time is available to root around in the PD material and find what is to be put into action. Even the quality of that action, or lack of quality, is a product of a sound action plan. If you have ever facilitated a PD session you're aware of the deviation in evaluation responses. Some folks think it's great or very helpful, probably most, but some will not like it and will forget about it as soon as they walk out the door. Heck, even many of those who thought it was great will lose the packet of info on their way home. You know all of this, of course, but I have to go through the ABCs in my mind as I address the issue (one of the penalties of getting older.) Yes, to me it begs qualitative. Once the plan is in place, the instrumentation and technique determined, and the measurements taken, things get a little easier. I like SPSS software for qualitative measurement collection, but realize this is a subjective thing.
Jim


From: professionaldevelopment-bounces at nifl.gov [mailto:professionaldevelopment-bounces at nifl.gov] On Behalf Of Deborah Stedman
Sent: Wednesday, October 08, 2008 8:55 AM
To: 'The Adult Literacy Professional Development Discussion List'
Subject: [ProfessionalDevelopment 2585] Re: 'Currents' in PD

Steve and Jackie,

I really appreciate this question.  The Texas Family Literacy Resource Center has always struggled with these issues as well.  The current "accountability" environment values standardized assessment; however, there are none that I know of that measure professional development learning.  Changes in the organization/program and practitioner behavior can be assessed through qualitative data from interviews and surveys and such but the trick, obviously, is crafting the right questions.  Qualitative evaluation (especially the analysis) is mighty labor intensive and expensive.

Eventual learner gains is another whole issue and would require a controlled study to attribute gains or the lack of gains directly to professional development - an evaluation strategy that most professional development organizations do not have resources for.  I'm hoping for some responses to Steve's question that are "outside the box" thoughts and ideas.

Deborah




Deborah Stedman, Ph.D.

Grant Director, Texas Family Literacy Resource Center

Lecturer, Developmental and Adult Education
Texas State University
601 University Drive

San Marcos, TX 78666

Phone: 512-245-9757

fax: 512-245-8151

E-mail: ds43 at txstate.edu<mailto:ds43 at txstate.edu>

Website: http://www.tei.education.txstate.edu/famlit





________________________________
From: professionaldevelopment-bounces at nifl.gov [mailto:professionaldevelopment-bounces at nifl.gov] On Behalf Of steve.reuys
Sent: Wednesday, October 08, 2008 9:36 AM
To: The Adult Literacy Professional Development Discussion List
Subject: [ProfessionalDevelopment 2584] Re: 'Currents' in PD

Jackie: I would be interested in exploring the question of how to evaluate or assess the impact or outcomes of professional development activities that we provide. Using the five levels of evaluation delineated by Thomas Guskey, we are usually able to evaluate the first two--participant satisfaction with an activity and participant learning--but the other three--change in the organization/program, change in practitioner behavior, and eventual learner gains--are much more difficult to assess in adult basic education. It would be interesting to see if others doing professional development in our field are wrestling with the same issues and anyone has come up with any useful approaches or strategies for doing this.

Steve Reuys
Director, A.L.R.I./Boston SABES
University of Massachusetts Boston
Wheatley Bldg., 4th fl., rm 04-167
100 Morrissey Blvd.
Boston, MA 02125-3393
617-287-4071
steve.reuys at umb.edu<mailto:steve.reuys at umb.edu>


On Oct 6, 2008, at 7:16 PM, Jackie A. Taylor wrote:

Professional Development Colleagues:
Sometimes, we might find ourselves "caught" in a daily work regimen or it is challenging to find time to participate in this online community. Good questions about PD confront us but we have little time to investigate them. (You might even feel a little guilty about it!)
But here's an idea -- what if we combined efforts by intentionally exploring a PD question or two a month together -- where, for example, a question about professional development would be posted for discussion every-other Monday?
The question would come from you -- something you are currently involved in or grappling with in providing professional development. You could post the question or send it to me for posting. Questions might cover, for example:

 *   Helping administrators build instructional leadership skills
 *   Providing effective PD around college transitions
 *   Identifying what's new and what's working in PD
 *   Showing return on investment in professional development
 *   Sharing Web 2.0 tools that others use to support communities of practice
 *   Extending practitioners' learning experiences beyond 'one shot' workshops
 *   Providing PD on demand for volunteer tutors
 *   Being an effective... facilitator...teacher...trainer...PD liaison, etc.

This would not preclude anyone from being able to post a question anytime; rather, it would be a new way we could focus our energy to advance our work. Responses could be compiled in the PD area of the ALE Wiki.

What do you think?

If you like it, what's a question we might explore together?

jackie

Jackie Taylor, PD List Moderator, jackie at jataylor.net<mailto:jackie at jataylor.net>
----------------------------------------------------
National Institute for Literacy
Adult Literacy Professional Development mailing list
professionaldevelopment at nifl.gov<mailto:professionaldevelopment at nifl.gov>

To unsubscribe or change your subscription settings, please go to http://www.nifl.gov/mailman/listinfo/professionaldevelopment

Professional Development section of the Adult Literacy Education Wiki
http://wiki.literacytent.org/index.php/Adult_Literacy_Professional_Development

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.nifl.gov/pipermail/professionaldevelopment/attachments/20081009/329e8713/attachment.html