Return-Path: <nifl-health@literacy.nifl.gov> Received: from literacy (localhost [127.0.0.1]) by literacy.nifl.gov (8.10.2/8.10.2) with SMTP id j28FgmC20608; Tue, 8 Mar 2005 10:42:52 -0500 (EST) Date: Tue, 8 Mar 2005 10:42:52 -0500 (EST) Message-Id: <5.2.0.9.2.20050308102649.0118f9c8@hsph.harvard.edu> Errors-To: listowner@literacy.nifl.gov Reply-To: nifl-health@literacy.nifl.gov Originator: nifl-health@literacy.nifl.gov Sender: nifl-health@literacy.nifl.gov Precedence: bulk From: Rima Rudd <rrudd@hsph.harvard.edu> To: Multiple recipients of list <nifl-health@literacy.nifl.gov> Subject: [NIFL-HEALTH:4685] RE: Research on the use of X-Listprocessor-Version: 6.0c -- ListProcessor by Anastasios Kotsikonas Content-Type: text/plain; charset="us-ascii"; format=flowed X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9 Status: RO Content-Length: 15086 Lines: 368 Hello, While I agree with Bill's point that rigorous formative research is an absolute professional obligation, there are additional evaluative questions that can be asked and examined. This is true for the use of any materials that 'deliver' a message, present issues and information, and support a wider program effort. First, it is critically important that materials or a message is not confused with a program effort [if passing out material constitutes the entire effort than we have a very weak 'program']. The formative evaluation includes a thorough evaluation by members of the intended audience so that the materials will be revised as needed. A process evaluation will look at how people use the materials. Here are some examples of process evaluation research questions: Are the materials being distributed as planned. Are they getting into the hands/homes of the people for whom it was intended? Are they being used as planned? Is there expanded use? For example, are people sharing the materials and information with others? Is the material and information sparking discussions with others [friends, family, doctors???]. Are the materials being used more than previous materials were used and/or in a different way? How does the wider audience respond to the materials and the messages? Assuming the development of materials was local and participatory, how do those in other areas and who do not know the people involved respond to the materials and message? One can also look at increased efficacy, increased knowledge..... etc. An outcome evaluation will look at the power of the intervention to bring about desired change. One might also look at the specific contribution of material X to enhance the program, increase the likelihood of change, bring about actual change over and above the program without the added materials or the program with other standard materials. You will see in articles by Roter, Rudd, Comings.... some discussion of evaluations and evaluation findings. An added note: we found that locally produced materials, developed by and with local people, with familiar places and faces -- might indeed be more salient and effective than professional looking, expert derived materials. Rima Rudd 50 AM 3/8/2005 -0500, you wrote: >Evaluating fotonovelas. What would you evaluate and how would you do it? > FotoN are like radio programs - they are a channel of communication. >Some radio programs fail and others succeed, but radio doesn't succeed >or fail. I have never seen anyone waste money on a case control study >of fotonovelas, whioch be the gold standard. You would have to take one >fotonovella, compare it against another fotonovel and then compare it >against some other alternative format to determine what?....in a given >population at a given moment fotonovella A was better(read more widely, >understood?) than B or not as good as alternative C? What would that >tell you that could generalize to different audience, subjects and >times? > >I will check some international sources that do research on >entertainment media - but I am very skeptical of finding any evaluation >of fotonovells as stand alone, or as simple evaluations that would not >really show you much becasue they were so case specific. > >What is it that you wopuld want an evaluation to tell you? > >I think what we know about communication suggests that your focus on >formative research makes a great deal of sense for a channel like this. >Obiviously a program would need to demonstrate that the reach and >frequency of a footonovella was sufficient for a given population. In >a given setting who reads fotonovelas? How often? What do they get out >of them? And most important what part are they playing in a much >broader strategy to accomplish a public health objective. > >Once you decide they are a viable channel for a specific audience - >what content do you put through it (what are people going to learn from >a fotonovel - facts, positioning, branding ?) > >We did a series of fotonovels in Ecuador to help fight the racism >against rural Indians. The novel featured a super hero Indian who >solved real problems with great street smarts. It was critical that the >problems were perceived to be real and that he was not fighting aliens >from outer space. Manual Santi was his name and he had to solve things >like water rights problems to be credible. > >Manual Santi became a symbol of the smart savvy Indian that we used in >a dozen other channels to address the existing stereotypes. We used >photos and the lead character made personal appearances around the >country. > >In another case we needed to get very specific information about how to >mix a rehydration solution to mothers of newborns. There, the accuracy >of the information was important and we knew that just reading the >fotonovel would not be enough. We designed it as a school project. >Kids got the fotonovel in school, read it in class and then took it >home. It was the story of how a school kid had saved his baby sister's >life by mixing oral rehydration properly. The kids took the fotonovel >home---read it to his parents as a school assignment -- and had mom >answer four test questions about ORS which the kid brought back to >school as homework. The fotonovel became a reminder in the home of the >mixing formula. The plot line was dramatic - with the child becoming >more and more dehydrated etc. > >This was part of a much larger program that reduce infant mortality due >to diarrhea by some 37% I think. But you could not say the fotonovel >was the only reason. It was part of a broader strategy. > >The key evaluation issues are really formative. In designing a >fotonovella you test whether it is compelling - whether the story is >clear and NOT considered "learning". If they think it is a textbook it >will loose its unique power. You want to test the drawings individually >- actually we use photos in both caseas above because we found drawings >too difficult to identify for our audience. We were targeting women age >18-35. Young boys loved the drawings better, but they were not our >audience. This was an audience specific finding, not realy >generalizable. > >There is a large literature on entertainment media and social change. >The key motion ios to embed call for action and information within an >entertainment format. My experience is that too many people test only >whether people "liked it" and not did it have the intended behavioral >effect. > >We have so little dollars for evaluation I always try to suggest using >it during the format process. > >What I think experience shows is that fotonolas can be an effective >channel for communication on topics that have an emotional appeal. They >can be targeted to specific audiences through their distribution >channels. We used traveling herbalists to sell our fotonovelas because >that's where our mothers bought things like that. Putting them in >stores would have gone no where. > >The final thing I will share is that the production quality of a >fotonovel matters. We tested various formats with our audiences and >found that unless it looked professional it was not taken seriously. >Again this was all part of the formative testing. > >I know this does not answer the science question....do fotonvoleas >work? But then do we have an answer to the question...does surgery >work?....not to cure a cold it doesn't and not if it is done by a >plummer. >Surgery is a procedure with practice standards and fotonovelas are a >channel with practice standards. Has anyone ever submitted surgery to a >case control trial? Why not? Don't we need to know if it works? >Even in the face of documented medical errors we don't submit surgery >to case control trials. > > >>> CBaur@OSOPHS.DHHS.GOV 3/7/2005 3:31:26 PM >>> >Bill, > >Bill, > >Would you mind posting some specific references to evaluation studies >of >fotonovelas? The original inquiry related to efficacy. The listserv >responses have focused on formative research and production processes. >The >Healthy People 2010 objective 11-3 encourages both formative research, >which >is what most people have discussed, and evaluation research. > >Cynthia Baur >HHS > >-----Original Message----- >From: nifl-health@nifl.gov [mailto:nifl-health@nifl.gov] On Behalf Of >William Smith >Sent: Monday, March 07, 2005 3:20 PM >To: Multiple recipients of list >Subject: [NIFL-HEALTH:4680] RE: Research on the use of Photonovellas? > >I spent a good deal of my early life developing photonovels in Latin >America. > >I'd be glad to offer that experience if people want specific guidence. >I'm pretty fluent in Spanish if that helps. > >Wm. Smith >Executive Vice President >Academy for Educational Development >1825 Connecticut Ave., NW >Washington, D.C. 20009 > >A little change hurts a lot. >A lot of change doesn't hurt that much more. >Phone: 202-884-8750 >Fax: 202-884-8752 >e-mail: bsmith@aed.org > >Wm. Smith >Executive Vice President >Academy for Educational Development >1825 Connecticut Ave., NW >Washington, D.C. 20009 > >A little change hurts a lot. >A lot of change doesn't hurt that much more. >Phone: 202-884-8750 >Fax: 202-884-8752 >e-mail: bsmith@aed.org > > >>> rrudd@hsph.harvard.edu 3/7/2005 12:30:23 PM >>> >Hello, >It is good to see discussions about this topic. I'd like to make a >brief note >about photonovels. As you all now, this form of wrting material is >very > >popular in Central and South America. The format is set like a comic >book. >However, photos are used instead of comics. The story line is carried >by >dialogue bubbles and aided by a brief note on top of a new page [e.g. >the next >day...]. The dialogue bubbles must be very brief and this helps with >literacy >related issues. >However, as you examine a wide variety of photonovels developed to >address an >array of health issues..... do look into the development process. In >some >cases the photonovel was developed by experts. It will be important >for >you to >consider the rigor with which it was developed, piloted, and revised >by > >members of the intended audience. In some cases the photonovel was >developed >by members of the intended audience through a process facilitated by >an > >outside or inside expert. Here too, be sure to consider the rigor of >the >process. >Debra Roter, John Comings and I [Rudd] wrote about this participatory >approach >and indicated the importance of authentic voice. >Also -- for those of you thinking about developing such material, you >will >find a how-to approach on the following website: >www.hsph.harvard.edu/healthliteracy. >We've found this to be a rich experience, a lot fun, and a worthwhile >endeavor. Participatory materials development enables members of the >intended >audience to share a story of their own about critical topics. The >power >of >this story, told visually and in words that are the participants', >captures >the attention of readers and most often speaks to them in 'plain' >everyday >terms. >Rima Rudd > > > >===== Original Message From nifl-health@nifl.gov ===== > >Hi- > >I just started working on an article related to our use of the >photonovels > >we developed for prenatal care. (We've discussed the De Madre A >Madre/From > >Mother to Mother bilingual photonovels on this listserv previously.) > >Although the article is not published yet, you might be interested in >our > >work. > > > >After developing the photonovels (a hybrid design- combining a >traditional > >photonovel format with a literacy component) we received several >small > >grants to develop a method to teach prenatal education classes using >the > >photonovels. We've developed a simple 6 step process. Over a period >of > >about 4 years, we have tested the 'Teach-With-Stories' (TWS) method >in > >several sites across North Carolina with pregnant Latino women with >low > >literacy skills in English and/or Spanish. The facilitators also had > >different skill and literacy levels. Each class they would read and >discuss > >a story together. Group members took the photonovels home. They >reported > >keeping them, rereading and sharing them with friends and other >family > >members. > > > >The TWS method addresses health literacy, health empowerment, and >health > >education needs. While our focus was on refining the method and >learning how > >to train facilitators in the process, we did some informal >qualitative > >evaluation. The results and stories were fairly consistent across >sites. > >Many of our findings are similar to the published findings in studies >on the > >efficacy of patient-centered care. Both clients and clinic staff >benefit. > >Note: A great summary of these studies is in the Institute of >Medicine's > >Health Professions Education: A Bridge to Quality report (2003). > > > >At the core of the TWS method is a 'power-sharing' way of relating >and > >educating (i.e., educator as facilitator rather than expert). In >addition, > >we have tried to build on your work, Dr. Rudd's work and the work of >Rick > >Arnold (who developed the 'Spiral Model'- an educational empowerment > >process). > > > >Just recently, the TWS method was selected as a national >demonstration > >project and will be more formally evaluated. We are looking at how >to > >evaluate its impact on the group members' health literacy (and also >the > >impact on the group itself). We are especially interested in how to > >evaluate change in levels/types of social support and how it relates >to > >health literacy of individuals and the group.) If anyone has any > >suggestions about this, please let me know! > > > >If you would like additional information, feel free to contact me >directly. > > > >Hope this helpful- > > > >Susan Auger > >Executive Director > >Auger Communications/Aprendo Press > >sauger@mindspring.com > >tel: 919-361-1857 > > > >-----Original Message----- > >From: nifl-health@nifl.gov [mailto:nifl-health@nifl.gov] On Behalf Of > >Lendoak@aol.com > >Sent: Monday, February 21, 2005 12:05 PM > >To: Multiple recipients of list > >Subject: [NIFL-HEALTH:4657] Research on the use of Photonovellas? > > > >Several of us are preparing a paper on the efficacy of visuals in >patient > >education. Our question is: What research or practice has been >published > >on the > >use of Photonovellas? > > > >We would appreciate any information. > > > >Len and Ceci Doak > >Patient Learning Assoc. > >4 Chilham Ct. > >Potomac, Md. 20854 > >Rima E. Rudd, MSPH, Sc.D. >Department of Society, Human Development,and Health >Harvard School of Public Health >677 Huntington Avenue >Boston, MA 02115 >Phone: 617-432-1135 >fax: 617-432-3123 >Note new listings on our Health Literacy Web Site: >www.hsph.harvard.edu/healthliteracy Rima E. Rudd, MSPH, Sc.D. Department of Society, Human Development, and Health Harvard School of Public Health 677 Huntington Avenue Boston MA 02115 Phone: 617-432-1135 fax: 617-432-3123 Note the new items on our health literacy web page: www.hsph.harvard.edu/healthliteracy
This archive was generated by hypermail 2b30 : Mon Oct 31 2005 - 09:49:38 EST