SurveyMonkey has recently introduced three short tutorial videos that are open to anyone to view, without need for passwords or logging in. Their plan is to introduce more videos over the upcoming months. (These videos, by the way, are really nice examples of how Camtasia can be used)

These first three tutorials demonstrate the following areas of SurveyMonkey:

• Getting Started (the fundamentals and a tour of the website)
• Survey Design (the basics of designing a survey using SurveyMonkey)
• Using Collectors (how to collect responses; use of settings and restrictions)

A lot of what these videos cover will already be known to those who have used SurveyMonkey, but they can serve as a good resource for classes or for introducing newbie colleagues to the resource. I did learn some very useful items from the tutorials, though, such as:

• the difference between the “exactly one” and “at least one” settings for multiple-choice questions
• how to use rating questions to force respondents to rank their choices
• the bad news about what happens when respondents leave a SurveyMonkey survey uncompleted and return to it later

(…and that bad news is that the information about the partially-completed survey is cookie-dependent, stored by the web browser on the respondent’s computer. To go back and complete a survey, the respondent must be using the same browser on the same computer–and not have cleared the cookies!)

The goal of this study is to develop a research plan and proposal to measure the value of the health sciences library, library services, and the librarian. The idea for a study on the value of libraries came from a discussion of the MAR’s RAC meeting held at New York University Medical Center on November 17, 2006. Out of that discussion emerged the issue of establishing the value of health sciences libraries to administrators in hospitals, academic health sciences centers, and other organizations where librarians work.

One frequently cited study in the RAC discussion was the “Rochester study”.  In discussing both the need for a value of libraries study, and the impact of the Rochester study - cited over 95 times since its publication in 1992 - the RAC came to the conclusion that before engaging in the research study itself considerable time, effort and thinking needed to go into what a research study would look like.

Objectives of the Value of Libraries Planning Study:

1. To inform the research plan by seeking input from health sciences libraries of various types and sizes and in various geographic locations within MAR
2. To determine the value domain(s) to be measured: patient care (information for providers, patients), education, research, administration, community health
3. To determine the library products or services to be evaluated, and to identify key activities that are directly related to the quality of those products or services.
4. To identify and outline the outcomes to be measured.
5. To develop a study design and timeframe for the study.
6. To determine MAR participants in the initial research study.
7. To develop a plan that is replicable, and to suggest methods for replication.
8. To determine likely sources of funding for the research study.

Please visit the web site for further details on the study, a listing of committee members, and the project timeline,  http://nnlm.gov/mar/about/value.html

Shaping Outcomes, an online course on outcomes-based planning and evaluation, will be available free to museum and library professionals this summer and fall. The instructor-mediated course, which will help participants improve program designs and evaluations, was developed through a cooperative agreement between the Institute of Museum and Library Services (IMLS) and Indiana University-Purdue University, Indianapolis (IUPUI). Through the approximately five-week course, participants will work at their own pace to learn outcomes-based planning and evaluation concepts and apply the concepts to a program or a project at their own institutions. A special course for those interested in teaching Shaping Outcomes or incorporating it into their own curricula will be offered in October 2007. Those interested in learning more about Shaping Outcomes or registering for one of the courses should visit http://www.shapingoutcomes.org or e-mail outcomes@iupui.edu.

For those of you interested in attending the 2007 American Evaluation Association Conference in Baltimore (November 7-10), information is now online at the AEA conference Web site.  I also recommend AEA’s pre- and post-conference professional development workshops, which are offered Nov. 5-7 and Nov. 11. Workshop descriptions and cost information are available here.  Many of these workshops have been taught by the same instructors year after year and have received high evaluations. The quality can vary, however — so maybe some of you who have previously attended the AEA conference can add their comments about the workshops they liked.

Because so many RMLs conduct online surveys, I thought I would mention a couple of books recently recommended in discussions at EvalTalk, the American Evaluation Association’s listserv. The first is Don Dillman’s book “Mail and Internet Surveys” (updated 2007, Sage). Don Dillman is probably the best known survey researcher – that is, he researches the best ways to conduct survey research. I have used this particular book and found it to be very useful, although I have not seen this edition. The second book is by Sue and Ritter called “Conducting Online Surveys” (2007, Sage). One listserv member said he liked the book because (among other things) it addressed issues related to response rates – a concern that most of us have regarding online surveys. If you are a member of AEA, you may get similar information from an upcoming issue of New Directions in Evaluation which deals directly with use of online surveys in evaluation. This issue is edited by Sue and Ritter and is due out in September. New Directions in Evaluation is free to AEA members.

Apparently Survey Monkey has undergone a major re-design. You might want to take a look before you need to use it again.

In my quick tour around the updated site, I noticed the list manager feature is much improved. It stores your recipient list and messages with the survey, so there is less opportunity for error (like sending your survey to someone else’s list!). You now create and store your recipient list with the survey and access it under the collect button: just hit the collect button for a specific survey, and you’ll find the distribution list for that survey. (You create your distribution list there too.)

The other major change I discovered is that inactive surveys are archived and it can take up to 24 hours to restore data from archived surveys. (I couldn’t find any notations about how long surveys are inactive before they are archived.) Only the data takes time to restore, though. You do not have to restore questionnaires if you want to copy them for the basis of a new survey. When you hit the “create survey” button, a list appears of the surveys in the account that can be copied and archived surveys are included in that list.

It would be great if others using Survey Monkey could give us all a heads-up on changes you find (for better or worse) by inserting comments to this post. Instructions for adding comments can be found by clicking on “How to Participate” under “About OERC Blog” to the left.

The Joint Commission (the health care organizations’ accrediting agency formerly known as JCAHO), has launched a public policy initiative to address broad issues that may undermine patient safety and quality of health care. It has published two papers that address important concerns related to patient safety and quality of care: linguistic and cultural diversity of patients; and low health literacy. A recent publication titled Hospitals, Language, and Culture: Snapshot of a Nation. A Report of Findings presents findings from a qualitative study of how 60 hospitals dealt with linguistic and cultural diversity. A white paper, “What did the Doctor Say? Improving Health Literacy to Improve Patient Safety” addresses the problem of health literacy and strategies for meeting patients’ communication needs.

The Joint Commission recognizes that the problems rising from diversity and low health literacy must be resolved through engagement of “multiple publics.” And medical librarians represent one “public” that should be engaged. Yet neither publication specifically mentions librarians, who could play a key role in providing access to information needed to support the strategies suggested in the publication. It appears that librarians may have to advocate for their role.

Fortunately, the MLA’s 2006-2007 priorities include a broad media effort to inform health care administrators of “librarians’ value (ROI) in providing consumer health information and patient education that improves patient safety, welfare and empowerment.” Another MLA priority — research defining the relationship between consumer health information and health information literacy on patient outcomes, safety and health care costs — will provide an “evidence-based” foundation for advocacy. The Joint Commission’s public policy initiative (and the accreditation standards that result from it) could pave the way for collaboration between health care organizations facing the challenges of caring for a diverse patient population and medical librarians who can support their efforts through access to much-needed information.

MLA News is running a two-part series about conducting needs assessments this spring. The first article, Needs Assessment for a Consumer Health Library Service, Part One: The Planning Process appears in the March 2007 issue. The second part of the series, Needs Assessment for a Consumer Health Library Service, Part Two: Implementation and Analysis, will appear in the May 2007 issue of MLA News. Part One focuses on planning a needs assessment. It lists some good examples of methods – surveys, key informant interviews, etc. It also gives examples of the different types of stakeholders and audiences who might be interviewed or surveyed as well as examples of existing information (publications, databases) that are useful sources of data in needs assessments. The article provides a lot of ideas in a small amount of news space.

The Aspen Institute Roundtable on Comprehensive Community Initiatives has a Measures for Community Success database with descriptions of over 100 evaluation tools and methods for evaluating community initiatives. Many of these tools have been designed for very specific types of initiatives, such as child development or substance abuse prevention. However, the resources are useful for getting ideas for community-based indicators or for evaluation of collaborations. There are some entries that appear to have more generic applications, such as the Collaboration Index and Collaborative Assessment of Capacity. (Note: I could not find these online, so they may be more context-specific than their descriptions indicate.) Some of these entries will lead you to names of experts who might have articles of interest to you.

This is mainly a database with descriptions. If you find something of interest, you will have to dig for more information, by either doing a literature search on the instrument or the author’s name or calling the contact information listed in each entry. However, it seems to be a rich resource for learning how to work with community partners.

I attended a session at the 2006 American Evaluation Association conference about building “evaluation capacity” in organizations and one presenter talked about “learning circles” as a way to teach evaluation methods. Others from NN/LM attended the session and confirmed to Susan and I that learning circles would be a worthwhile activity for the OERC to offer. (more…)