Questions to Ask and Answer
Why Developing and Pretesting Messages and Materials Are Important
Steps in Developing and Pretesting Messages and Materials
Planning for Production, Distribution, Promotion, and Process Evaluation
Common Myths and Misconceptions About Materials Pretesting
Selected Readings

Questions to Ask and Answer

• What materials will fit our strategy, appeal to our intended audience, and adequately convey our message? How can we make the materials as effective as possible?
• Do we need to create new materials? What types?
• How do we develop culturally appropriate messages and materials?
• How do we develop effective materials for low-literacy intended audiences?
• How can we make sure the materials will be used?
• When and how should we pretest our materials?
• How can we keep pretesting costs down?
• What should we do with pretest results?
• How can we get the best results from creative and research professionals? From reviewers?

Why Developing and Pretesting Messages and Materials Are Important

Developing and pretesting messages and materials are important because they allow you to learn early in the program which messages will be most effective with the intended audiences. Knowing this will save your program time and money by ensuring that you do not go through the entire development process with an ineffective message. Positive results from pretesting can also give you early buy-in from your organization.

See Appendix B for descriptions of theories and models that suggest important audience factors to consider when creating messages so they are both acceptable and persuasive to the intended audience.

Steps in Developing and Pretesting Messages and Materials

In Stage 1, you created a communication strategy statement. In Stage 2, you will use the strategy you developed as a guide to:

1. Review existing materials
2. Develop and test message concepts
3. Decide what materials to develop
4. Develop messages and materials
5. Pretest messages and materials

1. Review Existing Materials

Message and materials development and production can be time-consuming and costly. Because this process is creative and has tangible results, it is frequently the key developmental step for a health communication program. Before you begin developing and producing new materials, however, determine whether creating them is necessary.

You may have discovered existing communication materials (booklets, leaflets, posters, public service announcements, videotapes) while gathering data to plan the program. If not, look now.You may find materials at the following sources:

• Health departments (in your state or other states)
• University or public libraries
• Voluntary organizations
• Health professional associations
• Community-based coalitions
• Clearinghouses, Web sites, and telephone information services relevant to the issue
• Materials produced by the National Institutes of Health, the Centers for Disease Control and Prevention, or other agencies in the U.S. Department of Health and Human Services
• Healthfinder®, the Federal gateway to health information, to identify relevant Federal clearinghouses and other Federal information sources ( www.healthfinder.gov)

See Appendix C for additional contact information for these suggested sources.

If you find materials related to the health issue, decide whether they are appropriate for your program, either as they are or with modifications. Using the communication strategy statement as a guide, consider the following questions:

• Are the messages accurate, current, complete, and relevant?
• Are the materials appropriate for the intended audience in format, style, cultural considerations, and readability level? If not, could they be modified to be appropriate?
• Are the materials likely to meet the communication objectives?

Pretesting, discussed later in Stage 2, can help you answer these questions. Check with the group that originally produced the materials to learn about:

• Results of any pretesting
• Effectiveness of the materials to date
• Whether the group has advice or recommendations related to your program’s needs

If you are considering using existing materials, also ask the original producer these questions:

• Are they available?
• Could your organization receive permission to use the materials? Modify them? Note: Materials produced by the Federal Government are not copyrighted and may be used freely.
• Are they affordable?
• How have they been used?
• How have they been received?

See Appendix A for a sample form that you can modify to help you conduct a materials review.You also may want to test promising materials with the intended audience (see step 2). If the materials prove to be inappropriate, you will have gained valuable information for modifying them or developing new materials.

2. Develop and Test Message Concepts

The communication strategy statement and the other planning you did in Stage 1 form the basis for developing message concepts. Message concepts are messages in rough form and represent ways of presenting the information to the intended audiences. These may include statements only or statements and visuals. Do not develop the actual messages at this point. (If you create two or more concepts for each message, you will be able to explore which alternative works best.)

In this step, you will learn about the components that go into developing and testing message concepts, including working with creative professionals, creating culturally appropriate communication concepts, choosing a type of appeal, and testing concepts.

To develop, pretest, and eventually produce messages and materials, assemble a team of creative professionals, market research experts, and others. See the Communication Research Methods section for tips on working with market research experts.

Working With Creative Professionals

Developing a communication campaign usually involves working with creative professionals, either within your organization or on a contract basis. In either case, managing the relationship effectively is critical to getting the creative materials you want:

• Get to know and feel comfortable with the people who will be working on the project. If you are considering
  a contract with an advertising agency, public relations firm, or consulting firm, interview the professionals who will staff your effort (not just the agency representatives who solicit your business) and review samples of their specific work (not just the agency’s). Write into the contract who will work on the project.
• Be a good client. Use the creative brief to lay out the communication strategy (developed in Stage 1) and make
  sure the team understands the brief and that it must be followed. Think about what you want before you discuss the assignment and show the creative team examples of other materials that worked well or didn't and explain why. If you say, "I don't know what I want; you're the creative one," you lose a valuable opportunity to give creative professionals the fundamental direction they want and need. This does not mean asking for a blue brochure; it means helping members of the creative staff understand the objectives and concerns and what you've learned about the intended audience so that they can use their expertise to suggest effective approaches. Discuss sensitive issues, key content points, and other aspects that you want to see conveyed in the messages and materials, based on your knowledge and expertise.
• Agree at the outset to what pretesting and approvals will be required, when they will occur, and how long they will take.
• Discuss the theoretical grounding of the communication effort and help creative professionals understand and apply health communication theory to messages and materials development. Brainstorm with them about how the theory might shape the messages and materials and evaluate works in progress with this perspective in mind.
• Involve the creative team in concept exploration and pretesting. Ask its members what questions they would like addressed and make sure they can observe (not participate in) concept exploration sessions. Listening to the intended audience can help them craft messages and materials that use language and ideas that the audience will identify with.
• Assess draft messages and materials against the creative brief and what you know about an intended audience member's point of view. If the intended audience is urban teens at high risk of pregnancy, and you are a middle-aged suburbanite, recognize that the materials most likely to be effective with the intended audience may not appeal to you at all.
• Trust the team's professional expertise, provided that the material is consistent with your program's strategy and the intended audience's culture. While you have a key role to play in ensuring the appropriateness and accuracy of substantive content and in maintaining the program's strategic focus, developing the team's insights and commitment will keep the team involved.

Developing Culturally Appropriate Communications

Culture encompasses the values, norms, symbols, ways of living, traditions, history, and institutions shared by a group of people. Culture affects how people perceive and respond to health messages and materials, and it is intertwined in health behaviors and attitudes. Often, an individual is influenced by more than one culture; for example, teenagers are influenced by their individual family cultures as well as the norms, values, and symbols that comprise teen culture in their locale.

To develop effective health communications, you must understand key aspects of the cultures influencing the intended audience and build that understanding into the communication strategy. Messages must take into account cultural norms in terms of what is asked (e.g., don't ask people to make a behavior change that would violate cultural norms), what benefit is promised in exchange (in some cultures, community is most important; in others, individual benefit is), and what image is portrayed. The symbols, metaphors, visuals (including clothing, jewelry, and hairstyles), types of actors, language, and music used in materials all convey culture.

While it is important to acknowledge and understand the cultures within an intended audience, developing separate messages and materials for each cultural group is not always necessary or even advisable. For example, when print materials for a state program for low-income people depicted people of only one race, some intended audience members who were of that race felt singled out and said the materials suggested that only members of their racial group were poor. Careful intended audience research can help your program identify messages and images that resonate across groups–or identify situations in which different messages or images are likely to work best.

According to a Center for Substance Abuse Prevention Technical Assistance Bulletin, culturally sensitive communications:

• Acknowledge culture as a predominant force in shaping behaviors, values, and institutions
• Understand and reflect the diversity within cultures. In designing messages that are culturally appropriate, the following dimensions are important:
• Primary cultural factors linked to race, ethnicity, language, nationality, and religion
• Secondary cultural factors linked to age, gender, sexual orientation, educational level, occupation, income level, and acculturation to mainstream
• Reflect and respect the attitudes and values of the intended audience; some examples of attitudes and values that are interrelated with culture include:
  • Whether the individual or the community is of primary importance
  • Accepted roles of men, women, and children
  • Preferred family structure (nuclear or extended)
  • Relative importance of folk wisdom, life experience, and value of common sense compared with formal education and advanced degrees
  • Ways that wealth is measured (material goods, personal relationships)
  • Relative value put on different age groups (youth versus elders)
  • Whether people are more comfortable with traditions or open to new ways
  • Favorite and forbidden foods
  • Manner of dress and adornment
  • Body language, particularly whether touching or proximity is permitted in specific situations
• Are based on concepts and materials developed for and with the involvement of the intended audience. (Substituting culturally specific images, spokespeople, language, or other executional detail is not sufficient unless the messages have been tested and found to resonate with the intended audience. Formative research with audience members takes on added importance when planners and designers have different cultural backgrounds than the intended audience does.)
• Refer to cultural groups using terms that members of the group prefer (e.g., many people resent the term "minority" or "nonwhite." Preferred terms are often based on nationality, such as Japanese or Lakota.)
• Use the language of the intended audience, carefully developed and tested with the involvement of the audience

Choosing the Type of Appeal

To capture the intended audience’s attention, you can scare people, tug at their hearts, make them laugh, make them feel good, or give them straight facts.What will work best? The answer generally depends on the intended audience’s preferences, what your program is asking people to do, and how you plan to use the appeal in asking them to do it.

Positive emotional appeals show the benefits intended audience members will gain when they take the action portrayed in the message. Research has shown that, in general, messages that present a major benefit but do not address any drawbacks tend to be most appropriate when intended audience members are already in favor of an idea or practice. In contrast, messages that present a major benefit and directly address any major drawbacks work best when people are not favorably predisposed.

Humorous appeals can work for simple messages, especially if most competing communication is not humorous. The humor should be appropriate for the health issue and convey the main message; otherwise, people tend to remember the joke but not the message. Also, humorous messages can become irritating if repeated too frequently.

Threat (or fear) appeals have been shown to be effective with two groups. Research has shown that such appeals tend to be more effective with "copers" (people who are not anxious by nature) and "sensation seekers" (certain youth), and when exposure to the message is voluntary (picking up a brochure rather than mandatory attendance at a substance abuse prevention program). Research has also shown that, to be effective, a threat appeal should include:

• A compelling threat of physical or social harm
• Evidence that the intended audience is personally vulnerable to the threat
• Solutions that are both easy to perform (i.e., intended audience members believe they have the ability to take the action) and effective (i.e., taking the action will eliminate the threat)

In general, however, the effectiveness of threat appeals is widely debated.

The most appropriate type of appeal may differ from this general guidance, depending upon gender, age, ethnicity, severity of the problem, and the intended audience's relationship to the problem. For more information, please consult the following sources under Selected Readings at the end of this section: Backer, Rogers, and Sopory (1992); Goldberg, Fishbein, and Middlestadt (1997); Kotler and Roberto (1989); Maibach and Parrott (1995); Palmgreen et al. (1995); Siegel and Doner (1998).

Concept Testing

Once you have defined intended audiences and communication strategies and have developed message concepts, testing the concepts with intended audiences can help you decide on message appeals (e.g., feararousing versus factual), spokespersons (e.g., a scientist, public official, or member of the intended audience), and language (determined by listening to research participants' language). Testing is especially important if the program deals with a new issue, because it will help you understand where the issue fits within the larger context of the intended audience's life and perceptions.

Concept testing will help save your program time and money because it will identify which messages work best with intended audiences. Use concept testing to identify:

• Which concept has the strongest appeal and potential for effect
• Confusing terms or concepts
• Language used by the intended audience
• Weaker concepts that should be eliminated
• New concepts

Concepts can be presented in a number of ways. The key is to convey the major characteristics of the appeal along with the action your program wants intended audience members to take and the benefit they will receive as a result. Focus groups or in-depth interviews are most appropriate for concept testing because they permit discovery of:

• How an intended audience thinks about an issue
• How its members react to different appeals or aspects of a message concept
• Why they react that way

Message concept tests often ask participants to rank a group of concepts from most to least compelling and then to explain their rankings. Participants then discuss benefits and problems associated with each concept. Health communicators often use a sentence or brief paragraph to describe a concept to participants. For example, the following are two "don’smoke" concepts for teens:

1. Smoking harms your appearance.
2. Cigarette advertisers have created a myth that smoking makes a person more attractive. They’re lying.

While both concepts address attractiveness, the first concept uses it as the focal point of a negative appeal (to avoid becoming less attractive, don’t smoke), whereas the second concept uses a factual approach and a different benefit—avoid being manipulated by the tobacco industry—designed to appeal to teens’ strong desire not to be manipulated.

In each of the concepts above, both the action the intended audience members should take and the benefit are implied, not stated. This approach works in situations where the desired behavior is obvious. In other situations, the behavior or the benefit will need to be mentioned, as in the following examples:

• Mammograms detect breast cancer long before a lump can be felt. And finding it early can save your life
• If you’re concerned about getting breast cancer, getting a mammogram may give you peace of mind

More detail on the structure of a concept test is provided later in this section under Conducting Concept Tests and Materials Pretests and in the Communication Research Methods section.

3. Decide What Materials to Develop

Once you have message concepts that are effective with the intended audience, determine the material formats (e.g., brochure, videotape) that will best suit your program by evaluating:

• The nature of the message (e.g., its complexity, sensitivity, style)
• The function of the message (e.g., to call attention to an issue or to teach a new skill)
• The activities and channels selected during Stage 1 (e.g., whether you will be most likely to reach the intended audience through a school, library, physician, the media, or a combination of these)
• The budget and other available resources

Most important: Make sure that your program's decisions about materials fit with the activities and channels you selected and will contribute to reaching the communication objectives developed in Stage 1.

If your program has to develop new materials, doing so will probably represent a major expenditure. Make sure to choose formats that your program can afford; don't allot so much of your budget to materials production that you can't afford sufficient quantities, distribution promotion support, or process evaluation. The list of materials options below will give you ideas of possible approaches. Don't stop there; use your knowledge of the intended audience to combine, adapt, and devise new ways to get the message across. Explore your program's preliminary decisions about materials with the intended audience or partners, perhaps in conjunction with getting reactions to message concepts.

4. Develop Messages and Materials

The following guidelines will help your program develop materials that intended audiences understand, accept, and use:

• Ensure the message is accurate:
  • Scientific accuracy is vital to producing desired outcomes and to your program's credibility. This is particularly important because of rapid changes in advice and information for many health topics; often, what is accurate one year is no longer current the next. For this reason, experts on the health topic should always review your program's messages. However, you may have to work with them to get them to accept language that is simple enough for a nonscientific intended audience to understand. At times it may be helpful to share pretest results with them or get them to sit in on research with the intended audience so that they are exposed to the intended audience's level of knowledge and scientific sophistication.

Patches: An Innovative Format to Reach Girls

The Department of Health and Human Services, Substance Abuse and Mental Health Services Administration’s Center for Substance Abuse Prevention’s Girl Power! campaign partnered with the Girl Scouts of the USA to develop a Girl Power! Girl Scout patch program. Girl Power!, a national public health education program, addresses a wide range of issues affecting adolescent girls. The overall goal of Girl Power! is to delay and reduce the use of alcohol, tobacco, and illicit drugs among girls ages 9–14. This program also addresses related issues such as physical activity, nutrition, and mental health. Girl Power! and the Girl Scouts cooperate in an ongoing relationship. Through the Girl Scouts, the Center for Substance Abuse Prevention’s Girl Power! campaign materials are distributed to over 2.8 million girls around the country. To see other formats both groups use to reach girls, visit www.girlpower.gov or jfg.girlscouts.org.

• Be consistent:
  • All messages in all materials and activities should reinforce one another and follow the communication strategy. No matter how creative, compelling, or wonderful a message is, if it does not fit the strategy statement, objectives, and identified intended audiences, throw it out. Don't compete with your own campaign for attention.
  • Recognize inconsistencies between the message and what the intended audience may have heard from other sources due to controversies among scientists, government agencies, and advocacy groups. The best way to determine whether and how to address such inconsistencies is to ask the intended audience what impact the disparities have on them and what they need from your program to make decisions and take the desired action.
  • Use the same graphic identity in all campaign elements. In print materials, use the same or compatible colors, types of illustrations, and typefaces throughout the campaign. If there is a logo or theme, use it in all materials. Graphics and messages should reinforce each other, not send different signals.

• Be clear:
  • Keep it simple. Clear messages for lay intended audiences contain as few technical/scientific/bureaucratic terms as possible
    and eliminate information that the audience does not need in order to make necessary decisions or take desired actions (such as overly detailed background about disease physiology, research uncertainties, or background about your organization). Readability tests (see the Communication Research Methods section for instructions and limitations) can help determine the reading level required to understand material and can help writers be conscientious about the careful selection of words and phrases.
  • Prominently feature the action you want the intended audience to take. Give people an explicit recommendation of what
    you want them to do as a result of the message. Don't assume they will figure it out for themselves. For example, pretesting showed that a brochure about taking part in AIDS research studies was very well received by a grateful and emotional intended audience. However, no one recognized that the purpose of the booklet was to encourage people to consider participating in research studies and to talk about it with their health care providers. Instead, almost everyone thought the purpose was to give them hope and comfort. The booklet was revised to be more straightforward without compromising the hopeful tone intended audiences liked.
  • Demonstrate the health behavior or skills (if appropriate).

• Be relevant to the intended audience:
  • Include incentives for the intended audience to take the recommended steps, beyond the health benefits they will receive. Audience research will help you understand what the intended audience might value. Incentives may be psychological (you'll feel more in control); altruistic (you'll help others by participating in a research study); economic (not smoking saves money); social (exercise is a fun group activity, where you can make friends, see friends); family-driven (do it for them); or ego-driven (sun exposure causes wrinkles).
  • Choose a presentation style appropriate to the intended audience's norms and expectations; people must be able to see themselves in what your program presents. For example, intended audience preferences can help determine whether to take a rational or emotional approach, a serious or light tone. Use a light, humorous approach if appropriate, but pretest to be sure that it works and doesn't offend the intended audience. Responses to humor vary greatly.
  • Keep regional differences in mind. A lush, green park will not look like the Southwest and an urban clinic will not be familiar to people who live in a rural area. The organization of the health care system and the way medical and social services agencies work together also differ from region to region.
  • Use intended audience experience when creating materials. The intended audience will learn new information more easily when you build from the familiar to the unfamiliar (e.g., "When you have a headache that won't go away, you take an aspirin to relieve it. But that is not the best way to control cancer pain. Cancer pain medicines work best if you take them before your pain becomes severe and keep taking them on the schedule your doctors advise.")
  • Create the message to match the readiness of the intended audience to make a change. The Stages of Change model (see Appendix B) describes five stages that people pass through in making behavior changes: precontemplation, contemplation, preparation, action, and maintenance. Messages created to match readiness to change start where the intended audience starts. For example, it is unrealistic to expect an intended audience that has never heard of the problem to immediately make changes. A realistic outcome is beginning to raise awareness of the issue (precontemplation) and helping move the intended audience to a consideration of the change (contemplation).

• Be credible:
  • Use celebrity spokespeople selectively. Choose celebrity spokespeople who are directly associated with the message (e.g., using an athlete to promote exercise or cancer survivor to promote early detection) and who practice the desired health habit. Check with the intended audience about the suitability of the celebrity. (For example, while a famous television personality was well liked by members of the intended audience for a physical activity promotion program, they said she was not relevant to them because she could afford a personal trainer to make exercise easier.) Realize that while celebrities can help gain attention for the message, they may also compete with the message for attention or be unappealing to some intended audiences. In addition, your program will probably need to build its schedule around theirs, and celebrity involvement can cause production delays and extra costs. Caution: A network may not use TV PSAs featuring a rival network's star.
  • Be sure that the person who presents the message is seen as a credible source of information, whether as an authority, celebrity, or intended audience representative.You may also want to partner with organizations that are credible with the intended audience and emphasize their involvement. If the message involves health services and health care coverage, it is particularly important to have a source that the intended audience believes is unbiased, i.e., does not have a vested (profit-driven) interest in people taking the recommended action. For example, consumers may dismiss health practices an employer or government agency suggests they take if they perceive the motivation is just to save the organization money.

• Be appealing:
  • Produce variations of materials to appeal to specific intended audience segments. For example, NCI used several different covers on a mammography booklet to appeal to different cultures. Similarly, audio materials can be produced using culturally specific voices or music. Of course, changing executional detail to appeal to specific intended audience segments is appropriate only if the underlying communication strategy and messages have been tested with those audiences.
  • Get the intended audience’s attention. Given the number of health and other messages intended audiences receive, yours must stand out to be noticed. The best way to command attention will differ among intended audiences. It can be useful to know what has interested them before, but concept testing and message testing will help ensure that your program's approach will grab their interest.
  • Produce high quality materials. If you feel you have to skimp on production, choose a simpler way of presenting the message. Producing poor quality materials wastes funds and can damage your program’s credibility and your own.
  • Entertain while you educate when using mass media. Whether you are pitching a news story or producing an ad, remember that the mass media are viewed as a source of information and entertainment, not education. Therefore, if the complete message is too complicated, or simply not considered interesting enough for use by the media, redesign the message so that it is more appealing to media professionals and their perceptions about what their intended audience wants.Working with media professionals will help ensure that your program’s messages are interesting as well as accurate and may help you obtain greater exposure for the program.

Tailored Communications

Tailored communications are much more refined than communications created for a particular intended audience or segment of the population. If you can tailor each message for each individual, it is likely to be even more effective. Tailored communications are produced for each person based upon what is known about the individual. Obtain this information from health plan data, surveys, medical records, and other sources. This information permits matching people with messages appropriate for them. The following are two examples of tailored communication:

• A series of letters (e.g., to give smokers information about and support for quitting) from which paragraphs are selected to send to each individual based on past behavior—on barriers to, for example, quitting, or on other unique combinations of characteristics
• A cover letter, referring readers to the pages of an enclosed booklet that most relate to their interests and concerns

With the help of a computer and ordinary word processing software, individual letters can be tailored within a mass mailing, brochures can be printed on demand for a specific patient, or telephone counselors can appropriately focus their assistance and follow up with messages tied to the advice they gave.

When personal data are not available, interactive computer programs can prompt individuals to input key personal characteristics that direct the software to prepare individually tailored messages immediately (e.g., on a Web site or at a kiosk in a public location). NCI's 5 A Day Web site (www.5aday.gov/) allows users to chart their fruit and vegetable consumption and exercise and offers positive reinforcement in response to the behaviors they report.

Evaluation has shown that tailored communications can, in some circumstances, increase message effects, although much research remains to be done on how and why.

To use tailored communications, review the planning information from Stage 1 and determine:

• The important individual characteristics that affect a particular health behavior (i.e., differences in readiness to change behavior, or perceived benefits of or barriers to changing the behavior)
• Appropriate messages tailored to address individual characteristics (these messages may be identified or confirmed through primary research with each group, or through collaboration with a behavior change specialist or a subject matter expert, such as a genetic counselor or physician)

Then:

• Create a "library" of messages tied to each relevant characteristic, such as gender or risk factors ("as a woman who began smoking in the last five years...")
• Use word processing software with a mail merge feature that will match variations in personal characteristics with appropriate messages and produce the materials in the desired format

Tailoring is not always possible or necessary. However, think about using some of the principles of personalizing messages when appropriate and when funding is available. Find more information about tailored communications in the Selected Readings at the end of this section or use NCI's quick-start tutorial on the Internet at http://dino.nci.nih.gov/public/glassman/TailoringGuide.

Tailored Communications: Healthy Birthdays
In two 1999 projects, researchers designed tailored birthday cards and newsletters to increase breast and cervical cancer screening and smoking cessation, primarily among low-income African Americans. In one study, the cards and newsletters were individually tailored based upon ethnicity, gender, and the individual’s readiness to change according to the Stages of Change model. Smokers received either: Provider prompting Tailored cards and newsletters Tailored cards and newsletters and telephone counseling The tailored cards and newsletters showed a highly significant quitting effect. Thirty-three percent of smokers who received only the tailored cards and letters quit smoking compared to thirteen percent of smokers who received provider prompting and nineteen percent of smokers who received the tailored cards and newsletters and telephone counseling. In the other study, people who received the tailored birthday cards and newsletters showed higher rates of Pap tests and general cancer screening.	Image created by Bernard Glassman and Barbara K. Rimer. All rights reserved.
Note. From Glassman, B., Rimer, B. K. (1999). "Is There a Use for Tailored Print Communications in Cancer Risk Communications?" (Monograph of the Journal of the National Cancer Institute, No. 25) In the public domain.

Developing Effective Print Materials for Low-Literacy Intended Audiences

Many intended audiences have a hard time understanding health materials written in technical language. This is especially true of intended audiences with low literacy skills. Present the message in a more easily understood way to these intended audiences by making specific choices about writing style, vocabulary, typography, layout, graphics, and color. These choices affect whether the message is read and how well intended audiences with low literacy skills understand it. A great deal of health information and promotion is organized around the use of print materials, often written far beyond the literacy skills of intended audiences. Differences in the ability to read and understand materials related to personal health appear to contribute to health disparities. The link between literacy and health disparities underlies the Healthy People 2010 objective to "improve the health literacy of persons with inadequate or marginal literacy skills" (objective 11-2).

A common misconception is that low-literacy materials are synonymous with low-reading-level materials. That is, if you avoid polysyllabic words and long sentences, you've met the need. In fact, low literacy encompasses more than reading level, and meeting this need requires that you complete the same planning and research steps and adhere to the same fundamental communication principles that you would use for any other health communication material. The difference is that certain aspects of the process must be done with particular rigor. The following list includes pointers for designing materials for low-literacy intended audiences:

• Include only the information needed to convey the behavioral objective and support the intended audience in attaining it. Strictly limit content because poor readers struggle with every word, often reading letter by letter. Keep the piece short and focused, and let the communication strategy statement guide you in answering the question, "From the many possible information points, which ones will predispose and enable the reader to take the desired action?" The biggest challenge will be excluding concepts and content points that fall outside of the category of "information the reader must know." Do not include information just because it may interest the reader or because you are trying to promote your organization's work.
• Organize topics in the order the reader will use them. Less skilled readers have particular difficulty connecting topics and processing the flow of an argument.
• Present the most important points first and last. Studies show that intended audiences with limited literacy skills remember these best.
• Group information into chunks, with a clear, ordered format. Use steps (1, 2, 3), chronology (by time of day), or topical arrangement (main heading, subheadings), depending on how the person will use the information.
• Respect the intended audience. This is especially critical when designing low-literacy materials. Examples abound of well-intentioned materials that talk down to readers or have childlike or simple cartoon illustrations. The low-literacy population encompasses people of different ages, genders, cultures, and socioeconomic status, including highly intelligent adults with significant life experience who just cannot read very skillfully.
• Follow these guidelines.
  • Use short sentences and paragraphs.
  • Write in the active voice.
  • Clarify concepts with examples.
  • Avoid jargon, technical terms, abbreviations, and acronyms.
  • Include a glossary if necessary (but define key words within the sentence).
  • Give the reader an action step he or she can take right away (e.g., call your clinic, send in a request); this tends to improve retention of information and encourages the reader to begin practicing the desired behaviors immediately.
  • Use graphics and design to make the reader's job easier and to increase comprehension and recall; make sure they support, rather than compete with, the text.
  • Don't assume that pictorial signs, symbols, and charts are more effective than words for low-literacy intended audiences. Some experts suggest that "universal" symbols, such as a stop sign, an arrow, or a big black "X," usually test well. Don't confuse this intended audience with large, busy matrices—for example, functionally illiterate individuals have trouble using a bus schedule.
  • Avoid using all capital letters; they are more difficult for everyone to read, particularly so for less skilled readers.
  • Use captioned illustrations that are relevant to the subject matter and model the desired behavior.
  • Use headings and subheadings to convey a message and help reinforce the flow and content.
  • Use bullets and other graphic devices to highlight key messages and to avoid large blocks of print.
  • Avoid right-justified margins.
• Pretest all materials with the intended audience. This is absolutely crucial with low-literacy intended audiences.Writers and communication specialists are highly literate by definition. It is impossible for a person who reads well and has a good vocabulary to guess what people without those skills will understand. For example, an FDA brochure on food safety used the key message, "Keep hot foods hot, keep cold foods cold." Pretesting showed that low-literacy readers had no idea what they were supposed to do based on this message, nor did they understand what foods fell into the hot and cold categories.

A final note: You will find that most intended audiences of any reading level prefer well-produced materials that follow these guidelines.

Developing Effective Web Sites

A Web site should be graphically appealing and provide information about health issues in an informative manner. Some organizations begin by creating sites that primarily provide information to their stakeholders, employees, or members. To extend outreach, create an additional section in the site to appeal to the intended audience. For consumers, you might call this section the Help Center and provide a place for users to receive information about a particular health topic, participate in online surveys, or download your organization's consumer information. Many sites contain useful public health information and resources, but too often this information is buried within the site. Keep visitors interested in the site by making it easy to navigate.

To ensure that users will find the site well designed and easy to use, pretest the site as you would any other materials. Usability testing, which tests the site to see how well it helps users meet their goals, is crucial to creating an effective site. The best time to do this testing is as you are developing the site, not after it's completed. If the site is not yet running on a computer, test using paper or poster board mock-ups of pages. Conduct usability testing by having people who represent the intended audience actually sit down and use the site to complete tasks, either by themselves or in pairs. Observe how they interact with the site and ask specific questions once they have completed the tasks.

Their experiences and responses will allow you to improve the site before it is used. If you make major modifications to the site after usability testing, test again before the site goes live. For more information on usability testing, see www.usability.gov. Remember, your well-designed and attractive site is useless unless people know it exists. Therefore, consider launching a Web site by conducting both traditional and online media outreach. Online outreach can include alerting search engines such as Google or Yahoo about the site as well as selecting publications that specialize in online issues or exist only online.

Characteristics of Well-Designed Web Sites

Compliant with W3C accessibility guidelines ( www.w3.org/wai), which ensure access to the Web by everyone regardless of disability, and, for government publications, Section 508 guidelines (www.usability.gov/accessibility) Clean and consistent design (e.g., a simple background, legible type, a few carefully selected colors) A search engine (program that helps users find information) and a link to the search engine on all Web pages Fast display of graphics and text Clear and consistent navigation elements (to make it easy for users to move to and from different information sources) Interactivity and fun elements Short/concise pages (e.g., a Web page should not exceed 250 words on average) Compatibility with major browsers and earlier versions of major browsers (e.g., Netscape, Microsoft Internet Explorer) Mechanisms to track site usage and invite user response

5. Pretest Messages and Materials

Although working with advisory groups and gatekeepers can add useful input for developing intended audience-appropriate materials, only testing with members of the intended audience will tell you what their reactions might be. It is always better to conduct research with intended audience members than to do without it, especially with audiences that you do not know well (e.g., cultural groups other than your own) or whose education or skill levels are different from yours (e.g., audiences with limited literacy skills). When resources are an issue, use lower cost methods and money-saving strategies to keep costs down (see sidebar, Keeping Pretest Costs Down).

Pretest preproduction draft materials. Testing at this stage permits you to identify flaws before spending money on final production. To test materials in draft form, use a facsimile version of a poster or pamphlet, a video version of a television PSA, or a prototype of text materials like a booklet. Test these materials with members of the intended audience to accomplish the following:

• Assess comprehensibility—Does the intended audience understand the message?
• Identify strong and weak points—What parts of the materials are doing their job best—for example, attract attention, inform, or motivate to act? What parts are not doing their jobs?
• Determine personal relevance—Does the intended audience identify with the materials?
• Gauge confusing, sensitive, or controversial elements—Does the treatment of particular topics make the intended audience uncomfortable?

A Review Process Adds Value

As you test materials in rough form with the intended audience, it is also valuable to obtain gatekeeper or other reviewer comments. Health communicators often ask gatekeepers (e.g., public service directors, physicians, teachers, partner organization leaders) to review materials both to get input from people close to the intended audience and to increase the likelihood that the gatekeepers will use the materials with the audience. If the graphics style or illustrations depart from what gatekeepers or other reviewers expect, focus on these issues when testing draft products with the intended audience. Use favorable responses from the intended audience to persuade gatekeepers to accept your program's approach. Gatekeeper review should not be used as a substitute for pretesting materials with members of the intended audience.

Many organizations have established review procedures that can seem like hurdles to timely production of materials. However, reviewers can potentially add value, and review by some experts can be imperative for producing accurate, accepted communication materials. Reviewers will help create accurate documents for pretesting. After pretesting, reviewers can help synthesize the results and help with revising the document. The following tips will help you structure a value-added review process:

• Choose reviewers carefully. Reviewers should have relevant knowledge to contribute and be conscientious enough not to delay the process. Make sure there is a range of expertise represented among the reviewers, including subject experts, communication specialists, intended audience experts/representatives, and those who understand your organization and your partners' policies and priorities. In some cases, reviewers with other skills or viewpoints may be important, such as legal professionals, law enforcement officials, social workers, school personnel, or clergy.
• Explain exactly what you want reviewers to do and not to do. One organization's national campaign materials were delayed when a top administrator was given camera-ready layouts for his final approval, and he wrote comments on them. Reviewers are commonly handed materials without guidance; they may not understand the purpose or the context of the material and have no way of knowing what you want from them. Often this results in reviewers' simply tinkering with words or proofreading the copy, delaying the process without adding value. Instead, give them a list of questions to answer or other guidance.
• Never skip technical review by an expert, who may catch concerns or inaccuracies others miss. For example, the facts you present may be accurate but not reflect new information that only an expert would know.
• Incorporate as many comments as possible.You may need to get clarification on some comments or to talk with reviewers whose comments seem off base.You may find that you can compromise, but remember you are not required to act on all the comments you receive.
• Allow sufficient time in the development process to deal with reviewers' comments. This includes giving reviewers enough time to do a thorough job and giving creative professionals enough time to make revisions thoughtfully and carefully.

Conducting Concept Tests and Materials Pretests

To plan and conduct concept tests and materials pretesting, complete the following steps:

1. Determine test objectives.
2. Choose methods.
3. Secure vendors, facilities, and moderators or interviewers (if required).
4. Identify, screen, and recruit respondents.
5. Draft test instruments (discussion guides, questionnaires).
6. Conduct pretesting.
7. Analyze results.
8. Make the best use of results.

Key aspects of each step are presented below; see the Communication Research Methods section for more detail.

Determine Test Objectives

Develop a description of which segments of the intended audience you want to include and exclude in testing. Do you want to include individuals who have attempted a particular health behavior and succeeded? Failed? Ever thought of trying? Consider excluding individuals whose professions may make them too experienced with the topic or the method (e.g., health professionals, market researchers, employees of advertising agencies, or public relations firms). In some instances, exclude consumers whose past experience makes them too knowledgeable. For example, sometimes people with a chronic disease know too much about the condition to objectively assess materials designed for patients recently diagnosed.

Don't rely on these participants for creative guidance. They are expert consumers, and it is important to learn their preferences. They are not, however, communication or creative professionals, and their ideas for substantive message or materials changes are likely to be off the mark.

Choose Methods

A variety of research methods can and should be used to explore message concepts and test messages and materials. Which you choose depends on the research questions you want to have answered, the nature of the materials, the intended audience, and the amount of time and resources available for pretesting.

The following is a list of methods to use to test materials:

• Concept testing with intended-audience members
  • Focus groups (face-to-face or telephone)
  • In-depth interviews
• Pretesting with intended-audience members
  • Self-administered surveys/questionnaires (by mail, handout, or computer)
  • Interviewer-administered surveys/questionnaires (by telephone, through central-location
    intercepts, or other face-to-face scenarios)
  • Theater testing (large groups respond to messages via questionnaire or electronic)
  • Observational studies (e.g., observing behaviors of shoppers in a store or patients waiting
    in a clinic)
• Other assessment methods
• Readability assessments
• Expert/gatekeeper review

Sometimes, using several methods in combination will help overcome the limitations of individual methods. For example, readability testing should be used as a first step in assessing draft manuscripts, followed by self-administered questionnaires or interviews or, for some long documents, a combination of the two with intended audience respondents. Central-location intercept interviews for short audio, video, or print materials permit contact with larger numbers of intended audience respondents, which is especially useful prior to final production of materials. See the Communication Research Methods section for definitions of commonly used pretesting methods and descriptions of the pros and cons and common uses of each.

Secure Vendors, Facilities, and Moderators or Interviewers

Some research methods require securing appropriate facilities. If you are using commercial facilities (e.g., for focus groups, central-location intercept interviews, or theater tests), the American Marketing Association’s New York Chapter’s Green Book provides an extensive directory of vendors, indexed by geographic locations and services offered. The Communication Research Methods section provides tips for working with vendors and for securing alternative facilities if location or budget precludes using a vendor. The following are vendor services your program may need:

• Teleconference services to set up telephone focus groups. Most allow observers to listen without being heard, and some provide remote viewing programs to allow the moderator to see a list of participants (with a symbol next to the name of the one currently speaking) or notes sent by a technician from observers listening to the call. Some teleconference services can recruit participants; with others, recruit or contract with a recruiter separately.
• Focus group facilities to recruit participants that also offer audio and video recording equipment as well as one-way mirrors with observation rooms for viewing both focus groups and in-depth interviews. These are usually available only in larger metropolitan areas.
• Commercial facilities to conduct central*#8211;location intercepts at shopping malls.You can also use other facilities or public locations that attract a large number of pedestrians (e.g., other stores, train stations, university campuses, large medical centers); usually permission is required to interview people in these types of locations.

If you are conducting focus groups or in–depth interviews, identify trained, experienced moderators or interviewers. If your organization has no experience in such studies, consider hiring a good, experienced moderator or interviewer to conduct the project or to train internal staff to develop in–house skills. Local advertising agencies, the American Marketing Association’s Focus Group Directory, the Qualitative Research Consultants Association, or other health communication program managers may be of assistance in identifying a good moderator. Before choosing a moderator or interviewer, ask for and check references.

Identify, Screen, and Recruit Respondents

Use the intended audience description developed in Stage 1 to draft questions for recruiters to use to identify participants. See Appendix A for a sample form to use to screen potential participants.

Draft Test Instruments (Discussion Guides, Questionnaires)

Involve the creative team in developing the discussion guides or questionnaires. The team will often have specific issues or questions. Assess how much the intended audience likes each set of material, because research has found this to be a leading indicator of success. Other specific questions should be used to identify strengths and weaknesses in rough messages and materials. See Appendix A for a sample focus group moderator’s guide and an intercept questionnaire.

Conduct Testing

If possible, have other team members, such as creative professionals, content experts, partners, and gatekeepers (including decision-makers who control your program), observe at least some of the focus groups or interviews. Creative professionals’ observations are particularly important during concept exploration, because they often spot comments or trends important for creative development. Hearing the responses improves team members’ understanding of intended audience reactions and can illustrate, more vividly than any report, the need for simple language or the power of a particular creative approach.

Analyze Results

To analyze and communicate the results of materials testing, write a report outlining the process and the findings. The report should include the following sections:

• Background—Who was tested? Why? How? What did you hope to learn? (Describe each in detail.)
• Highlights—Summarize the main points that emerged from testing that answer the research questions.
• Findings—Present a complete report of findings.Where appropriate, describe participants’ reactions, quote participants, and use examples from the test documents to support the findings.
• Conclusions—Describe patterns that emerged or significant differences observed between groups. (If no patterns are apparent, more research may be needed.)
• Recommendations—Suggest revisions for materials or planned approaches based on findings and conclusions.
• Appendixes—Include copies of test instruments used, such as discussion guides, screening questions, and questionnaires.

Decide who should prepare the report. Ideally, choose a team member with a background in market research and health communication, and have that person include input from the creative team on recommendations. If such a person is not available, whoever conducted the test may be able to prepare a basic report on the findings but, depending on his or her background, this person may not be able to draft useful recommendations. Before relying on vendors to write reports, ask to see samples of reports they’ve written for other clients.

One problem that can arise in pretesting concerns interpretation of respondent reactions to a sensitive or emotional subject such as breast cancer or AIDS. Respondents may become unusually rational when reacting to such pretest materials and cover up their true concerns, feelings, and behavior. Therefore, the pretester must examine and interpret responses carefully.

The following are some tips for using pretest results:

• Involve the creative professionals who designed the materials in recommending creative solutions or revisions, rather than expecting researchers to make recommendations that creative professionals may later legitimately reject.
• Consider pretesting again if your program has made major changes to a message or product based on the original pretest conclusions.You may have addressed the right problems, but you can’t be sure the new solutions will be effective without intended audience testing. The key is allowing time for this possibility in the initial schedule.

Make the Best Use of Results

Pretesting findings can be used to solve problems, plan programs, develop materials, or refine materials or messages. It is important to avoid misuse of market research results. Perhaps the most common error is to overgeneralize. Qualitative, diagnostic pretest methods should not be used to estimate broad-scale results. If 50 of the 100 respondents in an intercept test do not understand portions of a pamphlet, it does not necessarily mean that 50 percent of the total intended audience will be confused. The lack of understanding among those pretest respondents suggests, however, that the pamphlet may need to be revised to improve its comprehensibility. Remember that pretesting is indicative, not predictive.

Planning for Production, Distribution, Promotion, and Process Evaluation

Once you have decided which materials to produce, determine how many copies of each will be needed and develop a production schedule. To write a realistic production schedule, review or revise the following three components of the communication plan:

1. Distribution channels you plan to use (including specific organizations, companies, sites, etc.), how many materials you plan to disseminate through each, and how many intended audience members you propose to reach.
2. Promotion plans, which describe how you intend to promote use of the materials.
3. Process evaluation plans, which describe the way the use of materials will be monitored (e.g., using bounceback cards to identify how respondents are using the materials). Go to the process evaluation section of Stage 3 for more details.

Distribution and promotion sections of the program plan will define both production costs and the quantity of materials to produce. It is a good idea to get input on the distribution, promotion, and evaluation sections of the program plan from partners and others involved with implementation.

Common Myths and Misconceptions About Materials Pretesting

Myth: I don’t have the time or money.

Fact: Pretesting needs to be planned as an integral step in the materials development process from the beginning. Include time and resources for pretesting and for any changes you might need to make as a result in the project budget and timeline. Otherwise, your program may not have the funds, and your boss may see the time for pretesting and alterations in materials as a delay in production rather than evidence of careful planning and development.

Myth: My boss won’t support pretesting.

Fact: Use the information in this guide and in the Selected Readings to convince your boss that you need to pretest. Beautiful materials and an elegant design can’t guarantee that the intended audience will pay attention to, understand, or relate to the messages. It’s cheaper to find out whether the materials might work before they are produced than to have to start over later or, worse, to have an unsuccessful program. Once you have pretested, be sure to explain to your superiors (in a report) how it worked and what resulted. Build a case for their acceptance of future pretesting. Using quotes from the intended audience or anecdotes to illustrate the findings can make the report more interesting and memorable.

Myth: I can tell the difference between good and bad materials, so I don’t need to pretest.

Fact: Many people have said this, only to find out they can be wrong.Your training and experience are essential credentials, but are you sure you can react objectively to materials you have created or are responsible for? Can you really assume the role of someone who is different from you (if you are not representative of the intended audience) and see your materials through his or her eyes? Can you defend your decision, without objective evidence, to those who may disagree?

Myth: Our artist/producer says that pretesting can’t be used to judge creativity.

Fact: Graphics staff, artists, and creative writers may be sensitive to criticism from "nonprofessionals," including the intended audience. Explaining the purpose of pretesting and involving them in the pretesting may help them understand and appreciate the process. Explain that you are testing all elements of the communication and not just their work. By testing alternative concepts or executions, you can provide the creative staff with direction without telling them their work "failed."

Selected Readings

American Marketing Association, New York Chapter. (2001/2002). Green Book: International directory of marketing research companies and services. New York.

Backer, T. E., Rogers, E. M., & Sopory, P. (1992). Designing health communication campaigns: What works. Thousand Oaks, CA: Sage.

Campbell, M. K., DeVellis, B. M., Strecher, V. J., Ammerman, A. S., DeVellis, R. F., & Sandler, R. S. (1994). The impact of message tailoring on dietary behavior change for disease prevention in primary care settings. American Journal of Public Health, 84, 783–787.

Center for Substance Abuse Prevention. (1994). Careful concept development paves the way to effective prevention materials [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.

Center for Substance Abuse Prevention. (1994). Conducting focus groups with young children requires special considerations and techniques [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.

Center for Substance Abuse Prevention. (1994). Following specific guidelines will help you assess cultural competence in program design, application, and management [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.

Center for Substance Abuse Prevention. (1994). A key step in developing prevention materials is to obtain expert and gatekeeper reviews [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.

Center for Substance Abuse Prevention. (1994). Pretesting is essential:You can choose from various methods [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.

Center for Substance Abuse Prevention. (1994). You can manage focus groups effectively for maximum impact [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.

Center for Substance Abuse Prevention. (1994). You can prepare easy-to-read materials [Technical Assistance Bulletin]. Washington, DC: U.S. Government Printing Office.

Center for Substance Abuse Prevention. (1994). You can use communications principles to create culturally sensitive and effective prevention materials [Technical Assistance Bulletin].Washington, DC: U.S. Government Printing Office.

Centers for Disease Control and Prevention. (1994). Listening to your audience: Using focus groups to plan breast and cervical cancer public education programs (CDC Publication No. PDF-245K). Denver: AMC Cancer Research Center.

Centers for Disease Control and Prevention. (2000). Beyond the brochure (CDC Publication No. PDF-821K). Atlanta.

Debus, M. (1988). Methodological review: A handbook for excellence in focus group research. Washington, DC: Academy for Educational Development.

Goldberg, M. E., Fishbein, M. F., & Middlestadt, S. E. (Eds.). (1997). Social marketing: Theoretical and practical perspectives. Mahwah, NJ: Erlbaum.

Jones, J. P. (Ed.). (1998). How advertising works: The role of research. Thousand Oaks, CA: Sage.

Kotler, P., & Roberto, E. L. (1989). Social marketing: Strategies for changing public behavior. New York: Free Press.

Lipkus, I. M., Lyna, P. R., & Rimer, B. K. (1999). Using tailored interventions to enhance smoking cessation among African Americans at the community health center. Nicotine and Tobacco Research, 1(1), 77–85.

Maibach, E., & Parrott, R. L. (Eds.). (1995). Designing health messages: Approaches from communication theory and public health practice. Thousand Oaks, CA: Sage.

Merton, R. K. (1987). Focused interviews and focus groups: Continuities and discontinuities. Public Opinion Quarterly, 51, 550–556.

Merton, R. K., Riske, M., & Kendall, P. L. (1996). The focused interview (2nd ed.). New York: Free Press.

Morgan, D. L., & Krueger, R. A. (1998). The focus group kit. Thousand Oaks, CA: Sage.

National Cancer Institute. (1994). Clear and simple: Developing effective print materials for low-literate readers (NIH Publication No. 95-3594). Bethesda, MD.

National Cancer Institute. (1998). How the public perceives, processes, and interprets risk information: Findings from focus group research with the general public. Bethesda, MD: U.S. Department of Health and Human Services.

National Cancer Institute. (2000). Multiethnic focus groups to test motivational messages on mammography and breast cancer. Bethesda, MD.

Palmgreen, P., et al. (1995). Reaching at-risk populations in a mass media drug abuse prevention campaign: Sensation seeking as a targeting variable. Drugs and Society, 8(3), 29–45.

Rice, M., & Valdivia, L. (1991). A simple guide for design, use, and evaluation for education materials. Health Education Quarterly, 18(1), 79–85.

Rimer, B. K., & Glassman, B. (1998). Tailoring communications for primary care settings. Methods of Information in Medicine, 37(3), 171–177.

Selden, C. R., Zorn, M., Ratzan, S., & Parker, R. M. (2000). Health literacy, January 1990 through October 1999. Bethesda, MD: National Library of Medicine.

Siegel, M., & Doner, L. (1998). Marketing public health: Strategies to promote social change. Gaithersburg, MD: Aspen.

Strecher, V. J., Kreuter, M.W., DenBoer, C. H., Kobrin, S. C., Hospers, H. J., & Skinner, C. S. (1994). The effects of computer-tailored smoking cessation messages in family practice settings. Journal of Family Practice, 39, 262–270.

U.S. Department of Health and Human Services. (2000). Healthy people 2010 (2nd ed.; in two volumes: Understanding and improving health. and Objectives for improving health).Washington, DC: U.S. Government Printing Office.

Wells, W. D. (Ed.). (1997). Measuring advertising effectiveness. Mahwah, NJ: Erlbaum.

Back to Top

< Previous Section  |  Next Section >