Introduction: Evaluation of IHC |
|
Introduction Interactive health communication (IHC) applications, through their information, emotional support, decision support and behavior change services, have the potential to dramatically improve the public's quality of life and reduce the total burden of illness and injury. Their emergence has been fueled by the growth and increasing sophistication of the Internet, which allows geographic barriers to fall and offers people an opportunity to learn from widely diverse resources. But their growth will be fertilized by the propensity for the health care system to push health care away from institutions, specialists, and even primary care providers. As patients are spending less time in the hospital and with their health care providers, they are being forced to assume more responsibility for their care, but often not provided resources to do it. Moreover, the increasing complexity of medicine, pressures for increased productivity, and growing recognition that there is more to dealing with illness than the standard medical model, is prompting providers to share responsibility with patients. There has been a separate but reinforcing trend toward patients and families wanting to participate in their own health care. The elderly, a growing percentage of the population, are becoming more assertive participants. Baby boomers, who have generally been more assertive, are moving into their late 40s and 50s, a time when chronic health conditions start to emerge and interactions with the medical system increase.[1] This assertiveness is generally good, if support is available to help patients be effectively assertive. IHC applications can help fill this growing need for individuals to understand and take greater control of their health and that of their families. However, IHCs also pose a risk to consumers who cannot sort out inadequate and misleading applications from ones of high quality. The purpose of this article is to explore the potential risks and benefits of IHC applications to consumers, and discuss initiatives that may help consumers make more informed choices among IHC applications and make more-effective use of the applications they adopt. While the consumer is the ultimate beneficiary, the article is also intended for audiences who act as advocates for the consumer. For the consumer, this report addresses issues such as reasons why evaluation should be important to them, encourages development of an evaluation template customized for them in relevant language, and offers a list of consumer rights and responsibilities and how to exert them in this field. For those interested in consumer welfare, we identify opportunities in areas such as nutrition-labeling systems, research, Web-based surveys, consumer report services, and oversight through existing or new mechanisms. The Need Initially, the Web and other computer-based health support systems provided little more than electronic page flipping and ways to talk with other people. As the technology improves it becomes possible to use more sophisticated methods for learning, social support, decision support, behavior change, and, unfortunately, manipulation. Expert systems, video, and access to large databases are state of the art in stand-alone systems and becoming practical on Web-based systems accessible at home, at work, in clinics, and in public places such as libraries and/or kiosks. This extends opportunities for patients and families to become much more informed about their disease and potentially become a valuable partner in care. At the same time, the amount of health information available from the Web and other electronic resources is vast. There are high-quality Web sites and stand-alone products developed by organizations and individuals committed to helping people in need. Other resources are factual but incomplete. They sometimes tell only one side of the story and are frequently out of date. Other IHCs are organized from a clinician's point of view, written in medical jargon, or assume the reader already knows a lot about a disease. In still other cases, the products are illogically structured or hard to use. Most consumers will rapidly detect any of these weaknesses. However, an increasing number of products have more sophisticated deficiencies. These products can project a credibility that may not be deserved and may be potentially dangerous. They can be used as subtle ways to market products and services of dubious value with the reader being victimized by those with a financial interest in the information they provide. In fact, this biased or incomplete information may not appear to be marketing at all. Moreover, many products are based on untested algorithms, formulas, and premises. Even the personal stories placed on the Web can be misleading if they emphasize the rare health care disasters or miracle cures rather than give the reader a balanced presentation and discussion of average results necessary to place the personal stories in context. It is important, therefore, to move away from a perspective that these systems can do no harm. Misinformation can lead patients with life-threatening conditions to lose trust in their provider, take actions that undermine the effectiveness of their treatment (e.g., by taking substances that interact in a negative way with prescribed medications), use their limited time with a provider unproductively in ways that ultimately increase costs of care, and even abandon a provider delivering high-quality care to pursue ineffective therapies. Vulnerable people may also be victimized by biased or incomplete information from those with a financial interest in the information they provide. As a result, they may take actions that may have a negative effect on their economic and social health. In a recent study, the Federal Trade Commission found more than 400 potentially false and deceptive claims on the Web; products and services were claiming to help cure, treat, or prevent AIDS, arthritis, cancer, diabetes, heart disease, and multiple sclerosis. J. Bernstein, Director of the FTC's Bureau of Consumer Protection, said, "Hopeful and sometimes desperate consumers spend millions of dollars on unproved, deceptively marketed and often useless `miracle cures' and the Internet should not become the newest medium for this age-old problem. In addition to wasting consumers' money, some products or treatment may even cause them serious harm or endanger their lives." For instance, Web sites that portray only disasters can cause unnecessary worry and lead consumers to take unwarranted actions. Such risks are present in most media. Advertising for "Health Foods" and nutritional supplements sell billions of dollars of products without extensive evaluations. Health infomercials lead people to spend billions on fitness equipment and other health products that have little, if any, evaluation associated with them. Moreover, many health care services in the mainstream health care system have evolved without carefully developed and replicated clinical trials. Still, there is a need to hold health information products to a high standard because of their increased potential for negative impact. Emerging research finds people assigning more credibility to computer-based information than television and other media,[2] because they can return time and again to a product that is becoming increasingly sophisticated at using expert systems, graphics and video, as well as tailoring messages to single consumers. In many other delivery systems (e.g., health professionals, health maintenance organizations, hospitals), the consumer is at least partially protected by certification, licensing, and credentialling systems. But there are no oversight mechanisms for IHC developers. Risks to privacy and confidentiality are another reason for
holding IHCs to a higher standard, particularly as technology becomes more proficient in
securing and using information about users. A consumer may have no idea of what happens to
information they enter into an IHC application A Consumer Focus to Evaluation Consumers need to understand the risks associated with these products. They need a way of separating truth from fiction, anecdote from established fact. They need a way to judge the quality of the information and support provided as well as the privacy of the information they provide in return. Unfortunately, evaluation of these products can be time-consuming and difficult. Many savvy evaluators shy away from providing a single evaluation or ranking of health information resources because content changes so often (especially on the Web), because one cannot depend on "authorities" to keep up with or evaluate material, and because definitions of quality vary depending on the needs of the user. In fact, the difficulties make traditional methods of evaluation or certification of limited value. There are several approaches that can be taken to assist consumers. These might include external controls and policing, industry self-policing with verification, certification by some independent body, and consumer education. Consumers (and evaluators) need to understand that good evaluations must consider more than just the quality or accuracy of the information. These products may make them feel anxious or overly confident, or may affect the trust they have in people important to them. What kind of emotional support does the product offer? Did it actually help them? Moreover, different consumers learn in different ways. Two products that have the identical information may have different effects on the same person. Consumers need to consider what style of learning works best for them in choosing IHC applications. In essence, consumers should develop their own evaluation framework, which teaches them not only what information to look for but also to consider how the IHC application made them feel and how it affected the confidence they have in their decisions. When there is potential for harm, consumers need to be equipped to make informed choices among products. Evaluations need to be conducted and the consumer needs to be informed of, or at least have access to, the results. The greater the potential for harm the more extensive the evaluations should be. Consumers need to understand and act on the potential risks as well as the potential rewards of health support technologies. To that end, carefully crafted examples and stories portraying both sides of an issue should be developed. Consumers need to know how valid the health information is. At the same time, they need to be realistic about what, if any, evaluation standards should be placed on developers and what evaluation expectations should be set for third parties such as consumer watchdog groups. It may not be reasonable (or even appropriate) to expect that a developer will conduct a randomized controlled trial of every product they develop. Some developers believe that evaluations are less likely to sell products than high-quality marketing. One of the goals of the Science Panel is to find ways to make it in the best interest of developers to secure high-quality evaluations of their products by promoting consumer demand for products of demonstrated quality. Therefore, it is important that we not focus all our efforts on evaluations of products after they are developed. Resources must be put into preventing problems, making it easier for developers to create high-quality products in the first place. Quality background R&D could be promoted through public and philanthropic grant programs. For example, carefully constructed needs assessments,[3] conducted by an independent third party and made available at no cost to the developers, could raise the standard of quality without asking the developer to collect their own needs-assessment data. For instance, a research team could interview breast cancer patients and their families to better understand what it is like to have breast cancer. From these interviews a set of patient and family needs would emerge. Then a survey containing these needs could be distributed to many breast cancer patients and family members at varying times after diagnosis, asking them to rate each need in terms of importance and the degree to which the need has been satisfied. Software templates (e.g., for tracking patient behavior or for making decisions) could be developed and made available to developers so they would not have to develop their own. Databases of frequently asked questions (and high-quality answers) as well as high-quality published articles could be made available in the public domain for all developers to use. In this way, we can raise the minimum standard of information that could be easily provided by any product. Goals This article addresses opportunities for action that are intended to achieve the following goals:
Opportunities for Action At present, the digital media may be too fluid for rigid regulation. As an alternative, if enough people are concerned about finding good information, their demand will drive someonenonprofit or commercialto offer such evaluations. The government can foster, standardize, and promote the development and dissemination of these evaluations. There are no studies of what consumers need in the way of information from evaluations of computer-based health support systems. However, there have been studies of what consumers would need from evaluations of health plans as they choose among competing alternatives. The findings of one study[5] offer some insight into how to construct evaluations of IHC applications.
A multifaceted approach can help raise consumer awareness of the risks and potential benefits of using IHC applications and to equip consumers to be discerning in their selection and use of such systems. Key possibilities include: 1. A statement of protections and responsibilities that are appropriate to the emerging IHC field should be adopted and promoted.[6] This is important not only for the consumer but for developers and providers of these applications and services because it establishes a baseline of trust while at the same time asserting the responsibilities of the consumer. This list could cover:
2. A consumer-oriented evaluation and certification entity to evaluate IHC applications could promote standards, conduct evaluations of health information, certify high-quality applications, point consumers to other evaluations of IHC applications, and help them recognize the quality of and effectively use evaluations. This entity could raise consumer awareness of issues (such as privacy) and conduct and publish (through print and electronic venues) evaluations of new Web-based as well as stand-alone programs. The service could be designed to support not only consumers but others concerned about their welfare. The service could examine the needs assessment on which a program is based, the intended uses of the program, and the quality and bias of content. It might evaluate usability, system security, user acceptance, and use, as well as the impact of the system. The evaluation process and results could be presented in easy-to-understand formats and be widely disseminated. If evaluations have already been completed, the service might comment on the quality of those evaluations.
This service could encourage consumers to seek evaluation information about these applications and provide them with tools for interpreting evaluation results and for conducting their own evaluations. The service could work with advocacy groups to search out, identify, and have competitions for the best software in a particular area. The service and the advocacy groups could publicize the results to their. 3. An evaluation template[7] could be developed to help consumers conduct their own evaluation of IHC applications. One could be a Web-based interactive program including:
4. A Web-based source of product evaluation data could collect and disseminate additional data about applications. Advocates could use it to address privacy and confidentiality issues. Developers could use this site to complete a survey providing uniform information to consumers about applications and their evaluations. Consumers could get explanations of what the questions mean and how to interpret the answers. To use this database, the developer might certify the accuracy and currency of their information. This site could also act as a forum where some of the best IHC Web sites and stand-alone applications (without implied endorsement for the future) could help pinpoint some of the questionable practices and strategies already prevalent on the Net. 5. A research agenda supported by government, foundation, and industry could include:
6. Vision. A panel of expert developers, providers, change theorists, and influential consumers might be convened to create a vision of a high-quality consumer health support. Principles that might be an important part of such a vision were developed by the Vision Roundtable.[8] This vision could be crisp, realistic, and easily understandable, and endorsed by leaders from the consumer and provider sectors. A respected opinion leader could act as spokesperson for the activity. The vision could be widely shared by advocacy groups, government, and providers. Like the needs assessment results, this vision needs to be disseminated through several venues, not the least of which would be the Internet. 7. Public domain raw materials for developers. Public domain raw materials and tools could be created that would make it easier for developers to produce high-quality IHC applications. In this way, we might be able to raise the minimum standard of information that could be provided easily by any new product. These resources could include:
8. Health literacy programs designed to help consumers understand and interpret health science. Understanding how to interpret relevant scientific discoveries will be increasingly important as individuals assume more responsibility for health-related decisions. To gain that capability, training might be needed in areas such as critical thinking, judgment, risk communication, understanding of bias, or levels of evidence. A long-term strategy must include elementary and secondary education programs in such subjects. But a more short-range strategy could include community education programs as well as continuing education programs offered through health care providers and on the Web. 9. A labeling system. A labeling process could be developed and widely deployed, requiring that all interactive electronic communication technologies purporting to help people prevent or cope with illness or injury, prominently display (e.g., through a paper or electronic package insert) certain data about the program. Essential data might include: target audience, target health issues; identification, affiliation, and credentials of the authors; content sources; disclosure of program ownership, sponsorship, commercial funding, potential conflicts of interest; dates when content was posted; types of evaluations conducted, results including side-effects to date. Additional details (populations studied, sample size, recency of data, base rates) of the evaluation could be filed with the consumer reports service and be available through a Web site. Interactive Health Communication applications hold potential for dramatically improving the quality-of-life and reducing the total burden of illness. The panel hopes this article will contribute to the discussion on how consumers can make more informed choices among IHC applications and help those consumers make more effective use of the applications they adopt. [11] Acknowledgements The authors are grateful to Paul Kim; Andy Maxfield, PhD; Anne Restino, MA; and John Studach, MA; for their contributions to the panel's work, and to Linda Friedman for assistance with copy editing. In addition, the authors thank the liaisons to the Science Panel on Interactive Communication and Health, especially the following persons who offered valuable suggestions for improving this manuscript: Loren Buhle, PhD; David Cochran, MD; Connie Dresser, RDPH, LN; Alex Jadad, MD; Craig Locatis, PhD; Ed Madara; Georgia Moore; Kent Murphy, MD; Scott Ratzan, MD, MPA, MA; Helga Rippen, MD, PhD; and Christobel Selecky. 1. Vision Roundtable. Vision for health learning: a partnership model. Queenstown, MD: Aspen Institute, November 13, 1994. 2. Hawkins R, Gustafson DH, Chewning B, Bosworth K, Day P. Interactive computer programs as public information campaigns for hard-to-reach populations: the BARN Project Example. J Commun 1987; 828. 3. Degner L, Kristjanson L, Bowman D, et al. Information needs and decisional preferences in women with breast cancer. JAMA 1997. 4. Brennan P, Moore S, Smyth K. The effect of a special computer network on caregivers of persons with Alzheimers disease. Nurs Res 1995; 44:166 72. 5. Gustafson D. Consumer health plan information needs: results of pilot interviews and focus group meetings. University of Wisconsin-Madison: Center for Health Systems Research and Analysis, 1995. 6. Presidents Advisory Commission on Consumer Protection and Quality in the Health Care Industry, 1998. 7. Robinson TN, Patrick K, Eng TR, Gustafson DH. An evaluation-based approach to interactive health communication: A challenge to medicine in the information age. JAMA 1998; 280:12649. 8. Vision for health learning: a partnership model. Queenstown, MD: Aspen Institute, 1994; November 1-3, 1994. 9. Gustafson D, Hundt A. Findings of innovation research applied to quality management principles for health care. Health Care Manage Rev 1995; 20(2):16 33. 10. Flanagan J. The critical incident technique. Psychol Bull 1954; 51:32758. 11. Gustafson D, Hawkins R, Boberg E, et al. Impact of a patient-centered, computer-based health information/support system. Am J Prev Med 1999; 16:19. a Departments of Industrial
Engineering and Preventive Medicine (Gustafson), University of Wisconsin-Madison, Madison,
Wisconsin, USA Other panel members and staff: Farrokh Alemi, PhD, Cleveland State University, Cleveland, OH; Molly Joel Coye, MD, MPH, The Lewin Group, San Francisco, CA; Mary Jo Deering, PhD, Thomas R. Eng, Office of Disease Prevention and Health Promotion, US Department of Health and Human Services, Washington, DC; Holly Jimison, PhD, Oregon Health Sciences University, Portland, OR; Albert Mulley Jr, MD, MPP, Massachusetts General Hospital, Boston, MA; John Noell, PhD, Oregon Center for Applied Science, Inc. and Oregon Research Institute, Eugene, OR; Thomas C. Reeves, PhD, University of Georgia, Athens, GA; and Kevin Patrick, MD, MS, Graduate School of Public Health and Student Health Services, San Diego State University, San Diego, CA; and Victor Strecher, PhD, MPH, University of Michigan Comprehensive Cancer Center, Ann Arbor, MI. Address reprint requests to: Mary Jo Deering, PhD, Office of Disease Prevention and Health Promotion, US Department of Health and Human Services, 200 Independence Ave., SW, Washington, DC 20201. Copyright © 1999 American Journal of Preventive
Medicine |
Home | IHC | Publications | About SciPICH | Glossary | Feedback | Disclosure Statement | Site Map |
Comments: SciPICH@nhic.org Updated: 05/01/08