SciPICH: Interactive Health Communication, An Evidence-Based Approach to Interactive Health Communication

An Evidence-Based Approach to Interactive Health Communication: A Challenge to Medicine in the Information Age

Authors: Thomas N. Robinson, MD, MPH; Kevin Patrick, MD, MS; Thomas R. Eng, VMD, MPH; and David Gustafson, PhD; for the Science Panel on Interactive Communication and Health

Citation: JAMA. 1998; 280:1264-1269

Full Text Article - October 14, 1998

Objective.—To examine the current status of interactive health communication (IHC) and propose evidence-based approaches to improve the quality of such applications.

Participants.—The Science Panel on Interactive Communication and Health, a 14-member, nonfederal panel with expertise in clinical medicine and nursing, public health, media and instructional design, health systems engineering, decision sciences, computer and communication technologies, and health communication, convened by the Office of Disease Prevention and Health Promotion, US Department of Health and Human Services.

Evidence.—Published studies, online resources, expert panel opinions, and opinions from outside experts in fields related to IHC.

Consensus Process.—The panel met 9 times during more than 2 years. Government agencies and private-sector experts provided review and feedback on the panel's work.

Conclusions.—Interactive health communication applications have great potential to improve health, but they may also cause harm. To date, few applications have been adequately evaluated. Physicians and other health professionals should promote and participate in an evidence-based approach to the development and diffusion of IHC applications and endorse efforts to rigorously evaluate the safety, quality, and utility of these resources. A standardized reporting template is proposed to help developers and evaluators of IHC applications conduct evaluations and disclose their results and to help clinicians, purchasers, and consumers judge the quality of IHC applications.

JAMA. 1998;280:1264-1269

Advances in telecommunications and computer technologies, unimaginable a generation ago, have become routine. These technologies are changing the nature of interactions between individuals and health professionals. The following analysis results from the efforts of the Science Panel on Interactive Communication and Health (SciPICH), convened by the Office of Disease Prevention and Health Promotion of the US Department of Health and Human Services. The SciPICH includes experts in clinical medicine and nursing, public health, media and instructional design, health systems engineering, decision sciences, computer and communication technologies, and health communication. The mandate of the panel is to clarify major medical and public health issues raised by the rapidly growing field of communication technology. The SciPICH is focusing its attention on interactive health communication (IHC), which is defined as the interaction of an individual—consumer, patient, caregiver, or professional—with or through an electronic device or communication technology to access or transmit health information or to receive guidance and support on a health-related issue.

For the purposes of this article, this definition does not include electronic applications that focus exclusively on administrative, financial, or clinical data, such as electronic medical records, dedicated telemedicine applications, or expert clinical decision/support systems for physicians. Discussions of these areas are available elsewhere.[1-3] This article focuses on IHC applications, that is, the operational communication and computer software programs or modules geared toward users (ie, individuals who use IHC applications) rather than the hardware and infrastructure technologies that run or disseminate these applications.

Because IHC applications have the potential both to improve health and to cause harm, there is an opportunity and a professional responsibility for physicians and other health professionals to help ensure the quality, safety, and effectiveness of IHC applications. As an example of one of the ways this can be accomplished, the SciPICH proposes an Evaluation Reporting Template to promote standardized reporting of evaluations of IHC applications.

Benefits and Risks of Interactive Health Communication

Interactive media are changing the nature of health communication. Some health communication strategies that use "old" media, including radio, television, and printed text and pictures, have been successful in conveying information and promoting healthful behaviors.[4,5] New media, however, have potential advantages for health communication efforts including the following:

1. Improved opportunity to find[6] information "tailored" to the specific needs or characteristics of individuals or groups of users. The degree of interactivity may be limited (eg, selecting an option for specialized information) or involve a series of complicated interactions over long intervals of time (eg, monitoring of a chronic health condition or individual risk behaviors);

2. Improved capabilities of various media[6] to be combined with text, audio, and visuals and of matching specific media to the particular purposes of the interention or the learning styles of users;

3. Increased possibility for users to remain anonymous[7] by providing access to sensitive information that people may be uncomfortable acquiring in a public forum or during a face-to-face discussion.[8] Computer-based interfaces also can increase a participant's willingness to engage in frank discussions about health status, behavioral risks,[9] and fears and uncertainties[10];

4. Increased access to information and support on demand,[6,7] because these resources often can be used at any time and from numerous locations;

5. Increased opportunity for users to interact with health professionals or to find support from others similarly situated through the use of networking technologies,[6-8,11] such as e-mail, which enables direct communication between individuals despite distance or structural barriers;

6. Enhanced ability for widespread dissemination and for keeping content or functions current.[7] This can be accomplished for an expanding audience at a limited incremental cost once the necessary hardware infrastructure is in place. As these technologies become pervasive in both public and private settings, more people, including traditionally underserved persons (eg, rural, poor, disabled), may gain access to information that has been out of reach.[12]

The IHC applications include 6 specific functions, which are as follows:

1. Relay information: They can provide general or individualized health information.[13,14] Examples of these technologies include Web sites, online services, and telephone-based applications that use interactive voice response and fax-back technology.[15,16]

2. Enable informed decision making: Decision/support applications can foster communication among health care professionals and patients by helping patients understand prevention, diagnosis, or management of a health condition.[17,18] Some applications assist individuals with health care decisions, such as selecting a health care professional or a health management plan.[19,20] More sophisticated applications assist individuals in thinking through and selecting options that are consistent with their desired health outcomes.[21-23]

3. Promote healthful behaviors: Some applications promote and sustain healthful behaviors not only on an individual level but also on a community-wide level. Such applications include risk assessment and health promotion modules typically based on theories of behavioral change.[8,14,24,25]

4. Promote peer information exchange and emotional support: An increasing number of applications enable persons to discuss their specific health conditions, needs, or perspectives with others who have similar concerns. Through "virtual support communities," which are available on a wide array of medical conditions, participants may share information and provide peer and emotional support[26-34] that typically cannot be obtained from health care professionals. This phenomenon may reflect people's tendency toward socialization and is one of the most common health-related uses of the Internet.[26,35] Participants in such support networks include consumers, patients, health professionals, and other caregivers.[26,36]

5. Promote self-care: Some applications help users manage health problems without direct intervention from a health care professional and help supplement existing services.[8,37] Some consumers using these resources may have limited access to a health care professional, have a particular interest in alternative medicine, or want information on therapies that may not be available from their health care provider.

6. Manage demand for health services: This function of IHC, increasingly being used by insurers, health plans, and employers,[39] provides answers to specific health questions through computer-assisted telephone advice systems, interactive voice response systems, and/or electronic consultation with health care advisers.[38] Providing specific information, tools, and other resources to support wellness, self-care, and self-efficacy may enhance use of effective health care services and reduce unnecessary services.[8,40,41] Exchange of patient-collected data also is a component of some of these demand management programs.

Potential for Harm

Although early IHC applications were limited primarily to academic or research institutions, many systems are now directly available to the public, especially through the Internet.[15,42] The growing use of IHC applications should raise legitimate questions about their quality, cost, and potential to cause harm.[43,44] Even though some health communication interventions have been shown to be efficacious,[8,14,24,25,29,38,45] minimal research has been reported to date about the risks associated with their widespread use. Inaccurate or inappropriate health information and/or poorly designed applications can result in harmful outcomes, such as inappropriate treatment or delays in seeking necessary medical care.[46-52] Potentially misleading claims for medical products are endemic on the Internet.[53] Within a few hours, a Federal Trade Commission initiative identified more than 400 Web sites and Usenet newsgroups that contained potentially false or deceptive advertising claims for products or services for 6 diseases.[54] Similarly, although online support groups have facilitated informational sharing and support among millions of users, they are also susceptible to the proliferation of incorrect or inappropriate information.[55,56]

Moreover, misleading information can damage people's trust in their health care clinicians and prescribed treatments. Although such risks exist with most media, IHC applications must be held to a high standard because emerging research shows that people put more credibility in information from computers than from television and other media.[57] Furthermore, privacy and confidentiality may be breached. A user may have little knowledge or control over what happens to personal information he or she enters into an IHC application; it may be sold, used to discriminate against the user, or applied to a personalized marketing effort.

Many IHC applications do not have consistent standards of evaluation to enable users to compare one with another or with less expensive technologies. As with other health and medical technologies deployed prior to evidence about whether or not they work, we may be on the threshold of an era in which considerable investment in these tools precedes knowledge of their effectiveness or their impact on costs. Without necessary feedback, this is likely to result in wasted resources and delayed innovation. There is concern about the ultimate impact that the widespread deployment of IHC applications will have on the quality of health care, the clinician-patient relationship, the organization of medical systems, and the health of the public.[17,58-61]

Toward an Evidence-Based Approach to IHC

Many aspects of the development, evaluation, and dissemination of IHC applications are in need of input and guidance from the scientific and professional communities to achieve an optimal future for these technologies.[61,62,63] Reliable and valid evaluation guidelines and tools to assess and improve IHC applications need to be developed and disseminated.[59] For example, many organizations have proposed rating systems, guidance, or criteria for assessing the quality of health-related "sites" on the World Wide Web.[59,64-68] Most of the Web site rating systems in use, however, are cursory and inadequate for the task.[64] Although a particular Web site may have received awards or high ratings, such accolades may not indicate high quality. It is unknown how many health-related Web sites or other IHC applications have been independently assessed or how many have internal quality assurance and improvement policies. The current challenges in evaluating IHC applications include the following: (1) the media and infrastructure (eg, Internet, cable television, wireless technologies) that underlie these tools are in a dynamic state; (2) the applications themselves may be highly fluid because of the relative ease of changing content and function; (3) many IHC applications are used in situations in which a variety of influences on health outcomes exist, few of which are subject to easy assessment or experimental controls; (4) developers of IHC applications often lack familiarity with evaluation methods and tools; and (5) developers of IHC applications often believe that evaluation will delay development, increase "front-end" costs, and have limited impact on sales. Addressing these issues requires additional research to improve monitoring of quality and effectiveness of newer IHC technologies (eg, automated updating of Web sites) and policy and educational initiatives to promote evaluation. At the same time, existing evaluation methods can and should be adapted to assess IHC.

Proposed Evaluation Reporting Template

The SciPICH proposes an Evaluation Reporting Template for Interactive Health Communication Applications, which is based on the rationale that all applications should undergo some level of evaluation and that the nature and results of such evaluations should be available to potential users and purchasers of the application. The template is designed to assist health professionals, consumers, and purchasers in judging the appropriateness of a given IHC application for their needs and, perhaps, compare one application with another. A standardized approach to planning and reporting evaluations can also help IHC developers explore and clarify expectations of potential purchasers and users. Finally, the template can help address the consideration of commonly used, but sometimes difficult to define, concepts such as the relevance, efficiency, cost-effectiveness, and practicality of a given application. The template is designed to apply to essentially all IHC applications, regardless of the specific technologies involved, communication strategies used, or stated goal(s). The background for evidence-based approaches to the development and diffusion of IHC applications is addressed in detail elsewhere[69-72] and also will be addressed on the SciPICH Web site (URL: www.scipich.org).

The template is proposed in a spirit similar to the call for structured abstracts and for standardized reporting of results of randomized controlled trials as endorsed and required by JAMA and other journals.[73-76] However, unlike journal standards, what we propose is voluntary. All IHC stakeholders can benefit from a voluntary standard of reporting that promotes evaluation. This template and future generations of it can (1) assist developers as they plan, conduct, and report the results of their evaluations and, ultimately, help prevent the development of flawed applications and wasted resources; (2) help users determine which applications are most likely to be of benefit; (3) assist clinicians in selecting relevant applications for their patients; and (4) help purchasers and policymakers focus on the best IHC applications and strategies for investment and dissemination.

Methods

We derived our template through an ongoing consensus process. The SciPICH considered findings of published studies, online resources, and opinions of outside experts to construct an initial list. This list was formally presented for feedback from developers of IHC applications, health care industry representatives, patients, and patient advocates who attended the Partnerships for Networked Consumer Health Information conference in Rancho Mirage, Calif, on May 14, 1996. During 8 additional panel meetings over the following 2 years, feedback and suggestions for improvement were elicited from invited experts and liaisons representing developers of IHC applications, government agencies, academic researchers, health care organizations, health care consumers, and consumer advocates and comprised more than 24 federal agencies and offices and 25 nonfederal or private-sector organizations. In addition, the work of the SciPICH was presented to audiences at 3 national conferences to obtain feedback and to ensure that the template was comprehensive and general enough to accommodate various forms of IHC. Nine developers of IHC applications completed the template for the 1998 Partnerships for Networked Consumer Health Information Technology Showcase and Games, held in Washington, DC, on April 28, 1998, and in Philadelphia, Pa, from May 27 to May 28, 1998. The developers demonstrated that the templates could be completed appropriately and correctly among various disciplines. Before completing the current template, version 1.0, we received additional feedback and made further revisions.

The IHC Template

The template is divided into 4 sections. The first section focuses on identification of the developer(s), the source(s) of funding for the application, the purpose of the application, its intended audience(s), technical requirements, and issues of confidentiality—a particular concern among consumers and consumer advocates.

The second section focuses on the results of formative and process evaluations, as contributors to application design and development. These items elicit information to help potential users and purchasers judge whether the content is valid, whether the application addresses the user's needs, and whether the application was sufficiently tested so that its intended functions are ensured. In addition to providing descriptive information, this section attempts to encourage disclosure of whether and how potential users and other "experts" were involved in the application's development and how extensively the application was tested prior to release.

The third section focuses on the results of any outcome evaluations performed. The listed outcomes include those most commonly encountered, ranging from whether users like the application to whether it produces changes in morbidity or mortality, reduced costs, or organizational change. Potential outcomes are broadly defined because individual developers, users, and purchasers may have different needs and expectations. For example, while 1 developer or potential purchaser may be interested in an application that improves management of a specific chronic disease symptoms, another may be solely interested in improving patient satisfaction. The SciPICH seeks to provide a tool to promote evaluation and evaluation reporting without imposing any particular opinion about the most appropriate outcomes to assess. Classifications of evaluation designs from the US Preventive Services Task Force[77] are included to provide information relevant to the internal validity of the results (ie, the strength of evidence that the observed results are due to the intervention) and descriptions of samples are included to provide information relevant to the ability to generalize results. These considerations may be new to some IHC users or developers but are included to promote a standard that is evidence based rather than opinion based.

The final section of the template focuses on information about evaluators and funding to disclose potential biases or conflicts of interest relevant to the evaluation. The notion of disclosing conflicts of interest is complex[78] and while previously discussed models may not be sufficient to ensure consumer protection, they serve as useful precedents.The template also attempts to increase accountability for IHC applications by encouraging the disclosure of those responsible for its design and content and for evaluation. This concept is consistent with the recent initiative to make contributors to scientific studies more accountable for the content of their published work.[79,80]

Since IHC applications are diverse, evaluation targets need not include all the categories specified but should reflect the specific needs of the target audience(s) and the developer's objectives. Similarly, randomized controlled trials are not expected for all IHC applications to assess their immediate and long-term outcomes, since they are not appropriate or practical for all interventions. Rather, the panel proposes a level of evaluation that is sufficient to support the intended purposes of the application and the resources it consumes. That is, applications that have substantial potential risk or require a large investment should require a higher level of evidence, such as an appropriately designed and implemented randomized controlled trial. The level of confidence in the evidence of safety and efficacy for such interventions (eg, shared decision support applications for serious illnesses) should be "beyond a reasonable doubt." However, for interventions that have minimal potential risk and require few resources (eg, Web sites that provide general information from trusted and reliable sources), formative and process evaluations may be sufficient to provide a "preponderance of evidence" that the application would benefit users. Developers should be proactive and implement quality-control and evaluation methods throughout the development process to prevent the release of ineffective or harmful applications. In situations in which developers are unwilling to implement such controls or evaluation methods, health professionals (as individuals and through professional organizations), purchasers, consumers, and consumer advocates will need to exert pressure on them to do so.

Will developers of IHC applications voluntarily disclose information about their products? Many developers may perceive few advantages to conducting evaluations and/or disclosing results to users or purchasers. Potential incentives to spur appropriate evaluation of all IHC applications include increased demand among users and purchasers for evaluated applications, awareness of the potential for harm, and the fear of possible government regulation or legal intervention in this area. The proposed template should help guide application developers in planning and implementing appropriate evaluation methods and in assisting them in these voluntary efforts.

Refinement of Template

The template is a first step toward promoting appropriate evaluation and disclosure about IHC applications and toward helping the IHC field advance with more rapid innovation and greater benefits to individual and public health. Although the current template arose from an extensive multi-year development effort, it will need to be updated as it is used and as the field itself evolves. The effectiveness of the template must also be evaluated, a process that was started recently when the template was used by developers participating in the previously mentioned 1998 Partnerships meeting. However, rather than wait for evaluation to be complete, the current near absence of valid information with which to judge the safety and effectiveness of most available IHC applications mandates that the template be widely disseminated at this time.

Comment

Interactive health communication technologies have the potential to change dramatically both the practice of medicine and the structure of health care systems.[17,81,82] The ultimate direction these technologies take will depend partly on the response and participation of physicians and other health care professionals. If health care professionals make it a priority to understand these systems, play an active role in assessing and assuring their quality, and contribute to application development and dissemination, outcomes will more closely approximate the ones they desire. An evidence-based approach to the development and diffusion of IHC applications is necessary to ensure that these technologies benefit individual and public health. A culture of appropriate evaluation and disclosure of evaluation results is central to this process. However, if these technologies are ignored, disparaged, or treated with benign neglect, the quality of health information available to the public may suffer and harm may result. As IHC applications continue to grow, there is little doubt that consumers will increasingly turn to them for health information and support. The challenge of the next decade will be to transcend the surface appeal of these technologies and to understand and harness their power to improve the health of individuals and communities.

The views expressed in this article represent those of the individual authors and not necessarily those of the US Department of Health and Human Services or any of the authors' institutions. Stanford, Calif (Dr Robinson); Graduate School of Public Health, San Diego State University, San Diego (Dr. Patrick); Office of Disease Prevention and Health Promotion, US Department of Health and Human Services, Washington, DC (Dr. Eng); and University of Wisconsin, Madison (Dr. Gustafson).

Other members and staff of the Science Panel on Interactive Communication and Health are as follows:

Linda Adler, MPH, MA, Interactive Technologies Initiative, Kaiser Permanente, Oakland, California
Farrokh Alemi, PhD, Cleveland State University, Cleveland, Ohio
David Ansley, Consumer Reports, Yonkers, NY
Patricia Flatley Brennan, RN, PhD, School of Nursing and College of Engineering, University of Wisconsin, Madison
Molly Joel Coye, MD, MPH, The Lewin Group, Fairfax, VA
Mary JoDeering, PhD, Office of Disease Prevention and Health Promotion, US Department of Health and Human Services, Washington, D
Joseph Henderson, MD, Interactive Media Laboratory, Dartmouth Medical School, Hanover, NH
Holly Jimison, PhD, Oregon Health Sciences University, Portland
Albert Mulley, Jr, MD, MPP, Massachusetts General Hospital, Boston
John Noell, PhD, Oregon Center for Applied Science, Inc, and Oregon Research Institute, Eugene
Thomas C. Reeves, PhD, University of Georgia, Athens
Victor Strecher, PhD, MPH, University of Michigan Comprehensive Cancer Center, Ann Arbor

Acknowledgements:

We are grateful to Paul Kim, Andy Maxfield, PhD, Anne Restino, MA, and John Studach, MA; for their contributions to the panel's work. In addition, we thank the liaisons to the panel, especially the following persons who offered valuable suggestions for improving this article: Peter Abbott, MD; Loren Buhle, Jr, PhD; Kevin Burke; David Cochran, MD; Connie Dresser, RDPH, LN; Tom Ferguson, MD; William Harlan, MD; Nelson Hazeltine; Thomas Hertz, PhD; John Hoben; Don Kemper, MD; Craig Locatis, PhD; Ed Madara; Rika Maeshiro, MD; Kent Murphy, MD; Gregory Pappas, MD, PhD; Scott Ratzan, MD, MA; Samantha Scolamiero; Anna-Lisa Silvestre; Laura St. Martin, MD; and Micaela Sullivan-Fowler.

Reprints: Mary Jo Deering, PhD, Office of Disease Prevention and Health Promotion, US Department of Health and Human Services, 200 Independence Ave SW, Room 738G, Washington, DC 20201.

Corresponding author: Thomas R. Eng, VMD, MPH, Office of Disease Prevention and Health Promotion, US Department of Health and Human Services, 200 Independence Ave SW, Room 738G, Washington, DC 20201 (E-mail: teng@osophs.dhhs.gov).

References

1. Field MJ, ed. Telemedicine. A Guide to Assessing Telecommunications in Health Care. Washington, DC: National Academy Press; 1996.

2. Perednia DA, Allen A. Telemedicine technology and clinical applications. JAMA. 1995; 273:483-488.

3. Balas EA, Austin SM, Mitchell JA, Ewigman BG, Bopp KD, Brown GD. Clinical value of computerized information services: a review of 98 randomized clinical trials. Arch Fam Med. 1996; 5:271-278.

4. Flay BR. Mass media and smoking cessation: a critical review. Am J Public Health. 1987; 77:153-160.

5. Flora JA, Miabach EW, Holtgrave D. Communication campaigns for HIV prevention: using mass media in the next decade. In: Institute of Medicine. Assessing the Social and Behavioral Science Base for HIV/AIDS Prevention and Intervention. Washington, DC: National Academy Press; 1995: 129-154.

6. Harris LM, ed. Health and the News Media: Technologies Transforming Personal and Public Health. Mahwah, NJ: Lawrence Erlbaum Associates Inc Publishers; 1995.

7. US General Accounting Office. Consumer Health Informatics: Emerging Issues. Washington, DC: US General Accounting Office; July 1996. Publication GAO/AIMD-96-86.

8. Robinson TN. Community health behavior change through computer network health promotion: preliminary findings from Stanford Health-Net. Comput Methods Programs Biomed. 1989; 30:137-144.

9. Locke SE, Kowaloff HB, Hoff RG, et al. Computer-based interview for screening blood donors for risk of HIV transmission. JAMA. 1992; 268:1301-1305.

10. Gustafson D, Wise M, McTavish F, et al. Development and pilot evaluation of a computer based support system for women with breast cancer. J Psychosoc Oncol. 1993; 11:69-93.

11. Pingree S, Hawkins R, Gustafson D, Boberg E, Bricker E. Can the disadvantaged ride the information highway? hopeful lessons from a computer assisted crisis support system. J Broadcasting Electronic Media. 1996; 40:331-353.

12. Eng TR, Maxfield A, Patrick K, Deering MJ, Ratzan S, Gustafson D. Access to health information and support: a public highway or a private road? JAMA. 1998; 280:1264-1269.

13. Skinner CS, Siegfried JC, Kegler MC, Strecher VJ. The potential of computers in patient education. Patient Educ Couns. 1993; 22:27-34.

14. Strecher VJ, Kreuter M, Den Boer DJ, Kobrin S, Hospers HJ, Skinner CS. The effects of computer-tailored smoking cessation messages in family practice settings. J Fam Pract. 1994; 39:262-270.

15. Wingerson L, Simon K, Northrup L, Restino A, eds. Patient Resources on the Internet: 1997 Guide for Health Care Professionals. Wingerson L, Simon K, Northup L, Restino A, eds. New York: Faulkner & Gray Inc; 1997.

16. Buhle EL Jr, Goldwein JW, Benjamin I. OncoLink: a multimedia oncology information resource on the Internet. Proc Annu Symp Comput Appl Med Care. 1994; XX:103-107.

17. Office of Technology Assessment. Bringing Health Care Online: The Role of Information Technologies. Washington, DC: US Government Printing Office; September 1995. Publication OTA-ITC-624.

18. Taylor E. Interactive technology enters realm of cancer education. J Natl Cancer Inst. 1994; 86:1272-1274.

19. Firshein J. US physicians' malpractice data goes on Internet. Lancet 1997; 349:1155.

20. Meyer H. Information systems: surfing the Net for a health plan. Hosp Health Network. 1996; 70:37-38.

21. Wennberg J. Shared decision making and multimedia. In: Harris LM, ed. Health and the New Media: Technologies Transforming Personal and Public Health. Mahwah, NJ: Lawrence Erlbaum Associates Inc Publishers; 1995: 109-126.

22. Barry MJ, Fowler FJ Jr, Mulley AG Jr, Henderson JV Jr, Wennberg JE. Patient reactions to a program designed to facilitate patient participation in treatment decisions for benign prostatic hyperplasia. Med Care. 1995;33: 771-782.

23. Gustafson D, Hawkins R, Boberg E, et al. Impact of patient centered computer-based health information and support system. Am J Prev Med. In press.

24. Campbell MK, DeVellis BM, Strecher VJ, Ammerman AS, DeVellis RF, Sandler RS. Improving dietary behavior: effectiveness of tailored messages in primary care settings. Am J Public Health. 1994; 84:783-787.

25. Krishna S, Balas EA, Spencer DC, Griffin JZ, Boren SA. Clinical trials of interactive computerized patient education: implications for family practice. J Fam Pract. 1997; 45:25-33.

26. Ferguson T. Health Online: How to Find Health Information, Support Groups, and Self-Help Communities in Cyberspace. Reading, Mass: Addison-Wesley Publishing Co; 1996.

27. Peters R, Sikorski R. Digital dialogue: sharing information and interests on the Internet. JAMA. 1997; 277:1258-1260.

28. Gustafson DH, Bosworth K, Hawkins RP, Boberg EW, Bricker E. CHESS: a computer-based system for providing information, referrals, decision support and social support to people facing medical and other health-related crises. Proc Annu Symp Comput Appl Med Care. 1992: 161-165.

29. Gustafson DH, Hawkins RP, Boberg EW, Bricker E, Pingree S, Chan CL. The use and impact of a computer-based support system for people living with AIDS and HIV infection. Proc Annu Symp Comput Appl Med Care. 1994: 604-608.

30. Feenberg AL, Licht JM, Kane KP, Moran K, Smith RA. The online patient meeting. J Neurol Sci. 1996; 139(suppl):129-131.

31. Weinberg N, Schmale J, Uken J, Wessel K. Online help: cancer patients participate in a computer-mediated support group. Health Soc Work. 1996; 21:24-29.

32. Fernsler JI, Manchester LJ. Evaluation of a computer-based cancer support network. Cancer Pract. 1997; 5:46-51.

33. Gleason NA. A new approach to disordered eating—using an electronic bulletin board to confront social pressure on body image. J Am Coll Health. 1995; 44:78-80.

34. Bluming A, Mittelman PS. Los Angeles Free-Net: an experiment in interactive telecommunication between lay members of the Los Angeles community and health care experts. Bull Med Libr Assoc. 1996; 84:217-222.

35. Scolamiero SJ. Support groups in cyberspace. MD Comput. 1997; 14:12-14.

36. Brennan PF, Moore SM, Smyth KA. The effects of a special computer network on caregivers of persons with Alzheimer's disease. Nurs Res. 1995;44: 166-172.

37. Ferguson T. Health care in cyberspace: patients lead a revolution. Futurist. 1997; 31:29-33.

38. Balas EA, Jaffrey F, Kuperman GJ, et al. Electronic communication with patients: evaluation of distance medicine technology. JAMA. 1997; 278:152-159.

39. Mullich J. Patient heal thyself. Healthcare Inform. July 1997: 27-32.

40. Fries JF, Koop CE, Beadle CE, et al. Reducing health care costs by reducing the need and demand for medical services. N Engl J Med. 1993; 329:321-325.

41. Vickery DM. Demand management, self-care, and the new media. In: Harris LM, ed. Health and the New Media: Technologies Transforming Personal and Public Health. Mahwah, NJ: Lawrence Erlbaum Associates Inc Publishers; 1995: 45-63.

42. Kieschnick T, Adler L, Jimison H. 1996 Health Informatics Directory. Baltimore, Md: Williams & Wilkins; 1996.

43. The web of information inequality [editorial]. Lancet. 1997; 349:1781.

44. Consumers Union. Finding medical help online. Consumer Rep.1997; 62:27-31.

45. Shiffman S, Gitchell J, Strecher V, et al. Real-world efficacy of computer-tailored smoking cessation materials as a supplement to nicotine replacement. Paper presented at: the 10th World Conference on Tobacco or Health; August 28, 1997; Beijing, China.

46. Weisbord SD, Soule JB, Kimmel PL. Poison on line—acute renal failure caused by oil of wormwood purchased through the Internet. N Engl J Med.1997; 337:825-827.

47. Food and Drug Administration. FDA warns consumers on dangerous products promoted on the Internet [press release]. Washington, DC: Food and Drug Administration; June 17, 1997.

48. Keoun B. Cancer patients find quackery on the Web. J Natl Cancer Inst. 1996; 88:1263-1265.

49. Saksena S, Nickelson DE. Medical information and disinformation: the perils of a new era. Pacing Clin Electrophysiol. 1995; 18:2216-2217.

50. Impicciatore P, Pandolfini C, Casella N, Bonati M. Reliability of health information for the public on the World Wide Web: systematic survey of advice on managing fever in children at home. BMJ. 1997; 314:1875-1879.

51. Scolnick A. WHO considers regulating ads, sale of medical products on Internet. JAMA. 1997; 278:1723-1724.

52. Goldwein JW, Benjamin I. Internet-based medical information: time to take charge. Ann Intern Med. 1995; 123:152-153.

53. Bower H. Internet sees growth of unverified health claims. BMJ. 1996; 313:381.

54. North American Health Claim Surf Day targets Internet ads hundreds of e-mail messages sent [press release]. Washington, DC: Federal Trade Commission; November 5, 1997. Available at: http://www.ftc.gov/opa/9711/hlthsurf.htm. Accessed August 28, 1998.

55. Bulkeley WM. E-mail medicine: untested treatments, cures find stronghold in on-line services. Wall Street Journal. February 27,1995:A1

56. Gray D. Faking pain and suffering in Internet support groups. New York Times. April 23,1998:E1, E7.

57. Hawkins R, Gustafson DH, Chewning B, Bosworth K, Day P. Interactive computer programs as public information campaigns for hard-to-reach populations: the BARN Project example. J Commun. 1987; 37:8-28.

58. Blumenthal D. The future of quality measurement and management in a transforming health care system. JAMA. 1997; 278:1622-1625.

59. Silberg WM, Lundberg GD, Musacchio RA. Assessing, controlling, and assuring the quality of medical information on the Internet: caveant lector et viewor—let the reader and buyer beware. JAMA. 1997; 277:1244-1245.

60. Health Commons Institute. Shared Decision Making and New Information Technologies. Transforming the Health Care System: An Invitational Conference for Policy Leaders. Portland, Me: Health Commons Institute; 1994.

61. Kassirer JP. The next transformation in the delivery of health care. N Engl J Med. 1995; 332:52-54.

62. Sonnenberg FA. Health information on the Internet: opportunities and pitfalls. Arch Intern Med. 1997; 157:151-152.

63. Thomas JR. Danger on the information superhighway. Mo Med. 1996; 93:742.

64. Jadad AR, Gagliardi A. Rating health information on the Internet: navigating to knowledge or to Babel? JAMA. 1998; 279:611-614.

65. Health Information Technology Institute, Mitretek Systems. Criteria for assessing the quality of health information on the Internet. Available at: http://www.mitretek.org/hiti/showcase/index.html. Accessed January 27, 1998.

66. Health on the Net Foundation. Health on the Net code of conduct (HONcode) for medical and health web sites. Available at: http://www.hon.ch/HONcode/Conduct.html. Accessed August 28, 1998.

67. Murray PJ, Rizzolo MA. Web site reviews and evaluations. Nursing Standard Online. Available at: http://www.nursing-standard.co.uk/vol11-45/ol-art.htm. Accessed September 16, 1998

68. Wyatt JC. Commentary: measuring quality and impact of the World Wide Web. BMJ. 1997; 314:1879-1881.

69. Gustafson DH, Robinson TN, Ansley D, Adler L, Brennan PF, for the Science Panel on Interactive Communication and Health. Consumers and evaluation of interactive health communication applications. Am J Prev Med. 1999; 16(1):23-29.

70. Henderson J, Noell J, Reeves T, Robinson TN, Strecher V, for the Science Panel on Interactive Communication and Health. Developers and evaluation of interactive health communication applications. Am J Prev Med. 1999; 16(1):30-34.

71. Jimison H, Adler L, Coye M, Mulley A Jr, Eng TR, for the Science Panel on Interactive Communication and Health. Health care providers and purchasers and evaluation of interactive health communication applications. Am J Prev Med. 1999; 16(1):35-42.

72. Patrick K, Robinson TN, Alemi F, Eng TR, for the Science Panel on Interactive Communication and Health. Policy issues relevant to the evaluation of interactive health communication applications. Am J Prev Med. 1999; 16(1):16-22.

73. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA. 1996; 276:637-639.

74. Rennie D. How to report randomized controlled trials: the CONSORT statement. JAMA. 1996; 276:649.

75. Haynes RB, Mulrow CD, Huth EJ, Altman DG, Gardner MJ. More informative abstracts revisited. Ann Intern Med. 1990; 113:69-76.

76. Taddio A, Pain T, Fassos FF, Boon H, Ilersich AL, Einarson TR. Quality of nonstructured and structured abstracts of original research articles in the British Medical Journal, the Canadian Medical Association Journal and the Journal of the American Medical Association. CMAJ. 1994; 150:1611-1615.

77. US Preventive Services Task Force. Guide to Clinical Preventive Services. 2nd ed. Washington, DC: US Dept of Health and Human Services; 1996.

78. Rodwin MA. Physicians' conflicts of interest: the limitations of disclosure. N Engl J Med. 1989; 321:1405-1408.

79. Rennie D, Yank V, Emanuel L. When authorship fails: proposal to make contributors accountable. JAMA. 1997; 278:579-585.

80. Susser M. Authors and authorship—reform or abolition? Am J Public Health. 1997; 87:1091-1092.

81. Council on Competitiveness. Highway to health: transforming U.S. health care in the information age. Washington, DC: Council on Competitiveness; 1996.

82. Coiera E. The Internet's challenge to health care provision. BMJ. 1996; 312:3-4.

Comments: SciPICH@nhic.org Updated: 05/01/08