Evaluation of IHC
Consumers & IHC
Policy Issues Relevant
Providers, Purchasers & IHC
Wired for Health and Well-Being: The
Emergence of Interactive Health Communication
Editors: Thomas R. Eng, David H. Gustafson
Suggested Citation: Science Panel on Interactive
Communication and Health. Wired for Health and Well-Being: the Emergence of Interactive
Health Communication. Washington, DC: US Department of Health and Human
Services, US Government Printing Office, April 1999.
Download in PDF format: [Entire Document] [References]
for Key Stakeholders
The Panel has identified several stakeholder groups that
need to participate in IHC application development, evaluation, and quality assurance
activities if meaningful evolution and quality improvement of IHC is to occur. These
include consumers (including patients, families, caregivers, and communities), IHC
developers, purchasers and investors, health professionals, and policymakers, all of whom
have related roles.7 Developers of IHC
applications are a vital link between the consumer and various sources of information and
support. They also have ultimate control of evaluation and quality assurance aspects of
these interventions, and are clearly influenced by the needs of the purchasers and users.
Potential health care purchasers, including health plans and employers, determine whether
IHC applications are implemented for their plan members or employees and directly
influence the financial success of commercial developers. Investors determine which
commercial applications receive funding to reach the marketplace. Health care and public
health professionals often mediate the use of these applications with patients, and are
involved in the development of IHC applications. Policymakers influence the climate in
which stakeholders make decisions about the development, use, or purchase of IHC
Understanding the various concerns, motivations, and
potentially different perspectives of all stakeholders may be valuable in establishing
collaborative efforts and in arriving at a consensus on appropriate directions and policy
for IHC. In this chapter, the Panel outlines some of the major IHC-related issues among
As mentioned previously, many types of developers create
applications. Although their background and available resources for product development
and evaluation may vary widely, they face similar obstacles in producing effective and
useful programs (Jimison et al., 1999; Henderson et al., 1999). Many of these barriers
have their origins in the competitive marketplace. The rapidly shifting nature of
technology and the marketplace for IHC makes economic success difficult for many
Balancing Science and the Realities of the Marketplace
How do developers balance their responsibility to use
scientific principles in product development with the realities of the marketplace? IHC
developerscommercial or nonprofit, large or smallare typically under
tremendous pressure to deliver their products on time and within budget. Developers are
pressed to produce high-quality applications in an extremely competitive environment with
limited resources. This is the norm for the software industry. The emphasis on releasing
products to the market as quickly as possible often results in applications that have
fewer features than originally envisioned or ones that have not been fully tested and
evaluated. The competitiveness of the market, from some developers perspectives,
precludes them from investing appropriate time and resources to fully evaluate a system
Investing in Marketing Versus Evaluation
What is the appropriate balance between investing in
marketing and evaluation activities? Some developers believe that aggressive marketing
efforts, rather than positive evaluation results, ultimately drive sales and commercial
success. In some larger companies and organizations, decisions about evaluation activities
may be made by managers and executives, and developers may be frustrated by the need to
convince these decisionmakers about the importance of evaluation.
Obtaining Funding for Development and Evaluation
How do developers get adequate resources for application
development? Venture capitalists and large corporations may invest in commercial
developers. Government or private foundation grants are often the primary sources for
smaller developers. This is particularly true for developers who are part of nonprofit
organizations. Competing for capital is of utmost concern for many developers. As a
result, many are reluctant to collaborate or share their ideas with potential competitors.
A list of potential resources for locating funding for IHC development is presented in Appendix C.
The scenario in Box V-1 illustrates some of the difficult
issues that developers face when they try to balance time-to-market considerations with
their responsibilities for quality assurance and evaluation.
A Developers Dilemma
The PDM company was formed by the Jones brothers, Paul (a
programmer), Bob (a business school professor), and Greg (a physician), in response to the
news that their father had been diagnosed with prostate cancer. As their father informed
each of them about his condition, he also expressed his confusion and doubt about what to
do. Each brother commiserated with him about the difficulty of working through the
available options for prostate cancer treatment. One evening not long afterwards, the
brothers were having dinner together and talking about their father. They agreed that men
like him needed a program to help sort through their options in a way that would help them
make the best possible decisions during a difficult time. Collectively, they felt that
their unique assemblage of talents gave them a special understanding of what was needed
and how to do it. As a result, they created the PDM Company (for prostate decisionmaking)
with a goal of creating (and selling) an interactive application that would help men
decide about treatment optionsthus making some money while meeting a real need.
With expert medical guidance recruited by the physician
brother and decisionmaking expertise from the business school brother, the programmer
began creating their flagship program. Due to major disagreements among the medical
experts about which treatment options actually were most desirable, the project schedule
started slipping. Meanwhile, Bob was holding focus groups to determine what men thought
would be most useful and to get their reactions to mock-ups of the program. Enthused by
the reactions of the men he spoke with, Bob began encouraging Paul to expedite development
of the program. Greg was very concerned about the potential to lead men into the wrong
decisions. He was worried about both legal liability and the moral/ethical issues of
encouraging specific decisions, even inadvertently. Bob, an expert in marketing, explained
how easy it is to influence decisions while providing seemingly neutral information. Paul
suggested an extensive program evaluation to make sure the program actually helped men
make the right decisions. They all agreed this sounded like a good idea.
Paul was coordinating the production of multimedia assets
(graphics, audio files, and video) as well as programming. The video subcontractor kept
delaying production and Paul could not finish the programming without the assets. Then, a
major new microcomputer operating system (OS) version was announced and Paul realized his
program would need to be changed if it was to run under the new OS. Each delay meant more
days and weeks of effort, rent, subcontracts running over budget, and other expenses.
Money was becoming a real worry. Finally, Paul announced that a pre-beta version was
available for testing. Bob tried to run it on his office computer but he did not have the
right multimedia software installed. He gave the program to others but all of them ran
into some type of bug or installation glitch. Paul tried to create new versions for each
problem that was encountered. Bob began to worry about just what types of computers could
run the program. Was the installed base a big enough market? Maybe they should simplify
the program so that it would run on more computers or make it a Web-based application?
Paul was beginning to feel stressed out. Greg kept asking about the big delay.
Not much later, Paul produced a "final candidate"
program and they began pilot testing with five men recently diagnosed with prostate
cancer. Despite some glitches, the program worked pretty well and the men generally liked
it. They felt it definitely was useful and had helped them make good treatment decisions.
Greg was called in and the three Jones brothers began discussing an evaluation that would
include enough men to have a statistically meaningful sample and would use a randomized
control trial design. Greg and Paul agreed that anything less would not really tell them
whether the program worked well or lead to any bad decisions. It would take about six
months, but it was all going to be worth the effort. The euphoria of being so close to
completion lasted three days. Then, Bob was at a conference where he overheard two men
talking about the prostate decisionmaking program they had recently seen demonstrated.
"Very slick," said one man. "Yes, theyve got a winner Id
guess," added the other. "Do you really think they can ship it within the next
three months like they claimed?" asked the first. "I think so," was the
reply. Bob ran to a phone and called Paul. To his surprise, Greg answered and immediately
interrupted his brother to announce that Paul had completed the "final" program.
"Great news!" he added. "Were ready to begin the randomized clinical
trial. My friends, the ones with that huge urology practice I told you about, have agreed
to start referring patients Monday." Bob brusquely replied, "Forget the trial,
weve got to ship right away or well get scooped! I just heard about another
program coming out." What ensued was a bitter argument about whether or not they had
sufficient data to begin shipping the program.
"We can get data from actual users," said Bob.
"Without random assignment, we cannot be sure what that kind of data would
mean," said Greg. Paul added, "I dont want to do any harm but I dont
really see how we could." Greg explained how leading a man into the wrong decision
could mean an unnecessary end to his sex life, on top of urinary incontinence. "Think
about no more intercourse and being incontinent the rest of your life," Greg said.
"But we probably have been more careful about creating a balanced program than our
competitors. If we dont go to market now we may not make anything out of this!"
Bob said with emotion. "I dont think we should market it unless we have data to
back up its efficacy and safety," said Greg. "But thats what the pilot
data says, isnt it?" asked Paul. "Its not conclusive, there were
only five men," said Greg. "Yes, but its the best we can do for now,"
added Bob. "We can run the trial while we start marketing, cant we?" said
Paul. Silence followed as each brother thought about the choices and implications. Paul
spoke, "Well, what about it? We could start the trial as planned and Bob can start
the marketing plan. By the time we ship a significant number, we should have preliminary
data from enough patients to see if there are any major problems. We will keep analyzing
the trial data as they come in and if we see any problems we yank the program and fix
whatever has come up." Greg shook his head, "If we cause a problem Id have
a hard time living with it. First we test, then we ship." Bob looked glum. He said,
"You do understand we could lose everything weve put into this?" Greg
sighed and said softly, "Yes, I do."
The next three months were tense ones for PDM. After
enrolling the first 20 men, they discovered that about half of the users were missing a
critical section of the program due to a confusing interface element. The experienced
computer users understood the interface metaphor Paul had programmed, but the others
missed it. It was a serious problem, but easily fixed. Moreover, their competitor did
release a comparable program. They heard about it while reading an industry newsletter.
There was a review of their competitors program by a physician. Their dismay turned
to glee as they read the review. The gist of the review was that the program had offered
terrible advice, "evidently based on faulty algorithms," concluded the reviewer.
The reviewer strongly suggested that the program be pulled off the market before the
developers (and any clinics making it available to patients) faced lawsuits. The brothers
carefully reviewed their own program to make sure it did not have similar problems. Once
the randomized clinical trial had produced enough data, Greg gave a peer-refereed
presentation of the scientific data at a national meeting. A medical director for a very
large HMO approached Greg after the talk and asked if he was prepared to negotiate for
"the purchase of a rather large number of copies to be used across all of their
- The first guiding principle must be: "Do no harm."
- Two key ingredients are content expertise and accurate
knowledge of the target population (i.e., their desires, capabilities, and needs). Both
are necessary for optimal program development.
- Key design decisions (e.g., specific behavioral goals,
target platform, structure) must be made carefullyand early in the development
processto avoid wasted efforts and costly delays.
- Adequate testing of the programming and the content are
critically importantboth to ensure efficacy and to avoid liability.
- Premature marketing can result in problems worse than
reduced market share.
- Good efficacy data can be an important sales factor.
Health care and public health professionals have a vested
interest in offering patients and consumers reliable and useful health information because
providing access to quality health information has the potential to both improve the
quality of health care and lower costs (Jimison et al., 1999). The challenge for health
professionals and purchasers is to identify systems that are effective and warrant further
investment of resources, time, and integration into clinical practice and preventive
programs. In a shifting paradigm where patients may have just as much or more access to
the latest health information, clinicians and other health and safety professionals will
be challenged to find ways to become involved in the health information-seeking process
and be able to add value to this process.
Employee health and safety directors, occupational health
nurses, industrial hygienists, and others in workplace settings, are examples of health
and safety professionals who could benefit from IHC applications. The recommendations for
reducing the risks associated with workplace hazards often require the development and
implementation of complex engineering and administrative controlsmeasures which
require expertise and often need to be tailored to individual work settings. As a result,
controlling occupational health and safety hazards can be costly and time-consuming for
employers and workplace health and safety professionals. IHC has the potential to reduce
costs and occupational risks by alerting health and safety professionals about newly
identified hazards, and by providing accessible, comprehensive, and tailored information
for developing and implementing controls.
Impact on Role of the Clinician
How will IHC impact on the traditional role of clinicians?
How should IHC applications be integrated into clinical practice? It is unclear exactly
how IHC will change the role of clinicians but it is likely to be a gradual rather than an
abrupt process. As discussed previously, some health care professionals view IHC as a
threat to their traditional relationship with patients. Others become discouraged at the
thought of managing patients who are avid health information seekers given the confines of
a 15-minute appointment. These concerns are somewhat magnified for some clinicians because
they are occurring in the context of the growth of managed care. Regardless of whether
their practice setting supports use of IHC, clinicians can make use of applications to
improve patient care. For example, referring patients to quality Web sites for additional
information or to well-regarded online support groups for peer support may help supplement
the information and support provided during an office visit.
A related concern for clinicians is whether they will be
reimbursed for their activities as information intermediaries as described in the next
chapter. In many cases, administrators rather than individual clinicians ultimately make
decisions about integrating IHC into clinical practice, particularly if they involve the
purchase of hardware or software. Administrators, therefore, have a key role in successful
Development of New Skills and Training
A 1997 survey among a nationally representative sample of
1,905 physicians showed that 42 percent use a computer for personal or professional
practice purposes (Trotto, 1998). It is clear that as the roles of health professionals in
the context of IHC evolve, additional training in new areas of responsibility will be
required. These areas may include both technology-related skills (e.g., computer use,
information searching and information evaluation skills) and communication-related skills
(e.g., facilitating shared decisionmaking, interfacing with proactive information-savvy
patients). Such training will need to be available during both health professional student
and post-graduate training. In addition, IHC-related technology is rapidly becoming an
essential tool for educating health professionals.
The scenario in Box V-2 illustrates several issues that may
challenge clinicians in integrating IHC applications into their practice.
The Proactive Health Care Professional
Dr. Emily Chen is a senior physician with more than 20
years of experience in a large oncology group practice. During the last year or so, the
clinicians in the practice have noticed a dramatic increase in the number of people who
have been coming in with realms of printouts from Medline searches and other online
resources. The clinical staff have been overwhelmed by these patients because much
clinician time is required to address all their questions and concerns. Some of the
clinicians also believe that most of the time spent with these patients is spent
correcting inaccurate information or explaining why the information is not relevant to the
patients situation. As a result, these clinicians have been discouraging their
patients from using online health resources. In addition, because the practice receives a
fixed pre-negotiated reimbursement rate from the insurance company for most services
regardless of the actual time involved, the practice administrator has strongly encouraged
time limitations on many types of visits.
Dr. Chen has been reluctant to accommodate the more
information-savvy patients in the practice and will sometimes only brusquely answer these
patients questions. Then, one day, a long-time patient of Dr. Chens abruptly
announces that, while he respects the quality of clinical care at the practice, his
information and support needs are not being met. He is switching to a new practice across
town because they not only answer all his questions, but they actually encourage him to
conduct online research about his illness. In fact, this practice encourages him to e-mail
the clinic staff with any questions he might have at any time. Asking around, Dr. Chen
finds out that several other clinicians in the practice have had similar experiences
recently. Concerned, Dr. Chen asks the practices accountant to review the clinic
logs. She finds out that there has been a significant drop in the number of new patients
coming into the practice during the last year, and an alarming number of long-time
patients have elected to leave the practice.
At their next meeting, the practice management team decides
to conduct a survey to assess the information and support needs of patients. The team
learns that an overwhelming number of patients are dissatisfied with the information and
support they receive from their clinician. It becomes clear to the team that the policy of
not fully supporting patients who use the Internet to access health information and
support is shortsighted and potentially ill-fated. Using the survey results, the team sets
out to develop a comprehensive "patient information support" policy with input
from an IT consulting company.
The team decides that the new policy would revolve around a
proactive and comprehensive approach to managing patient information and support needs.
Their goal is that all patients will be fully supported in seeking information and
support, and fully satisfied with their clinical encounters.
The management team quickly realizes that, although most of
the office staff have used personal computers, only a few of them have experience in
searching for health information on the Internet. They hire a local medical librarian to
hold a series of in-service training sessions on locating and evaluating health
information on the Internet. The librarian also identifies several well-respected online
cancer support groups that the clinicians start to use to get a sense of what questions
and concerns are circulating among cancer patients and caregivers. The nurses,
technicians, and other office staff are included in these training sessions because they
will assume much of the responsibility for supporting patient information and support
needs under the new policy.
A subgroup of the team is tasked with building the Web site
for the practice. They start by locating and conducting reviews of relevant,
health-information Web sites. This group carefully evaluates these Web sites using
standardized evaluation criteria and creates links to sites that seem to be of highest
quality and relevance to the practices patient population. Sites that inform and
guide cancer patients to appropriate clinical trials of new therapies are highlighted.
They also contract with a regional cancer center to use some of their online content and
adapt it to their patient population. To complement the health-information resources on
their site, they purchase licenses for several risk-assessment and clinical
decision-support tools from a university group after examining several peer-reviewed
research articles documenting the effectiveness of these tools. In addition, during the
course of the next few months, each clinician submits several "frequently asked
questions" from patients and the answers are posted on the practices Web site.
All of the clinicians are now available by e-mail to handle nonemergency questions.
Patients really like this because they no longer feel that they have to think of all their
questions during their visits, and many have been reluctant to call their clinician for
The team also wants to make sure that all patients have
some access to their Web site. On a trial basis, several interactive kiosks are installed
in the practice waiting room. An "information assistant" is available to show
patients how to use the system. New patients are instructed to use the kiosks to provide a
detailed medical history so that the clinicians can focus on their responses rather than
administering the history during their visit. Because the kiosks are linked to the
Internet, patients also can access the online resources on the practice Web site while
they wait. The office finds that the kiosks are well received, and that they actually free
up time that the clinicians would have otherwise spent interviewing the patients and
explaining basic concepts about cancer.
Several months after implementing their new "patient
information support" policy, the team implements structured and open-ended surveys of
patients to get their feedback on the new approach. They find that most patients are
satisfied with the information and support they receive through the practice. They use the
survey data to further refine their quality improvement initiatives. Dr. Chen and the
other clinicians also have the following observations: 1) most patients seem to be well
informed about their illnesses and their questions during visits are better focused; 2)
many patients have become active participants in their care and seem to be more satisfied
with their care; and 3) the number of patients in the practice has increased.
- Clinicians have important roles in helping patients locate,
assess, and interpret health information.
- Many health professionals will need training and
skills-building in health information technology to adequately address the information and
support needs of their patients.
- A proactive and comprehensive approach to addressing patient
information and supporting needs can help improve patient satisfaction.
- Encouraging patients to use online resources does not
necessarily increase the amount of time that clinicians have to spend with patients during
Currently, the major purchasers of IHC applications are
health plans, other health care organizations, and large employers. Emerging purchasers
include government agencies, schools, and voluntary health organizations.
Direct-to-consumer applications are becoming increasingly common. Health plans may be
primarily interested in the potential of IHC to promote healthy behaviors and manage
demand for services. Many employers consider purchasing an IHC application to help
decrease health care insurance premiums and to reduce losses in worker productivity due to
illness. Ultimately, both types of purchasers want to improve satisfaction among members
or employees and lower health care costs. After purchasers make a purchasing decision,
they then have responsibility for implementation and evaluation of the selected product. A
checklist of evaluation-related questions for potential purchasers is presented in Appendix D.
Selecting and Evaluating Applications
How should purchasers compare competing products and what
outcomes should they consider besides price? Major outcomes of interest to most purchasers
include cost (i.e., all expenditures potentially influenced by the use of the system) and
quality, including clinician and patient satisfaction, increased knowledge and healthy
behaviors, improved health status outcomes, process control, improved access to and more
appropriate utilization of health care services, and concordance between utilization and
expressed preferences (Jimison et al., 1999). One outcome of importance to health plans is
improved market growth with the use of IHC applications that may increase client
satisfaction, retention, and new enrollments. Cost-savings may result from applications
that facilitate disease management, self-care, and self-triage. Purchasers should keep in
mind, however, that many individuals are much more interested in outcomes that reflect
improvements in health, service, convenience, social support, and general ease of use,
rather than cost-effectiveness and improved market share. Therefore, the former types of
outcomes should also be important criteria for purchasers.
Strength of evidence is also important to consider when
examining evaluation results (Robinson et al., 1998). The results should show internal
validity (this is higher with randomized controlled trials, lower with observational
studies), statistical significance (higher with larger samples and/or larger effect size),
and clinical or system significance (sufficiently large effect). In addition, purchasers
may wish to consider how applicable evaluation results are to their own circumstances and
population. For example, in the case of a health plan, comparison of its patient
demographics and organizational characteristics in terms of culture, incentives, and
willingness to change or adapt, may shed light on whether evaluation findings are
"generalizable" to their circumstances. As with all interventions, it is
important to document whether the reported effect of a system is actually obtained in
routine use among the purchasers population.
Supporting Evaluation of IHC
From the purchaser perspective, the importance of
supporting the evaluation of IHC applications is based on both quality-of-care and
business criteria. Evaluation information can be used to identify more effective products,
and the quality of health care will be improved if patients are using more effective
systems for their health care decisionmaking. Without purchaser demand for evidence, there
is little incentive for developers to evaluate their systems and evaluation information on
competing IHC applications would be limited. Some purchasers, however, may not be
concerned about demanding health status outcome evaluations because they are more
interested in providing IHC applications as a marketing mechanism or as a means for
improving member or employee satisfaction rather than in the actual impact of the product
on health status. Appropriate evaluation results, however, would provide information about
user satisfaction rates that could predict satisfaction in the purchasers
Developing and Implementing IHC
What types of applications are most appropriate for members
and employees? In considering whether to implement IHC in a health care organization,
administrators will be concerned about several major issues. First and foremost may be
whether IHC can address the needs of their members or employees. Addressing this question
requires information about the populations health information and support needs, and
an understanding of realistic outcomes of the application. Another major consideration is
whether implementing a technology-mediated application adds real value compared with a
more traditional method of providing information and support. Administrators also need to
determine how to relate the IHC application to clinical information systems and to care
delivery. Another important issue they face is whether to purchase external systems or
develop applications in-house.
The scenario in Box V-3 describes several major issues
related to how health care organizations may approach the implementation of IHC
applications into their organization.
Implementing IHC in a Health Plan
Janice Rodriguez, an accomplished project manager, has just
been recruited from an interactive health software company to be the regional chief
information officer for a large progressive health plan with several facilities in the
area. The CEO of the health plan wants the organization to be on the
"cutting-edge" of using IHC applications to improve member health and to reduce
costs of delivering care.
Eager to please, Janice embarks on an ambitious plan to
eventually implement a comprehensive suite of Web-based applications to provide seamless
health information and support for members, allow electronic member transactions, provide
clinical decision support for members and clinicians, and facilitate electronic
communication between clinicians and members. For the first phase of her plan, she focuses
on selecting and implementing a shared decision-support application for members and
Janice ponders whether to outsource the development of the
application or to do it in-house. After an assessment of her departments resources
and an environmental scan of available applications, she decides on the former option. She
surmises that, with a pilot project, it is better to minimize uncertainty. The health
plans IT department has some expertise in general Web site development, but they are
unfamiliar with decision-support applications. If she outsources to a company with an
established product, she is assured of prompt delivery and can more easily predict costs.
In addition, she is worried about potential liability issues. Relying on an established
and rigorously evaluated application, she concludes, may help in that regard.
She sends out notice of what she is looking for to her
former company and a few other contacts. Within a week, Janice is deluged with calls from
more than a dozen companies that want to sell their product. Each of them claims that
their product is of high quality and very effective.
Janice understands that she needs to base her decision on
more than surface appeal or which salesperson can do a better presentation, because her
boss is looking for bottom-line benefits to the health plan, such as increased member
satisfaction and retention. She promptly assembles a team of people from the IT and
clinical research departments with experience in evaluation. They review the literature on
evaluation of IHC, talk to colleagues in other plans, and brainstorm, coming up with a
list of questions for themselves and prospective vendors. The questions include: What are
reasonable objectives for such an application within the context of the organization? Was
the application evaluated with regards to these objectives, and what were the results? Has
it been shown to be cost-effective? Is the product flexible and adaptable to the plans
After reviewing additional information from the vendors,
the team decides to purchase a hundred licenses of the application from a company that
presented very convincing data on effectiveness which was published in a respected
scientific journal. Janice and the team make plans to deploy the application in the
primary care department in one of the plans facilities because their medical
director happens to be a "techie" and is very enthusiastic about the program.
Things look positive; the application runs well with their existing clinical information
system, the training session is well attended, and several of the primary care staff say
that they are impressed with the technology.
Janice and her team check in weekly to see how things are
going. After a month, only a small proportion of the clinicians has used the application
with their patients. Janice interviews the staff. "I dont want to support using
something that may eventually replace me," says one physician. "I like the
program but I dont have time to use it during my appointments
I am also afraid
that it will actually increase the amount of time I have to spend with some
patients," says another. Yet a third comments, "Its too much of a hassle
for me to use it because it crashes my personal scheduler program when I do. When I ask
our IT administrator to help, he says he will not support it because he is too busy
troubleshooting the clinical information system that the plan just purchased last
Janices team realizes that they have formidable
implementation problems on their hands, so they start again from square one, and recruit
more troops. They fan out, interview additional stakeholders, hold extensive "town
meetings," and form a plan-wide implementation team.
The team is divided into a number of smaller groups that
are dedicated to a specific aspect of the implementation process. The development team
takes responsibility for assessing the value of available applications, rating them
according to compatibility with the health plan, and modifying and designing interfaces to
reflect the organizational culture. The implementation working group will oversee the
process of defining constituencies and determining how best to represent their needs in
the dissemination process. The operations working group considers how the application will
be merged with existing clinical care, how quality will be assessed, and how the
application will be maintained. The evaluation group defines how the impact of the program
will be determined, designs usability testing protocols, analyzes potential return on the
investment, and determines how study results will be disseminated. The technology working
group hammers out the "nuts and bolts" of the installation and designs the
architecture to support the smooth operation of the application. Finally, a planning
working group is formed to examine future opportunities and directions for new
applications and development.
Meanwhile, a communication plan is set into action so that
support for the program can be gathered from the stakeholders and opinion leaders
throughout the organization. Key decisionmakers are identified to ensure continued
financial support; they are encouraged to join the working groups and inject their
concerns into the development process. Clinical staff members from all the facilities are
asked to visit and report on changes in medical care and health plan policies. In
addition, organization leaders continually address the working groups to inform them of
organizational changes so that the applications are consistent with organizational goals.
Finally, the working groups meet every few months with
their customers, including members of the health plan, clinical staff, and plan
administrators. They define and refine the dissemination process, provide feedback on the
success of the program, and brainstorm on future options for added technology.
A year later, a survey shows that 70 percent of members who
have used the decision-support application are highly satisfied with their experience and
believe that it not only helped them make a better decision, but also improved their
relationship with their health care provider. A similar survey among providers shows that
more than half have already adapted the application into their routine practice and
believe that it improves the quality-of-care they deliver.
- Health care organizations and other potential purchasers
should carefully assess their internal capabilities for application development and
maintenance before attempting to develop their own applications.
- Purchasing decisions require careful consideration of the
objectives for the application and evidence of application effectiveness.
- Substantial attention to, and planning of, the
implementation process are critical. The process should be consistent with the
organizations structure and culture, and address institutional barriers.
- Continuous evaluation and quality improvement are essential.
- Successful implementation is a team process; involvement of,
and communication with, all stakeholders are necessary.
Consumers, perhaps more than any other stakeholder group,
vary in terms of their ability and experience in evaluating applications, and, thus, may
be the most "vulnerable" stakeholder group. This is because a
"consumer" may be a scientist or health professional by training, someone
trained in a different field, or someone who has no formal education. Therefore, of all
the stakeholders, consumers are probably at greatest risk of potential harm and need to be
cautious because of the general lack of disclosure of information about developers and
sponsors of IHC applications. Many applications are currently being used with no or
limited guidance from a health professional. Most consumers are concerned about being able
to select and use the best applications for their needs and require guidance and tools for
doing so (Gustafson, Robinson et al., 1999). A "consumers guide" for
interpreting evaluation results reported by developers is presented in Appendix E.
Selecting Applications for Use
How do users select the most appropriate applications? Even
if consumers recognize the potential for harm and wish to evaluate applications before
use, will they be able to? The growing array of applications available makes the selection
process extremely daunting. If independent evaluations of products are available, and they
are presented in consumer-friendly terms, consumers would be better able to make informed
selections of applications for their use. In many cases, however, no evaluations of
specific products will be available, and the consumer will need to independently judge the
quality of the application.
Consumers need to consider more than just the quality or
accuracy of the information or content. Some products may make users feel anxious or
overly confident, or may affect the trust they have in people important to them. What kind
of emotional support does the product offer? Did it actually help them? For example, a
program that offers only information and no social support may be less likely to promote
behavior change or enhance coping with a disease (Brennan et al., 1995; Gustafson, Hawkins
et al., 1999). Moreover, because different consumers learn in different ways, potential
users need to consider which style of learning works best for them. Two applications that
have identical information but different presentation styles may have very different
effects on the same person. In essence, consumers may consider developing their own
evaluation framework that includes consideration of content quality and outcomes related
to personal coping and self-confidence in decisionmaking. Tools will be needed to assist
consumers on how to be an educated consumer and implement an evaluation framework.
Building Skills to Use IHC Applications
What skills and training do consumers need to use IHC
applications? Because of the newness of IHC, relatively few consumers have the skills
needed to take full advantage of applications. In some cases, skills must be acquired
under circumstances of high stress, such as in the case where an individual has just been
diagnosed with a life-threatening illness. Unless application design takes into account
the diversity of potential users (e.g., educational/computer skills level, disabilities),
many resources will be unusable. Allowing users to provide feedback to developers is
important to improving usability. In addition, enabling users to share their experiences
with others may help them use applications more effectively.
The scenario in Box V-4 illustrates several important
issues faced by consumers in selecting and evaluating appropriate IHC applications.
A Consumers Experience with IHC Applications
Mary Smith returned home from the hospital with her
week-old twins, a girl and a boy. Like many first-time mothers, Mary was both enchanted
and overwhelmed with the challenges of her new family. The practical advice offered during
the childbirth preparation classes seemed to escape her awareness at the very moment that
questions about nursing the babies entered her mind. Her questions seemed too minor and
insignificant to warrant a call to the nurse midwife. However, she remained worried, and a
bit unsure about many of the day-to-day rituals of mothering twins. Encouraged by her
partner, Mary decided to use some of the health information resources available on the
Internet. During one of her first trips away from home, Mary stopped at the public library
to use their Internet computer.
First Time: Finding Resources
Mary began looking for health information at her
hospitals Web site. She brought along the address for the hospitals Web site,
which she had found in her discharge instructions. Mary typed the address into the
browser, and soon pages appeared on the computer screen presenting information by her
hospital. Most of the information addressed logistics such as how to get to the hospital
or where to park; however, several sets of keywords suggested that specific clinical
advice might be available. Selecting the phrase "Help for New Parents" led Mary
to discover a series of pages with helpful hints and explanations about taking care of
One page described common sleep patterns for newborns,
noting that smaller babies had more sleep difficulties than larger babies. This
information confirmed Marys experience with her five-pound twins, neither of whom
slept more than three hours at a time. Mary read with eagerness the suggestions regarding
how to establish successful sleep routines with small infants. Another page addressed
bathing infants. Mary was able to view a short movie demonstrating how to hold small
infants while bathing them. She learned that many soaps caused skin irritation in
newborns, and reviewed a list of recommended bathing products.
None of the material presented on the hospitals Web
site specifically addressed the needs of twins. Mary noticed the message "Search For
Other Resources" just as she needed to leave the library to return home. She made a
note to come back and check the new information.
Second Time: Follow-up and Exploration
One week later Mary returned to the library. By this time,
she had begun to feel a bit more confident feeding and bathing the babies, but had found
herself becoming increasingly weepy and sad over very small events. The babies seemed to
be nursing successfully, but she continued to worry about her ability to feed them. Mary
launched another session to her hospitals Web site.
Mary located the appropriate section within the
hospitals Web site and looked through the listing until she found the message
"Search For Other Resources." At this point Mary initiated a search. In response
to a question requesting her topic, Mary typed in the words "newborn twins." She
received a list of 27 different Web sites, all denoting "twins" somewhere in the
title of the pages. She began reviewing the pages one at a time.
A commercial baby food company presented the first page
that Mary discovered. Included on this page were pointers to descriptions about various
formula preparations and strategies for promoting health nutrition. Pretty colors and
pleasing pictures of well-nourished babies made the page seem interesting; however, with
her commitment to breast-feeding, Mary determined that the content on the page was not
relevant to her current situation.
A second page led Mary to the Web site developed and
supported by a national voluntary society, Parents of Twins. This page differed from the
ones Mary had already viewed. In addition to the usual helpful reference materials, this
page had a public bulletin board area where interested persons could post or read
messages. Mary focused her attention here.
Mary discovered several hundred messages with topics
ranging from "Twinsdouble trouble" to "Helpwhere can I find
cheap diapers." As Mary reviewed the messages, she noticed that parents of twins
posted most of the messages. Many messages offered humorous or unnerving stories about
child care experiences. Some posed specific questions, such as "Has anyone tried to
nurse both babies at the same time?" to which other participants offered suggestions
or advice. Occasionally a user identified himself or herself as a health care
professional; their comments seemed different from those posted by the parents, offering
clinical descriptions or recommendations to consult with clinical care providers.
Mary observed that the quality of advice in the postings
ranged from very believable to almost ludicrous. Some postings clearly were advertisements
in disguise, suggesting that a specific product may help with a given problem. Others
offered helpful strategies, such as how to use pillows to position the babies comfortably
during feedings. Occasionally a poster would direct readers to additional Web sites, which
Mary herself sometimes subsequently viewed.
Mary noticed that a small set of individuals seemed to do
most of the posting. Five or six users seemed to post several times a week, sometimes
addressing their messages directly to other individuals or to the group as a whole.
Phrases such as "Twin Moms Unite" or "Parents in the Know" created a
sense of community that Mary knew she would like to join. Since no funds were needed to
use the computer or to participate in the conversations in the bulletin board area, Mary
resolved to return when she could and partake in this shared experience of parenting
Maintenance: Linking IHC With Professional Care
When the twins were 4-weeks-old, Mary returned to the
midwife for her post-partum checkup. In the conversation about her emerging role as
mother, Mary described for the midwife the assistance and peer support she had discovered
on the Web. Mary talked about the range of helpful (and sometimes unhelpful) advice
provided by fellow users, and about the different Web sites that addressed child-care and
parenting. Her midwife listened attentively, expressing support for Marys abilities
to seek out and interpret health-related information for herself. She reviewed Marys
strategies for determining the quality and relevance of the health information provided
and identified some additional Web sites that she already reviewed and believed Mary would
- IHC applications are an important resource for consumers and
public access to these resources is critical.
- Consumers need to be critical of the quality and relevance
of online health information and support.
- The use of IHC applications can be more effective when their
use is linked with regular care from a health professional.
Policymakers, both in the public and private sectors,
determine the nature of the environment in which IHC development and implementation occur
(Patrick et al., 1999). IHC policymakers in the private sector typically include
executives of health care organizations and large corporations, and they are driven by
their primary responsibility to ensure the financial viability of their organization.
Their purchasing and investment decisions essentially determine the commercial viability
and development of specific types of applications. Policymakers in the public sector,
especially legislators and regulators, have decisionmaking power over the general field of
IHC, and have to ensure that their decisions are consistent with the interests of their
constituents and the general public. Both types of policymakers help determine the role of
IHC in the overall mix of public and private investment in individual and community
health, in disease prevention, and in health care and rehabilitation. Ultimately,
policymakers are concerned with the impact that IHC applications may have on the
structure, process, and outcomes of health and health care. Understanding how IHC
applications relate to, enhance, and/or potentially detract from other determinants of
individual and community health is important in developing policy. Major policy issues
related to IHC are reviewed in the next chapter.
Promoting Quality and Access
How do policymakers promote quality and ensure safety of
IHC while not hindering innovation and use? Policymakers will need to weigh the role of
voluntary standards and self-regulation versus government regulation, and decide if or
when it is appropriate to intervene or regulate. In some cases, such as in the case of
fraudulent online marketing of health care products, Federal agencies such as the Federal
Trade Commission have taken some action (FTC, 1997). Policymakers may consider
implementing appropriate incentives to promote voluntary adoption of quality goals.
How do policymakers ensure that all populations have access
to IHC? Some policymakers are increasingly concerned that the growing reliance on
technology in health care and public health may leave the most needy populations without
access to health information and support (Eng et al., 1998). The debate about who should
pay to enhance access to telecommunication services is being played out in the context of
the universal service provisions of the Telecommunications Act of 1996 (FCC, 1998). In
addition, there are concerns about how applications that address issues of relatively
small populations ("orphan" issues) can be supported if they do not have
The scenario in Box V-5 outlines some critical issues that
policymakers must deal with in developing and implementing IHC applications that contain
personal health information.
A Policymakers Dilemma
A county with a population of 2.5 million and a handful of
dominant health care providers is working to develop a community-wide immunization
registry for all children and youth under the age of 21. The vision is to have
"anytime, anywhere" instant availability of information on the immunization
status of the target population. The rationale for this is that having such information
available at the time of either medical office visits or school and college health
immunization initiatives, will reduce the likelihood that these individuals will
"fall through the cracks" and remain un- or under-immunized against important
and sometimes deadly diseases such as measles, rubella, and hepatitis B.
More than two years ago, the county director of public
health convened leaders from each of the major health care providers, the council of
community clinics (the members of which serve most of the uninsured), three major
employers who, combined, provide approximately 60 percent of the countys jobs, and
expert faculty from the local schools of public health and medicine. With support from
staff of the health department, they have been meeting monthly to discuss ways in which
the immunization registry can be built. Many of the technical problems that existed
several years ago when the idea of a registry first surfaced are now more tractable in the
face of growing acceptance of Internet-based technologies. These include problems such as
interfacing disparate databases, sufficient connectivity between health care providers and
the health department, acceptance of common definitions and language to use for
procedures, and methods for secure and reliable transmission of information via the
Internet, including safe storage behind adequate "firewalls."
However, in recent meetings, the group has become bogged
down over three issues. First, while there is a desire to fully engage consumers in the
process of helping to keep immunization data on their children up to date, there is worry
over whether public access to even part of the registry might be put to unintended
purposes (e.g., separated parents contesting custody of a child might use the registry to
locate one another). Second, the general issue of making available personal medical
information outside the confines of traditional health care settings is beginning to be
the subject of call-in talk shows in the community. One county supervisor is particularly
entrenched in his views about "big government" and how the potential for misuse
exists in any governmental collection of information about individuals. Finally, there is
concern about the accuracy and general quality of the information to be contained in the
registry. The health care providers readily admit that when they perform audits on their
own internal data sets for immunization information, they find errors and problems. This
is in spite of substantial efforts on their part to maintain the currency and quality of
What role can the local health department play to help this
group move beyond these problems and create a functional immunization registry? What
liability issues are raised by the first concern? How can these issues be mitigated? Can
the trade-offs between concerns about how access will be used and the need to engage
consumers fully in helping the registry stay current be made to come out in favor of
As for the second concern, how can the group move to
community acceptance of the functions served by the registry? Is it acceptable to
characterize the registry as similar in kind to other public health data functions and
thus invoke the public health department authority and responsibility? If the county does
not assume this responsibility what sort of entity could be developed to operate the
registry that might reassure those concerned about "big brother?" What
safeguards can and should the data communication and storage technology partners provide
to assure stakeholders that data will be secure?
Finally, how can the group handle the third area of
concern, that of the quality of the data present in the registry and any liability that
may pertain? This is considered especially important given the potential for consumer
access to the registry to enable, for example, a simply query about the need for a
vaccination for a child. What if the registry says there is no need when, in fact, there
is? Who is responsible for thisthe public health department, all participants, the
"health provider of record" for the child, or is there some other chain of
responsibility? If it is an independent new entity, how can these problems best be
anticipated and lessened?
- Policymakers should educate and work closely with all
- The potential for abuse or harm from publicly accessible
databases should be carefully evaluated.
- Policymakers should help define and coordinate roles and
responsibilities of professionals who are involved in delivery of services.
- Public health goals need to be balanced with the rights of
individuals to privacy and confidentiality.
7 The term
purchaser is used here to refer to purchasers of health services and products,
such as health plans, other health care organizations, large employers, and government
Return to Table of Contents