Expanding the Reach and Impact of
Consumer e-Health Tools

June 2006

Appendix 1. Environmental Scan of 40 e-Health Tools

Between August 2003 and February 2004, project staff conducted an environmental scan of consumer e-health tools in the academic, nonprofit, and commercial sectors. The scan was based on review of two major e-health research programs (the National Cancer Institute’s [NCI] Small Business Innovation Research [SBIR] program and the Robert Wood Johnson Foundation [RWJF] Health e-Technologies Initiative); articles and citations in the peer-reviewed social science, biomedical, and public health journal literatures; and recommendations from tool developers and other experts in the field. Project staff identified 40 tools for in-depth investigation. The purpose of the scan was to learn about the major characteristics, intended audiences, and evaluation practices of a range of tools. Examples were sought that are recognized by major research funders, use methodological rigor in their evaluations, have public health significance and commercial viability, and/or are technologically innovative. There was no expectation that this exercise would “cover the waterfront” or collect generalizable information. Inclusion in this exercise does not in any way imply an endorsement or evaluation of the quality or effectiveness of the tool.

For consistency, and to glean as much information as possible, the scan was conducted using a standard instrument (see Instrument Used to Conduct Environmental Scan for Consumer e-Health Report). Questions were based on theories, methods, concepts, and terminology from the peer-reviewed literature; reports on the state of e-health technologies; and handbooks on health communication research. The questions were pilot-tested with experienced e-health developers and researchers and revised based on their comments and suggestions. The instrument was used to conduct interviews with e-health tool developers and other experts (see Appendix 2 for names of interviewees) and review the tools themselves. Staff also sought out information on the tools in journal articles and other documents that were either publicly available or supplied by the tool developers.

Information was collected on the following topics, using the form at the end of this appendix:

• Functionalities of the e-health tool

• Methods of delivery

• User groups, populations served, and their effect on design and evaluation

• Payer(s) for use of the e-health tool

• Prospective purchasers and stakeholders other than consumers/patients

• Research and evaluation practices, including data elements collected

• Privacy, confidentiality, and security practices

• Mechanisms for dealing with adverse events

The resulting descriptions, presented below, represent a late 2003 and early 2004 snapshot of the e-health phenomenon. That phenomenon is rapidly moving: new technology is routinely being introduced; the market surrounding digital applications is in flux (at least two companies in the interview group were acquired during the short interview phase); and grant cycles begin and end. Nearly all of the interviewees described forthcoming products, services, research, or publications that will change the profile of their tools.

Back to Top

Tool Functions

Function	Number offering (of 40)
Health Information	39
Behavior change	34
Other (one or more additional functions)	24
Personal health data entry	22
Decision support	21
Social/emotional support	21
Disease management	19
Secure provider/patient communication	17
Risk assessment	17
Personal health record	12

All of the e-health tools in this group offer users multiple functions. Counting the “other” category as a single function (which understates the reality), the average tool has more than 5 functions of 10 possible choices. The core function, unsurprisingly, is health information, followed closely by behavior change facilitation. The large number of behavior change/prevention tools is partly accounted for by the presence in this pool of 20 NCI and RWJF grantees. At the time, both of these research programs stressed prevention-oriented projects. Significantly, 24 of the 40 tools offer one or more functions other than the nine specified in the interview form. This reflects the uniqueness and originality of e-health tools. The number of tools offering specific functions is shown, in order of frequency, in the following table.

Back to Top

Delivery Methods

The “average” e-health tool in this group of 40 uses at least two delivery methods—once again treating the “other” category, for simplicity, as representing a single method. In fact, as with functions, the “other” category is large and diverse and includes several unique devices for collecting and transmitting personal health data. The overwhelming number of e-health tools in the interview group—34 of 40—are delivered through the Internet, either through restricted-access (member/subscriber) Web sites, public Web sites, or a combination. Some tools that were initially developed for delivery via CD-ROM, notably, the Comprehensive Health Enhancement Support System (CHESS), have been converted to the Internet. e-Health tools generally use more than one delivery method. However, in most cases a primary form of delivery (e.g., secure, restricted-access Web sites) is combined with one or more ancillary methods (e.g., e-mail notices).

Back to Top

Audiences and Audience Segments

The findings show the complexity of e-health audience variables and the many ways developers think about reaching their intended audiences or user groups. The primary strategy used by developers in this group of 40 is audience segmentation. The findings align with the observations made in the Institute of Medicine report, Speaking of Health, about the adaptation of health communication for diverse audiences (2002):

• Some tools are developed for narrowly defined audiences (e.g., people older than age 65 with chronic obstructive pulmonary disease [COPD], or binge-drinking college students). Some developers have an array of such specialized tools or modules.

• Some tools are developed for a broad cross-section of users but are subsequently adapted to serve different audience segments (e.g., a Spanish-language version, a module for pregnant women, a chat room for caregivers). The broad cross-section may exist because the tool is available to all comers (e.g., through a public Internet site) or because it is distributed to a restricted but diverse constituency (e.g., the employees of a distributor or health plan enrollees).

• Some tools are developed for a broad (and therefore presumably heterogeneous) user group in a way that focuses on what all users have in common.

Back to Top

Transferability of Personal Health Information

The interviewees were asked what would be required for the user to transfer personal health data (e.g., history of tobacco use, blood sugar, blood pressure) to another organization’s application or device. The findings were varied and sometimes ambiguous. Of the 23 tools on which there is information for this question, some respondents focused on users’ ability to get their data in any form, including print, while others focused on interoperability issues related to standards and other technical matters. Only 7 tools have technical interoperability with other electronic systems. Another 7 make users’ data available to them in print. In general, the answers indicate the distance yet to go to make applications interoperable and to provide alternatives to proprietary approaches.

Privacy, Confidentiality, Security, and HIPAA

For these tools, security and confidentiality protections are generally addressed at the design stage, with a monitoring protocol thereafter. All interviewees in this group indicated awareness and, where needed, detailed knowledge of the Health Information Portability and Accountability Act (HIPAA). This fact is tempered by the reality that the HIPAA Privacy Rule does not apply to many e-health tool providers. The interviews highlighted the limits to privacy and confidentiality protection in online communities, as well as participants’ willingness to continue to share despite these limitations. The developers and distributors of open-access e-health tools with chat rooms and listservs make a serious effort to call users’ attention to the fact that the confidentiality of their contributions is not protected; theoretically, consumers use these sites with their “eyes open.” Participants must register, and the chat rooms in both open- and closed-system e-health tools in this group are monitored, and in some cases moderated by trained people, to minimize inappropriate behavior. For the e-health tools that are distributed as part of closed systems (the large majority in this group), chat room and listserv participants’ privacy seems more assured, as a function of the restricted access combined with stringent security measures.

Back to Top

Research and Evaluation

In-house or self-evaluation is the most common form of evaluation, done for 36 of the 40 e-health tools. Nearly one-half (18) are also evaluated by a nonaffiliated third party (i.e., an independent researcher). Only 10 e-health tools are evaluated by an affiliated third party (e.g., a sponsor or purchaser). Two-thirds of the evaluations (26 of 40) use at least one validated measure. All 40 e-health tools have undergone some kind of formative research. Almost all of the e-health tools (36) undergo process evaluation, described as usability testing or “ongoing feedback” (associated with continuing quality improvement). Some form of outcome evaluation has been conducted on the majority of e-health tools (33 of 40), with 17 e-health tools being evaluated in randomized controlled trials. (This is likely an unrepresentatively high proportion and reflects the requirements of the NCI SBIR and RWJF programs.) Many tools have an individual user feedback mechanism, such as a “comments” box or phone line. Developers report using the feedback to modify the tools on an ongoing basis.

The Federal Government emerged as significant, both as a funder of developmental or evaluation research and as a dissemination partner or purchaser of e-health tools. Some of the leading research and development on consumer/patient e-health (notably, on personal health records and disease management) is being done by Government agencies, including the U.S. Department of Veterans Affairs and the Centers for Medicaid & Medicare Services (CMS). In addition, at least 15 of the 40 tools have Government funding (usually research-related), in addition to several that are purchased by Medicare or Medicaid for enrollee use. As noted, several developers indicated that they see CMS as a potential purchaser of their tools. The need for Federal and foundation research funding can also be inferred from the fact that the only tools being rigorously evaluated are those with grant funding. Several interviewees mentioned that they had applied for research funding but did not receive it, and thus were unable to do the desired level of evaluation.

Back to Top

Payers, Purchasers, and Dissemination Partners

Payer/Purchaser	Number of tools (of 40)
Government (usually as research support)	15
Other	15
Health plans or insurers (includes Medicare and Medicaid)	13
Healthcare providers	12
Consumers/patients	9
Employers	9
Third-party sponsor (e.g., drug company, device manufacturer)	8

The questions in the instrument focused on payment for tool development rather than on the mechanisms for dissemination. The information collected shows that most developers in this group have multiple funders or purchasers, and that very few are consumers. Consumers pay to use only 9 of the 40 tools in this group, and of those 9, only 3 tools are exclusively made available directly to consumers (i.e., the tools are also disseminated through intermediaries). The following list shows the number of e-health tools in the interview group that fall in each payer or purchaser category.

The largest number of e-health tool developers (21) say they see health plans or insurers, including Medicare and Medicaid, as “ultimate purchasers or stakeholders” for their products (consumers/patients are always regarded as the “ultimate end-users”). To evaluate and demonstrate their products’ return on investment for purchasers, many tool developers conduct cost-benefit studies to compare health service utilization, absenteeism, or other variables with the cost of distributing the tool.

As noted above, 37 of the 40 tools are disseminated through various dissemination partners, a mechanism used for both for-profit and not-for-profit tools. The partners are in the following categories (with some developers partnering with several):

• Public health organizations

• Schools or childcare facilities

• Healthcare organizations/individual providers

• Employers

• Health insurance companies

• National health advocacy organizations

In these cases, consumers gain access to and experience the tools as a function of their relationship to the distributing entity (e.g., as employees, health plan members, and constituents of a national health organization). Some distribution partners purchase or license the tools and provide them to customers, employees, or members; others distribute the tools as part of healthcare or public health services. Some developers produce both direct-to-consumer and restricted-access versions of their products, with the latter offering more interactive services that are customized to the distribution partner’s specifications.

Instrument Used to Conduct Environmental Scan for Consumer e-Health Report

Name: ____________________________

Date: _____________________________

1. Sources of information on application or device. (Check all that apply.)

          __  Interview

          __  Web site

          __  Peer-reviewed literature

          __  Self-published report or other non-peer-reviewed document

          __  Other (specify) ____________________________

2. Bibliographic references available?

          __  Yes

          __  No

3. Brief description of application available?

          __  Yes

          __  No

1. Description of the application or device

1. Application title and URL ____________________________

2. Developer organization ____________________________

3. Division or unit ____________________________

4. Contact name ____________________________

5. Contact address, e-mail, and phone number

   ____________________________

6. Function of application or device. (Check all that apply.)

   __  Personal health record

   __  Secure provider-patient communication

   __  Health information

   __  Decision support

   __  Social/emotional support

   __  Risk assessment

   __  Behavior change

   __  Disease management

   __  Personal health data

   __  Clinician-entered

   __  Captured by device

   __  Consumer-entered

   __  Other (specify) ____________________________

7. Method of delivery of application. (Check all that apply.)

   __  Public Web site

   __  Member or subscriber only Web site

   __  CD or DVD

   __  Kiosk

   __  Game console

   __  PDA

   __  E-mail or listserv

   __  Bulletin board

   __  Telephone (any type)

   __  Device other than game or PDA

   __  Other (specify) ____________________________

8. Intended user group or population served? (Examples: ethnic group, gender, age, income, literacy skills)

   ____________________________

9. Please describe briefly how you take into consideration the characteristics of your intended users in the design and evaluation of your application or device.

   ____________________________

10. Who pays for the use of the application or device? (Check all that apply.)

    __  Consumer/patient

    __  Healthcare provider

    __  Health plan or insurer, including Medicare and Medicaid

    __  Employer

    __  Third-party sponsor, such as a drug company or device manufacturer

    __  Government (as part of access to health care, such as a community health center, or as part of a research project)

    __  Foundation grant

    __  Other (specify) ____________________________

11. Whom do you think of as the ultimate purchaser(s) or stakeholder(s) of your application or device? (Check all that apply.)

    __  Consumer/patient

    __  Healthcare provider

    __  Health plan or insurer, including Medicare and Medicaid

    __  Third-party sponsor, such as a drug company or device manufacturer

    __  Government (as part of access to health care, such as a community health center, or as part of a research project)

    __  Other (specify) ____________________________

    __  How is this consideration of purchasers and stakeholders reflected in your design and evaluation?

12. If, as a result of using your application or device, a user creates an electronic history (e.g., tobacco use, blood sugar or blood pressure levels), what would be required for the user to transfer this informationto another organization’s application or device, such as a personal health record? ____________________________

2. Application or device research and evaluation

13. Who has conducted/is conducting evaluations of the application or device? (Check all that apply.)

    __  Non-affiliated third party (example: independent researchers)

    __  Affiliated third party (example: sponsor or purchaser of application or device)

    __  In-house or self-evaluation

    __  Other (specify) ____________________________

14. Does the evaluation use validated measures?

    __  Yes

    __  No

15. Which types of research and evaluation have you conducted on the application or device? (Check all that apply and please provide a brief description of what you did as part of each type.)

    __  Formative research ____________________________

    __  Process evaluation ____________________________

    __  Outcome evaluation (note data source) ____________________________

    __  Adequacy of confidentiality and security mechanisms ____________________________

16. On which of the following elements are/were data collected as part of the research and evaluation of the application or device? (Check all that apply.)

    __  Cost-effectiveness for individuals, providers, payers, or sponsoring organizations

    __  Utilization of health services

    __  Frequency of use

    __  Intensity of use

    __  Satisfaction

    __  Convenience

    __  Relevance for users’ needs

    __  User appeal (likability)

    __  Health status change

    __  Attitude or belief change

    __  Knowledge change

    __  Intention change

    __  Behavior change

17. Please tell us about any other elements that you collect data on as part of the research and evaluation.

    ____________________________

18. Given current concerns about patient safety and adverse events, some people hypothesize that the use of some applications and devices could have unintended, harmful effects. Do you have any mechanism for identifying harmful effects that might occur as a result of using the application or device? ____________________________

19. Users typically have to provide anywhere from “some” to “a lot” of personal information to use an e-health application or device. Do you assess if your application or device is HIPAA compliant? (Check only one.)

    __  Yes, I’ve done such an assessment.

    __  No, I haven’t done such an assessment.

    __  I have determined that the application or device is exempt and does not require such an assessment.

20. Can you suggest other developers/researchers you think I should talk to? ____________________________