The Next Revolution: The Role of Informatics in Improving Health Care
Evaluation of English and Spanish Health Information on the Internet
Slide Presentation by Gretchen Berland, M.D. (Text Version)
On August 1, 2001, Gretchen Berland, M.D., made a presentation during the Web-assisted teleconference, The Next Revolution: The Role of Informatics in Improving Health Care entitled "Evaluation of English and Spanish Health Information on the Internet."
This is the text version of Dr. Berland's slide presentation. Select to access the PowerPoint® slides (468 KB).
Slide 1
Evaluation of English and Spanish Health Information on the Internet
Gretchen Berland, M.D.
The RAND Corporation
Slide 2
Number of Persons Seeking Health Information on the Internet
Has Nearly Doubled in the Last 3 Years
This slide features a line graph
showing the increase in people seeking information on the Internet:
- July 1998—54 million
people.
- June 1999—69 million
people.
- March 2001—97 million
people
Slide 3
Health Information on the Internet
- High expectations:
- Consumers will
have ready access to accurate health information.
- Access will
motivate consumers to participate more actively in their care.
- Significant concerns:
- Information
may be incomplete, inaccurate and misleading.
- But little is known about the accessibility, quality and reading level
of that information.
Slide 4
Conducted
a Three-Part Study to Evaluate English and Spanish Health Information on
the Internet
- What are consumers likely to find when they search for specific health
topics online? How easy is it to find relevant information?
- How comprehensive, accurate and current is the information on selected
e-health Web sites?
- What is the level of literacy required to understand the information provided
by these sites?
Slide 5
Overview of Approach for Studying Search Engines
- Conditions studied: breast cancer, childhood asthma, depression, obesity.
- Selected search engines:
- 10 English-language.
- 4 Spanish-language.
- Conducted standardized searches using simple search terms.
- Categorized the results of these searches.
Slide 6
Few Searches Lead to Relevant Information
This slide features a flow chart following an Internet search. The chart begins with entering 4 simple search terms. From there, 66% of the total links were irrelevant, and 34% were relevant. Of the relevant links, 26% resulted in a no content page, and 74% resulted
in a content page. Of the content pages, 21% contained irrelevant information, and 79% contained
relevant content.
Slide 7
Conducted a Three-Part Study to Evaluate English and Spanish
Health Information on the Internet
- What are consumers likely to find when they search online? How easy is
it to find relevant information?
- How comprehensive, accurate and current is the information on selected
e-health Web sites?
- What is the level of literacy required to understand the information provided
by these sites?
Slide 8
Approach Used to Evaluate Web Sites
- Convened panels of patient advocates and clinical experts for each condition.
- Developed 5-7 key "need-to-know" topics and "consumer-oriented" questions.
- Developed standardized answers to questions based on literature reviews.
Slide 9
Example of Consumer-Oriented Question and Standardized
Answers
"I have a lump in my breast. What should be done to check
this?"
- New
breast lumps should be brought to the attention of a physician.
- Mammography
and ultrasound are useful in evaluating lumps.
- A
negative mammogram does not eliminate the need for further evaluation.
- A
persistent, non-fluid filled breast mass felt by a physician should be
biopsied.
Slide 10
Finding and Evaluating Health Information on Web Sites
- Two searchers visited each selected Web site looking for information related
to the consumer-oriented questions.
- Results from the searches were saved and assembled into notebooks.
- Developed a standardized rating form for each of the four medical
conditions.
- Recruited 34 physicians to rate information for both coverage and accuracy.
Slide 11
Examples of Rating Coverage: Breast Cancer Screening
No coverage:
- No mention of
mammography.
Minimal coverage:
- Mentions mammography
as a way to identify breast cancer early.
- Does not mention
who, how often, or why.
- Does not discuss
pros and cons of mammography.
More than minimal coverage:
- Mentions who,
how often, or why, or
- Discusses pros
and cons of mammography.
Slide 12
Coverage
of 4 Conditions on English Sites
This slide features a bar
graph of four conditions and the percentage of topics on these conditions
receiving more than minimal and accurate coverage on English Web sites:
- Breast
Cancer, 63%.
- Childhood
Asthma, 36%.
- Depression,
44%.
- Obesity,
37%.
Slide 13
Coverage
of 4 Conditions on Spanish Sites
This slide features a bar
graph of four conditions and the percentage of topics on these conditions
receiving more than minimal and accurate coverage on Spanish Web sites:
- Breast
Cancer, 39%.
- Childhood
Asthma, 23%.
- Depression,
12%.
- Obesity,
15%.
Slide 14
Conducted
a Three-Part Study to Evaluate English and Spanish Health Information on
the Internet.
- What are consumers likely to find when they search online? How easy is
it to find relevant information?
- How comprehensive and accurate is the information on selected e-health
Web sites?
- What is the level of literacy required to understand the information provided
by these sites?
Slide 15
Approach
to Measuring Reading Levels
- Used widely accepted readability formulas.
- Measured grade levels as a function of sentence and word complexity in
a sample of text.
- Applied formulas to randomly selected passages of text.
Slide 16
Health Information Not Accessible to Many
This slide features two
pie charts measuring the level of education necessary to understand the
information on these Web sites.
For
English Language sites:
- Middle School: 0%.
- High School: 37%.
- College: 52%.
- Graduate 37%
For
Spanish Language sites:
- Middle School: 14%.
- High School: 43%.
- College: 43%.
- Graduate: 0%.
Slide 17
Conclusions
- Choice of search engine matters.
- Overall coverage varies by language and condition.
- Spanish-language availability and quality are lower.
- Even if one finds information, not everyone can read it.
Slide 18
Next
Steps
This slide features a diagram
illustrating how the use of computers can help contribute to a health outcome. On
the left are a column of four computers, two of which lead to a family,
and two of which lead to a provider. The
patient and provider then meet together, to produce the desired health
outcome.
Slide 19
Limitations
of Web Site Evaluations
- Reviews based on content found between October-November 2000; content may
have changed.
- Reviews based on the materials that were found by searchers in 90 minutes;
materials not found in this time were not evaluated.
Current as of August 2001
Internet Citation:
Berland, G. Evaluation of English and Spanish Health Information on the Internet. Slide Presentation (Text Version) presented at The Next Revolution: The Role of Informatics in Improving Health Care, Web-Assisted teleconference, August 1, 2001. Agency for Healthcare Research and Quality, Rockville, MD.
http://www.ahrq.gov/news/ulp/informat/berlandtxt.htm
Return to Web Conference
State & Local Policymakers
AHRQ Home Page