Evidence Report/Technology Assessment

Number 46

Prepared for:
Agency for Healthcare Research and Quality
U.S. Department of Health and Human Services
2101 East Jefferson Street
Rockville, MD 20852

http://www.ahrq.gov/

Contract No. 290-97-0017

Prepared by:
McMaster University Evidence-based Practice Center
Tim J. Whelan BM, BCh, MSc
Principal Investigator

Mary Ann O'Brien, MSc
Miguel Villasis-Keever, MD, MSc
Paula Robinson, MD, MSc
Aimee Skye, BA, PhD student
Amiram Gafni, DSc
Melissa Brouwers, PhD
Cathy Charles, PhD
Fulvia Baldassarre, MSc, BScN
Mary Gauld, BA
Investigators

AHRQ Publication No. 02-E004

July 2002

On December 6, 1999, under Public Law 106-129, the Agency for Health Care Policy and Research (AHCPR) was reauthorized and renamed the Agency for Healthcare Research and Quality (AHRQ). The law authorizes AHRQ to continue its research on the cost, quality, and outcomes of health care and expands its role to improve patient safety and address medical errors.

This report may be used, in whole or in part, as the basis for development of clinical practice guidelines and other quality enhancement tools, or a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.

This document is in the public domain and may be used and reprinted without permission except those copyrighted materials noted for which further reproduction is prohibited without the specific permission of copyright holders.

Whelan TM, O'Brien MA, Villasis-Keever M, et al. Impact of Cancer-Related Decision Aids. Evidence Report/Technology Assessment Number 46. (Prepared by McMaster University under Contract No. 290-97-0017.) AHRQ Publication No. 02-E004, Rockville, MD: Agency for Healthcare Research and Quality. July 2002.

The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-based Practice Centers (EPCs), sponsors the development of evidence reports and technology assessments to assist public- and private-sector organizations in their efforts to improve the quality of health care in the United States. The reports and assessments provide organizations with comprehensive, science-based information on common, costly medical conditions and new health care technologies. The EPCs systematically review the relevant scientific literature on topics assigned to them by AHRQ and conduct additional analyses when appropriate prior to developing their reports and assessments.

To bring the broadest range of experts into the development of evidence reports and health technology assessments, AHRQ encourages the EPCs to form partnerships and enter into collaborations with other medical and research organizations. The EPCs work with these partner organizations to ensure that the evidence reports and technology assessments they produce will become building blocks for health care quality improvement projects throughout the Nation. The reports undergo peer review prior to their release.

AHRQ expects that the EPC evidence reports and technology assessments will inform individual health plans, providers, and purchasers as well as the health care system as a whole by providing important information to help improve health care quality.

We welcome written comments on this evidence report. They may be sent to: Director, Center for Practice and Technology Assessment, Agency for Healthcare Research and Quality, 6010 Executive Blvd., Suite 300, Rockville, MD 20852.

Decision aids have been developed to improve communication between health professionals and patients, and to help involve patients in health care decisions. The area of cancer-related decisions has been found to be particularly problematic with respect to health professional-patient communication and decisionmaking for a number of reasons, including difficulties in communicating information about poor prognoses and modest benefits of treatments used. The objectives of this review were to describe the different cancer-related decision aids (DAs) that have been developed and to evaluate the effectiveness of these interventions.

Studies were identified by searching MEDLINE (1977 to April 2001), HEALTHstar, Cancerlit, Cinahl, Sociological Abstracts, PsycINFO (1977 to August 2000), and EMBASE (1995 to August 2000); the Cochrane Library (issue 3, 2000), reference lists of included studies, and personal files of experts. The main search terms were decisionmaking; decision analysis; patient education; patient participation; and neoplasms.

Primary studies about prevention, screening, and treatment decisionmaking focused on cancer that met the definition of a decision aid were included. Exclusion criteria were studies of benign prostatic hyperplasia, hormone replacement therapy, and smoking cessation as well as unpublished studies or those published as abstracts only.

Two reviewers independently extracted data, including methodological quality items for all studies. Disagreements were resolved by consensus. Descriptive statistics were calculated for all fields of the database. Evidence tables were constructed to describe the most salient features of the studies according to the review questions. Data were not pooled, because clinical heterogeneity existed across the studies (different types of cancer, diverse range of decisions: prevention, screening, treatment, and different study designs), outcomes measurements were inconsistent, and, overall, the studies had low methodological quality scores.

• 61 unique studies (including 18 randomized controlled trials, 5 nonrandomized controlled trials, as well as other study designs) were included after all screening processes were completed.
• 22 studies examined the development process of the DAs. In general, all studies had the same phases: assessment of construct validity and reliability in noncancer participants, followed by field-testing in cancer survivors in some studies. The majority (14/22, 64%) studied breast cancer treatment decisions.
• The effectiveness of the DA was assessed in 39 studies; only 16 were randomized controlled trials (RCTs). Various DAs or a combination of strategies were evaluated: brochures, audiotapes, videotapes, interactive computer programs, educational scripts, decision boards, counseling, and informal decision analysis. Breast (23) and prostate cancer (11) were the most frequent types of cancer.
• Across the studies, patients' decisions, knowledge, anxiety, depression, satisfaction, and acceptability of the DA were the most frequent outcome measures evaluated.
• Overall, among RCTs, DAs appeared to increase knowledge and patient involvement in decisionmaking. Anxiety and depression scores appeared not to be increased by the DA. In patients making prostate cancer screening decisions, significantly fewer men decided to proceed with screening after receiving a DA.

Our results support that decision aids are helpful for some cancer screening decisions. In these situations, DAs can increase knowledge, do not increase anxiety, and, in some circumstances, can influence the decision made. In contrast, there is very little data available evaluating decision aids for cancer-treatment-related decisions, and further evidence is still needed. The early stage of development of this field and the gaps in our knowledge determined by this systematic review underline the need for further research. A number of different areas were identified, such as developing a better understanding of how and when decisionmaking occurs; who is involved (clinician, patient, or others); and the extent of their involvement. The key features of quality decisionmaking need to be determined from patients and clinicians to help investigators develop appropriate interventions and to identify and prioritize outcome measures of effectiveness. Multicenter collaboration to formally set a research agenda is needed because integration of different research efforts in the field appears to be suboptimal. National or international collaboration would permit development of consensus about important basic concepts regarding decisionmaking, decision aids, and important outcomes.

Decision aids are mechanisms or interventions that have been developed to improve communications between health professionals and patients and to help involve patients in making decisions regarding their health care. Decision aids can include brochures, videotapes, or interactive computer programs. Recent reviews have suggested that decision aids may be effective in supporting general health care decisions.

Cancer screening or treatment have been found to be particularly prone to difficulties in communication and decisionmaking between health professionals and their patients. There are a number of reasons for these problems, including difficulties in communicating information about poor prognoses and the modest benefits of the treatments used. The objective of this study was to conduct a comprehensive, systematic review of the literature to determine the impact of decision aids on cancer prevention, screening, and treatment decisions.

A set of questions was initially proposed by the National Cancer Institute's Division of Cancer Control and Population Sciences, and was further refined with input from members of the McMaster University Evidence-based Practice Center (MU-EPC) and the project officer at the Agency for Healthcare Research and Quality (AHRQ), which funds the EPC program.

The Technical Expert Panel (TEP) for this project included individuals who represented providers of health care, experts in study methodology, and researchers. After consultation with the TEP, the following key questions were selected as the focus of the Evidence Report.

• What models of decisionmaking (e.g., informed, shared) underpin decision aids that have been used?
• What clinical contexts (e.g., prevention, screening, and treatment) have been investigated?
• What has been the clinical focus of the decision aids (e.g., type of cancer and extent of disease)?
• What has been the mode of delivery (e.g., print, interactive video)?

• On what populations has the research been conducted?
• Have decision aids been developed for or used by members of special populations (e.g., the elderly, ethnic groups, and people with a low level of education)?

• What outcomes have been evaluated (e.g., increase in knowledge, satisfaction, and behaviors)?
• Are there any key outcomes that are associated with specific characteristics of decision aids?

• What is the effectiveness of decision aids?
• What is the effectiveness of decision aids in different clinical contexts?
• What is the effectiveness of different modes of delivery?
• What is the effectiveness of decision aids on special populations?

• What specific direction is needed in future research on cancer-related decision aids?

The authors regarded as potentially eligible any article that (1) described a study in humans and (2) was about the development or evaluation of a cancer-related decision aid. There was no exclusion based on study design or language of publication. Primary studies about prevention, screening, and treatment decisionmaking; that focused on cancer; and that met the definition of a decision aid were included. A decision aid was defined as "an intervention designed primarily to help patients (or patients and clinicians together) with making cancer-related health care decisions, when options are available for prevention, screening, and treatment. At a minimum, it should target some component of decisionmaking (e.g., information exchange, involvement in the decision process)."

Studies of benign prostatic hyperplasia, hormone replacement therapy, and smoking cessation were excluded, as were studies published in abstract form only.

The research team used a two-stage screening process. In the first step, six raters worked in pairs to screen the titles and abstracts identified by the searches. In the second step, randomly assigned pairs of raters screened full text articles, then three reviewers checked all included studies and categorized them according to the context of the decision and type of study. Discrepancies were resolved by discussion.

Citations of potentially relevant studies were identified through a systematic research of: MEDLINE from 1977 to the end of April 2001; HealthSTAR, CANCERLIT, CINAHL, Sociological Abstracts, and PsycINFO from 1977 to August 2000; EMBASE from 1995 to August 2000; The Cochrane Library (Issue 3, 2000); reference lists of included studies; and the personal files of research team members. The development and refinement of the search strategy followed an iterative process using the MEDLINE database. The refined MEDLINE strategy was modified to meet the specific features of the other electronic databases.

In consultation with the TEP and project officer, all data extraction forms were developed, pilot-tested, and revised by members of the local research team. Two reviewers completed data extraction independently for all studies. Any disagreements were resolved by consensus. Following consensus on each item, the data forms were scanned into a Microsoft Access database using Teleform software.

Descriptive statistics were calculated for all fields of the database. Evidence tables were constructed to describe the most salient features of the included studies according to the review questions. The local research team at the MU-EPC, in consultation with members of the partner organizations and the project officer, evaluated the overall quantity and quality of the data available. A draft of the report was sent to an international Peer Review panel, comprised of researchers in the field of decisionmaking, methodologists, and consumers. This report incorporates many of the suggestions of the Peer Review panel and represents a detailed qualitative synthesis of the existing evidence, emphasizing the directions that future researchers could take to fill knowledge gaps.

The analysis of the yield of the literature and the general characteristics of the studies showed that:

• A total of 1,056 full text articles were retrieved and screened. After a preliminary screening process, 207 articles met the inclusion criteria. Of the 207 articles, there were 168 unique studies with 39 reported in more than one publication. After the final screening process, 61 unique studies that focused on either the development or effectiveness of a cancer-related decision aid were included and form the basis of the Evidence Report.
• Sixty-seven percent of studies were published between 1996 and 2001.
• Ninety-seven percent of studies were published in English.
• The setting for 74 percent of studies was North America.
• Overall, 18 studies were randomized controlled trials (RCTs), five were nonrandomized controlled trials, and the remaining studies had a mix of designs.
• Overall, the studies had low methodological quality scores.
• Twenty-two studies examined the development process of decision aids. In general, all studies had the same phases: testing of content and construct validity, followed by the assessment of reliability in noncancer participants. There were three studies of prevention or screening decisions and 19 of treatment decisions. There were 14 studies involving breast cancer patients; two each of prostate, ovarian, and lung cancer patients; and one study each of colon cancer and leukemia patients. Only two developmental studies focused on special populations (Mexican-American women and impoverished African-American women).
• The effectiveness of a decision aid was assessed in 39 studies: 16 RCTs, 4 nonrandomized studies, 2 nonconcurrent controlled studies, 6 pre/post designs, and 11 case series. Various decision aids or a combination of strategies were used: brochures, audiotapes, videotapes, interactive computer programs, educational scripts, decision boards, counseling, and informal decision analysis. Breast (23) and prostate cancer (11) were the most frequent types of cancer included.
• Of the 39 studies that evaluated a decision aid in a clinical context, the ethnicity of participants was reported in 11 studies. In 10 studies, the majority of participants were Caucasian. Only one study evaluated the effect of a decision aid in a special population.
• Across the studies, patients' decisions, knowledge, anxiety, depression, satisfaction, and acceptability of the decision aids were the most frequent outcome measures evaluated.
• Overall, among RCTs, the decision aids appeared to increase knowledge and patient involvement in decisionmaking. Anxiety and depression scores did not appear to be increased. In patients making prostate cancer screening decisions, significantly fewer men decided to proceed with screening after receiving a decision aid.

These results support the proposal that decision aids are helpful for a number of cancer-screening decisions. In these situations, such instruments can increase knowledge, do not increase anxiety, and can influence the decision made. In contrast, there are few data available evaluating aids for decisions related to cancer treatment. Unfortunately, further evidence is still needed before making specific conclusions regarding decision aids in this situation.

The early stage of development of this field and the gaps in knowledge outlined in this systematic review underline the need for further research. A number of different areas were identified. Future research efforts should:

• Develop a better understanding of how and when decisionmaking occurs in the real world, who is involved (clinician, patient, or others), and the extent of their involvement. Further work is needed to identify the processes involved and when they occur. Presumably, information transfer is the first step, but what are the stages of deliberation and how do patients and clinicians interact at this stage? How do they ultimately make a decision?
• Determine the key features of quality decisionmaking from patients and clinicians. Such information will have a number of important benefits to help investigators develop instruments to facilitate quality decisionmaking and, perhaps most importantly, to identify, prioritize, and measure outcomes of effectiveness.
• Determine patients' understanding of numerical estimates of risk. Are such numbers meaningful for them? What is the impact of providing risk estimates on real-life decisions?
• Determine whether decision aids are effective for cancer-related treatment decisions. Research in other disease sites besides breast and prostate cancer and for metastatic disease is also necessary. The latter may be particularly challenging in terms of explicit discussion of benefits and risks of proposed treatments.
• Focus on which components of a decision aid are necessary and effective (e.g., besides exchanging information, is counseling helpful)? How should it be instituted? Are different types of decision aids more effective than others?
• Investigate whether decisionmaking regarding cancer is really different from decisionmaking in other chronic medical illnesses. In view of the life-threatening nature of this disease, are special approaches necessary here (e.g., psychosocial support techniques, patient support groups, teleconferences, or use of repetition)?
• Determine what patient, clinician, or decisionmaking factors influence the effectiveness of decision aids. Are decision aids more or less useful in particular situations (i.e., do decision aids facilitate communication for clinicians who are less likely to spend time talking with their patients)? Or, alternatively, do decision aids impede communication in a more interactive clinician-patient relationship? Are there particular groups of patients that benefit from decision aids? Who are they (e.g., patients having difficulty making a decision)? Can they be identified a priori?
• Establish whether decision aids are useful for members of special populations (e.g., the elderly, ethnic groups, or people with a low level of education). Should decision aids be modified for these populations, and how should this be done?

In addition to focusing on these areas, future research efforts should consider:

• Multicenter collaboration to formally set a research agenda. From this review, there appeared to be poor integration of different research efforts in the field. National or international collaboration would permit development of a consensus about important basic concepts regarding decisionmaking, what a decision aid is, and important outcomes.
• Development of an accepted conceptual framework for decisionmaking, standardized definitions of a decision aid, and a core set of outcomes would have important benefits for patients, clinicians, and policymakers. Outcomes should be important to all parties and could include for patients and clinicians: knowledge, satisfaction, comfort with decisionmaking, involvement in decisionmaking, and resources utilized both for the decisionmaking and the treatment chosen.
• With respect to evaluation, larger studies with more rigorous design, more comprehensive reports, and studies with longer-term followup are needed to clearly establish effectiveness and adverse effects (if any) of decision aids, especially for cancer-related treatment decisions. Ideal studies would include evaluation of instruments developed based on sound principles compared to usual practice, with random allocation of intervention. Cluster randomization may be necessary so that the control group does not inadvertently receive the intervention. Appropriate outcomes should be assessed using survey instruments soon after administration of the intervention and with long followup to determine any latent effects. Studies should have sufficient statistical power to detect important differences and to look at factors predictive of effect. Multicenter collaboration is likely to facilitate this process and may have additional benefits in terms of increasing opportunities for dissemination of research results.
• Other collaborative efforts, such as workshops and the development of practice guidelines by policymakers, clinicians, and patients, and other methods to improve the dissemination and implementation of decision aids should be instituted.
• More involvement of consumer groups in helping to set the agenda, advocate for funding, facilitate the development of research studies, and disseminate research results should be considered.

The report concludes that funding should be sought from government and industry sources to support this research.