Evidence Report/Technology Assessment

Number 11

Prepared for:
Agency for Healthcare Research and Quality

U.S. Department of Health and Human Services
2101 East Jefferson Street
Rockville, MD 20852
http://www.ahrq.gov/

Contract No. 290-97-0017

Prepared by:
McMaster University, Hamilton, Ontario, Canada
Alejandro R. Jadad M.D., D.Phil.
Project Director

Michael Boyle, Ph.D.
Charles Cunningham, Ph.D.
Marie Kim, M.D.
Russell Schachar, M.D.
Investigators

AHRQ Publication No. 00-E005

November 1999

The Agency for Healthcare Research and Quality (AHRQ), formerly the Agency for Health Care Policy and Research, through its Evidence-based Practice Centers (EPCs), sponsors the development of evidence reports and technology assessments to assist public- and private-sector organizations in their efforts to improve the quality of health care in the United States. The reports and assessments provide organizations with comprehensive, science-based information on common, costly medical conditions and new health care technologies. The EPCs systematically review the relevant scientific literature on topics assigned to them by AHRQ and conduct additional analyses when appropriate prior to developing their reports and assessments.

To bring the broadest range of experts into the development of evidence reports and health technology assessments, AHRQ encourages the EPCs to form partnerships and enter into collaborations with other medical and research organizations. The EPCs work with these partner organizations to ensure that the evidence reports and technology assessments they produce will become building blocks for health care quality improvement projects throughout the Nation. The reports undergo peer review prior to their release.

AHRQ expects that the EPC evidence reports and technology assessments will inform individual health plans, providers, and purchasers as well as the health care system as a whole by providing important information to help improve health care quality.

We welcome written comments on this evidence report. They may be sent to: Director, Center for Practice and Technology Assessment, Agency for Healthcare Research and Quality, 6010 Executive Blvd., Suite 300, Rockville, MD 20852.

John M. Eisenberg, M.D.	Douglas B. Kamerow, M.D.
Director	Director, Center for Practice and Technology Assessment
Agency for Healthcare Research and Quality	Agency for Healthcare Research and Quality

The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services of a particular drug, device, test, treatment, or other clinical service.

This report was funded by the Agency for Healthcare Research and Quality (AHRQ) and developed with the help of many dedicated contributors. We are especially grateful for the input received from our partner organizations, the American Academy of Pediatrics (AAP) and the American Psychiatric Association. The members of the AAP Subcommittee on ADHD were especially helpful during the formulation of the research questions and the development of the data extraction forms. The support and commitment from our contacts at AHRQ were invaluable: Dr. David Atkins, the AHRQ Task Order Officer; Jacqueline Besteman; Al Deal; and the late Therese Dick, the AHRQ Contract Officer for this Task Order. It was with great sadness that we learned of Therese's untimely passing in the fall of 1998.

Thanks go to the San Franciso Cochrane Center for coordinating the peer review process, to the peer reviewers, and to Dr. Patricia Huston for her development of a coherent and constructive synthesis of the peer review commentaries. Several people provided us with information that was otherwise unavailable to us, including Drs. G. DuPaul, P. Jensen, and J. Swanson and members of the MTA Cooperative Group and the Canadian Coordinating Office for Health Technology Assessment (CCOHTA) research team. Special thanks to our colleagues at McMaster University for their enthusiasm, vision, and continued support.

To determine (a) the long-term and short-term effectiveness and safety of pharmacological and nonpharmacological interventions for attention-deficit/hyperactivity disorder (ADHD) in children and adults and (b) whether combined interventions are more effective than individual interventions.

MEDLINE (from 1966), CINAHL (from 1982), HEALTHStar (from 1975), PsycINFO (from 1984), EMBASE (from 1984), and the Cochrane Library searches were completed in November 1997. Reference lists of eligible studies and files of members of the research team and partner organizations were also searched.

Studies were selected if they focused on the treatment of ADHD in humans and were published in any language as a full report in peer-reviewed journals. Studies including conditions other than ADHD were reported if separate subgroup analyses for patients with ADHD were provided.

3 Two reviewers independently extracted data for 41 variables on general characteristics, along with detailed information on interventions, outcomes, and tests. Differences were resolved by consensus or by a third researcher. Studies were not combined quantitatively because the quality of reporting was low and heterogeneity existed across outcome measures and tests.

• Seventy-eight studies (77 randomized controlled trials) met the inclusion criteria.
• Twenty-three studies compared drugs and showed few, if any, differences among methylphenidate (MPH), dextroamphetamine (DEX), and pemoline; studies comparing stimulants with tricyclic antidepressants (2) were inconclusive.
• Six studies compared drugs with nondrug interventions and showed consistently that stimulants, particularly MPH, may be more effective than nonpharmacological interventions.
• Twenty studies compared combination therapies with a stimulant or a nondrug intervention alone; no additional beneficial effects for combination therapies were shown.
• Nine studies compared tricyclic antidepressants with placebo and showed that desipramine may be more effective than placebo; no consistent effect was shown for imipramine.
• Fourteen studies (13 in school children and 1 in adults) evaluated long-term therapy (>12 weeks) and showed a trend to general improvement regardless of treatment, but the length of followup was inadequate. MPH may reduce behavioral disturbance in children with ADHD while it is taken. Academic performance does not appear to be improved with stimulants.
• Twelve studies evaluated treatment in adults with ADHD. For MPH vs. placebo, the results were contradictory. Antidepressants may be effective in adults, but no beneficial effect was seen with pemoline, nicotine, or phenylalanine compared with placebo.
• Thirty-two reports (29 studies) evaluated adverse effects of drug therapy; many of the side effects associated with stimulant use appear to be relatively mild and of short duration and to respond to dosing or timing adjustments. Data are inadequate on the long-term effects and severity of adverse effects of most interventions.

This report describes rigorous systematic reviews on the treatment of ADHD, ready for incorporation into evidence-based clinical practice guidelines or performance measures. The report also provides a detailed description of the many limitations of the evidence available and provides recommendations to fill existing knowledge gaps. Studies on ADHD have low reporting quality, methodological flaws, and heterogeneity across outcome measures and tests. A detailed description is included of the many limitations of the available evidence plus recommendations to fill existing knowledge gaps. Fulfilling such knowledge gaps will not be easy and will require genuine collaboration among decisionmakers.

This document is in the public domain and may be used and reprinted without permission except those copyrighted materials noted for which further reproduction is prohibited without the specific permission of the copyright holders.

Jadad AR, Boyle M, Cunningham C, et al. Treatment of Attention-Deficit/Hyperactivity Disorder. Evidence Report/Technology Assessment No. 11 (Prepared by McMaster University under Contract No. 290-97-0017). AHRQ Publication No. 00-E005. Rockville, MD: Agency for Healthcare Research and Quality. November 1999

Attention-deficit/hyperactivity disorder (ADHD) is one of the most common disorders diagnosed in children and adolescents. The American Psychiatric Association (APA) describes the essential feature as "a persistent pattern of inattention and/or hyperactivity-impulsivity that is more frequent and severe than is typically observed in individuals at a comparable level of development." Terms that have been used to describe children with "distractability, impulsivity, and usually overactivity" include minimal brain dysfunction/damage (MBD), hyperkinetic reaction, and hyperkinesis.

ADHD has been surrounded by great controversy involving clinicians, teachers, policymakers, parents, and the media. The range of opinion regarding the validity of ADHD extends from those who do not believe it exists and regard it as a myth, to those who believe that there is genetic and physiological evidence supporting its existence.

Prevalence estimates of ADHD vary according to the methods of ascertainment, diagnostic criteria, informants, and population sampled. According to the Diagnostic and Statistical Manual of Mental Disorders IV (DSM-IV), the prevalence of ADHD in school-age children is 3 to 5 percent. However, prevalence studies using the two previous versions of the DSM (DSM-III and DSM-III-R) in the United States, Canada, United Kingdom, Germany, and New Zealand have shown rates that vary from 1.7 to 16.1 percent. Although it was previously thought that ADHD remitted before or during adolescence, it has been estimated that more than 70 percent of hyperactive children continue to meet criteria for ADHD as adolescents and up to 65 percent as adults.

Problems with the diagnosis and treatment of this condition can also arise because approximately 65 percent of ADHD patients may have at least one comorbid disorder in the form of anxiety, communication, mood, conduct, oppositional defiant, and learning disorders; Tourette's syndrome; and subnormal intelligence. ADHD has been associated with impaired academic achievement, rejection by peers, and family resentment and antagonism. The rich terminology may reflect the broad spectrum and the high frequency with which it is described with other comorbid conditions.

There is also variation and controversy around the treatment of ADHD, which often includes stimulant medication. To reduce inappropriate variation in treatment, major organizations in North America have developed, or are in the process of developing, practice parameters or clinical practice guidelines to guide treatment decisions.

In 1997, the Agency for Healthcare Research and Quality (AHRQ) charged the McMaster University Evidence-based Practice Centre (MU-EPC) with producing an evidence report on the treatment of ADHD. The objectives of this work were to conduct a comprehensive systematic review of the literature on the treatment of ADHD, with input from different groups of stakeholders, and to support guideline development initiatives, while building on existing work and focusing on answerable, clinically relevant questions.

A multidisciplinary research team was assembled, with participation of members of the nominating organizations, the American Academy of Pediatrics (AAP) and the APA, local experts, and research staff. After multiple consultations and the evaluation of published systematic reviews and meta-analyses, the following general questions were selected as the focus of the evidence report:

• What is the evidence from comparative studies on the effectiveness and safety, both short and long term, of pharmacological and nonpharmacological interventions for ADHD in children and adults?
• Are combined interventions more effective than individual interventions?

To answer these questions, while avoiding the duplication of work, making efficient use of the resources available, and ensuring maximum added value, the scope of the evidence report focused on the following seven categories of research studies:

• Studies with drug-to-drug comparisons of pharmacological interventions.
• Placebo-controlled studies evaluating the effect of tricyclic antidepressants.
• Studies comparing pharmacological with nonpharmacological interventions (drug vs. nondrug studies).
• Studies evaluating the effect of long-term therapies (>12 weeks).
• Studies evaluating therapies for ADHD in adults (>18 years of age).
• Studies evaluating therapies given in combination.
• Studies evaluating adverse effects of pharmacological interventions.

Numerous systematic reviews and meta-analyses have examined placebo-controlled trials of stimulant medication and have established consistently the short-term efficacy of these agents for core symptoms. Consequently, placebo-controlled trials evaluating stimulant medication were reviewed in this report only if they met the criteria for inclusion in any of the other six categories. In addition, the report focuses on head-to-head comparisons of pharmacological interventions and on head-to-head comparisons of pharmacological and nonpharmacological interventions. This was identified as the area of prime interest to clinicians rather than effectiveness of stand-alone interventions-either pharmacological or nonpharmacological.

In preparing the Report, we followed current standards for assessing and distilling research evidence. Citations of individual studies were regarded as potentially eligible and selected for further evaluation if they were randomized controlled trials (RCTs) that focused on the treatment of ADHD in humans and if they were published in peer-reviewed journals, in any language, as a full report. If the studies included conditions other than ADHD, they were included only if they provided separate analyses for patients with ADHD. We acknowledge that randomized controlled trials are imperfect tools, but so far they are our best probe to evaluate health care interventions. Non-RCTs were included if they provided data on adverse effects of interest collected over more than 16 weeks. More refined inclusion and exclusion criteria defined within each of the seven categories of research studies are described in the body of the report.

Inclusion of a study in the evidence report was decided by two members of the research team, by consensus, and on the basis of the information available in the full published articles.

Citations of potentially eligible studies were identified through a systematic search of:

• MEDLINE (from 1966), CINAHL (from 1982) and HEALTHStar (from 1975), PsycINFO (from 1984), and EMBASE (from 1984) using the search strategy described below. The searches were completed in November of 1997 using the terms behavioral symptoms, attention-deficit disorder with hyperactivity, attention-deficit, hyperactivity, cognition disorders, minimal brain damage, minimal brain dysfunction, hyperkinetic syndrome, hyperkinetic reaction, impulsivity or inattention, and random, clinical trial, comparative, case control, or cohort.
• The Cochrane Library (issue 4, 1997).
• The reference lists any eligible article identified in any of the above sources.
• Web sites of organizations funding research on the treatment of ADHD.
• Files of members of the research team and partner organizations.

Data extraction forms were especially developed and tested for this project. The local research team, partner organizations, and the Task Order Officer (TOO) were consulted and the forms approved for content. More than 41 different variables describing the general characteristics of the study were recorded. In addition, detailed information on interventions, outcomes, and tests were extracted from each of the full reports, independently by two reviewers, with differences resolved by consensus or by a third member of the research team.

Descriptive statistics were calculated for all the variables. Evidence tables were constructed to summarize, globally and question by question, all the information extracted from the study reports. The local research team, in consultation with members of the partner organizations and the TOO, evaluated the overall quantity and quality of the data available and decided that meta-analysis would be inappropriate to summarize the evidence on each of the research questions and for each of the main categories of interest. The main reasons for this decision were substantial clinical heterogeneity across the studies (e.g., therapies evaluated, patient populations, duration), inconsistency in outcome measurements, low methodological quality, and incomplete data reporting (see detailed descriptions within each category). Therefore, this report represents a systematic qualitative review of the existing evidence, emphasizing the implications for clinical practice and the opportunities for future research to fill existing knowledge gaps.

• A total of 2,405 citations were identified by the search strategies. Ninety-two reports, describing 78 different studies, met all the inclusion criteria.
• Overall, numerous deficiencies in the reporting of available RCTs limit the assessment of their validity, relevance, precision, and, therefore, their clinical application. Most studies did not clearly describe clinically important information such as the primary outcomes of interest, the presence of comorbid disorders, the characteristics of the patients' families, compliance with treatment, and baseline measurement of outcomes of interest. There was little information on the treatment of ADHD in minority groups.
• The small sample size of most studies limited their power to detect meaningful clinically important differences among the interventions.
• Ninety-seven percent of the reports of RCTs did not describe the method of randomization. Ninety-five percent did not describe efforts to conceal allocation from the investigators who recruited the patients into the study (e.g., allocation codes were obtained by telephone after a patient accepted to enter the study). Eighty-seven percent did not describe the number of withdrawals and dropouts and the reasons for such in each of the groups. These limitations increased the likelihood of biased results.
• Comparison or synthesis of data across studies was limited by the low quality of reporting and by the large number and heterogeneity of outcome measures and tests used in the studies. Researchers often used modified versions of the same tests (e.g., Conners) across studies but did not provide enough information on the modifications. This led to the conclusion in many instances that there is a lack of evidence on the effectiveness of clinically important interventions. It is important to recognize that this is different from finding evidence of the lack of effectiveness of the same interventions.

The following is a description of the main conclusions from each of the seven categories of interest:

• Drug vs. drug comparisons: The limited evidence available from studies comparing different stimulants suggests that there are few, if any, short-term differences in effectiveness among methylphenidate (MPH), dextroamphetamine, and pemoline. The studies comparing stimulants to tricyclic antidepressants had many limitations and presented conflicting results.
• Drug vs. nondrug comparisons: Despite the limitations in the individual studies, the results indicate consistently that stimulants are more effective than nonpharmacological interventions when compared head-to-head.
• Combination therapies: Evidence is lacking to support the superiority of combination therapy over stimulant alone or superiority of combination therapy over nondrug intervention alone. A recent large trial found that combined treatment offers modest additional benefits over single-component treatments for non-ADHD areas of functioning.
• Tricyclic antidepressants vs. placebo: The studies on desipramine, despite their heterogeneous designs, small sample sizes, and variable quality, suggest that desipramine is more effective than placebo. The studies evaluating imipramine show inconsistent results.
• Long-term therapy: All but one of the studies available were restricted to school-age children. Few studies followed children for a period of time equivalent to the length of time children typically remain on these treatments or reported side effects or used outcome measures that are situation specific (e.g., measure outcomes at home and at school). These studies show a trend to general improvement over time regardless of treatment and support the need for long-term placebo-controlled studies. MPH appears to reduce behavioral disturbance in ADHD children as long as it is taken. However, there is no information on the reasons so many children discontinue medication. The studies available provide little evidence for improvement in academic performance with stimulants, even though MPH treatment appears to produce consistent behavior improvement. The largest and most comprehensive study to date (Multimodal Treatment Study of Children with Attention-Deficit/Hyperactivity Disorder-the MTA Cooperative Group study) indicates that intensive behavior therapy, comprising child, family, and school-based interventions, adds little to the effects of long-term stimulant therapy. The MTA study also identified that the quality of supervision of medication may be an important factor in optimizing long-term therapeutic benefit. Lithium does not appear to be an effective alternative in patients who do not respond to stimulants.
• Treatment of ADHD in adults: The few studies evaluating MPH vs. placebo show contradictory results. The study with the highest methodological scores suggests that MPH may be effective for the treatment of ADHD in adults. Antidepressants may be effective in adults. Studies (one each) comparing pemoline, nicotine, or phenylalanine with placebo did not produce evidence in favor of these medications. There were no studies designed to determine the proportion of adults with ADHD who will use and benefit from other interventions.
• Adverse effects: Many of the side effects associated with stimulant use appear to be relatively mild and of short duration and respond to dosing or timing adjustments. However, data are inadequate on the long-term effects and severity of the adverse effects of most interventions. No comparative studies were identified with data on important adverse effects of interest, including potential for abuse of stimulants, liver toxicity due to pemoline, or major arrhythmia with tricyclic antidepressants in patients with ADHD.

Areas needing further research include the following:

• Effective strategies are needed to improve the quality of the study designs and reports. Journals that publish articles on the treatment of ADHD could benefit by the endorsement of criteria such as those included in the Consolidation of the Standards of Reporting Trials (CONSORT) statement, which has been adopted by over 70 leading peer-reviewed journals.
• Larger studies with more rigorous design and longer term followup are needed to establish the effectiveness and adverse effects of most interventions in both children and adults.
• The field would benefit from empirical methodological research evidence indicating the added value of nonrandomized within-subject and single-subject research designs for direct head-to-head comparisons between psychosocial interventions and other treatments.
• Research groups should make efforts to select a core set of validated and clinically relevant outcomes to be measured in all the studies in addition to any other outcomes of interest to the specific groups of researchers.
• More rigorous studies are clearly needed to establish the relative effectiveness of stimulants and tricyclic antidepressants and to compare the effects of stimulants with clonidine, buproprion, or selective serotonin-reuptake inhibitors.
• More definitive studies are needed to determine the added value of nondrug interventions when patients are already receiving stimulants, as well as the value of adding stimulants when nondrug interventions fail to achieve the desired outcomes. These studies, however, will require complex designs, substantial amounts of resources, and efficient collaboration among research groups. The MTA study is an example of this type of collaborative effort.
• Studies are also needed to determine whether comorbid factors (e.g., anxiety and depressive disorders) influence response to treatment.
• Studies are required to assess the severity of most adverse effects associated with stimulant medication and to evaluate, explicitly, the tradeoff between improvement in ADHD symptoms and signs and adverse effects. However, such a study is only worthwhile if the perspectives of all interested parties (parents, teachers, and patients) are included in the exercise.
• Reports of effectiveness and adverse effects almost always come from parents and/or teachers. One study (of adolescents) showed some important differences between parents and adolescents in the side-effects profile reported by each group. Data need to be collected from children on effectiveness and adverse side effects in order to gain a better understanding of the implications of treatment from the patients' perspective.
• A better understanding of the distinctions between "adverse effects" of therapy and concomitant characteristics of ADHD is needed. A number of reports discuss the high prevalence of "side effects" reported on placebo. Many of these may be associated with problem behaviors. Including such behaviors distorts the context for evaluating the importance of side effects.
• There were very few female patients in the available studies. A possibility exists that effectiveness and adverse effects vary by gender. This is an issue that needs to be examined or at least discussed.
• RCTs are of limited power to evaluate adverse effects, particularly rare ones or those that appear during long-term therapy. Only one comparative non-RCT with adequate data was found and it provided limited information. More observational studies are required (particularly case-control or cohort studies). Knowledge of adverse effects may also improve through more creative use of existing drug databases.
• Few studies have been supported financially by sources other than the Government or pharmaceutical companies. There is a great opportunity for consumer groups to support more research activities, given the number of important questions that remain unanswered and the implications of the results of research on the public.
• Conducting research on the treatment of ADHD is not easy, given the complexity of the disorder, the frequent presence of comorbidity, and the variety of interventions and outcomes available. Future research efforts will require commitment among different groups of stakeholders.

In summary, this report includes seven systematic reviews that incorporate state-of-the-art methodology, represent the most rigorous systematic review conducted to date, and are ready for incorporation into evidence-based clinical practice guidelines or performance measures. The report also provides a detailed description of the many limitations of the evidence available and provides recommendations to fill existing knowledge gaps. Filling such gaps will not be easy and will require highly innovative efforts and collaboration among different groups of decisionmakers. The MTA study confirms that large-scale, long-term collaboration among researchers is possible. If this field continues to produce small, incompletely reported studies with heterogeneous designs, instead of the high-quality collaborative efforts required, research in this area will continue to be abundant but will be of little value to guide most clinically relevant decisions.