Comments and suggestions regarding this draft document should be submitted within 60 days of publication in the Federal Register of the notice announcing the availability of the draft guidance. Submit comments to the Division of Dockets Management (HFA-305), Food and Drug Administration, 5630 Fishers Lane, rm. 1061, Rockville, MD 20852. All comments should be identified with the docket number listed in the notice of availability that publishes in the Federal Register.
For questions regarding this draft document contact (CDER) Howard Chazin at 301-796-0700, or (CBER) Leonard Wilson at 301-827-0373.
Guidance for Industry1
Integrated Summary of Effectiveness
|This draft guidance, when finalized, will represent the Food and Drug Administration’s (FDA’s) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance.
This guidance describes how an integrated summary of effectiveness (ISE) should be prepared by industry for inclusion in a new drug application (NDA) or biologics license application (BLA) 2 submitted to the Food and Drug Administration (FDA). The recommendations in this guidance reflect the FDA’s current thinking regarding what industry should include in an ISE to provide a truly integrated analysis, rather than a summary of efficacy results from individual clinical trials, and to satisfy FDA regulatory requirements. 3 Although there are no corresponding regulations requiring an ISE for BLA submissions, applicants are encouraged to provide these analyses in their applications.
This guidance supersedes section G, Integrated Summary of Effectiveness Data, of the 1988 guidance on Format and Content of the Clinical and Statistical Sections of an Application (Clin-Stat guidance). It also incorporates the conceptual framework of section 2.7.3, Summary of Clinical Efficacy (SCE), from the ICH guidance for industry M4E The CTD — Efficacy. 4
FDA’s guidance documents, including this guidance, do not establish legally enforceable responsibilities. Instead, guidances describe the Agency’s current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidances means that something is suggested or recommended, but not required.
The ISE is an overall integrated analysis that comprehensively examines the effectiveness data from individual clinical studies. In addition to the adequate and well-controlled studies that should be submitted as the basis for approval, the ISE should include other controlled trials (e.g., incomplete trials or trials that do not support the claim) and the results of any other studies, published or unpublished, not conducted by the applicant, of which the applicant has become aware.
Under 21 CFR 314.50(d)(5)(v), the ISE must include:
- An integrated summary of the data demonstrating substantial evidence of effectiveness for each claimed indication
- Evidence that supports the dosage and administration section of the labeling, including support for the recommended dosage and dose interval
- Effectiveness data analyzed by sex, age, and racial subgroups
- Evidence that is pertinent to individualization of dosing and the need for modifications of dosing for specific subgroups
Since 1985, the ISE has been required as part of an NDA submission, but the regulation does not describe the specific components of the ISE in detail except for the components listed above. The FDA provided recommendations for what should be in an ISE in the Clin-Stat guidance, as follows:
“The individual controlled studies to a great extent speak for themselves with respect to their ability to provide the evidence of effectiveness required by law. This section should provide an overview of the results, showing that they do satisfy the regulatory requirements for approval, i.e., represent adequate and well-controlled studies demonstrating the claimed effect, particularly if results are inconsistent or marginal. For example, the sponsor would explain here his basis for seeking to rely on a single study. Equally important, this section should include an examination of study-to-study differences in results, effects in subsets of the treated population, dose-response information from all sources, any available comparisons with alternative drugs, and any other information, so that the nature of the drug’s effectiveness can be as fully defined as possible, and the user of the drug can be given the best possible information on how to use the drug and what results to expect.”
Since the Clin-Stat guidance was published, several International Conference on Harmonisation (ICH) guidances, including the ICH guidances for industry E3 Structure and Content of Clinical Study Reports, E9 Statistical Principles for Clinical Trials, E10 Choice of Control Group and Related Issues in Clinical Trials, and M4E The CTD — Efficacy, 5 have provided additional recommendations for describing individual trials and providing an efficacy analysis. The recommendations in this guidance reflect the FDA’s current thinking regarding what should be included in an ISE to provide a truly integrated analysis, rather than a summary of efficacy results from individual clinical trials, and to satisfy FDA regulatory requirements.
The draft guidance for industry Integrated Summaries of Effectiveness and Safety: Location Within the Common Technical Document 6 outlines where to place the ISE within the framework of the common technical document (CTD) and electronic common technical document (eCTD). That guidance is intended to resolve the confusion concerning the difference between the document that should be included in Module 2, section 2.7.3, SCE, and the document that should be located in Module 5, section 18.104.22.168, Reports of Analyses of Data from More than One Study (Including Any Formal Integrated Analyses, Meta-Analyses, and Bridging Analyses). Although one of the goals of the ISE is to summarize the available effectiveness data, the ISE primarily is an integrated analysis of these data, going beyond a simple summary. For example, the regulatory requirements in 21 CFR 314.50(d)(5)(v) for support of dosing recommendations and analyses of demographic subset responses usually involve overview approaches that need both detailed explanation and documentation. The document in section 2.7.3 should summarize these analyses, but, in most cases, the ISE will be substantially larger than what would be appropriate for the section 2.7.3 summary of these data and analyses.
III. FORMAT AND CONTENT OF THE ISE
The format (including section titles) of the ISE should be flexible, but in many cases (and as broken down below) it can closely follow the format of the SCE that appears in Module 2, section 2.7.3, of the CTD. Applicants should take note of the following suggestions for content but should choose the format that best suits the data.
The ISE should not recapitulate detailed results of single studies, which are described in individual study reports, but instead should provide a comprehensive, detailed, in-depth analysis of the efficacy results in aggregate, with a clear rationale for the methods used in the analysis. Studies should be presented briefly while noting critical design and analytic features as well as important differences between studies (e.g., population, dose, duration, endpoints).
The background and overview section of the ISE should show clearly why individual studies should be considered adequate and well-controlled studies that demonstrate the claimed effect, and thus fulfill the legal requirements for approval. This demonstration is particularly important if results are inconsistent or marginal.
Critical study design features of the controlled studies supporting effectiveness should be discussed and compared among studies, especially when results are inconsistent. Examples of these features are as follows (see ICH E3 for additional important study characteristics):
- Choices of control treatment, particularly noninferiority designs
- Statistical approaches
- Dosing modifications
- Choice of patient population, and use of enrichment approaches to identify patients with high likelihood of events or high likelihood of response
- Particular design features such as crossover or randomized withdrawal designs
- Use of run-in periods as a method of enrichment
- Handling of dropouts in the study and analysis
- Choice of study endpoints (with particular attention to use of surrogate endpoints)
- Study duration
- Prespecified plans for analysis of the study results, with attention to multiplicity and primary and secondary endpoints
- Other features that may be critical in particular cases
As part of a comprehensive summary of experience related to efficacy, the ISE also can refer to nonclinical data and clinical pharmacology data as appropriate.
Particular attention should be paid to any recognized limitations of the efficacy studies. Thus, if an effectiveness claim is based on a surrogate endpoint, the basis for choice of the endpoint should be discussed and its validity as a predictor of clinical outcome should be supported. In some cases, this discussion may not be needed if the surrogate endpoint has been sufficiently validated and relied upon as the basis of approval for previously approved drugs (e.g., antihypertensive, oral hypoglycemic, and lipid-lowering drugs generally have been approved on the basis of demonstrated effects on blood pressure, blood sugar and HgA1c, and LDL cholesterol or triglycerides, respectively, without evidence, at the time of approval, of an effect of the particular drug on survival or morbidity). Nevertheless any novel surrogate endpoint (e.g., not used previously by the FDA as a basis for approval) should be discussed and supported.
If the objective of some of the studies was to show equivalence or noninferiority to an active control, the basis for the choice of noninferiority margin and support for the assay sensitivity of the trial, including support for the constancy assumption, should be provided in detail. The results should be evaluated by using the predefined criteria for defining equivalence or noninferiority, and any modification of plans should be explained (see ICH E10).
Graphs, technical discussion, mathematical derivations, or presentation of formulae should be included in an appendix, rather than in the main body of the ISE.
Generally, a tabular listing of all studies with data relevant to drug efficacy should be provided, together with study narrative descriptions. Both positive and negative studies should be included. The study narrative descriptions should be brief (e.g., similar in level of detail to an abstract for a journal article) and describe critical design features and critical results, including the prospectively identified endpoints and statistical analysis plan for each study. Similar studies can be described together, noting the individual study results and any important differences among the studies. These brief study narrative descriptions should include references or electronic links to the full study reports.
Using text, figures, and tables as appropriate, the Comparison and Analysis of Results Across Studies section should present all available data that characterize the efficacy of the drug. These comparisons and analyses should use appropriate methods for comparing studies of similar design, controlling for design differences, weighting by sample size, and examining by common covariates or stratifications. This section should include analyses of all data, irrespective of the strength of evidence of their support for the overall conclusion, and should discuss the extent to which the results of the relevant studies reinforce or do not reinforce each other. Any major inconsistencies in the data regarding efficacy should be addressed, and any areas needing further exploration should be identified. Data should be examined with respect to control groups, duration of exposure, patient populations enrolled, endpoints (including how defined), dropout profiles and analyses, and statistical methods (time-to-event versus cumulative rates).
Generally, this section should use two kinds of analyses: comparison of results of individual studies, and analysis of data combined from various studies.
1. Demographics of Efficacy Study Populations
The demographic and other baseline characteristics of study populations across all efficacy studies should be described. The following information should be included:
- The disease characteristics (e.g., severity, duration), prior treatment of the study subjects, concomitant treatment allowed or required, and study inclusion and exclusion criteria, including concomitant illness.
- Differences in baseline characteristics of the study populations in different studies or groups of studies, which may include any differences in age, sex, race, and geographic region (United States versus non-U.S. regions).
- Any differences among all populations included in critical efficacy analyses and the overall patient population expected to receive the drug when it is marketed.
2. Efficacy Results
- Assessment of the number of subjects who dropped out of the studies (separately for each treatment group), time of withdrawal (a defined study day or visit during treatment or follow-up period), and reasons for discontinuation.
Results from all controlled studies, including those controlled studies that did not favor the study treatment and other pertinent controlled studies that were included under clinical pharmacology, should be summarized, examined, and compared as appropriate using tables and figures (such as forest plots). Important differences in study design (such as endpoints, control group, study duration, patient population, dose, and statistical methods) should be identified.
Comparisons of results across studies should focus on prespecified primary endpoints. When the primary endpoints involved different variables, exposure durations, or time points in different efficacy studies, it can be useful to provide cross-study comparisons of important data elements that can be found in all studies even if not the primary endpoint in some of them. If results over time are important, study results can be displayed in a figure that illustrates the change over time in each study. Important secondary endpoints, particularly when examined appropriately after success on a primary endpoint, also should be shown.
The analysis should be consistent with the report of individual study results and can be accomplished with the help of tables that show major study design features, number of subjects, number of dropouts, and major outcomes.
When comparing study results for those cases in which many variables or time points were analyzed, representative ones (usually those that were identified as the primary endpoints in the individual studies) should be selected for display and evaluation.
Ordinarily, studies with similar controls (placebo control, active control) should be discussed together. A variety of methods have been used to display study results including tables, although graphic displays are more often helpful. It is increasingly common to use forest plots to display each study result on a common vertical axis, giving results (absolute effect versus placebo or hazard ratio) measured on the X axis. These plots can show p-values. To aid in the interpretation of point estimates, confidence intervals for treatment effects also should be given. If differences in change from baseline are shown for placebo and test drugs, the baseline values and the magnitude of effect in all treatment groups, including placebo and active controls (if used), generally should be presented in the table or in text accompanying a figure.
If there are important differences in outcome among studies of generally similar design, these differences should be displayed and an attempt should be made to explain why results were different, even though doing this can be difficult. Factors such as differences in subjects (disease definition, disease stage, severity, prior treatment, genetic differences among populations), drug dose or regimen, methods of observation, adherence to protocol or inadequate power (high beta error rate) may offer such an explanation and should be considered. Often such analyses will raise questions for future exploration rather than provide definitive answers.
Studies may exist that can serve as bridging studies intended to allow reliance on studies conducted outside the United States by showing that the clinical effect can be demonstrated in a U.S. population (see the ICH guidance for industry E5 Ethnic Factors in the Acceptability of Foreign Clinical Data). These studies should be particularly noted in this section. There also should be an analysis of the similarity of efficacy for subjects in different regions either study by study, subsets of studies, or pooled analyses. Any other information that may support extrapolation of the efficacy data to the new region also should be provided.
3. Analysis Issues
Support for the proposed claim should be described in terms of the strength of statistical evidence (including consistency of findings, individual study strength, p-value, and confidence interval) and findings that appear to weaken the actual claim. The applicant should include a summary and discussion of the analyses of each individual study, the extent of evidence in support of claims, statistical issues that may affect the conclusion on efficacy, and any related comments. Statistical issues should be summarized study by study, as well as collectively, for all studies conducted. Resolution of these issues and any effect on overall efficacy assessment should be discussed.
The following are examples of important statistical issues that may affect the results:
- Breaking the blind
- Unblinded or unplanned interim analyses
- High percentage of dropouts
- Inappropriate methods of imputation for missing values
- Change of primary endpoints during conduct of trial
- Dropping or adding treatment arms
- Sample size modification
- Adaptations to the design not planned in advance
- Inconsistency of results across subgroups
- Type I error inflation caused by multiplicity
4. Integrated Data Analyses
For the purposes of this guidance, the term integrated analysis refers to synthesizing the results of individual studies in an appropriate manner to collectively provide support for the claimed effectiveness of the study drug. Statistical literature formally uses the term meta-analysis, which was defined by Gene V. Glass as “the statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating findings.” 7 Examples of related terms used in literature include: analysis of combined data, combined analysis, analysis of pooled data, and pooled analysis. No matter what term is used, the objective is to use appropriately sound methods when formulating an integrated analysis.
As a general rule, when individual studies fail to show effectiveness on the basis of their planned analyses, a meta-analysis will not provide persuasive evidence of effectiveness. The decision to pool data usually is made after the fact and is potentially biased. In some cases, however, meta-analyses involving the entire study populations can provide useful information in an efficacy evaluation. Interpretability of meta-analyses depends upon the ability to minimize biases during the selection of studies for the analyses. Careful prospective planning is necessary to reduce these selection biases.
Meta-analyses or pooled analyses of clinical trials can provide useful information in a number of cases including:
- When examining various patient subgroups, such as those based on demographics, 8 etiology or severity of disease, or geographical regions, where individual studies might be expected to lack power.
- When assessing the dose-response relationship, especially if individual studies were conducted using different doses, and particularly in demographic subgroups.
- When assessing the drug effects on a secondary endpoint or on a component of a composite endpoint.
It is particularly important to plan such analyses in study protocols. For example, a mortality effect can be a planned pooled analysis of two studies using a composite outcome measure for each study. Pooled analyses also can be useful in evaluating time to effect and response rates (where the primary endpoint is a continuous variable).
Such results also can be used to design future trials including trials used to satisfy phase 4 commitments.
If a pooled analysis (or meta-analysis) of the clinical studies is performed, it should be clearly stated in the ISE whether this analysis was performed according to a predefined protocol or was a post hoc exercise. To avoid producing potentially biased results, decisions on how data will be analyzed should be done prospectively, before the results are known. It also should be clear whether including or excluding studies based on their observed outcomes will introduce bias.
Any differences in trial designs or populations, or in efficacy measurements, between studies should be described to allow an appropriate assessment of the relevance and validity of the results and conclusions (see ICH E9). A detailed description of the methodology and results of the meta-analysis should be provided.
It is important that the analysis avoid pooling studies with drastic heterogeneity, and it is also important to maintain the randomization procedure. Therefore, studies having different allocation ratios generally should not be lumped together and analyzed as a single study.
In summary, the points to consider with respect to meta-analyses are as follows:
- A meta-analysis without positive individual studies is not likely to be accepted as support for a claim unless a pooled analysis is specified as the primary endpoint.
- The measures and/or endpoints used in pooled or meta-analysis, such as survival, effects on less common endpoints, and dose response, should be specified and justified.
- Heterogeneity of the measure across trials should be considered and summarized.
- Measure and effect of the weight or influence of each study on the overall estimate of the measures should be given.
- The effect of varying trial durations, possibly including nonconstant hazard functions, should be considered. How duration is considered is critical to understanding therapies used in chronic diseases.
The results of individual studies or overview analyses of efficacy in specific populations should be summarized in this section. The purpose of these comparisons is to evaluate the observed treatment effect across all studies and to show whether the claimed treatment effects observed are not drastically inconsistent across all relevant subpopulations, especially those populations where there are special reasons for concern.
The comparisons can highlight apparent variations in efficacy that call for further investigation and discussion. However, the limitations of such analyses should be recognized (see ICH E9), and it is important to note that their purpose is not to provide the basis for specific claims or to attempt to improve the evidence of efficacy in situations where the individual study results are disappointing.
Given the limited sample sizes in individual studies, pooled analyses across multiple studies should be performed to evaluate effects of major demographic factors (e.g., age, sex, and race) and of other predefined or relevant intrinsic and extrinsic factors (e.g., disease severity, prior treatment, concomitant illness, concomitant drugs, alcohol, tobacco, body weight, renal or hepatic functional impairment) on efficacy. This analysis should be stratified and weighted by study size to minimize the chance of confounding caused by inherent differences in design specifications including baseline characteristics of the patient populations and use of concomitant medications across the studies to be pooled. Regional differences may need to be evaluated in multinational studies.
Factors of special interest can arise from general concerns (e.g., regarding the use of drugs in the elderly) or from specific issues that are related to the pharmacology of the drug or that have arisen during earlier drug development. If the application is being submitted for a new active ingredient, new indication, new dosage form, new dosing regimen, or new route of administration, the Pediatric Research Equity Act of 2007 (21 U.S.C. 355c) requires that the safety and efficacy of the drug be assessed in all relevant pediatric populations. A deferral and/or waiver of the pediatric study requirements may be appropriate in some cases, and pediatric studies do not necessarily need to be conducted before seeking approval of a drug for use in adults.
The comparison of results should be displayed using tables or figures/plots and should refer to the number of exposed subjects in the major subsets (e.g., age, race, sex). The examination of subsets need not routinely involve formal inferential statistical analysis.
Of interest are differences of clinically meaningful size. If these differences are not observed, there may be minor differences that reflect the fact that multiple subsets have been analyzed. These minor differences should be described although they need not be analyzed further. If a tendency toward a difference in the pooled analysis is seen, it also may be useful to look at study-by-study results. Differences that are consistently large across studies may generate a hypothesis for further study.
There might be cases in which formal inferential statistical analysis for specific subgroups may be needed to provide useful information in the drug label. For instance, the treatment effect may be most attributed to a specific subgroup. For such a labeling purpose, the integrated analysis should follow the clinical trial principle and statistical principle. The following issues that merit statistical consideration include:
- Whether or not the treatment effect in the overall population is conclusive
- Whether the integrated analysis is pre-planned
- Whether the studies pooled are drastically different with respect to patient population, dosage and study duration, pattern of missing values, among others
- How each individual study is weighted in the integrated analysis (usually it should be weighted by the study size)
- Whether proper multiple comparison adjustments are made for testing the multiple hypotheses (e.g., primary versus secondary endpoints) in the integrated analysis
- Whether the pooled study is acceptable as one randomized clinical study equivalent
- Whether the results of the integrated analysis are reproducible, if this analysis is acceptable for generating one-study worth evidence
For many NDA or BLA submissions, a well-constructed presentation of the results of several individual studies should be adequate to summarize the effectiveness results. Even in this case, however, a combined analysis usually will be needed to look at effects in demographic subsets and other subsets of the overall population. Interpretability of such a combined analysis rests upon careful prospective planning before commencement of the combination process. Careful prospective planning is needed to reduce the biases resulting from the combination process that is influenced by examination of study results.
This section should provide an integrated summary and analysis of all data, including data from individual dose-response clinical studies, relevant pooled analyses, and clinical pharmacological studies, that pertain to the dose-response or blood level-response relationships of effectiveness (including dose-blood level relationships). These data contribute importantly to dosing recommendations, including the choice of dose interval. The individual study results and any cross-study analyses that will be used to support the dosing recommendations (including the recommended starting and maximal doses, the method of dose titration, schedule, and any other instructions regarding individualization of dosage) should be summarized here. These results and analyses should include descriptions of relatively simple dose-response or blood level-response relationships as well as any identified deviations caused by nonlinearity of pharmacokinetics, delayed effects, tolerance, or enzyme induction. Limitations of the data (e.g., because titration designs were used instead of fixed-dose designs) should be candidly assessed.
Any evidence of differences in dose-response relationships that result from a subject’s age, sex, race, disease, or other factors should be described. Any evidence of different pharmacokinetic or pharmacodynamic responses also should be discussed, or discussions in section 2.7.2, Summary of Clinical Pharmacology Studies, CTD Module 2, can be cross-referenced. The ways in which such differences were looked for, even if no differences were found, should be described (e.g., specific studies in subpopulations, analysis of efficacy results by subgroup, or blood level determinations of the test drug).
Assessments of treatment response over time can be of interest. For example, is the persistence of efficacy over a time interval erratic or fairly consistent and how does the response over time relate to the dose? Therapeutic effects of a treatment can decline over time because of tolerability issues (patients who experience adverse events and refuse treatment) or from the development of drug resistance or tolerance. In such cases, all available information on persistence of efficacy over time should be summarized. The number of subjects for whom long-term efficacy data are available, the dose, duration of exposure, and the reason for censoring should be provided. Any evidence of tolerance (loss of therapeutic effects over time) should be noted. Examination of any apparent relationships between dose changes over time and long-term efficacy can be useful.
The primary focus should be on controlled studies specifically designed to collect efficacy data. These studies should be clearly differentiated from other, less rigorous studies, such as open extension studies. This distinction also applies to specific studies designed for evaluation of tolerance and withdrawal effects. Data concerning withdrawal or rebound effects pertinent to drug safety should be presented in the appropriate safety sections. Extension studies that enroll subjects who leave the controlled studies can provide valuable information and can permit true intent-to-treat analysis. Such analyses provide important information about the sensitivity of the analysis, especially for mortality or irreversible morbidity.
G. Exploratory Investigations
Results of exploratory analyses on nonspecified endpoints, patient subgroups, and pooled data can be reported. However, if such exploration was not prospectively defined, it will not meet the regulatory criteria for adequate and well-controlled clinical study, and, therefore, cannot be used as substantial evidence of efficacy (21 CFR 314.126). Nevertheless, these analyses can lead to proposals for additional studies and in some cases in which the analyses refine evidence of effectiveness that has already been established (e.g., demographic subset analyses), they can provide useful labeling information as well.
IV. Tables and Figures
Tables and figures should be embedded in the text when they enhance understanding of the document. Lengthy tables can be placed in the appendix at the end of the section. For eCTDs, hyperlinks to tables can be provided within the body of the ISE.
Tables should identify all studies pertinent to the evaluation of efficacy (including studies that were terminated or are not yet completed, studies that failed to show effectiveness for any reason, studies available as publications only, studies reported in full technical reports (ICH E3), and studies described in abbreviated reports) and should provide the most important results of those studies. However, it should be noted that unplanned interim analyses on ongoing studies generally are not needed and are not encouraged. When more than one ISE is provided for an application with more than one indication, usually each section should have its own appendix with tables.