Note from the National Guideline Clearinghouse (NGC): The National Institute for Health and Clinical Excellence (NICE) commissioned an independent academic centre to perform a systematic literature review on the technology considered in this appraisal and prepare an Evidence Review Group (ERG) report. The ERG report for this technology appraisal was prepared by the Peninsula Technology Assessment Group, Peninsula Medical School and Wessex Institute for Health Research and Development, University of Southampton (see the "Availability of Companion Documents" field).
Clinical Effectiveness
Description and Critique of the Manufacturer's Approach to Validity Assessment
The manufacturer uses items based on the CONSORT statement to critically appraise the randomised controlled trials (RCTs) of both natalizumab and comparators. They also provide a Jadad score. The submission concludes that the trials of natalizumab are as good or better than those for the comparators although, as they themselves note, as they have access to trial information on their own databases, this is perhaps not surprising – all assessment of comparators were based on the published trial reports only.
The conclusion that the natalizumab trials are well conducted is reasonable (refer to the assessment of quality in Appendix 3 of the ERG report) (see the "Availability of Companion Documents" field). Methods of randomisation and blinding were adequate, and although there was some drop out, this was small and similar in both arms. However, little consideration is given to relevant external validity. The assessment concludes that epidemiology of multiple sclerosis (MS) in the multicentre trial is likely to be similar to that in the UK but do not note that the studied population does not match the licensed populations.
Description and Critique of the Manufacturer's Statistical Approach
The statistical analysis of the AFFIRM trial was well reported.
The AFFIRM trial was adequately powered. The sample size was based on 90% power to detect an assumed annualised relapse rate of 0.6 with natalizumab and 0.9 with placebo with 15% drop out and required 900 people at 5% significance. Progression rates at the end of two years were assumed to be 35% with placebo and 23% with natalizumab. These effect sizes were met in the trial.
Appropriate statistical methods were used to compare the two groups. The U.S. Food and Drug Administration (FDA) statistical review did raise the issue that since randomisation was stratified by site, site should have been incorporated as a covariate in the primary analysis. However, such adjustment is not statistically mandatory and as the FDA statistical reviewers concede, sparse data in some sites may have made such adjustment for centres problematic.
The rapidly evolving severe (RES) subgroup is based on a smaller subgroup of the AFFIRM trials (n=209). As the impact of natalizumab is greater than predicted, significant results are seen in this subgroup. Details of withdrawals are not given.
Data Syntheses
The submission does not statistically pool information about natalizumab treatment effect and this is appropriate given the different treatment and comparator regimes in the AFFIRM and SENTINEL trials, and the short term follow up in the MS201 and 231.
The submission does pool information from AFFIRM, MS201 and MS231 on safety. As the length of follow-up in these three trials is different (2 years, 12 weeks and 24 weeks), it may have been more appropriate to use rate ratios, rather than the risk ratios used in the submission. Given the shorter follow up period in MS231 and MS201, it is possible that these trials may bias the results in favour of natalizumab, as there may be less adverse effects with less exposure to the drug.
Refer to Section 4 of the ERG Report (see the "Availability of Companion Documents" field) for more information.
Cost-Effectiveness
Overview/Summary of Manufacturer's Economic Assessment
The manufacturer submission reports cost effectiveness analyses (CEA), presenting cost per quality-adjusted life year (QALY) estimates for natalizumab compared to:
- Best supportive care (BSC) (BSC reflects the placebo arm of RCTs)
- Beta-interferon (IFN-beta)
- Glatiramer acetate (GA)
Cost per QALY estimates are presented for the RES and sub-optimal therapy (SOT) subgroups of patients with relapsing-remitting multiple sclerosis (RRMS).
The CEA uses a decision-analytic model (Markov-process cohort model) to estimate the incremental costs and benefits associated with natalizumab treatment, versus stated comparators. The main components of this model are summarised in the ERG Report (see the "Availability of Companion Documents" field).
Sensitivity Analysis
Extensive one-way sensitivity analysis is reported, plus multi-way sensitivity analysis and probabilistic sensitivity analysis (PSA). (Table 10 in the ERG Report [see the "Availability of Companion Documents" field])
Model Validation
The submission reports consideration of model validation, including tests for internal consistency, and consideration of the model in the context of model inputs (transit probabilities) and other analyses of disease modifying therapies (DMTs) for MS.
Critical Appraisal of Manufacturer's Economic Evaluation
Internal Consistency
The ERG has undertaken extensive checking of the Excel programming and mathematical logic of the model, and find the model to be well set out and accurate (with the exception of the items listed in the ERG Report [see the "Availability of Companion Documents" field]). All equations in the model have been checked for internal consistency/accuracy. The model is fully executable and the ERG has been able to replicate CEA results presented in the submission, the sensitivity analysis presented (in almost all scenarios), and the probabilistic sensitivity analysis presented.
External Consistency
The manufacturer submission reports that the external consistency of the model has been considered. The issue of validity is considered in the context of (1) external independent review, (2) predictions of the model compared to data inputs, and (3) consideration of model outputs against other studies of a similar nature.
The submission states that the methods used were valid. The submission compares AFFIRM data to the multi-state-model (MSM) used to derive transit probabilities, presenting evidence that the predictive power of the transit probabilities derived from the MSM model is high.
The ERG has concerns over the different rate of disability progression predicted in the manufacturer model and that presented in the AFFIRM trial against the primary endpoint of AFFIRM. They have undertaken additional analysis to compare the prediction of the model against the endpoint of the AFFIRM trial.
The ERG's analysis of the manufacturer's model shows a much higher cumulative probability of sustained disability progression (at 2-years) for the SOT subgroup than in the AFFIRM trial. Results show a cumulative probability of progression of 33% in the natalizumab treated group versus 53% for BSC (absolute risk reduction of 20%). A similar conclusion applies for the RES subgroup.
One-Way Sensitivity Analyses
The manufacturer submission and subsequent addendum present extensive sensitivity analyses on the key parameters in the model.
The ERG has checked all of these univariate-sensitivity analyses, and agrees with all except two. In the two cases shown in Table 16 of the ERG Report they find different cost per QALY data.
Refer to Section 5 of the ERG Report (see the "Availability of Companion Documents" field) for more information.
