Note from the National Guideline Clearinghouse (NGC): The National Institute for Health and Clinical Excellence (NICE) commissioned an independent academic centre to perform a systematic literature review on the technology considered in this appraisal and prepare an assessment report. The assessment report for this technology appraisal was prepared by Peninsula Technology Assessment Group (PenTAG), University of Southampton (see the "Availability of Companion Documents" field).
Critical Appraisal Strategy
Assessments of randomised controlled trial (RCT) quality were performed using the indicators shown below. Results were tabulated and these aspects described.
Internal Validity
- Sample size
- Power calculation at design
- Selection bias
- Explicit eligibility criteria
- Proper randomisation and allocation concealment
- Similarity of groups at baseline
- Performance bias
- Similarity of treatment other than the intervention across groups
- Attrition bias and intention to treat analysis
- All patients are accounted for
- Number of withdrawals specified and reasons described
- Analysis undertaken on an intention to treat (ITT) bases
- Detection bias
- Blinding
- Objective outcome measures
- Appropriate data analysis
- Any potential conflict of interest was noted (for example, financial support provided to studies and/or authors by manufacturers of the devices).
External Validity
External validity was judged according to the ability of a reader to consider the applicability of findings to a patient group in practice. Study findings can only be effectively generalisable if they (a) describe a cohort that is representative of the affected population at large or (b) present sufficient detail in their outcome data to allow the reader to extrapolate findings to a patient group with different characteristics.
Generalisability of included studies was assessed by examining the age, the percentage of participants with AF and the gender profile of the included patients, as well as their baseline QRS and left ventricular ejection fraction (LVEF) levels. Studies that appeared representative of the United Kingdom (UK) population with regard to these factors were judged to have high external validity.
Methods of Analysis and Synthesis
Given the time-related nature of mortality and morbidity, where possible these outcomes were reported as hazard ratios (HRs) (with their 95% confidence intervals). Where not reported, hazard ratios were derived from Kaplan-Meier curves or log rank test using the method of Parmar and colleagues. The trials in this review reported outcomes at differing follow-up points. Pooling results at different time points depends on the assumption of a constant treatment effect over time. Using the outcome of time to all cause death, the Assessment Group tested and confirmed the appropriateness of this assumption (see Appendix 5 of the Assessment Report [see the "Availability of Companion Documents" field]). Binary and continuous outcomes were summarised as relative risks and weighted mean differences respectively. Given the potential for repeated events, hospitalisation related to heart failure was also expressed as a rate ratio. Risks of adverse events were combined using simple pooling, i.e., without weights and by study.
Where appropriate, data were pooled using a fixed-effects model, except where statistical heterogeneity existed (p<0.1) according to the chi-squared statistic, when a random-effects model was used instead. Reasons for heterogeneity were explored using meta-regression. Data are expressed as means and 95% confidence intervals. All analyses were performed using Stata Software. Forest plots were produced using Stats Direct.
Five subgroups were identified at the outset. These were age, atrial fibrillation, New York Heart Association (NYHA) class, degree of left ventricular systolic dysfunction (LVSD) (i.e., % LVEF) and degree of dyssynchrony (i.e., QRS duration). The study reports of included trials were examined for data on these particular subgroups.
Potential publication bias was assessed by visual inspection of funnel plots and inferential testing using the Egger test.
Cost-Effectiveness
Study Quality Assessment
The methodological quality of the economic evaluations was assessed according to the international consensus-developed criteria list of questions developed by Evers and colleagues. Any studies based on decision models were also assessed against the ISPOR guidelines for good practice in decision analytic modelling.
Data Extraction Strategy
Data were extracted by one researcher into two summary tables: one to describe the study design of each economic evaluation and the other to describe the main results. See Appendix 6 of the Assessment Report (see the "Availability of Companion Documents" field).
In study design table: author and year; model type or trial based; study design (e.g., cost-effectiveness analysis (CEA), cost utility analysis (CUA) or cost-analysis); service setting/country; study population; comparators; research question; perspective, time horizon, and discounting; main costs included; main outcomes included; sensitivity analyses conducted; and other notable design features were recorded. See Table 80 in the Assessment Report (see the "Availability of Companion Documents" field).
For modelling-based economic evaluations a supplementary study design table recorded further descriptions of: model structure (noting its consistency with the study perspective, and knowledge of disease/treatment processes; sources of transition and chance node probabilities; sources of utility values; sources of resource use and unit costs; handling of heterogeneity in populations; evidence of validation (e.g., debugging), calibration against external data and comparison with other models).
In the results table, for each comparator, we show incremental cost; incremental effectiveness/utility and incremental cost-effectiveness ratio(s) (ICER). Excluded comparators on the basis of dominance or extended dominance were also noted. The original authors' conclusions were noted, and also any issues they raised concerning the generalisability of results. Finally the reviewers' comments on study quality or generalisability (in relation to the TAR scope) of their results were recorded.