The analytic goals were expanded from the previous guideline. In addition to meta-analyzing the randomized controlled trials to determine if there were significant differences among the treatments, the Panel also decided to develop outcomes tables and to actually provide estimates of outcomes for the different treatment modalities. To generate an outcome table, estimates of the probabilities and/or magnitudes of the outcomes are required for each intervention. Ideally, these come from a synthesis or combination of the evidence. Such a combination can be performed in a variety of ways depending on the nature and quality of the evidence. For example, if there is one good randomized controlled trial, the results of that trial alone may be used in the outcome table while findings of other studies of lesser quality are ignored. Alternatively, if there are no studies of satisfactory quality for certain outcome table cells or if available studies are not commensurable, expert opinion may be used to complete those cells. Finally, if a number of studies have some degree of relevance to a particular cell or cells, then meta-analytic mathematical methods may be used.
A variety of specific meta-analytic methods are available, and selection of a particular method depends on the nature of the evidence. For this guideline, the Panel elected to use the confidence profile method, which provides methods for analyzing data from studies that are not randomized controlled trials.
Three different meta-analyses of the efficacy data were performed:
- Meta-analysis of the comparable randomized controlled trials to determine the differences between pairs of available treatments. This analysis provides estimates of the absolute differences.
- Meta-analysis of the individual arms of the randomized controlled trials to combine all the data from such trials for each treatment. This "single-arm" analysis provides an estimate of the actual rate of occurrence of each outcome.
- Meta-analysis of the individual arms from all studies regardless of study design. For complications and side effects, only this method was used.
The Fast*Pro software was used to perform the meta-analyses. Many of the studies included in the meta-analysis had varying results. The variation in outcomes from study to study may have resulted from differences in patient populations, in how the intervention was performed, or in the skill of those performing the intervention. Given these differences, a random effects or hierarchical model was used to combine the studies. A random-effects model assumes that there is an underlying true rate for the outcome being assessed for each study. It further assumes that this underlying rate varies from site to site. This site-to-site variation in the true rate is assumed to be normally distributed. The method of meta- analysis used in analyzing the data attempts to determine this underlying distribution.
The results of the Confidence Profile Method are probability distributions that are described using the median of the distribution with a confidence interval. In this case, the 95% confidence interval indicates that the probability (Bayesian) of the true value being outside the interval is 5%. These Bayesian confidence intervals are sometimes called credible intervals.
The Bayesian method of computation assumes a "prior" distribution that reflects knowledge about the probability of the outcome before the results of any experiments are known. The prior distributions selected for this analysis are among a class of "noninformative" prior distributions, which means that they correspond to little or no prior knowledge. The existence of such a prior distribution can cause small changes in results, particularly for small studies. The prior distribution for all probability parameters is Jefferey's prior (beta distribution with both parameters set to 0.5). The prior for the variance for the underlying normal distribution is gamma distributed with both parameters set to 0.5.
In addition to the outcomes tables, graphs (Appendices 6 to 8 in the original guideline document) were developed to visually show selected treatment differences.
Efficacy Analysis
The outcomes analyzed for efficacy included recurrence and progression. A variety of methods of measuring recurrence were extracted, including probability of recurrence (percentage of patients with recurrence), time to recurrence, and time between recurrences. However, only probability of recurrence provided sufficient data for analysis. Similar measures also existed for progression, including time to progression and probability of progression. Moreover, there were different types of progression recorded including stage, grade, metastasis, and cystectomy. Ultimately, the Panel decided that only probability of progression could be analyzed. Progression was defined as progression in stage or to cystectomy.
The meta-analyses were conducted in three ways:
- Meta-analysis of comparable randomized controlled trials—this method used controlled trial data as reported to determine the difference between two treatments. The meta-analytic result gives an estimate of the absolute magnitude of the difference and whether it reaches statistical significance (p<0.05).
- Meta-analysis of comparable arms of randomized controlled trials—this method combines the individual arms reflecting the same treatment from controlled trials. For example, if one randomized controlled trial compared transurethral resection of bladder tumor (TURBT) alone to mitomycin C and another compared mitomycin C to bacillus Calmette-Guérin (BCG), the two mitomycin C arms would both be included in creating the mitomycin C estimate.
- Meta-analysis of comparable arms from all studies—this method combines arms as in method two but includes data from clinical series as well as randomized controlled trials.
Thus three outcomes tables exist for the efficacy data. The outcomes tables for methods two and three are formatted the same. Because the first method produces pair-wise results, the table is necessarily formatted differently.
Data from randomized controlled trials dealing with mitomycin C and/or BCG from the data extracted for the previous guideline were included in all three analyses. Other data from the previous guideline were not included.
One issue that is problematic when meta-analyzing data about time points is how to deal with losses to follow-up. Although most studies reported Kaplan-Meier data for recurrence (fewer for progression), not all studies provided the number of patients at risk. In order to avoid penalizing those studies which included numbers at risk, the initial study size was used as the denominator in all meta-analyses at all time points.
Complications
Different studies grouped complications into varying categories. They also used different terms for similar complications. The Panel grouped complications in an attempt to include all similar complications. Complications were variably reported. Only studies that specifically reported data concerning occurrences of complications were included in complication analyses. The Panel did not assume that the lack of reporting implied the lack of occurrence of any specific complication.
Also, some investigators may only have reported complications that had occurred and did not report that a complication did not occur. Combining complications into categories reduced the possibility of an overestimation of the complication rate. The probability that a patient would have a complication was still most likely slightly overstated because some patients experienced multiple complications. Thus, the result of the meta-analysis was best interpreted as the mean number of complications the patient may experience rather than as the probability of having a complication. There were insufficient data to permit meaningful meta-analyses of patient deaths. The estimates of death rates provided in the guideline result from the Panel's expert opinion and the limited available data.
Patient Groups
The Panel attempted to evaluate outcomes based on a variety of patient characteristics including stage, grade, tumor multiplicity, and recurrence. However, in most cases, the outcomes data were not fully or consistently stratified by these conditions. Ultimately, the Panel elected to analyze the combined data from all studies and also the individual data sets for high- and low- risk patients. Low risk was defined as Grade 1. High risk included groups that had no Grade 1 patients or were entirely carcinoma in situ and/or T1.
Treatments
The Panel considered a wide variety of treatments. However, limited data were available for many of the treatments of interest. Ultimately, the Panel decided that it could not distinguish between the different types of TURBT, including repeat TURBT. All forms of TURBT were considered the same. The Panel also considered maintenance therapy versus induction only. A wide variety of induction and maintenance schedules have been used and reported in the literature. The Panel ultimately decided that any treatment administered for a longer time period than an initial induction regimen would be considered as maintenance therapy. Finally, a single postoperative dose of mitomycin C was examined as a third alternative dosing regimen.
Because the issues surrounding the comparison of BCG and mitomycin C maintenance therapy and induction alone were so important, the Panel elected to combine data from the randomized controlled trials included in the original guideline with the data from the current analyses. Nonrandomized studies or studies of other regimens from the earlier guideline were not included.