 |
|
2005 Assisted Reproductive Technology (ART) Report: Appendix A |
|
How to
Interpret a Confidence Interval |
Findings from
Validation Visits for 2005 ART Data |
Discrepancy Rates by Data Fields Selected for Validation
How to Interpret a
Confidence Interval
What is a confidence interval?
Simply speaking, confidence intervals are a useful way to consider margin
of error, a statistic often used in voter polls to indicate the range
within which a value is likely to be correct (e.g., 30% of the voters
favor a particular candidate with a margin of error of plus or minus
3.5%). Similarly, in this report, confidence intervals are used to
provide a range that we can be quite confident contains the success rate
for a particular clinic during a particular time.
Why do we need to consider confidence intervals if we already know
the exact success rates for each clinic in 2005?
No success rate or statistic is absolute. Suppose a clinic performed 100
cycles among women younger than 35 in 2005 and had a success rate of 20%
with a confidence interval of 12%–28%. The 20% success rate tells us
that the average chance of success for women younger than 35 treated at
this clinic in 2005 was 20%. How likely is it that the clinic could
repeat this performance? For example, if the same clinic performed
another 100 cycles under similar clinical conditions on women with
similar characteristics, would the success rate again be 20%? The
confidence interval tells us that the success rate would likely fall
between 12% and 28%.
Why does the size of the confidence interval vary for different
clinics?
The size of the confidence interval gives us a realistic sense of how
secure we feel about the success rate. If the clinic had performed only
20 cycles instead of 100 among women younger than 35 and still had a 20%
success rate (4 successes out of 20 cycles), the confidence interval
would be much larger (between 3% and 37%) because the success or failure
of each individual cycle would be more significant. For example, if just
one more cycle had resulted in a live birth, the success rate would have
been substantially higher—25%, or 5 successes out of 20 cycles.
Likewise, if just one more cycle had not been successful, the success
rate would have been substantially lower—15%, or 3 out of 20 cycles.
Compare this scenario to the original example of the clinic that
performed 100 cycles and had a 20% success rate. If just one more cycle
had resulted in a live birth, the success rate would have changed only
slightly, from 20% to 21%, and if one more cycle had not been
successful, the success rate would have fallen to only 19%. Thus, our
confidence in a 20% success rate depends on how many cycles were
performed.
Why should confidence intervals be considered when
success rates from different clinics are being compared?
Confidence intervals should be considered because success rates can be
misleading. For example, if Clinic A performs 20 cycles in a year and 8
cycles result in a live birth, its live birth rate would be 40%. If
Clinic B performs 600 cycles and 180 result in a live birth, the
percentage of cycles that resulted in a live birth would be 30%. We
might be tempted to say that Clinic A has a better success rate than
Clinic B. However, because Clinic A performed few cycles, its success
rate would have a wide 95% confidence interval of 18.5%–61.5%. On the
other hand, because Clinic B performed a large number of cycles, its
success rate would have a relatively narrow confidence interval of
26.2%–33.8%. Thus, Clinic A could have a rate as low as 18.5% and
Clinic B could have a rate as high as 33.8% if each clinic repeated its
treatment with similar patients under similar clinical conditions.
Moreover, Clinic B’s rate is much more likely to be reliable because the
size of its confidence interval is much smaller than Clinic A’s.
Even though one clinic’s success rate may appear higher than
another’s based on the confidence intervals, these confidence
intervals are only one indication that the success rate may be better. Other
factors also must be considered when comparing rates from two
clinics. For example, some clinics see more than the average number of
patients with difficult infertility problems, whereas others discourage
patients with a low probability of success. For more information see
important factors to consider when using the tables to
assess a clinic.
Findings
from Validation Visits for 2005 ART Data
Clinic site visits for validation of 2005 ART data were
conducted April through June 2007. During each visit, data reported by
the clinic were compared with information recorded in patients’ charts.
Records for 1,458 cycles at 30 clinics were randomly selected for
validation. These selected cycles included 560 cycles that resulted in a
pregnancy and 464 cycles that resulted in a live-birth delivery.
Discrepancy rates are listed below for key data
items that were validated for each of the selected cycles. Review of the
discrepancies indicated that in the majority of cases, the error did not
affect the success rates (included in the national summary table and in the
individual clinic tables). In addition to fully validating data for the
randomly selected 1,458 cycles, during each visit the validation team also
reviewed the documentation for every live birth that had been
reported to CDC. There were no cases found in which a live birth had been
reported erroneously. In all, validation indicated that the clinic success
rates presented in this report are valid.
Discrepancy Rates by Data Fields Selected for Validation
|
Data
Field Name |
Discrepancy
Rate*
(Confidence Interval†) |
Comments |
| Patient date of birth |
2.3%
(0.0–5.0) |
Nearly all discrepancies were within 1–2 years and did not result in
a change in categorization of age groups. |
| Diagnosis of infertility |
14.3%
(4.5–24.1) |
For slightly more than half of these cases, multiple causes of
infertility were found in the patient’s chart, but only a single
cause was reported. |
Type of ART (i.e., fresh
vs.
frozen; donor vs. nondonor) |
<1% |
|
| Use of
ICSI |
2.2%
(0.0–5.3) |
For
approximately half of these cases, there was no indication in the
patient’s chart that ICSI was used. |
Number of
embryos
transferred |
<1% |
Nearly all discrepancies involved higher-order
(>2) embryo transfers and were only a one-or two-embryo difference. |
| Outcome of ART treatment
(i.e., pregnant vs. not pregnant) |
<1% |
In most of these cases, the information in the patient’s chart
indicated that the patient was pregnant, but the pregnancy was not
reported. In two cases, the information in the chart indicated there
was no pregnancy. |
| Number of fetal hearts on
ultrasound |
<1% |
Of those with misreported number of fetal hearts, seven cases
resulted in a change in categorization of single-versus
multiple-fetus pregnancy. |
Pregnancy outcome
(i.e., miscarriage, stillbirth, and live birth) |
<1% |
In most of these cases, there was no information on pregnancy
outcome in the patient’s chart. In one case, the live birth was
reported as a stillbirth. In one case, the live birth was reported
as a spontaneous abortion. |
| Number of infants born |
2.6%
(0.9–4.3) |
In most of these cases, the patient’s chart had no information on
the number of infants born in the patient’s chart. In one case, a
multiple birth was reported as a singleton birth. |
| Cycle cancelation |
<1% |
In most of
these cases, the information in the patient’s chart indicated the
cycle was cancelled, but the canceled cycle was not reported. |
| Notes: ART = assisted
reproductive technology; ICSI = intracytoplasmic sperm injection.
* Discrepancy rates estimate the proportion of all treatment cycles
with differences for a particular data item. The discrepancy-rate
calculations weight the data from validated cycles to reflect the
overall number of cycles performed at each clinic. Thus, findings
from larger clinical practices were weighted more heavily than
findings from smaller practices.
† This table shows a range, called the 95% confidence interval,
which conveys the reliability of the discrepancy rate. For a more
information, see
explanation of confidence intervals. |
Previous ART Reports
Implementation
of the Fertility Clinic Success Rate and Certification Act of 1992
Assisted
Reproductive Technology: Embryo Laboratory
Page last reviewed: 12/12/07
Page last modified: 12/12/07
Content source: Division
of Reproductive Health,
National Center for Chronic Disease
Prevention and Health Promotion
|
|
|
