|
2003 Assisted Reproductive Technology (ART) Report: Appendix A |
|
How to
Interpret a Confidence Interval |
Findings from
Validation Visits for 2003 ART Data |
Discrepancy Rates by Data Fields Selected for Validation
How to Interpret a
Confidence Interval
What is a confidence interval?
Simply speaking, confidence intervals are a useful way to consider margin
of error, a statistic often used in voter polls to indicate the range
within which a value is likely to be correct (e.g., 30% of the voters
favor a particular candidate with a margin of error of plus or minus
3.5%). Similarly, in this report, confidence intervals are used to provide
a range that we can be quite confident contains the success rate for a
particular clinic during a particular time.
Why do we need to consider confidence intervals if we already know
the exact success rates for each clinic in 2003?
No success rate or statistic is absolute. Suppose a clinic performed 100
cycles among women younger than 35 in 2003 and had a success rate of 20%
with a confidence interval of 12%–28%. The 20% success rate tells us that
the average chance of success for women younger than 35 treated at this
clinic in 2003 was 20%. How likely is it that the clinic could repeat this
performance? For example, if the same clinic performed another 100 cycles
under similar clinical conditions on women with similar characteristics,
would the success rate again be 20%? The confidence interval tells us that
the success rate would likely fall between 12% and 28%.
Why does the size
of the confidence interval vary for different clinics?
The size of the confidence interval gives us a realistic sense of how
secure we feel about the success rate. If the clinic had performed only 20
cycles instead of 100 among women younger than 35 and still had a 20%
success rate (4 successes out of 20 cycles), the confidence interval would
be much larger (between 3% and 37%) because the success or failure of each
individual cycle would be more significant. For example, if just one more
cycle had resulted in a live birth, the success rate would have been
substantially higher—25%, or 5 successes out of 20 cycles. Likewise, if
just one more cycle had not been successful, the success rate would have
been substantially lower—15%, or 3 out of 20 cycles. Compare this scenario
to the original example of the clinic that performed 100 cycles and had a
20% success rate. If just one more cycle had resulted in a live birth, the
success rate would have changed only slightly, from 20% to 21%, and if one
more cycle had not been successful, the success rate would have fallen to
only 19%. Thus, our confidence in a 20% success rate depends on how many
cycles were performed.
Why should confidence intervals be considered when success rates from
different clinics are being compared?
Confidence intervals should be considered because success rates can be
misleading. For example, if Clinic A performs 20 cycles in a year and 8
cycles result in a live birth, its live birth rate would be 40%. If Clinic
B performs 600 cycles and 180 result in a live birth, its live birth rate
would be 30%. We might be tempted to say that Clinic A has a better
success rate than Clinic B. However, because Clinic A performed few
cycles, its success rate would have a wide 95% confidence interval of
18.5%–61.5%. On the other hand, because Clinic B performed a large number
of cycles, its success rate would have a relatively narrow confidence
interval of 26.2%–33.8%. Thus, Clinic A could have a rate as low as 18.5%
and Clinic B could have a rate as high as 33.8% if each clinic repeated
its treatment with similar patients under similar clinical conditions.
Moreover, Clinic B’s rate is much more likely to be reliable because the
size of its confidence interval is much smaller than Clinic A’s.
Even though
one clinic’s success rate may appear higher than another’s based on the
confidence intervals, these confidence intervals are only one
indication that the success rate may be better. Other factors also must be
considered when comparing rates from two clinics. For example,
some clinics see more than the average number of patients with difficult
infertility problems, whereas others discourage patients with a low
probability of success. For further information see,
important factors to consider when using the tables to assess a clinic.
Findings
from Validtion Visits for 2003 ART Data
Clinic site visits for
validation of 2003 ART data were conducted March through June 2005. During
each visit, data reported by the clinic were compared with information
recorded in patients’ charts. Records for 1,849 cycles at 39 clinics were
randomly selected for validation. These selected cycles included 651
cycles that resulted in a pregnancy, including 533 cycles that resulted in
a live-birth delivery.
Discrepancy rates are listed on the next page for key data items that were
validated for each of the selected cycles. Discrepancy rates were low
(below 3%). Additionally, review of the discrepancies indicated that in
the majority of cases, the error was minor and did not
affect the success rates (included in the national summary table and in
the individual clinic tables). In addition to fully validating data for
the randomly selected 1,849 cycles, during each visit the validation team
also reviewed the documentation for every live birth that had been
reported to CDC. There were no cases found in which a live birth had been
reported erroneously. In all, validation indicated that the data are being
accurately reported by the clinics and that the success rates presented in
this report are valid.
Discrepancy Rates by Data Fields Selected for Validation
Data
Field Name |
Discrepancy
Rate |
Comments |
Patient
age |
1.9% |
Nearly all discrepancies were within
1–2 years and did not result in a change in categorization of age
groups. |
Diagnosis
of infertility |
2.4% |
For many discrepancies, multiple causes of infertility had been
diagnosed in the couple, but only a single cause had been reported
to CDC. |
Type of
ART (i.e., fresh versus frozen; donor versus nondonor) |
<1% |
|
Use of
ICSI |
1.6% |
|
Number of
embryos
transferred |
1.5% |
Nearly all discrepancies involved higher-order (>2) embryo transfers
and were only a 1– or 2–embryo difference. |
Outcome of
ART treatment (i.e., pregnant versus not pregnant) |
<1% |
In most of these cases, there was no information on pregnancy in the
patient chart. In 2 cases, the information in the chart indicated
there was no pregnancy. |
Number of
fetal hearts on
ultrasound |
1.2% |
Of those with misreported number of fetal hearts, only 2 cases (<1%
of total) resulted in a change in categorization of single- versus
multiple-fetus pregnancy. |
Pregnancy
outcome
(i.e., miscarriage, stillbirth,
and live birth) |
<1% |
In most of these cases, there was no information on pregnancy
outcome in the patient’s chart. In 2 cases, the information in the
chart indicated there was no live birth. |
Number of
infants born |
<1% |
In most of these cases, there was no information
on the number of infants born in the patient’s chart. In 5 cases, a
twin delivery was recorded in the patient chart and a singleton
delivery was reported to CDC. In 1 case, a singleton delivery was
recorded in the patient chart and a twin delivery was reported to
CDC. |
Cycle Cancellation |
<1% |
|
Notes: ART
= assisted reproductive technology; ICSI = intracytoplasmic sperm
injection. |
Previous ART Reports
Implementation
of the Fertility Clinic Success Rate and Certification Act of 1992
Assisted
Reproductive Technology: Embryo Laboratory
Date last reviewed:
03/23/2006
Content source: Division
of Reproductive Health,
National Center for Chronic Disease
Prevention and Health Promotion
|
|
|