Evidence-based
Evaluation and Decision Process for the
Advisory Committee on Heritable Disorders
and Genetic Diseases in Newborn and Children:
A Workgroup Meeting Summary
October 23, 2006
On this page:
I. Introduction
II. Background on the ACHDGDNC
III. Overview of the Nomination
Form
IV. Issues in Evidence Review
for Genetics, Pediatric and Newborn Screening
V. Case Study: Evaluation
of Pompe Disease
VI. Discussion
The nomination
form
Evidence evaluation
process
Expertise needed
on the review workgroup
Revisiting the current
uniform panel
VII. Moving Forward
Appendix A: Agenda
Appendix B: Participant
List
I. Introduction
The Children’s Health Act of 2000
was passed by Congress as an amendment
to the Public Health Service Act to revise,
extend and establish programs with respect
to children’s health research, health
promotion and disease prevention activities
conducted through Federal public health
agencies. Initiatives mentioned in the
Act included efforts related to autism,
asthma, childhood obesity and, in Title
XXVI, improvements in the ability of States
to provide newborn and childhood screening
for heritable disorders. This provision
also stated explicitly the need to create
an Advisory Committee “to provide
advice and recommendations to the Secretary
for the development of grant administration
policies and priorities, and to enhance
the ability of the Secretary to reduce
mortality or morbidity from heritable
disorders.” Subsequently, the Advisory
Committee on Heritable Disorders and Genetic
Diseases in Newborns and Children (ACHDGDNC)
began convening in 2004 to advise and
guide the Secretary on the most appropriate
application of universal newborn screening
tests, technologies, policies, guidelines
and programs for reducing negative outcomes
of heritable disorders.
In 2006, the ACHDGDNC made a formal recommendation
to the Secretary of Health and Human Services
outlining an initial uniform screening
panel of 29 conditions suggested for inclusion
in State newborn screening programs. The
selection of these conditions was based
on a report authored by the American College
of Medical Genetics (ACMG) and commissioned
by the Health Resources and Services Administration
(HRSA). Concurrently, the ACHDGDNC has
been attempting to establish criteria
for assessing the inclusion of other diseases,
based on available evidence.
On October 23, 2006, the ACHDGDNC’s
decision-making criteria workgroup convened
a group of experts to discuss how to establish
a rigorous evaluation and recommendation
process to determine the suitability of
screening for certain rare inherited disorders.
The experts—who came from a range
of backgrounds in pediatrics, genetics,
and public health—were requested
to:
Examine a draft nomination form and comment
on the criteria that an evidence-based
workgroup would need for evaluating conditions
for screening newborns and children for
heritable disorders;
Develop options for the ACHDGDNC on
the structure of the evidence-based workgroup;
and
Determine whether an iterative process
should be tied to the decision-making
process of the ACHDGDNC, and if so, how
to tie the evaluation and recommendation
process to an established infrastructure
accompanied by this iterative process.
Nancy Green, M.D., Medical Director of
the March of Dimes Birth Defects Foundation
and one of the ACHDGDNC members, who helped
to develop the sample nomination form,
noted that the day’s discussion
would feed into the next ACHDGDNC meeting
in mid-December. Michele Lloyd-Puryear,
M.D., Ph.D. of HRSA stressed that the
recommendations arising out of this meeting
were not being directed to any Federal
agency, rather, they were feeding into
the criteria working group, which in turn
would present recommendations to the ACHDGDNC
about this process.
II. Background
on the ACHDGDNC
R. Rodney Howell, M.D., Ph.D. of the
University of Miami School of Medicine
and Chair of the ACHDGDNC, provided further
background on the ACHDGDNC and the issues
it faces. The ACHDGDNC is comprised of:
several ex officio voting members appointed
by the Secretary for Health and Human
Services from HRSA, CDC, NIH, and AHRQ;
medical, technical, public health and
scientific professionals with expertise
in heritable disorders or newborn screening;
non-voting representatives from other
medical and public health organizations;
and non-voting liaisons from the Advisory
Committees on Infant Mortality and Genetics,
Health and Society.
The preliminary tasks/issues the ACHDGDNC
has been tasked with addressing have included
the following:
Assessing capacity needs of States/newborn
screening programs including workforce
needs, States’ ability to conduct
technology analysis and translation into
practice, public education, and secondary
screening.
Parental education/notification/informed
decision-making.
Recommendations for a uniform panel
of conditions (which, as noted earlier,
has been submitted to the Secretary).
The process by which new tests/technology/conditions
will be considered.
Dr. Howell said that perhaps the most
complex of these tasks is determining
what evidence is used to decide what will
go on the uniform screening panel. He
noted that some issues are currently being
examined by sub-committees, such as laboratory
quality assurance issues and how to handle
long-term follow-up. The ACHDGDNC also
will establish workgroups with responsibility
for interagency coordination (particularly
the relationship with the HRSA regional
collaborative program) and to work with
NIH and others to set up research agendas
related to newborn screening.
While there has been an enormous amount
of publicity related to newborn screening,
Dr. Howell observed, most reviews have
found few detriments to doing such screening.
He argued that the day’s discussion
would touch upon some of the most important
issues surrounding newborn screening,
including the evidence related to diagnosis,
treatment and long-term outcomes of potentially
screenable disorders. Assessment of evidence
for screening newborns for rare heritable
disorders is unique in many regards, as
such an assessment must deal with extremely
rare conditions, the initial diagnosis
and immediate follow-up occurs within
the public health sector, and once a treatment
is deemed life-enhancing or –saving,
controlled studies most likely will not
be undertaken.
III. Overview
of the Nomination Form
Dr. Green and Piero Rinaldo, M.D., Ph.D.,
Mayo Clinic College of Medicine, proceeded
to brief the group on the development
of the nomination form and process for
adding conditions to the uniform panel.
The nomination form includes space for
information on the condition, the screening
test and the treatment, and was largely
based around the ACMG categories for assessing
evidence. However, Dr. Rinaldo noted that
there were some diversions from the ACMG
model, as this nomination form added mention
of risks (and benefits) associated with
screening and treatment. In addition,
the ACHDGDNC was careful not to include
on the form any request for cost data,
or any subjective measures.
The ACHDGDNC-approved draft process for
review also was presented for discussion.
Once a nomination form is completed, it
is sent to HRSA, who conducts an administrative
review. HRSA can request more information,
decline the form as incomplete, or send
it on to be reviewed by the ACHDGDNC.
The ACHDGDNC then examines the form and
determines whether to decline it, or to
form an ad hoc evidence workgroup to conduct
further research and analysis on the condition.
The expertise and the members of this
workgroup are to be determined. After
its analysis, the workgroup presents the
evidence for screening newborns and the
quality of that evidence to the ACHDGDNC.
The ACHDGDNC determines whether to decline
the addition of the condition or to recommend
adding it to the uniform panel for screening
of heritable disorders. Because there
will be no scores applied to the criteria
the challenge to ACHDGDNC will be to determine
where to set the bar for inclusion. This
process for nomination and review will
have a mechanism for constant re-evaluation.
Dr. Green added that addition of conditions
to the uniform panel is a constantly evolving
process, and that as new treatments emerge,
conditions that may have been rejected
for past inclusion can be reviewed again
in the future.
No process, however, is currently in
place for reviewing the current 29 conditions
that make up the uniform panel. This topic
was mentioned frequently throughout the
day and is discussed in further detail
in section VI.
Participants raised the question of what
data might lead the ACHDGDNC to reject
a nominated condition. Some of the reasons
include: issues related to the analytical
performance of screening (e.g., testing
leading to an excessive amount of false
positives) and a need for better understanding
of the natural history of the disease.
Dr. Green said that the ACHDGDNC has never
arrived at a particular cut-off point
for metrics of specificity and sensitivity,
nor has it begun the address the difficulty
of screening tests that move beyond the
dried blood spot model. In simplest terms,
the ACHDGDNC looks at a nomination form
and tries to assure that what they’re
looking at is indeed a disease, that it
can be identified reasonably well in screening,
and that there are desirable actions that
can be done after the screening that lead
to positive outcomes.
IV. Issues
in Evidence Review for Genetics, Pediatric
and Newborn Screening
Michael Watson, Ph.D. talked about the
origins of the ACMG report that was published
in 2006 and some of the thinking that
led up to the report’s recommendations,
including its primary recommendation of
the 29 conditions for the uniform screening
panel.
ACMG was asked by HRSA to outline a process
of standardization of outcomes and guidelines
for State newborn screening programs and
to define responsibilities for collecting
and evaluating outcome data, including
a recommended uniform panel of conditions
to include in these programs. Newborn
screening often has been characterized
as arising from uneven policies that have
led to perceptions of inequities in selection
of diseases for screening. The ACMG expert
panels recognized at the start that the
evidence base for most conditions under
scrutiny was largely made up of observational
studies and expert opinion. They designed
a system to collect expert evidence from
providers and families and have researchers
and other experts in the conditions review
the assembled evidence.
Once all the evidence had been collected
and reviewed, the group eliminated any
conditions that did not meet the minimum
of having a screening test available,
having a well-understood natural history,
and having an available treatment that
significantly alters outcomes. Following
the evidence review, the ACMG concluded
with two other observations/recommendations:
Tandem mass spectrometry should be used
for newborn screening on the basis of
MCAD, VLCAD and LCHAD, and is a screening
process that allows for improved identification
of other disorders, including PKU, IVA
and MSUD.
Per the overriding principles set by
the ACMG expert group, the newborn screening
program should report any other results
of clinical significance. This led to
delineation between clinically significant
primary targets of screening and other
conditions that may be included in the
differential diagnosis of a primary target
analyte.
Some of the confounding factors that
the ACMG group had to consider when doing
their evidence review included: avoiding
unwarranted assumptions of the type inherent
to the health technology assessment reports
being aware of the self-evident evidence
paradox (i.e., if an intervention is truly
effective, no one will study it); how
to incorporate the views of patients and
families with clinical and scientific
evidence; and how to state that referral/consultation
with experts may be necessary for PCPs.
Dr. Watson also detailed the issues faced
in assessing the evidence, such as:
Natural history: needs to start from
at-risk newborns, needs as many patients
as possible to further minimize bias,
confounded by continuous availability
of new treatments.
Incidence: general population data are
best, mostly rare diseases with wide range
of estimates, multiple genetic etiologies
for phenotypes, splitting conditions screened
by same analytes into multiple groups.
Onset: poor capture of penetrant cases,
newborn screening can detect both early
and late onset forms.
Burden: there is a “bias”
toward severe conditions.
Screening test: there is variability
among States and a slow evolution of the
gold standard. Iterative changes in screening
algorithms may detect clinically significant
secondary conditions.
Diagnosis: the process needs to recognize
increasing complexity with permutations
of genes involved.
Treatment: studies of treatments often
with limited involvement of pediatric
patients. Dilemmas such as how to integrate
treatment when the phenotype is common
to a number of rare conditions and many
different conditions may require similar
treatment and have comparable outcomes.
However, one of the primary issues is
the quality of evidence itself. Natural
history studies are increasingly difficult
as a result of ongoing intervention. Moreover,
there is limited data collection on the
national level despite obvious need and
recommendations to do so, and some data
that are collected are proprietary. Treatment
data are increasingly subject to reduced
pre-market studies with conditional approval
subject to more post-market surveillance.
Other evidence challenges for newborn
screening and pediatrics include the difficulty
in running randomized clinical trails
(RCTs), given the extreme rarity of the
conditions in question, the rapid growth
in interventions and technologies used
to screen and treat them, and the problem
enlisting infants and children in such
trials. The health of infants also is
laden with emotion and any missed case
resulting in the death of a child can
be compelling. Likewise, there are questions
related to what constitutes a “benefit”
in early detection and whether the evidence
is good enough—what is the likelihood
of getting better evidence in the near
future?
There are numerous evidence assessment
systems for public health that, while
valuable for making recommendations for
broad populations, have not been used
for newborn screening and genetics due
to concerns about the evidence base for
newborn screening and rare genetic disorders.
Dr. Watson briefly highlighted a number
of these—including the US Preventive
Services Task Force (USPSTF), Evidence-based
Practice Centers (EPCs), and Centers for
Education and Research on Therapeutics
(CERTs), Evaluation of Genomic Applications
in Practice and Prevention (EGAPP), to
name a few—and discussed the varying
reasons why they have failed for these
disorders, including problems related
to expense, inflexibility, lack of applicability
to rare diseases, or more treatment- rather
than screening-aimed foci.
The state of the genetic disease testing
evidence base largely is in disarray not
because of lack of diagnostic confirmation
or treatment data, but because of paucity
of incidence/prevalence data and the difficulty
in comparing screening test evidence.
After touching briefly upon developing
the evidence base for common versus rare
conditions, Dr. Watson stressed the need
for creation of registries to house data
on newborn screening patients and genetics,
which would in turn contribute to prospective
and retrospective studies of these conditions.
This point was taken up by the participants
in their discussion, who noted that the
practice model for metabolic (and genetic)
disease is not well organized or established.
Great value could arise from developing
consistent standards of care and from
gathering consistent data across sites
(much as has happened with pediatric oncology
work).Many conditions may have good small
sample testing of screening, but lack
any larger and more population-based testing.
There was much discussion of the potential
for State pilots and the value of moving
to population testing in one or more States
prior to more general use of a screening
test (detailed further in section VI).
V. Case Study:
Evaluation of Pompe Disease
To give the group an idea of how the
nomination form would work in practice,
Priya Kishnani, M.D., Associate Professor
in Pediatrics at Duke University, reviewed
the submission she created for Pompe disease.
A lysosomal enzymatic deficiency, Pompe
disease appears both in infantile and
late onset form. For the former, babies
typically present with severe hypotonia
and hypertrophic cardiomyopathy, resulting
in death within the first year of life.
These infants fail to meet most major
milestones for motor skills, especially
anything other than rolling over. The
late onset version of Pompe disease may
occur in childhood or adulthood and is
associated with progressive muscle weakness
and respiratory failure that also eventually
leads to death.
An article detailing a comprehensive
study into the natural history of Pompe
disease in 160 patients worldwide was
published in Pediatrics in 2003. This
article showed that for infantile Pompe,
the median age that symptoms appeared
was at two months, and that traditional
diagnosis (i.e., not using newborn dried
blood spot screening) generally was made
four months after those symptoms appeared,
thus leaving a short time period for therapeutic
intervention that could save the life
of the child. Indeed, traditional treatment
of Pompe was limited to supportive care.
In 2006, however, the Federal Drug Administration
licensed the first effective treatment
for Pompe disease, the drug Myozyme. In
clinical trials, Myozyme was shown to
have significantly altered the natural
course of Pompe disease: all infants in
the first phase of clinical studies lived
past the critical age of one year. Subsequent
studies proved that infants given Myozyme
made swift gains in motor skills, such
as standing, walking and running; had
improved muscle tone and cardiac function;
and the vast majority (14/18 patients
in one trial) were free from invasive
ventilators.
Integrating Pompe disease onto the newborn
screening uniform panel would allow for
more timely diagnoses and significantly
improve outcomes for those children with
infantile Pompe. The challenge with newborn
screening for this condition is that it
cannot distinguish infantile versus late
onset variations of the disease and the
screening test has not been validated
within a broad population.
Alex Kemper, M.D., M.P.H., M.S. then
reviewed some of the methodological challenges
facing those evaluating the evidence for
newborn screening of rare conditions.
The criteria used in the initial selection
of the 29 conditions on the uniform panel
were based on the following:
Clinical characteristics: such
as incidence, presentation during the
newborn period and burden of disease.
Screening tests: the
availability of sensitivity and specificity
tests, expense, and the ability to integrate
into existing newborn screening programs.
Diagnosis and follow-up:
availability and cost of treatment, its
efficacy and prevention of mortality,
etc.
Dr. Kemper reported that some individuals
expressed concerns about the methodology
of the ACMG report during the public comment
period; some of these concerns have since
been published. Examples include:
There was an inappropriate use of opinion
and survey methods.
The review of the literature lacked
an analytic framework or strategy, or
clear inclusion/exclusion criteria.
There was a limited scope, and issues
such as false positives, secondary targets
and ethics were not addressed in the report.
Dr. Kemper then presented his thoughts
on approaches to weighing evidence for
important policy decisions. First, there
are two strategies for doing a review
of the evidence. The traditional strategy
relies on narratives, usually written
by experts, which can themselves be biased.
The second strategy of a systematic review
includes methods and criteria for identification
of studies and data, criteria for inclusion/exclusion
of studies and data, and criteria for
judging study quality. Systematic reviews
may combine qualitative and quantitative
data, and may form the basis of cost-effectiveness
studies. The challenges to systematic
reviews relate to synthesizing the evidence,
especially when the evidence is sparse
because of a rare condition or new or
understudied test or treatment. Studies
may use different populations, study designs
and measurements. In addition, studies
of rare conditions and studies focused
on children are difficult to come by.
Systematic reviews and users of such reviews
face the challenging question of what
is enough evidence for drawing a conclusion.
Dr. Kemper added that not making a decision
is a decision in and of itself.
Other organizations have structured evidence
reviews in a variety of manners. The Cochrane
Collaboration—which produces a database
of systematic reviews—focuses on
RCTs and provides a plain language summary
as well as a detailed scientific summary
to accompany its recommendations. The
USPSTF, which makes recommendations about
preventive services in primary care, utilizes
EPCs to conduct systematic reviews, and
outside experts provide peer review. The
Task Force members then review the evidence
and issue recommendations on a scale of
A, B, C, D or I, ranging from strongly
recommended practices to recommending
against an intervention or concluding
that there is insufficient evidence to
make a recommendation.
In a paper submitted to the group, Dr.
Kemper suggested that a similar multi-tiered
voting scheme could be used by the ACHDGDNC.
For example, the different tiers of such
a scheme might include: universal screening
recommended, targeted screening recommended,
pilot study of screening recommended,
pivotal studies required, no general recommendations
or recommendations against screening.
This voting scheme would allow the ACHDGDNC
to guide the process of acquiring some
critical piece of information—for
example, a population-based pilot study. The
pilot efforts in MA and CA of MS/MS could
be examples of this.
Using Pompe disease as an example, Dr.
Kemper led the group through his vision
of a process by which to weigh the evidence
on Pompe and consider it for inclusion
in the panel. Underlining this decision-making
process was what he termed “important
decision #0” or that the
success of a review will be determined
if the results are replicable. The following
rounded out the steps of the Pompe evidence
review:
Important decision #1:
Focus on the infantile form of the disease,
and determine answers to key questions
regarding its natural history, diagnosis,
treatment and the experience of screening
programs.
Important decision #2: Cast
a broad net for data that looks at published
studies of all types, government databases,
unpublished data and anything except animal
data.
Important decision #3:
Do not use standard study quality assessment
instruments. Quality is still important,
though the instruments developed do not
translate well into smaller studies.
Regarding this last decision point, Dr.
Kemper suggested looking at the Canadian
Centre for Health Evidence for guidelines
for quality and assessment. The Centre
offers a number of key questions for primary
and secondary guidance to determine whether
study results are valid, as well as to
assess results and determine whether they
can be applied to patient care.
Dr. Kemper concluded by listing some
of the general challenges facing an evidence
review group for rare and heritable disorders.
Chief among these was deciding whether
or not to use unpublished data—a
topic which raised some debate later on
in the day and is discussed on page 11
of this report. Other issues included
how to adjust for standards of quality
given that RCTs are unlikely to occur
with these rare disorders, expectations
for how the data should be summarized,
the relationship of the methodologist
to the content expert, and how the final
reports should be structured and disseminated.
VI. Discussion
The remainder of the meeting provided
an opportunity for participants to discuss
some of the issues raised by the presentations
and the sample case of Pompe disease,
and to make recommendations to the working
group that would in turn advise the ACHDGDNC
on the uniform panel and its decision-making
processes. While tasked with commenting
specifically on the three discussion points
outlined on page 1 of this report, the
discussion also raised other relevant
points that must be taken into consideration
by the ACHDGDNC, as described below.
The nomination form
There was general consensus that the
nomination form as currently drafted can
provide the basic information about whether
there is evidence regarding available
diagnosis and treatment. An example was
offered regarding the EGAPP project, and
how it handled the nomination process.
An EGAPP representative stated that the
program has a similar process, receiving
a short summary about the disorder, test
and clinical scenario. Those nominations
that are given top priority are then asked
to give a longer summary based on criteria
set forth by EGAPP; this serves as the
jumping off point for the key questions
and analytic framework established for
the evidence review.
What were observed as lacking from the
ACHDGDNC form, however, were quantitation
of specificity and sensitivity, prevalence,
and the cost implications of the four
possible outcomes to screening (true positives,
true negatives, false positives and false
negatives). There was general consensus
to add a request for quantitation of specificity
and sensitivity to the nomination form,
however, there was some debate within
the group that it is unlikely that the
ACHDGDNC (or an evidence support group)
will be able to agree on absolute figures
for adequate sensitivity and specificity
(or other metrics). Presumably, the ACHDGDNC
could agree on minimum floors below which
a test is not satisfactory. A decision
analytic framework, outlined in the next
section on evidence evaluation process,
can help to put the information into a
set of questions and decision points for
the ACHDGDNC that allows better weighing
of alternatives.
It also was recognized that there will
be nominated conditions that might have
a good screening test but insufficient
evidence to demonstrate clinical validity
in a larger population base. Therefore
another key suggested improvement to the
form was adding a pilot study that could
prove that one could successfully screen
for the disease with the technology applicable
to population screening, i.e., proof of
a study offering clinical validity. There
are several ways to do controlled pilot
trials. For example, a disorder can
be tested and treated in one State using
a different State as a control. The
ACHDGDNC might also recommend that pilot
studies be conducted to close evidence
gaps: if there are no pilot studies for
tests or treatment, the ACHDGDNC could
recommend a population-based study to
obtain such information in lieu of conducting
an evidence review (and making a recommendation
to add to the core panel or not).
Finally, as assistance to those who might
submit nomination forms, one person recommended
that the ACHDGDNC provide an example of
a condition on the uniform panel that
successfully underwent the nomination
process, so that they might see how the
structure and supporting data should be
presented. MCAD was suggested to be the
sample case.
Evidence
evaluation process
The group went on to discuss the steps
that need to be in place for the evidence
review to occur. First and foremost, the
ACHDGDNC needs to establish an overall
goal or principle for how the evidence-based
working group will operate, such as, “The
overall benefits of screening must outweigh
the potential harms for the individual,
the family or society.” A decision-analytic
framework was suggested for the decision-making
process. Decision trees represent the
most common analytic framework, and should
be used to frame the evidence review process.
It was observed that a decision-analytic
framework makes explicit the assumptions
that go into the process. The group suggested
that a prototype decision tree could be
developed to guide nominators regarding
the types of data most needed in an evidence
review.
The evidence-based working group then
needs to define the overarching principles
for an acceptable evidence base for the
condition in question, and review those
recommendations with the ACHDGDNC. For
example, for many types of disorders,
the group might not require evidence in
the form of RCTs or double blind studies;
instead, it must realize that likely levels
of evidence for clinical efficacy may
come in the form of case studies and expert
opinions. Criteria for quality of case
studies and expert opinions must be set.
Sensitivity analysis was also recommended
as a part of all reviews; such analyses
may define what level of evidence is needed
and should be built into the decision
analytic framework.
Other key considerations noted during
discussion of the evidence review process
included the following:
Transparency. There
were several references to the need for
the ACHDGDNC to make their decision-making
process “transparent”, and
participants asked what is actually meant
by the term transparency in this discussion.
One participant stated that it is the
ability to be so explicit in one’s
methodology that someone could take the
same evidence and draw the same conclusions.
Others suggested that transparency includes
making others aware of where assumptions
have been made in the decision-making
process due to gaps in the knowledge/evidence
base. Being explicit about harms/potential
harms resulting from the screening was
also cited as a transparent approach,
and it was noted that all evidence-based
reviews involve judgment at some point,
so making clear where this judgment occurred
is crucial to transparency.
This clarification of transparency opened
the discussion up to the wider issue of
how the ACHDGDNC makes decisions when
there is no apparent “evidence bar”.
As one person noted, to improve the credibility
of the process, there needs to be a discussion
about what evidence the ACHDGDNC is going
to look at and how to proceed. Much literature
has been generated on how to go about
doing an evidence review, but there is
a large question mark around how to assign
value to the evidence. However, as also
was noted, the ACHDGDNC should not waste
its time trying to rate evidence when
there is general consensus that the evidence
is “good enough”; instead
they should focus on outcomes.
Unpublished/grey data and literature.
The question of whether and how
to use unpublished or grey data and literature
was raised again in the afternoon session.
Dr. Kemper suggested that the circumstances
under which is it possible to bring in
unpublished data for the evidence review
can be either structured or inclusive,
but that any use of grey literature must
be flagged and explained within the context
of the review. Another participant added
that if the ACHDGDNC does choose to accept
grey data and literature, there needs
to be a list of questions that they can
ask about that data to determine whether
it can be used. There was caution, though,
about delving too deep into unpublished
materials: using grey literature and data
is worth little if it becomes cost-prohibitive
to examine closely, and there are still
no criteria for weighing the quality of
evidence in unpublished studies.
Harms vs. benefits.
One of the major areas of discussion was
on the need for the ACHDGDNC to establish
overall goals for harms versus benefits.
No screening test occurs without harm,
and the ACHDGDNC needs to be prepared
to determine what level of harm they are
willing to allow, whether that be an invasive
second diagnostic test on a false positive
patient, delivery of treatment to those
who may get no benefit from it (e.g.,
late onset Pompe patients), and the value
of unnecessary therapies to obtain one
benefit. There was much debate around
the adverse effects of false positives,
and the rate at which these would be acceptable
to a screening panel. Harms also extend
beyond the purview of the public health
arena—for instance, diagnosis can
have psychological and legal ramifications
for the child and family that scientists
are often unaware of. Thus, it is important
for reviewers to consider carefully the
different actions that might occur after
diagnosis and make their ultimate decisions
about outcomes and whether a decision
would cause more harm than benefit.
Decision analytic frameworks allow inclusion
of both harms and benefits, with some
means to assign weights or costs to certain
harms and benefits – thus allowing
consideration of the relative costs and
benefits of different choices in screening.
Conflict of interest.
There always are going to be biases (e.g.
drug companies having a market edge on
approved product) in any nomination of
a condition, therefore the ACHDGDNC needs
to weigh evidence from a variety of sources
that may not include just the literature.
Other kinds of evidence that need to be
weighed are colloquial (testimony of advocates,
parents, experts) and contextual evidence
(things like cost).
Consumer input and support.
There were conflicting ideas on where
consumer input was needed in the review
process. The ACMG only involved parents’
input in the early stages, and several
others noted that it is not necessary
to have consumer input in the evidence
review, as long as there are experts on
board who can ensure that outcomes are
family-oriented. From an evidence-based
standpoint, any purported consumer benefit
should be measurable, though as one participant
reminded the group, it is important to
realize that consumers aren’t only
the families of those infants who screen
positive, but any family with a newborn
undergoing the screening process. Informed
consumers can have active roles in helping
to determine critical branch points in
the screening process and reviewing the
evidence gathered on the outcomes at each
branch. Likewise, it was observed that
parental groups can be incredibly powerful
and help to push down walls where legislative
structures are standing in the way, just
as they can be biased in pushing an agenda
to get a condition added to the screening
panel.
Expertise
needed on the review workgroup
Participants suggested a broad spectrum
of expertise that should be brought in
to assist the ACHDGDNC with the analysis
of evidence. In the Canadian province
of Quebec, for example, their population-based
genetic screening programs make decisions
after bringing in experts such as:
Representatives who can offer expertise
on the content of the disease, laboratory
and clinical aspects and treatment;
Economists;
Public health officials;
Those with methodological expertise;
and
Representatives from organizational
services that don’t follow immediately
under public health but may intersect
with long-term follow-up care.
Quebec also has a mechanism for interacting
with consumers, though they are not included
as part of the formal review process.
As one person noted, it is good to get
a content expert on the ad hoc review
workgroup because that person will already
know the literature and the spectrum of
evidence that exists. This is part of
the procedure of the USPSTF, which also
identifies a specialty representative
to give comments and analyze the recommendations.
The USPSTF process is heavy on the methodological
expertise to analyze the outcomes of studies;
this may be one reason that the timeframe
for full review of a new topic takes between
12 and 18 months for the Task Force.
The value of consumer consultation was
raised again as well, with the idea that
if the evidence review group is going
to assign/place value on the benefits
of a certain treatment, it is important
to understand how families view those
benefits.
Revisiting
the current uniform panel
One issue that was brought up repeatedly
was the feasibility of revisiting the
29 conditions on the current uniform panel.
ACMG has discussed this issue extensively
and there was general consensus among
the group participants that there needs
to be a clear mechanism for follow-up
review of conditions and possible exclusion
of a condition if the evidence base alters,
but that this should not be the first
priority of the Committee. It was noted
that one the of sub-committees is trying
to define what long-term follow-up is,
and it is expected that once this definition
is in place, whatever constitutes long-term
follow-up will be fed back into the screening
and eventual re-evaluation of conditions
on the panel.
Some mechanisms are already in place
to assist with data efforts that could
feed into this. California, for example,
has a data collection feedback loop and
other States have longitudinal outcomes
data for metabolic disorders. NIH is likewise
looking to establish a data set around
all positively screened infants.
Dr. Rinaldo added one final note that
it is important not to lose sight of the
fact that support and consensus on the
initial report and panel overwhelmed whatever
criticism they might have generated.
VII. Moving
Forward
Gerard Vockley, M.D., Ph.D., Chief of Medical
Genetics at the University of Pittsburgh
Medical School and a working group member,
wrapped up the meeting with a final summary
of the major suggestions arising from the
day’s discussion (see box). He noted
that it will be an ongoing struggle for
the ACHDGDNC to come up with a set of criteria
for weighing the evidence that will be pertinent
to every disease nominated. Instead, he
took up the suggestion
recommended
by the group that the ACHDGDNC should
initially focus on better defining
in advance the pieces of the decision
process and identifying criteria
that will be used when considering
the addition of a condition to the
panel. This will ultimately facilitate
actually assessing the evidence
for any one condition.
Dr. Vockley reiterated the emphasis
on transparency, especially the
importance of revisiting the current
uniform panel in a quick and inexpensive
manner to ensure they are still
relevant. Finally, he repeated the
suggestion that the Committee should
be trusted to keep the process broad,
not disease-specific and include
other stakeholders (family/consumer
|
Final
Recommendations
The ACHDGDNC needs to better
define the pieces of the process
and decision nodes that must
be navigated in order to make
the decision from nomination
to addition to the uniform panel;
at each of these points, the
evidence must be applied in
as rigorous a way as possible.
Transparency is critical to
the legitimacy of the process.
How evidence is weighted in
the final decision is as important
as what evidence is used.
Small adjustments need to be
made to the nomination form
related to the inclusion of
pilot studies and quantitative/specificity/sensitivity
data.
It will be important to revisit
the disorders currently on the
uniform panel to periodically
“reaffirm” their
inclusion against the current
evidence.
The ACHDGDNC itself should
be trusted to keep the process
broad, not disease-specific
and involve input from patient
& family groups and potentially,
specialty societies. |
|
groups and potentially specialty societies)
when making the final decisions.
Appendix
A: Agenda
Evidence-based Evaluation and Decision
Process for the
Advisory Committee on Heritable Disorders
and Genetic Diseases in Newborns and Children
(ACHDGDNC)
October 23, 2006
8:30-8:45 |
Introductions and Charge
to Group
Jim
Perrin and Nancy Green
|
8:45-9:00 |
History of Committee and
its Charge
Michele
Puryear
|
9:00- 9:30 |
American College of Medical
Genetics (ACMG) Project and Process
R.
Rodney Howell
|
9:30-9:45 |
Nomination Form and Committee’s
Proposed Decision-making process
Nancy
Green and Piero Rinaldo
|
9:45-10:15 |
Issues in Evidence Review
for Genetics, Pediatrics and Newborn
Screening
Michael
Watson
|
10:15-10:30 |
Break
|
10:30-11:30 |
Evaluation of Pompe Disease
Priya
Kishnani
Alex
Kemper
|
11:30-12:30 |
Lunch
|
12:30-4:30 |
Discussion
Jim
Perrin and G. Vockley
|
4:30-5:00 |
Summary
G.
Vockley
|
5:00 |
Adjourn
|
Appendix
B: Participant List
Work Group Members
Ingeborg Blancquaert,
M.D., Ph.D.
Research Consultant
Agence d'Evaluation des Technologies
et des Modes d'Intervention en Sante
(AETMIS)
2021 Avenue Union, Bureau 1040
Montreal, Quebec, H3A 2S9 CANADA
Phone: (514) 873-7853
Fax: (514) 873-1369
Email: ingeborg.blancquaert@aetmis.gouv.qc.ca
|
Fred Lorey, Ph.D
Research Scientist Manager
Genetic Disease Branch
California Department Health Services
850 Marina Bay Parkway
Richmond, CA 94804
Phone: (510) 412-1490
Fax: (510) 412-1560
Email: florey@dhs.ca.gov |
Linda Bradley, Ph.D.
Centers for Disease Control and Prevention
4770 Buford Highway
Mail Stop K-89
Atlanta, GA 30341
Phone: (770) 488-8399
Fax: (770) 488-8336
Email: lbradley@cdc.gov |
Glenn Palomaki
Associate Director, Division of Medical
Screening
Pathology and Laboratory Medicine
Women and Infants Hospital
2 McConkey Road, Unit 2
Gray, ME 04039
Phone: (207) 657-7888
Fax: (207) 657-7887
Email: gpalomaki@ipmms.org |
Ned Calonge, M.D., M.P.H.
Chief Medical Officer and State Epidemiologist
Colorado Department of Public Health
and
Environment
4300 Cherry Creek South Drive
Denver, CO 80246-1530
Phone: (303) 692-2662
Fax: (303) 691-7702
Email: ned.calonge@state.co.us |
James Perrin, M.D.
Professor of Pediatrics, Harvard Medical
School Director, Division of General
Pediatrics
Director, Center for Child and Adolescent
Health Policy
Harvard Medical School
MassGeneral Hospital for Children
50 Staniford Street, #901
Boston, MA 02114
Phone: (617) 726-8716
Fax: (617) 726-1866
Email: jperrin@partners.org |
Stephen Downs, M.D., M.S.
Associate Professor and Director
Children's Health Services Research
Indiana University School of Medicine
699 West Drive, Room 330
Indianapolis, IN 46202
Phone: (317) 278-0552
Fax: (317) 278-0456
Email: stmdowns@iupui.edu |
Sharon Terry
President and CEO
Genetic Alliance
4301 Connecticut Avenue, Suite 404
Washington, DC 20008
Phone: (202) 966-5557
Fax: (202) 966-8553
Email: sterry@geneticalliance.org |
Alex Kemper, M.D., M.P.H., M.S.
Associate Professor
Pediatrics, General Pedatrics
Duke University
P.O. Box 17969
Durham, NC 27715
Phone: (919) 668-8038
Fax: (919) 668-7058
Email: alex.kemper@duke.edu |
Gerard Vockley, M.D., Ph.D.
Professor of Human Genetics/Pediatrics
Chief, Medical Genetics
University of Pittsburgh Medical School
Children's Hospital of Pittsburgh
3460 Fifth Avenue, Rangos Building,
2nd Floor
Pittsburgh, PA 15213-2583
Phone: (412) 692-7775
Fax: (412) 692-7816
Email: gerard.vockley@chp.edu |
Participants
Coleen Boyle, Ph.D., M.S.
Centers for Disease Control and Prevention
1600 Clifton Road, Mailstop E86
Atlanta, GA 30333
Phone: (404) 498-3907
Fax: (404) 498-3550
Email: cboyle@cdc.gov |
R. Rodney Howell, M.D., Ph.D.
The University of Miami School of
Medicine
Department of Pediatrics (D820)
P.O. Box 016820
Miami, FL 33101
Phone: (305) 243-3993
Fax: (305) 243-3990
Email: rhowell@miami.edu |
Amy Brower, Ph.D.
Medical Informatics and Genetics
Executive Director
Third Wave Molecular Diagnostics
315 South Fork Place
South Sioux City, NE 68776
Phone: (608) 358-1574
Email: abrower@twt.com |
Michele Lloyd-Puryear, M.D., Ph.D.
Health Resources and Services Administration
Parklawn Building, 18A19
5600 Fishers Lane
Rockville, MD 20857
Phone: (301) 443-1080
Fax: (301) 443-8604
Email: mpuryear@hrsa.gov |
Denise Dougherty, Ph.D.
Agency for Healthcare Research and
Quality
540 Gaither Road
Rockville, MD 20850
Phone: (301) 427-1868
Fax: (301) 427-1561
Email: denise.dougherty@ahrq.hhs.gov |
Marie Mann, M.D., M.P.H.
Health Resources and Services Administration
Parklawn Building, 18A19
5600 Fishers Lane
Rockville, MD 20857
Phone: (301) 443-1080
Fax: (301) 443-8604
Email: mmann@hrsa.gov |
Nancy Green, M.D.
Medical Director
March of Dimes Birth Defects Foundation
1275 Mamaroneck Avenue
White Plains, NY 10605
Phone: (914) 997-4649
Fax: (914) 428-7849
Email: ngreen@marchofdimes.com |
Piero Rinaldo, M.D., Ph.D.
Professor of Laboratory Medicine
Chair, Division of Laboratory Genetics
Mayo Clinic College of Medicine
200 First Street, S.W.
Rochester, MN 55905
Phone: (507) 284-5859
Fax: (507) 266-2888
Email: rinaldo@mayo.edu |
Speakers
Priya Kishnani, M.D.
Associate Professor in Pediatrics
Interim Division Chief, Medical Genetics
Duke University Medical Center
237 Bell Building
Durham, NC 27710
Phone: (919) 684-2036
Fax: (919) 694-8944
Email: kishn001@mc.duke.edu
Michael Watson, Ph.D.
American College of Medical Genetics
9650 Rockville Pike
Rockville, MD 20814
Phone: (301) 634-7127
Fax: (301) 634-7275
Email: mwatson@acmg.net |