Evidence-based Evaluation and Decision Process for the Advisory Committee on Heritable Disorders and Genetic Diseases in Newborn and Children: A Workgroup Meeting Summary

October 23, 2006

On this page:

I. Introduction
II. Background on the ACHDGDNC
III. Overview of the Nomination Form
IV. Issues in Evidence Review for Genetics, Pediatric and Newborn Screening
V. Case Study: Evaluation of Pompe Disease
VI. Discussion
The nomination form
Evidence evaluation process
Expertise needed on the review workgroup
Revisiting the current uniform panel
VII. Moving Forward
Appendix A: Agenda
Appendix B: Participant List

I. Introduction

The Children’s Health Act of 2000 was passed by Congress as an amendment to the Public Health Service Act to revise, extend and establish programs with respect to children’s health research, health promotion and disease prevention activities conducted through Federal public health agencies. Initiatives mentioned in the Act included efforts related to autism, asthma, childhood obesity and, in Title XXVI, improvements in the ability of States to provide newborn and childhood screening for heritable disorders. This provision also stated explicitly the need to create an Advisory Committee “to provide advice and recommendations to the Secretary for the development of grant administration policies and priorities, and to enhance the ability of the Secretary to reduce mortality or morbidity from heritable disorders.” Subsequently, the Advisory Committee on Heritable Disorders and Genetic Diseases in Newborns and Children (ACHDGDNC) began convening in 2004 to advise and guide the Secretary on the most appropriate application of universal newborn screening tests, technologies, policies, guidelines and programs for reducing negative outcomes of heritable disorders.

In 2006, the ACHDGDNC made a formal recommendation to the Secretary of Health and Human Services outlining an initial uniform screening panel of 29 conditions suggested for inclusion in State newborn screening programs. The selection of these conditions was based on a report authored by the American College of Medical Genetics (ACMG) and commissioned by the Health Resources and Services Administration (HRSA). Concurrently, the ACHDGDNC has been attempting to establish criteria for assessing the inclusion of other diseases, based on available evidence.

On October 23, 2006, the ACHDGDNC’s decision-making criteria workgroup convened a group of experts to discuss how to establish a rigorous evaluation and recommendation process to determine the suitability of screening for certain rare inherited disorders. The experts—who came from a range of backgrounds in pediatrics, genetics, and public health—were requested to:

Examine a draft nomination form and comment on the criteria that an evidence-based workgroup would need for evaluating conditions for screening newborns and children for heritable disorders;

Develop options for the ACHDGDNC on the structure of the evidence-based workgroup; and

Determine whether an iterative process should be tied to the decision-making process of the ACHDGDNC, and if so, how to tie the evaluation and recommendation process to an established infrastructure accompanied by this iterative process.

Nancy Green, M.D., Medical Director of the March of Dimes Birth Defects Foundation and one of the ACHDGDNC members, who helped to develop the sample nomination form, noted that the day’s discussion would feed into the next ACHDGDNC meeting in mid-December. Michele Lloyd-Puryear, M.D., Ph.D. of HRSA stressed that the recommendations arising out of this meeting were not being directed to any Federal agency, rather, they were feeding into the criteria working group, which in turn would present recommendations to the ACHDGDNC about this process.

II. Background on the ACHDGDNC

R. Rodney Howell, M.D., Ph.D. of the University of Miami School of Medicine and Chair of the ACHDGDNC, provided further background on the ACHDGDNC and the issues it faces. The ACHDGDNC is comprised of: several ex officio voting members appointed by the Secretary for Health and Human Services from HRSA, CDC, NIH, and AHRQ; medical, technical, public health and scientific professionals with expertise in heritable disorders or newborn screening; non-voting representatives from other medical and public health organizations; and non-voting liaisons from the Advisory Committees on Infant Mortality and Genetics, Health and Society.

The preliminary tasks/issues the ACHDGDNC has been tasked with addressing have included the following:

Assessing capacity needs of States/newborn screening programs including workforce needs, States’ ability to conduct technology analysis and translation into practice, public education, and secondary screening.

Parental education/notification/informed decision-making.

Recommendations for a uniform panel of conditions (which, as noted earlier, has been submitted to the Secretary).

The process by which new tests/technology/conditions will be considered.

Dr. Howell said that perhaps the most complex of these tasks is determining what evidence is used to decide what will go on the uniform screening panel. He noted that some issues are currently being examined by sub-committees, such as laboratory quality assurance issues and how to handle long-term follow-up. The ACHDGDNC also will establish workgroups with responsibility for interagency coordination (particularly the relationship with the HRSA regional collaborative program) and to work with NIH and others to set up research agendas related to newborn screening.

While there has been an enormous amount of publicity related to newborn screening, Dr. Howell observed, most reviews have found few detriments to doing such screening. He argued that the day’s discussion would touch upon some of the most important issues surrounding newborn screening, including the evidence related to diagnosis, treatment and long-term outcomes of potentially screenable disorders. Assessment of evidence for screening newborns for rare heritable disorders is unique in many regards, as such an assessment must deal with extremely rare conditions, the initial diagnosis and immediate follow-up occurs within the public health sector, and once a treatment is deemed life-enhancing or –saving, controlled studies most likely will not be undertaken.

III. Overview of the Nomination Form

Dr. Green and Piero Rinaldo, M.D., Ph.D., Mayo Clinic College of Medicine, proceeded to brief the group on the development of the nomination form and process for adding conditions to the uniform panel. The nomination form includes space for information on the condition, the screening test and the treatment, and was largely based around the ACMG categories for assessing evidence. However, Dr. Rinaldo noted that there were some diversions from the ACMG model, as this nomination form added mention of risks (and benefits) associated with screening and treatment. In addition, the ACHDGDNC was careful not to include on the form any request for cost data, or any subjective measures.

The ACHDGDNC-approved draft process for review also was presented for discussion. Once a nomination form is completed, it is sent to HRSA, who conducts an administrative review. HRSA can request more information, decline the form as incomplete, or send it on to be reviewed by the ACHDGDNC. The ACHDGDNC then examines the form and determines whether to decline it, or to form an ad hoc evidence workgroup to conduct further research and analysis on the condition. The expertise and the members of this workgroup are to be determined. After its analysis, the workgroup presents the evidence for screening newborns and the quality of that evidence to the ACHDGDNC. The ACHDGDNC determines whether to decline the addition of the condition or to recommend adding it to the uniform panel for screening of heritable disorders. Because there will be no scores applied to the criteria the challenge to ACHDGDNC will be to determine where to set the bar for inclusion. This process for nomination and review will have a mechanism for constant re-evaluation. Dr. Green added that addition of conditions to the uniform panel is a constantly evolving process, and that as new treatments emerge, conditions that may have been rejected for past inclusion can be reviewed again in the future.

No process, however, is currently in place for reviewing the current 29 conditions that make up the uniform panel. This topic was mentioned frequently throughout the day and is discussed in further detail in section VI.

Participants raised the question of what data might lead the ACHDGDNC to reject a nominated condition. Some of the reasons include: issues related to the analytical performance of screening (e.g., testing leading to an excessive amount of false positives) and a need for better understanding of the natural history of the disease. Dr. Green said that the ACHDGDNC has never arrived at a particular cut-off point for metrics of specificity and sensitivity, nor has it begun the address the difficulty of screening tests that move beyond the dried blood spot model. In simplest terms, the ACHDGDNC looks at a nomination form and tries to assure that what they’re looking at is indeed a disease, that it can be identified reasonably well in screening, and that there are desirable actions that can be done after the screening that lead to positive outcomes.

IV. Issues in Evidence Review for Genetics, Pediatric and Newborn Screening

Michael Watson, Ph.D. talked about the origins of the ACMG report that was published in 2006 and some of the thinking that led up to the report’s recommendations, including its primary recommendation of the 29 conditions for the uniform screening panel.

ACMG was asked by HRSA to outline a process of standardization of outcomes and guidelines for State newborn screening programs and to define responsibilities for collecting and evaluating outcome data, including a recommended uniform panel of conditions to include in these programs. Newborn screening often has been characterized as arising from uneven policies that have led to perceptions of inequities in selection of diseases for screening. The ACMG expert panels recognized at the start that the evidence base for most conditions under scrutiny was largely made up of observational studies and expert opinion. They designed a system to collect expert evidence from providers and families and have researchers and other experts in the conditions review the assembled evidence.

Once all the evidence had been collected and reviewed, the group eliminated any conditions that did not meet the minimum of having a screening test available, having a well-understood natural history, and having an available treatment that significantly alters outcomes. Following the evidence review, the ACMG concluded with two other observations/recommendations:

Tandem mass spectrometry should be used for newborn screening on the basis of MCAD, VLCAD and LCHAD, and is a screening process that allows for improved identification of other disorders, including PKU, IVA and MSUD.

29 conditions were found to be appropriate for newborn screening.

Per the overriding principles set by the ACMG expert group, the newborn screening program should report any other results of clinical significance. This led to delineation between clinically significant primary targets of screening and other conditions that may be included in the differential diagnosis of a primary target analyte.

Some of the confounding factors that the ACMG group had to consider when doing their evidence review included: avoiding unwarranted assumptions of the type inherent to the health technology assessment reports being aware of the self-evident evidence paradox (i.e., if an intervention is truly effective, no one will study it); how to incorporate the views of patients and families with clinical and scientific evidence; and how to state that referral/consultation with experts may be necessary for PCPs.

Dr. Watson also detailed the issues faced in assessing the evidence, such as:

Natural history: needs to start from at-risk newborns, needs as many patients as possible to further minimize bias, confounded by continuous availability of new treatments.

Incidence: general population data are best, mostly rare diseases with wide range of estimates, multiple genetic etiologies for phenotypes, splitting conditions screened by same analytes into multiple groups.

Onset: poor capture of penetrant cases, newborn screening can detect both early and late onset forms.

Burden: there is a “bias” toward severe conditions.

Screening test: there is variability among States and a slow evolution of the gold standard. Iterative changes in screening algorithms may detect clinically significant secondary conditions.

Diagnosis: the process needs to recognize increasing complexity with permutations of genes involved.

Treatment: studies of treatments often with limited involvement of pediatric patients. Dilemmas such as how to integrate treatment when the phenotype is common to a number of rare conditions and many different conditions may require similar treatment and have comparable outcomes.

However, one of the primary issues is the quality of evidence itself. Natural history studies are increasingly difficult as a result of ongoing intervention. Moreover, there is limited data collection on the national level despite obvious need and recommendations to do so, and some data that are collected are proprietary. Treatment data are increasingly subject to reduced pre-market studies with conditional approval subject to more post-market surveillance.

Other evidence challenges for newborn screening and pediatrics include the difficulty in running randomized clinical trails (RCTs), given the extreme rarity of the conditions in question, the rapid growth in interventions and technologies used to screen and treat them, and the problem enlisting infants and children in such trials. The health of infants also is laden with emotion and any missed case resulting in the death of a child can be compelling. Likewise, there are questions related to what constitutes a “benefit” in early detection and whether the evidence is good enough—what is the likelihood of getting better evidence in the near future?

There are numerous evidence assessment systems for public health that, while valuable for making recommendations for broad populations, have not been used for newborn screening and genetics due to concerns about the evidence base for newborn screening and rare genetic disorders. Dr. Watson briefly highlighted a number of these—including the US Preventive Services Task Force (USPSTF), Evidence-based Practice Centers (EPCs), and Centers for Education and Research on Therapeutics (CERTs), Evaluation of Genomic Applications in Practice and Prevention (EGAPP), to name a few—and discussed the varying reasons why they have failed for these disorders, including problems related to expense, inflexibility, lack of applicability to rare diseases, or more treatment- rather than screening-aimed foci.

The state of the genetic disease testing evidence base largely is in disarray not because of lack of diagnostic confirmation or treatment data, but because of paucity of incidence/prevalence data and the difficulty in comparing screening test evidence. After touching briefly upon developing the evidence base for common versus rare conditions, Dr. Watson stressed the need for creation of registries to house data on newborn screening patients and genetics, which would in turn contribute to prospective and retrospective studies of these conditions.

This point was taken up by the participants in their discussion, who noted that the practice model for metabolic (and genetic) disease is not well organized or established. Great value could arise from developing consistent standards of care and from gathering consistent data across sites (much as has happened with pediatric oncology work).Many conditions may have good small sample testing of screening, but lack any larger and more population-based testing. There was much discussion of the potential for State pilots and the value of moving to population testing in one or more States prior to more general use of a screening test (detailed further in section VI).

V. Case Study: Evaluation of Pompe Disease

To give the group an idea of how the nomination form would work in practice, Priya Kishnani, M.D., Associate Professor in Pediatrics at Duke University, reviewed the submission she created for Pompe disease. A lysosomal enzymatic deficiency, Pompe disease appears both in infantile and late onset form. For the former, babies typically present with severe hypotonia and hypertrophic cardiomyopathy, resulting in death within the first year of life. These infants fail to meet most major milestones for motor skills, especially anything other than rolling over. The late onset version of Pompe disease may occur in childhood or adulthood and is associated with progressive muscle weakness and respiratory failure that also eventually leads to death.

An article detailing a comprehensive study into the natural history of Pompe disease in 160 patients worldwide was published in Pediatrics in 2003. This article showed that for infantile Pompe, the median age that symptoms appeared was at two months, and that traditional diagnosis (i.e., not using newborn dried blood spot screening) generally was made four months after those symptoms appeared, thus leaving a short time period for therapeutic intervention that could save the life of the child. Indeed, traditional treatment of Pompe was limited to supportive care. In 2006, however, the Federal Drug Administration licensed the first effective treatment for Pompe disease, the drug Myozyme. In clinical trials, Myozyme was shown to have significantly altered the natural course of Pompe disease: all infants in the first phase of clinical studies lived past the critical age of one year. Subsequent studies proved that infants given Myozyme made swift gains in motor skills, such as standing, walking and running; had improved muscle tone and cardiac function; and the vast majority (14/18 patients in one trial) were free from invasive ventilators.

Integrating Pompe disease onto the newborn screening uniform panel would allow for more timely diagnoses and significantly improve outcomes for those children with infantile Pompe. The challenge with newborn screening for this condition is that it cannot distinguish infantile versus late onset variations of the disease and the screening test has not been validated within a broad population.

Alex Kemper, M.D., M.P.H., M.S. then reviewed some of the methodological challenges facing those evaluating the evidence for newborn screening of rare conditions.

The criteria used in the initial selection of the 29 conditions on the uniform panel were based on the following:

Clinical characteristics: such as incidence, presentation during the newborn period and burden of disease.

Screening tests: the availability of sensitivity and specificity tests, expense, and the ability to integrate into existing newborn screening programs.

Diagnosis and follow-up: availability and cost of treatment, its efficacy and prevention of mortality, etc.

Dr. Kemper reported that some individuals expressed concerns about the methodology of the ACMG report during the public comment period; some of these concerns have since been published. Examples include:

There was an inappropriate use of opinion and survey methods.

The review of the literature lacked an analytic framework or strategy, or clear inclusion/exclusion criteria.

There was a limited scope, and issues such as false positives, secondary targets and ethics were not addressed in the report.

Dr. Kemper then presented his thoughts on approaches to weighing evidence for important policy decisions. First, there are two strategies for doing a review of the evidence. The traditional strategy relies on narratives, usually written by experts, which can themselves be biased. The second strategy of a systematic review includes methods and criteria for identification of studies and data, criteria for inclusion/exclusion of studies and data, and criteria for judging study quality. Systematic reviews may combine qualitative and quantitative data, and may form the basis of cost-effectiveness studies. The challenges to systematic reviews relate to synthesizing the evidence, especially when the evidence is sparse because of a rare condition or new or understudied test or treatment. Studies may use different populations, study designs and measurements. In addition, studies of rare conditions and studies focused on children are difficult to come by. Systematic reviews and users of such reviews face the challenging question of what is enough evidence for drawing a conclusion. Dr. Kemper added that not making a decision is a decision in and of itself.

Other organizations have structured evidence reviews in a variety of manners. The Cochrane Collaboration—which produces a database of systematic reviews—focuses on RCTs and provides a plain language summary as well as a detailed scientific summary to accompany its recommendations. The USPSTF, which makes recommendations about preventive services in primary care, utilizes EPCs to conduct systematic reviews, and outside experts provide peer review. The Task Force members then review the evidence and issue recommendations on a scale of A, B, C, D or I, ranging from strongly recommended practices to recommending against an intervention or concluding that there is insufficient evidence to make a recommendation.

In a paper submitted to the group, Dr. Kemper suggested that a similar multi-tiered voting scheme could be used by the ACHDGDNC. For example, the different tiers of such a scheme might include: universal screening recommended, targeted screening recommended, pilot study of screening recommended, pivotal studies required, no general recommendations or recommendations against screening. This voting scheme would allow the ACHDGDNC to guide the process of acquiring some critical piece of information—for example, a population-based pilot study. The pilot efforts in MA and CA of MS/MS could be examples of this.

Using Pompe disease as an example, Dr. Kemper led the group through his vision of a process by which to weigh the evidence on Pompe and consider it for inclusion in the panel. Underlining this decision-making process was what he termed “important decision #0” or that the success of a review will be determined if the results are replicable. The following rounded out the steps of the Pompe evidence review:

Important decision #1: Focus on the infantile form of the disease, and determine answers to key questions regarding its natural history, diagnosis, treatment and the experience of screening programs.

Important decision #2: Cast a broad net for data that looks at published studies of all types, government databases, unpublished data and anything except animal data.

Important decision #3: Do not use standard study quality assessment instruments. Quality is still important, though the instruments developed do not translate well into smaller studies.

Regarding this last decision point, Dr. Kemper suggested looking at the Canadian Centre for Health Evidence for guidelines for quality and assessment. The Centre offers a number of key questions for primary and secondary guidance to determine whether study results are valid, as well as to assess results and determine whether they can be applied to patient care.

Dr. Kemper concluded by listing some of the general challenges facing an evidence review group for rare and heritable disorders. Chief among these was deciding whether or not to use unpublished data—a topic which raised some debate later on in the day and is discussed on page 11 of this report. Other issues included how to adjust for standards of quality given that RCTs are unlikely to occur with these rare disorders, expectations for how the data should be summarized, the relationship of the methodologist to the content expert, and how the final reports should be structured and disseminated.

VI. Discussion

The remainder of the meeting provided an opportunity for participants to discuss some of the issues raised by the presentations and the sample case of Pompe disease, and to make recommendations to the working group that would in turn advise the ACHDGDNC on the uniform panel and its decision-making processes. While tasked with commenting specifically on the three discussion points outlined on page 1 of this report, the discussion also raised other relevant points that must be taken into consideration by the ACHDGDNC, as described below.

The nomination form

There was general consensus that the nomination form as currently drafted can provide the basic information about whether there is evidence regarding available diagnosis and treatment. An example was offered regarding the EGAPP project, and how it handled the nomination process. An EGAPP representative stated that the program has a similar process, receiving a short summary about the disorder, test and clinical scenario. Those nominations that are given top priority are then asked to give a longer summary based on criteria set forth by EGAPP; this serves as the jumping off point for the key questions and analytic framework established for the evidence review.

What were observed as lacking from the ACHDGDNC form, however, were quantitation of specificity and sensitivity, prevalence, and the cost implications of the four possible outcomes to screening (true positives, true negatives, false positives and false negatives). There was general consensus to add a request for quantitation of specificity and sensitivity to the nomination form, however, there was some debate within the group that it is unlikely that the ACHDGDNC (or an evidence support group) will be able to agree on absolute figures for adequate sensitivity and specificity (or other metrics). Presumably, the ACHDGDNC could agree on minimum floors below which a test is not satisfactory. A decision analytic framework, outlined in the next section on evidence evaluation process, can help to put the information into a set of questions and decision points for the ACHDGDNC that allows better weighing of alternatives.

It also was recognized that there will be nominated conditions that might have a good screening test but insufficient evidence to demonstrate clinical validity in a larger population base. Therefore another key suggested improvement to the form was adding a pilot study that could prove that one could successfully screen for the disease with the technology applicable to population screening, i.e., proof of a study offering clinical validity. There are several ways to do controlled pilot trials. For example, a disorder can be tested and treated in one State using a different State as a control. The ACHDGDNC might also recommend that pilot studies be conducted to close evidence gaps: if there are no pilot studies for tests or treatment, the ACHDGDNC could recommend a population-based study to obtain such information in lieu of conducting an evidence review (and making a recommendation to add to the core panel or not).   

Finally, as assistance to those who might submit nomination forms, one person recommended that the ACHDGDNC provide an example of a condition on the uniform panel that successfully underwent the nomination process, so that they might see how the structure and supporting data should be presented. MCAD was suggested to be the sample case.

Evidence evaluation process

The group went on to discuss the steps that need to be in place for the evidence review to occur. First and foremost, the ACHDGDNC needs to establish an overall goal or principle for how the evidence-based working group will operate, such as, “The overall benefits of screening must outweigh the potential harms for the individual, the family or society.” A decision-analytic framework was suggested for the decision-making process. Decision trees represent the most common analytic framework, and should be used to frame the evidence review process. It was observed that a decision-analytic framework makes explicit the assumptions that go into the process. The group suggested that a prototype decision tree could be developed to guide nominators regarding the types of data most needed in an evidence review.

The evidence-based working group then needs to define the overarching principles for an acceptable evidence base for the condition in question, and review those recommendations with the ACHDGDNC. For example, for many types of disorders, the group might not require evidence in the form of RCTs or double blind studies; instead, it must realize that likely levels of evidence for clinical efficacy may come in the form of case studies and expert opinions. Criteria for quality of case studies and expert opinions must be set. Sensitivity analysis was also recommended as a part of all reviews; such analyses may define what level of evidence is needed and should be built into the decision analytic framework.

Other key considerations noted during discussion of the evidence review process included the following:

Transparency. There were several references to the need for the ACHDGDNC to make their decision-making process “transparent”, and participants asked what is actually meant by the term transparency in this discussion. One participant stated that it is the ability to be so explicit in one’s methodology that someone could take the same evidence and draw the same conclusions. Others suggested that transparency includes making others aware of where assumptions have been made in the decision-making process due to gaps in the knowledge/evidence base. Being explicit about harms/potential harms resulting from the screening was also cited as a transparent approach, and it was noted that all evidence-based reviews involve judgment at some point, so making clear where this judgment occurred is crucial to transparency.

This clarification of transparency opened the discussion up to the wider issue of how the ACHDGDNC makes decisions when there is no apparent “evidence bar”. As one person noted, to improve the credibility of the process, there needs to be a discussion about what evidence the ACHDGDNC is going to look at and how to proceed. Much literature has been generated on how to go about doing an evidence review, but there is a large question mark around how to assign value to the evidence. However, as also was noted, the ACHDGDNC should not waste its time trying to rate evidence when there is general consensus that the evidence is “good enough”; instead they should focus on outcomes.

Unpublished/grey data and literature. The question of whether and how to use unpublished or grey data and literature was raised again in the afternoon session. Dr. Kemper suggested that the circumstances under which is it possible to bring in unpublished data for the evidence review can be either structured or inclusive, but that any use of grey literature must be flagged and explained within the context of the review. Another participant added that if the ACHDGDNC does choose to accept grey data and literature, there needs to be a list of questions that they can ask about that data to determine whether it can be used. There was caution, though, about delving too deep into unpublished materials: using grey literature and data is worth little if it becomes cost-prohibitive to examine closely, and there are still no criteria for weighing the quality of evidence in unpublished studies.

Harms vs. benefits. One of the major areas of discussion was on the need for the ACHDGDNC to establish overall goals for harms versus benefits. No screening test occurs without harm, and the ACHDGDNC needs to be prepared to determine what level of harm they are willing to allow, whether that be an invasive second diagnostic test on a false positive patient, delivery of treatment to those who may get no benefit from it (e.g., late onset Pompe patients), and the value of unnecessary therapies to obtain one benefit. There was much debate around the adverse effects of false positives, and the rate at which these would be acceptable to a screening panel. Harms also extend beyond the purview of the public health arena—for instance, diagnosis can have psychological and legal ramifications for the child and family that scientists are often unaware of. Thus, it is important for reviewers to consider carefully the different actions that might occur after diagnosis and make their ultimate decisions about outcomes and whether a decision would cause more harm than benefit.

Decision analytic frameworks allow inclusion of both harms and benefits, with some means to assign weights or costs to certain harms and benefits – thus allowing consideration of the relative costs and benefits of different choices in screening.

Conflict of interest. There always are going to be biases (e.g. drug companies having a market edge on approved product) in any nomination of a condition, therefore the ACHDGDNC needs to weigh evidence from a variety of sources that may not include just the literature. Other kinds of evidence that need to be weighed are colloquial (testimony of advocates, parents, experts) and contextual evidence (things like cost).

Consumer input and support. There were conflicting ideas on where consumer input was needed in the review process. The ACMG only involved parents’ input in the early stages, and several others noted that it is not necessary to have consumer input in the evidence review, as long as there are experts on board who can ensure that outcomes are family-oriented. From an evidence-based standpoint, any purported consumer benefit should be measurable, though as one participant reminded the group, it is important to realize that consumers aren’t only the families of those infants who screen positive, but any family with a newborn undergoing the screening process. Informed consumers can have active roles in helping to determine critical branch points in the screening process and reviewing the evidence gathered on the outcomes at each branch. Likewise, it was observed that parental groups can be incredibly powerful and help to push down walls where legislative structures are standing in the way, just as they can be biased in pushing an agenda to get a condition added to the screening panel.

Expertise needed on the review workgroup

Participants suggested a broad spectrum of expertise that should be brought in to assist the ACHDGDNC with the analysis of evidence. In the Canadian province of Quebec, for example, their population-based genetic screening programs make decisions after bringing in experts such as:

Representatives who can offer expertise on the content of the disease, laboratory and clinical aspects and treatment;

Economists;

Public health officials;

Those with methodological expertise; and

Representatives from organizational services that don’t follow immediately under public health but may intersect with long-term follow-up care.

Quebec also has a mechanism for interacting with consumers, though they are not included as part of the formal review process.

As one person noted, it is good to get a content expert on the ad hoc review workgroup because that person will already know the literature and the spectrum of evidence that exists. This is part of the procedure of the USPSTF, which also identifies a specialty representative to give comments and analyze the recommendations. The USPSTF process is heavy on the methodological expertise to analyze the outcomes of studies; this may be one reason that the timeframe for full review of a new topic takes between 12 and 18 months for the Task Force.

The value of consumer consultation was raised again as well, with the idea that if the evidence review group is going to assign/place value on the benefits of a certain treatment, it is important to understand how families view those benefits.

Revisiting the current uniform panel

One issue that was brought up repeatedly was the feasibility of revisiting the 29 conditions on the current uniform panel. ACMG has discussed this issue extensively and there was general consensus among the group participants that there needs to be a clear mechanism for follow-up review of conditions and possible exclusion of a condition if the evidence base alters, but that this should not be the first priority of the Committee. It was noted that one the of sub-committees is trying to define what long-term follow-up is, and it is expected that once this definition is in place, whatever constitutes long-term follow-up will be fed back into the screening and eventual re-evaluation of conditions on the panel.

Some mechanisms are already in place to assist with data efforts that could feed into this. California, for example, has a data collection feedback loop and other States have longitudinal outcomes data for metabolic disorders. NIH is likewise looking to establish a data set around all positively screened infants.

Dr. Rinaldo added one final note that it is important not to lose sight of the fact that support and consensus on the initial report and panel overwhelmed whatever criticism they might have generated.

VII. Moving Forward

Appendix A: Agenda

Evidence-based Evaluation and Decision Process for the
Advisory Committee on Heritable Disorders and Genetic Diseases in Newborns and Children (ACHDGDNC)
October 23, 2006

Appendix B: Participant List

Work Group Members

Participants

Speakers

Priya Kishnani, M.D.
Associate Professor in Pediatrics
Interim Division Chief, Medical Genetics
Duke University Medical Center
237 Bell Building
Durham, NC 27710
Phone: (919) 684-2036
Fax: (919) 694-8944
Email: kishn001@mc.duke.edu

Michael Watson, Ph.D.
American College of Medical Genetics
9650 Rockville Pike
Rockville, MD 20814
Phone: (301) 634-7127
Fax: (301) 634-7275
Email: mwatson@acmg.net