Public Meeting - Advancing Regulatory Science for Highly Multiplexed Microbiology/MCM Devices, October 13, 2011#

These diagnostic devices present several advantages, such as identifying potential disease etiology in situations where many different pathogens share a common clinical manifestation and the simultaneous detection of co -infections. However, establishing and validating the performance of these devices to make informed clinical and public health decisions pose significant scientific challenges.

The purpose of the public meeting is to discuss performance evaluation of highly multiplexed microbiology/medical countermeasure (MCM) devices, their clinical application and public health/clinical needs, and quality criteria for establishing the accuracy of reference databases. These considerations are essential to establish the safety and effectiveness of highly multiplexed devices when used for the clinical diagnosis of infectious diseases from a human specimen.

FDA is holding this public meeting to obtain input from academia, government, industry, clinical laboratories, and other stakeholders on the performance evaluation approach to be proposed by FDA, which includes validation methods, reference panels, and bioinformatic concepts needed to address the clinical and analytical performance requirements for highly multiplexed microbiology/MCM devices. The ultimate goal is to advance regulatory science for highly multiplexed devices used in pathogen detection in order to ensure their safety and effectiveness and thereby provide potential clinical and public health benefits.

• Date, Time, and Location
• Federal Register Notices
  • Initial Federal Register Notice
  • Notice of Comment Period Reopening
    (Comments should be received to the docket by December 21, 2011)
• Agenda
• Topics for Discussion
• Questions for Afternoon Discussion Session
• Archived Webcast
• Meeting Transcript
• Contact Us

Date, Time and Location

This meeting was held October 13, 2011, from 8:00 a.m. to 6 p.m. at the following location:

FDA White Oak Campus
10903 New Hampshire Ave
The Great Room (Room 1503), White Oak Conference Center, Bldg 31
Silver Spring, MD, 20903

The meeting was webcasted.

Agenda   *Denotes Link to Presentation

Topics for Discussion

• Concept Paper: Advancing Regulatory Science for Highly Multiplexed Microbiology/Medical Countermeasure Devices (PDF)

FDA sought input on the following topics:

1. Clinical Application of Highly Multiplexed Microbiology Devices: Their clinical application and public health/clinical needs; inclusion of MCM -related pathogens that are expected to be rarely present in the tested specimens; the composition of clinically relevant panels of pathogens; the interpretation of the test results taking into consideration the possible detection of microorganisms that are not clinically relevant, and what is known and unknown about co -infections.
2. Device Evaluation: How to evaluate the analytical and clinical performance of highly multiplexed microbiology devices; approaches to device validation when positive specimens are not easily available, which is the case for many MCM pathogens; is the proposed FDA evaluation approach sufficient to establish device performance and is it feasible and practical.
3. Reference Databases: Quality criteria for establishing the accuracy of reference databases; methods for curating, maintaining, and updating these databases; what is the current practice for creating and maintaining reference databases.

Questions for Afternoon Discussion Session

Clinical Need and Other Considerations

1. Is the use of syndromic panels the appropriate pathway forward for multiplexed diagnostics? What is the role of large all-inclusive multiplexes?
2. What is the risk of a false positive diagnostic result for very low prevalence analytes (including MCM targets), in the context of their inclusion in a highly multiplexed device? For example, would the use of diagnostic algorithms reduce the false positive risk?
3. Does a multiplexed testing approach benefit the clinical diagnostic evaluation of a potential pathogen(s) relevant to a specific specimen type (for example, CSF, blood culture, respiratory), or does it introduce factors that complicate the clinical interpretation of the test results? Would the use of diagnostic algorithms or panels aid in reducing complexity?
4. In the context of colonization vs. infection, would quantitative or semiquantitative multiplexed devices improve diagnostic outcomes? How could this approach be implemented without having international standards (e.g., WHO) to compare results obtained by different platforms? In the context of qualitative assays, could different assay cut-offs be used to evaluate colonization vs. infection?
5. Are there benefits to report test results as “For Information Only”; for example, in the context of infection control for hospital surveillance (e.g., for resistance markers or for detection of newly emerging pathogens without well established medical significance)? What are the potential risks?

Validation

1. Are there risks of using banked positive specimens for the evaluation of clinical sensitivity? Consider a scenario where the positive banked specimens were selected and banked based on a sensitive method; then, prior to inclusion in the study, the positivity of these specimens was confirmed (selected again) using a less sensitive comparator method. In this case the candidate device is likely to have an "inflated" sensitivity (PPA). How can this potential bias be minimized?
2. What are the criteria (e.g., technology dependant, physio-chemical properties) to select organisms to evaluate various analytical device parameters instead of including all pathogens detectable by the multiplexed device in conventional analytical testing? How many strains/isolates should be included for each organism based on the selection criteria?
3. For the evaluation of LoD using multiplexed NAAT assays with nucleic acid purification, is it necessary to evaluate all sample matrices (e.g., NP swabs, BAL, Blood, etc.). Can specimen types from the same anatomic “compartment” be considered equivalent and be pooled during the device analytical evaluation? When is this not a viable approach? What are the risks of applying the same specimen “pooled” approach to the clinical evaluation of the device?
4. The proposed validation approach uses a combination of clinical sample type and pathogens appropriate to that sample type to provide some inherent ceiling to the size of the assay multiplexing involved. How does the panel envision the validation approach scaling for technologies that are inherently metagenomic at much larger scales (e.g., pan-microbial microarrays or deep genomic sequencing)?
5. When is an in-silico approach to the evaluation of multiplexed devices justified over conventional analytical bench testing and when should it be used to augment conventional analytical bench-testing? Should this approach be used only when the organisms being detected are not readily available?
6. Culture-based approaches for quantification of infectious organisms have traditionally shown large variability in performance between operators and between testing sites. Is nucleic acid quantification (G.E., copies, etc) a plausible replacement for culture-based quantification for NAAT-based multiplexed microbiology device validation? What are the risks of using this as the sole approach?

Reference Databases

1. Could in silico predictions for new signatures or potential cross-reactivity replace or augment lab testing? When an in silico approach is necessary, what are the criteria that should be used to validate the in silico data?
2. Given the potential differences in device databases, what are some of the fundamental principles, regardless of sequence, mass, etc., that can be applied to the validation of these databases to ensure reliable and accurate test results?
3. For many targets new mutations will arise and therefore there will be a constant need to update any device database. What are the challenges associated with the validation of these changes and what is the most appropriate way to accomplish this? How would the end-users be notified of updated information?
4. Some devices query public databases to retrieve sequence results. Given some of the inherent problems with this approach due to the inconsistency of the information deposited in these databases, is there a viable alternative to ensure that the information used to affect a decision is complete and valid, without becoming overly burdensome to developers?

Webcast

• Webcast for October 13

Contact Us

For information regarding the program, contact:

Raquel Peat
Center for Devices and Radiological Health
Food and Drug Administration
10903 New Hampshire Avenue, Bldg 66
Silver Spring, MD 20993
Phone: 301-796-6218
Email: Raquel.Peat@fda.hhs.gov