FDA Guidance on Clinical Trial Data Monitoring Committees (DMCs)
FOOD AND DRUG ADMINISTRATION
CENTER FOR BIOLOGICS EVALUATION AND RESEARCH
OPEN PUBLIC MEETING FDA GUIDANCE ON
CLINICAL TRIAL DATA MONITORING COMMITTEES (DMCs)
TUESDAY, NOVEMBER 27, 2001
Hyatt Regency Bethesda
Rockville, Maryland
Contents
Welcome
Opening Remarks
History and Background of DMCs
Which trials need DMCs?
Panel Discussion:
Rick Ferris, M.D., NIH, NEH
William G. Henderson, Ph.D., Dept. VA
LeRoy B. Walters, Ph.D., Georgetown
Janet T. Wittes, Ph.D., Statistics Collaborative, Inc.
Open Public Discussion
Establishment of DMCs and Operational Issues
Independence of DMCs
Panel Discussion:
Norman C. Fost, M.D., MPH, U. Wisconsin
Lawrence M. Friedman, M.D., NIH. NHLBI
Ira Shoulson, M.D., U. Rochester
Steven M. Snapinn, Ph.D., Merck Research Lab
Open Public Discussion
DMCs and Other Regulatory Requirements
Panel Discussion:
Robert M. Califf, M.D., Duke Univ.
David M. DeMets, Ph.D., U. Wisconsin
Robert J. Levine, Ph.D., Yale Univ.
David C. Stump, M.D., Human Genome Sciences
Open Public Discussion
Closing Remarks
Proceedings
Welcome
DR. LEPAY: Good morning. On behalf of FDA I'd like to welcome you to today's workshop on data monitoring committees. The purpose of this workshop is to introduce FDA's new guidance for clinical trial sponsors on the establishment and operation of clinical trial data monitoring committees.
We planned this workshop several months ago with the expectation certainly that this guidance document would be out with ample time for individuals to review it in advance of the workshop. We may not have had quite as much time for this review process as we would have hoped but we are very pleased to at least see that the document is available and is, in fact, available for general circulation today outside.
I want to start by just mentioning, of course, that this guidance document has been a while in planning, in preparation and in clearance. We've certainly been talking about it at FDA for well over a year now and it is a very integral part of our move certainly to look at subject safety, subject protection in real-time and as part of our overall unit of overseeing clinical trials respective to FDA's regulatory responsibilities.
The draft guidance came out just about a week ago, announced in the Federal Register on the 20th of November, and for those who otherwise need to access it by means other than the formal copies that have been distributed at the outside of this conference room, it is available on various of FDA's websites, either through the CBER website, www.fda.gov/cber/guidelines/clindatmon.htm. Or if you can't remember that, simply go to FDA's general website, www.fda.gov, to the clinical trials section and you'll see this in the What's New? and in the New Guidances Section.
We're currently in the beginning of a 90-day comment period, which began at the time of publication of this guidance in the Federal Register. The comment period will be open until the 19th of February 2002. Comments can and should be submitted to a docket which has been established for this purpose. The identification of this docket is listed here, 01D-0489. In fact, we can accept comments either in writing directly to the Dockets Management Branch at the address shown here, and this is also provided in the Federal Register announcement, or more simply as electronic comments again off of the FDA website at a specific link to our Dockets Management Section. Again you'll need to reference the docket number.
We think this meeting is a very important step in providing us with input on this guidance document. As we've remarked many times over the past several years, public comment is integral to the process of FDA rulemaking and development of guidances. Certainly what we're going to be talking about today in the presentations that you will hear reflect FDA's current thinking in the area of data monitoring committees but clearly that thinking is very much an interactive process that depends on the contributions of everyone here in the audience, as well as those at your respective companies or institutions who we strongly encourage to read and provide comments to us.
So with that, I'm going to open the meeting.
Oh, let me also remind everyone here that the proceedings of this meeting are being audio-recorded. The transcripts of this meeting will be made available, as well as transcripts will be filed to the docket, so comments made here will, in fact, be captured and will be part of our consideration as we review the guidance document and move forward toward its finalization.
And with that, I would then like to introduce our opening speaker and I have the very great pleasure of presenting Dr. Greg Koski, who's head of the Office for Human Research Protection in the Department of Health and Human Services. Greg has certainly been a tremendous moving force in the area of human subject protection since he came on board just a little over a year ago and has been an extremely important and successful colleague with FDA in moving forward initiatives pertaining to human subject protection and the oversight of clinical trials.
So with that, I'll ask Greg to open the meeting with a few introductory remarks.
Opening Remarks
DR. KOSKI: Thank you very much, David, for the kind words. It's really a pleasure to be here. It's nice to see so many people out there, as well. You know, we've been accused in government of holding public meetings in order to get more people to come to Washington in order to support the economy. I hope that some of you have come from farther than Bethesda or downtown, but it's great to see all of you here. I think it reflects the very high level of interest in this very important topic as it pertains not only to the oversight of research, protecting the validity and the objectivity of the research, but also protection of human subjects.
I'm sure that all of you recognize that over the last 30 years or so the FDA and the former Office for Protection from Research Risks have shared responsibility for protection of human subjects in research. Since the Office for Human Research Protections was created a little over a year ago, not only have we continued that tradition of collaboration but indeed have worked very, very hard to strengthen it as we go forward and I think that David has been absolutely critical to the success of that effort.
I think all of you are aware that the system for protection of human subjects in research is undergoing some remodeling currently. Over these last 30 years we've really had two schemes under which we have operated, that which applied primarily to federally supported and conducted research, a system that really focussed primarily on an assurance process before research was to be initiated, whereas we had a system that FDA was primarily responsible for that dealt largely with corporate sponsored, privately sponsored research that focussed far less on an up-front assurance process but instead focussed very significantly on audits of investigators and IRBs and sponsors in order to ensure the process.
And while both of these approaches, they have good reasons for their existence, have had both strengths and weaknesses, when the Office of the Inspector General and the General Accounting Office looked at our processes they both concluded that although each of these emphasized particular areas, there was a gap and that gap that they identified as a weakness in the overall process was in that area that I describe as what happens after the IRB says okay. In other words, it's when we're actually conducting the research activities.
Clearly we do have processes for reporting adverse events, for interacting with investigators and subjects. We have seen data and safety monitoring boards utilized effectively over the years. But as we've gone forward we've begun to realize that indeed there are opportunities to utilize the stronger aspects of each of these systems in a more effective way and this effort by FDA, in conjunction with the rest of the colleagues here in the Department of Health and Human Services, to provide guidance on data monitoring committees I think is a very, very important step toward achieving a greater level of uniformity and to provide a component of the system that can work across the entire domain, which, of course, is something that we're very anxious to achieve.
So this document that has just been published a week ago with some relief, I believe, to everyone, it reflects the enormous effort and thinking that has gone into this by the folks at FDA, with input from many others, toward defining these committees, how they should be constituted, how they might be positioned, how they can interact with the IRBs and with investigators and sponsors as they carry out their important activities.
And in bringing this document forward I think it's quite clear that FDA is emphasizing the fact that this is not a fait accompli. This is a piece of work that they have put out there in order to stimulate discussion, to get your input, and today I think they're very, very serious in asking you to interact with them, with the panels. I think it's very interesting and also rewarding, I find, satisfying that if you look at the agenda for today's meeting, if you look at the participants in the panels, as well as here in the audience, you can see that there is a coming together of the minds of these two systems in important ways so that what we hope will emerge from this again will be a set of guidance that will strengthen the process for everyone.
There's an awful lot to talk about here today. Again we encourage you to really jump in, get involved in the discussions so that the final product is one that will serve everyone's interest.
With that, David, I wish you the very best of luck, and Susan, in your meeting today. I encourage you to take it seriously and get down to business. Thank you very much.
DR. LEPAY: Very good. With that, we'll begin with the discussion of our guidance document. Our first presentation this morning will be by Susan Ellenberg, who chaired the working group involved with the drafting of this guidance document. Susan will outline the history and background of data monitoring committees. With that, I will turn this over to Susan and with luck, hopefully she can get us started on track here.
History and Background of DMCs
DR. ELLENBERG: I'm very glad to see all of you here today. I notice there's still a few empty seats, mostly toward the front. So people who are coming in in the back, don't be shy; just wander up and you'll find a seat.
Let's start with a definition of a data monitoring committee. This is the definition exactly as it appears in our document. It may not be everybody's favorite definition but I think it's serviceable. A data monitoring committee is a group of individuals with pertinent expertise that reviews on a regular basis accumulating data from an on-going clinical trial. The data monitoring committee advises the sponsor regarding the continuing safety of current participants and those yet to be recruited, as well as the continuing validity and scientific merit of the trial.
So this is the kind of committee that we're going to be talking about today. Many of you have seen this slide. I just would like to clarify on the terminology. We are talking about data monitoring committees but these committees have gone by a lot of other kinds of names, so you can pick as many as you like from column A and put it together with something from column B and something from column C and I don't know whether all the permutations and combinations have been used but many of them have been. In particular, the other phrase that's used frequently is data safety monitoring board. As far as I've been able to ascertain, all of these things mean approximately the same thing and are consistent with the definition.
We are using the phrase data monitoring committees because that is the terminology that was selected by the International Conference on Harmonization, who, as I'll talk about in a minute, is a collaboration of industry and regulatory scientists in the United States, Europe and Japan who are putting together guidance documents on regulated clinical trials and other aspects of regulated research and have used this phrase, so we're being consistent with that.
In the document we mention some other oversight groups because it's important to recognize that the data monitoring committee, while there may be some overlap of oversight, is a separate group from any of these others. Many trials have a steering committee. This is an internal group to the trial. This is the trial leadership who designs the trial, monitors the conduct of the trial, will prepare the final presentation. That is an internal group where a data monitoring committee is an external group.
Institutional review boards, sometimes called institutional ethics committees, are charged with evaluating the acceptability and appropriateness of a trial in a specific clinical setting. While they have some oversight responsibility as the trial progresses, it's not at the level of detail and looking at specific data that the data monitoring committee has. So again there is a difference. These are not the same groups.
Another kind of oversight committee that would be internal to a trial would be an end point assessment or an end point adjudication committee. This is a committee often of trial participants who would review data on the reported primary outcomes to ensure consistency with the protocol specified criteria--for example, to look at reports of an acute myocardial infarction and make sure that all the data were there to meet the protocol criteria.
There are often in trials also site monitoring groups. The responsibility of these groups is to basically do an overall quality control. They may go out to the sites, look at the data, make sure that what's in the record is consistent with what's on the form. Again that's another type of oversight but it's different from the kind of monitoring that we're talking about here that a data monitoring committee would do.
When did data monitoring committees start? This is one story that I've heard other people may have other stories, but in a clinical trial that the NIH sponsored back in the 1960s called the University Group Diabetes Project several investigational anti-diabetic agents were compared to placebo and this, you have to remember, was sort of the very beginning of clinical trials. Randomized clinical trials were brand new in the 1960s. There were no oversight groups. There was a group of investigators who were mounting this trial and I notice that increased cardiovascular mortality was emerging early for one of the agents, not what was expected in this trial. These agents were hoped to improve mortality. There was no established statistical monitoring plan. This was well before the era of statistically based sequential designs and the investigators and sponsors were wringing their hands, not really sure what to do about this, but their gut feeling was let's get some outside experts who are not invested in the trial in the way we are to have a fresh look, to help us really make the best decision we possibly can, based on the data.
So it was this sense of needing some objective kind of look that may have led to a recognition that it would be generally good to have some kind of external advice on this sort of thing.
In 1967 a report was issued to what was then the National Heart Institute, now NHLBI, regarding the conduct of clinical trials. This report is widely referred to as the Greenberg Report because the committee that put it together was chaired by Dr. Bernard Greenberg, who was chair of the Department of Biostatistics at the University of North Carolina. This covered the range of good clinical trials practices for that time and it included a recommendation that a formal committee be established to review the accumulating data on safety, efficacy and trial conduct.
I don't think the phrase data safety monitoring board or data monitoring committee was used in this report. It was published after a number of years ultimately in Controlled Clinical Trials in 1988 so if you're interested in the report, you can find it there.
I'm not going to say too much about history. Data monitoring committees have been components of federally funded trials for a very long time, particularly the NIH and the VA, but there are probably other agencies, as well. Department of Defense and CDC have done clinical trials probably that have used data monitoring committees. They've been used primarily in multi-centered trials with mortality end points or end points of major morbidity, things that will have a permanent impact on people's fundamental health.
And the reason that these committees have been felt to be needed for these kinds of trials is because in these trials efficacy and safety end points essentially overlap. If you have a mortality end point and you expect to see deaths in the course of the study, if you have a safety problem with your drug where there's excess mortality, you can't really see that by looking at individual cases. You need to look overall at the number of deaths being observed. So it's an efficiency end point but it's also a safety end point and somebody needs to be looking as the trial progresses to see if there's any kind of difference emerging.
Because of the importance of these end points, there's a real ethical imperative to monitor. If the trial is part-way through and it's very clearly established that more lives are being preserved on one arm than the other, it would be important not to continue to enter patients on that trial. And as was noted in the UGDP example, there is a need, because the stakes are so high, a need to insert some objectivity into the interim assessments, to try and make sure that the decisions that are made are based on the data and not on possible extraneous influences from which few of us are free.
Now in industry data monitoring committees were not used so frequently in industry trials prior to the 1990s. For some trials they were used, particularly trials with mortality end points, primarily but not entirely in the cardiovascular area. But recently there's been a lot more use of data monitoring committees in industry trials for some of these reasons. Industry is sponsoring more trials with mortality end points or other major end points. Again we're still in an early phase of evolution of clinical trials methodology. There's been a heightened awareness of the value of independent monitoring in some of these circumstances, I think, and there's also, I think, increased government-industry collaboration that has introduced industry to some of the data monitoring approaches that have long been used in trials that are sponsored by government agencies.
Now data monitoring committees are almost entirely absent in FDA regulations. There's only one type of trial that actually requires a data monitoring committee and those are trials in which informed consent is waived. And some of you will remember that a regulation was issued in 1996 dealing with emergency research in which informed consent was simply not feasible, and I have the CFR reference up there. Why would it not be feasible? If a patient is unconscious or otherwise unable to provide consent and no proxy can be available within the time frame in which treatment would be required to be started.
So this was a regulation aimed specifically at being able to do research in this kind of circumstance but the circumstances were very limited. There was great concern at FDA and outside the FDA about allowing a trial to proceed without informed consent. It had to be a life-threatening situation. The trial could not be feasible without the waiver. There had to be a strong scientific basis established for the investigational treatment.
And because we were not having such a fundamental protection as informed consent, additional protections were required in such trials, such as prior community consultation, public notification, and the establishment of an independent data monitoring committee. So this is the only place where data monitoring committees had been required.
Data monitoring committees have been mentioned in several FDA guidance documents, mostly those developed through the International Conference on Harmonization, including the E3 document, Structure and Content of Clinical Study Reports, E6, the Good Clinical Practice document, and E9, Statistical Principles for Clinical Trials.
E3, this is sort of an after-the-fact document. It tells you how to report once you've completed the trial and it says well, if you had a data monitoring committee you've got to tell us about it. Who was on it? How did it operate? What statistical monitoring plan was used? How did you make sure that people who were supposed to be blinded stay blinded? You need to describe the interim analysis and you need to provide all the minutes of the meetings and the interim data reports. So that's in one of the guidance documents.
E6, the Good Clinical Practice document, has a section that mentions the independent data monitoring committee, basically provides a sort of definition and specifies that it should have written operating procedures and maintain written records. So it's not a whole lot of detail.
A little more detail in the E9 document, Statistical Principles for Clinical Trials. Again it notes what a data monitoring committee does. It evaluates interim data and makes recommendations to the sponsor--that it should have written operating procedures and maintain meeting records. This is the first document where the notion of confidentiality of interim data is mentioned and the protection of the trial integrity, that an independent data monitoring committee will help with those. It notes that it is separate from an IRB or an IEC, not the same thing, that its composition is multidisciplinary, and it notes that if there are sponsor representatives participating in the data monitoring activities, then those roles must be clearly defined and it must be clearly understood how interim results within a sponsoring organization would be controlled.
So today data monitoring committees are increasingly used. NIH and the various NIH institutes have established policies requiring data monitoring committees for many extramural and intramural trials and you can find those guidelines on the NIH websites.
Data monitoring committees have become a standard in industry trials with major end points, for the most part, and they've been suggested even for some early phase trials when you have a novel high-risk treatment and we're going to be discussing some of those possibilities.
There are a variety of models for data monitoring committee operation. People who have been doing this for a long time--I've talked to a lot of people and different people do it different ways and most people think that their way is right, so I would not say that there is an absolute consensus on what the optimal approach is and there may be multiple approaches that could be acceptable in any given circumstance.
In 1998 the Office of the Inspector General of HHS issued a report on institutional review boards and while the focus was on IRBs, there were two recommendations that dealt specifically with data monitoring committees.
The first recommendation was that data monitoring committees be required for trials under NIH and FDA purview that meet specified conditions, didn't say what those conditions would be but said that NIH and FDA would need to define those conditions and would need to specify requirements for data monitoring committee composition.
Well, this document is, in a sense, a response to this, although the word "required" doesn't really fit with a guidance document but we have tried to respond to this recommendation.
The second recommendation was that data monitoring committees should have primary responsibility for reviewing and evaluating adverse experiences occurring in the trial and that data monitoring committee assessments, along with summary data, could be shared with IRBs. We've certainly had a lot of discussion about this. We're not entirely sure that the data monitoring committee is the best place for primary responsibility for review of individual adverse events, although they certainly do have a role overall in considering adverse events in a trial and I think we'll have some discussion of that.
The development of this guidance was a joint effort of three FDA centers plus the Office of the Commissioner. Center for Biologics, Center for Drugs, Center for Devices and Radiological Health all were involved in the development of this document, as well as the Office of Good Clinical Practice, the new Office of Good Clinical Practice headed by Dr. Lepay.
We did get interim comments, very helpful interim comments from our colleagues at NIH on this document. We also solicited some interim comments from two FDA advisors that were considered in putting together what is our final draft.
And you've seen this slide. This is the title of the guidance document.
Just a couple of introductory comments to the document before I turn this over to Dr. Campbell. The document frequently refers to the sponsor and there could be a question as to who is the sponsor, who acts as the sponsor. Generally at FDA we regard the sponsor as the group, the organization that holds the IND but we acknowledge in the opening of the document that sometimes sponsors delegate authority for decision-making to some entity. It could be a steering committee, could be a contract research organization or even a principal investigator. And when you read the sponsor does this or the sponsor may do this in the document, you should also read the group, the entity to whom the sponsor may have delegated such decision-making authority. It seemed awkward to continue to write "or the steering committee" or whatever throughout the document. So that should be understood. The sponsor may be a company or may be a government agency.
We discuss briefly the issue of government and industry sponsors. We believe the issues discussed in this guidance document are relevant to all trials, whatever the sector of the sponsor, so we don't distinguish between government and industry sponsors but we do recognize that there are differences in type and extent of conflict of interest that exist for government and industry sponsors and those may have implications for the types of data monitoring committee approaches that are established.
Now the intent of this guidance document is to describe generally acceptable models for data monitoring committee establishment and operation, to discuss possible advantages and disadvantages of different approaches, and very importantly, to increase awareness of the potential concerns that can arise in trials when comparative data are subject to interim monitoring and we've had some experience with this, which we'll be discussing today. I know that some of these issues I had not been aware of before coming to FDA so I think it is important to consider these.
We also address the relationship of data monitoring committees to the regulatory requirements for monitoring and reporting, to understand who maintains who responsibility.
What it's not intended to be is prescriptive. It's not intended to lay out the exact single model of data monitoring committees that everything should adhere to. We are really trying to raise issues and help those who are sponsoring clinical trials to understand what some of the issues are so that we can develop optimal strategies.
That's it. Thank you for your attention.
DR. LEPAY: Thank you, Susan. I think that was a very good introduction to our guidance document today, to some of the history on data monitoring committees.
We've organized the program today in three sections, as you'll see, with ample opportunity for both open discussion as well as panel discussion with each of these sections.
The first section covers the chapters 1 through 3 of the guidance document and with that, I will turn over to Greg Campbell for our second presentation. Greg is the director of the Division of Biostatistics in the Center for Devices and Radiological Health and he will be talking about certainly one of the most important topics addressed within this guidance document, some of the thinking behind which trials need data monitoring committees.
Which Trials Need Data Monitoring Committees?
DR. CAMPBELL: Thank you, David. Well, I get the pleasure of trying to explain when one should consider using a data monitoring committee and when not.
The first question and the important one, I suppose, is are data monitoring committees always needed or always advised? And the answer quite simply is no, that there are lots of situations where it's less than clear that a data monitoring committee would be helpful. Although it's not advised in every trial, there are advantages, there are situations where a data monitoring committee might prove valuable.
So Susan Ellenberg in her opening remarks mentioned that there is a situation where a data monitoring committee is required and it's in the case where one is dealing with some emergency therapy and there is waived informed consent. An example of this would be the automatic external defibrillators that you see now in airports and sometimes on airplanes. Those external defibrillators were tested in a clinical trial with a data monitoring committee. What one needs there is to act very quickly. There's no possibility of informed consent except as a community, and that's an example where the DMC is required.
What is clear and what is in the regulations is that all clinical trials do require safety monitoring but this doesn't necessarily mean that every trial needs a formal committee that's external to the trial organizers and to the investigators. One could, for example, in nonconfirmatory studies imagine an independent safety monitor who would essentially in real time evaluate the safety considerations of each and every patient in the study.
So what I'd like to do now is present an outline of the other times when one should consider a data monitoring committee and there are essentially three main bullets here. The first is risk to trial participants and this is the first and foremost situation that one wants to consider for data monitoring committees. The important thing is to be able to protect the subjects by insulating the decisions about continuing or curtailing the trial from those that may have a financial interest or even a scientific interest in the trial's success.
More generally, the overall welfare of patients with the disease and others in future clinical trials is also a consideration for the data monitoring committee. The implication here is that if one had a failed clinical trial, that might stymie the development of an entirely new technology completely.
There are pragmatic issues having to do with the practicality of the data monitoring committee and its review and I'll go into each of these in great detail.
The third point is the assurance of scientific validity. There's a major advantage for data monitoring committees in terms of safeguarding the scientific validity of the trial and so without that independence, there may be a perception that the trial was not conducted in a scientifically valid manner.
So let's turn attention to the first of these three points, the first and foremost, that of protecting trial participants from risk.
A first and major factor to consider here is what is the end point, primary or secondary? Is it, in fact, mortality or major morbidity? If the answer to that question is yes, then a data monitoring committee should be considered very seriously.
And there are lots of examples where this could arise. For example, in a randomized clinical trial for a cancer chemo prevention strategy, one would consider strongly a data monitoring committee. In cardiovascular device randomized clinical trials one of the major end points is called MACE. It's the major adverse cardiac events and that's, of course, either mortality or MI or future reoperation. Those are major mortality/morbidity end points and a data monitoring committee should be in effect there.
One could also imagine a randomized clinical trial for a new retroviral therapy for HIV and as a fourth example, a randomized clinical trial for a new regimen for adjuvant treatment of colon cancer.
So here are four examples where the primary end point is mortality or severe morbidity, major morbidity in a randomized clinical trial and a data monitoring committee is clearly indicated.
A second point is to answer the question would a favorable or unfavorable result early in the trial suggest termination? So this is an ethical question. If you're a manufacturer of some medical product and your product performs in an extremely optimal fashion, you and your investigators may be no longer having equipoise. You may want to stop that trial right away, rather than expose subjects in the control arm to the inferior therapy.
And that goes actually in the other direction, as well. If it turns out that the new product, be it a device or a pharmaceutical drug or biologics, if there is some disadvantage in the trial that shows up early, for the safety of future patients in that trial you would want to discontinue enrollment for ethical reasons.
A third question to ask in this section about risk to trial participants is is the new treatment so novel that there is very little prior information on its clinical safety? For example, one might have a new molecular entity for which there is not any information in the confirmatory setting about its safety, for example. Then a data monitoring committee should be strongly considered.
Another example would be a medical device, a novel technology for which its operation is poorly understood. It's not clear to everyone exactly how the device might appear to be delivering benefit. In those situations a data monitoring committee should be considered seriously.
And a fourth question here is is there a particular safety concern? Has some safety concern already shown up perhaps in phase II trials that might cause one to look carefully in the confirmatory study? For example, perhaps there's a hint that there might be a liver toxicity problem. In those cases it would be well advised to have a data monitoring committee to follow up.
The fifth point is the fragility of the population that's being studied. If, for example, one is looking at a trial that involves children, then data monitoring committees should be something that one considers. For example, in vaccines one might have a childhood vaccine trial. In those cases why would you worry about in particular a data monitoring committee? Well, one point has to do with informed consent. In situations where the population is fragile, the issue about informed consent would be of concern and it's something that data monitoring committees can help to safeguard.
The second point, the elderly, there are certainly lots of studies where the therapies involved are for the elderly population, who may not be well equipped to make decisions.
A third fragile population are patients in very ill health; for example, patients with HIV entered into a randomized clinical trial. In those cases a data monitoring committee is indicated. In a study for congestive heart failure where you're talking about people with severe disease, NYHA class three or four, again data monitoring committees would be a very good idea.
Are there adverse events that are expected or likely? These are sometimes difficult to protect. It may be difficult to anticipate in advance what's expected and what's unexpected but a data monitoring committee can help safeguard these, as well as unanticipated or unexpected events that might occur.
And the last point in this section on risk to trial participants, are the participants at an elevated risk of mortality, major morbidity or toxicity? For example, in a confirmatory phase III drug trial, there might be the potential for severe liver toxicity. In those cases one might strongly consider a data monitoring committee.
If one were looking at an earlier phase trial having to do with dose finding in the case of a drug, one might consider a data monitoring committee there, as well, particularly if liver toxicity is something of worry.
Okay, so that's the first point. Let me go on now to the practicality of the clinical trials and data monitoring committees. The first point here has to do with the time lag. It could be that if a data monitoring committee is set up that the trial is so swift in its enrollment, so swift in the follow-up with the patients that the monitoring committee doesn't have anything to do; the study's over before the monitoring committee could even meet. In those cases it's not clear that a monitoring committee adds any value at all.
Now what one might want to do in cases where it's possible to enroll very fast is to stage the enrollment so that that does not necessarily happen, to allow the monitoring committee to be able to look at what's happening over the course of the trial.
There are examples where the enrolment is very fast but the follow-up on the individuals is not. For example, in a vaccine trial, people can be vaccinated very quickly but the follow-up may take years before the evaluation of whether that vaccine is effective or not and safe can be done. In those cases one should consider a data monitoring committee not because you're going to stop future patients from enrolling in the trial but if you, for example, stop early that vaccine trial, you may be able to switch people over from the control arm to the vaccine arm. You may be able to allow the product into the public arena much more quickly. So this is an example where even though you can enroll people right away, there are still advantages to a data monitoring committee in terms of early stopping.
Is the trial large? If the trial tends to be large, then that's certainly a suggestion that a monitoring committee might be used. And certainly the tradition of clinical trials, if you go back in terms of the history of DMCs, the NIH trials tended to be quite large; the trials for the Department of Veterans Affairs tend to be large, as well.
If one has small trials it's not so clear. One could imagine that you're doing a relatively moderately sized trial but the implications in terms of the population that would be affected by the therapy could be quite large, in which case you might want to consider a monitoring committee nonetheless.
If the trial multi-centered? Is it a multi-centered randomized clinical trial? If the trial were only to involve a single institution it may be that the IRB could serve many of the roles that a data monitoring committee would ordinarily do. But most of the confirmatory trials that are submitted to the FDA are multi-center ones, so the conduct of these kinds of trials is much more complex and in those cases a data monitoring committee can be quite helpful.
Another point here has to do with globalization and the fact that there are now multinational clinical trials and this is so because not only is there the ICH effort for pharmaceutical products and biological products but there's also for medical devices a global harmonization effort, as well. If one has a multinational trial that's multi-centered, there are additional issues for monitoring committees that may have different implications for the different regulatory bodies that might be affected.
So, for example, if some of the centers are in the United States and it's being used as a confirmatory trial for the U.S. FDA, there may be some issues about whether the data shows safety and efficacy or safety and effectiveness for the U.S. part of the study.
Is the trial conducted over a long period of time? As we know, over a long period of time the practice of medicine can change; new therapies can be introduced. A DMC can provide some element of insurance for long trials because, as I'll talk about in a little while, there are changes that DMCs can easily effect that are much harder to manage if one would not have the data monitoring committee.
More points on the practicality of the trial. Could the enrollment of investigators or subjects be a problem? In some trials enrollment may not occur as one might plan. In those cases it may be possible that the data monitoring committee, in conjunction with the steering committee, may be able to make some suggestions of how to improve enrollment. There may be some inclusion/exclusion criteria that need to be contemplated for a change. And changes, I'll talk about later.
The whole issue about equipoise in terms of the ethical nature of the trial may be a problem for some of the investigators. Investigators may drop out as a source of new subjects not because necessarily anything from the trial has been released, because presumably the trial might be masked or blinded, but things may have changed over time and they may no longer feel comfortable as individuals in terms of equipoise.
If the trial is not blinded, if it's not a masked trial, and this happens sometimes in medical devices, then equipoise can be, in fact, more of a problem because different investigators may have some impressions that they've built up over the conduct of the trial.
Can the sponsor afford to have a data monitoring committee or could they afford not to? Data monitoring committees are somewhat expensive. There's an issue about who pays. In the case of industry-sponsored trials it's usually the companies.
And the last point, and this really goes to the question of do we need data monitoring committees for every trial that comes to the FDA; if that were the case, we'd run out very quickly of well qualified individuals to serve on these monitoring committees. There simply aren't enough. Although there are lots of experts in this room, there are many, many more trials than there are experts.
More, of course, can be trained and there are issues about how to effectively do that but there are not enough, I suspect, experts for all the scientifically important questions that come up.
Okay, the third major point has to do with the assurance of scientific validity. A first question to ask is is it important that the perception of independence of the sponsor from the trial be preserved?
Now this afternoon Dr. Jay Siegel will talk in greater detail about the whole issue about independence and data monitoring committees but at least for now the whole issue about scientific preservation of validity can be helped to be ensured by employing a body that is independent of the sponsor and independent of the company, that doesn't have some vested financial and/or scientific interest in the trial. And this has advantages, of course, in terms of ethical behavior, as well, and the perception of ethical behavior.
Would the scientific validity of the trial be questioned without a data monitoring committee? And that's related to the point that I just made; namely, that if there were financial ties by the people who served on the data monitoring committee, that could create difficulties.
A third question to ask in terms of the assurance of scientific validity is is the interim analysis contemplated with the probability of stopping early for success or failure? As an example, there was a medical device that came on the scene in the 1980s called ECMO, which stands for extracorporal membrane oxygenation, and this is a treatment for newborns, neonates, who are in some respiratory distress and if those trials were conducted now it would be very clear that one would want to have a data monitoring committee not only for the ethical nature of it but also to preserve the scientific validity.
What tended to happen was there were a number of trials that were done. There were different ways of randomizing babies to the two arms. One was the ECMO arm; one was the standard of care arm. And interim analysis played a key role in deciding when to stop those trials.
Another example when one would want to stop early and preserve the scientific validity has to do with an indication of a mortality advantage. So, for example, if the new product has some survival advantage, one would want to stop early but still be able to preserve the scientific validity. A data monitoring committee enables you to be able to have your cake and eat it, too.
And the last point on this slide has to do with the statistical analysis. In stopping early, in particular, there are lots of statistical issues that come up having to do with bias and without a data monitoring committee it's much more difficult to consider how to handle those.
In addition, in medical devices in particular, there are situations that sometimes come up where a company comes in early for what was a fixed size trial and the suspicious person might ask well, why did they come in early? Were they continually monitoring the trial, even though that wasn't part of the plan? Those create nontrivial statistical implications in terms of trying to figure out how valid scientifically are the results.
The fifth point in terms of assurance of scientific validity is that during the trial is it possible that another study might be released that could compromise the trial? There may be well known other studies that are going on at the time that the trial is being conducted that may have implications in terms of the control arm or in terms of the treatment arm in the current trial and the release of information on these other trials could have grave implications in terms of the conduct of the trial and a data monitoring committee can help buffer that and provide, in the case of independent data monitoring committees, provide decisions of what to do in those cases.
There's an example of a device, for example, that's used now in stenting that has recently been approved by the FDA which allows for distal protection or embolic protection and the approval of this device has probably had implications in terms of other devices that are currently in clinical trials.
And the last point here is modifications to the trial. It's possible during the trial that different kinds of things could happen. A clinical trial, after all, is not a fixed quantity. It's almost like a living thing. It evolves; it changes; it can change. One of the obvious ways in which a clinical trial might need to be modified has to do with the sample size. When the sample size is calculated, different things are assumed about the rate in the control arm, the rate in the treatment arm. Those assumptions may or may not be valid and it may turn out that the trial is underpowered and the sample size needs to be adjusted. A data monitoring committee, although it's not easy, can grapple with this. If it's left only to a sponsor it creates difficulties. There are questions about the scientific validity in those cases.
A similar discussion can be made for changes to the primary end point. This has to be done with great, great care and I should hasten to add that when these sorts of changes to the protocol are made, it is extremely important that the FDA be informed about those changes and different products have different schedules that require the notification thereof.
It could be that the inclusion/exclusion criteria might be changed during the trial. There might be issues that the monitoring committee sees during the course of the trial that are red flags. It could be that there are some enrollment difficulties and without a data monitoring committee it might be extremely difficult for a sponsor to be able to make the case about changing the end point or changing the inclusion/exclusion criteria on the fly.
It could be possible, in fact, that a trial design could be modified. For example, dropping an arm in a three-arm trial might be something that could be considered by a monitoring committee. In the case of medical devices it's not unheard of that during the course of the trial the device needs to be modified because of some problem that might have arisen and how do you do that? Without a data monitoring committee it's much more difficult.
So in conclusion, what I guess I would say is that for significant risk products, be they pharmaceutical drugs, biologics or medical devices, it's extremely important that companies and their sponsors come to the FDA and talk with the respective center, either the Center for Drugs, the Center for Biologics, or the Center for Devices and Radiological Health, at the planning stage. So if you have an IND or in the case of a medical device it's called an IDE, an investigational device exemption, come early, come even at the pre-IDE stage or the pre-IND stage and have a conversation about data monitoring committees and get the best advice that you can.
The ultimate decision about whether to employ a data monitoring committee or not is a complex one and the unique aspects of the particular medical product and where it fits in the plan study need to be taken into account in the determination of this very complicated issue about when do you need a DMC and when you don't. Thank you very much.
DR. LEPAY: Greg, thank you very much.
With that, we're going to take our first break of the morning and resume at 10:30 with our first panel discussion. Thank you.
[Recess.]
DR. LEPAY: Again can I have everyone's attention so that we can resume with the panel? Very good.
I'd like to introduce our distinguished panel this morning, the first of our three panels today. Starting on my left first is Edward Connor, senior vice president for clinical development at MedImmune, Incorporated. Dr. Rick Ferris, director of the Division of Epidemiology and Clinical Research at the National Eye Institute at NIH. William Henderson, director of the Hines Cooperative Studies Program Coordinating Center at the Hines VA Hospital, Department of Veterans Affairs. LeRoy Walters, senior research scholar at the Kennedy Institute of Ethics, Georgetown University. And Janet Wittes, president of Statistics Collaborative, Incorporated.
Again, as I said, a major focal point of this particular meeting is to get discussion, public discussion, as well as panel discussion. We're going to first then move into our panel and what I'd like to do is I'd like to invite each of our panelists to perhaps provide some of their own perspective, some of their own experiences in a few minutes. Then from there we can move more broadly into comments across the panel.
With that, I think we'll just go in the order I had mentioned here, starting with Dr. Connor.
DR. CONNOR: Thank you. I'd just like to make a couple of brief comments by way of background and experience. I guess I've been involved with various aspects of DSMBs or DMBs for the last 15 years or so through a variety of experiences, the first of which involved as a committee chair and protocol chair for some of the AIDS clinical trials group studies that were conducted over the past decade or so; as a committee chair involved in a portfolio of studies that interacted regularly with NIH's DSMB.
And as a protocol chair for 076, which was a trial of perinatal transmission using AZT, as a protocol chair involved in the conduct of that trial and ultimately with the DSMB as a decision-maker, having been on the receiving end of the DSMB's decision to stop that trial early because of efficacy, first-hand was able to demonstrate the actual immediate impact of having such committees involved in certainly high-profile and important clinical trials. In those instances the rapid decision of efficacy in the studies allowed immediate implementation actually of that prophylactic regimen and had substantial public health benefit that was able to be facilitated through the intimate involvement with the DSMB.
For the last eight years or so I've been involved in the sponsor side as a clinical development person at MedImmune and in that capacity have obviously been involved in several instances of the development of large phase III clinical trials and have been involved in implementing and managing DSMB activities related to those trials.
So I think in general, the document that has been produced as guidance has really done a very good job at being able to capture the issues related to the implementation of DSMBs within clinical studies and by and large represents the paradigm by which decision-making is arrived at regarding how those agencies are actually involved in clinical development.
I think some of the issues that we'll ultimately be discussing have to do with the resource of folks who are expert in those areas and how that resource can be efficiently used to optimize involvement in the major trials and also in some of the issues related to how you take the trials that don't necessarily fit into the clearly needing SMC or DMB or clearly not needing a DMB and make decisions around those issues. So that's all. Thank you.
DR. LEPAY: Dr. Ferris?
DR. FERRIS: In 1973 I had the privilege of my first data monitoring committee chaired by Jerry Cornfield and in the succeeding years I've been on a number and as time has gone on I'm more and more convinced of the value of these from a number of perspectives. Most importantly, rarely--never are we dealing with a perfect experiment and rarely do you find that everyone looks at the accumulating data and comes to the same decision.
I think one of the most important reasons for having the data monitoring committees, as was discussed earlier today, is these are living things and it takes a group of people to develop a consensus. The FDA often has panels to review data because these aren't perfect data. There's always missing data, there's always bias, so there's always interpretation of the results and I think the committees are important.
To that end, at the National Eye Institute now all of our interventional studies have data monitoring committee review and I think it's important to note the differences that were pointed out earlier today between IRB review and data monitoring review. I don't think IRBs have the kind of expertise that is outlined in the document for reviewing accumulating data in a way that data monitoring committees do.
So at the National Eye Institute now all of our studies have on-going review. The intermural trials have one data monitoring committee. Many of the studies are very small. The committee probably reviews more than 20 different studies. They meet regularly but also have conference calls, interim conference calls, and when something comes up they review it.
Just one anecdote. I was reminded as I listen today, years ago a friend of mine in the Cancer Institute was talking to me about what he considered to be a very difficult situation. He was a statistician. He was looking at on-going accumulating data and noticed that there seemed to be more deaths than in the untreated group and he felt very concerned about noticing this difference. He talked to the investigator and as a clinician, we're all pretty adept at coming up with reasons why this person had this bad event or that person did and I think having this independent review is really an important part of clinical research.
DR. LEPAY: Thank you. Dr. Henderson?
DR. HENDERSON: I found the guidance document to be very well written, very well done, and I'd like to congratulate the authors. I think Greg Campbell did an excellent job this morning of pointing out the aspects and determining whether or not a data monitoring committee should be established.
Just a little bit about the VA. The VA is a very large health care system in the country. We do many different types of trials--drug trials, device trials, surgical trials, and lately we've been getting into trials dealing with health care organizations where the unit of randomization is not the patient but it might be the physician or the clinic or the hospital.
I found this document to be a very good exercise for me because it's just standard in our program that every one of our trials has a data monitoring committee. So I ask himself, why is this so? Are there some trials where we might not need it? And what are the reasons why we have a data monitoring committee for every trial? I mean we have some trials where the risk is not very great, like it's just symptomatic relief for the patient, but we still have a data monitoring committee and I came up with these reasons.
We do large-scale trials, multi-centered trials, mostly long-term trials. We have a vulnerable population that we're dealing with. But I think another very important reason, which is the third point that Greg Campbell brought up, and that is the scientific validity of the trial. I think an independent data monitoring committee gives the trial better credibility than if you don't have one.
One other thing I wanted to just raise and that is the perspective of the patient. I've been the head of a coordinating center doing these clinical trials for 25 years and I've always asked myself, would I participate in this trial that we're doing? I think the patient deserves protection and I think the data monitoring committee gives some of that protection to the patient in terms of having an independent body reviewing that trial.
So I would argue that most trials should have data monitoring committees, even the small trials. You can combine the small trials and have one committee review several trials if you have small trials but I would argue in terms of having a data monitoring committee in most instances.
I think it's also important to, in every protocol, to specify that you've thought about the data monitoring committee, whether or not it's needed, if it isn't needed, the reasons why, if it is needed, standard operating procedures, and so forth.
I agree with the other comments that data monitoring committees have been extremely valuable in our program and I would highly recommend them.
DR. LEPAY: Thank you. Dr. Walters?
DR. WALTERS: I, too, would like to commend the FDA and in particular, Susan Ellenberg for this very thoughtful guidance document.
I'd like to make three points in my comments. The first is that there's a gaping hole in the document as it stands and it begins with the title of the document. All of the focus is on the role of data monitoring committees and nothing is said in the title about the role of statisticians or coordinating centers and I think that these two groups, or in some cases it's an individual statistician, are equal partners and equally important partners in the monitoring of clinical trials.
In fact, I'd go a step further and say that the data monitoring committee meets quarterly or perhaps twice a year, takes a look at the data each time and renders a judgment. In an emergency the committee can be convened in person or by conference call but the individual or the group that's in the trenches day after day is the coordinating center or the statistician or statisticians responsible for the trial.
So I would like to see the role of the statisticians included in the title. I'd like to add "and the role of trial statisticians" to the title of the document. In part 3 of the document where it talks about DMCs and other oversight groups I'd like to take out "oversight" and just talk about the DMCs and other groups or individuals and include a separate section on statisticians or coordinating centers.
Secondly, if statisticians or coordinating centers have such an important role in studies then everything that's said in this document about the independence of data monitoring committees I think should apply equally to statisticians or coordinating centers. If the trial is going to be viewed as having integrity then the statisticians have to have independence and an insulation from the sponsors. I think Section 6 in this document on the importance of the independence of the data monitoring committees is an eloquent section of the document and I would like to see something similar said about these important statisticians or coordinating centers.
And third and finally, I'll say something about the composition of the data monitoring committees. Here I'm cheating a bit because we're supposed to only focus on parts 1 through 3 of the document.
Early in part 4 there's something said about the importance of having clinicians and biostatisticians on data monitoring committees. This is not simply an attempt to drum up jobs for people trained in ethics. I actually think it's very important to have an additional perspective on data monitoring committees; that is, one that complements the perspective of clinicians and biostatisticians. It may be a person formally trained in ethics. It may be somebody trained in law, as long as the person is not too adversarial. It may also be a consumer representative. But what I'm really interested in is broadening the viewpoint of the data monitoring committee and it's a kind of triangulation in a nonpolitical sense within the committee, to make sure that all important points of view are being heard.
I'll use an example from a recent DMC experience. Having someone from a Caribbean country in which a clinical trial was being conducted gave the data monitoring committee insights and points of view that we North Americans would never have had.
So the composition of the committee should be looked at carefully and I think in addition to clinicians and biostatisticians, it might be very useful to have one or two additional perspectives.
DR. LEPAY: Thank you. Dr. Wittes?
DR. WITTES: I'd like to echo the congratulations that everybody has made about the guidance document. I think that it struck really the right tone, that as a first guidance it's come out in a very flexible way addressing a lot of the issues and I think we'll all be fleshing out how it gets implemented over time.
I want to thank LeRoy for his very eloquent support of statisticians and also to comment that I, over the years, have found how useful it has been to have ethicists--and actually I like them trained in ethics--on the committees because they do bring a very, very different kind of orientation and perspective that I think is very useful.
I'd like to tell you a little bit about how I started in DSMBs or DMCs--I will try my best to change the initials--and then to argue for some training, which I think Greg alluded to but I want to emphasize.
My first experience was at NHLBI. I came in in 1983 and like the first day I was there Gordon Land, who's here, and Kent Bailey--I don't know if Kent is here--came up to me and he said, "Look, just go to every DMC"--then it was DSMB--"every DSMB that you can go to because you can learn a lot, it's the only way you're going to understand it and it's really fun."
So I did that. Now, of course, unfortunately in these days we can't do that anymore because now there's many more rules about who can attend and who cannot attend, but it provided for us at the Biostat Branch, for the Biostatistics Branch at NHLBI, the ability to go to committees to really understand--and I echo what Rick said--the fact that these decisions and the discussions are very complicated, they're very nuanced, and they reflect a certain sociology of a committee that varies from committee to committee.
And I would contend, and this is leading into the training, that if one plops a statistician onto a committee as the first time that person has ever been on a committee or one plops an ethicist or one plops in a clinician, although there's usually some other clinicians on the committee, it can actually be very harmful because the person is learning and training at the same time, learning him or herself and training the committee in statistical or ethical principles for DSMBs for the first time.
I do think that topic number two, the guidance talks a lot about the similarities between government and industry trials and roles of DMBs in the two and I've been vacillating over the months that I've thought about this but I've come to believe that there is actually a profound difference in the way in which these two sets of trials are run, that government trials, as several people here on the panel from either NIH or Bill from the VA, that they are really spending public money and they're sponsored by the public and there is a sort of public trust that I think is fundamentally different from an industry-sponsored trial and I think we do have to think about how that translates into what roles of DMBs, and it'll come out, I guess, in the afternoon, who attends.
The other issue I did want to raise, I have to respectfully disagree with Greg on his extension of the roles of DMC to recommending changes in certain aspects of protocol. And again I vacillate about this. I think it's very important to have flexible designs for trials but I think that a data monitoring committee--remember a data monitoring committee is seeing data on efficacy and for it to have the ability and the right to change end points and to change crucial aspects of design I think can sacrifice the integrity of the design. I think we have to think very clearly about who is responsible for that and whether that's a DMC role or not. Thank you.
DR. LEPAY: Thank you. I'd like to open this up now among the panel for any additional comments or questions, information, they could provide us with. So again any takers?
DR. CONNOR: I'd like to just follow up a little bit on what Janet said about training and the composition of the DSMB or DMBs. One of the things that happens during the years that I've been on the industry side of this is that obviously when you're approaching a phase III trial and a lot has gone into the development of a particular product you're in many ways handing over to this independent group a lot of very profound decisions. That obviously is true in the public sector, also.
But the talent base of folks who understand the role of the DSMB and the decision-making of the DSMB is really very critical and in all the instances that I've been involved with so far, we've been very lucky in the sense that both on the NIAD side and on the private industry side we've been able to have folks that are very talented and experienced involved in that process but I can imagine that there are instances where, as more safety monitoring committees are charged and more large clinical trials get done, the need for folks specifically experienced and mentored in the process of DMC activities is really very critical and the confidence with which folks are able to invest the responsibilities into the groups is very importantly based on the talent base that exists to be able to accomplish those goals.
So somehow as we implement this very important process more broadly than we have it right now, it's very important that an element of specific attention be paid to the development of folks with specific expertise in this area.
DR. FERRIS: I'd just like to follow up on that with regard to clinicians on data monitoring committees because it's clearly important to have that perspective.
One of the problems that I've seen over the years with clinicians on data monitoring committees is by nature we're interested in individuals and what happens to this individual and at times some of the clinicians have asked literally for every case report. Bring in the wheel barrows because they want to see every last piece of data.
I think it's important to have all perspectives but among the clinicians I think there has to be at least one who is experienced in clinical trials and clinical research so that the committee doesn't start down the wrong path.
DR. HENDERSON: I thought Janet raised a very interesting point and that is the trials at NIH and VA are government-sponsored, whereas the industry trials are sponsored by industry, funded by industry, and what implications does that have on the need for data monitoring committees or the operation of data monitoring committees? Did you have something in mind by your comment?
DR. WITTES: No. My comment was just that my goodness, they're different and that we need to think about--it's actually been precipitated by some issues where some of the institutes want to be in closed sessions of committees and some of them do not. Certainly in industry-sponsored trials--well, I shouldn't say certainly--I think the standard is not to be there.
So I've been actually struggling in my own mind about whether the same model should apply and whether it is ripe or not ripe for government sponsors--and whether the word is sponsor or not, I don't know--to be in closed sessions. So I don't have an answer but I do think the thinking needs to be different.
How's that as a cop-out answer?
DR. HENDERSON: But it seems to me that I think in the document they made reference to the independence of the data monitoring committee and the fact that the industry is actually excluded from the discussion of the outcomes broken down by treatment group or they aren't involved in the data monitoring committee at all, and that's the definition of independence.
It seems to me that in any case I think the independence is good but basically the data monitoring committee makes recommendations back to the sponsor and then it's the sponsor's job to act on that. They might act on it; they might not act on it. So the industry sponsor has the last word on those issues.
One question that was raised in my mind, what if there is a conflict between what the data monitoring committee recommends and what the sponsor wants to do? How is something like that resolved? Maybe that'll come up later on in operational issues.
DR. WITTES: I think what Bill raises is exactly the issue that I've been struggling with. If a committee comes and recommends to the sponsor, either the government or the industry sponsor, to make such-and-such a change, I think the tradition has been for such an industry recommendation the industry ought to make that change and the committee may not say why it's making the recommendation. It just says make this change or let me see these data or let us see these data, or so forth. Whereas when such a recommendation goes to a government sponsor it is very hard to not give the information that's leading to the recommendation and it's very hard to expect that somebody responsible for public monies is going to make changes without justifications.
DR. ELLENBERG: I just wanted to respond to a comment that Janet had made earlier about the role of the data monitoring committee in making protocol changes. I just wanted to clarify that we certainly agree that when a group has seen interim comparative data they're not in the best situation to make a recommendation on a change that could, in fact, be impacted by the data that they've seen. But the fact of having a data monitoring committee monitoring the trial actually frees up the trial leadership to make changes because there may be a need to make a change in a trial. Sometimes it comes from external information that comes out and if the only people who are in a position to make the change are people who have seen the interim data, you have no way out of this sort of conundrum. But if the data monitoring committee is reviewing the interim data, then that will free up the trial leadership to be able to make a change that they think is needed.
So our intent is not that the data monitoring committee would, in fact, be recommending a change in a protocol end point. It's that they protect the ability of the trial to make such changes.
DR. FERRIS: I'd like to just address the issue of whether the government and industry are the same. I think we can probably all agree that they're not and there are certainly perceived differences between how the trial comes out and how the government wants their trials to come out and how industry wants their trials to come out. I think we all want them to come out successfully but a lot of the trials I've been in, I would have been equally happy if we showed the treatment didn't work. So there is a difference.
However, I think it's important to remember that data monitoring committees aren't always correct. I was listening to the historical issue of the University Group Diabetes Project and I was thinking that based on UKPDS results, maybe the first data monitoring committee made a mistake.
I think there are times where the decisions from a data monitoring committee need review and I know at National Eye Institute a number of times we've either had ad hoc or in-place review committees review the data monitoring committee's assessment and there have always been times when the data monitoring committee is not unanimous. And a lot of data monitoring committee work--I think some of what Janet was talking about in terms of the training, they really are consensus development exercises as much as frequent statistician assessment of the data.
DR. ELLENBERG: We do recognize that government and industry trials are different. We do think, however, that the issues that are raised can really apply to both types of sponsors. What that means in terms of implementation of approaches may differ but it does not mean--what Rick just said about sometimes data monitoring committees may make the wrong recommendations, I think that's true. I mean I think the strongest support of data monitoring committees would never say they're right 100 percent of the time, but that's true for data monitoring committees in industry trials just as well as data monitoring committees for government-sponsored trials.
So I think the fundamental issues are ones that all sponsors need to think about. That's really the main point.
DR. LEPAY: Dr. Walters?
DR. WALTERS: Janet Wittes's suggestions about training reminded me of another point that we might want to consider today and that is the role of empirical research on the actual functioning of data monitoring committees and perhaps evaluation research on how well they're functioning.
Perhaps that component ought to be built in right from the start of the FDA guidance so that 20 years from now the Office of Inspector General won't have to do an independent analysis and say oh, there's some deficiencies in the way data monitoring committees function, as that office did for institutional review boards.
So some kind of periodic look at the composition of the bodies, how many members there are, how frequently they stop trials before the planned termination, might provide helpful feedback on how the whole enterprise is working.
DR. LEPAY: Dr. Wittes?
DR. WITTES: I'd like to distinguish two kinds of right decisions. This is in relation to Rick's comment. In light of data that come out later we can always learn that we've made a wrong decision and that can happen in science in many different ways and that's why we replicate experiments, because it's possible that one experiment shows one thing and one shows another thing.
I think the best we can hope for for data monitoring committees is that they act rationally and reasonably and develop good consensuses that other people can look back and say yes, confronted with these data, I, too--I being a reasonable person, also--would have made the same decision or I can't fault the process of the decision. But we can't assume that data later is going to confirm what we think we saw.
Open Public Discussion
DR. LEPAY: Thank you. I'd like to open this up now to the audience. What we'd like to do is focus our comments and focus attention in this particular section on the first three sections of the guidance document if at all possible, dealing particularly with the need for a DMC and the relative roles of DMCs and other groups that are involved in overseeing clinical trials.
So again I'd encourage people to step up to the microphone. Again these transcripts are being prepared and we'd appreciate it if you'd identify yourselves.
DR. LEVINE: Thank you. I'm Bob Levine. I'll have my opportunity to speak later but I want to make two quick points on what came up in this panel.
First, some people might leave this room thinking that LeRoy Walters and Janet Wittes made the same recommendation about having ethicists on the DMC. LeRoy though, when he spoke of ethicists, included people who are not trained in ethics and even included somebody whose only descriptor was that he or she came from the Caribbean. I think what LeRoy's trying to tell us is that we need a different perspective and it may be an ethicist; very commonly it would be.
I think the later comments that were made about people who are schooled and working on DMCs is extremely important. There are a lot of tyroethicists who can be really very disruptive, thinking they're going to apply their principles in the field of clinical trials.
The other point I want to address is that there are indeed great differences between the DMCs in industry and in the government. I agree with Susan Ellenberg that they can all be expected to follow the same basic principles as set forth in this excellent document. However, they could learn from one another. Industry tends to have much greater formality in the contractual arrangements and much greater specification of such things as confidentiality rules and I think people on NIH DMCs could benefit by being reminded of that sort of thing. It's just assumed that everybody who serves on a government DMC already knows all about that and often most of them do.
I think government could also learn from industry about how much to pay a DMC member.
And my final point would be that one major difference, and this, I think, reflects what's been said about--I think Rick Ferris brought this up about the different ideas about what a satisfactory outcome would be--I think that we see that manifested in the industry's strong tendency to try to set the stopping rules or guidelines themselves, rather than let the DMC engage in its own exercise of establishing the stopping guidelines. And I think that there should be some discussion of that, about who should set the stopping--I don't like stopping rules but stopping guidelines, and how to go about doing that. Thank you very much.
DR. LEPAY: Any comments from the panel? Okay.
MR. CONSTANTINO: Joe Constantino from the University of Pittsburgh Graduate School of Public Health. I'm also the associate director of a data coordinating center and I really came here today to reiterate Dr. Walters's comments. After I read the document it was very clear to me that there was a gaping hole in the document in terms of dealing with clinical trials, data coordinating centers and the role of a statistician of that coordinating center with the DMCs.
Having had over a decade worth of experience on dealing with independent data monitoring committees, it's clear to me that it's essential that the statistician who works with the data monitoring committee needs to be that statistician who's involved on a day-to-day basis with the data and who sees it in an unblinded fashion. He's the one that actually is monitoring the trial for safety and brings to the attention of the data monitoring committee things that occur.
To suggest that an individual who should be going to the data monitoring committee, as is done in the later portion of the document, should be totally independent of the day-to-day operations is not in the best interest of the primary goal of a data monitoring committee, and that's safety of the participants.
The document doesn't deal enough with the interchange and the balance that we need to achieve between protecting the confidentiality of the data, the integrity of the trial, and protecting the participants in the trial. There is a big play-off of all of these things and this is where some of the differences between industry-sponsored and government-sponsored contracts come into play. There's differences there.
There's also differences that must be recognized that come into play in terms of people who actually sit on data monitoring committees aren't totally devoid of conflict of interest. These people participate in cooperative groups who are doing similar trials to the ones they're investigating. They go back to the universities and have colleagues who participate. So there are pressures on them to breach confidentiality but we accept those levels of breaches to protect the risk of the participants. This kind of balance of protection of the risk to participants versus the integrity of the trial needs to be stressed more in the document.
DR. LEPAY: Thank you. Any comments from the panel?
DR. WALTERS: Perhaps one of the reasons that the role of coordinating centers and statisticians is not accented more is that biostatisticians are very modest people. Even in a wonderful book like "Fundamentals of Clinical Trials," I would say that the role of statisticians in the conduct of clinical trials is, if anything, underplayed, even though this book was written by a group of very distinguished statisticians.
So FDA may accurately be reflecting what's in the literature. It may be that the biostatisticians are just too self-effacing.
DR. TEMPLE: Some of them perhaps. Actually, I wanted to follow up on the same area that Dr. Walters raised. The obvious reason that the biostatistical center isn't covered is this was a document about data monitoring committees but you can see in the document considerable nervousness about who does the analysis.
One model is that somebody in industry, presumably very shielded from the corporate management and everything, analyses, the data, presents it to the committee, but that makes people a little nervous, as the document describes, because there are nonverbal signals and maybe you really reveal it.
So the alternative is a more or less independent statistical center. But nonetheless, I think the document continues to treat that center as more a creature of the sponsor, working for the sponsor, and I can tell you personally these centers vary considerably in whether they're really neutral or whether they're really advocates for the sponsor.
So for all those reasons, the document doesn't dwell on that very much but sort of accepts a wide range.
Now I'm wondering whether you and the other panelists think that we ought to be more insistent on saying at least for major outcome trials that the people who put the data together really ought to be arms-length from the sponsors. Is that what you're proposing? I couldn't quite tell but I think it needs more discussion.
DR. LEPAY: Comments? Yes, Dr. Walters?
DR. WALTERS: Yes, I do think that there should be independence of the individual or group collecting and analyzing the data by treatment arm and that what's said in this document about the importance of the independence of the data monitoring committee for the integrity of the data in the trial applies with equal force to the role of the statisticians that are analyzing the data.
DR. TEMPLE: Is it particular studies that need that treatment, all of them? You're basically describing a situation in which drug companies no longer analyze their data, period. Is that what you're saying? Or is it only certain major studies with important outcomes where you feel that that was essential?
DR. WALTERS: I guess as a rule of thumb I would say that where there's a data monitoring committee there ought to be an independent statistical center or an independent statistician who serves the data monitoring committee.
DR. WITTES: I think there are several issues being conflated here. There's issues of confidentiality, there's issues of conflict of interest, and then there's issues of credibility. I think these are different. And I think they're going to come up this afternoon but it's important to keep them separate and it seems to me that each one of them, as you think of each one separately, it speaks to a different kind of model and the issue we have to face is how do you have one model that satisfies them all?
DR. FERRIS: I'd like to make one comment regarding this and that is when it comes to rules for data monitoring committees I'm not sure there should be any. There are probably a lot of ways of doing the job and I'm not sure any one fits all. I think saying that never can a company do its own statistical analysis seems to go too far. If a company does do its own statistical analysis surely there will be skeptics and critics that are going to want to see that data and do the analysis another way. And I think we all realize that the data monitoring committee is beholden to the coordinating center and statistician. A lot of mischief can happen between the data and the data monitoring committee, so having good, competent people is the key. And, in the end, fudging the data is going to wind up being detrimental to everybody.
DR. LEPAY: I'll go to the speaker at the microphone.
ATTENDEE: Actually, I think I'll yield to the ones in front of me because I have a feeling they want to talk about the same vein and I want to take another one.
ATTENDEE: Just a follow-up on the point that was raised a little bit earlier. It is important for the data monitoring committee to deal with a biostatistical center which is also independent but there are levels of perceived independentness. Clearly a statistician who's working for a private research group around the beltway is different than one that's working for an academic-based clinical coordinating center. It's different than one that might be a private consultant working for an industry.
These are the types of things that need to be recognized as differences between the types of trials. And when I said there's a give and take between--an arm's length is an arm's length but it might be a two-foot arm or a three-foot arm and sometimes a two-foot arm is acceptable. These are the kinds of things that I think need to be brought out and made clear.
DR. ELLENBERG: Could I just ask for you to elaborate on the difference between, say, a coordinating center at an academic organization and one that's a private consulting group?
ATTENDEE: Sure. An individual who's working at an academic center has his primary boss as the university. He's a tenured person at the university. His job doesn't depend on whether or not, in a real sense, whether or not this trial turns out one way or the other.
So in a perceived sense--maybe it's not true in reality but in a perceived sense he's going to have "less of a conflict of interest" than somebody who works for a private company who makes their whole living by doing these kinds of things for industry or specifically for an industry group panel set up to do the analyses.
So these are all perceived levels of independentness that need to be weighed plus and minus against how far does the perception have to go to protect the integrity of the trial? That's the kind of thinking that I think is still missing in this document.
ATTENDEE: I reserve the right to go back to my original point but I can't let that one go. I think that you've gone too far. It's absolutely not true that everyone at an academic institution is not beholden to the sponsor.
ATTENDEE: I said perception. I didn't say reality.
ATTENDEE: But the reality is important. I mean many people are totally dependent on the grants or contracts from NIH or industry for their job and they don't have a paycheck if that contract ends for whatever reason. So I think we do have to be careful here.
Also, I think there is both a real and perceived difference between coordinating centers who are sponsored by the NIH and coordinating centers who are sponsored by government--I'm sorry, by industry. At NIH it's virtually impossible to have more than a two-inch length from the sponsor to the coordinating center. They hold the contract. In many instances, if not all, they actually interact quite closely with the DMC and the coordinating center. They also see the unmasked data, whereas in most industry studies, at least that I have some responsibility or interaction with, they're more like at a one-mile length as far as the blinded data. At least that's the way it's perceived. I'm not sure about the reality all the time.
I do want to say something else but I'll let Dave talk for a minute.
DR. CONNOR: I think a lot of the issues related to industry trials--and while I don't represent industry I do have some experience in doing that over the last couple of years--is that obviously the outcome, the desired outcome is approval of a drug and the ultimate arbiter of that is really going to be very dependent on that arm's length decision.
So a lot of effort gets put into really assuring that we're as separate from that decision as possible so that, in fact, at the end of the day the integrity of the trial is maintained.
So I think there's a lot of effort on the industry side, as folks have pointed out, to be sure that the arm's length is several arm's lengths away and how that gets accomplished is obviously dependent on the organization. In some organizations it may be eons away where the analysis gets done, rather than the corporate decision-makers are and in other places which are small organizations like ourselves, we really depend on the independence of separate organizations to do those analyses because it is a smaller kind of organization.
DR. LEPAY: You had another question?
DR. DeMETS: Dave DeMets, University of Wisconsin. I have two points: one on IRBs and one on training.
I'm not sure what the ultimate responsibility of IRBs will be but I'm pretty convinced as of right now that IRBs are not in a position to do much monitoring, as we're talking about here. The composition, the resources, the talent just isn't there. And while we may want them to do certain things about monitoring local studies, the fact is they can't do it and it would be a terrible disservice to patients and investigators if we dump that responsibility onto IRBs without a substantial investment in those IRBs. IRBs have had enough trouble meeting the paper requirements, as we've learned recently, but to ask them to do the other, do additional without substantial increases of resources and talents would be a recipe for disaster.
The second point, on training, I have to take an opportunity to put another plug in. Some wag said that this document is a full employment act for statisticians. The current situation before today might be that we already are desperately short of a training pipeline of biostatisticians. Those of us who are in academic departments training biostatisticians know that students go out and get four and five job offers. When we try to recruit faculty we work at it for a long time.
So the pipeline is already short and if this process, which I strongly endorse and support, nevertheless, we have a double training problem. We have to train those we have but we have to step up the training process and right now there's no initiative in place to do that.
DR. LEPAY: Thank you.
MR. VERDA: Joel Verda, George Washington University. I almost yielded too much because Dave actually started along the lines that I was heading for.
My concern is that the document, although it's specific for DMCs, has opened the door for another issue and that is the IRBs. Over the last 50 years as clinical trials have developed we've seen developments in coordinating centers, in design, in monitoring, in DMCs going from occasional trials to almost all to almost all industry trials of the nature described this morning.
But in the last five or six years we started to see a trend that's a little disturbing and that relates to the IRBs' responsibilities. We, for example, recently have received two or three requests from IRBs for blinded data, saying that they can't do their job unless they see blinded data. I think someone, and I'm not sure who it is; I'm sure it's not this panel but the FDA, NIH, OHRP--somebody has got to give these poor souls some guidelines, what they don't have to do and what they do have to do.
I certainly agree with Dave that it's impossible for a local IRB to become a DMC. In fact, it would be the death knell of any clinical trial if you had 12 or 160 IRBs trying to monitor the trial along with the DMC.
DR. LEPAY: Thank you. I was going to say I think that's an issue we're also going to take up this afternoon but certainly that's one of the major impetuses behind our discussions here today, is to come to reality with respect to the fact that there are certain responsibilities that need to be met in clinical trials and we need to look very carefully at where those can best be accomplished. And hopefully that is going to be one of the take-home messages at the end of the day, both for us and for those who will see this transcript.
If I could go to the next individual in the back?
DR. STUMP: Dave Stump from Human Genome Sciences. I'll have several comments to make in one of the afternoon panels but I did have one topic that I'd like to bring up and maybe elicit some comment from the panel. It has to do with when is a DMC needed?
In Dr. Campbell's presentation and in the guidance document it talks about a therapy that is so novel that there's very little information on clinical safety that exists. This can actually be the case with many phase I trials, any new molecule first entering man. I'll argue that for novel biologics, something I actually live with day in and day out, you may often not have relevant preclinical data because of species specificity of human proteins.
Would it be the panel's view that phase I trials require DMCs and if DMCs are required do these need to be external DMCs? We actually get IRB requests now for multi-center phase I trials for external DMCs, which in my mind seem to supplant a great deal the relationship historically that has worked between the sponsor's medical monitor and the FDA's product reviewer, where a constant dialogue takes place with frequent safety monitoring of these trials, but it's becoming an issue certainly for those of us on the sponsor side and I'd love to hear some discussion about it.
DR. LEPAY: I'd like to go down the panel, if possible, and see if we have any comments. This is an issue that's certainly very pertinent to us in developing this guidance.
DR. CONNOR: I think a lot of the issues, some of the issues are addressed in the guidance document but are a little unclear as to the answer to that question. From our perspective, we are also in the position, similar to the last speaker, where more and more is being demanded of the sponsor from the IRBs relative to separation and independence even early in clinical development, so much so that now very often the IRB will regularly request updated information, albeit blinded or unblinded, on a regular basis, demanding a lot of resource intensity to provide such information while the trial is actually on-going and, in addition to that, now actually making specific demands that there be an independent individual in early clinical safety monitoring committees even if the origin of those are actually internal.
I think we've debated a lot about the value of that, early on. The expectation is that there are specific reasons for such review; we've accommodated those reviews. And I think that it's important in other instances where there's not a specific safety concern or there's not an expectation that there's going to be the need for more broad review, we have tended to wait until the next set of trials, not the early dose escalation range-finding trials but the set of trials that's sort of the transition between early clinical development and phase III clinical development, which is where ideally most of the pertinent discussion resides.
DR. ELLENBERG: Before other people comment I just want to make a clarification that our intent in this document was not to suggest that a large majority of phase I trials would require data monitoring committees. We think that there could be, on occasion, an early phase trial of something where there really were important safety concerns and where a set of people without any particular investment in the trial might provide some useful advice, but our intent is not to suggest that that would be typical or even frequent but rather, a rare occurrence but a possibility that we wanted to raise.
DR. FERRIS: I said earlier, and I echo what Joel said, that I think the responsibilities of the IRB and the responsibilities of data monitoring committees, although each have factors that are similar, the differences are important. And to that end, what we've done, and I think on an institutional basis it doesn't have to be an NIH institute but any institute that has an IRB, they may want to consider what we've done. That is we've formalized the relationship between our data monitoring review committee and the IRB.
I don't think--I said before I don't think there should maybe ever be rules, stopping guidelines; DSMC guidelines are appropriate. Independent review I think is important, of the data, and if the IRB works something out with whether it's a DSMC or some other independent reviewers, I think that's helpful to have in place so that whenever the study is--these are all intervention studies I'm talking about now--is reviewed by the IRB, that there's a written document from some independent group saying we've looked at the data and at this point we don't see any evidence to modify the study.
DR. HENDERSON: We haven't had really any experience with phase I trials so I really can't comment on that.
I would like to make one comment about the IRB issue. We're also seeing the phenomenon of local IRBs in the VA system requesting unblinded data and what we've tried to do is we have a data monitoring committee reviewing each study and once the committee meets and decides on an action, we communicate that action in general terms back to the local IRBs because I think that many of these local IRBs aren't even aware that there's a central DMC reviewing the data, outcome data from that study. So we communicate back a general statement to them that these are the data monitoring board members, they reviewed the study on such-and-such a date and their overall recommendation was that it continue and there are no safety concerns, a general statement like that. Whether or not that's going to be adequate for the local boards, we've only been doing this for about six to 12 months so I'm not sure.
DR. WALTERS: The document deals with the question of independent safety monitoring on page 16 in 4.4.2 about early studies and I guess I would suggest that even in phase I studies, independent safety monitoring is really important and it's simply to guard against self-deception by the investigator who's trying out something new. It's another pair of eyes, just as a check. Very often it won't be a committee; it will just be another person within the same institution or the same company. But it provides a measure of safety for the participants even in phase I studies and it's something that IRBs simply are not equipped to do.
DR. WITTES: I actually think the question is backwards, that we shouldn't be asking whether phase I trials need DMCs but we should be asking what safety monitoring should be done for phase I trials.
I think the issues have come up because of at least three really unfortunate events--the liver toxicity death at NIH, the death at the University of Pennsylvania, the death at Hopkins--and I think that what it says to people is my goodness, maybe phase I trials are not being looked at in the way they ought to be. But I agree with LeRoy that the way that one can monitor trials for safety need not necessarily be a DMC.
My own personal experience being on DMCs for phase I trials is that we were singularly ineffective, that the trials go on, as Greg described, the trials can go on so quickly that the DMC doesn't function and that's really what happened to us in several trials.
So I think what has to happen is in a phase I trial of a novel entity there's got to be a really clear safety monitoring plan and we need to be very flexible about how it gets implemented.
DR. LEPAY: Thank you. I'd like to take each of the speakers who are currently at the microphone. I think I'll start on my left. Please identify yourself if you would.
MR. VENABLE: Tom Venable from Fujisawa Pharmaceuticals. I have a question about data coordinating centers, back to the arm's length or kind of a rock and an expensive hard place question.
Sponsors have to maintain the blind in-house, all right? That usually sets us on a model of doing the data coordinating center through a CRO. Will the guidelines emphasize that independence of data coordinating centers or will it invite the mechanisms to occur within a sponsor?
DR. ELLENBERG: We'll be dealing with that this in talks later on. We'll go into that in more detail.
DR. LEPAY: In the front?
MR. LEWIS: It seems like all three of us are Toms. Tom Lewis, RAND.
I'd like to get back, although the previous person did also, to the topic that vexes everyone in Statistics 1 and that is statistical independence, in this case independence of statisticians. I think the document is too vague on it because every DMC I've been on or every coordinating center I've been in, at least in the coordinating center role, we are totally collaborative with the investigators, that independence is not viable if you're going to be a statistical scientist, as opposed to one running the data.
But what's very important, and I think the document should focus more clearly on it, is independence in a certain role. It's that role of monitoring the study and preparing reports for the DMC and interacting with the DMC and with that kind of clarity I think it's a good concept. But the idea of just generally saying the statistical center or statisticians are independent of the sponsor is, in fact, promoting what is a very bad idea.
DR. FLEMING: Tom Fleming, University of Washington. Janet in her comments appropriately emphasized the importance of experience in the people who would be on monitoring committees. At the same time it's been acknowledged that these committees are much more broadly implemented. And Greg Campbell in his presentation, under the topic of practicality of DMC review, acknowledged then that one of the logical issues that follows is are there going to be adequate numbers of well qualified experts?
I think as we configure these DMCs we need to be thinking not only about today but about the future. And in configuring these committees to address Janet's issue of ensuring that there are people that can be available that are experienced, many of us have argued that we should be thinking about an apprentice approach where you intentionally select in your configuring these committees a combination of people with experience and without. So if you have two statisticians, for example, you try to bring in diversity, one with experience, one who really has important contributions but without the experience and they wish to gain that experience.
It is, in fact, an additional investment today but I think sponsors, both government sponsors, industry sponsors, and societies for clinical trials should be thinking carefully about this issue, about how can we work together to configure today's committees in ways, for example, through an apprentice-type approach, to broaden the population of experts who have the experience for future DMCs.
DR. LEPAY: Thank you. I'd like to thank our panelists for their excellent contributions, to those members of the audience who provided additional comments, and we're going to move on to a discussion of the next section of the document. So if we could give a hand to our panelists.
[Applause.]
DR. LEPAY: Our next speaker is Mary Foulkes, deputy director of the Office of Biostatistics and Epidemiology in the Center for Biologics, and she's going to discuss the section of the guidance document dealing with DMC establishment and operations. Mary?
Establishment of DMCs and Operational Issues
DR. FOULKES: Thank you very much, David. After this morning's discussion I'm going to start by assuming that we've already addressed the question of whether or not a DMC is necessary and then ask the question what's next, what follows?
If there is to be a data monitoring committee it's generally one that is appointed by the sponsor. And by that I'm terming the sponsor as a very broad use of that term. If there is, in fact, an existing steering committee, the appointments to the data monitoring committee are usually mutually agreed upon between the steering committee and the sponsor. Sometimes the sponsor delegates this responsibility, as has been mentioned already this morning. The DMC is also funded by the sponsor in the sense of covering expenses for the meeting, honoraria, et cetera.
And the specifics of the need to maintain some independence between the sponsor and the DMC, as we've already discussed a little bit this morning, will be discussed in much more detail after lunch by Jay Siegel.
There are multiple factors to be considered in the construction of a data monitoring committee. Not only does there have to be an agreement among those who are selecting and identifying the membership of this DMC; it needs to be multidisciplinary, as we have heard, and I'll talk a little bit more about that in a minute.
The size of the DMC is really a function, largely a function of the complexity, although we've just heard a few suggestions for expanding the size of the DMC, which certainly ought to be considered. Then the membership of the DMC have to be in general agreement with the clinical trial as it's proposed with the specific hypothesis that's to be addressed, with the design of the trial, and with the end point that's been chosen. And we've already touched on the issue of minimizing the overall conflict of interest.
To get back to the size of the DMC, the document does refer to an expected minimum size of three, approximately three. There have been examples of smaller size DMCs but they have generally had some serious problems, so the recommendation is to have a committee of at least size three.
And as I was looking over my slides this morning I realized that I actually made this slide before LeRoy's comments earlier this morning. I would suggest that the areas of expertise that need to serve on a DMC are first of all, obviously the relevant specialty of clinical medicine that's appropriate for the given trial; the expertise in biostatistics that we've already heard about, and modesty prevents me from going further; the involvement of biomedical ethicists. As you can see, the top three are highlighted in yellow.
If your DMC is larger than size three you should consider involving some other specialties as a function of the characteristics of the trial. And also it has been mentioned earlier this morning the involvement of possibly a patient advocate, community representative. So these are the various persons that would be suggested as possibilities.
Then there are other issues to be considered when you're constructing your DMC. We've already touched a little bit upon geographic representation, representation of the relevant demographic characteristics, which comes into play, for example, if you're dealing with a study that involves one segment of society versus another.
We've already also heard discussion of the involvement of individuals with prior DMC experience, which is very important.
The aspects of conflict of interest. I don't mean a very narrow definition of conflict of interest. Conflict of interest can involve lots of things. It can involve financial conflict of interest. Investigators enrolling in the clinical trial itself have a certain conflict of interest. Then there is a very broad category of intellectual conflict of interest. So this is not meant to be a very narrow aspect to be considered and all of these things need to be considered when you're constructing your DMC.
The other thing to be considered, which is a very important choice to make, is who is the individual who's going to serve as the DMC chair? In this context even in the situation we face right now with limited numbers of individuals with prior DMC experience, it really is important for the person who serves as the chair to have prior DMC experience. They also obviously have to have a very strong scientific background relative to the trial at hand. They have to have some appreciation for the administrative issues because a lot of the recommendations from a DMC have administrative implications.
We've talked about consensus-building and being a facilitator. That is a very important skill that this individual must bring to the process. You'll see in a moment that their skills as a communicator are going to be called upon, so that needs to be considered.
And lastly, they really should be in a position to make a commitment for the duration of the trial. It's somewhat disruptive to have changes in the investigators involved in the trial in the middle, it's somewhat disruptive to have changes in the individuals participating in the DMC but it's very disruptive to have a change in the DMC chair. So this individual should be willing to commit for the duration of the trial.
In the document we recommend that there exists a DMC charter or standard operating procedures and that such a document be developed in advance of the instigation of the trial, if possible, and in advance certainly of the initiation of any interim analyses.
The document also discuses the schedule and format of meetings. The schedule and timing of meetings is largely a function of the structure of the trial itself, the interim analysis plans that are an integral part of the trial, but that needs to be planned in advance believe obviously there are a lot of logistic and administrative issues having to do with that.
The frequency of the meetings, as we've heard earlier this morning, has a lot to do with the specifics of the trial--how rapidly the recruitment occurs, how rapidly the end points are observed, and that sort of thing. All of these have to be taken into account with regard to how frequently the meetings occur.
Also mentioned earlier this morning is the possibility of teleconferences. That sort of thing should really be a part of the discussion in developing a charter or an SOP. When do we meet face to face and when do we have teleconferences?
Also the question of what is a quorum for this DSMB is important. It's much more important when the size gets beyond the size of three because you can have DMC meetings scheduled and have the inability to get together the entire committee, so it really is important to discuss what in essence is a quorum.
And then this sort of charter or SOP needs to delineate the data access. Who has access to what data and how much of it? And is it blinded or unblinded? That ought to be delineated and spelled out at the beginning of the process, hopefully before the trial begins but certainly before the interim analysis begins.
And then some discussion of the meeting attendees, and that's also been brought up earlier this morning. I'll discuss that in a minute as we go through the structure of a DMC meeting.
There has to be some clear identification of how conflict of interest will be assessed. Some of the DMCs I serve on, there is a reassessment of conflict of interest on an annual basis and it's a very clear process. It's very helpful to have that clearly identified in this charter or SOP.
And then the method and timing of the distribution of reports. Obviously we're still in the stage where most reports are produced on paper and so they have to be physically delivered. So how the DMC reports are delivered, at what time they're delivered, are they delivered to the hotel the night before the meeting, is the DMC expected to receive the reports hand-delivered in their offices seven days prior to the meeting or by FedEx to their home doorstep? All of these things have to be considered.
There has been some discussion of the statistical methods already. All of this really does need particularly to be spelled out in advance of the trial. The statistical methods to be used may cover a broad variety of possible approaches--group sequential analyses, possibly Bayesian methods, other methods. Certainly we talked about trials being living things. Statistical methodology is a living thing, as well, developing over time so the approach that is intended for this trial does need to be spelled out.
Also very important is the discussion of how the type 1 error rate is to be handled, how the type 1 error rate is to be allocated throughout the course of the trial. All of this needs to be very carefully spelled out in advance.
There also should be some consideration in advance of the conduct of the trial if and when a futility analysis should be considered, so that should be an issue that is at least discussed in advance.
And one of the things that DMCs are charged with is finding a balance between the risk and the benefit, so how this risk/benefit assessment is expected to be conducted. On occasion, DMCs see data that provide a certain amount of information with regard to the benefit but they don't necessarily have a solid handle on the measure of the risks, so their recommendations to the sponsor may be somewhat a function of which side of this equation they have more information on.
Again these are the types of issues that need to be addressed and considered in advance of the interim monitoring process.
Confidentiality we have already discussed to some extent but I think it's a general agreement--I hope it's a general agreement--that the interim comparative data are generally considered confidential, highly confidential, during the process of the trial conduct. The sponsors should establish existing procedures to ensure the confidentiality of the data. We've already heard examples where the possibility of knowledge of the interim data could affect the trial conduct and some examples of those are when there is an unstable situation, things are fluctuating and changing very rapidly. There may or may not be an emerging trend. It may be a solid trend that we see. We see this morning how long it's taken the economic community to agree that we're in a recession so it may take a while for emerging trends to be recognized.
Then we have the situation of interim reports. The knowledge of the interim report is not necessary for the investigators and/or the sponsors to do their job. Otherwise they wouldn't be in the process of conducting a randomized control trial and particularly a blinded randomized control trial. So we have this scenario where we have a data monitoring committee charged with monitoring the on-going trial.
The interim reports obviously have to be based on a prior established analytic plan, which is spelled out usually in the protocol and possibly in greater detail in later documents. We've already touched on the discussion of the statisticians preparing the report and their level of independence from the sponsor.
I mentioned the issue of the timing and the distribution. The timing of when an interim analysis takes place should be a part of the plan, at least fleshed out in terms of how we intend to approach this issue, if not specifically nailing down the timing to the exact date for each of the interim analyses.
And then the comparative results usually are prepared in a printed report in a coded fashion, and by coded I mean blinded. The columns are labeled treatment A and treatment B or treatment 1 and treatment 2, and that sort of thing. Then in the process of the data monitoring committee meeting, the data monitoring committee has access to the unblinding of those codes. That is one additional level of protection.
I do remember a situation where a data monitoring committee member was en route to a data monitoring committee meeting and inadvertently left the monitoring committee report on the plane, so it really is useful to have these reports printed in a coded, blinded fashion for that reason, if for no other, but certainly there are many others.
Now with regard to the specifics of the meeting, there are separate parts of the report that are useful and used in the open and the closed sessions of the meeting and I'll go through the parts of the meeting that usually take place in a data monitoring committee meeting.
Here you see the meeting starts with an open session, followed by a closed session. There is potentially or optimally an executive session and lastly, a debriefing session. I'll go through each of these in some detail.
In the open session those attending the open session are possibly the steering committee, certainly the statistician who presents the interim reports for the DMC review. There may be some representative from the sponsor. There may be the individual, the principal investigator or the individual who serves as the study chair. There may in the open session be regulatory representatives attending.
In an open session only the aggregate data are presented--the total number of people who have enrolled in this trial to date, and so forth. There is an opportunity for communication of possible problems that the sponsor might be able to take some action about. For example, in an open session I have been involved in discussions of does this placebo taste like it's supposed to taste, and everyone in the room was given a placebo tablet to taste. Those are the kinds of issues that can be discussed in an open session.
Discussions of implications of possible external research. We've heard mention of this issue and possibly this is going to come up more frequently. As research of this type is more globalized we'll hear about results from trials in Japan and need to address the issue of how do those results impact the trial that we're reviewing in front of us?
Then there is the opportunity to communicate without disclosing the comparative data. One can communicate that there are some enrollment problems, there's some problem with the laboratory, there's some problem with getting the data submitted centrally in a rapid fashion and that sort of thing. All of these types of issues can be communicated in an open session.
The kinds of topics that I've already mentioned--the accrual rate, the baseline characteristics, whether or not there's a problem with regard to compliance, whether there are problems with missing data, if the amount of missing data or the timing of how rapidly that missing data is retrieved, if at all possible, or if it's impossible to retrieve. That sort of thing can be discussed in an open session. The overall toxicity picture, if it doesn't provide information that unblinds the trial, and then the site-specific issues--if there's a problem with one site or if, for example, in the VA system, and Bill can correct me if I'm wrong on this, they sometimes identify more clinical sites than they need so they have one or two back-up sites and if a site is not performing, then they bring in the next team.
Now to the closed session. In the closed session only the DMC members and the presenting statistician are recommended for attendance. The document discusses who should attend the closed session but it really should be a much, much more limited group of individuals than those in the open session, and we've already touched on this topic a little bit already this morning. And it is in this session that the comparative unblinded data are discussed and presented in detail and it is at this session that the recommendations, the formal recommendations to the sponsor are formulated among the DMC and a consensus is arrived at.
So that's the number of slides devoted to the open session, and the closed session don't necessarily reflect the relative amounts of time allocated to the open session and the closed session but they do delineate what gets covered in those two sessions.
Then there is the possibility of an executive session. As I mentioned, that box was a little off to the side because it doesn't necessarily occur at every meeting of the data monitoring committee. There is or is not an executive session when the sponsor representatives have participated in the closed session and the DMC wants to meet and discuss only among themselves. There may be other issues that are appropriate for discussion in an executive session--topics dealing with study conduct, dealing with how the interim analyses are being conducted, dealing with the review process itself, dealing with the external study results, et cetera. This is again the session wherein only those members of the DMC are present and no one else.
Then at the end of the process there is a debriefing session where the DMC chair meets with either the representative of the steering committee or the representative of the sponsor or whoever the individual is who represents the sponsor in the context of delivering the recommendation and possibly orchestrating, taking some action on the recommendation.
There may be other issues dealing with the study conduct that are discussed in this debriefing session. There may be some clarification of the concerns that the DMC has and the specifics of the recommendation from the DMC to the sponsor to the organizing team of the trial are conveyed in this context. They're conveyed in this debriefing session verbally but again they're conveyed in a written form, as well.
The specifics of the DMC responsibilities. The organizational structure, the individual expertise represented within the DMC, the SOPs that we've already discussed, the analysis plan, the interim reporting, the meeting structure are all put into place to support the DMC in fulfilling its responsibilities and those responsibilities are listed here, the primary ones being to evaluate the accumulating data with regard to both safety and efficacy, to provide a recommendation whether or not the trial is to be terminated or to be continued as it was originally designed or possibly to be modified in some sense.
The other responsibilities of the DMC are to review and approve the protocol. Possibly this comes in in some DMCs that they receive the protocol before the trial is initiated and they review and approve the protocol. This doesn't necessarily occur in 100 percent of the cases.
They have some responsibility for assessing the trial conduct and we've discussed the differences between the IRB level of review and the DMC level of review so there are a lot of ways in which the DMC can review the trial conduct, but they are certainly not the only ones involved in this and they may in some sense, recommend additional analyses either to be conducted at the time, at the moment, or just prior to the next DMC meeting, or possibly recommend analyses that the sponsor might want to undertake at the end of the trial.
The primary responsibilities--again, monitoring safety and effectiveness, to focus on the monitoring of trial conduct, to deal with any external information that might emerge. We've already talked briefly about involving DMCs in the process of early development, involving DMCs in monitoring phrase I trials. That sometimes is a responsibility of the DMC.
A major responsibility is to convey recommendations in a clear and useful fashion to the sponsors and the DMC is also responsible for meeting records--not only the terse, sometimes cryptic but hopefully usefully written but not conveying or unblinding the trial recommendations in writing. That's one of the meeting records but the other meeting records are transcripts or minutes of the DMC meeting, which are kept but usually are not widely available until the end of the process, until the trial is concluded.
Then there is the issue of who should have access to the treatment codes. Should the DMC review the comparative data? Some DMCs discuss this and choose to remain blinded until some later point in the interim analysis process when they choose to unblind themselves, but this is the kind of discussion that needs to go on at least within the context of each DMC: who should have access to these treatment codes and when should the treatment codes be identified?
There are arguments in favor of remaining blinded, that the recommendations with regard to termination or continuation are seen in a different light when it's known that the DMC is in favor of blinding and remaining blinded. Other emerging concerns are seen in a different light when they're known to remain blinded.
Then there are arguments against blinding, that the DMC, if anyone in the process should be knowledgeable about what treatment A versus treatment B means, it is the DMC. So this is the kind of issue that really at the moment remains up in the air for how the individual DMCs deal with this, whether they remain blinded from the beginning or they unblind themselves once they begin discussion of treatment A versus treatment B. That's the kind of thing that needs to be discussed in the development of the charter, of the SOPs, and how each DMC chooses to operate within itself.
The DMC reporting, as I mentioned earlier, needs to be a report to the sponsor, a face-to-face debriefing, but then a short report to the sponsor after each meeting. The minutes, as I've already described, they go into a lot more detail as to how the recommendations were arrived at and they are available only to the DMC during the conduct of the trial. Usually at the end of the trial those minutes and all the records involved in the process are made available to the sponsor and to the FDA at the completion of the trial.
So thank you very much.
DR. LEPAY: Mary, thank you very much. We're going to adjourn for lunch now and we'll resume again at 1:30, again continuing this particular section of the document, and then into our second panel. Thank you.
[Whereupon, at 12:04 p.m., the meeting adjourned for lunch.]
Afternoon Session
[1:32 p.m.]
DR. LEPAY: Okay, we're ready to resume for the afternoon to continue the discussion of the second group of sections of the guidance document. I'd like to open the afternoon session by introducing Dr. Jay Siegel, who's director of the Office of Therapeutics Research and Review in our Center for Biologics. Jay will be talking about a subject that I think we've hit on already on numerous occasions this morning but we'll certainly develop much more this afternoon and that is the independence of data monitoring committees.
DR. SIEGEL: Thank you, David. Well, based on this morning's discussion I anticipate that this topic should lead to a lot of lively discussion and valuable input and I very much look forward to that.
So let me start the next half hour or so by outlining what's in the document and also by providing some case studies or examples that are, in part, informative about why the document says what it does.
A lot of people, of course, talk about independence of a data monitoring committee and very few times is it well defined what one means by independence. When you write a document you sort of have to do that if you want people to understand the document.
So for the purpose of this document, at least, we start with a definition of what independence is and what we're addressing. No data monitoring committee is, in a true sense, fully independent by the sponsor. They're usually selected by the sponsor, paid by the sponsor, they make their recommendations through the sponsor, as some people have pointed out, but there are critical independence issues that are addressed in this guidance document.
So in Section 6 of the document at the very beginning on independence is this passage, which defines what we mean by independence. An independent data monitoring committee is a committee whose members are considered independent--good way to define it--of those sponsoring, organizing and conducting the trial. That is, they have no previous involvement in the design of the trial, are not involved in its conduct except through their role on the data monitoring committee, and have no financial or other important connections to the study sponsor or other trial organizers. And what we mean by important connections we have a little more detail on and that I'll come to in just a couple of slides.
So that's the working definition for this part of the document.
I would note that, as I said, we discuss both financial connections but we recognize that there are other types of connections that can compromise objectivity or create compromising situations, and I'll go into that in significantly more detail shortly.
The document then proceeds to discuss some of the typical relationships that a sponsor may establish in terms of their role on the DMC. At a time when they establish the DMC they'll define what their role is and that is a critical decision process with important implications.
There are two types of roles which are not consistent with the definition of independence, which is not to say that the document says that they're per se unacceptable; it just say that they're not independent, and it goes on to talk about the concerns or implications of that. Those are situations where the sponsor has a representative who is a voting member on the monitoring committee or where the sponsor has a representative as a nonvoting member on a monitoring committee but who is present at all sessions or, at the very least, at closed sessions, even if not executive sessions.
There are two other common conditions that are more consistent with the definition of independence where a sponsor representative is present only in the open meeting and they may well see enrollment, compliance and event rate data but no study on specific data, or situations where the sponsor has no direct representation on the data monitoring committee.
The document proceeds to discuss three reasons why independence of the data monitoring committee is a desirable trait. I noted that Janet Wittes this morning, in pointing out that we were blurring some distinctions of important issues, summarized these issues much more succinctly than we managed in the document when she said we were blurring issues of confidentiality, credibility and conflicts of interest. And indeed there are different implications for each of those and certain other factors that contribute to the desirability of independence, so we've tried to take them somewhat apart and address them somewhat separately of each other.
The first reason given is that independence ensures the ability of a monitoring committee to make recommendations on behalf of the subjects and the trial, their two principal responsibilities, that are not unduly influenced by the interests of the sponsor. That particular issue is addressed in a passage in Section 4.1 of the document, not in Section 6, which deals with independence per se, but in Section 1.4, which Mary alluded to briefly; that's the section on selecting a committee.
The second point, that complete blinding of the sponsor allows the sponsor to modify a trial or to take part in modifications of a trial without the introduction of bias. That's probably the issues that's the main focus of Section 6 and will be a substantial focus of the remainder of my presentation of Section 6.
And blinding also protects the sponsor from pressures toward premature disclosure. We've heard from CEOs of companies, for example, that if they learn the data and then attend shareholder meetings, get called by financial analysts, have to consider the lawyers telling them what they do or don't need to disclose to the Securities and Exchange Commission, that often they're put in rather compromising situations where there are pressures to do things that could endanger a trial.
Not explicitly on this list of reasons for independence but also addressed elsewhere in the document is the fact that keeping the DMC independent of investigators and sponsors decreases the likelihood that investigators, directly or through the sponsor, might become unblinded to the trial, which can impact recruitment practices, patient management practices, and so forth.
So in Section 4.1 is a passage on conflict of interest-type issues. It notes that data monitoring committee members should not have financial interests that could be substantially affected by the outcome of a trial, that they should not be investigators entering subjects into the trial. That reflects, as I just noted, not just conflicts of interest but also potential biasing impacts of unblinding.
They should not have strong views on the relative merits of the intervention and they should not have relationships with trial leaders that could be considered reasonable likely to affect their objectivity. This gets back to that issue in our definition of other important connections to the study sponsor.
We don't go into any detail on this issue. We recognize that the clinical trial community is a relatively small community, that members of the monitoring committee are, in fact, often people that may have important professional or other relationships with the people involved in managing the trial or conducting the trial. The critical issue, though, is to consider in these cases whether the nature of those relationships is such that they would be or would be viewed as being reasonably likely to affect objectivity.
Now there's a substantial value to a sponsor having certain types of involvement with a DMC, even an independent DMC, and that has already been discussed, I guess, in Mary's presentation regarding open sessions, and it's also discussed to some degree in Section 6.2 of the document.
These interaction can both facilitate the DMC's deliberations as well as facilitate drug development by the sponsor. And they may include sharing of information in both directions, and typically do, where the sponsor can inform a committee about what the sponsor's goals are, their plans for drug development, time lines, other trials, what indications they're seeking, how they feel about certain patient populations that are or are not in the study, dosing issues, and so forth, what resources they have committed to development of the product, what is and isn't feasible to do.
And conversely, by learning, the data monitoring committee can assist the sponsor in its role and the information in the open sessions can assist the sponsor in terms of discussion of issues with the trial regarding enrollment, compliance, event rates, and the like, that can be important determinants of cost, timetables, likelihood that the trial will successfully answer its questions, and so forth.
Section 6.3 of the document covers some of the risks that occur if a sponsor is exposed to interim comparative data, one of them being, as I alluded to before, the possible further unblinding of the trial so that investigators or participants in a trial, perhaps through a sponsor meeting with the steering committee and so forth, may learn directly or more indirectly about the data in the trial and that, of course, can affect various aspects of their role in dealing with the trial.
The other area which I've alluded to and will go into more detail on is, and also a number of examples shortly, is that the exposure to interim comparative data can significantly impact the ability of the sponsor and potentially others, as well, to manage a trial appropriately. And what we've seen over experience is that there are not infrequently, more commonly than anticipated by many, who would say you design a trial and you just stick with it to the end, there are not infrequently external factors that may suggest the need to change a trial. You learn something from other clinical studies of the same or related agents about what doses do, about what risks or adverse events are. You may have new financial resources or new financial constraints that may affect the way the trial can be conducted or should be conducted.
There can be internal factors to the trial, as well, problems, as I alluded to before, with compliance with the drug, with enrollment in the trial that may suggest a change in entry criteria or in the protocol that may be important for the success of the trial.
Knowledge of the interim data, when modifying the trials, may lead to unavoidable and uncorrectable biases. So if the sponsor and/or steering committee and other individuals involved in suggesting changes--changes to the analysis, changes to the entry criteria, changes to the protocol--are aware of results, unblinded results of the trial, they're likely aware of how that direct information as to whether changing that end point or entry criteria will increase or decrease the likelihood of success, that introduces biases to the trial.
Furthermore, these are not correctable biases in the sense that if you do multiple interim analyses you can apportion type 1 error to correct for that multiplicity to ensure that you don't have excessive type 1 error. When you biases that result from making decisions based on advanced knowledge, there is no statistical correction. You're just left with a trial result whose validity is called into question.
Section 6.4 is a section that has already received substantial discussion and I suspect will receive substantially more and I would like to take this opportunity to urge all of you to read that section, for starters, as there were some comments that indicated that the document didn't cover areas which it does or that it says things which it doesn't.
So please read that section and please comment on that section. We know there's a great deal of interest. We know that it's a very common practice in all settings for statisticians as well as data coordinating centers that are unblinded to the trial to also be interacting with and preparing data for data monitoring committees and also be interacting in various ways with the sponsor of the trial.
That topic is addressed in this section. The section doesn't say don't do that or you can't do that but it does warn rather explicitly about some of the potential that has occurred in some cases to seriously impair the ability to manage the trial, to modify the trial, or to render a trial uninterpretable when certain types of relationships like that exist and we feel that it's very important that in deciding on the relationship and role of the statistician and coordinating center and the communication links, that these issues be taken into account.
So the sponsor statistician frequently is the one who sees and prepares the interim data, interim data reports, and often, as well, presents them to the data monitoring committee. Experience has shown that separation of these statisticians from trial management may be difficult to effect or to demonstrate. It may be easier than we think but certainly in recent experience it hasn't always been accomplished to the extent one would hope.
So we find statisticians meeting with the trial team in the company; they're part of the project for that drug. We find these unblinded statisticians reviewing protocol and analysis amendments or sitting in those meetings even if not giving verbal communications, potentially giving informal or nonverbal communications and we tried in this section to explain what sorts of concerns arise from that--the notion that if a company or sponsor--it doesn't have to be a company; it could be a governmental institute--is considering a modification that impacts spending of millions of dollars and the statistician is there knowing potentially that the modification is futile, unnecessary, going to turn the trial into a failure, you know, and everybody knows that the statistician knows and he's just sitting there in the room not saying anything, that's a difficult situation and a difficult situation which really, I think, runs the risk of transformation of information, even nonverbally or verbally.
In other settings where maybe a corporate management is responsible for making those decisions there may be further pressures.
I think even where those pressures don't exist one of the concerns and one of the concerns we've raised is simply it's hard to participate in a decision knowing information and not letting that information contribute to the decision and it's hard to be present as a decision is being discussed or made and not be totally nonparticipatory. Those issues are addressed in Section 6.4.
One issue you used to hear discussed a lot at meetings and I guess still is sometimes on data monitoring committees and on interim analysis is the notion that was sometimes referred to as administrative looks, although I don't think we've used that term in this document. But the sponsor does frequently desire access to interim data for what are legitimate business purposes. They may want to know that they should upscale production, they need to plan another trial, they can get the drug to market perhaps a year earlier if they have an educated guess as to whether or not the trial is likely to be successful than if they don't.
However, there are some significant problems with these sorts of looks at the data. As I've just pointed out, they may impair the ability to manage a trial. They may make the results uninterpretable due to bias. And although not mentioned in this section although discussed elsewhere, they may lead to further unblinding of the trial. So presumably if the sponsor sees the interim data and then starts building a new plant, that might well tip somebody off that there's a problem.
In addition to cautioning about reasons to consider not doing this in the first place, the document does provide some substantial guidance based on experience in terms of cautions that could be taken if a sponsor does choose to access interim data.
First, to consider discussing the issue with the FDA in advance. Think about the implications. Think about how to do it.
Second is that there should be a prospective stopping rule in a type 1 error allocation. We reject the notion that you can look at the data and have no chance of stopping the trial and therefore don't need to allocate any type 1 error. We believe that from an ethical perspective any time you look at the unblinded data you might see something that leads you to believe the trial should be stopped, that even if you assign a very low type 1 error if you think it's improbable, it's much better to do that prospectively than retrospectively.