FOOD AND DRUG ADMINISTRATION

	CENTER FOR DRUG EVALUATION AND RESEARCH

	MEETING OF THE

	CARDIOVASCULAR AND RENAL DRUGS ADVISORY COMMITTEE

	8:37 a.m.

	Friday, October 20, 2000

	Jack Masur Auditorium
	Clinical Center, Building 10
	National Institutes of Health
	900 Rockville Pike
	Bethesda, Maryland

	ATTENDEES

COMMITTEE MEMBERS:

MILTON PACKER, M.D., Chair
Chief, Division of Circulatory Physiology
Columbia University
College of Physicians and Surgeons
630 West 168th Street
New York, New York  10032

JOAN C. STANDAERT, Executive Secretary
Center for Drug Evaluation and Research 
Food and Drug Administration
234 Summit Street, Room 117
Toledo, Ohio  43604

PAUL ARMSTRONG, M.D.
Professor, Department of Medicine
University of Alberta
251 Medical Science Building
Edmonton, Alberta, Canada  T6C#7

MICHAEL F. ARTMAN, M.D.
Professor of Pediatrics
Pediatric Cardiology
New York University Medical Center
530 First Avenue, FPO Suite 9-V
New York, New York  10016

JEFFREY BORER, M.D.
Director, Division of Pathophysiology
Weill Medical College at Cornell University
525 East 68th Street, Room F467
New York, New York  10021

THOMAS FLEMING, PH.D.
Professor and Chair
Department of Biostatistics
University of Washington
Box 357232
Seattle, Washington 98195-7232

JOANN LINDENFELD, M.D.
Professor of Medicine, Division of Cardiology
University of Colorado Health Science Center
4200 East Ninth Avenue, B-130
Denver, Colorado  80262

	ATTENDEES (Continued)

COMMITTEE MEMBERS:  (Continued)

ILEANA PINA, M.D.
Director Congestive Heart Failure
  and Transplant Programs 
Case Western Reserve University
University Hospital of Cleveland
11100 Euclid Avenue
Cleveland, Ohio  44106-5062

FDA CONSULTANTS:  

GEORGE BAKRIS, M.D.
Department of Preventive Medicine
Rush Medical University
St. Lukes Hospital, Suite 470
1700 West Van Buren Street
Chicago, Illinois  60612

MARVIN KONSTAM, M.D.
Professor of Medicine
New England Medical Center
750 Washington Street, Box 108
Boston, Massachusetts  02111

LEWIS SHEINER, M.D.
University of California at San Francisco
521 Parnassus Street
San Francisco, California  64143

FDA GUESTS:

CARL PECK, M.D.
Center for Drug Development Science
Georgetown University
Med-Dent Building, Room NE 403
3900 Reservoir Road
Washington, D.C.  20007

	ATTENDEES (Continued)

FDA GUESTS:  (Continued)

DONALD B. RUBIN, PH.D.
Professor, Department of Statistics 
Harvard University
1 Oxford Street
Cambridge, Massachusetts  02138

FOOD AND DRUG ADMINISTRATION STAFF:

RAYMOND LIPICKY, M.D.
Division of Cardio-Renal Drug Products
Center for Drug Evaluation and Research 

	C O N T E N T S

AGENDA ITEM	PAGE

CONFLICT OF INTEREST STATEMENT
	by Ms. Joan Standaert       	6

INTRODUCTION
	by Dr. Raymond Lipicky	8

DOES THE CURRENT DEVELOPMENT FIND THE RIGHT DOSE?
	by Dr. Carl Peck	10

COMMITTEE DISCUSSION 	20

ANTIHYPERTENSIVE DRUG DEVELOPMENT, AS AN EXAMPLE
	by Dr. Raymond Lipicky	34

COMMITTEE DISCUSSION 	49

DOSE FINDING - WHAT DO WE WANT TO KNOW?
	by Dr. Lewis Sheiner	102

COMMITTEE DISCUSSION 	130

HOW CAN WE KNOW WHAT WE WANT TO KNOW?
	by Dr. Donald Rubin	149

COMMITTEE DISCUSSION 	159

	P R O C E E D I N G S
	(8:37 a.m.)
		DR. PACKER:  This is the 91st meeting of the Cardiovascular and Renal Drug Products Advisory Committee.  Today's topic is a discussion on the need and mechanisms of defining dose-response relationships in the evaluation of drugs for the treatment of hypertension.  
		The nature of today's discussion does not deal with a specific product, although there will be examples of data that has been collected from earlier development programs, although the drugs involved in these earlier development programs will not be specifically identified.  So, there is no approval or nonapproval that is required from the committee in terms of a recommendation.  
		That is being sought from the committee today is a sense as to what direction this area should be moving towards, and we will have a very good discussion about the principles involved, the issues involved, and we will try to address many, perhaps most of the questions which have been posed to us in the time that is allotted to us. 
		I will have Joan read the conflict of interest and the administrative matters for today's meeting. 
		MS. STANDAERT:  Thank you. 
		The following announcement addresses the issue of conflict of interest with regard to this meeting and is made a part of the record to preclude even the appearance of such at this meeting.  
		In accordance with U.S.C. 208(b), general matters waivers have been granted to all committee participants who have interests in companies or organizations which could be affected by the committee's discussion of dose-response.  Copies of these waiver statements may be obtained by submitting a written request to the agency's Freedom of Information Office, room 12A-30, Parklawn Building. 
		With respect to FDA's invited guests, Dr. Donald Rubin has reported interests which we believe should be made public to allow the participants to objectively evaluate his comments.  Dr. Rubin would like to disclose that he has minor holdings in various drug companies.  He consults for Amgen, Pfizer, Merck, and Roche.  He also speaks for Pfizer, Merck, and Roche, and serves as a scientific advisor to Pharsight.
		In the event that the discussions involve any other products or firms not already on the agenda for which an FDA participant has a financial interest, the participants are aware of the need to exclude themselves from such involvement, and their exclusion will be noted for the record.  
		With respect to all other participants, we ask in the interest of fairness that they address any current or previous financial involvement with any firm whose products they may wish to comment upon. 
		This is part of the public record on the 20th of October 2000.
		DR. PACKER:  The committee needs to receive a charge and an introduction about the events that will unfold today, and I'll ask Ray to introduce today's topic and describe to us what is going to happen and what he expects from the committee. 
		DR. LIPICKY:  I think I can do it from sitting here, but I will need to see the slides that start at 60.  If you can show the first slide please, and then I'll ask you to advance the slide because the podium is occupied. 
		We have three individuals who have agreed to come to a rather loosely structured meeting that has no singular objective, but rather to get involved in a discussion of the need for dose-response and how one might go about evaluating dose-response.  I thank them for that because they've taken time to come as far as from California with really no real agenda and no real purpose except to get involved in some intellectual discussion about stuff that we're not sure anybody is interested in.
		So, that is the purpose of the meeting, and if we can see the next slide.  The reason that it's the purpose of the meeting is because when one writes a prescription, there are really only two parts.  You have to decide what drug you want to use and ordinarily the committee is involved in deciding whether a drug can be used or not.  But once that decision is made, the only other decision left is the dose and/or dosing interval.  I hope that we will be able to convince the committee that current developing programs don't define that and that if one thinks they do, they don't know what they're talking about.  And there has to be something new and different thought about and done in order to handle that properly.
		So, we'll be talking about development programs, and in particular, we'll be talking about, I hope, what properties of the drug with respect to dose ought to be defined in a development program.  I don't think that's very clear at the moment, and maybe by the end of the day, it will be a little clearer. 
		The notion is to evolve a framework of reference.  I know my framework of reference.  Each individual up here knows their framework of reference.  How do you approach this?  How do you look at it?  What do you want to get?  What do you want to know?  How should you go about getting it?   And the question is whether anybody's framework of reference is the same.  We want to explore that and find out whether what is where and what is how.
		I think that there are no rules.  There's an agenda, but I'm not sure we'll stick to it.  There are questions.  I'm not sure they'll get asked.  But I think the notion is that whatever interaction seems to be necessary ought to occur, and if anybody who is talking says something stupid, they ought to be called on it.  Otherwise, the framework of references won't evolve properly.
		And that is the purpose of the meeting. 
		DR. PACKER:  Does anyone on the committee have any questions about what this is all about?  If you do, you'll find out shortly.  
		We'll move forward to the first presentation, which is by Carl Peck.  It's going to focus on does the current development find the right dose.
		DR. PECK:  Thank you, Dr. Packer and Dr. Ray Lipicky, for inviting me to present my views about this important subject today. 
		I've been challenged to lead off with a brief discussion concerning the question, does current drug development find the right dose?  I'm going to answer that question in the end, but first I'm going to pose five sort of leading questions that I think provide a background for being able to provide an answer.  These questions will be:  What do we mean by "right" dose?  Is there regulatory encouragement for finding the right dose?  How good are approved doses?  How good are dose-response studies?  What might be the consequences of getting the dose wrong?  And after I've answered those questions, I'll then provide my view of how well we're doing. 
		What is the right dose?  Well, my personal view -- and this reflects practice of medicine many years ago and several decades of trying to get the right dose for others -- is that right dose will be a range, not a single dose, but a range of safe and effective doses which is accompanied by information for achieving benefit in individual patients.  So, the emphasis here is on a range of doses and on individualization of the dosage.
		That leads to the desirable features of an adequate drug label.  In my view, there ought to be quality dose-response information displayed prominently in the label so that the practitioner can understand the linkage between dose and effect and the covariates that affect that.
		There should be specific instructions relating to a safe starting dose that are keyed to individual patient characteristics when appropriate, such as age, gender, body size, ethnicity, concurrent medications, concurrent medical conditions, and severity of the disease being treated. 
		After the dose is started and monitoring has occurred, then there ought to be some advice as to how to adjust the dosage upwards, in the case of inadequate effectiveness, or downwards in the case of an adverse reaction. 
		Finally, in italics I've got that labeled dosage should remain stable.  In other words, there shouldn't be a dosage recommendation that's changing over time.  I'll get back to that because this comes closest to what we've discovered as a method for evaluating the adequacy of current approved dosages. 
		The second question is, is there regulatory encouragement for getting the dose right?  Well, in fact, there's at least a century of regulatory encouragement that began at the beginning of the last century with requirements for truthful labeling and, more particularly, in the mid part of the century, the emphasis on safety; in the 1960s, the emphasis on effectiveness; and during the last several decades, a move toward emphasis on individualization.  Perhaps in the future, there will be an even greater step when genomic information becomes importantly available for personalization. 
		But this is just not a theoretical concept.  There are literally tens of thousands of pages of regulatory guidance in this country and in Europe that give instruction and counsel on how to get the dose right. 
		FDA realized in the late 1970s that there were severe imperfections in dose finding in drug development, and that led Bob Temple and Ray and others in the early 1980s to begin to publicly discuss the matter and to move toward written regulatory guidance.  I'm citing here a DIA journal article that Bob wrote in 1982 and a chapter that he wrote in 1989, which are rich in information advising on the qualities of titration in parallel dose-response trials that, surprisingly, don't seem to be well-recognized even today. 
		There's a multitude of FDA guidelines.  The ones that I would cite here that are particularly applicable are the guidelines for format and content of the clinical statistical sections of the new drug applications published in 1988 and the geriatric guidelines published a couple of years before, specific information on dose-response trials.  And several International Committee on Harmonization guidelines that basically reflect the FDA guidelines, the geriatrics guidelines, general considerations for clinical trials, ethnic factors, and one specifically entitled dose-response information to support drug registration. 
		In my view, any serious practitioner of drug development science ought to have read this document, the dose-response document, and know it really well, because it contains contemporary knowledge on this subject. 
		Here's the way that document starts out.  It says, the purpose of dose-response information is to express knowledge of the relationships among dose concentration and clinical response, the importance being that it leads to safe and effective use of drugs in individual patients.  It goes on to say that this can help to identify the appropriate starting dose, the best way to adjust the dose to the needs of the individual patient, and increases or decreases as needed.
		Now, the next question I wanted to consider was how good are the first approved doses.  Remember I said that a desirable feature of a drug label would be that it doesn't change over time.  Well, here you're going to see some real data, not just hortatory comments. 
		We've been undertaking a study in our Center for Drug Development Science at Georgetown University of drug doses as first approved.  The principal investigator on this is sitting in the audience, Mr. Jamie Cross.  He's been assisted with this study by several other post-doctoral fellows and faculty and staff of our center. 
		The purpose of this study was to evaluate all 499 approved new molecular entities that were approved from January 1980 to December 1999.  These are U.S. approvals.  The study has a very simple design.  It seeks to capture the dosage recommendations for the first approved indication in the first approved label and to compare that dosage recommendation with the current recommendation in the approved label.  Not so simply done, however.  Of the 499 drugs, we've only been able to find actually both labels for 354 evaluable drugs.  We've used every possible source of information I believe known.  We've gone to the Physicians' Desk Reference.  We've gone to FDA.  We've gone to multiple Internet sources.  Some of these simply fade away.  Ray and I were discussing and confirming that between us this morning. 
		Nevertheless, we've got over a 70 percent representation here.  What we've found is that 1 in 5 of the approved dosages during this two-decade period have undergone a dosing change.  That's 79 out of 354, 80 percent of which were dosage reductions.  So, there's a systematic error in dose finding, it appears, that's leading to a dosage reduction.  We concluded from this cohort as a whole that premarketing drug development is improvable with regards to safe doses. 
		Now, that might not interest you as much as antihypertensive drugs, which is in fact the focus of this particular committee's work at present.  During this period of time, there were at least 34 new antihypertensive drugs approved.  This is probably a pretty accurate figure.  There were at least 9 label dosing changes, which is about 1 in 4.  8 of those resulted in dosage reductions of 33 to 50 percent, or a contraindication from a previously approved cohort of subjects, or a full-market withdrawal because any dosage was considered to be unsafe.
		This is the list of those 9.  The panel on your left contains those labels that show a change in dosage of the starting dose.  You can see, surprisingly, there's one in which the dosage has been increased.  That is, there's been an expansion of the dosage allowed for the first dosage.  But the other 5 have had a 30 to 50 percent decrease, two of which Ray tells me -- these are the asterisks, captopril and indapamide -- were a result of competent dose-response trials done following marketing approval.  
		Now, the other 3 you might find arguably included here.  Maybe these are oranges rather than apples.  Mibefradil you know only too well was completely withdrawn from the market because of its liability in certain drug-drug interactions, especially for lipid-lowering agents, and lisinopril and enalopril have had a specific contraindication announced for subjects that have angioedema with ACE inhibitors.  Even if we take these out, this still results in a 17 percent of new antihypertensives that have undergone a dosing reduction. 
		So, in summary, for antihypertensive drugs, 26 percent underwent a dosing change, which is about the same as the 22 percent for all drugs.  90 percent were reductions, which is comparable to the 81 among all drugs.  So, I would propose that antihypertensive drug development is no better than or no worse than drugs in general.  So, there's perhaps a systematic error in dose finding across the whole of drug development. 
		Now, the question is how good are dose-response trials.  Let me say that I believe that there were few proper dose-response trials done before the decade of the 1980s.  I think that's what motivated Bob Temple's articles and the plethora of regulatory guidance. 
		In our center, we study contemporary drug development practices, and to date we've had an experience in over 160 IND programs with over 60 companies.  So, we get a peak into how many clinical trials are being done, what the nature of those trials are, and here's a snapshot.
		Current NDAs continue to employ very large numbers of clinical trials, typically 25 to 100.  It's phase I and phase II and phase III.  We're seeing a remarkable fraction of phase IIb -- that's the dose finding in phase II trials -- are randomized, blinded, controlled, dose-response trials, typically parallel dose-response trials.  What's amazing to us is that most of them are analyzed as hypothesis-testing trials using only comparisons of each assigned dose against placebo or sometimes against each other.  Few studies are thoroughly analyzed for the shape and location of the dose-response relationship or for covariate influences even though the studies are designed in such a way as to make that an appropriate statistical assessment. 
		A few years ago, Hans Melander of the Swedish Medical Products Agency presented a study undertaken in his own statistical section of dose-response trials submitted to their agency in the early 1990s.  They documented 46 dose-response trials that were submitted during this time period.  20 of the 36 used only pair-wise comparisons in the analysis of the data.  19 of those 20 used unadjusted multiple comparisons in arriving at their p values.  Almost all of them were overpowered for assessing the significance of a dose response, a linear dose response, and only 10 of them actually had any kind of a dose-response analysis or regression analysis undertaken to evaluate the shape and location of the dose response. 
		Informally, I've heard from colleagues at FDA that the situation is very little different here, and we certainly see this in the companies that we work with, a very inadequate analysis of dose-response trial data. 
		The next question, what will be the consequences of poor selection of dose for the label?  Well, clearly patients who get the wrong dose, especially if it's high and if 80 percent of that quarter that undergo a dosing change are high, then there's injury.  Moreover there may be suboptimal effectiveness.  Some patients may be denied the opportunity to benefit from the drug if they're withdrawn from the drug because the dose was too high.  
		Manufacturers encounter liability.  
		Erosion of pricing, and I would think at this moment in time, this should be particularly important to manufacturers since pricing is often based on the first marketed dosage, and if you must reduce the dosage, you can take the bottom out of your profit margin.  
		Increased costs by having multiple dose-response trials that are not fully exploited for the subsequent trials, and then, of course, the post-approval costs. 
		Regulatory agencies can be subject to criticism and loss of public confidence with drug withdrawals or drug dosage changes. 
		So, the question I was asked to answer, does current drug development find the right dose?  Well, you could say most of the time, although let me say that the study that Jamie Cross and the rest of us are undertaking is not clear on that yet.  This 20 to 25 percent dosing change rate is for the whole two decades, but we're doing some Cox regression analysis on these data using epoch or decade as a covariate.  And we're discovering that the rate of dose changes is higher in the decade of the 1990s than in the decade of the 1980s.  That doesn't mean necessarily that the effort is any worse, but we're picking up dosing imperfections and making label changes seemingly more frequently. 
		Dose changes that are too high can harm patients.  There's no doubt about that. 
		The rate of dose approval changes, 20 to 25 percent, including antihypertensive drugs, can be improved, and it's the purpose of this day to discuss some approaches for doing that. 
		Finally, regulatory guidance is ubiquitous, but for some reason these trials continue to be inadequately analyzed and that information is inadequately exploited.
		Thank you very much. 
		DR. PACKER:  Does anyone on the committee have questions for Dr. Peck?
		I have a question.  The definition you use in your study of determining whether the right dose was found is a ratio of a denominator over a numerator where the denominator is all of the new chemical entities, for example, approved for the treatment of hypertension and the numerator is all of the drugs that have undergone labeling changes.  
		Excluding the three drugs for which a dosing change was not really part of what happened -- it really was withdrawal or some other major event -- you have six drugs that underwent labeling changes.  Were the labeling changes, in terms of dose, for the general population or for subpopulations? 
		DR. PECK:  Those five -- 
		DR. PACKER:  It's six I think.  I think it was six and three. 
		DR. PECK:  Six.  One was an increase, yes.
		That was in the first dose section of the label, and that would apply to all patients.  I think that two of them actually referenced patients with renal dysfunction and so there was significant dosing adjustments for renal dysfunction.  So, it was, in effect, a discovery postmarketing of a significant covariate that had not been taken into account.  So, in my view that's part of the dose-response issue, identifying the right dose for the right subpopulation. 
		DR. PACKER:  The reason for asking the question is it was also my impression, by just looking at the examples and relying on memory, that many if not most of the six examples or five examples were dosing adjustments in sub populations as opposed to an overall population, which now addresses another layer of complexity.  It's not dose response for the general population; it's dose response for an increasing number of populations of interest.  So, it's not only patients with underlying comorbid disorders, it's not just demographic distinctions amongst populations, and that increases the number of complexities.  And frequently the development program for dose response is not a development program for dose response in subpopulations.  It's for the general population.  So, we wouldn't be too surprised to see that there may be tweaking of doses in subpopulations. 
		So, a more interesting or perhaps more targeted question is, how many times were there adjustments in the general population in dose response.  Just looking at this, my sense is that it only applied to maybe two of the examples.  That would make the denominator now quite small.
		DR. PECK:  Well, you're more forgiving than I am.  Actually the data is that of the six dosing changes, only two were in the subpopulation of patients with renal dysfunction.  The other four were for all comers. 
		However, this particular subpopulation, patients with renal dysfunction, we've known for three decades is a special population, that any drug that has any significant excretion to the kidney should be taken care of.  So, it's not so much, I think, a layer of complexity.  It's routine clinical pharmacology.  It's what most development programs discover in the phase I and, for some reason, forget when they get to phase II and phase III.  So, I think it's a matter of awareness and interpretation of this knowledge rather than anticipating an increase complexity.
		DR. PACKER:  Okay.  Lew, do you want to start, in terms of the discussion? 
		DR. SHEINER:  I just wanted to make the point that I think this study is a useful study, but I think we have to regard it as sort of the tip of the iceberg in the following sense, than an official label change of dosage is unlikely to occur unless there's a safety issue.  So, a drug, if it's essentially nontoxic, that comes out at a dose that's three times larger than you need and then gets lowered and winds up having economic consequences for the manufacturer and in some sense we'd like to use the least dose that will do the job because there's a general bias against giving excessive doses, but I don't know that they would show up as labeling changes. 
		DR. PACKER:  Ray?
		DR. LIPICKY:  Well, the other thing is I think the idea that renal problems or deficiencies in eliminating the drug are well known, and sometimes at the time of writing the initial label, one forgets to put that in.  I guess there are people in the audience who might know the answer for the ones where there was dosage reduction.  I don't know if that came from studies, adverse events, or, geez, we just forgot to put that in, we ought to.  Does anybody recall?
		DR. PACKER:  I'm not certain that we can poll the audience.
		DR. LIPICKY:  Well, no.  The audience could too because probably somebody knows the answer to that. 
		DR. PACKER:  The one thing that strikes me as being interesting about the renally impaired subpopulation is the fact that this committee, to my knowledge, although we've seen a fair number of dose-response studies, we have never seen a dose-response study in patients who have renal disease. 
		DR. LIPICKY:  Right, but you do see biopharmaceutics, pharmacokinetic studies in people with renal disease, and every question you are asked asks you to think will this influence the dose response, but you never do.
		DR. PACKER:  But what we see are studies in an extremely small number of patients with renal disease, usually like 20 or 30.  George is saying, well, even less than that, but that's okay.  Usually all the patients who have renal disease are lumped together as if they were one kind of patient, and some formula based on creatinine clearance or something else is determined that allows for "dosing adjustment."  We don't get dose-response information.  We just get crude reductions in dose based on reductions in renal clearance of a drug.  My sense is that that has been "satisfactory" up to now.  Is that wrong?
		DR. LIPICKY:  Well, I think we are here to ask you whether that's wrong.  You have described the practice properly.
		DR. PACKER:  George?
		DR. BAKRIS:  Since Milton threw the question down, Ray, I'll answer it.  I would agree with Milton a thousand-fold and say that, in fact, the papers that I've reviewed for clinical pharmacology journals do exactly what you said, Ray.  They look at pharmacokinetics and they look at mathematical models and they look at n's -- I thought Milton was being very generous talking about 30 and 40 patients.  I'm used to seeing single digits in each arm.  That's totally inappropriate.  
		Frankly, the renal insufficiency seen in diabetes is totally different than the renal insufficiency seen in heart failure in the sense of the mechanisms may be but the duration and natural history is different.  Certain drugs, for example, ACE inhibitors, to my knowledge, that are renally cleared may accumulate more, but I'm unaware that there's any substantial side effect profile seen.  Other drugs clearly do have that, and certainly in the transplant setting, that's important.  
		So, I think Milton is right to say this is extremely complicated, and I think the whole focus on patients with renal insufficiency needs to be looked at as a separate entity practically or at least a parallel entity to the general population rather than doing these certainly inadequately powered window-dressing type studies that are put in there just to give you some pharmacokinetic data because the pharmacodynamics is absolutely important in those people longer term. 
		DR. LIPICKY:  Well, there is an important distinction to make and I think we need to have that out in the beginning.  Clearly defining dose response for anything, whether it's an antihypertensive or anything else, might be able to be done in a fairly efficient way so that for the molecule and for some patient population, one knows what's going on. 
		Clearly the business of what is the "clinically relevant" outcome isn't known from those studies.  So, if the notion is that one needs to know the clinically relevant outcome with respect to dose, we will talk about that, hopefully, today somewhere late on.  
		But for the moment -- and maybe that's what the complaint is for the renal problems -- I'd like to keep the discussion focused around how you tell dose response, because with the clear recognition that that tells you one very small part of what you really want to know.  The question is, do you want dose response in every subpopulation?  If that is the issue that's in front of us.
		DR. PACKER:  Jeff?
		DR. BORER:  I'd like to ask what information you have, Carl, about the problem or situation that you touched on but didn't give us any data about, probably because there aren't any, and that is the other end of the spectrum, that is, labels that are written for doses that actually are less than, could be safely given, at least to some people, with increasing benefit.  
		You noted that one of the drugs on your list actually had a labeling change to increase the dose.  My perception is that there are other drugs -- I won't say many -- for which the dose that's commonly used probably exceeds the label as it was first written.  I'm thinking, for example, of diltiazem particularly, but there are others.  That may not seem like so much of a problem, but it sort of is because there are potential benefits that aren't known and there are patients who don't benefit optimally, though they're taking a drug, because the dose-response curve hasn't been explored to the high level.  
		The usual argument seems to be, well, we know there are side effects up there, so won't be marketing at that level because if 50 percent of the people have side effects, they won't take it.  Or we have to study too many people at too many doses to be able to determine safety at the high level, which is different from determining the dose-response curve. 
		So, I wonder if you have any information about how many drugs -- this may be impossible to determine -- are being used commonly at doses higher than the label because of experience plus published literature.  The label may not have been changed, so it wouldn't show up on your chart, but it would still be the wrong dose and it would be unfortunate that that is the case.  I believe it's the case.  What do you think about that? 
		DR. PECK:  Our study would capture a label dose increase.
		DR. BORER:  No, no.  I'm saying you would only capture a label dose increase.  
		DR. PECK:  I'm affirming your comment.  It would not be able to capture the practice of deviations from the labeled dosage.  I'm sure you're correct, that there are wide deviations from the labeled dose that are being practiced. 
		DR. BORER:  Well, my question really is, is there something we should be doing about that in terms of guidance or requirements or whatever so that the upper levels of the dose-response curve are explored more fully, even just to provide information, even if the company isn't going to market at that level?
		DR. PACKER:  I'd like the committee to, in general, until the next discussion period, hold questions about potential discordances between what a drug is labeled for and how it is used by practitioners.  I think that's an important issue, but it is more appropriately handled in the next discussion period rather than this discussion period.  So, Jeff, hold that thought.  There are many related thoughts to that.
		DR. BORER:  It's held. 
		DR. PACKER:  We'll bring it up.
		Marv and then Tom.  Carl, I'm sorry.  You wanted to answer the question? 
		DR. PECK:  I have a comment to make but I'll take my turn.
		DR. PACKER:  Okay.  Then Marv, Tom, Carl.
		DR. KONSTAM:  I'd like to understand how much of the problem is a function of population dose response and inadequate exploration of population dose response and, therefore, fixable with full exploration of population dose response, and how much of the problem might be related to distribution of responses within the population.  I don't necessarily mean definable subpopulations.  
		But actually in one of your constructs, you talked about individualization and maybe you could expand on that because dose response with regard to hypertension was a really good example.  Dose response of a population with regard to a certain millimeter mercury change in blood pressure in no way really describes what's really going on from patient to patient.  There may be huge variations in response within an individual.  I wonder if that kind of information is being fully captured in NDAs, and is it important? 
		DR. PECK:  My view is that the data is being collected.  Contemporary clinical trials are extremely burdened with data.  The analyses are pitifully meager relative to the data that's collected.  The typical phase III trial is viewed as an intention-to-treat hypothesis test with literally thousands of data items collected per patient but only a very small fraction actually used in the statistical test of interest.  Not that that's not unimportant.  Confirmation of effectiveness and documentation of safety is important, but there's a gold mine of relationships that is typically unexplored.  We'll hear a lot about that as the day proceeds.  It's subject to issues of causality and randomization and so forth.  But nevertheless, your basic point I think that I'm supporting is that there's a huge amount of data available for evaluation that is simply left alone. 
		DR. SHEINER:  Let me just make a quick response to that, as I'll define later.  I think we should, in fact, take the point of view that you're taking.  That's the entire distribution of responses to doses that we're after, not just the mean.  And how we might get that is a more technical issue, and Carl is saying the data might already exist or we might have to change what we do.  But there's no question that the distribution is what's of interest, not just the center.
		DR. PACKER:  Tom?
		DR. FLEMING:  Carl, I'd like to maybe expand on your opening summary here in terms of maybe defining what the problem is and defining what the strategies or approaches would be.  Some of these elements you have already identified.  
		It seems to me the challenge is quite broad.  Certainly part of that challenge is finding the right dose.  As you pointed out, though, it might actually be a reformulated, though, as what is the right dose concentration in blood.  There's also the question of what's the right schedule.  What's the right route of administration?  Within schedule, what is the right frequency or what's the right duration of administration of an intervention?  Related to the drug concentration in the blood, it might actually be do you dose to a given achieved normalization of a marker like hematocrit. 
		There is a huge array of specific challenges that I've seen in various settings, and often when we think what's the right dose, we're really just looking at one element of a huge array that define the options that one could use in formulating a regimen.  So, I'd like to just expand on what you've already articulated to say that the challenge is enormous in coming up with the right specific regimen formulation. 
		In terms of the solution, certainly the strategies that are in place right now are attacking this problem, obviously as you would point out, not fully efficiently at an array of stages, at the preclinical stage in phase I and IIa with PK and early safety and biologic activity measures and in phase IIb and in phase III.  So, the question is, as I see it, ultimately as you're attacking this very broad problem of identifying the right regimen, there is the whole myriad of the drug development phases that could be approached in terms of defining how to best do this. 
		Very much too, it becomes an issue of confirmatory versus exploratory which was also just pointed out.  I think what we do frequently in the phase III trial is we do try to focus, if at all possible, on an intention-to-treat or hypothesis testing stage.  At least in my experience, the myriad of issues that have to be addressed in formulating the right regimen are, as much as possible, relegated to the stages that precede the confirmatory step.  Right or wrong.  But that's in fact the way I've seen the strategy approached. 
		So, ultimately today it seems to me that we ought to be looking at the totality of this issue of finding the right regimen and then also recognizing that the strategies that we use ought not be focused just on what we do in phase IIb or phase III but in all of the phases. 
		DR. PACKER:  I think that's true. 
		Carl, I know you wanted to add something, but my sense is that all of the points that have been raised thus far are just the first step in a much greater process that will unfold quite quickly.  With everyone's indulgence, what I would like to do is for everyone to hold their thoughts and go on to the next presentation when these ideas will be developed more fully.  We will have more specific examples of issues we can focus on going forward. 
		So, Ray?
		DR. LIPICKY:  Thanks. 
		Can I have the first slide please?  I guess what I'm going to try to do -- and I don't know whether I will do it very well is to say that our guidance in this area, whether it be explicit in words or implicit in written or spoken, what we're doing is giving bad guidance.  The question is whether in the final discussions today you'll agree with that.  
		In particular, for hypertension, we focused on it because it's an easy thing to measure.  It's a continuous variable.  You can play games all you want, and it's not an irreversible thing that one is dealing with.  But I think the principles that apply here are the same as you would apply in any other field for any other purpose.  So, it is focused on hypertension, but it isn't meant to be exclusive to hypertension.
		There is no such thing, I will assert, as a phase III trial for antihypertensive drugs.  Antihypertensive drugs get approved for dose-ranging trials.  That's classically thought of as phase II and there is no morbid/mortal trial that is required to get an antihypertensive drug approved.  
		So, I think the thinking process should change from what do we do in phase I, in phase II, phase III, to what are we trying to accomplish in this development program.  Indeed, one of the major things is to define dose response.  That indeed is one parameter.  Maybe then there should be a phase III in antihypertensives, but at the moment, the whole development program devotes itself to finding a dose. 
		So, what we did, with the help of the Office of Clinical Pharmacology and the two individuals that are named there, is embarked on looking at what we have gotten in the way of dose response from our trials.  What we generally recommend is parallel fixed dose, placebo-controlled trials.  There are 15 million other trials that could be done conceptually designed, but this is generally what we do. 
		If I show a slide, it is X, 3X, 10X, 30X, and usually everybody laughs.  They say, yes, X, 50 percent more, 50 percent more, and 50 percent more.  Basically they go from X to about 1.75X in the three arms, and that is a trial.  So, then they do another and they do another and then they do another. 
		A successful development program looks like this where basically there isn't any question about what's going on, and there is an effect here and these data points out here because these are called phase III even though they're dose response, are indeed statistically significant from placebo.  So, basically the drug clearly works. 
		But the limitations of the drug as a clinical tool are very obvious:  the side effects of the dose-limiting problem.  Indeed, this point is statistically significantly different from that point.  
		But the thing that is convincing here is that the shape tells you the whole thing.  The question is, is this point different from placebo or is this point different from placebo?  The thing you get here is that this is the limiting factor and that there is a relationship to what you want in relationship to dose.  I'll assert that, given a figure like this, I can make all the decisions I need to without a p value.  
		So, in fact, it leads to doses that say the usual dose is 5, the maximum dose is 10.  The adverse reactions list adverse reactions for 2.5, 5, and 10.  But since 15 milligrams is not part of the dosing information, the 15 milligrams adverse reactions are absent.  In fact, that data point is the one that really convinces you something is going on.  So, the most important part of this development program is left out of labeling.  Our choice.  We do that and I agree to doing that. 
		Then you get a thing that looks like this, where there's dose here and decrease in blood pressure.  All of the data you'll been seeing is parallel placebo-controlled, fixed dose stuff where the placebo arm's mean is subtracted from dose 1's mean, and then it's the delta of deltas.  So, that's the data that is shown in all of this stuff. 
		Clearly there's a dose-response curve here.  Every single dose, almost, with a couple of exceptions, as you go up is sort of bigger, and it even looks like it reaches the maximum effect.  Then you just do a silly thing.  You plot that same thing on a log axis, and all of a sudden, it looks like it's not increasing to a maximum.  It looks like it would keep going up forever.  All that's been done here is to go from a linear axis to a log axis.
		So, you say how should I look at this.  What's going on here?  How come these two graphs look so different, if you're graphically oriented?  Did it really get to a maximum effect?  I don't know.  
		But indeed, what we do is we look at this data point and we say, before that data point might be approved, how many people received that dose?  It might only be 50.  In the event that only 50 people received that dose, we say, it's smaller than that, the point estimate.  That dose can't be approved.  So, we do some kind of funny stuff here even though this is all one nice continuous curve and there isn't a dose-related side effect in the database.  This is an ACE inhibitor.  There isn't any such thing as there being a dose-related side effect that I'm aware of.
		So, then you get to another guy, and again plotted there linearly, the data is all over the place.  It's a bunch of trials, different dosing regimens, q.d., b.i.d., and so on and so forth.  But the sense you get here is that it reaches a maximum.  Semi-log doesn't really tell you much and you get very confused.  At least I do.  What's the framework of reference here?  How should you plot this stuff?  Is it really going to a maximum?  Where are we on the dose-response curve?  
		But, indeed, some kind of dosing recommendation gets made, usual starting dose.  Ignore the 9.  That's a typo, and the total daily dose is from there to there.  There isn't any data that justifies that outside of the fact that that's all the doses that were studied, so that's all you have information about.  But it has nothing to do with the drug or the drug's intrinsic properties. 
		So, I need a framework of reference, and I say to myself -- and I have a problem.  I use a Mac and when you put a Mac Power Point thing into IBM Power Point, the electrons don't flow right.
		(Laughter.)
		DR. LIPICKY:  So, this is the number of drug molecules here.  There's supposed to be a drug there.  This RD is supposed to be over there.  So, it's the number of drug molecules interacting with a receptor with two first order reversible rate constants, because the universe operates that way.  Right?  Everything is first order.  That leads to a dose-response combination, and it may be and drug-response, and that that's identical to the effect.
		So, we can write that model.  You can write equations for that model.  You can put them in a computer and you get curves where you can vary the ratio of K1 to K2, and that's the affinity constant.  So, as you change the affinity constant, the curve moves, but the distance from here to there stays constant, but the ED50 changes, if you would, or the affinity constant changes.  
		The thing to point out is nothing funny that has been done here.  This model goes over a range of from here to one, two, three, four orders of magnitude.  Not four times.  Orders of magnitude. 
		Indeed, you can vary the number of drugs that attaches to the receptor, and then you get changes in slope.  So, this model you can worry about where its affinity constant is and what its slope is.  Those are the things that you can worry about. 
		If you have this kind of conceptualization in your head, you can say, well, this is the effect that I care about.  These are the bad effects.  There are some red points there.  And the distance in the ED50 between these two guys basically gives me the safety margin.  Indeed, this starts here, so with this particular drug you'd have a pretty good effect before you got any adverse effects. 
		Now, the problem is that we see data that looks like this, and these four data points are those four data points from the previous curves.  So, my worry is that if you don't see the whole curve, you don't know where you are because this looks terrific.
		So, we said, well, the way you make decisions nowadays when you don't know what to do, you ask a computer.  But computers want numbers and with numbers you've got to have equations.  
		So, we said, well, the simplest thing you can do is say that sum Y is equal to mX plus b, a straight line.  That translates to an effect equals an effect at 0 dose plus a slope times the dose. 
		So, you can use this model to see if the data you have collected is consistent with the model.  The behavior of this, when you plot it linearly, dose versus effect, look like that.  If you plot that as log of dose -- the same data, log of dose -- versus effect, the curve looks like that. 
		Well, you can also say the effect is equal to the slope times the log of the dose plus b.  I should have written that differently.  That translates to that equation that was put into the package that was sent to you.  Now you have dose effect.  That curve looks like that.  If you plot it on a log axis, it looks like that. 
		Now, those are very different looking curves. So, you say to yourself, geez, I ought to be able to tell whether this model or this model applies to the data that I've obtained. 
		But then you say to yourself, yes, but this is stupid.  This says that as long as the dose keeps going up, the effect will keep going up.  And you know that's not true.  You could be a walking crystal of an ACE inhibitor and your blood pressure is still finite.  Right?  So, that can't be a relevant model.  That's just a stupid model.  But it's reasonable to ask yourself can I tell that that model that isn't operative. 
		This guy has similar properties.  As long as the dose goes up, the effect is going to keep going up forever.  It has the unusual property that the 0 effect is not at 0 dose because there isn't any 0 dose on a log scale.  So, you've got numerical problems here, and people who do numerical things don't like that. 
		So, you say, okay, it must be some nonlinear thing like the effect is equal to the effect at 0 dose plus some complicated thing that contains the maximum effect and the ED50 and the dose.  You can tell the computer for this model, that this is my data.  I know this and I know this.  Tell me what this is and this is, both E0 and slope.
		Similarly, for the log thing.  And here the effect is equal to this.  So, you know this and you know this and that, and you tell the computer, tell me what the other things are.  That's called overspecifying the model sometimes.  That is, the data we're analyzing is placebo subtracted.  So, we know what that number is, or it is part of the observations if we assign it that way.  
		But we didn't do that in this first pass, and I can say we have a second pass of the data that we sent out to you that, in fact, did not have an intercept term for either the linear or log linear or the Emax model.  And the qualitative information you get out of looking at this data is exactly the same.  The numbers are different and so on. 
		One minute longer here.  When you tell the computer I know this and I know this, tell me what this is and what this is, the computer gives that to you with a confidence limit.  So, from the data you have and this equation, you can tell how well you know that number, and you can tell how well you know that number.  If you have two sets of data, you can even ask the question, is this number for the one set of data different from that same number for another set of data with some confidence that you can, in fact, answer that question. 
		So, if you buy the Emax model, you can simulate it.  In fact, the Emax model is here where this is a drug that looks like that from 0 effect to 100 percent effect, a drug that is less potent, but equally effective, that looks like that, then another pair of drugs that are less effective and they look like that and they look like that.  But all of those drugs have the same ED50, if you would.  So, you know what effectiveness is.  This is less effective than that.  You know what potency is.  This is more potent than that.  
		And if one studies this drug at that dose and that drug at that dose, and finds that the two point estimates are different from one another, you know that that has absolutely nothing to do with effectiveness.  And if you really wanted to know whether something was more effective than the other, you've got to have an appreciable part of the two curves to be able to make that differentiation.  Otherwise, you are guessing at what is going on. 
		Now, this is exactly the same simulation, the same equation, the same ED50, everything else, but 10 percent of the baseline blood pressure was added as noise to every point estimate.  Boy, you're less able to make differentiations there.  In particular, if you take any one section down here and you have three data points, you tell me that anyone would be able to tell you what's going on.  That is, however, what we do and we label drugs because someone has three points there and gives that to us and we make sense out of it. 
		Then why do we want to know anything about what the maximum effect of a drug is?  Well, in the hypertension area, our policy with fixed dose combination labeling is you should never use a second drug until you have explored the entire range of the effects of the first drug because you shouldn't expose people to two sets of the independent side effects.  So, if we're serious about this statement, we ought to be equally serious about knowing what dose gets the maximal effect.  
		So, the notion then was to take the data we had and see what we have.  So, if we could see the first overhead please.  This is a graph of whatever the number of drugs that we had.  This is all approved ACE inhibitors and/or angiotensin 2 receptor blocking agents.  Don't pay any attention to the lines that are present on any of these graphs.  This is linear scale. 
		A lot of these guys, data point-wise, look like they've hit a maximum effect.  It happens to be that this line is a linear fit, and if you happen to do a linear fit, in fact, the line goes through the points.  Not too bad.
		But if the same data is plotted semi-log, all of a sudden all of them straighten out.  Indeed, the line that goes through the data points that is a log dose line, to my eye this all looks better than the first set of graphs did.
		Then if you want to look at the Emax model, now you should pay attention to the line on a linear scale.  Well, the Emax model line goes through the data points, more or less, when it's plotted on a linear scale, and it even looks like it curves.  That's what we want to see. 
		Then when you plot it semi-log, it stills goes through the data points, but now the curvature in the wrong direction.  What the hell is going on?  So, then you say to yourself, well, how am I going to decide?  This doesn't make sense to me at all.  I'm at a loss.  What does this data show?
		Well, you go back to the computer and you say, tell me which of these models fits the data the best, because that's supposed to answer your question.  You say, I ought to be able to tell a log-linear, linear, or Emax model from the data because the linear and log-linear models are obviously nonsense.  They can't be applicable. 
		So, one of the ways you try to decide is to look at the objective function from this program and a smaller number says you have fit the data better.  Your model is more consistent with the data.  Well, you tell me that any of those models are more consistent with the data than another.  Maybe out here Emax does a little bit better, but otherwise any of those models are applicable.  So, the data will not allow us to reject or accept a model.
		You can look at another kind of how well does the data fit, which is the Akaike criteria.  Here it's the same thing.  You just cannot tell the difference between those models by criteria.  You can look at the standard error of your parameter estimates, and all of those numbers were sent to you.  The standard error of the parameter estimates are pretty big, so you'll have a lot of confidence in the parameter estimates you got.  
		So, the long and short of it is when you look at the data and you try to do something very simple but crude -- we did the wrong thing.  We had the intercept in for all of this stuff.  We shouldn't have had the intercept in.  When you take the intercept out, it looks a little better but the qualitative thing doesn't change.  Well, it says you can't tell.  So, I hope you didn't spend a lot of time trying to figure out what those things tell you because you can't tell anything from them. 
		What that means to me is the data is not collected.  The data you need you don't have.  We have not planned the experiment properly.  We haven't made the proper observations. 
		So, then you say, I'm going to make the proper observations.  I'm going to take this thing.  
		I forgot to point out.  If you look at those data, most of that data goes over several orders of magnitude.  We have one of those drugs that goes over three orders of magnitude.  Three orders of magnitude wasn't good enough.  
		So, you start looking at what you got.  You got a few orders of magnitude, and you're going to change dose by 50 percent increases.  How many arms is that?  Well, people tell you to go home, you're crazy.  
		Well, you can cover that same range with just six doses if you increase things by a factors of three.  So, maybe that's how we should be thinking.  If you're doing to do a dose-ranging trial, at least use factors of three because you can cover a wider range, if you think you need to cover a wide range.  In fact, we have told people who just want to do two three-arm trials, placebo, dose, two times dose, don't do the two times dose.  Pick dose or two times dose and put all your resources into a two-arm trial so you really know whether you found something or not because you're wasting your time studying two doses that are the same. 
		Now, I said things aren't planned right.  Well, how should they be planned?  The first statement is, why dose range?  Whole curve.  Big increments between doses.  The second is, where on the curve should you collect that data?  Over the whole curve or parts of the curve?  So, if you have a model like Emax, and data that has error in it, you can say, if I pick this dose and this dose and this dose over the whole curve, does that give me better information than if I pick this dose, this dose, this dose or this dose, this dose, this dose, or this dose, this dose, this dose?  
		When you boil it all down, if the question you're asking is, I know this is an Emax model, because that's how I made the data -- it's got to be an Emax model -- and I introduce some error in the point estimates that is generated, and I want to reject a linear model or reject the log-linear model, because I know this is an Emax model, it turns out simulation 4, putting your doses there -- you only want to pick 3 -- gives you your best chance at being able to have data with noise in it that, in fact, allows you to make some decision.  Any of the others are less adequate. 
		This just lays it out graphically where if you have an Emax model that you believe and you want to know the probability of rejecting the linear model, pick the Emax model over the linear model.  If you have simulation 4, and you do some Monte-Carlo simulations, you need a smaller sample size to have pretty good chance of rejecting it.  If you do this simulation 2, this guy, you're not so well off, and the thing that you think would make you well off -- you have the whole limits together, the simulation 1 -- is not bad but not as good as if you just concentrate in some part. 
		Now, I'm not offering is as this being meaningful.  In fact, it probably isn't and we probably should have done a different simulation, and we probably should do more work and so on and so forth.  What I'm offering this for is that if you think you are going to analyze the data with a model, you think you know the model, in fact you have a pretty good idea of where you want to concentrate your efforts, and to put it in Lew Sheiner's terms, if you tell me what you want to know and you tell me the accuracy with which you want to know it, I will be able to tell you what you have to do. 
		So, the discussion today is sort of going to be you telling us what you want to know, and Lew Sheiner and Don Rubin telling us what you have to do to get it.
		I'm open to questions. 
		DR. PACKER:  I think what we need to do is pause and consider the issues that Ray has put forward because, I think as Ray concluded in his closing remark, it would not be useful to go on to the next presentations until the committee concluded what it wanted to know.  So, we need to focus on what we want to know.  I think that the illustrations that have been made by Ray clearly identifies some of the inadequacies of the databases which have been presented for approval to date and the difficulties that are encountered in writing labeling based on the data which is presented. 
		There are, as I can see it, two somewhat different situations that the agency faces in looking at adequacy of dose response.  
		There are databases where the range of doses is limited by an identifiable and prohibitive adverse reaction, and you gave us one example of that.  I think you emphasized that in spite of the fact that that development program did not cover a wide range of doses, it clearly identified a "safety margin," a relationship between the doses that define efficacy and the doses that produce an adverse reaction.  
		My sense, Ray, is that to the extent that most of the drugs that we have available for the treatment of hypertension have such dose-limiting side effects, that you have not been unhappy with the dose-response information that has been developed for those drugs that have dose-limiting side effects.  Is that a fair statement? 
		DR. LIPICKY:  Right.  I must admit that we generally ask people, although they don't actually always do it, to find a dose that hurts people and to clearly demonstrate it.
		DR. PACKER:  But I think that that's the point.  Up to now, in order to qualify for what might be called the easier path, the path in which there is a dose-response relationship or any useful dose range, defined by a dose which produces an adverse reaction profile which is unsatisfactory, companies actually were somewhat grateful when they had a drug that produced a problem.  I know that sounds a little strange, but their development was simplified when they identified a drug that produced a dose-limiting side effect that essentially capped the upper range of the dosing curve. 
		However, in order to identify that, they had to make some people sick, and the hope was they had to make as few people sick as possible, and that if they could do so and they could do so early in the development program, then the delineation of dose response was simplified. 
		DR. LIPICKY:  Right. 
		DR. PACKER:  The problem that comes to the fore is that to the extent that the pharmaceutical industry has now been increasingly able to develop drugs that may not have dose-limiting side effects like ACE inhibitors and A2 antagonists, then the previous model of trying to define or guess at a dose that produced a problem was not workable.
		DR. LIPICKY:  Right. 
		DR. PACKER:  And that is really part of the problem that we have today. 
		Let me just ask two questions so we can define the questions for the committee.  In the model which is the more common of the two, where there is an AE that limits dosing, you can write labeling for use.
		DR. LIPICKY:  Correct. 
		DR. PACKER:  However, to an increasing degree, industry is also interested in determining whether their drug is better than another drug, something that you hinted at when you showed two dose-response curves that had different effectiveness.  And they want to say, well, our drug is better than another drug, and your response is, well, it's hard for you to say that unless you have defined the entire shape of the curve for both drugs and have shown me that your drug has a higher plateau, a higher Emax, that the slope of the curve is different than the slope of the competitive curve. 
		DR. LIPICKY:  Correct.  That is what my response would be as an individual.  The agency's response, however, is that if a dose of a drug gives a better effect than another dose of another drug, that is reasonable information to communicate.  And if you show me two trials of such nature, you can communicate that.  That is mistaken by many people as a difference in effectiveness. 
		DR. PACKER:  Right.  Your concept that, in order for a company to claim superiority, you need to define the entire shape of both curves -- 
		DR. LIPICKY:  Yes.  You assert that. 
		DR. PACKER:  How should that concept be modified when you have a development plan for both drugs that has a dose-limiting AE?  In other words, since it isn't, as I understand it, necessary to define the entire shape of the dose-response curve, if you have a dose-limiting adverse reaction, can a sponsor claim superiority?  You can clearly write labeling if you have a dose-limiting AE.  Can you claim superiority if you have a dose-limiting AE if you compare the new drug to an old drug at the highest recommended dose?
		DR. LIPICKY:  Yes, I think so.  To me the thing that you're asking is very complicated because it would require your knowing that the effect that you get at the best usable dose of the drug that has dose-limiting side effects, you'd have to have a pretty precise estimate of the magnitude of that effect.  Then you would have to say that at some range in the dosing of the new drug that has no adverse effect, dose-limiting side effect, that you know at what range in that dosing regimen you indeed exceed the effect of the other drug.  Then you'd be able to say, I can produce a bigger effect than this drug, and more than that, I don't produce side effects, so you don't have to worry about it.  
		Did I answer the question you asked?
		DR. PACKER:  No.  You may have, but I wouldn't know yet. 
		(Laughter.)
		DR. PACKER:  Let me try it a different way.  It is possible for a sponsor to say that their drug is better than another drug -- and let's just say it has to be in two trials and it has to be persuasive, internally consistent ‑‑ either because the shape of their sigmoid curves are different and their Emax's are different or, in fact, their Emax's may be the same.  But since one can push the dose of one drug higher than another because of the absence of adverse reactions, would you allow a company to claim superiority even if the shape of the curves were the same if in fact one could push to a higher dose with a drug that had fewer AEs?
		DR. LIPICKY:  I think so.  What I was addressing was the difficulty one would have in getting that information.  That would be hard to get that information.  But, indeed, I think if one showed that, that would be claimable. 
		DR. PACKER:  So, the problem is not in the validity of the concept, but in the ability to get there.
		DR. LIPICKY:  The design and the analysis that would allow you to conclude that you, in fact, had that data.
		DR. PACKER:  And one couldn't simply do that by going after the highest dose of the comparator drug which is in their labeling?  Because presumably you made an educated --
		DR. LIPICKY:  I would assert that that would not be an appropriate thing to do, but I would probably lose the argument if it went to the place -- here's what I'm saying I guess.  
		Incidentally, for conversations, there is transparency in ink and overheads because sometimes you've got to have the curves you're looking at because you don't know you're talking about the same thing. 
		But if drug A has the dose-limiting side effects and drug B does not, and drug B's dose-response curve goes to effect sizes that are substantively greater than the effect size of the usable dose of drug A, that would be fine.  But it's the question of how would you tell it is substantively greater.
		You're suggesting studying a dose of drug A and then a dose of drug B and then doing a pair-wise comparison on that.  I think if I were pushed to the wall, I'd have to say, sure, that would be okay, but I'd try to convince you not to do that because that just seems slovenly. 
		DR. PACKER:  Slovenly but not inaccurate. 
		DR. LIPICKY:  Not inaccurate, right. 
		DR. PACKER:  Okay.  So, in fact, the development program that has a dose-limiting AE is not the issue before us today because you are comfortable with how a sponsor would define the appropriateness of use and therefore the labeling, and it therefore also becomes reasonable how one at least operationally, not ideally, would define differences between usable doses. 
		DR. LIPICKY:  Right. 
		DR. PACKER:  So, the issue before us today is what happens with the drug doesn't have a dose-limiting AE.
		DR. LIPICKY:  That is one issue and it is a real problem.  
		The second is, what are the properties of the drug, as it affects the effect in relationship to dose, that you want to know about?  Because, in fact, if you boil all of this down, all we're doing in this antihypertensive stuff is saying we will approve you if you beat placebo.  And that's the nuts of what we're doing.  I would personally like to see that changed but I don't have any support.
		DR. PACKER:  Yes, let me just make sure.  Just so that we clearly have defined this, sponsors in the past have pursued with great justification development of drugs that have the least possible side effects and the greatest safety.  What you're saying is that with the increasing ability to develop agents that have that profile, there should be an increasing responsibility on the part of the sponsor to develop a much more complete database on dose response.  In other words, although they can celebrate that they have a better tolerated agent, there should be an enhanced responsibility that goes along with that. 
		DR. LIPICKY:  And, in fact, one of the drugs that might have come to the advisory committee at one point, which is what this whole orientation was meant to address, was it could be that for ACE inhibitors, angioedema is dose-related.  And one just hasn't figured out that because there isn't a large enough dose that's been studied.  One will find that out by experience and by postmarketing and all that sort of stuff, or maybe that's not the case.  
		But the problem is I feel uncomfortable, in fact, if people have not identified the dose-limiting side effects.  That was the problem with rhabdomyolysis in  whatever those drugs are that lower cholesterol.
		DR. PACKER:  Statins.
		DR. LIPICKY:  Yes, statins. 
		That that was related to the dose of statin was not known until mibefradil came along and clarified it.  Now, that's a drug development problem that should have been known at the time it was approved.  It wasn't.  I do think that that ought to be considered to be necessary, although what do you ask people to do, and that's what we're talking about. 
		DR. PACKER:  So, what we need to do is have a discussion using what might be called the non-AE model, the example of drugs that they may or may not have side effects.  The idea is we're not talking about drugs which are free of side effects, but which at least from the data which can be collected and are available for analysis do not have a relationship between the frequency of a specific AE and dose.
		DR. LIPICKY:  Right, because that is the hardest problem, and that's the one to address.
		DR. PACKER:  So, let me ask the committee.  I hope that this has been clarified sufficiently.  The reason that this has come to the committee -- it never came to the committee before because we always were plagued with drugs that had dose-limiting side effects, and so no one had to talk about this.  Now we've got an increasing ability for industry to develop drugs that may or appear not to have dose-limiting side effects.  In an era where there were dose-limiting side effects, it was relatively easy to write instructions for use and define doses that could be used for comparative studies.  Now we're in an era where the appearance of -- there's got to be some relationship between AE and dose, but I guess not over the useful range.  Under those circumstances, how well should one define and how broadly and precisely should one define the relationship between dose and effect. 
		So, I just want everyone to focus on this because no one is suggesting here -- and the division is not even putting forward for discussion defining six orders of magnitude of dose for a drug that has dose-related side effects.  The focus today is for drugs that appear not to have dose-related side effects and the responsibilities on a sponsor to define a broader range of doses than they have conventionally defined in previous development programs.  I just want to make sure we've accurately focused on the question.  Ray, is that fair?
		DR. LIPICKY:  Yes.
		DR. PACKER:  Lew?
		DR. SHEINER:  I just want to say I think that is a particularly difficult question, as you bring up, and may be it's a good metaphor for us to consider because if we can answer that question, we perhaps can answer others.  But I don't want to say that the only situation that we're not handling adequately with respect to dose response is that case.  I think there's an argument that can be made for finding something about dose response even for a drug where clearly the upper limit -- we don't dose aspirin until we have tinnitus anymore.  We don't dose digoxin until you throw up.  So, we would like to find the right dose response.  But I agree, this is a nice metaphor to talk about because it presents a wider range for --
		DR. PACKER:  The reason, Lew, for framing it this way is because if we don't frame it this way, we're going to get lost today.  We will get lost not only in the broadness of the scope of what we can discuss, but also in our unlimited ability to have our clinical judgment intercede here, especially when it comes to the use of multiple drugs for the treatment of hypertension because we will commonly say, of course, we'll use a second drug, and it is no problem in using a second drug.  There's no reason to define the range of useful doses for a first drug because we will always put the second drug in place.  That's a very appropriate clinical response because invariably the reason that we're going to the second drug is because the first drug has a dose-limiting side effect we do not want to encounter.
		The discussion is far more interesting if the first drug doesn't have a dose-limiting side effect that we want to avoid and then the argument becomes more interesting as to why one would go to a second drug if one hasn't fully utilized the first drug.  So, we've got to discard the first paradigm because we can find all sorts of excuses for not looking at the dose-response curve for the first paradigm.  If we don't focus on the second paradigm, we won't have a useful discussion. 
		Jeff.
		DR. BORER:  I would like to get back to Lew's statement here for a minute because I think that there's something very important that we need to focus on here.  Following the line of reasoning we've begun to discuss, I think it is important to attempt to define the dose-response curve even in drugs that have side effects at the upper limits, at some high level because, first of all, in theory -- well, there are several thoughts here.  So, let me try and put them in order. 
		Number one, for the sponsor, to define a drug as effective, the demonstration of a dose-response curve is one form of acceptable evidence.  So, if you define that a dose-response curve exists -- forget about the relation to safety for a moment -- you've shown efficacy for approvability purposes. 
		Now, a problem, it seems to me, is the assertion that we're in an era where there's a class of drugs that doesn't have dose-limiting side effects.  It may be true, but I don't know how you know that until you give high doses of the drug.  That means you've got to expose a certain number of people to the drug at high doses, and how are you going to predict that there will be a problem?  Well, you do animal studies at high doses, but maybe you don't see what you're looking for in the animals. 
		I'm going to bet, because I've heard Lew speak before, that he will show us mathematical models or methods that would allow us to use relatively small numbers of data points so that you don't have to expose too many people to very high doses that might be deleterious in order to define the shape of the dose-response curve.  If we have the shape of the dose-response curve, the only other thing we'd need to write a label would be the dose-related side effects and what the side effects are at each dose.  That's what we'd need. 
		So, I think it is important to define the dose-response curve throughout its entirety or as high as you can go without running into side effects that potentially kill people or are irreversibly damaging because, without that, in fact we really don't have any starting point for determining the level at which we want to look for dose-limiting side effects.  So, I think it's probably wrong and probably a little naive to suggest that we're now developing drugs that don't have dose-limiting side effects because we don't know that until we test it. 
		So, anyway, I would argue that we ought to be looking at a larger segment of the curve, that we probably can do it with relatively small numbers of patients, that that is a basis for approvability, so it's not a lose-lose situation for a sponsor, and that then we have to figure out somehow at what level we want to start looking for dose-limiting side effects.  I would say that part of that decision would be based on the increment of effect as you increase the dose, which you'd know if you did the D-R curve.
		DR. PACKER:  Jeff, I think what you're saying is that the two paradigms may not be different, the one which has AE-limiting and the other one has no AE-limiting.  It's just that maybe it would be better to say that a dose-limiting AE has been identified in paradigm 1 and has not been identified in paradigm 2, not that it doesn't exist, but it hasn't been identified.
		DR. BORER:  I would agree with that. 
		The other point is the one that I was beginning to ask about before and now I'm glad you told me to hold it till now.  The issue is that, of course, not all individuals respond identically to all drugs in all situations.  So, even if there is a side effect that's nonlethal or non-permanently damaging, that would preclude the use of the drug in 70 percent of the population.  If you knew what the D-R curve was, you could titrate up that D-R curve to get the extra increment that you know might be there until you hit that side effect.  Why be limited by the fact that 70 percent of people are going to have a side effect?  
		So, I think I'm arguing just what you say.  The paradigms aren't different, and what we ought to do is what's being suggested here, study the whole D-R curve.
		DR. PACKER:  Well, it's not clear that the committee would agree with that. 
		DR. BORER:  They may not. 
		DR. PACKER:  So, what we need to do is to define what we want to know.  Let me pose to the committee a specific point of discussion which is very important.  It's a very specific example.  
		Just suppose you had a drug that the sponsor had shown reduced blood pressure by what is conventionally referred to as a meaningful degree.  Let's assume that the identified doses that lowered blood pressure in the realm of 5 to 8 millimeters of mercury, which is the common range.  There are no dose-limiting side effects.  Is that database adequate based on everything that you have heard today?  Because clearly, the question that arises, based on the data presented today, is you could maybe get more -- you could get 10 millimeters, 12 millimeters, 15 millimeters -- if you went up and avoid the need for a second drug entirely in patients who require a greater antihypertensive effect. 
		Or should we congratulate the sponsor for having developed a drug which does a reasonably good job based on conventional criteria and that they should relax and feel good and be approved? 
		That is really the question because if we say that the second response is appropriate and that drug is approvable, everything else being equal, then we don't have to discuss how we're going to define the dose-response curve because we're saying we don't want to define the dose-response curve.  
		So, this is really a pivotal point of discussion right now.  It deals not only with questions 3, 4, and 5, but it deals with the whole purpose of going on with the further presentations.  If we say we don't care, then we can go home. 
		DR. LIPICKY:  No.  You'd have to listen to Lew and Don anyhow. 
		(Laughter.)
		DR. PACKER:  But if we say we should care, then we need to find out how we should care.  I really want the committee to focus on that.  It's a very important question.  Each member really needs to think through this.  We need to have a full discussion on this issue and all of the issues that pertain to it, and we'll begin with Marv.
		DR. KONSTAM:  I think, Milton, you've broadened the question from an intellectual question to getting into the realm of practical drug approvability.  I am becoming increasingly convinced that dose responses are not being adequately explored and we need to move toward adequate exploration of them.  So, I think that's going to wind up being my conclusion. 
		I think, though, you're pushing it because getting into the question of, okay, but now you've got an NDA in front of you and there is a set of doses.  That set of doses clearly achieves a response at a very acceptable side effect profile.  So, by definition, that range of doses is safe and effective.  Now, in my experience on the committee, we've never been asked anything beyond that.  Yes, it's safe and effective, but we don't know that the drug might not be more effective at a higher dose.  So, this is the first time at least I'm involved in getting into that. 
		Now, I don't know whether we're going to be able to, at the end of the day, really answer that question.  I wonder whether this discussion is really specific for hypertension.  It makes more sense in hypertension because, after all, there are so many antihypertensive agents, and for the next antihypertensive agent to come along, let's get more information.  We don't need another antihypertensive agent necessarily to be approved. 
		But let's say we were talking about a disease that didn't have 500 drugs treating it.  Let's say it really is the next big thing in heart failure or some other area where there clearly is an added benefit shown, but the full dose response has not been defined.  I guess I would begin to get very shaky in saying I don't think a drug is approvable until the full dose response is defined under that circumstance. 
		DR. LIPICKY:  Just for Marvin's sake, limit it to antihypertensives, because that will make it easier.  I think you do need to generalize it, but limit it because that is the easier setting and it's where you can be most rigorous. 
		I guess what I'm looking for is what your intellectual sense is.  You can tell us, well, you shouldn't approve this.  It's not worked up right.  Yes, it's safe and effective, but you shouldn't approve it.  We won't follow that recommendation probably, but if you say do the bad things, we'll keep doing the bad things.  So, I'm asking you what you think.  We don't care so much what you think in the sense of the actions we take.
		(Laughter.)
		DR. LIPICKY:  But if you tell us to do things that are not meaningful, it's very hard for us to argue that we should do things that are meaningful.  If the dose response is important and you tell us, ignore it, well, then how can we say, no, we're not going to?  Say what you think.  Don't worry about the practical parts here.
		DR. PACKER:  Ray, in all fairness, I had toyed with the idea of asking the committee to take a poll, so to speak, and ask them whether they thought that antihypertensive development programs up to now were adequate in terms of defining dose response.  And I thought that that was sort of a silly question because my sense is that each of us would look at the presentation that you've shown us and say, no, it's not adequate.  I don't know if that's a very meaningful question. 
		The real question is, to the extent that it is not adequate, are we prepared to say it is not adequate and are we prepared to hold sponsors or recommend that sponsors be held to the conclusion that it is not adequate because, otherwise if we say, well, it's not adequate -- but I would still approve a drug which lowers blood pressure 5 to 8 millimeters and doesn't produce side effects -- then there's no meaningfulness to the declaration of inadequacy.  Then we will simply say, you know, you're right, Ray.  The whole thing is a mess and they should do better, but we really don't think that that sentiment should be translated into any action. 
		DR. LIPICKY:  So, I will make an assertion then to help you in your deliberations because I want you to enunciate what you think.  I will pledge -- if you say you've got to have the whole dose response to be approved ‑‑ to try to stick to that, and I'll fight to make that happen.  If you say I don't want to see the whole dose response and it would be all right if it didn't, then I'll say, what the hell do I care?  How's that for terms?
		DR. PACKER:  Well, I think those are fair terms, which is why I presume we're here today because otherwise this is just an analytical, theoretical, and potentially meaningless discussion, and all of us will scratch our heads and say, this was pretty interesting, but nothing will change.  
		Lew has been arguing for years that lip service is paid to this, nothing ever changes.  Nothing ever changes.  So, why should today be different than any other day.  Is that right, Lew?
		DR. SHEINER:  Yes.
		DR. PACKER:  I'm not saying today is going to be different than any other day, but I think what we should do is take the matter seriously and determine how we feel about this because that's what we're here for.  We're not here to have an elucidation and description of the failures of previous drug development. 
		Lew?
		DR. SHEINER:  Just a very quick comment.  I think one of the things that may help us think about this ‑‑ and I think you are on the right track of what you should be thinking about -- is to remember that you don't want to make the best be the enemy of the good, recognizing that it is extremely difficult, as Tom has pointed out, as others have pointed out, to understand the entire distribution of dose responses, conditional upon all the varieties of things that could be different among patients.  That doesn't mean you should give up, which is fundamentally I think what we've done.  So, I think there will always be a judgment of how much do you need to know. 
		Ray has abstracted the three questions that I will present to you that I think you need to answer before you do anything, the first of which is, what do you want to know?  But there are two others, and we can talk about those.  But they bear on the issue of essentially what it's going to cost and how much you're going to demand.  So, I think we can recognize that we could do better than we're doing without necessarily doing everything that was possible. 
		DR. PACKER:  I anticipate that every single member on the committee, whether they have their hand up or not, has something important to say and contribute to this.  What I want to do is have some general discussion, and then I want to move rapidly to addressing questions 3, 4, and 5.
		I'm going to skip questions 1 and 2 because my sense is that whether we admit to it or not, we use models, and that frequently those models are intuitive and frequently they are quantitative, and we will use both kinds of models.  Does anyone not use models?  We all use models.  We do all use models.  Right, Tom?
		DR. FLEMING:  The fact that the answer to that is yes is leaving out a huge part of this though.  I.e., there is a huge range in the level of model assumptions that are made and, in turn, the robustness of those model assumptions.  The real interesting discussion then comes down to how dependent on model assumptions are you willing to be because certainly the more dependent you are, the smaller the amount of patients or doses or whatever that you have to look at.  So, the essence of the issue here goes beyond just saying, do you use models, yes or no, but how reliant on model assumptions are you willing to be and what are the tradeoffs for what you gain by being more reliant against the lack of robustness or reliability of your conclusions. 
		DR. PACKER:  The only reason I think that this question -- and it's 1 and 2 -- is being asked is that, in fact, if we don't use models at all, then elucidation of dose-response relationships is impossible because they're all model dependent.  So, let me set aside the degree of dependency on the assumptions of models for a moment because that relates to the feasibility as opposed to the desirability. 
		DR. FLEMING:  But this gets down to one of the real philosophical issues, and actually I really would like to hear the other presentations as well because there will be important input, I'm sure, from Lew and Don's presentations. 
		But one of the philosophical difficulties, as I encounter all this, is in phase III are you looking still at an exploratory stage or a confirmatory stage.  It's not to say that you can't be doing both, but if you are still heavily relying on models, as you point out that you are, when you're looking at dose response, when you get to the confirmatory stage, are you at a point where you want to be able to say something that is less model dependent?  It's one of the reasons that in phase III studies, as Carl pointed out, people are frequently looking at pair-wise comparisons because they're trying to confirm whether a given specific dose is proven to be efficacious against a control.
		DR. LIPICKY:  Just to follow that, I have seven slides I didn't show because I thought I had formulated the problem okay.  It sort of tends to address what Tom is saying now with respect to where in the stage of drug development do you get what information.  I wonder if I ought to do that now or wait until some later time.
		DR. PACKER:  Maybe one should wait. 
		Before opening this up for discussion -- it's a very important discussion -- can we just have one other point of clarification on the paradigm that we're not focusing on today, which is the paradigm where there is a dose-limiting side effect which has been clearly identified, what might be called the old model.  I hate to use that because there are lots of new drugs that fall into the old model, the AE-limiting model.
		A lot of sponsors will say, well, we have identified a dose or range of doses which lower blood pressure and have an acceptable safety profile.  We have gone higher than that and we've gotten into problems.  Do we need to go lower than that and identify the minimally effective dose of a drug that has dose-limiting AEs?  That is not something specifically that we are likely to address in the next few minutes because we're focusing on the model and all the questions before us are in the model in which dose-limiting AEs have not been identified.  
		So, let us just complete the picture of the model where dose-limiting AEs have been identified if a sponsor has identified a range of dosing which they believe is useful and the agency would be likely to recommend that range for purposes of labeling.  There's a clearly identified dose or dosing which would not be recommended.  Do they have to now identify what is commonly referred to as the minimally effective dose or a no-effect dose?
		DR. LIPICKY:  Are you asking me?
		DR. PACKER:  Yes. 
		DR. LIPICKY:  I'm sorry.  Well, I don't think those words are useful words because they imply value judgments.  Is a half a millimeter important?  And they also imply sample size; that is, the minimally effective dose, if you're doing a pair-wise comparison, the difference will not be different from placebo unless the dose is pretty big if you have a small sample size.  If you have a large sample size, you can detect a smaller thing.  So, the concept of minimally effective or no-effect dose is very difficult and not a very meaningful concept.
		I think a better concept is you know where the ED50 is because you have defined the entire curve and you know what model is applicable, so that then you can, in fact, intelligently say, well, some dose down here really isn't going to do much and that that should not be an empirically derived property.  At the moment, we do require it to be an empirically derived property.  
		DR. PACKER:  Right.  In fact, we commonly see sponsors that make presentations in front of this committee where they do a pair-wise comparison between a low dose and placebo.  The p value is .07.  They say, see, we've identified a noneffective dose.
		DR. LIPICKY:  But I think that whole conceptualization is just totally misleading and isn't worth the time it takes to discuss it, which we could discuss for a long time.  It's really a matter of you've got the whole curve.  You know what the curve is.  You feel confident in the model.  I'm making an argument that you may not accept.  Then you, in fact, have the information you need to be able to make rational, reasonable choices.
		DR. PACKER:  Then let us set that question aside because what you're saying is that the principles that we are discussing for the model which has no identifiable AE-limiting side effect is just generally applicable to both models.  It's just that the first model is just easier to execute. 
		DR. LIPICKY:  Right. 
		DR. PACKER:  All right, terrific.
		Now, I want to have an open discussion on the example that I put forward because, although we can have a wonderful theoretical discussion on the desirability of dose-response relationships -- and my sense is that everyone would say that the history has been inadequate and we should do a better job.  So, I'm going to assume that everyone thinks we should do a better job.  The question is, how much better a job should we recommend to the division that they hold sponsors to?  
		The best example I can come up with, the most striking example, is development of an agent which lowers blood pressure to what might be called a meaningful degree, in which the side effect profile in that range is very, very good, let's say comparable to placebo.  And the dose has not been pushed to a range where the possibility is the drug may be even more effective with the possibility that dose-limiting AEs might be identified.  Is that kind of database adequate for approval?  Because that is question number 3 and that goes to the heart of identifying what we want to know today.
		We'll start.  Ileana, please, lead off.
		DR. PINA:  I'll state one of my biases that I've had for a long time about choosing dosages of drugs, and that is that I don't think we choose dosages adequately.  I am interested in a comment that Dr. Peck made that there are numerous data points that are never analyzed, and I'm not as interested in that mean as I am in the people that it didn't work on.  And if the people that it didn't work on, it's maybe that they need the higher dose and maybe that's the population that we need to push the dose to finding out a side effect.  We dose children by weight and by body mass.  Why don't we dose adults the same?  So, I have a problem with this initial basic starting dose, which as a clinician has never helped me. 
		So, no, I don't think we're exploring the dose response adequately.  I think we're taking this range of the middle population and the patient that is going to give us the most problem is the patient that's out there on either side, the patient that is not responding well to that dose that the average population may.  Maybe that person needs a higher dose, and yes, I'd like to push it up to a side effect profile that I'm going to have some side effects because I want to know what that maximum dose is, and that may be that maximum dose for that person and not for the small, frail, elderly woman who may do very well at a lower dose.  So, going back to the era of individualization, since we're not at personalization yet, that's where I'd like to see it go. 
		DR. PACKER:  Jeff and then George and then Carl and then whoever. 
		DR. BORER:  Going back to the suggestion or the dichotomy you set up -- that is, do we congratulate and say that's it or not approve -- I think that the straw man with the mutually exclusive choices isn't really the only option.  I think that you can conditionally approve and ask for more data.  We have to uncouple the approvability and the need for data collection in some way. 
		So, having said that, I think that there are a couple of options here.  If you have an effective and safe dosing range to give a drug, well, it may be reasonable to make that drug available, but to mandate that additional information should be required, for example, in phase IV.  The reason I say that is that the real world is there's just so much money to develop drugs with, and a safety database can cost money; whereas the efficacy database, I would maintain, probably doesn't cost as much money if you actually are studying a dose-response curve with polar extremes and a couple of points in between and going across orders of magnitude. 
		But having said that, I think that it would be useful to point out in a guidance that's easily understandable exactly what the principles of identifying or defining the dose-response curve are so you can know how many patients might be required, so you can know how to set up such a study appropriately and at the least possible cost.  If that option were available to sponsors, my guess is they might use that as part of their initial studies to gain approvability and then the phase IV problem, which could be a mandatory phase IV, would be to define safety at certain dose levels that might not have been defined in the pre-approvability phase. 
		So, I don't think it's an either/or here.  I think we need more information.  Perhaps it's time to mandate more information.  But you can't mandate so much that drug development stops because people can't afford to do it. 
		I would also say, just as an aside -- and I'm sure you didn't mean it this way, but I will answer it anyway -- that it is not reasonable to say that because a certain decrement in blood pressure has been achieved, that therefore we have a drug that we know how to use appropriately because our understanding of pathophysiology is changing over time and our understanding of not epidemiology but the utility of certain kinds of interventions is changing, as we gain more information.
		For example, 15 years go who knew that in people over age 55, blood pressure above 140 in systole actually was more important as a predictor of outcome than diastolic blood pressure?  Who knew?  Nobody knew so nobody was treating for it.  Now, the recommendations for hypertension seem to be that in people with coronary disease, systolic pressure ought to be below 130.  Well, maybe 10 millimeters of mercury average drop isn't particularly useful, and if we limit the information to doses that do that, we're not really providing the information that would be most useful to treat individual patients. 
		DR. BAKRIS:  If we were in a different forum, I would stand up and applaud what you just said because I wholeheartedly agree, not only with you but with Ileana as well.  
		I think, listening to this discussion -- and those of you who know me know that I'm not very quiet, but I've been quiet here because I'm trying to assimilate all of this -- it reminds me of the old military service -- I think it was the Army -- motto, "Be all you can be." 
		One of the problems that physicians have -- and I appreciate the comment that starting dose hasn't helped anybody.  It certainly hasn't helped me.  Most physicians in practice are scared to go beyond that, and we have a very limited dose range.  There are certain drugs that we're talking about here that in the very few studies where they have been looked at -- and by the way, most of those are animal studies -- the effects are incredibly good in a very few human studies in a very small number of patients.  They do quite well.  I think the potential is not realized in these agents. 
		I think Ray presented a very elegant model that needs some tweaking and needs some statistical help to help give us some guidance as to how to do this, but it's a very efficient model that I think will answer a lot of things.  You pick three doses in an appropriate mathematical range, and you get a tremendous amount of information across the gamut.  And all these other questions about safety and other issues, if they've already been looked at and they're pretty safe already, should be looked at more. 
		And yes, I do think it should be a precondition of approval because if you don't do that, you'll be going with the putzy definitions that we have right now.  I've published papers where we're talking about using three and four different drugs at maximal doses to control blood pressure because that's what it's taking.  Who knows, if we actually use the doses in higher quantities? 
		Milton, I know you know this.  The cardiologists know this and even the nephrologists know this, that super high doses of particular classes of drugs have very good benefits.  The GPs are terrified if you talk about these.  They think we're insane.  We should be locked up.  That's because there's no data to give us guidance.  So, I think this is a very important point. 
		DR. PACKER:  Carl?
		DR. PECK:  One thing that nags at me, and perhaps some others, about this discussion is that I think the FDA actually doesn't have the legal authority to withhold approval if you have demonstrated safety and effectiveness.  Now, the FDA often gets more out of sponsors than that sort of simple rule, using a bully pulpit or using their influence.  But this could be challenged legally.  The legal department of a company could come and say, look, we've done what everbody else did before us.  We've demonstrated safety and effectiveness with adequate and well-controlled trials.  Why are you withholding this? 
		So, I think I would challenge the committee to consider what compelling medical or public health rationale can you add to this interest in a broader range of dose-response information.  I think you can come up with that.  Ray made a very interesting comment earlier that the clinical pharmacology of Posicor, or mibefradil, was well-known but it was the lack of knowledge of an interacting drug that actually resulted in its withdrawal from the market. 
		So, drugs are exposed to patients at much higher levels than they are ever studied during drug development and there are a wide variety of circumstances.  Patients take overdoses.  Patients dose themselves higher.  Physicians dose off label at higher levels, or they encounter interacting drugs, which cause the effective dose to be much higher.  
		So, I would urge you to think about what is the medical or the public health rationale for this as the foundation, and then I think you'll have full authority. 
		DR. PACKER:  Carl, let me just add one minor response to that from a public health or medical point of view.  With the increasing evidence that previously recommended target levels of blood pressure are inadequate and that there is a need to lower blood pressure to levels that we previously had not imagined should be targeted but now should be targets -- and that may be particularly true in high risk patients like diabetics -- that there is a uniform sense that monotherapy, given the way that single drugs are used now, is going to generally not be adequate to achieve those targets and that we are going to have to rely on two or three or four drugs. 
		Therefore, a very cogent position could be put forward that the individual as well as public interests are best served by trying to minimize the number of drugs which would be required to achieve the new targets of blood pressure lowering that we are now hearing being recommended to physicians.  It is difficult for patients to take four drugs.  It is expensive for patients to take four drugs, and it may be unrealistic for patients to take four drugs.  And if a drug can be identified that would achieve those target blood pressures more effectively as a single agent, then in fact an individual as well as public health purpose could be served.  Does that seem reasonable?
		DR. PECK:  That's exactly what I think would serve your purposes best, to identify and inventory the strong public health and medical rationale for having the full dose-response curve available at the time of approval.
		DR. LINDENFELD:  To add to that too, I think we can then know the side effects that we might anticipate with a full dose of a single drug, but we can't possibly know the side effects from a combination of four drugs that people would have to use.  So, that's a very major advantage of getting these dose-response curves for a single drug.
		DR. BAKRIS:  Let me just chime in on this.  Certainly we have the data from the NHANES.  There's a paper I recently reviewed that will be coming out in JAMA that actually looked at the diabetics, as Milton brought up, and we all know that the goal blood pressure there is lower.  Only 11 percent of the diabetics with hypertension are actually achieving the goal recommended by the JNC-6.  So, there's a multitude of reasons for that, but we're not helping the situation by not providing adequate information about single agents that potentially could give you a lot more power than they currently are. 
		DR. PACKER:  Marv, Paul, and Ileana.
		DR. KONSTAM:  I guess I'm thinking somebody has to put in a word of balance in the discussion.  I'm very sympathetic to this discussion and we may well want to move in this direction.  But I'm troubled by it.  I guess I'm thinking that we are potentially setting a precedent for asking something of a pharmaceutical company that really is very different from what we've ever asked before in the sense that previously we've been say, well, just show that your drug is safe and effective. 
		Now, we're beginning to develop rationale, for example, saying, well, in an era where we need multiple drugs to manage hypertension, I want you to show us maybe you can help in that regard.  Maybe this drug could have an economic impact on that or other kinds of impacts if we really knew the full dose range.  I guess I'm hearing this argument for the first time and I'm troubled by the precedent that that establishes. 
		Let's expand it.  For example, I've never heard any serious discussion about cost effectiveness, at least on the panel and I think within the agency.  Now, I'm in favor of cost effectiveness, but it's beginning to sound a little bit like moving in that direction to me.  I just want to point that out.  I think there is potential precedent being set here. 
		DR. LIPICKY:  It's not as big a leap as you think it is.  Although it is true we approve because it beats placebo currently, all of the guidelines, ICH guidelines, the draft guidelines that exist from FDA, guidelines that Bob Temple has in draft, the European guidelines, all say for an antihypertensive drug there has to be adequate exploration of dose response.  So, really you can view this as one step further; that is, you're defining adequate.  They left adequate undefined.  So, it isn't that much of a leap.
		DR. KONSTAM:  That's fine and I guess I'm comfortable with that.  I guess just as the discussion has begun to expand, that's really what I'm beginning to get a little bit uncomfortable with.  I think there could be a significant precedent for saying, well, the company really has a greater responsibility well beyond showing that a drug is safe and effective.  Maybe that is a direction to go, but I just want to point out that that seems like it could be precedent setting. 
		DR. PACKER:  First of all, I think it would be safe to say that a level of uneasiness exists for every member of this committee.  This can't be viewed by anyone as being a no-brainer because I don't think that that would be appropriate.  I think this is hard at many, many different levels and you've identified a few of them. 
		I'm not a 100 percent certain that this represents necessarily either an incremental burden or a disadvantage to sponsors.  Let me just outline what I mean.
		Sponsors now, on a regular basis, do dose-response studies.  The only question is not whether they should do dose-response studies, but what should be the doses that they evaluate in dose-response studies.  In other words, instead of the range of doses being, as Ray said, 1.75X, or whatever it happens to be, or 2X, what Ray is putting forward is the proposition that maybe they should still do their dose-response studies but do it over a range of dosing which is more likely to yield useful dose-response information than the doses that they've been exploring to date.  So, this should not necessarily be viewed as a major incremental imposition on the drug development process.  
		Second, another point is that the hesitancy they have in pursuing higher doses has been I think motivated twofold.  One is they're afraid that if they hit higher doses, they're going to get side effects, and the side effects at higher doses will appear in labeling.  Ray has made it clear -- and Ray, you might need to say it one more time so everyone hears it -- that if side effects appear at a dosing range outside of that which is then recommended, it doesn't appear in labeling because there is a big perception in industry that if you hit doses that are going to be outside the recommended range, the labeling will suffer as a result of that because what you will see in labeling is placebo and active therapy and all of the side effects at the highest dose, which they evaluated because they believed in your proposition, would now be counted against them.  So, maybe you could just reiterate your statement that that in fact won't happen. 
		DR. LIPICKY:  That is correct.  It will not.  It's hard to say.  I always hate to say it will not because under some circumstances it might.  But given the current perceptions and the current practices and the current way in which people think, it will not. 
		DR. PACKER:  The other advantage that comes to industry is that -- my sense is it's not sufficient for us here to say that we think this is a good idea.  Industry has to think that this is in their interest as well.  My sense is that if, in fact, they are able to show that their drug has a broader range than they previously had expected of the agent -- remember what we're talking about today is a model where there are no AE-limiting side effects not the model where there are AE-limiting side effects.  If, in fact, you can show that your drug is a lot more effective than you thought, then a whole host of superiority claims become possible that previously were not possible. 
		Ray has said, reluctantly, that he would be receptive to such claims, even when the database is less than adequate, if in fact one compared against an old model drug that had AE-limiting side effects.  I didn't say you would be enthusiastic, but you would be receptive. 
		DR. LIPICKY:  I need to make just one comment.  There is no need to have a larger database.  The problem is that an antihypertensive NDA will contain all dose-ranging studies, 2,000 patients, sometimes 3,500 involved in parallel group, fixed dose, placebo-controlled dose-ranging studies, poorly conceived, not analyzed properly, incompletely thought through.  All the goals are wrong.  If I present my seven slides and Don and Lew are listened to, 150 patients, you've got all you need for defining dose response.  But people don't do it because they don't think anyone cares, and they don't know what they're looking for.  So, we need to just enunciate what do we want to know.
		DR. PACKER:  Paul has been waiting patiently.
		DR. ARMSTRONG:  Milt, let me make three points that I think are controversial but hopefully will be useful.  Ray started off this discussion by saying that blood pressure was very easy to measure and that's why he thought it was useful to talk about.  On the one hand, I think he's right. 
		On the other hand, I think that's deceptive.  The issue of whether systolic, diastolic, or pulse pressure is the right thing to measure -- and indeed, the technology about continuous blood pressure monitoring suggests that many of the blood pressures that have been measured in these trials bear little relationship to what we might hope to achieve.  So, I think that's deceptive. 
		I guess it leads to the question, why do we want to lower blood pressure because we don't have phase III trials relative to this?
		Which leads me to my second point, that a number of the agents that we know lower blood pressure have different effects on what we would like to achieve.  So, the target that we should be looking at presumably, as we extend this discussion, might be things, perhaps regression of left ventricular hypertrophy.  Maybe it's retinal arterial diameter.  Maybe it's a neurohormone.  I don't know.  But I think the issue of the objective of the exercise of lowering blood pressure, as opposed to just lowering blood pressure, needs clearly to embrace this conversation. 
		The third point is that, notwithstanding the beauty of classic dose-response curves from smooth muscle baths that many of us have worked in exploring vasodilators, the applicability of that to an intact human being or an intact human being who has disease is very difficult.  Indeed, although I support the decision to move upwards, to get some degree of flatness or to acquire some confidence that we've achieved maximum in whatever population, the likelihood that we'll be able to model that in a classic DRC seems to me unlikely.
		DR. PACKER:  Paul, let me just say that the doability of this is what the discussion is about afterwards.  Ray is saying that a lot of this information can be obtained very efficiently and hopefully in an interpretable fashion.  The focus now should be whether we think that this is important, whether it's desirable.  It could be that we're going to say it's important and then Lew is going to say, well, you can't do it.  So, really sorry to disappoint you. 
		DR. LIPICKY:  Paul, I guess it doesn't have to be blood pressure.  If you want to look at the dose-response relationship for left ventricular hypertrophy regression, that's perfectly fine.  But the question is, you don't think there is one?  You don't think you can define a dose-response relationship for whatever it is you're interested in?  This is really generally applicable.  It turns out it's easy to measure blood pressure.  That's why it was chosen.  We have lots of data.  But we can do it for mortality if you want.  It doesn't matter.  
		The question is, do you want to know?  And if you want to know, what do you want to know about that relationship?  Then you may decide, well, look, in this setting, I just can't have what I want to know, so I'll be satisfied in something less.  But in the hypertension arena, whether you care about blood pressure or not, you can know anything you want.  What do you want to know?  That's the question.  We're not talking in hypertension about death or anything else. 
		DR. PACKER:  Ileana?
		DR. PINA:  I have two comments.  I want to follow up on Jeff's statement that maybe we can approve a drug because of a broad range of safety with adequate blood pressure and then demand more studies that open up the range of dosing to higher dosages.  I think that's a problem because, the majority of times, the clinicians will never get the higher dosages relayed to them, or they will never develop comfort with them if they don't come out at the outset of the drug.  
		I share what was being said on this side, that the clinicians will only use smaller doses and will be very afraid to push beyond the original dosing that was given.  So, I would rather see it all done before the drug gets approved rather than later. 
		The second comment is that I think we're being naive if we're thinking that blood pressure now is what blood pressure was before.  It's analogous to saying if you give digoxin and diuretics to a heart failure, that's plenty and you really don't want to know what happens to neurohormones because they're not important.  I think we've even learned that blood pressure lowering enough may not be enough either, and I think the ALLHAT trial has proven to us that the lowering of the numbers may not necessarily affect the outcome.  
		So, I am very interested in harder endpoints.  It may be left ventricular hypertrophy.  It may be neurohormones.  If you look at JNC-6 -- and I'm imagining that JNC-7 even more so is being targeted to specific disease entities within the hypertensive population so that you target therapy depending upon what the underlying clinical position is.  So, I don't think we can just sit here and say let's approve drugs now the way we used to before when we're dealing with hypertension.  I think we've grown up quite a bit. 
		DR. PACKER:  Again, let me just focus on the issue of just blood pressure.  Blood pressure lowering is an acceptable surrogate, and we are not going to get into issues whether it is in fact or not an acceptable surrogate.  We can't go there.  We just can't. 
		One thing, I would like to make an observation.  There is an enthusiasm on the part of clinicians and I think enthusiasm on the part of sponsors to develop combination products for the treatment of hypertension in the hopes that if you add two drugs in the same tablet, you'll get a greater antihypertensive effect.  Maybe it's additive.  Maybe it's synergistic.  Maybe it's less than additive.  Who knows.  The hopes are by putting the two drugs in the same tablet -- the patient would normally be prescribed these two drugs separately, but they can be put into the same tablet to enhance convenience. 
		The same argument could be made for dose-response relationship for a single drug.  You can think of a higher dose of a single drug as being a fixed combination of a low dose plus a low dose of the same drug.  As long as there's no dose-limiting adverse reaction, then the same convenience issue pertains to a high dose of a single drug in a tablet versus a low dose of a single drug in a tablet.
		DR. LIPICKY:  But why are you asking that question? 
		DR. PACKER:  No.  It's just an example of the fact that convenience issues sometimes drive physician practices and therefore the use of a higher dose, which is better, which doesn't produce side effects, would enhance compliance. 
		DR. LIPICKY:  I guess I just want to say that the same concepts to developing a combination drug apply and that currently people do dose-range both drugs.  In fact, they would be able to do it better, cheaper, faster, if they did it differently.
		DR. PACKER:  My sense is that a lot of our responses are being influenced by perceptions, maybe accurate and maybe inaccurate, of how hard or easy this would be to do.  
		DR. LIPICKY:  Oh, duck soup. 
		DR. PACKER:  Before going forward, therefore, and asking the committee to formally consider the questions before us, maybe we need to have a better sense as to what this would entail.
		DR. LIPICKY:  Seven slides, about 30 seconds each.
		DR. PACKER:  Who's going to deliver it?
		DR. LIPICKY:  I will.  I didn't show them.
		DR. PACKER:  I thought Lew was going to deliver it. 
		DR. LIPICKY:  No.  I had seven slides that address the issue, and then Lew is going to address the broader issues.
		DR. PACKER:  Is the sequence you first, Lew second?
		DR. LIPICKY:  I had them in my first presentation.  I decided, when I got to them, that I should stop. 
		DR. PACKER:  Okay.  Then let's pause here for a moment.  Let's find out what this would entail from a conceptual framework, and why don't we hear Ray's 2-and-a-half minutes.  Then we'll pause for some questions and, Lew, we'll go directly to you for your presentation. 
		DR. LIPICKY:  So, these are two designs, fixed dose, parallel, one drug versus placebo.  Clearly this is easier to do because there are only two arms.  
		I apologize for the fact that IBM computers don't know how to make electrons flow right. 
		But if you have a placebo arm here, and for the first 2 weeks you're at 1X, and then you up it and you go to 3X, then you up it, and you go to 10X, and you up it and go to 30X, and you up it to go 100X, you've covered a 100-fold dosing range in 10 weeks, with sufficient time to get into steady state.  Lew is going to tell you how you analyze this data because it's complicated.  But, indeed, for the first 2 weeks, you have the standard stuff you could do. 
		If you are afraid of that, you could go back to the other design.  It's just that you have this fixed dose titration in the high-dose arm and then you haven't lost this.  Then if you buy into all that, you can actually go over 1,000-fold with three arms, placebo-controlled.  You have to be able to analyze this.  Here you, in fact, have at the end of the first 2 weeks a shot at standard, pair-wise comparisons over a dose range of 100-fold. 
		If this still bothers you, we've looked at ABPM data, and I'm not going to talk about it much, but basically I don't care what you want named, 24-hour average, a.m., p.m., nighttime, before breakfast, after breakfast, during eating, the coefficients of a Fourier curve, the coefficients of a polynomial, the mean of this, the mean of that, or anything you want named.  There ain't no placebo effect in short-term antihypertensive trials.  There's a nice drug effect.  So, you can get rid of the placebo arm.  
		In these same trials, no placebo effect in the placebo group, but in the -- I'm sorry.  This is the placebo group.  This is the treated group who had ABPM measurements, but there is a clear placebo effect for the office cuff, the same population.  So, the ABPM short term offers one the opportunity to perhaps use baseline blood pressure, no parallel placebo group.  Then you see you can do this, three arms, and you cover a 1,000-fold dosing range.  No big deal. 
		DR. PACKER:  Ray, what you're saying is that in order to elucidate the dose-response relationships, if one used ambulatory blood pressure monitoring, because of the absence of a placebo effect -- 
		DR. LIPICKY:  And a very different way of analyzing data.
		DR. PACKER:  Which we'll hear about.
		DR. LIPICKY:  Right. 
		DR. PACKER:  One can elucidate the entire dose-response relationship using this kind of study design.
		DR. LIPICKY:  150 patients, 9 weeks.
		DR. PACKER:  Let me ask a question because I'm sure it's in more than one person's mind.  How do you know you got steady state?  There are some drugs that don't reach steady state in 3 weeks.
		DR. LIPICKY:  You don't.
		DR. PACKER:  Does it matter?
		DR. LIPICKY:  Well, it does and it doesn't because, you see, you're going to have to study a couple thousand patients for a new chemical entity.  Right?  Now you got your dose range pretty well figured out.  Now you can figure out what you really want to know.  If you want to look at how long it really takes to come to steady state, fine. 
		DR. PACKER:  If you found it took 4 to 8 weeks to come to steady state, is this adequate?
		DR. LIPICKY:  Then this dose response would be a little bit wrong.  Right?
		DR. PACKER:  Right. 
		DR. LIPICKY:  But you would have a 1,000-fold dose range covered safely without any problem, and if it all worked out okay, 2 weeks would be enough.  For most of the drugs we see, 2 weeks is enough.  You're to about 80 percent, 90 percent of the final steady state effect-wise.
		DR. PACKER:  And your sense is this would provide information that would replace a lot of the dose --
		DR. LIPICKY:  Everything that people do now.
		DR. PACKER:  Would replace what they do now.
		DR. LIPICKY:  Right.  So, now you have another 2,000 patients --
		DR. PACKER:  That you can do something useful with.
		DR. LIPICKY:  -- that you can do something useful with.  Get rid of the dose response easy unless there are other things you want to know.
		DR. PACKER:  But people spend millions of dollars and years doing dose-response curves. 
		DR. LIPICKY:  It's stupid.  I don't understand why they do that. 
		DR. PACKER:  This is enough?
		DR. LIPICKY:  Yes.
		DR. PACKER:  Oh.
		(Laughter.)
		DR. KONSTAM:  What about adverse events?
		DR. LIPICKY:  A different problem.  Again, you're back to the 2,500 patients more you have to study.  You're talking about rare adverse events.  You'll pick up dose-related adverse events here pretty well.  You're talking about the 1 percent incidence of torsade.
		DR. KONSTAM:  Yes, but can't rare events be also dose-related?
		DR. LIPICKY:  Well, they might be, but then they will be in the category of angioedema in ACE inhibitors.
		DR. KONSTAM:  Right, but you said you wanted to know about that. 
		DR. LIPICKY:  No.  You said you did.
		(Laughter.)
		DR. KONSTAM:  Okay, we agree.
		DR. LIPICKY:  Yes, that is a problem.  There are still problems.  I'm just trying to get rid of the problem of what additional burden do you add if you say you want to see the whole dose range for the response.  There is the adverse effect stuff.  There is certainly the problem with no placebo group, how are you going to tell what the adverse effect is that's related to the drug. 
		I just wanted you to get a feeling for the fact that if you wanted to say I really want to know the dose-response curve, that you're not telling people they have to do another 3,000 patients.  That's all I wanted to do.
		DR. KONSTAM:  Can I just ask another question about it?  Let's say you do this and you still haven't found an Emax.
		DR. LIPICKY:  Yes.  Well, then you drop back to that simulation business and you top load the top arm because then you have most of your data in the higher doses, which is simulation 4, and you say, I'm going to analyze prospectively.  You say, I'm going to analyze this data with the Emax model.  You get your estimate of the Emax, your point estimate, and the standard error and 95 percent confidence limit, and I will believe it.
		DR. LINDENFELD:  Milton?
		DR. PACKER:  JoAnn?
		DR. LINDENFELD:  Just for clarification, I have a dose-response curve, and then I have an individual patient.  How will I know that there might be specific covariates that would affect the dose-response curve --
		DR. LIPICKY:  Well, Lew is going to tell you about that.  He says no.
		(Laughter.)
		DR. LIPICKY:  He wasn't prepared to do that.
		This won't tell you anything about that, unfortunately.  Well, it could.  So, he'll tell you then.  Don Rubin will tell you. 
		DR. PACKER:  Let's move on.  Does anyone have a specific clarification from Ray's supplemental presentation?  
		If not, let's go on to Lew's presentation.  Ray, my sense is that it will take both Lew's and Don's presentations before we can get into the meat of the matter.  Is that correct?
		DR. LIPICKY:  Probably true.
		DR. PACKER:  So, let's go on to Lew's presentation and see if there are any issues that come up here. 
		DR. SHEINER:  Well, thank you very much.  A lot has been promised about what I'm going to say, and I'm not going to actually give you a technical talk.  Part of the reason I'm not going to give you a technical talk is in one of my slides, because I really do think we need to keep these things separate.  So, I'm taking the title that Ray gave me, which is what do we want to know, and I'm going to present a somewhat theoretical point of view and with an example.  But in trying to clarify exactly what are the issues, the assumption I'm making, of course, is that there is something that we want to know.  
		I just wanted to start out with a slide referring to a paper I wrote a number of years ago, which tries to say about the whole drug development process exactly what Tom was saying, that there are things that we have in exploratory mode or what I call here learning, and there are things that we want to confirm, and that the entire drug development process can be seen as two cycles of learn/confirm, one early one that gives you the confidence to go ahead and spend all the money that you're going to spend in the later one.  In this process, phase IIb and III, as everyone has said, is the phase in which one is mostly interested for our purposes today in figuring out the optimal use of the drug.
		Now, just as Tom said, there is no reason why you can't learn while confirming.  I actually had that over here.  It requires a different view and possibly different portions of the data, but there is generally no reason why you can't do both and that's probably the most efficient way to do things. 
		But the point that I wanted to make about this slide is that exploratory analysis, learning as opposed to confirming, confirming meaning I have something that I believe is true and I want to test it, learning meaning I want to understand a whole set of relationships -- that this learning activity is quite different and is what can be applicable to the dose finding.  Let me elaborate on that somewhat. 
		First, let me get these three questions.  As I said before, there isn't one.  There are three.  The first is, what do you want to know?  Maybe Don will expand on this more.  That's not data.  That is to say, the answer to that question is something that exists in your mind in principle that you use data to try to learn about, but you don't want to know the average in this group of people.  What you want to know is something about the underlying mean of the population perhaps or something like that.  So, what do you want to know is a theoretical construct that you need to figure out what that is. 
		Then you need to know how certain you need to be.  This is where, again, what Tom was saying earlier comes in.  There's a tradeoff between certainty and assumptions, and that's the next question, which is what are you willing to assume.  What do you think you know already on which you're going to base your data gathering to learn some more?  It turns out that questions 2 and 3 have a very, very strong interaction, which I will elaborate on more. 
		I want to make a couple of points about these three questions, though.  These are questions that have to be answered, by domain-specific experts, by regulators, by physicians, by clinicians.  Technical people, people who know how to run clinical trials, people who know how to analyze clinical trials really have very little to contribute to the answers to these questions.  Mainly the contribution, for example, of the statistician to these questions is to tell you when you have stated how certain you need to be and what you're willing to assume, that that will take you 40,000 patients and 50 years, and you might want to think about that again.  But the technical people who are not the domain experts don't really have anything to say about these fundamental questions that are scientific or clinical questions. 
		The second is that I said before.  The second and third questions interact strongly.  The more you're willing to assume, the less certain in a sense you can be unless those assumptions are rock solid.  But if those assumptions are open to question and if your conclusions are going to depend on them, if you're using those assumptions to gather less data or more specific data and not gather other data, then clearly, if the assumptions are wrong, the conclusions may be wrong. 
		Once you answer these questions, to turn the statement I made a moment ago around, study design and data analysis decisions are purely technical within technical constraints.  That is to say, that's when you bring in your experts on clinical trials.  That's when you bring in your statisticians.  With all due respect to Ray and what he's saying, a number of the things that Ray has been saying have been these technical issues.  What I want to assure you of is that there are plenty of technologies around that will allow us, when you give us a statement about how certain you need to be and what you're willing to assume, to be able to minimize the cost of learning those things with that degree of certainty.  And there are ways to do that that I think we need to understand only because we need to have some confidence in them.  But they're fundamentally technical questions that come to us from other experts.  
		So, I don't think, for example, that the domain expert should be someone who says, I want to use a t test or I want to use a model-based analysis or I want the design to look like this.  That's not actually a domain expert question.  These are the domain expert questions, and how you get the answers come to us through people who are experts in answering those kinds of things.
		So, what are the answers to those questions for dose selection?  And these are personal positions.  
		What do we want to know?  I say we want to know something called the response surface and utilities, and I'll elaborate on these more.  But that's what we need to know. 
		How certain do we need to be?  I say, not very.
		What are we willing to assume?  I would say that, given that we don't need to be very certain and the interaction between those two questions, then what we want to do is use as much as possible valid scientific knowledge of the relationship between dose, concentration or exposure, and effect. 
		So, let's get to what do I mean by these things.  Well, a very quick course in decision theory.  Decisions should maximize expected utility.  I'm going to use a little notation.  I'll say D sub i are the possible decisions so that decision 1, decision 2, decision 3, I'll give dose 1 or dose 2 or dose 3.  I'll give drug 1 or drug 2 or drug 3.  Those are decisions. 
		Then those are possible outcomes.  The simplest case, I get blood pressure below a certain level or I don't.  But there are many possible outcomes, toxicity and so on.  So, each one of them is indexed and there are many of those.
		Utility is the subjective value of an outcome.  So, utility of an outcome is what it's worth to you or what it costs you, negative utility.  What's the cost of a toxicity?  What's the value of getting your blood pressure lowered?   Those are subjective in the sense that they are issues for individuals to decide.  If I'm going to have a side effect, it's for me to say, given that you described that side effect, what that's worth to me or what that costs me.  For some people, having a limitation on their mobility might be a real tragedy.  For other people, they may not feel that way.  So, that's subjective, and there's not a lot we can do then as scientists about knowing about those.  We have to know about utilities, but they are not something that we can do as a public activity.  
		Expected utility is the average of the utility across all possible outcomes, each weighted by its probability.  So, there's a formula for it, but it essentially says that if you're going to get a given result that you want, and one treatment will give you a 20 percent probability of getting that result and another treatment will give you an 80 percent probability of getting that result, then the value of the first one is 20 percent times the value of the result, and the value of the second treatment is 80 percent times the value of the result.  The second one obviously is better, not because the result is better, but because it's more likely.
		So, to finally get down to the bottom line of this line of argument, we said, the optimal decision, what dose you should give, what drug you should give, is whatever decision maximizes expected utility, gives you the best average payoff.  The necessary empirical information, that which we can concern ourselves as scientists and regulators, that we can ask drug companies to give us or other sources are those distributions of outcomes given decisions.  What's the probability of lowering your blood pressure a certain amount at a given dose?  What's the probability of given side effect given that you use a certain drug?  Those are the empirical things that, in the process of drug development or, in general, the process of drug use, it's designed to learn about. 
		So, a simple example of optimal dose is let's just say it's a binary decision, treat or not; and a binary efficacy, we get efficacy or not; and a binary toxicity, we get toxicity or not.  Nothing could be simpler than that.  Life is much more complicated. 
		I'm now even going to make it simpler and get rid of utilities.  I'm going to say that the utility of toxicity is the negative utility of efficacy.  I'm essentially saying whatever this side effect is, this toxicity, getting that is about as bad.  In a sense it cancels out the value of getting whatever the efficacy is.  That's no so unrealistic if a drug can produce, for example, a very serious side effect and if the drug is used to treat a very serious illness -- and let's say death is the issue here -- then presumably the utility of death by either means is approximately equal and so it's not unrealistic to think that there are treatments that have this case. 
		But I'm not trying to make that point.  I'm trying to say, let's take the very simplest case.  If this were true, then it would be very simple.  We would treat, if the probability of efficacy given that we treated was greater than the probability of toxicity.  They've got the same utility.  So, if this is more likely than that, then we treat, and if it's the other way around, we don't.  It looks very simple in this very simple case. 
		The point I bring up here is even then here's the problem, that the probability of efficacy, given treatment, is a function of the patient and the dosage.  The probability of toxicity, given treatment, is a function of the patient and the dosage, not just of the decision to treat, and this is complicated.  This is what I mean then by the response surface. 
		That probability function, if you will, in this particular case, for example, talking about efficacy, this would be the probability of efficacy.  This is an axis that I've designed to talk about patient factors.  Obviously, there are many such factors, so it's a great simplification to just have one axis on which I'm going to put all the patients.  
		Then there are multiple dosage decisions, as Tom pointed out.  There's the length of treatment.  There's the frequency of doses.  There's the magnitude of each dose.  I'm just summarizing them all and saying that's dose. 
		Then there is some kind of a surface here that describes relationship between the probability of efficacy, the kind of patient you are, and the dose you get.  In this particular example, a patient down at the left-hand axis gets efficacy very quickly and it flattens out rather rapidly; whereas, a patient at the right-hand end has a little flat area where he's not getting efficacy as we raise the dose, and then he gets it and flattens out at a lower level.  So, for this kind of a patient, the drug is less probable, even in its maximal effect on this individual, to cause the good effect.  Well, that's one side of the equation. 
		Here's another side of the equation, same kind of a picture.  I just made this up, a picture of toxicity which says a person at this level tends to not get toxicity until the dose rises a fair amount; whereas, a person at this level -- maybe it's an old person or somebody like that -- starts to get toxicity earlier.  Notice that this kind of a person seems to get more toxicity at the range studied here, or a higher probability of toxicity, than this kind of person ever gets. 
		So, we can slide this curve over to there, and then we've got the net expected utility in this very, very simple situation.  That would be the maximum expected utility for somebody at this end of the scale.  It's the maximum distance between that curve and this curve in this very simple case.  There it is.  That says that that's the optimal dose for that person.  
		So, the whole thing is all very nice, and the only reason I'm showing you this is not because you could ever do these computations, but because it's focusing on what do we need to know.  What we need to know are those two surfaces in this simple case.
		Another point to be made about this is that for this kind of a patient, there isn't any optimal dose.  Toxicity always exceeds efficacy, so you don't want to use the drug in that kind of a person.  So, if we could know this kind of thing, we could, in principle, settle the problem.  That's what we want to know.  
		How certain do we want to be?  I say, not very.  Why?  Well, we've already heard why.  Not very certain is already the current standard.  Often only three or four doses are tested pre-release, and one of these is almost invariably chosen.  As Carl has pointed out to you, a lot of drugs require relabeling, so it's clear we're not doing very well on this.  So, we only have to do a bit better to justify what we're doing. 
		Also, it's true that for reasonably safe drugs, a wide dose range is tolerable, so you don't have to get it exactly right.  And unpredictable inter-individual variation is always going to make it uncertain for any individual.  Even if you know how to adjust the dose for age or sex or weight or renal function, there will still be unpredictable inter-individual variation, so that we'll never know the perfect dose for anybody until we treat that person. 
		That's the last point, which is that dose titration is a standard part of medical practice so that, in effect, we limit the harm of a wrong initial dose by adjusting it later.  
		So, these are all reasons why we don't have to be perfect here, but we certainly can do a lot better than we're doing and that will have a payoff not only for individuals but, of course, for society as a whole.  A small increment in benefit relative to risk multiplied by the number of people who tend to get antihypertensives can be a considerable benefit to society. 
		Finally, the last question is, what are we willing to assume?  I say, valid scientific knowledge of dose response or dose concentration response.  I guess the serious question here is why am I talking about what we're willing to assume.
		Let me first set up the problem, which I have already been setting up.  There's something called the curse of dimensionality and this response surface.  I made a very nice little picture there.  But there are obviously a large number of distinct dosage decisions, as we've already mentioned, each with multiple options.  There are a large number of distinct patient variables that affect the relationship between concentration and response, and they each have multiple values.  
		So, here's the claim.  It's impossible to study all possible combinations of dosage by patient type variables.  It can't be done.  Empirically it simply cannot be done. 
		So, response surface estimation then requires a parsimonious representation of that surface that's going to interpolate and extrapolate between and beyond what have got to be limited data.  We have got to be able to describe that with less than every single type of patient given every single size of dose and duration of therapy, et cetera. 
		These interpolating and extrapolating functions are the assumptions.  These are the models that we use to cause the surface to be described in a parsimonious way, and if we can't make those assumptions, we cannot proceed.
		The certainty assumption tradeoff then hinges on the scientific validity of those assumptions.  So, to quote one of my heroes here, in other words, nothing is wrong with making assumptions.  On the contrary, they're the strands that join the field of statistics to scientific disciplines.  The quality of those assumptions, not their existence, is the issue.  That's the certainty assumption tradeoff.  That was a quote from a paper by Rob Little and Don Rubin that came out this year.
		Now, how do assumptions help us?  They provide the required parsimonious representation.  We've talked about that already.  They increase the amount of information we can recover from the data, and I'll have more to say about that in a moment.  And they allow us to do what you might call meta-analysis.  They allow us to synthesize information from multiple sources so that all of those various trials that Ray is talking about can be put together and all analyzed together and you can get the dose response.  Ray has just made a proposal for one trial that has several arms that are escalated in different ways, but of course, you could have different trials.  There's no reason why you can't combine them, but you need models to do that.  Again, I'll show you very briefly why.
		Another very important part of this is that they can adjust for missing data and other deviations from protocol that make exploratory or explanatory analyses difficult because you're trying to learn what's really going on despite the fact that people didn't take the doses you asked them to or people didn't show up at the clinic for being measured when you wanted them to.  So, let me just say a few words about those two so that you're clear about that. 
		Models increase the information recovered.  Well, information is basically signal to noise, and in any given study the signal is the total variability in the data and that's kind of fixed.  But the signal is the variation due to identifiable and interesting causes such as differences in dose, and the noise is the residual or unexplained or uninteresting variation that, in fact, make it more difficult to figure out what the signal is. 
		As I say, the total variation, which is what information is, in any given data set is fixed.  So, models can't add information.  They increase information by turning noise into signal.  If information is the ratio of signal to noise, if I take some noise and I move it up into the numerator, I'm getting sort of a double benefit.  I'm making the noise less and the signal greater, and I'm getting more information out of this. 
		A simple example.  The noise in a dose-response relationship due to inter-individual variation in pharmacokinetics can turn into signal regarding the concentration-response relationship if we measure concentrations and if we try to make some kind of a model that relates concentrations to effects.  
		Well, what's the assumption, what's the model that allows us to turn that noise, that pharmacokinetic variability, into signal about concentration response?  It's simply the assumption that drugs act through concentrations, not doses.  If you don't believe that, then this concentration information can never be used to get signal.  If you believe that, then this is a sensible thing to do.
		Turning to the other thing, models adjust for design problems.  Explicit modeling of covariate effects, obviously, sex, age, et cetera, allows pooling of data from different patient groups because now we know how to adjust or we can estimate how to adjust for the different groups.  But as long as we don't explicitly model the effects, then we're comparing old people to young people and we know it's apples and oranges.  
		Serial observations and explicit modeling of system dynamics.  In other words, making time be a covariate that's a part of our model.  It shows up in our model.  The time of the dose, the time of the onset of response.  When we make that be a part of our model, then we can pool data that comes from studies with different treatments and different designs.  And that's very important.  If you observe at different times in different studies, you can't average the response at 1 week with the response at 3 weeks unless you can have somehow time in there and correcting for those features.  So, you have to make time be a covariate.  A model is required to do that. 
		Explicit modeling of variation in error structure allow pooling of data with different precision.  That's a subtle point that I don't want to go into, but it's important for pooling data from various studies. 
		Finally, explicit modeling of deviations from protocol -- adherence and dropout being the major ones -- can avoid biases that such deviations might otherwise cause.  I'll give you an illustration of that in a moment.
		So, here's the illustration.  Everybody who has heard me talk has heard me talk about this example, but it's a good one.  Modeling in the case of Ketorolac and dose finding allows pooling of several trials.  I just mentioned that that was a feature of modeling.  Allows us to avoid some bias due to deviations from protocol.  In this case, it's dropout.  And allows rational dose-finding which I want to end with and focus on a little more than the others.
		So, the standard analgesic trial design is a parallel dose, placebo versus two to four active treatment levels.  You have a pain-initiating event, for example, surgery.  Analgesic dose is given when the patient says they hurt.  Pain relief, which I'm going to call Y at time t, is measured on a categorical scale.  Often 0 is no pain relief, 4 is complete pain relief, and 1, 2, 3 are in between.  
		The difficult part, the part that produces the potential bias is that if somebody is not getting enough pain relief, they can demand rescue medication, and then they're effectively dropping at some time t.  That's the last scheduled observation time because after that, although of course they could be observed, the pain relief is not necessarily due to the agent you're testing but now it may be due to the rescue medication.
		The Ketorolac trials.  There were three of them.  So, we're pooling trials here.  They had different designs, but across all of them, we have these doses being tested and pain relief being elicited at these times.  For example, in one of the trials, it was at a quarter of an hour.  In most of the trials, pain relief was first inquired about at either a half an hour or 1 hour.  So, as I say, these are differences in designs, but we can put them together because we're going to have a model that models the time course of pain relief.  So, we don't have a problem with that. 
		There were a total of 254 patients.  As I said, our data are pain relief score and the time of rescue. 
		Here's just a picture of what the data look like.  These are each histograms.  I have six of them.  This is for placebo, 2.5 milligrams, 10, 12.5, 100, and 200.  Each bar is broken up into dark segments and light segments.  Just look over here for an example.  The dark segment down at the bottom.  These are people with no pain relief, and then, for example, at this time point here, which is 1 hour, you have some people with a little bit of pain relief, more, more, and then finally dark again when you have a lot of pain relief.  So, these are people with complete pain relief up here. 
		What we can see is as we go to higher doses, that the black up at the top tends to get more.  So, the drug is working and certainly at later times.  Maybe in the early times it doesn't seem to be doing very much, which is okay.  There's a delay of action.  
		But a key point here is if you look over here, you see this placebo.  What I've done is drawm the width of these bars to be proportional to the number of people who are still there at the time they're being inquired of.  What you see is that in the placebo group people are dropping out.  These bars are thinner than, for example, here in the 200 or the 12.5 or whatever.  What's happening, of course, is the people who are not getting pain relief are dropping out.  So, if you look at the people who are left, they actually have pretty decent pain relief.  That's almost as large a fraction of complete pain relief there as it is for clearly efficacious dose.  So, you can't just take the average.  That's the problem of bias that I talked about. 
		Well, we have an analgesic model.  There's a picture of it, and it's got really basically the following parts.  It's saying that the drug dose comes in and turns into concentrations, which eventually become a concentration at some active site, and that influences the pain relief.  At the same time, we've got some kind of a placebo or time evolution of pain that's causing pain, and these two are conflicting with each other.  So, we get to observe the green pain relief.  This model also entails the notion that there are individual effects, this gray bar here for randomness, that individuals differ with respect to their pharmacokinetics and with respect to their pain course. 
		Then the model says that the degree of pain relief will influence whether or not you drop out, and I've got dark gray here to suggest that there's variability in this part too.  These are probabilistic models rather than deterministic models. 
		What are the key assumptions that we require to analyze the data with a model like that?  That drug acts only through its concentration.  Notice that there's no line from dose to effect.  It goes through concentrations.
		Individuals are consistent over time.  The idea here is that individual effects are a characteristic of individuals, but they are persistent for that individual. 
		Individual differences affect baseline and drug kinetics and dynamics.  That's just saying that the individual effects here affecting this part, the natural time course, and they're affecting various parts of the kinetics. 
		The important assumption here is that this pharmacokinetic model is linear.  So, if I double the dose, I double the concentration.  That will allow me to extrapolate to other concentrations, but it doesn't say that the pharmacodynamic part is linear.  It's not true that if I double the dose, I double the effect. 
		The dropout depends only on the observed level of pain relief.  This is a technical point which allows me to deal with this bias.  So, I have no other arrows to drop out from these individual effects, for example, or from anything else.  It just says, you're going to drop out depending upon whether or not you have adequate pain relief.
		So, here's the basic result.  On the left-hand side, I've got the predictions and the observations of the pain relief.  So, these are the data that are observed.  Each of these four pictures is a kind of response surface, but it's a probability.  So, this is the picture of the probability that you'll have some pain relief, that is, pain relief greater than 0, and this is that you have greater than 1, so on, all the way out to here, that pain relief is greater than 3, or in other words, this is the complete pain relief model.  The z direction, the vertical direction, is the probability that you will have pain relief, as I say, here greater than nothing, some pain relief here, complete, and there in between.  Along this axis I have dose and along this axis time.  So, this is the response surface for dose and time against probability of pain relief.
		The circles are an attempt to estimate these things directly from the data.  They are simply, take the number of people who had pain relief 2 at dose 100 and take the fraction of people who had some pain relief and plot that along that probability axis.  
		The surface is what's estimated by the model.  But this is estimated for the observed data.  So, this is the probability, for example, here of some pain relief given that you're still in the study, that your dropout time is greater than the time at which you're being observed.  What you can see here is that you have generally sort of a flat curve.  That's not the curve we're interested in.  What we want to know here is what would happen if you didn't drop out.  What is the drug actually doing, and that's estimated from the model.  I can't put the data on this curve because the data don't conform to this situation of no dropout, but you can pull that out of the model.  
		For example, it shows you, if we just take one of the faces here, the placebo effect, which looks like it's rising in time.  You saw that on the histogram.  Is it really rising in time?  According to the model-based analysis, it's staying flat.  It's just that these are people who are dropping out.  So, that's why the level is going up here.  When we get rid of the dropout factor in our predictions, then we can see that there is a placebo effect, but then it flattens out. 
		Similarly, if you look over here, you can't really discern a dose-response curve, a difference in response at 6 hours depending upon the dose you receive, because we flattened everything out because of these dropouts.  But here it's pretty clear that there is a dose-response curve and, interestingly, the dose at which we seem to get this sort of a maximum response, about 50 milligrams.  So, here's a case in which the trials were done at doses well in excess of what turned out to be the maximum effect. 
		Now, let me just end with the dose-finding part.  The optimal dose is often considered to be the minimal effective dose, and that's generally defined to be the smallest tested dose such that the null hypothesis of efficacy equal to placebo or no efficacy is rejected. 
		Now, Ray said we can just forget about that definition, and maybe it is a straw man, but it is certainly one that has been used a great deal in the past and I wouldn't be surprised if it continues to be used.
		Well, the problem is that the minimum effective dose, defined in this way, is design dependent.  It isn't something that depends on the science.  It isn't something that depends directly upon the underlying situation.  It depends upon the design.  It depends on study size.  If I make a bigger study, I can obviously find a significant difference with a smaller dose.  But why should the dose we suggest depend upon the size of the study we use to determine it?  It must be one of the tested doses.  You can't find some dose that you didn't test.  So, again, why should the best dose be one that we happen to choose to test?  It must be a single dose because that was the design of all the studies, one single dose.  
		But why should it be a single dose?  Maybe the best way to give this drug is a dose, and then a couple of hours later, another dose, and a couple of hours later, another one. 
		So, this is an inappropriate use of the confirming paradigm of testing a hypothesis when we should be learning.  And what is a rational approach to this?
		Well, let's define these things we talked about earlier, efficacy and benefit and harm.  So, let's take the outcome that we want to maintain a certain minimum probability of pain relief greater than some value.  So, K is some value.  That could be 3, in which case we want to say we want to maintain a minimum probability we have complete pain relief.  But if K is 0, which is the example I'll show you, then we're saying we want to maintain a certain minimum probability that we have some pain relief between certain times and we're going to put a large utility on that.  
		Our "toxicity," since we didn't have any toxicity in this study is just dose magnitude.  What we're going to say is the bigger the dose, the more it costs us.  We're not going to big disutility on that, so that the combination of these two things will essentially yield us the smallest dose that will produce the minimum relief that we want between those times.
		Patient factors.  There won't be any.  No difference between men, women, or whatever.  But we could have put them in there. 
		The response surface that we're going to use is the one that I showed you a picture of.  It's the efficacy outcome obtained by a simulation from the model of the probability of pain relief being greater at any given level, and we have that model because we fit it to our data.
		To make things clear that we don't necessarily have to use that minimum effective dose kind of idea, we say, let's find the two-dose regimen where we give one dose at time 0 and we give another dose at some other time.  What we want to find is that other time and the value of that dose plus the value of the time 0 dose.  So, there are three things we need to find out.  Two dose sizes and the time of the second dose, something we never studied.  And we want to find it such that the probability that we have some pain relief at all times between 1.5 and 8 hours is no less than some minimum. 
		Before I get to that minimum, let me just talk about the 1.5 to 8 hours.  There's a time lag in the effect, so we don't see any pain relief before an hour.  So, it doesn't make sense to make that earlier, and I chose 8 hours just to go beyond the data.  The data in this study only went to 6 hours.  I just want to show extrapolation.
		Now, what's the level of pain relief?  I want to have a greater than 70 percent probability that I have some pain relief.  Why?  Because that's the value that you get between that time from the model for a single dose of 200 milligrams given at time 0.  That was the largest dose given.  So, I want to match the efficacy of a single dose given at 200 by giving two doses at different times. 
		Here's kind of a picture of the initial dose versus the second dose.  What this says is if I give 60 milligrams to start with and 60 milligrams at 6 hours later, I'll get just a good a result as if I get 200 at time 0 over 1.5 to 8 hours.  If I give about 50 milligrams at my initial dose or 40 and then 30 or 20 at 5 hours later, I get the same result.  And if I give 20 at time 0 and 40 at 3 hours later, then I'm giving a total of 60 milligrams, and I'm getting just as good a result as if I give 200 at time 0.  The winner actually is 4 hours.  If I give 20 at time 0 and 20 at 4 hours, I get just as good a result at a total dose of 40 milligrams versus 200 milligrams if I use it that way. 
		Now, if you think it's better to give 40 than 200, here's an example of a rational approach to dosage.  You don't have to use these same criteria, but this is the idea.  What do you want to know?  What are you trying to achieve?  Now, let's use the model to tell us how we get it.
		So, optimal dosage decisions maximize expected utility.  Utilities are subjective values of outcomes.  Expected utility is an average over outcomes, each weighted by the probability.  And the set of probabilities is the response surface.  It's a function of dosage regimen and patient features.  It's derived through experiment and observation. 
		I have not shown you, as Ray promised that I would, how you can analyze an escalating dose trial and find out about those models.  I showed you an arrow dose trial.  But again, that's a technical issue.  It can be done.  You use these things called mixed-effects models, and you can do it. 
		Response surface estimation is best viewed as a learning, not a confirming exercise.  You can't study all the possible combinations of regimens and patient types, so we have to interpolate and extrapolate.  I showed you interpolating and extrapolating as an example of doses for the Ketorolac.  We went beyond the time of the experiment and we used two doses rather than one.
		PK/PD model-based learning designs analysis can do so by making scientifically valid assumptions.  Assumptions reduce inferential certainty because if the assumptions are wrong, the conclusions are wrong, but fortunately dose-finding doesn't require great certainty.
		Thank you. 
		DR. PACKER:  We'll take questions from the committee.  Any points of clarification specifically focused on this presentation or any residual questions to Ray's supplemental presentation?  Tom?
		DR. FLEMING:  I've got a number of thoughts.  I think much of what Lew has said I find myself in agreement with.  I would maybe refine some of the perspectives, at least as I think about them. 
		Just as a quick aside, even this isn't the main focus, Lew had pointed out the importance of three questions -- what do we want to know, how certain do we need to be, and what are we willing to assume -- and pointed out the distinction between domain-specific experts and technical experts and more or less indicated that the domain-specific experts are the key people for answering those three questions, and once those answers are in, the technical experts are then charged with essentially the scientific modeling and analysis of data.  
		In a real sense, I would agree with how I would allocate primary responsibilities, but I would argue that the domain-specific experts need to have at least a given level of understanding of what the technical experts are about so that they have at least a general sense of the reliability of the analyses by the technical experts and how much the model assumptions can impact on that reliability. 
		Conversely, I would say the technical experts really need to have a base understanding of the clinical situation since they're the ones who really understand the robustness of the results from the modeling, they have a general sense of what makes reasonable justification in terms of the answers to those questions.  In fact, I've often heard it said that in an ideal situation if you sat down at a meeting of the technical experts and the domain-specific experts addressing issues such as this, you ought not be able to tell initially which is which because both disciplines should have a given understanding in order for this synergy to really achieve its maximal.
		Relative to the -- 
		DR. SHEINER:  Can I just make a quick response to that?  Because I think we are pretty much in line. 
		There are two best of all possible worlds.  One of them is the last one you described, which is that you can't tell the difference.  I don't know that that's achievable.  The other best possible world, of course, is that everybody is absolutely clear on just where their expertise is and makes completely clear to their counterpart just what their assumptions are.  
		Because that's also impossible, that's why you have to have what you were talking about, Tom.  That is to say, you need to be in the same room together because there are always hidden assumptions that we just kind of go on and don't even know we're making that the domain expert will make or the statistician will make.  Each one has to be able to say to the other one, how come we got there from there?  What did it take to do that?  But as I say, there are these two alternative best possible worlds, and if we could be in either one, that would be fine.
		DR. FLEMING:  To move to the other aspects of the presentation, it is exactly as you point out, Lew, critical to be able to augment the actual data with modeling in order to try to glean as much insight as possible from the data. 
		I would agree very much with what I think I heard you say toward the end, and that is the amount of reliance that we're able to make on modeling depends on whether we view ourselves to be in an exploratory versus a confirmatory stage of the overall development.  I see, exactly as you pointed out, the IIb stage as a critically important stage for exploration and, in turn, an invaluable contribution to that is modeling. 
		Of course, not all modeling is the same.  Certain models carry with them stronger assumptions and certain types of modeling assumptions are stronger.  It's important to distinguish those models that are making very strong assumptions that in fact in a certain sense can give us broader insights, but those insights may be at some risk in terms of their reliability. 
		So, to be more specific, if we're talking about modeling covariate effects or modeling for dose response, I think many of us are more comfortable if our conclusions are based on interpolations as opposed to extrapolations.  So, for example, if we're looking at the effect by age and we're really interested in the effect in 70-year-olds and our data is only on effects in 20- and 30-year-olds, then that extrapolation puts us at much greater concern than if the data is looking at results in 20-year-olds and 70-year-olds and we're interested in 50-year-olds.  We're interpolating rather than extrapolating. 
		And same issue occurs in dose response.  If we're trying to understand this dose-response surface and we fit a model and we only have information on lower doses, I'm very reluctant to extrapolate to a high dose, whereas I'm much more willing to interpolate. 
		And this same issue arises for looking at models over time as well. 
		This issue is actually something that occurs across a wide range of challenges that we face.  Do we need to randomize?  Well, technically we can adjust for differences between intervention and control groups based on baseline covariates that we would have.  Unfortunately, those baseline covariates are just the tip of the iceberg for what explains the differences, and hence, that's the concept of extrapolation.  That's the reason we argue we have to randomize because the covariates that we have for adjustment are just a fraction of the totality of covariates that we would need. 
		Is missing data important?  Is it informative missingness?  Well, if we can explain the mechanism of missingness by covariates, then we can fully adjust for the potential bias.  But it's the same situation as randomization.  We only know a tip of the iceberg of those covariates that truly explain the nature of missingness in most situations. 
		Surrogate endpoints.  Why is it that we often don't accept a surrogate endpoint?  Well, if we understood the causal pathways by which the disease process influences the outcome and those causal pathways are entirely mediated through the surrogate marker of interest and there are no unintended effects of the intervention, then we can rely on the surrogate.  But again, it's an extrapolation problem.  We're concerned that the effects of the disease process are only partially mediated through the biologic marker of interest, and there could be unexplained covariates. 
		So, basically what I'm trying to get at is the fundamental scientific principle here that very often we have abilities to set up models and those models are models we feel much more comfortable trusting if we are, in essence, interpolating as opposed to being forced to extrapolate.  It's the reason I have trouble with non-randomized trials or with missing data or with reliance on surrogate endpoints, or in the case of discussion here, fitting dose-response surfaces or looking at effects over time or looking at effects by covariates.  If I have data and I can interpolate, I'm going to be more comfortable than if I have to extrapolate.
		But my level of comfort is much greater is you're asking me to do this in an exploratory approach.  So, what you've laid out makes perfect sense for me, especially in a phase IIb trial where I can, in fact, arrive at the strongest possible conclusions from the data that you have given to me, the best insights possible, but then I can come back and confirm that in a confirmatory trial.
		DR. SHEINER:  All the principles that you enunciate are ones that I agree with, but I think there is a disagreement and I would like to sharpen it. 
		I think the basis on which you trust making a prediction of an unobserved result, interpolation or extrapolation, is not whether you're interpolating or extrapolating.  The basis is the credibility of the assumptions required to make that prediction. 
		Now, interpolation requires, especially if it's narrow interpolation, only the assumption of smoothness.  It doesn't require any knowledge of process.  Almost everything is smooth.
		DR. FLEMING:  Or at least monotonicity.
		DR. SHEINER:  Yes, monotonicity or something like that. 
		Whereas extrapolation, especially distant extrapolation, requires that the basis on which that extrapolation is made have actually credibility in terms of -- you're remodeling the real world -- the real world.  So, I'd say it's how good the science is, and the fact that you can rely on smoothness or monotonicity in many fields and you don't have to think too much about it does definitely make you feel better. 
		But I think the only solution to this problem, which is going as far as you can go with information, giving people the best information about how to dose patients, the only way you're going to get there is by carefully looking at which scientific assumptions you need to make, making sure you've got those straight, and then letting that make you be comfortable with your extrapolations. 
		So, again, the distinction between the exploratory and confirmatory for me in this particular instance is one in which we cannot possibly confirm every possible combination of dose and patient.  So, we are going to have to be left, in the end, making recommendations based on exploratory or explanatory statements.  Therefore, our obligation is to make those as good as possible within the constraints of how much money we're willing to spend, how much time we're willing to spend, and that means getting the science straight.  That's the key thing. 
		In the end, the label will be written with respect to dosage based on models or it will be written based on empirical observations, in which case it will be totally inadequate because it will be a dose you tested which you didn't know whether it was the right dose, and it will be a pattern of testing which you didn't know was the right one.  So, we've got that tradeoff.  
		We want to be certain that we know what the dose does.  We have to suggest one of the doses we tested.  On the other hand, there's no reason to believe, as Ray has shown us very nicely this morning, that the doses we tested are going to be the best doses. 
		So, I claim that, no, we can do better than that.  We can do reasonable interpolation and extrapolation and we will have to argue about that.  We will have to reach a compromise, and it will be the domain experts ultimately who will say I trust that assumption.  You can't block the beta system any more than completely.  Consequently, this thing is flat after a while, whereas here's a drug that operates by unknown mechanism and I'm not exactly sure that it's going to be flat within some reasonable range.  All kinds of things like that.  
		So, I think that's a difference in that I'm saying we are really going to have base the best possible recommendations on assumptions that we have not demonstrated in this case in detail. 
		DR. PACKER:  I just want to clarify.  Tom, what I did not hear Lew do was advocate a departure from randomized trials which would address many of the assumptions that you're most uncomfortable with in terms of accounting for unknown baseline covariates.  I didn't hear, Lew, that you advocated such a departure.
		DR. SHEINER:  No.  Again, in the general case, I'll advocate using totally observational, uncontrolled studies if that's the best you've got and that's all you can do.  I always want to learn the most I can because I've got to make a decision.  You have to make a decision.  And to base a decision on less than everything you know is a mistake. 
		But randomization.  Absolutely crucial.  Controls.  Absolutely crucial.  Nobody is suggesting that we abandon those.  Nobody. 
		DR. PACKER:  My sense is that the preservation of randomization minimizes a number of assumptions you make that are of low quality.
		DR. SHEINER:  I think Don will talk about this more.  The more you have departure from protocol -- so, let's say people don't take the doses that you prescribe and yet you still want to know what happens when they do take the dose -- then you have to use those actual departures -- and they break the randomization somewhat because you didn't randomize for people who were going to not comply.  They just did it or didn't.  So, if you strictly hold to the randomization, then you get certain nice features, but you answer what I consider to be the wrong question.  The use effect in this question rather than a method effect in this question.  What happens when you prescribe the drug rather than what happens when you take the drug. 
		On the other hand, if you don't start with randomization, you don't have a chance, and if you can design protocols that patients accept so that they follow the protocols, then you've got the benefits of both.  Then you can do a nice explanatory analysis, if you design it well, and you've got the benefits of the causal inference that Donald talked about more that's based on randomization.  But serious causal inference has to be based on randomization if you've got any doubt at all about the underlying assumptions. 
		DR. PACKER:  Carl?
		DR. PECK:  Well, I had wanted to comment on Lew's use of the word "assumptions," and I'll do that but Tom's comments actually allow me to say this in the context of his. 
		What I want to say was contemporary drug development programs, ones that are being run right now and the ones with clinical trials appearing in NDAs, almost all of the trials are randomized.  So, the issue of randomization is really not an issue except in oncology.  Oncology is sort of in a class of its own, not employing randomized trials except perhaps in a single phase III, and that's a special problem. 
		But when we count the 30 to 50 to 80 clinical trials in programs we're looking at, three-quarters or sometimes 90 percent or sometimes every single trial has had a randomized phase in it.  They're almost all blinded.  So, the quality of data coming from these clinical trials is very high from the point of view of randomization and blinding. 
		So, the point I wanted to make is the "assumptions" that can be available at the time of the IIb trial, exploratory dose-response trial -- typically there will be 30 to 50 trials that have preceded that trial.  There's a huge database from randomized, blinded trials, surrogate endpoints rarely, biomarkers that have putative linkage in a mechanistic way, often definitely clinical adverse reaction data that, if properly analyzed and integrated, provide a very strong set of "assumptions" that I would -- I don't like the word "assumptions" at this point because I think that they're so data-informed that they reflect a reality that's more than just an assumption pulled out of the air. 
		But I don't think we need to worry about the lack of randomization.  It's the rare clinical trial today that's not randomized. 
		DR. FLEMING:  Just to come back to my earlier comments, I'm using the concept of randomization to simply illustrate the point of where many of us have drawn a line in terms of our willingness to extrapolate.  So, specifically the focus of the discussion, as I see it today, is on the model assumptions associated with dose-response surfaces, as well covariate effects and how they influence those dose-response surfaces.  
		But I wanted to point out that we obviously in our research strategies make model assumptions in many other ways or are unwilling to make model assumptions in other ways. I used as examples of those other ways randomization, handling of missing information, and surrogate endpoints.  My comment is, in those three specific instances, the reason many of us believe in randomization and in the need for minimizing missingness of information and using clinical endpoints is the alternative of using non-randomized trials and models for missingness and surrogate endpoints requires very significant model assumptions that are more in the spirit of extrapolation.  I.e., we're having to make assumptions beyond the covariates we have in a model to specifically model the necessary relationships. 
		So, using that as a guide, I was arguing that, in the context of our discussion today, which is mostly focusing on dose-response surfaces and covariate effects, I'm much more confident with making conclusions about given safety and efficacy at doses if I'm interpolating as opposed to having to extrapolate.  Every single model that Ray put up at the beginning of today assumes monotonicity overall.  It's not necessarily the case that efficacy is monotonic.  I'm more comfortable, though, with an assumption of monotonicity when I'm interpolating than when I'm having to extrapolate. 
		DR. PECK:  Well, my comment about interpolation and extrapolation and the interpretation of phase IIb data is this is sort of an accident of the moment where we have incompletely analyzed the available information.  You cited, for example, you'd be uneasy about extrapolating beyond the maximum age that had been studied, say, on a phase IIb trial.  Many of us would be bolder than that because we know enough generally from these trials about what the mechanistic elements of the aging process are that are affecting drug disposition and pharmacodynamics. 
		So, if your maximum age is 50 and you're wondering what happens in the 80-year-old, you can use data-informed modeling to understand that between 50 and 80, your creatinine clearance is going to naturally decline another 20 to 30 percent, that the drug is excreted primarily in the urine, and that there's a dose and concentration effect related to side effect profile.  You can certainly expect a higher side effect profile in the 80-year-old given a common dose and the expected higher blood levels.  So, I would not feel my hands tied at extrapolation in phase IIb trials, given that the data that preceded it is adequately incorporated.  You see, it's not a frequentist situation at the end of phase II.  You are standing on a mountain of data and potential knowledge if you extract it.
		DR. PACKER:  I don't want to speak for Tom, but I think what he's saying is that he's not as convinced that we are that smart.  I think Lew's point specific to this is the fact that the extrapolation requires assumptions from domain-specific experts who aren't that smart either.  In fact, there may be other determinants of how an elderly patient responds to a treatment other than the fact that there is renal impairment in the elderly and one can measure blood levels in renally impaired as a function of creatinine clearance.  There are millions of changes that occur with aging that can affect response and renal excretion is only one of them. 
		My sense is that Tom is a little bit concerned that an extrapolation based on a single biological determinant just isn't good enough to allow for this, and if domain-specific experts say that that's the primary determinant, then the answer you may get may, in fact, be the wrong answer.  I hope I said that correctly.
		DR. SHEINER:  This is such a nice example, this whole thing, what we're doing now because it just is so clarifying, the difference between confirming and learning.  It's clarifying the difference between the attitude you take when there is no cost to being right or wrong, so to speak.  That is to say, you're just trying to figure out how the world works.  And science I think appropriately protects scientists from having to pay a big penalty if they make a wrong guess.  Well, that's it.  You still get paid your salary and somebody corrects it and you keep on going.  
		But we're now talking about making suggestions.  So, what's the competition here?  The competition is either saying nothing about the effects of age, in which case you are leaving people who are less domain expert than the domain experts that you're able to talk to to fend for themselves, or demanding that age be studied.  And then somebody else will come along and say, well, sex has to be studied and now this has to be studied.  And there's got to be a stop somewhere.  
		Either you study everything a little bit with a few people, in which case you've got to be very Bayesian about how you interpret those data because those are very few points, so you have to put them in a very strong assumption-rich context in order to do that.  Maybe you want to do that.  Or you put a huge effort into studying a bunch of old people and a bunch of young people, and then you don't study a bunch of women and a bunch of men. 
		So, you've got no choice.  You've got to make a recommendation.  It seems to me that backing off and saying, well, a million things might happen here and we don't know them all is actually doing a disservice to the public because you know -- you being the manufacturer, the experts, et cetera -- as much as there is to be known, and it's your obligation to communicate that in as clear a way, with the right hedges, as possible.  That's your obligation.
		Now, it's quite different than your obligation as a scientist if you're just trying to find out knowledge.  Then you say, look, I didn't nail this one down.  So, this hypothesis remains open.  I encourage other people to investigate it. 
		DR. FLEMING:  It certainly is exactly as you say, Lew.  The far and away most preferred strategy is what I call to be inclusive when I'm looking at eligibility requirements in clinical trials and, as has been advocated here, inclusive when you're looking at ranges of plausible doses or schedules or schemes that could achieve optimal benefit to risk.  
		I think sponsors are well served by trying to achieve that inclusiveness.  Those insights can readily in many settings lead to a better choice of dose and schedule that would optimize benefit and reduce risk.  By looking at being inclusive in eligibility criteria, it allows us to gain insights about whether, even if we can't say that they are definitive insights about subgroups, at least it gives us the opportunity in phase III trials to generate clues because, as you point out, ultimately we're going to have to make decisions as caregivers as to whether to use an intervention and if so, in what way. 
		From a regulatory perspective, I would say that considerations are also very consistent as they would be from the perspective of the clinical community and the insights that you'd want to know.  Ultimately, though, we want to be able to have evidence that we view to be adequately convincing or conclusive that a given regimen, as it's being proposed to be delivered, is in fact efficacious and has a safety profile that's consistent with a favorable benefit-to-risk profile. 
		DR. SHEINER:  Absolutely.  I'm not advocating abandoning having to have some empirical evidence that something you did worked out in the world in fact as observed better than if you hadn't done it or the control.  I absolutely agree you need confirmation.  It's just the next thing is what do you put that within.  I don't think we disagree.  
		I think actually a continuing dialogue on this, we'll see, in the end comes down to that slide that I took from Don.  It has to do with the quality of the assumptions and that interaction between people who understand the implications of assumptions and the people who know the field well enough to know how much credibility to put on them. 
		DR. PACKER:  I think that this would be a very good time to take a lunch break.  Let us take a break for about, say, 35-40 minutes.  Make it 35.  
		DR. LIPICKY:  We can't do it in 35. 
		DR. PACKER:  We'll reconvene at 1 o'clock, and at 1 o'clock we'll start with Dr. Rubin's presentation and move forward. 
		(Whereupon, at 12:21 p.m., the committee was recessed, to reconvene at 1:00 p.m., this same day.)

	AFTERNOON SESSION
	(1:15 p.m.)
		DR. PACKER:  We're going to resume by asking Don Rubin to cover the topic "How Can We Know What We Want to Know?"  Don, why don't you proceed with your presentation.
		DR. RUBIN:  I don't have any slides or transparencies to show you.  I'm going to be just speaking.
		We began with Carl Peck discussing whether even the analyses of current data that are collected are adequate for the purpose of decisions about dose response.  Carl basically came to the conclusion that the current analyses were not.  Even though the trials are often designed as dose-response trials, they're analyzed in this traditional, null hypothesis testing dichotomy which did not yield very helpful information for many decisions. 
		Then Ray extended that to say maybe the data that are collected are inadequate, inadequate in the range of doses, and suggested that we really should be doing trials with a greater variety of dose ranges than we have now. 
		I tend to agree with both those positions. 
		Lew has added support for those positions, showing you explicit examples and discussing how you might analyze the data.  
		I'm just going to make some general kind of comments from a statistical point of view.  I'm relatively recent into this area, but I have gotten involved in clinical trials and the complications in clinical trials recently, complications such as missing data, noncompliance, and how to analyze trials in that context.  I guess at the outset I should say that I completely support the idea that randomization should be done.  We're not talking about dropping randomized trials at all.  But I think we will be talking about trying to do a better job of analyzing the data from randomized trials and more flexible randomized trials than we have now.
		One thing that I want to mention is that there's a context that I think is fairly interesting about the causal inference randomized trials beyond drug development.  The context I'm thinking of right now is in social science.  A couple weeks ago, I was at a conference dealing with social science, and one of the topics that was discussed was the tremendous advances that have been made in social science in thinking about causal inference.  In that context, David Cox was there and I had a chance to talk to him about something.  I think it's kind of interesting. 
		When do you get to use the word "causes" in dealing with data?  He had this example.  He said, let's suppose you have a huge, randomized experiment with no complications at all.  There are no complications with missing data, dropout, noncompliance, nothing like that.  And there's an obvious effect shown.  Do you get to say that the treatment causes the outcome?
		Well, the interesting thing was that David said that he really wanted to see some evidence of mechanism before asserting that the treatment caused the outcome.  Now, why is that?
		I'm not sure when you want to use the word "cause" or not.  I'd be inclined to use the word "cause" and just say I don't understand the causal mechanism. 
		But David's point was that when you're dealing with a trial in a particular context, what you're really interested in is in generalizing outside that context.  Unless you have some understanding of the mechanism involved, you can't really assert how well it might generalize. 
		That's a comment, I think, on intention-to-treat analysis as well.  
		Why do I say that?  Well, intention-to-treat analysis is very much viewed as a confirmatory kind of analysis.  You confirm the results of it, but how well do you know that the results that you see can generalize?  If you think about the situation with noncompliance, for example, when you have noncompliance in a trial, as we all know, the standard way to deal with it is an intention-to-treat analysis.  The problem with intention-to-treat analysis is it bundles together personal behavior and compliance behavior in this trial with a scientific effect of the drug.
		Now, I'm not arguing for as-treated.  I'm certainly not arguing for that kind of analysis, nor am I arguing for per-protocol, nor am I arguing for anything simple like that.  But I am arguing for trying to disentangle the effects of behavior, which probably will not generalize outside this trial, to the effects of the drug, of actually taking the drug, which probably will generalize outside the trial.  And I've written some recent papers on that, which I hope have had some influence.
		The thing that you expect to generalize beyond a trial is the science.  The scientific part is in small steps.  That's an important idea, that the small steps, the small things are the things that you have a better understanding of.  
		The words "confirmatory" are a little ambiguous.  I think they promise more than they really deliver.  If we are in a perfect randomized trial, why do these results apply to people in the future?  Do we really have a random sample of people in the future?  No.  Why do we think they'll generalize then?  Well, because we have some confidence and smoothness of the world, that the effect of the drug is not going to be affected by ozone layers or ozone counts or something else like that.  So, we really never have the ideal randomized experiments because randomized experiments would take units from the future.  So, we're always relying for the generalization on some kind of science. 
		Now, the information that we require for approval of a drug is a dichotomous decision.  You eventually have to approve it or not approve it.  That information is very different from the information that's used by docs for making decisions because those decisions are not dichotomous decisions.  They're not up or down decisions.  They're not yes or no decisions. 
		One of the principles of statistics is that you should avoid summarizing data, avoid making summaries along the way until you really must, until you're really forced to do so.  That's part of the decision theory, the analysis that Lew has put up, that you want to weight things by probabilities of their occurrence and you want to wait till the very end until you make a decision.
		The issue is that as far as the use of dose-response trials by docs, they never really do have to make a dichotomous decision until the time when they make a recommendation for a patient. 
		So, what is a real ideal labeling for a drug?  Lew has pointed out the response surface.  I think what we really would like to do, if we could, would be to have a real response surface that would have something like the following.  We have all these covariates that describe the patient and the situation.  At each value of these covariates, you'd like to have the expected value of the effect, the variability of the effect for patients of that type.  You'd like to have the expected number of side effects, the type of side effects, and you'd like to have standard errors, measures of uncertainty about all those things.  
		Ideally, that's the information that you'd supply with an approved drug, not that the decision to approve or not to approve has to be continuous, but the information that you provide will be this continuous kind of information.  That information is very, very detailed.  I don't know whether we can ever move towards that, but that's the kind of information that would be most useful in actual prescribing use.  It's really this ideal of individualized medicine, very personalized medicine.  Ideally that's what a label would be in some generalized sense of what a label is. 
		I'm sort of cycling around here.  I hope I'm not cycling around too much. 
		What's the purpose of a trial?  Is it just to test the null hypothesis or is it to summarize evidence?  Now, I think many of us would think that ideally you would like to summarize evidence in the trial about the efficacy of a drug, and that means the whole dose-response curve.  That's something that I think we heard pretty much agreement on, that that would be the ideal thing to do.  That's sort of a Bayesian perspective.  It's the whole modeling perspective.  You want to summarize what we know about the drug and not test the null hypothesis. 
		When you think about testing a null hypothesis, it's actually not a very interesting thing to do.  If you get a very significant p value, what does it really mean?  Well, what it really means fundamentally is I have found a model that doesn't fit, and it really doesn't fit.  And the more significant, the more convinced you are it doesn't fit.  And that's not very exciting.  
		Why are we so focused on that?  We're focused on it because usually we have an alternative model in mind that we're willing to accept in place of the null.  But you have to admit it's not very interesting to just say I've found something that doesn't work that is a significant result.  I found a model that doesn't work. 
		So, why do we do that?  Well, it's a pretty easy question to answer, and I think the tradition of statistics, as it's evolved over the last half century, has been focusing on questions that are easy to answer and not answering the right questions.  I think John Tukey once said that it's better to address the right question, even if it's a poor answer, than to address the wrong question and get a good answer for it.  I think what statistics as a professional field has been too oriented at doing is getting a precise answer to the wrong question, or often the wrong question, the precise answer to a too simplistic question. 
		Now, if you address the more general questions, questions that are of interest, you often do get somewhat non-robust answers to the real question.  Now, this is a point that Tom Fleming made.  There are different questions.  If they're more like interpolation, you certainly get more robust answers than if they're more like extrapolation.  I think the difference between interpolation and extrapolation is kind of difficult to address in many cases.  In some cases it's pretty easy.  In the case that Ray was describing where he showed plots, some of those plots where you were extrapolating to beyond a dose that you had ever seen.  That's probably like extrapolation, although Carl had this example of age where there's other data that you have available that would suggest you really pretty much know what's going on there.
		But I do think if you take the attitude that the right thing to do is to try to get an answer, even if fairly non-robust, to the right question, that you actually do better in real practice with that. 
		So, what am I saying?  Well, if we accept the objective as to try to get this response surface which is a multivariate response surface, it's the expected value, the variance of the effect, the side effects, as a function of all these covariates and dosing levels, that's a hard question to address and you'll have to make a variety of assumptions when addressing it.  But it is most related to practice.  It is a question that really does address how practice works. 
		If you do get a non-robust answer to that question, what you often will find is if you're looking to that formulation, that response surface to address a simpler question, an average of some kind, then when you average over that function, you will get robust answers.  The margins of a high dimensional surface are often very well estimated and very robust even if you're using a model to do that.  So, what I'm saying is that if you use a model to try to address the right question, that you'll often get a great deal of uncertainty in the answer to that right question, but then when you integrate over that model to ask simpler questions, you'll get robust answers to those simpler questions.  The averages are robust.
		But for actual clinical practice, you'll get no worse decisions.  You'll get better decisions, in fact, than if you ignored the right question and provided no evidence at all, no argument at all for what the right question should be because then you're left -- I think this point came up at the end of maybe Lewis' talk.  If you just do not provide information, then the user, the doc has to make decisions based on no information at all.  If you can provide some information, it's better than none.
		Now, I want to say a couple of other comments. Then I'm going to close.
		I did say that no one is saying don't do randomized trials.  Absolutely, we have to do randomized trials.  In design, you must do it.  
		However, how do you deal with missing data in a randomized trial?  Well, you really can't if you're going to try to do it in a completely robust way and try to put in worst values or something like that, but that's not getting at science at all.  When dealing with noncompliance, a similar kind of problem.  When you deal with surrogate outcomes, you have a similar problem. 
		Where I started out with the David Cox comment was the fact that things that generalize best are small pieces of scientific understanding.  In order to decompose those, even in a randomized trial, you're going to have to rely on models to do that.  I think surrogate outcomes is an area which is very poorly understood.  I think future technical statistical work will have a big effect there in understanding how to do that.  That's in parallel with understanding, dealing with noncompliance in clinical trials, how to decompose the effect into the behavioral effect on compliance and the effect of the drug.
		I think that's all the comments I really want to make, and I will be happy to take any questions.  I think that there should be some. 
		DR. PACKER:  Any questions from anyone on the committee?  Actually we can open up for any of the presenters today.  Ray?
		DR. LIPICKY:  I'm just curious to know how you define scientific.
		DR. RUBIN:  Well, I'd say that's obviously a tricky question.  Scientific really means there are relationships among variables that we think we understand in kind of a mechanistic way, not necessarily really a deterministic way, but a mechanistic way.  Things such as reasons for dropping out of a study are not part of science.
		DR. LIPICKY:  Right, I understand. 
		DR. RUBIN:  They're reasons for behavior, not part of science. 
		DR. LIPICKY:  So, let's say that I have three sets of data that are really tables of random numbers, but I can deduce from each set something that the data says to me and I can somehow or another relate those three data sets to one another.  Do I have a scientific piece of stuff?
		DR. RUBIN:  I'm not sure I understood the question.
		DR. LIPICKY:  I have three tables of random numbers.  I've done an experiment, and I have three tables of random numbers.  I look at the first table and I say, I have dose and effect here and I can make something out of that.  But really the relationship between dose and effect is --
		DR. RUBIN:  Noise.
		DR. LIPICKY:  -- just noise.  But I can make something out of it.  So, now I think I have found something. 
		DR. RUBIN:  And you can make something out of it because? 
		DR. LIPICKY:  No.  I can.  I could.  Believe me, I can. 
		(Laughter.)
		DR. RUBIN:  If you keep working at it, you can.  Is that what you're saying?
		(Laughter.)
		DR. LIPICKY:  Then I have another table of random numbers, and I do some similar, clever thing, analysis.  Then I have a third table of random numbers, and I do the same thing there.  So, now I have three different pieces of relationships.  Would you call that scientific?
		DR. RUBIN:  It sure doesn't sound it to me.  I'm not sure what kind of analysis you're doing. 
		DR. LIPICKY:  Well, I guess really let me ask the question differently then.  The question I'm asking is in this set of data that you wish to analyze somehow, don't you have to have something in that data set that allows you to say that this is not a table of random numbers?
		DR. RUBIN:  Sure. 
		DR. LIPICKY:  Yes, and what is that something that you have that allows you to decide that?  Because I can fit some kind of a function to anything, and I'll call that my model.
		DR. RUBIN:  I think that you're confusing a couple different things.
		DR. LIPICKY:  Okay.  That's what I'm asking you to straighten out. 
		DR. RUBIN:  Right, and I'll try to straighten it out. 
		One is a paradigm that says before you begin an analysis, before you look at the data, you should have a specified plan of attack.  Another thing is that that plan of attack should end with a p value, and those are two very different things.
		DR. LIPICKY:  How do I get a p value out of an Emax model?
		DR. RUBIN:  If you're going to fit an Emax model, how would you get a statistic out of it?
		DR. LIPICKY:  How do I get a p value. 
		DR. RUBIN:  A p value.  You do a randomization test.
		DR. LIPICKY:  What is the test?
		DR. RUBIN:  The test is you take whatever parameter you fit from the Emax model and you run it through the randomization distribution.  Then you get a p value for it.  That's completely valid. 
		DR. LIPICKY:  You'll have to be more explicit.  The last math course I had was in high school.
		DR. RUBIN:  Something that's often confused in statistics is the value of randomization and what it brings to you and the traditional analysis that you do in a randomize experiment.  There's no reason why the only statistic that's of interest is a simple comparison between two means.  Randomization doesn't have anything to say but it justifies that.  
		If you look at the original work on -- where does randomization starts?  It starts with 1925, 1923 with Fisher and Neyman.  What was the great innovation about it? Well, everybody was sort of talking about randomized trials before then.  They were writing papers saying, if you assigned the plots as if they had been random....  
		Fisher's great innovation in 1925 was he actually proposed to physically randomize the plots.  He said that physical randomization would provide an internal test of the effect. 
		Actually Neyman did a paper in 1923, two years before Fisher proposed that, doing mathematical analysis with a randomized experiment.  And mathematical analyses of randomized experiments as if they had been physically randomized go back five years before that.  That sort of spirit was in the air in the 1920s. 
		But Fisher's great innovation was he actually said do it, do a randomized experiment, randomly assign the units.  Why?  Well, if you randomly assign the units, then you will have all permutations.  You'll have laid out in your mind at least or on a piece of paper all possible permutations of the units that would be done if you had done randomization.  So, if you have 20 units, 20 people, and 10 of them are randomly assigned, one treatment, and 10 of them are randomly assigned to another treatment, then there are 20, choose 10, randomizations.
		What is a p value?  A p value is like a proof by contradiction.  How do you prove something in mathematics by contradiction?  How do you prove that the square root of 2 is irrational?  What do you do?  You start off by assuming that it's the ratio of two relatively prime integers, and then you reach a conclusion that they're both even, that these two integers are both even.  I've reached a conclusion.  Therefore, what I assumed must be wrong.
		So, what is a randomization?  What is a p value?  A p value is what do you want to prove is wrong?  What you want to prove is wrong is both treatments have the identical effect.  Absolutely everywhere it's an identical effect.  So, if you have two treatments and you have these 20 people and I observe a value under one treatment for each person, but not the same treatment obviously, under the null hypothesis, you know the value under the other treatment.  So, you know all the values that you might have ever observed no matter what the randomization would be.  If all possible randomizations, you can calculate the value of the statistic. 
		DR. PACKER:  Ray?
		DR. LIPICKY:  Okay.
		DR. RUBIN:  I'm almost done, if I may complete this.
		DR. LIPICKY:  I don't understand what he's talking about. 
		DR. PACKER:  I think that's okay. 
		(Laughter.)
		DR. RUBIN:  The reason for doing this is just trying to say what a p value really is.  I don't see how you can be in love in a p value once you understand what it is.  I'm just saying that no matter what statistic you've put through this randomization test, it's completely valid and it has the same interpretation as a standard test for the means.  You've got no obligation to run the means through.  You can put any test of this that you want to, and it could be your estimate of an Emax parameter that you could put through, and it's just as valid as a test of the means, as long as it's the one you decided a priori, that's the one you're going to put through it. 
		DR. PACKER:  Really, Lew?  Do you have a question? 
		DR. SHEINER:  I want to bring it to the context that you're dealing with here.  One of the things that advisory committees ask about is the science, which is sort of what Don is talking about.  One of the things you ask about is what's the import in the world. 
		One thing that you haven't addressed, Don, and I think maybe we could all think about a little bit, is that there's an extra, little hooker in the whole thing, which has got to do with the fact that, not maligning anybody's intentions, the temptation to make money or to be an enthusiast for something, in this case profit, is more than the best of us can resist.  Consequently, there's the extra part of what does the FDA require to be sure that nobody is playing the game not straight, which is a little different than our usual assumption, which is that whatever data you show me, of course, they're the right data.  You may have interpreted them wrong, but that's them.  I'm not even talking about cheating here.  
		I'm just saying one of the problems that has been raised about model-based stuff -- and I'm going to be the devil's advocate here -- is that if you get to decide what you want to do with that, as Ray I think was suggesting, you can always do something that will make it look a little better for you.  So, society has the extra problem of perhaps costing itself a little bit in terms of the ultimate amount of knowledge they get out of data in order to protect itself against the temptation.  And the question is, how do you build that in?
		DR. PACKER:  Let me ask a question because I think it's essential for us to bring the concepts that have been elucidated and focus them on the topic today.  Lew, let me ask you a question directly related to what you have just brought up.  If sponsors are going to go forward and elucidate the response surface, to what degree do the models that they would use need to be prespecified? 
		DR. SHEINER:  That's the key issue.  Exactly what does prespecification mean?  Does it mean prespecifying the model, or does it mean prespecifying a procedure for coming up with a model?  
		Let's just say I'll do a multiple regression, and I'll pick the variables that meet a certain criterion.  Then I'll test the predictive ability through some kind of, let's say, cross validation or whatever.  Now, I haven't said what's my model is going to be because I have 10 variables and I may pick 3 or 4.  Is that okay, or do I have to say, no, I'm going to actually relate the response surface to age and sex and weight, but not to a million other things that I might measure because then we start to get into this world of my making it look like perhaps I can predict perhaps better than I really can. 
		What procedures do I need to check on how good my predictions are?  One of the things that's nice about intention-to-treat and all that is it's pretty tough to game it.  So, I think that's a serious issue.  When we start to say, let's get more information, it's a serious issue to say how should we set up the procedures so that everybody is reassured that nobody is cheating anybody?
		DR. RUBIN:  The point is not the complexity of the procedure, but as long as it can be prespecified in such a way that anyone can duplicate it, that's where its validity comes from.  Validity comes from the ability for a robot to carry out the analysis. 
		DR. LIPICKY:  The way in which I understand it -- and Tom can correct me if I'm wrong -- in a sort of morbid/mortal trial, the degree of specificity that is required up front is really pretty serious.  It's the specific thing.  It's the specific analysis.  If you are going to include covariates, you must specify.  If you have not done all of those things, then you are looked at with jaundiced eye.  
		So, what's being talked about now is really quite deviant from that, and maybe if it's descriptive and viewed as needing confirmation, that's okay.  I think that's sort of what's being talked about.  I think that that's right, and I'm not sure I'm articulating it properly.
		DR. PACKER:  But I think that clearly to the extent that this kind of information would be deemed desirable and that methods are elucidated in order to obtain this information, then the quality controls by which these methods are applied need to be clarified so that sponsors are not disadvantaged by trying to do the right thing and finding that a database is modeled by them but remodeled by the agency in a way that reaches a conclusion which changes the whole way that the dose-response curve has been defined.  
		In other words, I could foresee a circumstance whereby in the learning phases Lew has described, the response surface is described, and then they go and do some specific additional studies in order to either confirm or clarify the response surface, as Lew has described.  Then the NDA is submitted and the division then says, well, we looked at the data from the trial which defined your response surface from which you then determined everything else you were going to do in terms of subsequent development, and we just think you got it wrong.
		DR. LIPICKY:  There isn't any way you can prevent that.
		DR. PACKER:  Right. 
		DR. LIPICKY:  We looked at black/white and we looked at the U.S. versus the rest of the world.
		DR. PACKER:  I understand.  But my sense is that because of the unfamiliarity that people have with this process, unfamiliarity breeds fear, and one of the fears that could be brought to the table is how do we know that we're doing the right thing, one. 
		Over the past many, many years, there has been a narrowing of the number of possible ways one can interpret and analyze a randomized trial done in a conventional manner, and there are certain analyses that are what may be called beyond the acceptable range, which may be done but are not seriously applied.
		Right now we don't have that experience here, and one of the fears that may be brought to the table is in the absence of that experience -- and that is an experience that will evolve and get better over time -- what responsibilities are placed on the sponsor and what responsibilities are placed on the division in reviewing the data that allow people to feel comfortable that the conclusions reached will be concordant?
		DR. SHEINER:  I think that's a very serious question because one of the problems I think we all perceive now is that there's a certain inevitable incentivization for ignorance in the drug development process.  We heard about it earlier.  We don't want to try with that much higher dose because we might see something that is going to get us into trouble.  Well, any system with an incentive for ignorance is a system that's going to get in trouble, and we're going to get that trouble eventually, but we won't know about it.  We want to, if at all possible, make it a positive thing to find out as much as you can about your drug by the time you submit your NDA, within the limits of the resources you have to put to that.
		I think what you're getting at, Milton, is that as we allow for much more complex sorts of things, if it's an adversarial relationship, which to some degree it is -- and I was suggesting earlier -- there are more places for somebody to pick a hole in what you did and you've got to know that holes picked in this part are not going to stop me from getting approval.  They're just going to be a negotiation process for the label, and is it going to say 15 milligrams or is it going to say 10 milligrams or whatever, or is it going to have this graph in it or that graph in it, but that the approval at least is something that's separate, related to certain confirmatory evidence that you need to have, and that the rules for that and the complexity cannot impact on that.  I don't know how to work that out.  I know it can be, but I think a concern with it is completely well placed. 
		DR. PACKER:  I think it's important to be up front about recognizing that fear because fears provide a major disincentive here.  What we want to do is align the incentives so that people feel motivated towards the same goal, although the motivations may be different, but at least the incentives be aligned.
		One of the concerns here is that if there is a major confusional component as to how this can be approached, that that would be a disincentive.  Somehow some discussion or a clarification or initial guidance needs to be provided. 
		DR. LIPICKY:  But I think the two can be accomplished at the same time.  For the trivial example, one of the designs I had was basically parallel group, three-arm trial.  So, there you've got standard stuff.  So, that first arm could be said -- you know, and then the other stuff could be treated however one wanted to.  Then the only problem would be then the company would ask our statisticians, well, do we have to pay the price of multiple comparisons for our parallel three-arm trial, and I think we would say no.  You use that just the way you usually do, and we'll consider the rest of this stuff descriptive and then, depending on how you use it, in later development, find something that would be confirmatory or something along those lines.  So, I don't see any problem at all with integrating the two approaches in a practical development program and not penalizing anybody for adopting the approach.
		DR. PACKER:  Carl?
		DR. PECK:  I agree there's no problem in doing it in principle, but I think actually providing guidance on how to do it is really an issue here. 
		One thing that I'm impressed with in looking at protocols for phase III trials or even -- let's go back -- for phase IIb trials, dose-response, exploratory trials is the lack of information in the statistical analysis section about what will actually be done with the data.  Let me clarify.  
		The phase III trials typically are quite clear about what will be done.  It will be a frequentist hypothesis testing thing.  So, there's no issue there. 
		In the IIb trials, the dose response, there's often a statement, the data will be analyzed for dose response.  End of sentence.  End of section.  So, it's left up to the reader to guess what will actually happen.  In some cases it's nothing.  In other cases it's extensive.  Then there's the concern that, well, they looked for what they wanted to look for.  They didn't tell us about all the other things they looked for and didn't find.  So, there's a problem. 
		Our center, in collaboration with Lew Sheiner and many others, has been working in the area of modeling, anticipating its use in simulation of clinical trials for the purpose of improving the design in future trials.  Realizing this problem of the lack of standards, if you will, for the process of modeling, we have put forth on our web page a draft document called Simulation of Clinical Trials:  Good Practices.  It's an attempt to define the principles and the good things to do, if you want your modeling and simulation to be understood and credible.
		We've identified three principles.  One is clarity, which means that the models and the assumptions and the procedures are all laid out in advance in a simulation plan. 
		The second is completeness.  In other words, you're urged to be complete about that.  It's not just a one-sentence thing.  We will model and simulate this.
		The third is parsimony which has to do with a certain quality of the models to make sure that they're not over-specified. 
		So, my contribution on this point is that I think if the agency wants to encourage modeling of dose-response data, I would recommend that it engage then in creating some standards or principles of the process so that there's some prospective rigor in what will actually be done.  I think that will make it more credible.  It will certainly make it reproducible. 
		DR. PACKER:  It will make it reproducible, but I guess one question that arises is, to what degree can one realistically do this?
		DR. PECK:  I can speak to that.  It's not easy because modelers are like artists.  They have their favorite models.  They have their favorite procedures.  It's an area that needs work to get experts in exploratory modeling to agree on systematic, standard procedures that they're all willing to declare in advance and follow. 
		But there isn't actually a complete chaotic universe of options here.  For example, in modeling dose-response trials, there really are only a handful of plausible basic structural models.  There's a linear and there's a nonlinear, and there's usually only one or two nonlinear models that work.  Then it's a matter of incorporating the covariates.  That's were it can become much more complex and arbitrary.  But still there are thousands and thousands of models that are probable.  There might be a few tens, and those can be declared in advance by and large. 
		DR. SHEINER:  Can I say one more thing?
		DR. PACKER:  Lew, yes, please.
		DR. SHEINER:  First of all, remember right now we don't have an adequate procedure for finding doses.  So, anything that gets added on that attempts to do that should not represent a penalty.  I think that's got to be a principle. 
		But the second thing is there's an interesting interaction here, which is that the tendency for the pharmaceutical company would be to minimize the number of covariates that actually have to be taken into consideration because that makes using their drug more complex.  So, you reverse the usual situation.  The public, as represented by the FDA, would be inclined to want to watch out for age or other things.  Now, as they get to more and more covariates, the certainty will go down.  The regulatory agency will be on the side of less certainty because they're worried about safety. 
		Now, if you can prevent that from being accompanied by a demand for more data, which is what the pharmaceutical companies are going to be afraid of -- every single thing we did in this study, which was more inclusive, as Tom talked about -- now we had four people who were a little bit older, and suddenly maybe there's maybe a little hint of something and it's got low probability, but nonetheless, now the agency says I've got to study 30 old people.  That's what you want to avoid. 
		So, it seems to me that you're going to have to have rules on both sides, but if you do, then it will sort of naturally work itself out because as you get more and more fine, the certainty goes down and nobody will want to make that sort of a claim in the labeling. 
		But I think the basic principle has got to be, if this is an add-on, at least for some period of time, you never get hurt if you try to do a better job. 
		DR. PACKER:  That sounds like a good principle.
		DR. LIPICKY:  Just one more thing to add.  I guess I really would like to see, at least in the hypertension area, dose-ranging trials not be called phase II anything.  They are phase III.  That is the basis of approval.  You really have to discard this business of phase I, phase II, phase III.  It is where that thing fits in the necessity for getting approved.  So, the program in its entirety I think is a different issue, but a dose-ranging trial can be considered phase III, is phase III, and is actually phase III for a combination product.  It is the only trial that is required.
		DR. PACKER:  Lew, let me just ask one other question.  The approach that you've described would seem to have lots of the aspects of -- actually it's not the approach you described, Lew.  It's the approach that Ray described which, in fact, you might utilize in reaching conclusions about dose response.  
		One of the issues that we would normally worry about in interpretation of dose response is the issue of carryover effects because when one force titrates but still wants to look at the relationship between dose and effect at any individual dose, in a parallel group design, you don't worry about how you got there, but in an incremental dose-response design, as Ray outlined, you worry a lot about how you got there.  In general, the agency has seen lots of examples where crossover trials have not been easy to interpret because of carryover effects.  Yet, the trial design that Ray has put forward has more carryover effects than the typical design.  How much should we worry about that?
		DR. SHEINER:  This is a very complicated issue, which I think maybe Don and Tom could talk more about.  First of all, there's again the issue that you have to be within a scientifically valid context.  That is to say, the standard model for carryover and period effects and everything essentially says we don't know.  Anything could happen.  So, you could have any kind of carryover, you could have any kind of period effect, essentially proliferating parameters, and so the problem in figuring out whether they're there or not becomes a big issue and a lot of the tests for whether or not carryover is there are logically inconsistent, in that you fail to reject the null hypothesis and therefore you accept it.  And there are all kind of problems like that.  
		But if you are scientifically based, if you're willing to do that, then you say, well, what's the nature of carryover?  Sometimes it's pharmacokinetic and it's pretty simple.  The drug persists.  
		In the case of an escalation trial, the problem is that time and dose are confounded.  As time goes by and more time in the trial goes on, that's one thing that's happening, and also dose is rising over time.  So, is it the dose or is it the time?  It's sort of a way of thinking about carryover but in a more general way. 
		Of course, there are things you can do about that in terms of the design if you think there's a good reason to believe that it might to take a month or two to develop the full effect of a given dose.  One of the things is you can intersperse placebo periods in your escalation sort of randomly.  So, you can protect yourself against the things you're worried about, but you've got to make the list have scientific credibility.  The things you're worried about have to be as credible scientifically as the things you're trying to claim, and then it becomes tractable.  
		If, on the other hand, you say I have to protect myself against all possible weird things happening, then there's only one thing you can do, which is a very, very standardized trial, draw a limited set of conclusions, and not even know whether they extrapolate to the future because you've taken such a non-mechanistic view. 
		So, there's always this tradeoff.  The way I think we'll get around it is by recognizing which of the things we need high certainty for and which of the things that we're willing to accept assumptions and lower certainty because we need to give people a best guess as to what to do and it's never going to be for certain. 
		DR. PACKER:  Jeff and then I want Tom to address that as well. 
		DR. BORER:  I have two sort of general comments and then a question. 
		DR. PACKER:  Don, if you want to sit down, it will be probably easier. 
		DR. BORER:  First of all, I just want to support what Lew said a little while earlier.  Much of the discussion has gone on as if the goal is to design the perfect development program and anything short of perfect is a failure, when in fact what we have now is at sort of the lower end of good.  If we get to the middle end of good, that's a tremendous advance.  So, it doesn't have to be perfect. 
		The same thing is true with regard to the covariates.  We don't ask for information to the nth degree about every possible confound or every possible subpopulation.  We don't do that now.  It's unrealistic, can't be done.  Why are we talking as if it has to be done in some new strategy?
		Having said that, just with regard to the carryover effect, I want to ask a practical question, and Tom or Lew or whoever can answer this I'm sure.  In Ray's multiple-arm, forced titration scheme, at the end of forced titration 3 in one arm, you actually were at the same dose as you were in the beginning of another arm and you're 9 weeks later.  Could you not look and see that the pattern of effect was the same at the end of forced titration arm 3 as in titration 1 of the other arm and say, we've accounted for carryover?  Could you or couldn't you?
		DR. SHEINER:  It sounds good to me. 
		DR. PACKER:  I guess the answer is possibly yes.
		DR. LIPICKY:  Yes, but then Tom is going to say, is that a non-inferiority kind of a comparison?  Aren't you, Tom?
		DR. FLEMING:  That's what you'd be doing. 
		DR. LIPICKY:  Yes, and that could get very hairy.  At first blush, yes, that would be good. 
		DR. PACKER:  Tom, do you have any additional points?
		DR. FLEMING:  Actually I've been just trying to gather in all the perspectives here following Donald's talk.  Actually I have a few global comments that might be my attempt as a frequentist and someone who does believe in a role of hypothesis testing in confirmatory trials and yet trying to integrate into that philosophy the insights that came from Donald Rubin's talk. 
		He made a number of key points.  Just a couple of those key points are the issue as to whether a randomized trial itself is giving us a generalizable conclusion to the future because, in fact, if it doesn't, it really isn't serving the purpose that we intend. 
		He also pointed out the importance of being able to address issues, in my words, that are truly the clinically relevant ones to individuals in trials, and that in fact, involves what is the right dose and schedule for me given my specific background. 
		I guess to address those issues, the first point that I would make is that it's very important to design trials in ways to achieve maximal generalizability.  There are a number of ways to do that:  inclusive eligibility criteria; bringing people in from multiple centers instead of having a single center trial; designing the control regimen to represent truly standard of care and to allow a real world access to ancillary care, so that the answers that we're looking at are, in fact, as relevant as possible to the real world setting for the use of your therapy; adequately sized trials to be able to provide, even if it's secondary analyses, relevant insights into exploratory analyses that might be looking at what is the evidence for homogeneity of effect across various characteristics of patients; providing what evidence is still necessary to be obtained on dose and schedule, although I would argue, as I did toward the beginning of today, that that's an issue that we should be addressing from the very beginning of our drug development plan.  And we do.  Preclinical studies, PK studies back in phase I, in particular, looking at an array of doses in phase II and IIb, and in fact, in some cases as well, looking at these issues in phase III. 
		So, I would hope that even though one can never fully achieve it, that one is designing trials that are confirmatory trials to be as generalizable as possible and as relevant as possible to the real world setting.
		When one gets to the confirmatory trial, i.e., when one gets to what I call the phase III study, it is in fact important from my perspective that we have prespecified goals that we are addressing in a confirmatory fashion, and for multiple reasons, including those that Lew articulated, that there are a myriad of different exploratory analyses that can be done, many of which certainly lead to what some of us might refer to as data-driven aspects and it's difficult to discern level of convincingness.  
		Essentially ultimately what I want to be able to assess is the strength of evidence.  I wish to have a standard for that strength of evidence.  To my way of thinking, the paradigm that we follow at this point for prespecifying key hypotheses and providing hypothesis testing gives us one effective way of being able to address whether or not there is adequate evidence here to meet our standards for strength of evidence to establish benefit and to establish a safety profile that gives us a favorable benefit to risk.
		That doesn't mean, however, that these methods can't be adaptive.  There certainly are, depending on our level of confidence of how much we're willing to prespecify, valid methods that allow flexibility in the formulation of these statistical methods that are used that still allow us to meet this frequentist paradigm and p values and standards for strength of evidence. 
		But certainly we don't stop at that point.  If we did, you would only need a statistician to interpret results and make judgments.  As important as some of us might believe statisticians are to this process, obviously this is only one part of the process and a global judgment, looking at all of the results in the trial, strength of evidence not only of what we see in this primary hypothesis, but secondary measures, safety considerations and results from external trials, is critical and needs to be integrated in a way that is obviously subjective and requires considerable judgment. 
		In this process of designing trials, to come back to the specifics here of dose response, I would strongly support the argument that phase III can still be a setting -- and maybe should be a setting in some instances -- for looking at dose response and looking at multiple doses. 
		Being a frequentist, I worry about the issue of multiple testing.  Somewhat unsuccessfully, I've been arguing for a long time that multiple testing adjustments, though, shouldn't apply to this specific issue of looking at multiple doses against a control.  I've had multiple long discussions with FDA colleagues and others about this issue, and I know I'm in a minority in the FDA ranks about this.  I believe in adjustment for multiple testing.  If you are a frequentist and you're looking at strength of evidence, if you allow multiple endpoints, multiple test statistics to look at those multiple endpoints, testing over multiple points in time, looking at multiple subsets, all of these greatly inflate the risk of getting differences that would appear to be significant evidence of benefit when, in fact, they could be attributed to random variability. 
		I view, though, in a very different way looking at multiple doses, and the irony is if someone was so naive as to say, I'm going to look at low dose against control in one study and high dose against control in another, and not be asked to adjust for multiple testing, that's logically inconsistent with that they had the wisdom to say I could be more efficient to put high dose and low dose against a common control, that I'm going force them to adjust for multiple testing.  
		Or to say if low dose against control gives a two-sided p value of .04 and high dose control is a two-sided p value of .04, that's no longer significant because it doesn't meet some kind of multiple testing adjustment.  I've actually strengthened the argument when I see high dose against control being .04 when I also see low dose against control being .04.  
		So, I've argued that if one does look at multiple doses against a control, it is very appropriate to control the pair-wise comparison error rate, adjusting for all multiple comparisons done within that pair-wise comparison, but one should factor in the strength of evidence from the multiple doses using some kind of aggregation, whether it's a dose response or some other method, some type of aggregation for strength of evidence.
		Now, one last comment and that is should we be doing a primary analysis for efficacy.  Let's say we fit a log-linear model or a linear model and the slope is our strength of evidence for our primary hypothesis.  There certainly can be some ambiguities that can result.  In fact, one concern that I have is if you look at high dose and low dose and there's no control but the low dose is, in essence, thought to be the control for the high dose, if there isn't a dose response, I'm left without certainty as to whether the two doses are equally effective or equally ineffective. 
		If I have a control in a low dose and a high dose and I'm fitting a slope and I find a significant relationship, but I then also discover that the high dose has an unacceptable safety profile, I'm uncertain from that analysis whether the low dose, which has an acceptable safety profile, actually has had an established benefit.
		So, there are some complexities with these types of analyses, and I have tended as a result for this reason, even though Carl might call me inefficient, to favor designing these phase III trials in ways that do allow me to obtain adequate sensitivity to each dose against a control so that if there is, in fact, significant evidence or true, significant benefit for a given dose, I'm able to address that and have sensitivity to it. 
		But I would definitely take this opportunity to argue one more time that if someone has the wisdom to look at multiple doses against a control, that they ought not be paying an alpha spending penalty for that type of multiple testing.
		DR. PACKER:  I think for the sake of time and the sake of clarity, I think we heard Ray say before about an hour ago that you would not penalize a sponsor who wanted to undertake this path and exact a penalty for multiple comparisons, that you would allow each of these arms to be tested pair-wise at an alpha of .05.  Is that correct?
		DR. LIPICKY:  See, I don't understand the thinking.  I agree 100 percent with what Tom said, every single word.  I think I agree with what you said, Milton, but I don't see how anything that either of you said is applicable to this circumstance.  This circumstance of an antihypertensive drug in the current development regimens have no phase III.  There is no such as a phase III.  There is no such thing as an intent-to-treat, morbid/mortal trial where all these rules start to really play enormously important things.  
		So, I think what I was saying is I wanted to see a well-defined shape of a curve somewhere -- and maybe that could be done first in a learning phase and then a second trial in a complementary -- whatever the word is -- "you prove that your first learning was right" phase.  Then that defines the dose response.  I think you could do that in 300 patients.  Now you've got another 2,000 patients you can devote to other things to learn about the antihypertensive.  All I want to do is consider at this point in time what you can do from the dose-response point of view, if we could settle that.
		At the moment we have 6 and 8 trials of 300 to 500 patients each that are all parallel, fixed-dose, dose-ranging trials.  That's the whole NDA.  In fact, every one of those trials is treated by a frequentist type analytical process where something is being tested against placebo in a pair-wise comparison way or sometimes ANOVA, sometimes with some kind of more fancy thing.  But I think that's just all the wrong stuff.  So, although everyone is saying good stuff, it isn't applicable to the circumstance that basically needs to be changed, which I think it should be.
		DR. PACKER:  I think that speaks to the point that you made before that you made before that if one knew that the response surface would look like, one can reach conclusions independent of the calculation of a p value.
		Tom, my sense is that it's not either/or, that these are complementary approaches that enhance the quality of the database. 
		DR. LIPICKY:  I don't disagree with Tom.  I think he's 100 percent right.  It's not addressing the right problem. 
		DR. PACKER:  It sounds like this is much easier than the Middle East. 
		(Laughter.)
		DR. KONSTAM:  With your construct of what you're talking about in terms of not having to pay a multiplicity penalty, I assume that that really implies a very strong assumption regarding something about the model of the dose response.  Let me give you the ultimate which works with hypertension because I think we know a lot about dose response in hypertension.  The ultimate contrary example would be what would you do with a drug that you knew nothing about dose response, including the possibility that higher doses may go in the opposite direction.  And you may do placebo and two doses and find a low dose looks like it reduces mortality and higher doses actually looks like it kills people, which has actually been seen.  So, I guess it assumes that it fits some model of dose response, doesn't it?
		DR. FLEMING:  Well, what I'm assuming, when I'm arguing that I don't do alpha adjustment, is if I have set up the study to address multiple schedules, which could be multiple doses.  My primary analysis when I'm looking at it as a frequentist here for whether I hit the standard for strength of evidence, establishing efficacy, is to look at this in a pair-wise comparison of a single dose against control. 
		When, however, I look at the totality of the data, I will do some type of meta-analysis.  That meta-analysis may be multiple studies looking at that same dose, or it might be combinations of studies that look at multiple doses against a control.  Clearly then there's going to be in that level of meta-analysis, where I'm aggregating strength of data, some type of model.  When I'm aggregating data from multiple sources of information, I will then be using some type of model.  
		For example, if I had two trials looking at the same dose and one study showed a favorable effect and one study showed an unfavorable effect, the unfavorable result weakens my confidence in the result in the favorable.  If, in fact, I'm looking at multiple doses, yes, I'm certainly going to either formally or informally, intuitively, be fitting some kind of model that I'm assuming for dose response.  I'm happy with either one depending upon which approach you would prefer.  I will then use that model, formally or informally, to integrate all of the insights that I have to determine, A, whether there's evidence of benefit and, B, is there evidence to suggest what is the dose that maximizes benefit to risk. 
		DR. LIPICKY:  Really, you need a lot of particulars.  I think both of the conceptual aspects of what we're talking about can be satisfied by a single design.  It would take a lot of thought and careful thought, and I don't think one has to be dichotomous and say we can't do one or the other.  I think both can be done at the same time.
		DR. FLEMING:  Ray, admittedly I'm having trouble focusing my thoughts on this to the single case of antihypertensive drugs where we're going to assume that we can rely on a surrogate in this setting, i.e., blood pressure lowering, and we're going to assume that there is no dose-limiting toxicity.  I'm really thinking in a more global sense. 
		But the principles that I've laid out, if applied to this specific setting, would argue from my perspective that if I did believe that I had no dose-limiting toxicity, that it would be very prudent not just because of what the FDA would wish, but from the sponsor's perspective to set up a wide range of doses because this is an answer that I can get for a wide range of doses.  
		If you are looking at survival as the endpoint or stroke or MIs and it takes 10,000 patients to be able to address the effect of a single dose, those studies don't lend themselves as well to understanding the dose-response surface.  
		But if I can rely on a surrogate and I can understand the relationship of the effect of a given regimen on that surrogate endpoint in much smaller numbers, and now you're telling me I don't even have to worry about whether there's an upper limit of how far I can go because of a dose-limiting toxicity, then I would argue it makes perfect sense to look at, as you've advocated, a wide range of doses that allow me, with a reasonable sample size, to get an insight into the dose-response surface and to fit some model, if I'm a frequentist and want to look at strength of evidence, into how I'm modeling that relationship to establish whether I have conclusive evidence that the intervention is affecting this specific outcome. 
		DR. SHEINER:  I think that this issue of a single study being used for two purposes -- And, Tom, I admire your pure position of integrating it all and having a an overarching view of it.  But let me segregate it for a moment.  Let me propose something that's been proposed in the past. 
		There are some people who really want to see something clearly demonstrated by the data, so I'm going to do two doses:  placebo, which is a dose, and a dose that I think will work.  I'm going to do that in 400 people:  200 placebo, 200 with this dose.  And I'm going to do it for 1 month or whatever it is.  That's what I'm going to use to demonstrate that the drug works. 
		Now, in that exact same study, I will, at the end of the 1 month, randomize all of the people to some other dose.  So, I'll now actually have a second period in which they'll all get some other dose.  And now I might choose for just that one dose that I used in the first period.
		Secondly, from the very front of this study, I'm going to add on another 200 people who are much more inclusive.  They're not having blood pressure exactly where I wanted them to be in the study.  They're not without any other complicating disease.  I might take people with angina when I didn't before.  And I put them also in this same study. 
		I can do two completely separate analyses of the core of 400 people in a straight-ahead, hypothesis testing way.  Then I can take that data, add it on to all the other data, and be a complete Bayesian, for example, and just do a total model-based approach with informative priors designed to come up with the best understanding I have of the response surface.  There's nothing illogical.  There's nothing wrong.  The fact that I got all that extra data in that same study does not mean that that analysis of the core 400 all by themselves, having stated beforehand that that's the way I was going to do it -- it doesn't change in any way the frequentist properties associated with that and the conclusions we might draw from it.  So, although obviously it's in some sense more satisfying to look at the whole thing as all one thing, you can do that, what I just said, and that would be an example of getting extra power for the exploratory analysis and no loss of power for the confirmatory one. 
		DR. FLEMING:  In fact, I would argue that's precisely the approach that we should be taking and, in general, are taking although aren't necessarily doing it with quite the level of rigor as you've explained for the exploratory analysis.  
		The approach in general that I would advocate is to design trials to be generalizable, to be relevant to the real world setting, and to be able to be analyzed for strength of evidence from a frequentist perspective, but then ultimately the assessment of benefit to risk and approvability and as a consumer, whether I'm going to use a given agent, should be based on an aggregation of all of the data that will clearly have to involve an exploratory element of this.  
		A Bayesian approach is one relevant approach.  If I want to use a prior and get a Bayesian inference, that's a very relevant exploratory approach.  Of course, we have to understand the influence of how my choice of the prior would influence the interpretation, but that can all be laid out. 
		DR. SHEINER:  Let me just add one thing to that, Tom.  Having said all that and we're in 100 percent agreement, my extra little claim that I'm not sure whether you're for or not or what you want to say about this is that that explanatory -- I like to call it explanatory rather than exploratory because it makes it sound a little better -- analysis, having satisfied the frequentist test, might show up -- the conclusions of that might show up in the label as a graph of dose response or as a set of suggestions for doses or something rather than the label simply saying we tested, in a standard confirmatory way, 100 milligrams, so that's what's suggested. 
		Rather it might say now, given that 100 milligrams definitely works, this is where it appears to be on the dose-response curve and just interpolation, this is what the picture looks like.  So, if your patient has these characteristics, you might consider lowering the dose to that and so on and so forth.
		DR. FLEMING:  I guess I would be unwilling to make an absolute statement as to what would always be the case.  It strikes me that the possibility that such insights could be very important beyond what came from the primary prespecified frequentist analyses certainly exists, that those results could be important additional insights and the reliability of those analyses sufficiently established that it's possible. 
		DR. LIPICKY:  So, we just have to convince somebody that it is worth their undertaking these things so we can get some data set to examine and see whether anything we're talking about is real or not.
		DR. PACKER:  Well, I think that brings us to the final part of today's agenda.  I'm going to take the chairman's prerogative to skip all of the questions. 
		DR. LIPICKY:  Okay.
		(Laughter.)
		DR. PACKER:  I want the committee to address two questions, and we can do so in detail or in a general manner, depending on what the responses are.  But I think the two questions that summarize -- I mean, what we have heard is that a problem has been identified in terms of the adequacy or inadequacy of characterizing the relationship between dose and response in the evaluation of drugs for the treatment of hypertension.  An approach or, should I say, approaches have been proposed that would greatly expand the knowledge base, and this approach does not conflict with our conventional approach to hypothesis testing.  The two approaches are complementary, and that this additional approach does not constitute an undue burden on sponsors and, in fact, may alleviate them from the responsibility, expense, and time of doing conventional analyses that may contribute little to the total knowledge of what the appropriate relationship between dose and response may be.  Therefore, the new approach would provide benefits to not only industry but to the regulatory process and, most importantly, to patients.  And we've outlined many of those reasons. 
		So, that leaves us really with only two questions.  One question is, are there reasons that sponsors should, in fact, elucidate the full range of the dose-response relationship or, as Lew says, the response surface?  Are there reasons?  
		I think we've elucidated many, many reasons why it's in the sponsor's interest, in the interest of the information gained, the clarity of labeling, the clarity of the development program, and in fact in the public health interest.  We mentioned some of those issues earlier.  Especially as the target blood pressures decrease, the number of drugs would increase unless the drugs that we have had a wider dose-response relationship than previously assumed. 
		So, the last question to this group is, are there reasons that sponsors should not elucidate the full relationship between dose and response?  Are there reasons that they shouldn't do that?  I guess I'd like to hear any proposals why they shouldn't.
		DR. KONSTAM:  I still think there are reasons why they wouldn't.
		DR. LIPICKY:  No.  Why they shouldn't. 
		DR. PACKER:  We have outlined several, maybe six or seven, reasons why they should.  Are there reasons why they shouldn't? 
		DR. BAKRIS:  Milton, let me just make a comment to that.  Marvin is right.  There are reasons they wouldn't, but why they shouldn't, the only reason I could imagine would be an economic one, and I think we've made a pretty compelling argument that, in fact, from an economics standpoint, it's in their favor.  It probably is more cost effective.  So, you've actually stumped me with that question.  I usually can come up with something, but I can't think of anything for this one.
		DR. PACKER:  The question is framed specifically with intent.  The word "should" is in fact the intended word.  Are there reasons why they shouldn't?  If there are many reasons why they should and there are no reasons why they shouldn't, then they should.
		DR. BAKRIS:  That was subtle. 
		(Laughter.)
		DR. SHEINER:  The same compelling logic as hypothesis testing. 
		(Laughter.)
		DR. PACKER:  Consequently, we've outlined reasons why they would.  So, if they should and they would, then they might. 
		(Laughter.)
		DR. PECK:  Well, I'm going to rise to the occasion and try to anticipate a shouldn't.  That would be under the circumstance that the prospect of harm to a subject, even remote, would be unacceptable to a company.  That is, their view that having one adverse reaction of a particular kind, say, a teratogenic reaction, would be so devastating to their image or reputation and the drug's image that they would be unwilling to creep too far into the unknown.
		DR. LIPICKY:  I think there's a very good answer to that.  Much better that that should be in the development phase than postmarketing.  So, that just is not an adequate reason.  People aren't going to get sued in the development program.  They're going to get sued postmarketing.
		DR. PECK:  Yes, that's all right, but small companies whose finances are subject to private financing or mouth-to-mouth, month-to-month, or flaky stock market support might not buy that argument. 
		DR. LIPICKY:  No.  But in fact they do.  One of the most recent development programs that we've been associated with with a small company -- the liability issue postmarketing was a big deal, and they were willing to put more into the development program thing than suffer the pangs of having postmarketing liability.  They looked into it in great detail.  So, if that's logically approached premarketing or during development, I think one would come to the same conclusion, and I know one that did. 
		DR. KONSTAM:  But, Ray, that might not apply when you're talking about a 1,000-fold increase in the dose range.  
		DR. LIPICKY:  It's only three times more than 300.  So, that's only one more arm. 
		(Laughter.)
		DR. KONSTAM:  Okay.  I suppose this is a question that Milton didn't ask, the "wouldn't" question.  I still think that there is going to be a lot of resistance.  I still think that your assurance that there would not be a penalty, in terms of approvability, will not sufficiently dissuade that concern.
		DR. LIPICKY:  All they have to do is look at all of the drugs that are on the market that have dose-related side effects.  There are.  We have approved drugs that do that.
		DR. KONSTAM:  I actually would like to ask another question.  Maybe this is a naive question, but an ethical question, regarding informed consent.  Particularly as we're getting into the issue of exploration of adverse events at very wide dose ranges, perhaps at dose ranges that, despite we don't know adequate dose response, some people might construe as unlikely to yield significant additional benefit.  
		Maybe there are studies done this way, but I've never been involved in one, where a specific goal of the study was to identify the dose range of adverse events.  Usually you say, well, we're exploring efficacy.  We don't know everything about adverse events, so there's risk.  Here are the adverse events we know about, and you should know there might be others that we don't know about.  But we're really exploring efficacy. 
		This is now different.  This is now saying, well, one of the purposes of this program is actually to find adverse events. 
		DR. LIPICKY:  Right.  I know and I shouldn't have said what I said.  The purpose of the program is to find the greatest effect one can find.  So, that requires increasing dose.  One may not be able to increase dose sufficiently to find that greatest effect, in which case adverse effects would be limiting.  The only reason for my assertion of we ask people to hurt people is that the excuse that, as you are looking for the greatest effect you can get, might hurt somebody doesn't seem like a reasonable excuse to me.  It's much better to know that, once again, in the premarketing circumstance than in postmarketing.  But you're searching for the greatest benefit.  
		DR. KONSTAM:  No.  I understand that part.  I guess I'm saying one of the constructs that I think that had floated around as a situation that would be acceptable, that you've adequately explored the dose range, despite absence of a maximal effect, is that you've identified dose-limiting adverse effects.
		DR. LIPICKY:  Well, no.  I understand, but I think the notion is that if you have been able to identify a maximal effect -- and I would accept Emax with tight confidence limits as having demonstrated that -- and you haven't found adverse effects, that's fine.  I wouldn't push for going to a higher dose.
		DR. KONSTAM:  So, you'd never be, despite some comments that have been said, proposing a program that specifically would be seeking --
		DR. LIPICKY:  That is correct.  The purpose is to find the greatest effect, beneficial effect, and that is the purpose.  The thing that bothers me, all told, is we have this one ACE inhibitor over 1,000-fold dose range and it still couldn't show it had the found the maximum effect.  Now, that's a bother to me.  So, that means it probably should have gone higher. 
		Well, then up comes the business of side effects and you might hurt somebody.  My response to that is, sure, you may.  You may not too.  And if you do hurt somebody, the best time to hurt them is in the development program.  It should not occur postmarketing.  So, if there are things that are dose-limiting, you ought to find that out as you're working the drug up.  You should not find that out when it goes on the market. 
		DR. BAKRIS:  Milton, let me just build into what Carl mentioned and what the discussion was right here because it occurred to me, as a former IRB member, that there may be a limiting factor and it will have nothing to do with the company.  That is, the company may say fine and they may propose it, and the sites they go to and the IRBs that they use may not allow the maximum doses.  So, I think that's something else to --
		DR. PACKER:  I don't understand that.  When a sponsor proposes a dose, it does not know, so the IRB cannot know, what is in fact the appropriate dose-response relationship.  So, it is impossible for me to understand how an IRB could object to what is almost always an arbitrary selection of doses.
		DR. BAKRIS:  Right. 
		DR. LIPICKY:  Well, I can see where some people might say if you're studying a 1,000 times something, that that's just too much.  You don't have to know much to know 1,000 times is a big number.  A priori, they will say that's too much. 
		DR. BAKRIS:  We're not talking about physiology here.  This is gut feeling of people. 
		DR. SHEINER:  With respect to this, though, you have the tradition in oncology where they have now escalation studies not within an individual, not crossover type, but small numbers of people and you keep on increasing the dose.  So, if it was really important, there are ways to design studies that put minimum number of people at risk, and you can do that if you think it's important enough.  In oncology, they always have because they have wanted to be side effect-limiting in general.
		DR. PACKER:  From the sense that I've heard today, all we're encouraging sponsors to do is maximize their asset.  What we're saying is that if you have a product that lowers blood pressure and you can only give it in a limited dose range because it produces side effects, then you've understood how that drug should be used.  If you've given it in a range that lowers blood pressure but is very well tolerated, you haven't explored its full potential.  It may be the single best antihypertensive drug ever.  In fact, it may be so good that it is the only antihypertensive drug a patient will ever need and would, in fact, fulfill that role in everyone in the world.  You would never know that if you limited the potential to ask questions. 
		One would never, I think, willingly reduce the potential of an asset.  Everything that's being talked about here is simply to say there are straightforward, non-expensive, non-burdensome ways of expediting the process of exploring the full potential of an asset, which up to now has been under-utilized, perhaps non-utilized, and that doing so is in the sponsor's interest.  It's in the interest of regulatory clarity and, as Carl said, regulatory consistency over time, and it's in the interest of public health. 
		DR. SHEINER:  That being said, I think the reality is that without regulatory pressure it won't happen.  It won't happen for two reasons.  One, because pharmaceutical companies tend to function that whatever we did in the past, we'll do in the future because we have a system for that. 
		The second is that despite the fact that you said it's straightforward -- as far as I'm concerned it's straightforward -- it isn't.  It's a little more complex.  The design of the study is a little more complex.  The interpretation of the study is a little bit more complex.  And adding these two elements of fear and the fact that there will have to be an investment in a relatively unknown thing that they'd have to do because the guidance of exactly how to do it -- in fact, the knowledge of exactly how to do this -- in the most efficient way doesn't exist.
		So, what I think we have to acknowledge that without regulatory pressure, we will not get the pharmaceutical companies to do this and begin the exploration on a larger scale that we in academics have started on a smaller scale, but the exploration is going to be required to find out what are the most efficient procedures for getting this knowledge. 
		DR. PACKER:  Paul?
		DR. ARMSTRONG:  Milt, I'd like to pose a question in relationship to this discussion of dose.  If we establish the minimally effective dose of a new compound, do we have a sense as a group of what a feasible or realistic range would be to approve marketing of that compound in this era of medical errors?  Notwithstanding the wish to make the mistakes before it's marketed, we know for certain that there will be continuing problems after it's marketed based on inappropriate use.  Do we have a sense of feasibility and range that's realistic?  A 1,000-fold, for example, would be very difficult for me to accept.  Could we have some discussion on that point?
		DR. PACKER:  Paul, let me pose a way of thinking about it.  That is that if a sponsor would normally look at a dose range from 2 milligrams to 10 milligrams, just making this up, and found that that was acceptable, but because of this discussion, was encouraged to look at a dose of 100 milligrams and found out that 100 milligrams was really very much more effective, based on whatever model one would think, and was still very well tolerated, what the sponsor might decide to do is, rather than instruct physicians that ranges of dose from 2 to 100 milligrams, which seems quite wide, are recommended, they might want to further explore maybe whether that recommended dosing range might be usefully confined perhaps to the upper range rather than the lower range.  I'm just putting that out as a possibility.  And they might be so confident in that range because it was so much more effective, they would be able to do comparative studies with other drugs showing it was better.  Therefore, they would primarily develop the upper range, something they would have never done if they hadn't explored the full dose response. 
		Jeff?
		DR. BORER:  Paul raises I think a very important point that I would like to discuss a little further, that is, the need to define the minimally effective dose.  I don't think we have to define the minimally effective dose.  I don't think a sponsor has to define the minimally effective dose.  Just to throw it out on the table, it seems to me like defining the ED50 and knowing that that's safe and knowing that another dose, either higher or lower, is safe, and having some idea of the dose-response curve would allow someone -- having enough information to define a good range of the dose-response curve, if not the entire dose-response curve to maximally effective dose -- to titrate, to allow a doctor with a label to titrate to the effect that he or she is trying to achieve in his or her patient. 
		If it's blood pressure we're talking about, you don't have to know the minimally effective dose for the population.  You've got to know the effective dose to get to where you want to get in the patient.  And if you know that the dose-response curve is X, and it's shaped this way, and you know that a dose that's up here is safe, you can titrate down from it and figure out where you want to be.  Who needs to define the minimally effective dose for the population I would suggest.
		So, I think it's a very important point, but I don't think that that should be the goal of the sponsor, to define the minimally effective dose.  I think the goal should be to define the dose-response curve and then to define safety at a couple of points, however many points seems appropriate along that curve, so that some reasonable directions for use can be given.  
		It may be that in phase IV more points along the curve, higher up, have to be defined -- the safety of them, that is -- but a titratable drug, if you know the slope of the dose-response curve, why do you have to spend a lot of money on defining minimally effective population dose?
		DR. PACKER:  As I understand it -- and I'm just looking at the equations used for calculating the Emax model -- a sponsor doesn't have to define the minimally effective dose.  The sponsor simply has to define the full range of dose and describe the relationship.  What might be a minimally effective dose emerges from knowing the shape of the curve.  But the intent is not to identify the minimally effective dose.
		DR. BORER:  I didn't mean to suggest that the current rules indicated that one needs to do that.  We know that they don't.  But rather, just moving on from what Paul had suggested, that not only do you not have to do it, but it's not something that you would want to spend precious resources on. 
		DR. PACKER:  Jeff, I think you're emphasizing that's frequently misunderstood.  There appears to be a considerable energy frequently expended by sponsors to define a minimum dose in the absence of defining the whole shape of the dose-response curve.  I think what you're saying is no good purpose is served in that kind of energy expenditure unless one is going to do that as part of -- in other words, the motivation is to define the dose-response relationship, not to define the minimally effective dose.  Defining the minimally effective dose without defining the dose-response relationship serves no purpose. 
		DR. BORER:  That's right, although I would suggest that there is no such thing as a minimally effective dose for an individual except for 0.  The dose-response curve is a continuum, and you got to titrate in your individual patient. 
		DR. PACKER:  Any other comments, questions?
		(No response.)
		DR. PACKER:  Ray, I think you have our strongest recommendation and encouragement to take measures to encourage the elucidation of the full dose-response relationship for antihypertensive drugs, both for the individual and for the public health.
		DR. LIPICKY:  Thank you. 
		DR. PACKER:  And we are adjourned. 
		(Whereupon, at 2:54 p.m., the committee was adjourned.)