Agenda for March 1, 2000 Meeting

Chairperson: Harold C. Sox
Executive Secretary: Sharon Lappalainen

Minutes of March 1, 2000 Meeting

OPEN SESSION

In accordance with the provisions of Public Law 92-463 this meeting was open to the public; no part of this meeting was closed to public participation

Health Care Financing Administration
Main Auditorium
7500 Security Boulevard
Baltimore, MD 21244

Attendees

Harold C. Sox, MD
Chairperson

Robert H. Brook, PhD
Vice-Chairperson

Sharon K. Lappalainen
Executive Secretary

Voting Members
Thomas V. Holohan, MA, MD, FACP
Leslie P. Francis, MD, JD
John H. Ferguson, MD
Robert L. Murray, PhD
Alan M. Garber, MD, PhD
Michael D. Maves, MD, MBA
Frank J. Papatheofanis, MD, PhD
Ronald M. Davis, MD
Daisy Alford-Smith, PhD
Joe W. Johnson, DC

Consumer Representative
Linda A. Bergthold, PhD

Industry Representative
Randal E. Richner, MPH

HCFA Representatives
Jeffrey L. Kang, MD, MPH
Hugh F. Hill, III, MD, JD

Open Public Session – Morning
Guido Tricot, MD, PhD
Richard Justman, MD
Morgan Downey
Donald S. Baim, MD
Wayne L. Roe
Vicki Gottlich
Larry M. Weisenthal, MD, PhD
Sandy Sherman
Thomas Mesken
Whitney W. Addington, MD, FACP
(Written Testimony)

Open Public Session – Afternoon
Steven J. Northrup
Kenneth Cook
Thomas Meskan
Steven Lascher, DVM, MPH

Wednesday, March 1, 2000, 8:11 a.m.

The Executive Committee (EC) of the Medicare Coverage Advisory Committee (MCAC) met to hear a report from its Subcommittee and to discuss and consider the levels of evidence and types and presentation of information that it believes should be considered by the medical specialty panels at future MCAC meetings. The meeting began with the introduction of the Committee, a reading of the conflict of interest statement, and the call to order.

Opening Remarks. The Office of Clinical Standards and Quality (OCSQ) made opening remarks, stressing that HCFA continues to see its mission as beneficiary-focused and stated its goal to assure access to health care for the Medicare population. The representative stressed that science should have a role in coverage decisions, keeping in mind the Congressional mandate that Medicare will pay only for those items or services that are reasonable and necessary for the diagnosis or treatment of an illness or injury. He concluded that critical examination of the scientific literature is complex and that HCFA is grateful for the Committee’s work in this area.

Subcommittee Report. The Chairperson introduced the members of the Subcommittee and presented their report entitled "Recommendations for Evaluating Effectiveness." He stated the document’s two purposes: 1) to provide general guidance to the panels on how to evaluate evidence, and 2) to suggest specific procedures the panels should follow when drawing conclusions about the adequacy of the evidence and the magnitude of its effect. The goals of the document are to make the evaluation process more predictable and consistent, so that panel decisions are more understandable, accountable, and transparent. He then went through the report on an item-by-item basis. He concluded by proposing a timeline for a typical MCAC evaluation and solicited comments from the Subcommittee. The Industry Representative raised three concerns about the report: 1) its overall tone and assumption that new technology must first be proven, 2) the availability and rigor of evidence over time for a technology was not taken into account, and 3) its implementation process was unclear. The Consumer Representative emphasized the amount of effort and discussion that occurred during Subcommittee deliberations, including editing tone and verbiage before reaching the present draft.

Open Public Comments and Scheduled Commentaries. During the morning session, panelists heard from nine speakers and testimony from the ACP-ASIM was recited by the Executive Secretary.

Dr. Guido Tricot, Director, Myeloma Transplant Center, University of Arkansas, expressed a number of issues. Firstly, in considering autologous stem cell transplants for multiple myeloma patients, exclusion criteria should be used to assess co-morbid conditions, not biological age. Secondly, he felt that mechanisms are needed to assure that the explanation of the panel’s conclusion reflects the view of the panel and not just that of the Chairperson. Thirdly, he stated that the conclusions of a medical specialty panel should be publicly available to afford review and comment prior to the EC meeting. Dr. Tricot then addressed the difficulties of performing randomized clinical trials. He cited barriers in obtaining data using new treatment regimens, such as the affects of patient referral patterns on randomization. Finally, he suggested that time lapses between the process initiation and implementation of coverage determinations could be shortened using a temporary approval mechanism.

Dr. Richard Justman, National Medical Director of United Health Group, spoke on behalf of the American Association of Health Plans (AAHP). He stated that AAHP endorses a rigorous, evidence-based approach to coverage determinations. He applauded the establishment of MCAC, and asserted that the Subcommittee’s report will promote systematic and consistent evaluation of the clinical evidence. Dr. Justman stated that the variation, use, and quality of heath care services and the proliferation of unproven treatments, illuminate the importance of promoting a delivery of care that is based on robust, scientific evidence. He then gave some illustrative examples.

Morgan Downey, Executive Director of the American Obesity Association, gave statistics on the increase in obesity in the U.S. He proposed that when the EC evaluates evidence for new services or items, it consider whether a representative sample, by weight, of the current Medicare population was included. He also proposed that the EC begin a process to clarify Medicare coverage of obesity and cited examples of discrepancies in the Coverage Issues Manual.

Dr. Donald S. Baim, Chief of Interventional Cardiology, Beth Israel Deaconess Hospital, spoke on behalf of the Health Industry Manufacturers Association (HIMA). His three main points were: 1) a variety of evidentiary sources, such as randomized clinical trials, registries, equivalence trials and objective performance criteria (OPCs), are needed; 2) evidence trials used to support FDA approval should be considered; and, 3) delays in coverage decisions restrict application of new therapies and add to hospital’s and industry’s financial burden. Dr. Baim then gave a history of the use of patient registries in the area of interventional cardiology that led to the development of OPCs.

Wayne I. Roe, Chairman of Covance Health Economics & Outcome Services, spoke at HIMA’s request. He stated the importance of the EC encouraging better science, but questioned whether the report’s tone would have that effect. He said the EC should be realistic about levels of evidence, rather than requiring randomized controlled trials (RTCs) and that such a requirement would prevent decision-making and stall medical progress. He suggested the EC use data gathered in support of FDA approvals. He asked the EC to question old technologies as well as new innovations. Lastly, he advocated implementing a provisional coverage process so as not to discourage innovation and research.

Vicki Gottlich, an attorney for the Center for Medicare Advocacy, Healthcare Rights Project, stated that her organization represents low-income Medicare beneficiaries. She expressed concern that the current process available to beneficiaries (national coverage, claims and appeals processes) does not protect beneficiary rights and is too slow. She applauded the EC for its efforts in clarifying coverage issues, but pointed out that a lengthy process deprives beneficiaries of prescribed services because of delays incurred while evidence is reviewed. She advocated an efficient coverage determination process, and an effective and expeditious appeals process.

Dr. Larry Weisenthal, a medical oncologist in private practice, expressed his concerns that the Subcommittee’s report lacked adequate consideration of laboratory issues. He stated his frustration after a recent experience at an MCAC meeting on human tumor assay systems (HTAS). He felt that key parts of the proponent’s presentations were not given adequate opportunity for the panel’s review or consideration. He emphasized that his purpose was to help develop a better process for future reviews, not to complain about the past. He encouraged the EC to appreciate important advances in medicine that are occurring outside the traditional NIH and university research arenas and that HCFA must ensure these proponents hold a central role in the evaluation of new technology. He proposed a modification to the overall process by which panel members receive, review, and transmit information to the proponents prior to a meeting. Dr. Weisenthal then submitted written remarks for consideration.

Sandy Sherman presented a statement from Dr. E. Ratcliffe Anderson, Executive Vice President and CEO of the American Medical Association (AMA). Dr. Anderson stated that he was impressed and gratified by the commitment of the EC and HCFA to ensure that MCAC recommendations would be grounded in scientific evidence. The AMA commended the EC for recommending that panelists play an active role in the process. Dr. Anderson stated that the report has allayed fears that those with vested interests, rather than medical evidence, might drive coverage decisions. He concluded by encouraging adoption of the Subcommittee report.

Thomas Meskan, President of Medical Alley, provided a number of suggestions and modifications for improving the report. He requested that a panel’s written explanation be specific about which health outcome (e.g., mortality, morbidity, quality of life, etc.) factored into its decision. Additionally, Mr. Meskan suggested amending some of the report’s wording and volunteered to assist the EC.

The Executive Secretary read into the record a letter received from Dr. Whitney W. Addington, President of the American College of Physicians-American Society of Internal Medicine (ACP-ASIM). The letter offered a number of suggestions on the Subcommittee report and was generally supportive of the report’s recommendations and objectives.

HCFA Presentation. Dr. Jeffrey L. Kang discussed his experience in evaluating new technologies. He asked the EC to consider, during its deliberation how the patient’s condition, availability of alternatives, and risks associated with the service should affect viewing the evidence. Dr. Hugh F. Hill noted HCFA’s intent to balance timeliness with a national coverage process that is accountable. He stated HCFA’s goal to reach well-reasoned, scientifically sound decisions as rapidly as can be consistent with the level of quality. He raised a couple of concerns about the report. He stated that written panel explanations should not delay a decision; and, indicated HCFA’s commitment to present high-quality and well-organized information within stated timeframes –using outside expert review of evidence reports only if required. Dr. Hill reiterated HCFA understood its responsibility for making final coverage decisions, but that MCAC input is integral to the review process.

Open Committee Deliberation. One panelist raised a process issue. He proposed adopting the current Subcommittee report for the next round of medical specialty panel meetings while soliciting written comments to be considered at the next EC meeting. Panelists agreed the present document was an interim version, but preferred to go through the document section by section in order to make comments or suggestions on its language. They expressed their intent to revisit the document at future meetings, as needed and as new information became available. The Chairperson asked the EC to focus on the technical content of the document rather than on its tone, recognizing that written public comments will be forthcoming. The committee agreed with the Chairperson’s request, adding that revisions to the report’s tone could be made if time permitted. The report was then computer-projected as an overhead and the Chairperson solicited remarks from the EC on a paragraph by paragraph basis. Panelists had a number of suggestions and comments. A few panelists were assigned tasks to revise specific passages. These panelists then submitted their remarks for the entire EC’s consideration. The EC openly discussed each suggestion or comment, and if agreed upon, the report was amended to reflect the change. The Committee deliberated extensively on the report’s requirement that a panel must explain its conclusions in writing, commenting on the form and process that should be used, and whether or not the Chair should be responsible for the written explanation.

Motion, Discussion and Vote. The EC voted on the following motion related to the Subcommittee’s document. A quorum of 11 voting members was present, no member was recused due to an appearance of a conflict of interest:

A motion was made and seconded that the operational documents of a panel meeting should consist of the transcripts and meeting summary (prepared by HCFA, using the transcripts and certified by the Chairperson) including the content/reason for the vote. At the panel meeting, the agenda will include an opportunity for each panel member to give their rationale for his/her vote, and an opportunity for the Chairperson to provide his/her opinion. Each member also has the option of providing his/her written opinion, including any dissenting or minority views. The motion carried with 8 for, 2 against, and 1 abstention.

Those opposed were uncertain about when these documents would be prepared or available. Others felt that the Chairperson should be responsible for drafting the explanation of the panel’s conclusions, which should then be circulated to panel members for their comments and/or approval. Another panelist felt the Federal Advisory Committee Act (FACA) reporting requirements were sufficient.

Open Committee Deliberation. The EC continued its discussion on the Subcommittee report and made recommendations regarding the structure of the evidence to be provided to the medical specialty panels. One member suggested the medical specialty panels gain experience creating evidence reports before the EC make recommendations. Another panelist stated her preference to receive the literature, not just a summary report of the evidence. The Industry Representative raised concerns about timeframes, panel involvement, and assembly of the evidence report. The EC also discussed having the evidence report reviewed by experts.

Open Public Comments. During the afternoon session, panelists heard testimony from 4 speakers:

Steven Northrup, Executive Director of the Medical Device Manufacturers Association (MDMA), asked the Committee to keep in mind the incremental nature of medical advances and that they don’t necessarily require de novo review. He asked that the evidence hurdle not be set so high that manufacturers would have difficulty in meeting their requirements. He also stressed that the MCAC process should be timely as to prevent unnecessarily slowing down innovation.

Kenneth Cook, a facilitator for a cancer support group at the University of Maryland Medical Center, commented on issues of external validity and Medicare patient enrollment in clinical trials. He noted that since Medicare does not pay for experimental protocols, and most Medicare beneficiaries don’t carry separate insurance, they are often left out of the clinical studies if they have no other independent means. Secondly, he pointed out the number of patients enrolled in rare disease studies will be small compared to more prevalent disorders. He asked the EC to not require the same degree of proof for all studies.

Thomas Meskan of Medical Alley asked the EC if it would accept public comments concerning tone of the document. The Chairperson stated that the report would be available for comment and those comments will be considered on a periodic basis.

Dr. Steve Lascher, of ACP-ASIM, commented on an overhead that was presented. He noted that statistical power was mentioned on the issue of generalizability. He stated that since statistical power relates to a Type II error, perhaps sample size would be a more appropriate term than generalizability.

Motion, Discussion and Vote. The EC voted on the following motion as it related to the Subcommittee’s document. A quorum of 10 voting members was present, 1 member voted in absentia:

A motion was made and seconded that the EC approve the Subcommittee’s report and recommendations as amended, as an interim report, and that the EC would revisit the report and revise it as needed, in response to comments from MCAC members and the public. The EC would consider revisions to the document at its next meeting. The motion was carried on a unanimous vote, 10 in favor, and 1 one absentee vote in favor.

Adjournment, Wednesday, March 1, 2000, 3:40 p.m. The meeting adjourned at 3:40 p.m.

I approve the minutes of this meeting
as recorded in this summary.
______________________________
Harold C. Sox, M.D.
Chairperson

Discussion Paper for March 1, 2000 Meeting

• Recommendations to Assure Full Consideration of Issues
  
• Recommendations for Evaluating Effectiveness

RECOMMENDATIONS TO ASSURE FULL CONSIDERATION OF ISSUES
Report and Recommendations of the Subcommittee
Of the Executive Committee

At its first meeting last December, the (EC) Executive Committee of the MCAC (Medicare Coverage Advisory Committee) charged a subcommittee of its membership to recommend to the full EC guidance for the MCAC’s six panels. The posting below is the document the subcommittee will present to the EC at the next meeting, March 1, 2000. At that open meeting, this report will be discussed and the public invited to comment.

HCFA (The Health Care Financing Administration) appreciates the continued efforts of our advisory committee as we work together to perfect our new process for coverage decisions. The document that follows was prepared by our advisors and not drafted or controlled by HCFA. We wish it clearly understood that by this posting, we are simply alerting interested parties to the matters that will be discussed at the EC’s next meeting. Wording choices and substantive recommendations in this report are the subcommittee’s own. To enhance the likelihood of an appropriately focused discussion at the open EC meeting March 1, we state here the spirit in which we take the subcommittee’s report.

We view the document posted below as a list of suggested topics that should be considered and addressed to assure full and consistent discussion of issues by the MCAC panels. HCFA will not view this report as a prescription of criteria by which we are to determine coverage, or even an absolute standard by which we may judge the adequacy of evidence.

In short, this document is a list of suggested topics that the MCAC should consider and address in evaluating the clinical evidence and rendering advice to HCFA. Based on that advice and the record, HCFA will make its coverage decision. We are confident that the MCAC and its process will be an enhancement, not a barrier, to the fair and open consideration HCFA will give to proposals for coverage.

We anticipate, at least for now, asking for MCAC advice only on clinical and scientific questions around the medical effectiveness of new items or services, and the comparative effectiveness of new items or services relative to existing alternatives. Again, HCFA views the materials developed as helping to ensure that the MCAC panels have complete discussion around questions posed to them by HCFA. The subcommittee’s draft should not be construed as reflecting a rigid process, or as creating any decision criteria for entry or exit to the HCFA coverage process. Furthermore, we will not ask questions of the MCAC about dollar costs of new items or services.

Finally, we continue to work diligently to develop coverage criteria, explaining what we mean by "reasonable and necessary" in discriminating covered from non-covered items. The development of these criteria is HCFA’s responsibility and these too will not be delegated to any outside body. Nonetheless, we appreciate the EC’s efforts toward open, clear articulation of scientific and evidentiary standards. When the panels offer comments to HCFA about medical evidence, both HCFA and the public should understand the panels’ basis for making those judgements. Those standards are the MCAC’s; we do not take them to be criteria or processes binding to HCFA.

Our profound and continued gratitude goes to the subcommittee members who labored hard and long to produce this document for the EC’s consideration, as well as to all the MCAC members for their service in this vitally important work.

RECOMMENDATIONS FOR EVALUATING EFFECTIVENESS
Executive Committee Working Group

February 21, 2000

Preface: The Health Care Financing Administration (HCFA) convened the Medicare Coverage Advisory Committee (MCAC) to provide advice on scientific and clinical questions regarding coverage. MCAC has six panels, each addressing a different category of medical intervention, and an Executive Committee. The purpose of this Executive Committee document is to provide guidance to the six panels. The goals of this document are to promote consistency (within and between panels) in the reasoning that leads us to a conclusion about the evidence and accountability (to each other and to the public) to explain our reasoning.

Each panel of the Medicare Coverage Advisory Committee will evaluate the adequacy of the evidence and the size of the health effect in determining the effectiveness of new medical products and services (laboratory test, diagnostic procedure, preventive intervention, treatment). This document has two purposes:

These principles and procedures should make the evaluation process more predictable, more consistent, and more understandable. By making the reasoning behind each panel's conclusions more explicit, these principles should also make the MCAC process more accountable.

HCFA is formulating a proposed rule to outline coverage criteria. The following recommendations are provisional and are meant to assist the Panels in their deliberations until HCFA issues further guidance. We will modify these recommendations as needed to respond to the HCFA final rule about the definition and application of the concept of "reasonable and necessary."

Evaluation of Evidence

In advising HCFA about the evidence for a new medical item or service, MCAC will need to answer two questions. First, "is the evidence concerning effectiveness in the Medicare population adequate to draw conclusions about the magnitude of effectiveness relative to other items or services?" Second, "how does the magnitude of effectiveness of the new medical item or service compare to other available interventions?"

The MCAC panels should explore many sources of evidence in assembling the body of evidence to be used in their deliberations. The sources might include the peer-reviewed scientific literature, the recommendations of expert panels, and unpublished data used to secure FDA approval. The quality of the evidence from these sources will vary, and the panels should weigh the evidence according to its quality.

1. Adequacy of evidence: The Panels must determine whether the scientific evidence is adequate to draw conclusions about the effectiveness of the intervention in routine clinical use in the population of Medicare beneficiaries.

Comment: Assessing the adequacy of the evidence is a sine qua non of essentially all modern approaches to the evaluation of medical technologies. Defining what constitutes adequate evidence is a critical step. The committee's definition of adequate evidence includes the validity of the evidence and its general applicability to the population of interest.

Many forms of evidence can be valid, or not, depending on circumstances specific to the individual study. The most rigorous type of evidence is ordinarily a large, well-designed randomized controlled clinical trial. The ideal randomized clinical trial should have appropriate endpoints, should enroll a representative sample of patients, should be conducted in clinical practice in the patient population of interest, and should evaluate interventions (diagnostic tests, surgical procedures, medical devices, drugs) as typically used in routine clinical practice.

When several such well-designed trials yield consistent results, there is likely to be a strong consensus that the evidence is sufficient. This level of evidence will likely be unavailable for many of the interventions that the MCAC panels will evaluate. There may be randomized trials conducted in other populations (e.g., middle-aged men rather than men and women 65 years of age and older), randomized trials with important design flaws (e.g., they are not double-blinded), or non-randomized studies with concurrent controls. Deciding whether such studies constitute valid, applicable evidence can be very difficult.

The Executive Committee believes that general guidelines for deciding whether the evidence is adequate will serve our purposes better than a rigid set of standards. In considering the evidence from any study, the MCAC panels should try to answer these two main questions:

Bias: Does the study systematically over- or underestimate the effect of the intervention because of possible bias or other errors in assigning patients to intervention and control groups?

There are many potential sources of bias. In observational study designs, the investigators simply observe patient care without intervening to allocate patients to intervention or control groups. In such studies, the investigators cannot be sure that they have measured all of the ways in which treated patients differ from untreated patients. If some of these characteristics influence both health outcomes and the likelihood of receiving the intervention, at least part of the measured treatment effect will be a result of the patient characteristics rather than the treatment itself. This particular bias is called selection bias. For example, in comparing a new, extensive surgical procedure to a less extensive operation, researchers might measure survival one year after the two procedures. Surgeons might avoid performing an extensive operation on patients with severe comorbid illness. If, in an observational study, the researchers failed to measure comorbid conditions, they might conclude that the patient groups were similar. If patients who got surgery for a disease had better one year survival than those who did not get surgery, the reason could be the good health of those that the surgeons selected for surgery, rather than the surgery itself.

Random allocation of patients to the intervention under study eliminates systematic selection bias. In a properly designed and conducted randomized trial, apart from random differences, the group of patients receiving the intervention and the group receiving the alternative are identical with respect to all characteristics, measured and unmeasured. The investigators can be fairly certain that any observed difference in health outcomes is the result of the intervention. Unbalanced allocation can occur with randomized allocation of subjects, but it is very unlikely when the study groups contain a large number of patients.

In an observational, non-randomized study, it is usually very difficult to determine whether bias could account for the results. However, there may be important exceptions. For example, if a disease is uniformly fatal within six weeks, and an observational study demonstrates that half of all patients receiving a new treatment survive for at least a year, it is not necessary to conduct a randomized controlled trial to obtain adequate evidence that the treatment is effective. On the other hand, the outcomes of most diseases with and without treatment are less predictable than in this extreme case and depend upon difficult-to-measure aspects of each patient's health. In these diseases, bias can strongly influence the results of observational studies. Bias is especially likely if the intervention under study is dangerous or toxic, because physicians might avoid prescribing it for patients who are particularly likely to suffer ill effects. Clinical trials of treatments for cancers that have an unpredictable natural history, for example, have repeatedly demonstrated that the results of observational studies are misleading.

To detect important bias in observational studies, the Panel will need to carefully consider all of the evidence, including the comprehensiveness of the available data, how physicians selected patients to receive the intervention, and the extent of disease in intervention and control group patients. In some cases, the panel may decide that it cannot draw firm conclusions about effectiveness without randomized trials.

Although a body of evidence consisting of only uncontrolled studies - whether based on anecdotal evidence, testimonials, or case series and disease registries without adequate historical controls - is never adequate, in some cases the panel will determine that observational evidence is sufficient to draw conclusions about effectiveness. When these circumstances apply, the panel must describe possible sources of bias and explain why it decided that bias does not account for the results.

External validity: Do the results apply to the Medicare population?

Historically, many randomized controlled clinical trials excluded older men and women. An increasing number of randomized trials now include elderly men and women. However, simply enrolling older people in proportion to their number in the general population may not be sufficient to determine whether the results of the trial apply to Medicare patients. If the study has too few elderly participants, it might not have the statistical power to detect a clinically important effect in Medicare patients. Clinical trial populations might also differ from the clinically relevant population of Medicare beneficiaries because the trials exclude individuals who have significant comorbid illness or who take many medications. If the study population in the available trials is not the same as the general population of Medicare beneficiaries who would be candidates to receive the intervention, the Panel must state whether the results of the trials apply to typical Medicare patients and explain its reasoning.

Issues of external validity also apply to the intervention. For a drug or device, the intervention is the same when used in different settings. But other interventions may differ from one site to another. For example, the outcomes of a complex surgical procedure can depend heavily on the skills of the surgeons and other staff caring for the patient. If available trials only include sites where surgeons have the best outcomes, the outcomes might be considerably better than what is possible in typical practice settings. The panel must state whether the results are likely to apply to the general practice setting and explain its reasoning.

The second issue to address is the size and direction (more effective, as effective, or less effective) of the health effect that it demonstrates.

2. Size of Health Effect: Evidence from well designed studies (as discussed in preceding section) must establish how the effectiveness of the new intervention compares to the effectiveness of established services and medical items.

Comment: If the evidence is adequate to draw conclusions (as defined above), the next question is the size and direction of the effect compared with interventions that are widely used. In evaluating the evidence for an intervention, the panels should help HCFA make coverage decisions by placing the size and direction of effectiveness, as compared to established services or medical items, into one of these seven categories:

1. Breakthrough technology: The improvement in health outcomes is so large that the intervention becomes standard of care.

2. More effective: The new intervention improves health outcomes by a significant, albeit small, margin as compared with established services or medical items.

3. As effective but with advantages: The intervention has the same effect on health outcomes as established services or medical items but has some advantages (convenience, rapidity of effect, fewer side effects, other advantages) that some patients will prefer.

4. As effective and with no advantages: The intervention has the same effect on health outcomes as established alternatives but with no advantages.

5. Less effective but with advantages: Although the intervention is less effective that established alternatives (but more effective than doing nothing), it has some advantages (such as convenience, tolerability).

6. Less effective and with no advantages: The intervention is less effective than established alternatives (but more effective than doing nothing) and has no significant advantages.

7. Not effective: The intervention has no effect or has deleterious effects on health outcomes when compared with "doing nothing" (e.g., treatment with placebo or patient management without the use of a diagnostic test).

Suggestions for Panel Operations

3. Explanation: A panel must explain its conclusions in writing.

Comment: Adherence to this principle will help to ensure the integrity of the MCAC procedures and judgments and, by making the committee's reasoning processes more explicit and open, provide internal and external accountability. The explanations will serve as a body of "case law" to which the committee can refer in order to maintain consistency in its recommendations. The requirement for written explanations will help the panel structure its discussions and clarify its reasoning. It is also likely to diminish the scope for ambiguity and misunderstanding. The explanation should include a description of any additional research that would be required to strengthen the evidence. The panel chair is responsible for writing the explanation of the panel's conclusions.

4. Structure of evidence provided to the panels: Panels should receive well-organized, high quality background information before beginning its deliberations. The evidence should be summarized in a report, not simply presented as a collection of data or primary studies.

Comment: The integrity of the coverage decision process begins with a complete critical review of the evidence. The standard of excellence for the evidence report should be the best work in the private sector (e.g., Blue Cross-Blue Shield), by professional organizations (e.g., ACP-ASIM), and for other Federally sponsored panels (e.g., the Evidence-based Practice centers technical support for the U.S. Preventive Services Task Force). The evidence reports provided to the MCAC panels should equal, or improve upon, the best work being done by others under circumstances that are similar to those imposed by the schedule of MCAC panel deliberations. Thus, although there may be limited time in which to prepare an evidence report, the MCAC panels expect its quality to be the same or better than that achieved by others working under the same time constraint. In the opinion of the Executive Committee, production of a full evidence report on a typical MCAC panel topic should ordinarily require no more than six months after HCFA refers the topic to MCAC.

5. Panel member involvement: Panel members should take an active role in reviewing the evidence, (1) The panel chair should play an active role in framing the questions that the evidence report must address and the panel must answer. (2) Several panel members should participate actively in designing the evidence review and preparing the evidence report that will lead to coverage determinations. (3) Other panel members should do an in-depth evaluation of the evidence report prior to a panel meeting (primary reviewers)

Comment: The panel chair should assign at least two panel members to work closely with the authors of the evidence reports. The rationale for this recommendation is to ensure that the evidence report covers a sufficient scope of studies, that it considers relevant alternative interventions, and that it will be useful to the panels in other respects. The panel should include some people who have acquired expertise in the topic of a coverage recommendation, in part so that the panel can evaluate the oral presentations of proponents and in part to assure that the panel can fairly evaluate the evidence review. The best way to develop this independent content expertise is to assign panel members to work on the evidence report. Active participation is also the best way for a panel to develop a common level of skill in evaluating evidence.

Each panel member should read the evidence report carefully and understand the main issues that the report addresses. In addition, the Executive Committee recommends that the panel chair assign two primary reviewers for each topic. These reviewers will not be the individuals who assist in the development of the evidence report; they should be new to the topic. They will evaluate the evidence independently of one another. Each will write a 1-2 page report that will include a preliminary recommendation for the quality of the evidence and the strength of the recommendation and a justification of their recommendations. These reports will often form the core of the panel's explanation of its recommendation.

6. Expert review of evidence reports: To ensure that the evidence report is complete and free from bias, the Executive Committee recommends expert review of the evidence reports.

Comment: Ordinarily, this principle will mean subjecting evidence reports to external review. To allow adequate time for the panel to consider all of the evidence, it should ask independent experts to comment upon the evidence report in advance of panel meetings. The opinion of experts is the best way to assure everyone, the public and the panel, that the evidence report is complete and fair. Expert reviewers will have to be very timely in returning their comments and must explain their reasoning clearly. The evidence report and comments of expert reviewers will be part of the public record, which will provide an opportunity for members of the public to comment on the evidence used by the panel at the time of the panel meeting. The Executive Committee envisions that the panel will choose a small number of expert reviewers (perhaps no more than six) and will require a reply within one month. A reviewer may ask the panel's industry representative to obtain additional information from industry sources. Of course, experts and proponents will also have an opportunity to address the panel at the time of the panel meeting, but reading independent, expert reviews of evidence reports in advance of a panel meeting will help prepare the panel to understand the verbal testimony.