Guidance for Industry Development and Use of Risk Minimization Action Plans [PDF version] Additional copies are available from: Office of Training and Communication Division of Drug Information, HFD-240 Center for Drug Evaluation and Research Food and Drug Administration 5600 Fishers Lane Rockville, MD 20857 (Tel) 301-827-4573 http://www.fda.gov/cder/guidance/index.htm Office of Communication, Training and Manufacturers Assistance, HFM-40 Center for Biologics Evaluation and Research Food and Drug Administration 1401 Rockville Pike, Rockville, MD 20852-1448 http://www.fda.gov/cber/guidelines.htm. (Tel) Voice Information System at 800-835-4709 or 301-827-1800 U.S. Department of Health and Human Services Food and Drug Administration Center for Drug Evaluation and Research (CDER) Center for Biologics Evaluation and Research (CBER) March 2005 Clinical Medical TABLE OF CONTENTS INTRODUCTION BACKGROUND PDUFA III's Risk Management Guidance Goal Overview of the Risk Management Draft Guidance Documents THE ROLE OF RISK MINIMIZATION AND RISKMAPS IN RISK MANAGEMENT Relationship Between a Product's Benefits and Risks Determining an Appropriate Risk Minimization Approach Definition of Risk Minimization Action Plan (RiskMAP) Determining When a RiskMAP Should Be Considered TOOLS FOR ACHIEVING RISKMAP GOALS AND OBJECTIVES Relationship of RiskMAP Tools to Objectives and Goals Categories of RiskMAP Tools Targeted Education and Outreach Reminder Systems Performance-Linked Access Systems Description of RiskMAP Tools Selecting and Developing the Best Tools Mechanisms Available to the FDA to Minimize Risks RISKMAP EVALUATION: ASSESSING THE EFFECTIVENESS OF TOOLS AND THE PLAN Rationale for RiskMAP Evaluation Considerations in Designing a RiskMAP Evaluation Plan Selecting Evidence-Based Performance Measures Compensating for an Evaluation Method's Limitations Evaluating the Effectiveness of Tools in Addition to RiskMAP Goals Evaluating RiskMAP Tools Prior to Implementation FDA Assessment of RiskMAP Evaluation Results Making Information From RiskMAP Evaluations Available to the Public COMMUNICATING WITH FDA REGARDING RISKMAP DEVELOPMENT AND DESIGN ISSUES RECOMMENDED ELEMENTS OF A RISKMAP SUBMISSION TO FDA Contents of a RiskMAP Submission to FDA Background Goals and Objectives Strategy and Tools Evaluation Plan Contents of a RiskMAP Progress Report Summary Methodology Data Results Discussion and Conclusions Contains Nonbinding Recommendations Guidance for Industry1 Development and Use of Risk Minimization Action Plans This guidance represents the Food and Drug Administration's (FDA's) current thinking on this topic. It does not create or confer any rights for or on any person and does not operate to bind FDA or the public. You can use an alternative approach if the approach satisfies the requirements of the applicable statutes and regulations. If you want to discuss an alternative approach, contact the FDA staff responsible for implementing this guidance. If you cannot identify the appropriate FDA staff, call the appropriate number listed on the title page of this guidance. INTRODUCTION This document provides guidance to industry on the development, implementation, and evaluation of risk minimization action plans for prescription drug products, including biological drug products. 2 In particular, it gives guidance on (1) initiating and designing plans called risk minimization action plans or RiskMAPs to minimize identified product risks, (2) selecting and developing tools to minimize those risks, (3) evaluating RiskMAPs and monitoring tools, and (4) communicating with FDA about RiskMAPs, and (5) the recommended components of a RiskMAP submission to FDA. FDA's guidance documents, including this guidance, do not establish legally enforceable responsibilities. Instead, guidances describe the Agency's current thinking on a topic and should be viewed only as recommendations, unless specific regulatory or statutory requirements are cited. The use of the word should in Agency guidances means that something is suggested or recommended, but not required. Table of Contents BACKGROUND A. PDUFA III's Risk Management Guidance Goal On June 12, 2002, Congress reauthorized, for the second time, the Prescription Drug User Fee Act (PDUFA III). In the context of PDUFA III, FDA agreed to satisfy certain performance goals. One of those goals was to produce guidance for industry on risk management activities for drug and biological products. As an initial step towards satisfying that goal, FDA sought public comment on risk management. Specifically, FDA issued three concept papers. Each paper focused on one aspect of risk management, including (1) conducting premarketing risk assessment, (2) developing and implementing risk minimization tools, and (3) performing postmarketing pharmacovigilance and pharmacoepidemiologic assessments. In addition to receiving numerous written comments regarding the three concept papers, FDA held a public workshop on April 9-11, 2003, to discuss the concept papers. FDA considered all of the comments received in developing the three draft guidance documents on risk management activities. The draft guidance documents were published on May 5, 2004, and the public was provided with an opportunity to comment on them until July 6, 2004. FDA considered all of the comments received in producing the final guidance documents: 1. Premarketing Risk Assessment (Premarketing Guidance) 2. Development and Use of Risk Minimization Action Plans (RiskMAP Guidance) 3. Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment (Pharmacovigilance Guidance) B. Overview of the Risk Management Guidances Like the concept papers and draft guidances that preceded them, each of the three final guidance documents focuses on one aspect of risk management. The Premarketing Guidance and the Pharmacovigilance Guidance focus on premarketing and postmarketing risk assessment, respectively. The RiskMAP Guidance focuses on risk minimization. Together, risk assessment and risk minimization form what FDA calls risk management. Specifically, risk management is an iterative process of (1) assessing a product's benefit-risk balance, (2) developing and implementing tools to minimize its risks while preserving its benefits, (3) evaluating tool effectiveness and reassessing the benefit-risk balance, and (4) making adjustments, as appropriate, to the risk minimization tools to further improve the benefit-risk balance. This four-part process should be continuous throughout a product's lifecycle, with the results of risk assessment informing the sponsor's decisions regarding risk minimization. When reviewing the recommendations provided in this guidance, sponsors and applicants should keep the following points in mind: Many recommendations in this guidance are not intended to be generally applicable to all products. Industry already performs risk assessment and risk minimization activities for products during development and marketing. The Federal Food, Drug, and Cosmetic Act (FDCA) and FDA implementing regulations establish requirements for routine risk assessment and risk minimization (see e.g., FDA requirements for professional labeling, and adverse event monitoring and reporting). As a result, many of the recommendations presented here focus on situations when a product may pose a clinically important and unusual type or level of risk. To the extent possible, we have specified in the text whether a recommendation is intended for all products or only this subset of products. It is of critical importance to protect patients and their privacy during the generation of safety data and the development of risk minimization action plans. During all risk assessment and risk minimization activities, sponsors must comply with applicable regulatory requirements involving human subjects research and patient privacy.3 To the extent possible, this guidance reflects FDA's commitment to harmonization of international definitions and standards. When planning risk assessment and risk minimization activities, sponsors should consider input from healthcare participants likely to be affected by these activities (e.g., from consumers, pharmacists and pharmacies, physicians, nurses, and third party payers). There are points of overlap among the three guidances. We have tried to note in the text of each guidance when areas of overlap occur and when referencing one of the other guidances might be useful. Table of Contents THE ROLE OF RISK MINIMIZATION AND RISKMAPS IN RISK MANAGEMENT As described in section II.B, FDA views risk management as an iterative process encompassing the assessment of risks and benefits, the minimization of risks, and the maximization of benefits. Specifically, the premarketing guidance and the pharmacovigilance guidance discuss how sponsors should engage in evidence-based risk assessment for all products in development and on the market to define the nature and extent of a product's risks in relation to its benefits. The goal of risk minimization is to minimize a product's risks while preserving its benefits. For the majority of products, routine risk minimization measures are sufficient to minimize risks and preserve benefits. Only a few products are likely to merit consideration for additional risk minimization efforts (see section III.D). Efforts to maximize benefits to improve the overall balance of risks and benefits can be pursued in concert with risk minimization efforts and can be discussed with FDA. A. Relationship Between a Product's Benefits and Risks The statutory standard for FDA approval of a product is that the product is safe and effective for its labeled indications under its labeled conditions of use (see sections 201(p)(1) and 505(d) of the Federal Food, Drug, and Cosmetic Act (21 U.S.C. 321(p)(1) and 355(d)). FDA's determination that a product is safe, however, does not suggest an absence of risk. Rather, a product is considered to be safe if the clinical significance and probability of its beneficial effects outweigh the likelihood and medical importance of its harmful or undesirable effects. In other words, a product is considered safe if it has an appropriate benefit-risk balance for the intended population and use. Benefit and risk information emerges continually throughout a product's lifecycle (i.e., during the investigational and marketing phases) and can reflect the results of both labeled and off-label uses. Benefits and risks can result in a range of corresponding positive and negative effects on patient outcomes that may (1) be cosmetic, symptomatic, or curative; (2) alter the course of the disease; or (3) affect mortality. Benefits and risks are difficult to quantify and compare because they may apply to different individuals and are usually measured and valued differently. Examples of factors to weigh are (1) population risks and benefits, (2) individual benefits from treatment, (3) risks of nontreatment or alternative products, and (4) modest population benefits in the context of a serious adverse effect that occurs rarely or unpredictably. Benefits as well as risks are also patient-specific and are influenced by such factors as (1) the severity of the disease being treated, (2) the outcome of the disease if untreated, (3) the probability and magnitude of any treatment effect, (4) existing therapeutic options, and (5) the individual's understanding of risks and benefits and the value they attach to each of them. Thus, assessment and comparison of a product's benefits and risks is a complicated process that is influenced by a wide range of societal, healthcare, and individualized patient factors. B. Determining an Appropriate Risk Minimization Approach To help ensure safe and effective use of their products, sponsors have always sought to maximize benefits and minimize risks. FDA believes that, for most products, routine risk minimization measures are sufficient. Such measures involve, for example, FDA-approved professional labeling describing the conditions in which the drug can be used safely and effectively, updated from time to time to incorporate information from postmarketing surveillance or studies revealing new benefits (e.g., new indications or formulations) or risk concerns. Efforts to make FDA-approved professional labeling clearer, more concise, and better focused on information of clinical relevance reflect the Agency's belief that communication of risks and benefits through product labeling is the cornerstone of risk management efforts for prescription drugs. 4 For most products, routine risk management will be sufficient and a RiskMAP need not be considered. There are, however a small number of products for which a RiskMAP should be considered (see section III.D). FDA recommends that RiskMAPs be used judiciously to minimize risks without encumbering drug availability or otherwise interfering with the delivery of product benefits to patients. This guidance focuses on the development, implementation, and evaluation of RiskMAPs. C. Definition of Risk Minimization Action Plan (RiskMAP) As used in this document, the term RiskMAP means a strategic safety program designed to meet specific goals and objectives in minimizing known risks of a product while preserving its benefits. A RiskMAP targets one or more safety-related health outcomes or goals and uses one or more tools to achieve those goals.5 A RiskMAP could also be considered as a selectively used type of Safety Action Plan as defined in the International Conference on Harmonization (ICH) guidance E2E: Pharmacovigilance Planning (E2E guidance). 6 FDA recommends that RiskMAP goals target the achievement of particular health outcomes related to known safety risks. FDA suggests that sponsors state goals in a way that aims to achieve maximum risk reduction. The following are examples of RiskMAP goals: "patients on X drug should not also be prescribed Y drug" or "fetal exposures to Z drug should not occur." FDA recommends that goals be stated in absolute terms. Although it might not be possible to ensure that absolutely no one on X drug receives Y drug, FDA believes that a goal, as the term implies, is a statement of the ideal outcome of a RiskMAP. FDA recommends that RiskMAP goals be translated into pragmatic, specific, and measurable program objectives that result in processes or behaviors leading to achievement of the RiskMAP goals. Objectives can be thought of as intermediate steps to achieving the overall RiskMAP goal. A RiskMAP goal can be translated into different objectives, depending upon the frequency, type, and severity of the specific risk or risks being minimized. For example, a goal may be the elimination of dangerous concomitant prescribing. The objectives could include lowering physician co-prescribing rates and/or pharmacist co-dispensing rates. As described in greater detail in section IV, many processes or systems to minimize known safety risks are available or under development for use in RiskMAPs. These systems include: targeted education and outreach to communicate risks and appropriate safety behaviors to healthcare practitioners or patients reminder systems, processes, or forms to foster reduced-risk prescribing and use performance-linked access systems that guide prescribing, dispensing, and use of the product to target the population and conditions of use most likely to confer benefits and to minimize particular risks For certain types of risks (e.g., teratogenicity of category X drug products), it may be possible to develop systems with similar processes and procedures that can be used industrywide. The use of these systems can occur outside of a RiskMAP. For example, while most drugs do not need a RiskMAP, many would still benefit from a program of physician and patient education and outreach. At times, communication of potential product risks may be warranted before a sponsor agrees to do a RiskMAP or an agreed upon RiskMAP is completed. D. Determining When a RiskMAP Should Be Considered7 As described in the premarketing guidance and pharmacovigilance guidance, evidence-based risk identification, assessment, and characterization are processes that continue throughout a product's lifecycle. Therefore, a risk warranting the consideration of a RiskMAP could emerge during premarketing or postmarketing risk assessment. 8 The Agency recommends that the appropriate information for consideration in making such a determination include, as applicable, (1) data from the clinical development program, postmarketing surveillance, and phase 4 studies, and (2) the product's intended population and use. Although it is expected and hoped that sponsors will determine when a RiskMAP would be appropriate, FDA may recommend a RiskMAP based on the Agency's own interpretation of risk information. Decisions to develop, submit, or implement a RiskMAP are always made on a case-by-case basis, but several considerations are common to most determinations of whether development of a RiskMAP may be desirable: Nature and rate of known risks versus benefits: Comparing the characteristics of the product's adverse effects and benefits may help clarify whether a RiskMAP could improve the product's benefit-risk balance. The characteristics to be weighed might include the (1) types, magnitude, and frequency of risks and benefits; (2) populations at greatest risk and/or those likely to derive the most benefit; (3) existence of treatment alternatives and their risks and benefits; and (4) reversibility of adverse events observed. Preventability of adverse effects: Serious adverse effects that can be minimized or avoided by preventive measures around drug prescribing are the preferred candidates for RiskMAPs. Probability of benefit: If factors are identified that can predict effectiveness, a RiskMAP could help encourage appropriate use to increase benefits relative to known risks. Consider the following examples: Opiate drug products have important benefits in alleviating pain but are associated with significant risk of overdose, abuse, and addiction. The Agency recommends that sponsors of Schedule II controlled substances, including Schedule II extended release or high concentration opiate drug products, consider developing RiskMAPs for these products. Drugs that provide important benefits, but that are human teratogens would often be appropriate for a RiskMAP to minimize in utero exposure. Some drugs may warrant RiskMAP consideration because safe and effective use call for specialized healthcare skills, training, or facilities to manage the therapeutic or serious side effects of the drug. Involving all stakeholders during the initial phases of considering whether a RiskMAP is appropriate allows input and buy-in by all parties who will later have roles in implementing the RiskMAP. If a RiskMAP is appropriate, stakeholders can help shape the RiskMAP to foster its success in the healthcare delivery environment. Therefore, we recommend public discussion about the appropriateness of a RiskMAP through the FDA advisory committee process. Such public advisory committee meetings can also be used to address (1) whether a RiskMAP is appropriate, (2) what the goals and objectives of the RiskMAP could be (see footnote 6), (3) the circumstances under which a RiskMAP tool might be revised or terminated, and (4) whether a RiskMAP itself is no longer appropriate. The FDA advisory committee structure and processes are well suited to foster such discussions as they arise on a case-by-case basis. Table of Contents TOOLS FOR ACHIEVING RISKMAP GOALS AND OBJECTIVES A risk minimization tool is a process or system intended to minimize known risks. Tools can communicate particular information regarding optimal product use and can also provide guidance on prescribing, dispensing, and/or using a product in the most appropriate situations or patient populations. A number of tools are available; FDA encourages and anticipates the development of additional tools. A. Relationship of RiskMAP Tools to Objectives and Goals Risk minimization tools are designed to help achieve one or more RiskMAP objectives that are directed at the overall RiskMAP goal or goals. One or more tools can be chosen to achieve a particular objective. For example, a goal might be that patients with condition A should not be exposed to product B. An objective for achieving this goal might be to communicate to patients that if they have condition A, they should not take product B. Depending on the likelihood and severity of the adverse event associated with product B in a patient with condition A, a variety of tools could be applied to achieve this objective. One possible tool would be patient labeling explaining that a patient with condition A should not take product B. On the other hand, if the potential harm to a patient with condition A is severe and/or likely to occur, a more active tool may be appropriate. For example, the sponsor could choose to develop a patient agreement where, before receiving the product, the patient formally acknowledges their understanding and/or agreement not to take product B if he or she has condition A. B. Categories of RiskMAP Tools A variety of tools are currently used in risk minimization plans. These fall within three categories: (1) targeted education and outreach, (2) reminder systems, and (3) performance-linked access systems. A RiskMAP might include tools from one or more categories, depending on its risk minimization goals. FDA notes that the use of tools in different categories does not imply greater or lesser safety risks, but rather indicates the particular circumstances put in place to achieve the objectives and goals. 1. Targeted Education and Outreach FDA recommends that sponsors consider tools in the targeted education and outreach category (1) when routine risk minimization is known or likely to be insufficient to minimize product risks or (2) as a component of RiskMAPs using reminder or performance-linked access systems (see sections IV.B.2 and 3 below). Tools in this category employ specific, targeted education and outreach efforts about risks to increase appropriate knowledge and behaviors of key people or groups (e.g., healthcare practitioners and consumers) that have the capacity to prevent or mitigate the product risks of concern. FDA acknowledges that tools in this category are occasionally used for products where the benefit/risk balance does not necessarily warrant a RiskMAP. Educational efforts by sponsors might include one or more of the tools described below without a RiskMAP being in place. Sponsors are encouraged to continue using tools, such as education and outreach, as an extension of their routine risk minimization efforts even without a RiskMAP. Examples of tools in this category are as follows: healthcare practitioner letters training programs for healthcare practitioners or patients continuing education for healthcare practitioners such as product-focused programs developed by sponsors and/or sponsor-supported accredited CE programs prominent professional or public notifications patient labeling such as Medication Guides and patient package inserts promotional techniques such as direct-to-consumer advertising highlighting appropriate patient use or product risks patient-sponsor interaction and education systems such as disease management and patient access programs In addition to informing healthcare practitioners and patients about conditions of use contributing to product risk, educational tools can inform them of conditions of use that are important to achieve the product's benefits. For example, a patient who takes a product according to labeled instructions is more likely to achieve maximum product effectiveness. On the other hand, deviations from the labeled dose, frequency of dosing, storage conditions, or other labeled conditions of use might compromise the benefit achieved, yet still expose the patient to product-related risks. Risks and benefits can have different dose-response relationships. Risks can persist and even exceed benefits when products are used in ways that minimize effectiveness. Therefore, educational tools can be used to explain how to use products in ways that both maximize benefits and minimize risks. 2. Reminder Systems We recommend that tools in the reminder systems category be used in addition to tools in the targeted education and outreach category when targeted education and outreach tools are known or likely to be insufficient to minimize identified risks. Tools in this category include systems that prompt, remind, double-check or otherwise guide healthcare practitioners and/or patients in prescribing, dispensing, receiving, or using a product in ways that minimize risk. Examples of tools in this category are as follows: Patient education that includes acknowledgment of having read the material and an agreement to follow instructions. These agreements are sometimes called consent forms. Healthcare provider training programs that include testing or some other documentation of physicians' knowledge and understanding. Enrollment of physicians, pharmacies, and/or patients in special data collection systems that also reinforce appropriate product use. Limited number of doses in any single prescription or limitations on refills of the product. Specialized product packaging to enhance safe use of the product. Specialized systems or records that are used to attest that safety measures have been satisfied (e.g., prescription stickers, physician attestation of capabilities). 3. Performance-Linked Access Systems Performance-linked access systems include systems that link product access to laboratory testing results or other documentation. Tools in this category, because they are very burdensome and can disrupt usual patient care, should be considered only when (1) products have significant or otherwise unique benefits in a particular patient group or condition, but unusual risks also exist, such as irreversible disability or death, and (2) routine risk minimization measures, targeted education and outreach tools, and reminder systems are known or likely to be insufficient to minimize those risks. Examples of tools in this category include: the sponsor's use of compulsory reminder systems, as described in the previous section (e.g., the product is not made available unless there is an agreement or acknowledgment, documented qualifications, enrollment, and/or appropriate testing or laboratory records) prescription only by specially certified healthcare practitioners product dispensing limited to pharmacies or practitioners that elect to be specially certified product dispensing only to patients with evidence or other documentation of safe-use conditions (e.g., lab test results) Performance-linked access systems should seek to avoid unnecessary or unintended restrictions or fragmentation of healthcare services that may limit access by physicians, pharmacists, or patients, or that may lead to discontinuities in medical or pharmacy care. C. Description of RiskMAP Tools FDA plans to develop a RiskMAP Web site that will include (1) descriptions of tools that are currently used in RiskMAPs and (2) other information relevant to RiskMAP development (see section IV.D below). The information will be made available consistent with federal law and regulations governing disclosure of information by FDA to the public. The list of tools will be intended to assist sponsors in designing a RiskMAP but will not suggest that the listed tools are FDA-approved or -validated. On the contrary, FDA does not suggest that the tools listed on the Web site are the only tools that could be useful and encourages sponsors to develop tools that may be optimal for their particular products. See also Section V.D on making information from RiskMAP evaluations available to the public. D. Selecting and Developing the Best Tools Given the variety of available tools, FDA recommends that a sponsor carefully consider which tool or tools are most appropriate, given the goals and objectives of its product's RiskMAP. A tool could be developed or selected based on its individual impact and/or because of its impact when used in coordination with other tools. Generally, the best tools would be those that have a high likelihood of achieving their objective based on positive performance in other RiskMAPs or in similar settings and populations. Relevant non-RiskMAP evidence and experience can be found in healthcare quality initiatives, public health education and outreach, marketing, and other outcomes-based research (see section V for a more detailed discussion of evaluating tools' effectiveness). Although FDA suggests that the best tool or tools be selected on a case-by-case basis, the following are generally applicable considerations in designing a RiskMAP. In choosing tools for a RiskMAP, FDA recommends that sponsors: Maintain the widest possible access to the product with the least burden to the healthcare system that is compatible with adequate risk minimization (e.g., a reminder system tool should not be used if targeted education and outreach would likely be sufficient). Identify the key stakeholders who have the capacity to minimize the product's risks (such as physicians, pharmacists, pharmacies, nurses, patients, and third-party payers) and define the anticipated role of each group. Seek input from the key stakeholders on the feasibility of implementing and accepting the tool in usual healthcare practices, disease conditions, or lifestyles, if possible. Examples of considerations could include (but would not be limited to) patient and healthcare practitioner autonomy, time effectiveness, economic issues, and technological feasibility. Acknowledge the importance of using tools with the least burdensome effect on healthcare practitioner-patient, pharmacist-patient, and/or other healthcare relationships. Design the RiskMAP to be: compatible with current technology applicable to both outpatient and inpatient use accessible to patients in diverse locales, including non-urban settings consistent with existing tools and programs, or systems that have been shown to be effective with similar products, indications, or risks Select tools based on available evidence of effectiveness in achieving the specified objective (e.g., tools effectively used in pregnancy prevention). Consider indirect evidence of tool effectiveness in a related area that supports the rationale, design, or method of use (e.g., tools applied in modifying patient or healthcare practitioner behaviors in medical care settings). Consider, and seek to avoid, unintended consequences of tool implementation that obstruct risk minimization and product benefit, such as obstructing patient access or driving patients to seek alternative product sources (e.g., Internet sales, counterfeit products) or less appropriate products. FDA recognizes that once it approves a product for marketing, healthcare practitioners are the most important managers of product risks. FDA believes that by including information in the FDA-approved professional labeling on the conditions in which medical products can be used safely and effectively by their intended population and for their intended use or uses, the Agency and the sponsor encourage healthcare practitioners to prescribe medical products in circumstances that yield a favorable benefit-risk balance. However, as the Agency has long recognized, the FDCA and FDA regulations establish requirements governing the safety and effectiveness of medical products. FDA does not have authority under these provisions to control decisions made by qualified healthcare practitioners to prescribe products for conditions other than those described in FDA-approved professional labeling, or to otherwise regulate medical or surgical practice. E. Mechanisms Available to the FDA to Minimize Risks This guidance focuses on the tools that industry can incorporate into RiskMAPs. As noted, FDA has a variety of risk management measures at its disposal under the FDCA and FDA regulations (see e.g., FDA requirements for professional labeling and adverse event monitoring and reporting). FDA must occasionally invoke other mechanisms to minimize the risks from medical products that pose serious risks to the public health. These tools include: FDA-requested product recalls, warning and untitled letters, and import alerts safety alerts, guidance documents, and regulations judicial enforcement procedures such as seizures or injunctions Further information on these mechanisms is available on the Internet at http://www.fda.gov. Table of Contents RISKMAP EVALUATION: ASSESSING THE EFFECTIVENESS OF TOOLS AND THE PLAN As FDA and sponsors seek additional knowledge about the design, effectiveness, burdens, and potential unintended consequences of RiskMAPs, it is important to collect as much information as possible on plan performance. RiskMAPs and their component objectives and tools should be monitored and evaluated in a timely manner to identify areas for improvement. A. Rationale for RiskMAP Evaluation At least two studies have documented poor or limited implementation and effectiveness of traditional risk minimization tools. In particular, the studies examined situations in which labeling changes (with or without Dear Healthcare Practitioner letters) were used to reduce safety problems. 9 The iterative process of risk assessment, risk minimization, and reevaluation previously described is intended to avoid repeating these experiences by identifying poorly performing or ineffective RiskMAPs or RiskMAP components as soon as possible. Ultimately, RiskMAP evaluation is intended to ensure that the energy and resources expended on risk minimization are actually achieving the desired goals of continued benefits with minimized risks. FDA considers evaluation of the effectiveness of a RiskMAP to be important and recommends that every RiskMAP contain a plan for periodically evaluating its effectiveness after implementation (see section VII for a detailed discussion of RiskMAP submissions to FDA). 10 The evaluation of RiskMAPs can take several forms. Most critical is determining the performance of the overall RiskMAP in achieving its targeted health outcomes or goals. Separate but related assessments can be done for (1) individual tool performance, (2) acceptability of RiskMAP tools by consumers and healthcare practitioners, and (3) compliance with important RiskMAP processes or procedures. Generally, FDA anticipates that RiskMAP evaluations would involve the analysis of observational or descriptive data. The specific types of data gathered in a RiskMAP evaluation will determine whether it would be appropriate to include a statistical analysis of evaluation results. B. Considerations in Designing a RiskMAP Evaluation Plan FDA recommends that RiskMAP evaluation plans be tailored to the specific product and designed to assess whether the RiskMAP's goals have been achieved through its objectives and tools. The following are generally applicable guidelines for sponsors designing RiskMAP evaluation plans. 1. Selecting Evidence-Based Performance Measures The Agency recommends that sponsors select well-defined, evidence-based, and objective performance measures tailored to the particular RiskMAP to determine whether the RiskMAP's goals or objectives are being achieved. An appropriate measure could be a number, percentage, or rate of an outcome, event, process, knowledge, or behavior. Ideally, the chosen measure would directly measure the RiskMAP's health outcome goal. For example, for a RiskMAP with a goal of preventing a particular complication outcome from product use, a sample performance measure could be the complication rate. For evaluation purposes, a target for that measure could be established to be no more than a specified number or rate of that complication. In some cases, however, a health outcome cannot be practically or accurately measured. In those cases, other measures can be used that are closely related to the health outcome, such as the following: Surrogates for health outcome measures (e.g., emergency room visits for an adverse consequence, pregnancy test results for determining if pregnancy occurred). The sensitivity, specificity, and predictive value of surrogate markers should be established before their use as a performance measure. Process measures that reflect desirable safety behaviors (e.g., performance of recommended laboratory monitoring, signatures attesting to knowledge or discussions of risk). Assessments of comprehension, knowledge, attitudes, and/or desired safety behaviors about drug safety risks (e.g., provider, pharmacist, or patient surveys). FDA recommends that the validity of a measure be judged by how closely it is related to the desired health outcome goal of the RiskMAP. Simply stated, the more closely related a measure is to the RiskMAP goal, the greater its degree of validity. For example, if the RiskMAP goal is avoidance of liver failure, then ascertainment of the rate of liver failure in the user population would be a highly valid performance measure. Hospitalization for severe liver injury would be another, but less direct, assessment of the RiskMAP goal. The frequency of liver function monitoring in users could be used to see if RiskMAP processes to prevent liver failure were being followed, but since liver function monitoring may not be tightly linked to the occurrence of liver failure, such process monitoring would have limited validity as an indicator of successful prevention of liver failure. 2. Compensating for an Evaluation Method's Limitations Most evaluation measures have limitations. FDA suggests that, in choosing among evaluation methods and measures, sponsors consider their strengths and limitations. The following are examples of some of the limitations of evaluation methods: Spontaneous adverse event data are a potentially biased outcome measure because reporting of adverse events varies due to many factors and represents an unknown and variable fraction of the adverse outcomes that are actually occurring. As a result, systematic data collection or active surveillance of adverse events in populations with well-defined exposure to the product would be preferred for purposes of evaluation. Population-based evaluation methods can use administrative or claims-based data systems that capture service or payment claims to measure rates of events, although it is usually recommended that medical records be examined to validate the actual occurrence of coded diagnoses and procedures. Administrative data may come from various insurers, purchasing groups, or networks that are tied to employment or entitlement programs, so it is important to determine if an administrative data system is representative of the general population being treated with the product. Also, unless enrollment in an administrative claims system is large, the number of patients exposed to any single product is likely to be limited, as will be the power to detect uncommon adverse events. 11 In addition, there may be data processing time lags of several months or longer before administrative data can be retrieved and analyzed. Active surveillance using sentinel reporting sites may be useful for evaluating adverse events, but it is costly and may not detect rare events. Surveys of healthcare practitioners or patients using various modes (in-person, mail, telephone, electronic) can be another useful form of active surveillance of knowledge, attitudes, policies, and practices of healthcare practitioners, institutions, and patients about recommended RiskMAP tools and their associated processes. However, issues relating to response rates, representativeness, and reporting biases may limit the accuracy of survey results. 12 These examples illustrate how using only one evaluation method could skew assessment of the performance of a RiskMAP. Therefore, FDA recommends that, whenever feasible, sponsors design evaluation plans to include at least two different quantitative, representative, and minimally biased evaluation methods for each critical RiskMAP goal. By using two methods, one method can compensate for the limitations of the other. For example, surveys of healthcare practitioners may indicate high compliance with systems for preventing product complications. However, systematically collected or spontaneous reports might show that product complications are occurring, thus suggesting that prevention efforts in actual practice may be ineffective or incompletely applied. If it is not practical to use two complementary and representative methods, FDA suggests using other quantitative methods such as multiple site sampling or audits that aim for high coverage or response rates by the affected population. If RiskMAPs use multiple tools or interventions, it may be useful to consider using evaluation methods applicable to the program as a whole. For example, a systematic program evaluation model, such as Failure Modes and Effect Analysis (FMEA),13 ,14 can provide a framework for evaluating the individual RiskMAP components and the relative importance of each in achieving the overall RiskMAP goal or goals. 3. Evaluating the Effectiveness of Tools in Addition to RiskMAP Goals FDA recommends that sponsors periodically evaluate each RiskMAP tool to ensure it is materially contributing to the achievement of RiskMAP objectives or goals. Tools that do not perform well may compromise attainment of RiskMAP goals, add unnecessary costs or burdens, or limit access to product benefits without minimizing risks. Tools that are implemented incompletely or in a substandard fashion could result in additional tools being adopted unnecessarily. For all these reasons, evaluating tools is important. Data from such evaluations may make it possible to improve a tool's effectiveness or eliminate the use of a tool that fails to contribute to achieving a RiskMAP goal. By eliminating ineffective tools, resources can be concentrated on useful tools. Distinguishing between the evaluation of RiskMAP goals and tools is important because the achievement of goals and the performance of tools may not be linked. For example, the overall goal of a RiskMAP may be achieved despite individual tools performing poorly. The reverse situation may also occur, with component tools performing well but without appropriate progress in achieving the RiskMAP goal. This situation may occur if a surrogate objective correlates poorly to the desired health outcome. The first example (i.e., the RiskMAP goal may be achieved despite individual tools performing poorly) may afford an opportunity to discontinue a tool, whereas its converse may trigger the implementation of new or improved tools, or even a redesign of the overall RiskMAP. Two important factors that contribute to tool effectiveness are its acceptability and unintended consequences. Since tool performance will often depend upon the understanding, cooperation, efforts, and resources of healthcare providers, pharmacists, and patients, evaluation of acceptability and unintended consequences for individual tools may help to improve the use of tools and thus their performance. 4. Evaluating RiskMAP Tools Prior to Implementation FDA recommends that, to the extent possible, sponsors evaluate tools for effectiveness before implementation. As discussed in section IV.D, FDA suggests that in selecting tools to include in a RiskMAP, a sponsor consider tools that are likely to be effective. For example, the success of potential RiskMAP tools might be predicted to some extent by evidence in the scientific literature or from their use in other RiskMAPs. Application of computer modeling or simulation techniques may also assist in projecting potential outcomes of implementation of various combinations of RiskMAP tools. Besides using literature evidence and past RiskMAP experience to identify tools with a known track record of effectiveness, sponsors can pretest or pilot test a tool before implementation. Such testing, ideally with a comparison group or time period, can help to assess comprehension, acceptance, feasibility, and other factors that influence how readily RiskMAP tools will fit into patient lifestyles and the everyday practices of healthcare practitioners. Pretesting can potentially avoid wasted time, expense, and escalation of RiskMAP tools by discriminating between high- and low-performing tools. For example, if a preventable risk is identified in Phase 2 trials, Phase 3 trials could provide an opportunity to pretest targeted education and outreach tools. FDA recommends that pretesting methods be chosen on a case-by-case basis, depending on the product, tool, objective, and goal. For example, in certain preapproval situations, large simple safety studies may be a means of generating useful information about the effectiveness of RiskMAP tools in conditions close to actual practice. 15 On the other hand, for certain tools such as targeted education and outreach, published best practices could be used as guidelines for implementation. If time is particularly limited, multiple interviews or focus group testing can assist in determining acceptance or comprehension of a RiskMAP tool by major stakeholder groups. This action might be particularly useful in situations where risks and benefits are closely matched, and RiskMAP goals may include the making of informed therapeutic choices by patients and prescribers. FDA recognizes that, in some cases, tools cannot be pretested for logistical reasons. Pretesting of tools may not be practical in situations in which newly recognized adverse events dictate the importance of rapid implementation of a RiskMAP after approval and marketing. In such instances, sponsors should seek to employ tools with a proven track record of effectiveness. In general, the greater the rate or severity of risks to be minimized, the more critical it becomes to have compelling evidence of effectiveness of the tool through some form of testing or prior use. C. FDA Assessment of RiskMAP Evaluation Results FDA recommends that if a sponsor makes a RiskMAP submission to the Agency, the submission describe when the sponsor will send periodic evaluation results to FDA. As discussed in section VII.B, the Agency recommends that sponsors analyze evaluation results and requests that sponsors provide FDA with (1) the data, (2) all analyses, (3) conclusions regarding effectiveness, and (4) any proposed modifications to the RiskMAP. FDA, in turn, generally would perform its own assessment of RiskMAP effectiveness according to the principles of this and the other risk management guidances. At a minimum, FDA and sponsors would discuss their respective RiskMAP evaluations in a meeting or teleconference. In cases where risks are frequent and/or severe, or where results are ambiguous or uncertain, or where there is disagreement between the sponsor and FDA in the interpretation of the RiskMAP or tool effectiveness, public and expert input would be sought through the FDA Advisory Committee process. This will also allow airing and discussion of important information about effective and ineffective RiskMAPs and tools. D. Making Information From RiskMAP Evaluations Available to the Public As discussed in section IV.C, FDA plans to maintain a RiskMAP Web site that will describe all publicly available information about implemented RiskMAPs (and their tools). On the same Web site, FDA intends to make available, in summary format, information that has been publicly discussed or is otherwise publicly available (from sponsors or other sources) about the effectiveness of particular RiskMAP tools in achieving risk minimization objectives. The summaries may derive from materials presented and discussed at FDA Advisory Committee meetings where the effectiveness of a particular RiskMAP has been discussed and potential modifications have been entertained. Table of Contents COMMUNICATING WITH FDA REGARDING RISKMAP DEVELOPMENT AND DESIGN ISSUES As discussed in section III.D, because risk and benefit information emerge continually throughout a product's lifecycle, a sponsor could decide, or FDA could recommend, that a RiskMAP is appropriate at several different times. These times include: before approval, when a risk is identified from clinical studies, nonclinical studies, or in similar class of products, and risk minimization is appropriate as the product is introduced into the marketplace after marketing, if pharmacovigilance efforts identify a new serious risk and minimization of the risk will contribute to a favorable benefit-risk balance when marketing a generic product that references an innovator drug with a RiskMAP If a sponsor would like to initiate a dialogue with FDA to benefit from the Agency's experience in reviewing previously implemented plans, the Agency recommends that the sponsor contact the product's review division. The review division is the primary contact for a sponsor. The review division may choose to consult with other Offices in assisting the sponsor in developing a RiskMAP. These consulting offices could include CDER's Office of Drug Safety (ODS), CBER's Office of Biostatics and Epidemiology (OBE), or CDER's Office of Generic Drugs (OGD), as appropriate. In any particular case, it is helpful if the sponsor and FDA: share information and analyses regarding the product's risks and benefits discuss the choice of RiskMAP goals, objectives, and tools discuss the evaluation plan, including (1) times for evaluation, (2) performance measures and their targets, and (3) analyses Sponsors may wish to discuss RiskMAP issues with FDA at pre-defined meeting times (e.g., end-of-phase-2 meetings), if appropriate, or request meetings where RiskMAPs can be specifically considered. To maximize the value of their discussions with FDA, we recommend that sponsors who seek the Agency's guidance apprise reviewers of the rationale for and data underlying RiskMAPs under consideration. FDA requests that sponsors also share relevant background information and questions for discussion before their meetings with FDA. Both CDER and CBER will develop internal Manuals of Policies and Procedures (MaPPs) (or standard operating procedures (SOPs)) regarding the review of RiskMAPS. The procedures will define milestone points at which RiskMAP discussion is logical and will promote consistency in RiskMAP review and design. All RiskMAPs involving reminder tools or performance-linked access systems will be considered at the Center level as a secondary method of ensuring consistency across product classes and across divisions. If the sponsor decides to submit a RiskMAP before marketing approval of the product, most times the RiskMAP will be submitted to the new drug application (NDA) or biologics license application (BLA) for the product in question. However, if a risk is identified early (e.g., the product is a teratogen), and the sponsor wishes to institute formal risk management activities during Phases 1 to 3 studies, the sponsor can submit the RiskMAP to the investigational new drug application (IND). If a RiskMAP is being considered in a product's postmarket phase, FDA recommends that it be submitted as a supplement to the relevant NDA or BLA. Additional user fees will only be applicable to a supplement if FDA determines that new clinical data are required for its approval. This would be unlikely for a RiskMAP supplement. FDA encourages early and open discussion of safety concerns and whether such concerns may merit a RiskMAP. Early discussion of RiskMAPs could provide the opportunity to pretest risk minimization tools. Table of Contents RECOMMENDED ELEMENTS OF A RISKMAP SUBMISSION TO FDA A. Contents of a RiskMAP Submission to FDA FDA suggests that a RiskMAP submission to FDA include the following sections, as well as a table of contents: Background Goals and Objectives Strategy and Tools Evaluation Plan 1. Background FDA suggests that the Background section explain why a RiskMAP is being considered and created. We recommend that it describe the risks to be minimized and the benefits that would be preserved by implementation of a RiskMAP. Further, we suggest that this section describe, to the extent possible, the type, severity, frequency, and duration of the product's risks, with particular attention to the risk or risks addressed by the RiskMAP. The following are sample questions regarding risk characterization that we recommend be addressed in the Background section: What is the rationale for the RiskMAP? What is the risk the RiskMAP addresses? Is there more than one risk to be minimized? If there is, how do they relate to each other with regard to the following bulleted items? What is the magnitude and severity of the risk? Who is at highest risk? Are particular populations at risk (e.g., children, pregnant women, the elderly)? Is the risk predictable? Is the risk preventable? Is the risk reversible? Is the risk time-limited, continuous, or cumulative? These questions are similar in intent to what the ICH calls a Safety Specification in its E2E guidance. 16 FDA recommends that this section include a discussion that considers the product's risks in the context of its benefits. The following are sample questions that address benefit characterization. What is the overall nature or extent of benefit and what are the expected benefits over time (i.e., long-term benefits)? How do the populations most likely to benefit from this product compare to those that may be at highest risk? How would implementation of a RiskMAP affect individual and population benefits? Will it increase the likelihood that benefits will exceed risks in patients using the product? Will the RiskMAP affect access to the product by patients who benefit from it? Could certain individuals and/or populations likely to benefit from the product potentially have less access to the product because of the tools in the RiskMAP? We suggest that the Background section include a discussion, if pertinent, about the successes and failures of other regulatory authorities, systems of healthcare, or sponsor actions in minimizing the risks of concern for this product. Information provided by the sponsor regarding relevant past experiences, domestically or in other countries, will assist in harmonizing plans as well as avoiding the cost of implementing RiskMAP tools already deemed unsuccessful. We encourage sponsors to provide applicable information or evaluations from past experiences with products or programs that are similar to the proposed RiskMAP. 2. Goals and Objectives FDA suggests that the Goals and Objectives section describe the goals and objectives of the RiskMAP. 17 In addition, we recommend that this section describe how the stated objectives will individually and collectively contribute to achieving the goal or goals. 3. Strategy and Tools FDA suggests that the Strategy and Tools section define the overall strategy and tools to be used to minimize the risk or risks targeted by the RiskMAP. We recommend that the sponsor provide a rationale for choosing the overall strategy. We suggest that the sponsor describe how each tool fits into the overall RiskMAP and its relationship to the other tools. FDA suggests that the sponsor also provide the rationale for choosing each tool (see section IV.D for a discussion of considerations in choosing tools). In particular, we recommend that the sponsor describe the available evidence regarding the tool's effectiveness and, where applicable, provide results from pretesting. In addition, we suggest that the sponsor state whether it sought input from patient or healthcare interests, and if it did, we suggest that the sponsor describe the feedback that was received regarding the feasibility of its RiskMAP. FDA plans to maintain a Web site that will describe publicly available summary information about effectiveness of RiskMAP tools (see section V.D). We recommend this section also include an implementation scheme that describes how and when each RiskMAP tool would be implemented and coordinated. FDA suggests that sponsors specify overall timelines and milestones. For example, this section could address whether targeted education and outreach tools would be implemented before, or concurrently with, other tools. 4. Evaluation Plan FDA suggests that the Evaluation Plan section describe the evaluation measurements or measures that will be used to periodically assess the effectiveness of the RiskMAP's goals, objectives, and tools. For a detailed discussion of RiskMAP evaluation, see section V. We recommend that this section include: The proposed evaluation methods for assessing RiskMAP effectiveness (e.g., claims-based data systems, surveys, registries) and the rationales for the sponsor's chosen measures. Targeted values for each measure and the time frame for achieving them. FDA recommends the sponsor include interpretations of expected results under best- and worst-case scenarios. In addition, we suggest the sponsor specify what values of measures at specific time points will trigger consideration of RiskMAP modification. The nature and timing of data collection, analyses, and audits or monitoring that will be used to assess the performance of each individual tool in achieving the RiskMAP's objectives and goals. Again, we suggest specifying target values for measures. A schedule for submitting progress reports to FDA regarding the evaluation results for the RiskMAP's individual tools, objectives, and goals (see section VII.B for a discussion of progress reports). We recommend that the timing and frequency of progress reports be based primarily on the nature of the risk, tools used, and outcomes under consideration. FDA recommends that progress reports be included in periodic safety update reports or traditional periodic reports. Where applicable and possible, we recommend that the Evaluation Plan section discuss potential unintended and untoward consequences of the RiskMAP. Such a discussion would be particularly valuable if there are therapeutic alternatives with similar benefits and risks. We suggest that sponsors discuss how unintended consequences would be assessed after RiskMAP implementation. The goal of the assessment would be to ensure that overall population risks are minimized and specific product benefits, including access, are preserved. B. Contents of a RiskMAP Progress Report FDA recommends that a RiskMAP progress report contain the following sections, accompanied by a table of contents: Summary of the RiskMAP Methodology Data Results Discussion and Conclusions 1. Summary We suggest that the Summary section briefly provide background on and an overview of the RiskMAP, and describe the overall RiskMAP goals and objectives, as well as its strategy and tools. We recommend that this section also summarize (1) the evaluation methods used and (2) the relevant measures and time frames for achieving targeted values. 2. Methodology We recommend that the Methodology section provide a brief overview of the evaluation methods used (e.g., ascertainment of outcomes, comprehension testing, patient surveys, process audits). FDA suggests that it describe the evaluation plan, sources of potential measurement error or bias for the outcome of interest, and any analytical methods used to account for them. Since RiskMAP evaluations will often rely upon observational data, we recommend that the analytical plan address issues such as measurement errors, sensitivity, and specificity of the measures, as well as power for detecting differences where appropriate. 3. Data To the extent possible, we recommend that the Data section of a RiskMAP progress report contain data that would allow FDA to analyze the information and make conclusions independently. 4. Results To the extent possible, we recommend that the Results section of a RiskMAP progress report contain the primary data from each evaluation method and analyses of the evaluation data, statistical estimation if appropriate, and the sponsor's comparison of tool, objective, and/or goal achievement relative to targeted performance measures. 5. Discussion and Conclusions FDA recommends that this section describe whether the RiskMAP has met or is making progress in meeting the stated measures for each tool, objective, and goal. We suggest that this discussion take all available data, evaluations, and analyses into consideration. Progress towards achieving RiskMAP goals or performance measures should be reported. Where appropriate, sponsors are encouraged to propose modifications to the RiskMAP and discuss them with FDA. Table of Contents 1 This guidance has been prepared by the PDUFA III Risk Management Working Group, which includes members from the Center for Drug Evaluation and Research (CDER) and the Center for Biologics Evaluation and Research (CBER) at the Food and Drug Administration (FDA). 2 For ease of reference, this guidance uses the term product or drug to refer to all drug products (excluding blood and blood components) regulated by CDER or CBER. Similarly, for ease of reference, this guidance uses the term approval to refer to both drug approval and biologic licensure. Paperwork Reduction Act Public Burden Statement: This guidance contains information collection provisions that are subject to review by the Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995 (PRA) (44 U.S.C. 3501-3520). The collection(s) of information in this guidance were approved under OMB Control No. 0910-0001 (until March 31, 2005) and 0910-0338 (until August 31, 2005). 3 See 45 CFR part 46 and 21 CFR parts 50 and 56. See also the Health Insurance Portability and Accountability Act of 1996 (HIPAA) (Public Law 104-191) and the Standards for Privacy of Individually Identifiable Health Information (the Privacy Rule) (45 CFR part 160 and subparts A and E of part 164). The Privacy Rule specifically permits covered entities to report adverse events and other information related to the quality, effectiveness, and safety of FDA-regulated products both to manufacturers and directly to FDA (45 CFR 164.512(b)(1)(i) and (iii) and 45 CFR 164.512(a)(1)). For additional guidance on patient privacy protection, see http://www.hhs.gov/ocr/hipaa/ 4 For example, see the Proposed Rule on Requirements on Content and Format of Labeling for Human Prescription Drugs and Biologics; Requirements for Prescription Drug Product Labels that published in the Federal Register on December 22, 2000 (65 FR 81081). 5 Although all products with RiskMAPs would also have FDA-approved professional labeling, the term tool as used in this document means a risk minimization action in addition to routine risk minimization measures. Some tools may be incorporated into a product's FDA-approved labeling, such as Medication Guides or patient package inserts. As used in this document, the FDA-approved professional labeling refers to that portion of approved labeling that is directed to the healthcare practitioner audience. See section IV for a more detailed discussion of other non-routine risk minimization tools that focus on targeted education and outreach. 6 This ICH guidance is available on the Internet at http://www.fda.gov/cder/guidance/index.htm under the topic ICH Efficacy. The draft E2E guidance was made available on March 30, 2004 (69 FR 16579). ICH agreed on the final version of the E2E guidance in November 2004. 7 This guidance is directed primarily toward sponsors of innovator products. However, a generic product may have the same benefit-risk balance as an innovator product and so may be considered for a similar RiskMAP. 8 See section VII for a detailed discussion of RiskMAP submissions. 9 Smalley W, D Shatin, D Wysowski, J Gurwitz, S Andrade et al., 2000, Contraindicated Use of Cisapride: Impact of Food and Drug Administration Regulatory Action. JAMA 284(23):3036-3039; Weatherby LB, BL Nordstrom, D Fife, and AM Walker, 2002, The Impact Of Wording in "Dear Doctor" Letters and In Black Box Labels. Clin Pharmacol Ther 72:735-742. 10 As noted in section III.B, sponsors should not develop a RiskMAP for a product for which routine risk minimization measures are sufficient. Similarly, formal evaluation plans and performance measures should not be developed for these products. Instead, evaluation by routine postmarketing surveillance should be sufficient, although some products may also have a Pharmacovigilance Plan as described in the Pharmacovigilance Guidance. If a RiskMAP is later developed for this type of product based on new risk information, then a sponsor should consider submitting a formal evaluation plan. 11 For further discussion of administrative claims systems, please consult the pharmacovigilance guidance. 12 For a more detailed discussion of survey development and implementation, please consult the pharmacovigilance guidance. 13 Stamatis DH, Failure Mode and Effects Analysis: FMEA From Theory to Execution, Milwaukee: American Society for Quality, Quality Press, 2003. 14 Cohen Michael R ed, Medication Errors: Causes, Prevention, and Risk Management, Washington, DC: American Pharmaceutical Association, 1999. 15 For a detailed discussion of large simple safety studies, please consult the premarketing guidance. 16 Available on the Internet at http://www.fda.gov/cder/guidance/index.htm under the topic ICH Efficacy. 17 See section IV for a discussion of goals and objectives.

Updated April 19, 2005

CBER Home Page | CBER A-Z Index | CBER Search | Contact CBER FDA Home Page | Search FDA Site | FDA A-Z Index | Contact FDA | Privacy | Accessibility | HHS Home Page FDA / Center for Biologics Evaluation and Research