Figure I-1. The Three Components of Risk Analysis
In this section of the document, risk analysis is defined and the characteristics that describe CFSAN's approach to conducting high quality risk assessments are outlined. The organization of the risk analysis teams and the risk assessment activities as well as the interaction of risk assessors with risk management and risk communication teams are described. A brief description of the processes associated with risk assessment including planning, performing, reviewing, and issuing/publication a risk assessment is provided.
Risk analysis is a tool to enhance the scientific basis of regulatory decisions. It includes risk assessment, risk management and risk communication activities. As shown in Figure I-1, risk analysis is often illustrated using three circles, equal in size and overlapping to indicate both independent and dependent activities of the three risk analysis components. Each component has unique responsibilities:
Risk assessment provides information on the extent and characteristics of the risk attributed to a hazard.
Risk management includes the activities undertaken to control the hazard.
Risk communication involves an exchange of information and opinion concerning risk and risk-related factors among the risk assessors, risk managers, and other interested parties.
Transparent. Transparency includes stating any biases that impact the risk assessment, clearly and concisely documenting the assessment, and using a participatory process. Transparency includes clearly stating all assumptions used in the assessment, providing the scientific rationale, and documenting the data used to estimate the impact of the various factors influencing the risk.
Reveal biases. All assumptions used in the assessment, and the scientific rationale and data used to estimate the impact of the various factors influencing the risk must be clearly stated. Transparency ensures that biases will be eliminated or minimized, and that any introduced biases are clearly identified.Team-oriented. Quantitative risk assessments are too complex to be conducted by a single person. They require the input and critical evaluation of experts in a number of scientific disciplines including microbiology, chemistry, epidemiology, medicine, mathematics, statistics, toxicology, and food science, i.e., a multidisciplinary team.Clarity. Transparency includes ensuring the document is clear and understandable. In attempting to provide all of the information used in the risk assessment model, the resulting document tends to be lengthy and very technical. Documents that are so technical that only a few experts can understand them lack transparency. Therefore, CFSAN prepares an interpretive summary document as a companion to each technical risk assessment. The summary document provides a non-technical explanation of the assessment process, results, and conclusions in a manner that a non-scientist can understand.
Participatory process. The process for initiating, performing and finalizing a risk assessment should also be transparent. CFSAN invites public comment on planned assessments and encourages stakeholders to submit scientific data and information to support the assessment. The advice and opinions of advisory committees are solicited as well as peer review from experts within and outside of the Center. As part of its commitment to transparency and use of the best information available, CFSAN makes the draft risk assessment documents available to the public for technical review and comment. Assessments are posted on the CFSAN web pages and printed copies are also available through CFSAN's Outreach and Information Center.
Iterative. Risk assessment for complex topics should be an iterative process, and this iteration will require communication and collaboration among the various risk analysis teams and other interested individuals.
As additional scientific knowledge about the hazard, additional exposure or dose-response data, or improved modeling techniques become available, the assessment and its conclusions may have to be reevaluated or updated. Risk assessments can always be improved and the uncertainty in the model may be reduced when new data or modeling techniques become available. On the other hand, this makes it difficult to know when to stop and give the results to the risk manager. A general guideline is that the risk assessment should be kept as simple as possible, while providing the risk managers with the information they need to make decisions.
Similarly, the risk assessor often must choose between assumptions or models to use to describe exposure or dose-response. The risk management team should be consulted regarding influential choices. They may be able to make all necessary risk management decisions despite this model uncertainty, or they may choose to engage other resources to clarify the choice before proceeding further with the risk assessment (e.g., directed research, expert panels).
Risk analysis should also be conducted and thought of as a dynamic, iterative process. Key elements of successful risk analyses are clear understanding of the questions to be addressed by the risk assessors and wide acceptance of the assumptions used in risk modeling. Communication of the questions to be addressed often involves an initial dialogue and framing of the issues followed by clarification and elaboration once screening-level risk analysis has been carried out.
The iterative nature of risk analysis is illustrated in Figure I-2 by the use of double-headed arrows between risk management, risk assessment, and risk communication teams. The iterative nature of risk analysis might include a situation where:
the answer provided by a risk assessment results in a new risk management question; or
Note: Because a model may influence the experimental design and the type of data collected, research also has an important role in risk analysis, as noted in the section below entitled, "The Role of Research."The double-headed arrows shown in Figure I-2 imply a dialogue among the risk analysis teams, in contrast to a monologue (i.e., from risk assessors to risk managers). In a monologue-driven process, the risk management decision is conceived to be one that progresses in a single direction from analysis to policy. In a dialogue, policy is determined based on information exchanged between the risk assessors and the risk managers.
By focusing on the risk assessor--risk manager--risk communicator dialog, the risk assessment may be designed to evaluate the effectiveness of a putative action. That is, the risk management question is not just about the magnitude of the anticipated harm -- it is also about how we might mitigate the risk.
Structured. The generally accepted paradigm for microbial, as well as chemical food contaminates, includes separating the assessment activities into four components: hazard identification, exposure assessment, dose-response assessment, and risk characterization. See Appendix A for a description of these components.
Descriptive. An important part of risk assessment is determining the degree of uncertainty in relation to the results and distinguishing this from the variation that is inherent in any biological system. The accuracy of a risk assessment is dependent on the quality of the available data. When definitive data are lacking, the uncertainty about the available information is represented in the risk assessment using a range of possible data values. One way to decrease uncertainty is to conduct research to provide improved data. However, this does not mean that if one collects enough data that a single, "right" value will be reached. One of the challenges in using a risk analysis approach is that an "accurate" risk assessment captures the variability inherent in the food safety system instead of generating a single value.
Flexible. The risk assessment models should be flexible, such that they can be easily revised when new data or information become available.
Based on sound science. The risk assessment should be based on sound science that is decision driven and supported by systematic analysis that maintain integrity and protects the risk assessment from political and other pressure. The standards for sound science differ across different types of research. The risk assessment team should specify what it considers sound or competent science for each type of research that is used in the risk assessment.
The Role of Research A risk assessment should be based on sound science. Where data are lacking, assumptions must be made. Risk managers must provide a major contribution to those assumptions. There is a high likelihood that a risk assessment will raise uncertainties that might be addressed through further research. At times a risk management decision can be postponed until further research can be conducted, but it is more likely that the issues that gave rise to the present assessment are likely to persist or rise again. However, a risk assessment can be useful in planning the research that will have the most impact on future regulatory decisions. This relationship is shown in Figure I-3 with the placement of research (science) between risk assessment and risk management.
Risk assessments bring together research from many fields, including:
|
|
Supporting research could be conducted before, during, and after any quantitative risk assessment. Research carried out before the risk assessment could be the reason for the request for a risk assessment. For example, an outbreak of a particular food-borne illness may lead to epidemiological and medical research that points to the need for a risk assessment. This same early research will also play a large role in the hazard identification stage of the risk assessment. Research on environmental, technological, and behavioral sources of risk could all be part of the hazard identification.
During the conduct of the risk assessment, risk assessors will most likely discover missing information and request research. For example, the risk assessors may be unable to estimate exposure to a foodborne hazard without additional research on food consumption or food handling practices. They may ask for a survey on consumer practices to get the desired information. Or they may discover that the lethality of the time and temperature treatments of one path from processor to consumer is not well known. The risk assessors would then ask for laboratory research to simulate the properties of the path in order to estimate the lethality of the actual path. Yet another important type of research is on potential risk mitigations. The risk assessors might ask for research on the technical feasibility of some proposals for mitigation. They might ask for research comparing the lethality of chemicals, heat treatments, and other mitigations. Another line of research might be to ask economists to consider the incentives that industry or consumers have to voluntarily adopt the mitigations.
After the risk assessment is complete, several types of research might be in order. Economists might be asked to evaluate or identify the costs associated with risk mitigations identified by the risk assessment. The study of strategic behavior (mathematical game theory), for example, might prove useful in predicting how producers will react to regulations based on the risk assessment. Epidemiological studies, perhaps in conjunction with statistical and mathematical studies, could confirm or refute the range of predictions of the risk assessment. As new methods of hazard control appear, research will be needed to incorporate the potential mitigations into the risk assessment.
Research provides data and information needed to carry out assessments; risk assessments provide information risk managers need to make decisions. The link between research, risk assessment, and policy decisions means that decisions on the amount and type of research should be tied to decisions on the amount and type of information provided by the risk assessment. At various stages of the risk assessment risk managers will consult risk assessors to see what information is lacking. The risk assessment team should let risk management know how much each additional type of research can be expected to reduce uncertainties in the risk assessment, or how it will fill needs for information after the risk assessment. The risk manager will then determine if the research should be carried out. How much and what types of research and when to carry out that research, will depend on the importance of the policy that the risk assessment is designed to support.
The Risk Management Team (RM) is responsible for formulating the risk assessment questions to be addressed, providing key assumptions and oversight of the assessment, and developing a management action plan.
The Risk Assessment Team (RA) is responsible for conducting the assessment, and refining as necessary, the assumptions provided by the risk management team, explaining the uncertainty of the results and the impact of assumptions on the results.
In addition to the three risk analysis teams, CFSAN's risk analysis framework includes three unique positions: a science advisor, a risk analysis coordinator, and a risk assessment project manager.
Science Advisor. The role of the science advisor is to ensure that the science of the assessment is not compromised by the policy needs of the risk management team. The science advisor must act to preserve credibility and transparency in all decisions and is responsible for resolving any science vs. policy issues.Risk Analysis Coordinator. This individual assists the risk management, assessment, and communication teams by coordinating and managing the activities and helping to facilitate communication within and among these groups.
Risk Assessment Project Manager. This individual is responsible for administrative and technical management of the process. In contrast, the risk assessment team leader is mainly responsible for the technical conduct of the assessment. For risk assessments that are more limited in scope and complexity, a single individual might function as both the risk assessment project manager and the risk assessment team leader.
The relationships and interaction of the risk analysis teams, the science advisor, the risk analysis coordinator, and the risk assessment project manager are indicted in Figure I-4.
The various roles and responsibilities of participants in risk analysis are summarized in Table I-1.
| Participant | Roles and Responsibilities | |
|---|---|---|
| (Senior) Management team | [Includes Center director, deputy director, senior science advisor] Allocates resources (staff, budget); Selects risk assessments to be conducted | Responsible to ensure that risk assessment (RA), risk management (RM), and risk communication (RC) activities meet the Center's objectives and mission. |
| Leadership team | [Includes management team plus office/staff directors] Recommends risk assessments to be conducted | Responsible to ensure that recommended risk assessment projects meet the Center's needs. Allocates resources for risk analysis activities. |
| Science Advisor | [CFSAN's Senior Science Advisor] Arbitrates science/policy issues | Responsible to ensure that politics do not bias the assumptions, data, conclusions, or interpretations of the risk assessment. |
| Risk Management Team (RM) | Team leader coordinates activities with the Risk Analysis Coordinator (CO). | Poses the specific risk assessment question(s) to be answered; Identifies scope of the assessment and provides to RA in the form of a charge; Provides RA with key assumptions to be used; Conducts gap analysis of current RM strategies; Develops a RM action plan; Identifies RM options, implementation strategies, and measures effectiveness; and Identifies and implements plans for research needed to address data gaps. |
| Risk Communication Team (RC) | Team leader coordinates activities with the CO; and Interacts with stakeholders |
Identifies the strategies for the exchange of information with various stakeholders, including their risk
communication needs and concerns; Develops and distributes RC messages for the assessment and management action plan; and Develops outreach action plan. Evaluate success of risk communications throughout and at the end of the risk analysis process. |
| Risk Analysis Coordinator (CO) | Coordinates activities with RM, RC, and RA; and supervises the PM and clerical assistant. | Serves as contact person for the assessment within the Center; Works to remove barriers identified by the RA/RM/RC; Implements budget and tracks progress of the assessment; and Assists RA, RM, and RC teams with planning the roll-out. |
| Risk Assessment Project Manager (PM) | Coordinates assessment activities with the CO and works closely with RA team leader. | Identifies barriers to completing the risk assessment and informs the CO; Assists with identifying risk assessment team members; Manages the conduct of the RA including maintaining project files, word processing, document assembly, references, quality control, desktop publishing, timelines, scheduling of team meetings; and Assists CO with implementing roll-out plan. |
| Risk Assessment Team (RA) | Team lead reports any barriers and project concerns to the PM. | Members serve as technical experts for the conduct of the risk assessment in various identified subject matters; Develops assumptions used in modeling; Gathers and assembles data used in modeling; Performs the risk assessment; Prepares risk assessment report/document(s); and Identifies research data gaps. |
One difficulty in risk analysis is delineating the functional boundaries between assessors, managers, and communicators. A key aspect is that boundaries must be maintained but functions should be interdependent so as to provide appropriate checks and balances to the process. By clearly defining responsibilities of the risk analysis participants, CFSAN believes that appropriate boundaries can be maintained. The risk analysis coordinator should assist the risk analysis participants in understanding their respective roles and responsibilities.
While the goal of a risk assessment is to assist risk managers in making a decision, it is only one of many sources of information that CFSAN's risk managers use to make policy decisions. As such, it is the risk managers who must set the objectives of the risk assessment and provide the overall scope, including key assumptions, which should be used in the absence of data. It is, however, the responsibility of the risk assessors to do the following:
The following are the general areas or topics that the risk analysis teams should discuss at various points in the process.
During the Planning of the Risk Assessment
- The risk management question and its significance to the specific risk assessment question(s) and how the assessment answers relate to the risk management options
- How the teams should interact and at which specific points in the assessment process
- Interaction and communications with other interested parties, including other agencies
- The type of risk assessment (quantitative vs. qualitative; risk ranking, product pathway, risk-risk) and whether this is the best approach to answering the questions
- Resources needed and timeline for conducting the assessment
During the Conduct of the Risk Assessment
- A conceptual diagram of the major steps of the risk assessment model
- Method of collecting data, criteria for choosing which data to include in the assessment, how the data are used in the risk assessment model
- Specific data inputs and outputs for the conceptual model
- Use of key assumptions provided by risk managers
- Identification of modeling assumptions
- Criteria used to select the various distributions/models used
- Sensitivity of assumptions and impact on conclusions
- Methods of presenting of the results
During the Risk Analysis Team Review of the Risk Assessment
- Answer(s) to the risk assessment question(s)
- Uncertainty/sensitivity analysis
- Data gaps or research needed to refine the risk assessment
- Transparency of the document
Examples of topics (posed as questions) that the risk analysis team members should discuss are provided in Appendix B. The questions in Appendix B are not meant to be a prescriptive list but rather a list of possible topics designed to encourage a meaningful exchange of information.
To be effective, risk communication must include an exchange of information by all interested parties ensuring that the process and results of any risk analysis are considered transparent and are trusted by outside audiences. For this trust to occur, the information exchange must be encouraged and actively promoted throughout the risk analysis process, particularly during the planning, conduct, review and issuing phases of the risk assessment. A summary of specific risk communication activities conducted within a risk analysis framework is provided in Appendix C. The need for these activities is mentioned in other sections of this document but the focus of Appendix C is the specific activities that are the responsibility of the risk communication team.
CFSAN's risk analysis framework includes identification/selection, planning, performance, review, and issuing/publication activities. Responsible parties for each risk analysis activity are shown in Table I-2. The identification and selection process is described in Part II of this document. The activities directly related to the conduct of the risk assessment (planning, performing, reviewing, and issuing) once it is commissioned are described in detail in Part III of this document. The specific goals and end products of the activities that are the responsibility of the risk assessors are described below and summarized in Table I-3.
| Risk Analysis Activities | Responsible Party* | |||
|---|---|---|---|---|
| RA | RM | RC | ||
| Select the risk assessment | X | |||
| Plan and allocate resources | X | X | X | |
| Performance | Conduct the risk assessment | X | ||
| Develop management action plan | X | |||
| Develop communication messages | X | |||
| Review | Risk assessment documents | X | X | X |
| Management action plan | X | |||
| Communication messages | X | X | ||
| Issue | Risk assessment documents | X | ||
| Risk management action plan | X | |||
| Risk communication messages | X | |||
|
RA= risk assessment team; RM = risk management team; and RC = risk communication team. |
||||
During the planning phase, the specific resources needed are allocated, a timeline established, and a charge (questions for the risk assessment to answer) are prepared. Prior to commissioning the assessment, the members for each team should be identified and the teams assembled.
After the risk management team defines the risk assessment questions, the various risk analysis teams are formed and the risk assessment is performed. The risk assessment team collects and assembles data and information, conducts the risk assessment and drafts the technical and interpretive summary documents. Based on the results of the risk assessment the risk management team develops an action plan and the risk communication team develops communication strategies including public health advice messages about the assessment and management action plan.
The risk assessment, management action plan, and communication messages must be reviewed. The review of the risk assessment may result in another iteration of the assessment, a redrafting of the document or both. Reviewing activities are completed when the documents are cleared by the Center and are ready to be released to the public. This step also includes peer review of the risk assessment documents by FDA staff, other agencies, and non-government individuals, as needed. A suitable timeframe should be allotted for the review process. Following the peer review process, the draft risk assessment documents will be issued for public review and comment.
Issuing (or publishing) the risk assessment documents is another activity in the risk analysis process. A roll-out plan is prepared and implemented. A Federal Register notice is drafted and published, a press contact list is compiled, and a public meeting is planned and held to present, discuss, and clarify the findings of the risk assessment. Following the public comment period, the risk assessment may be revised and a final document issued. The final risk assessment document must include a summary of the public comments and how they were addressed in the revised document.
Follow up is also important and included in this framework as part of issuing or publishing the risk assessment. It includes an internal evaluation of the effectiveness of the process and also a determination of future actions required and changes needed.
| Step Description | Goal(s) | End Product(s) |
|---|---|---|
| Step 1: Planning | Define scope of risk assessment, identify resources, develop timeline | Assemble team, allocate resources, 'charge' for the risk assessment |
| Step 2: Performing | Answer risk management's questions | Risk assessment results and draft document(s) |
| Step 3: Reviewing | Peer review, approval, clearance | Document ready to be released to the public |
| Step 4: Issuing | Develop and implement roll-out plan | Publish risk assessment document(s) Conduct follow up |
Shaded bidirectional arrows indicate dialog among the teams: Risk Management Team, Risk Assessment Team, and Risk Communication Team.