Search Options
Index | Site Map | FAQ | Facility Info | Reading Rm | New | Help | Glossary | Contact Us

POLICY ISSUE
NEGATIVE CONSENT

SECY-00-0162

July 28, 2000

• PURPOSE:
• BACKGROUND:
• DISCUSSION:
• RESOURCES:
• COORDINATION:
• RECOMMENDATION:

PURPOSE:

To describe the staff's approach for addressing the issue of probabilistic risk assessment (PRA) quality in current risk-informed activities in response to the Commission's April 18, 2000, Staff Requirements Memorandum.

BACKGROUND:

In their March 1999 report ("Nuclear Regulation: Strategy Needed to Regulate Safety Using Information on Risk," GAO/RCED-99-95), the General Accounting Office (GAO) identified a number of issues that it believed required resolution for NRC to successfully implement a risk-informed regulatory approach. Among these, GAO indicated that more was needed to "develop standards on the scope and detail of risk assessments needed for utilities to determine that changes to their plants' design will not negatively affect safety."

PRA standards have been under development by the American Society of Mechanical Engineers (ASME) and American Nuclear Society (ANS). In addition, the National Fire Protection Association (NFPA) has under development a fire protection standard with an appendix on fire PRA. On June 14, 2000, ASME issued a second draft of a standard for a full-power, internal events (excluding fire) Level 1 and a limited Level 2 PRA (for a 60-day public comment period). A final standard is expected in January 2001. ANS is scheduled to issue a draft standard for external events risk (for a 60-day public comment period) in July 2000, with a final standard in December 2000. ANS also plans to issue a draft low power shutdown PRA standard in September 2000, with a final version in June 2001. NFPA is scheduled to issue a final standard on fire protection (including the appendix on fire PRA) in March 2001 (the draft was released for comment in January 2000).

Reactor owners' groups have been developing and applying a PRA peer review program for several years. In a letter dated April 24, 2000, the Nuclear Energy Institute (NEI) submitted NEI-00-02 (Probabilistic Risk Assessment Peer Review Process Guidance, Rev. A3) to the NRC for review in the context of the staff's work to risk-inform the scope of special treatment requirements contained in 10 CFR Part 50 (discussed in SECY-99-256).

Concerns regarding PRA quality and the standards development effort were discussed in the March 31, 2000, Commission briefing on the Risk-Informed Regulation Implementation Plan. The Commission, in their April 18, 2000, Staff Requirements Memorandum (SRM) on that briefing, indicated that the staff "should provide its recommendations to the Commission for addressing the issue of PRA quality until the ASME and ANS standards have been completed, including the potential role of an industry PRA certification process." The staff's response to this SRM item is provided below.

DISCUSSION:

The NRC staff has been using information from PRAs in many of its regulatory activities. These activities are outlined in Attachment 1. During this time, the staff has used a number of approaches for ensuring that a PRA used in a particular activity was of sufficient scope and technical quality for that activity. Regulatory Guide 1.174 and SRP Chapter 19 provide guidance on PRA quality and on how a PRA should be used in support of licensing actions. This guidance has been used successfully in numerous (approximately one hundred) licensing decisions. The staff's approach has involved review of licensee submittals and the extent of the staff review has varied depending on such matters as the type of decision being made and the extent to which the decision would be based on the PRA information.

In any regulatory decision, the goal is to make a sound safety decision based on technically defensible information. Therefore, for a regulatory decision relying upon risk insights as one source of information, there needs to be confidence in the PRA results from which the insights are derived. Consequently, the PRA needs to have the proper scope and technical attributes (see Attachment 1) to give an appropriate level of confidence in the results used in the regulatory decision-making. It is recognized that these aspects can vary depending on the specific decision under consideration.

If a certain aspect of a PRA is weak, and it can be shown that this weakness is not impacting the results to be used for the decision under consideration, then the confidence in the risk insights is not compromised. However, if the weakness does impact the confidence of the PRA results, a sound safety decision can still be made if there are appropriate compensatory measures (this process and measures are discussed in Attachment 2). Accordingly, the main goal of the PRA review is to identify weaknesses in the PRA that could be relevant to a decision so that they can be properly considered.

The risk-informed regulatory decision-making for various activities (as discussed in Attachment 1) use the same PRA results in deriving the risk insights. These results include:

• An evaluation of the core damage frequency (CDF), large early release frequency (LERF), and potential for late containment failure of the as-operated and as-built plant,
  
  
• An evaluation of the change in CDF and LERF,
  
  
• An identification and understanding of the major core damage sequences and their contributors,
  
  
• An identification and understanding of the core damage states and phenomena contributing to the large early release of radionuclides and late containment failure, and
  
  
• An understanding of the sources of uncertainty and their impact on the results.

Attachment 1 identifies the scope of the PRA that is needed to provide these results, and describes the minimal functional attributes necessary to ensure the PRA is capable of providing the above information. Although these are the minimal requirements at a functional level defining a PRA, they do not by themselves ensure confidence in the PRA results. This confidence can be gained by defining supporting technical requirements.

For example, in the technical element of systems analysis, one functional attribute is that "the model is developed in sufficient detail to capture the impact of dependencies." To ensure that the intent of this attribute is met, it is necessary to understand the dependencies that could impact the availability and operability of the system and components under consideration. However, what the dependencies are and how they support a specific system or component are not always evident. Dependencies such as the need for DC power for the Reactor Core Isolation Cooling (RCIC) system (in a BWR) are evident. However, for continued operation of RCIC, there is also a need for suppression pool cooling. The steam from the RCIC turbine exhausts to the suppression pool, and loss of cooling to the pool can cause the RCIC turbine to trip on high exhaust pressure. This type of dependency is not as evident. Consequently, to ensure that the PRA has properly accounted for the impact of dependencies, supporting technical requirements interpreting this functional requirement (and the others) are needed. In this example, the supporting requirements may specify the types of dependencies (e.g., motive and control power, design and operational conditions) that need to be considered in looking at the availability and operability of a particular type of component (e.g., turbine-driven pump).

Consensus PRA standards can be used to define these technical requirements, and an industry peer review program can provide an assessment of the weaknesses of a PRA. Therefore, the staff is reviewing the NEI-00-02 and the ASME and ANS PRA standards, as well as the PRA portion of the NFPA fire protection standard, in this light. To support this review, the staff is developing acceptance criteria for the technical requirements and peer review process. These reviews are summarized below.

• Review of the ASME, ANS and NFPA standards  The staff intends to use Attachment 1 to provide the general basis for its review of the ASME and ANS standards and the PRA portion of the NFPA fire protection standard. ASME and ANS intend to issue final versions of their standards near the end of this year (with ANS issuing just the external hazards risk standard at that time), and NFPA plans to issue their final standard in March 2001. The staff will review the ASME and ANS final versions as well as the final version of the appendix on fire PRA included in the NFPA fire protection standard. If appropriate, the staff will endorse them in an update of Regulatory Guide 1.174 or elsewhere to support other risk-informed activities. The staff endorsement may take exception to or include additional specific criteria to address any identified weaknesses in the standards to ensure that PRAs used in regulatory decisionmaking will have an adequate technical basis.
  
  
• Review of the NEI Peer Review Program  The staff review will encompass both the process and technical aspects of the peer review. The staff process review will involve comparing the NEI-00-02 review methodology against the criteria described in Attachment 1 to this paper. These criteria provide a set of "peer review attributes" on what constitutes an acceptable peer review process. The staff technical review will involve comparing the NEI-00-02 technical elements against the set of "functional technical attributes" in Attachment 1 and sub-tier criteria against the acceptance criteria being developed to supplement the functional attributes. The staff is reviewing NEI-00-02 in conjunction with its review of the draft ASME standard.
  
  The staff endorsement may contain specific criteria to address any identified weaknesses to ensure its adequacy to support risk-informing Part 50, Option 2 (e.g., categorization of structures, systems, and components).

Until the review of the standards and peer review program is completed and the staff endorsement final, the staff will continue to use the guidance of Regulatory Guide 1.174 and SRP Chapter 19 to ensure that PRA information used in regulatory decisions is of the appropriate scope and quality for the decision being made. To strengthen this guidance and thus improve the efficiency and consistency of the staff review process, the staff intends to include the information in Attachments 1 and 2 in the next update of the guide and SRP chapter. The staff is now developing an updated version of these documents with the intent of publishing them in September 2000 for public review and comment.

RESOURCES:

RES and NRR resources for development of the staff documents, review and possible endorsement of the standards and certification documents, and updating Regulatory Guide 1.174 and SRP Chapter 19 are included in the current RES and NRR budgets for FY2000 and FY2001.

COORDINATION:

The Office of General Counsel has reviewed this paper and has no legal objections. The Office of the Chief Financial Officer has reviewed this paper for resource implications and has no objections. Although this paper is written around reactor applications of PRA information, the general approach and concepts will be considered as the NMSS plans for risk-informed regulation are developed and implemented.

RECOMMENDATION:

The staff recommends that the Commission approve the approach described above for ensuring that safety decisions using PRA information continue to be adequate for the intended application; including the following:

• Continue with the current process,
  
  
• Review NEI-00-02 and industry standards (using Attachments 1 and 2), and endorsement (if appropriate) with exceptions as necessary, and
  
  
• Update RG 1.174 and SRP Chapter 19 with the information in Attachments 1 and 2.

Staff requests action within 10 days. Action will not be taken until the SRM is received. We consider this action to be within the delegated authority of the EDO.

ATTACHMENT 1

PRA SCOPE AND TECHNICAL ATTRIBUTES

• 1. INTRODUCTION
• 2. PRA SCOPE
• 3. PRA ELEMENTS AND TECHNICAL ATTRIBUTES
• Level 1 PRA --
• Level 2 PRA --
• Internal Floods --
• Internal Fire --
• External Hazards --
• Documentation
• 4. PEER REVIEW PROCESS
• 5. PRA TECHNICAL ACCEPTABILITY
• 6. EXPERT PANEL
• Risk Insights in Decision-Making
• Implementation of Decisions

1.  INTRODUCTION

Over the past 25 years a number of probabilistic risk assessments (PRAs) have been performed by both the U. S. Nuclear Regulatory Commission (NRC) and the nuclear industry. The scope, depth, and technical content of the PRAs have varied along with their purposes and uses. Results from PRAs have increasingly been used in the regulatory process, starting with generic safety issue prioritization and progressing to regulatory analysis in support of rulemaking and backfits and currently risk-informed regulation, which opens up the possibility of using PRA information in many ways not previously done.

The NRC issued a Policy Statement on the use of PRA in 1995, encouraging its use in all regulatory matters. Since that time, many uses have been implemented or undertaken, including the initiation of work to modify the reactor regulations and inspection program. As a result PRA is becoming a mainstream regulatory tool and, as such, is providing valuable input into the decision-making process regarding the design, operation and maintenance of plants. Consequently, confidence in the information derived from a PRA is an important issue. That is, the scope of the analysis must be sufficiently broad and the accuracy of the technical content must be of sufficient rigor to justify the specific results and insights from the PRA that are used to support the decision under consideration.

Each application may impose somewhat different requirements on the supporting PRA. Therefore, it is important to note what are the different risk-informed activities for which defining PRA technical acceptability is needed. Recent activities include the following:

• Risk-Inform 10 CFR Part 50: The NRC is evaluating the scope of the special treatment requirements and the technical requirements of 10CFR Part 50 and is considering revisions to them, as appropriate, based in part on risk insights obtained from PRAs.
  
  
• Reactor Oversight Process: The NRC is increasing the focus of inspection on those activities with the greatest potential impact on safety. Inspection results will routinely be evaluated to determine the risk importance of the findings. Likewise, enforcement sanctions for violations of regulatory requirements will be better linked to the safety significance of inspection findings.
  
  
• Operating Events Assessment: The NRC is continuing to evaluate the risk significance of operational events and trends in data in conjunction with risk assessments so that safety vulnerabilities can be identified, prioritized, communicated, and resolved on a timely basis.
  
  
• License Amendments: The NRC has developed Regulatory Guide 1.174 that provides guidance on an acceptable analysis approach to support changes to a plant's licensing basis using plant-specific risk information. Application specific regulatory guides have also been developed in the areas of inservice testing, inservice inspection, graded quality assurance and technical specifications. The staff is continuing its reviews of license amendments in these and other areas.
  
  
• Risk-informed technical specifications: The NRC is continuing to work with industry on several initiatives to further develop risk-informed improvements to the technical specifications. Examples of these initiatives include the replacement of fixed allowed outage times with a PRA-based configuration risk management program, and a definition of preferred end-states for technical specification actions.
  
  
• Maintenance rule: The NRC has required licensees to monitor the effectiveness of maintenance actions via the maintenance rule (50.65). A new section (a)(4) is being implemented (11-28-00) to help in controlling configuration-specific risks.

For each of the above activities, PRA results are used to determine the risk significance of structures, systems, and components (SSCs), the design and operational features critical to risk, and the events or scenarios important to risk. To make these determinations, the following are needed:

• an evaluation of the core damage frequency (CDF), large early release frequency (LERF) and potential for late containment failure of the as-operated and as-built plant
  
  
• an evaluation of the change in CDF and LERF
  
  
• an identification and understanding of the major core damage sequences and their contributors
  
  

• an identification and understanding of the core damage states and phenomena contributing to the large early release of radionuclides and late containment failure
  
  

• an understanding of the sources of uncertainty and their impact on the results.

The PRA scope needed to provide these results, and the minimal functional technical attributes necessary to ensure the risk analysis is capable of providing the above information are discussed in the following sections.

2.  PRA SCOPE

The scope of a PRA plays an important role in determining the role PRA results can have in the decision-making regulatory activity. The scope of a PRA is defined by the following characteristics:

• Degree of coverage of plant operating states (POSs) that define the plant's operating mode of concern: from full-power, to low-power, to shutdown modes of operation.
  
  
• Degree of coverage of initiating events, either internal or external to the plant boundary, that cause off-normal conditions.
  
  
• Level of characterization of risk:
  
  
• Level 1 PRA that estimates the CDF (given an event that challenges plant operation occurs).
  
  
• Level 2 PRA that estimates the containment failure and radionuclide release frequencies (given a core damage state occurs).
  
  
• Level 3 PRA that estimates the offsite consequences from a release, e.g., early and latent cancer fatalities (given a radionuclide release occurs).

For PRAs used in risk-informed activities (as outlined above), the scope and level of risk analysis are summarized in Table 1.

Table 1  List of Items Defining PRA Scope and Level of Risk Analysis

Initiating events are the events that have the ability to challenge the condition of the plant. These events include failure of equipment from either "internal plant causes" such as hardware faults, operator actions, floods or fires, or "external plant causes" such as seismic or high winds. The risk perspective should be based on the total risk connected with the operation of the reactor which includes events from both internal and external sources. Therefore, to gain the maximum benefit from a PRA, the model should address both internal and external initiating events.

The risk characterization used in risk-informed applications are CDF, LERF (as a surrogate for early fatalities), and the consideration of late containment failure; therefore, to provide the risk perspective for use in decision-making, a Level 1 PRA is required. A Level 2 PRA may be needed (i.e., estimation of the other release beyond a large early release is not needed) if the estimation of LERF for the level 1 PRA is not sufficient to provide insights on application-specific issues, or if late releases can become important for the application. A Level 3 PRA will not be required.

3.  PRA ELEMENTS AND TECHNICAL ATTRIBUTES

The technical elements of a PRA that provide acceptable results are summarized below in Table 2. A PRA that is missing one or more of these elements would not be acceptable and, in fact, would not be considered a PRA.

Table 2  Technical Elements of an Acceptable PRA

Each of the elements in Table 2 has associated with it technical attributes needed to ensure that the results are technically correct. These technical attributes are listed in Table 3.

Table 3  Summary of Characteristics and Attributes of an Acceptable PRA

In addressing the above elements, because of the nature and impact of internal flood and fire and external hazards, their attributes need to be discussed separately. This is because flood, fire and external hazards analyses have the ability to cause initiating events but also have the capability to impact the availability of mitigating systems. Therefore, in developing the PRA model, the impact of flood, fire and external hazards needs to be considered in each of the above technical elements. Table 4 provides a summary of the desired attributes of an acceptable internal flood and fire and external hazards analyses.

Table 4  Summary of Characteristics and Attributes of an Acceptable Internal Flood and Fire Analysis and External Hazards Analysis

Level 1 PRA --

Initiating event analysis identifies and characterizes those random internal events that both challenge normal plant operation during power or shutdown conditions and require successful mitigation by plant equipment and personnel to prevent core damage from occurring. Events that have occurred at the plant and those that have a reasonable probability of occurring are identified and characterized. An understanding of the nature of the events is performed such that a grouping of the events into event classes, with the classes defined by similarity of system and plant responses (based on the success criteria), may be performed to manage the large number of potential events that can challenge the plant.

Success criteria analysis determines the minimum requirements for each function (and ultimately the systems used to perform the functions) needed to prevent core damage (or to mitigate a release) given an initiating event occurs. The requirements defining the success criteria are based on acceptable engineering analyses that represent the design and operation of the plant under consideration. The criteria needed for a function to be successful is dependent on the initiator and the conditions created by the initiator. The code(s) used to perform the analyses for developing the success criteria are validated and verified for both technical integrity and suitability to assess plant conditions for the reactor pressure, temperature and flow range of interest, and accurately analyze the phenomena of interest. Calculations are performed by personnel qualified to perform the types of analyses of interest and are well trained in the use of the code(s).

Accident sequence development analysis models, chronologically, the different possible progression of events (i.e., accident sequences) that can occur from the start of the initiating event to either successful mitigation or to core damage. The accident sequences account for those systems and operator actions that are used (and available) to mitigate the initiator based on the defined success criteria and plant operating procedures (e.g., plant emergency and abnormal operating procedures and as practiced in simulator exercises). The availability of a system includes consideration of the functional, phenomenological and operational dependencies and interfaces between and among the different systems and operator actions during the course of the accident progression.

Systems analysis identifies the different combinations of failures that can preclude the ability of the system to perform its function as defined by the success criteria. The model representing the various failure combinations includes, from an as-built and as-operated perspective, the system hardware and instrumentation (and their associated failure modes) and the human failure events that would prevent the system from performing its defined function. The basic events representing equipment and human failures are developed in sufficient detail in the model to account for dependencies between and among the different systems, and to distinguish the specific equipment or human event (and its failure mechanism) that has a major impact on the system's ability to perform its function.

Parameter estimation analysis quantifies the frequencies of the identified initiators and quantifies the equipment failure probabilities and equipment unavailabilities of the modeled systems. The estimation process includes a mechanism for addressing uncertainties, has the ability to combine different sources of data in a coherent manner, and represents the actual operating history and experience of the plant and applicable generic experience as applicable.

Human reliability analysis identifies and quantifies the human failure events that can negatively impact normal or emergency plant operations. The human failure events associated with normal plant operation include those events that leave the system (as defined by the success criteria) in an unrevealed, unavailable state. The human failure events associated with emergency plant operation include those events that, if not performed, do not allow the needed system to function. Quantification of the probabilities of these human failure events are based on plant and accident specific conditions, where applicable, including any dependencies among actions and conditions.

Quantification analysis provides an estimation of the CDF given the design, operation and maintenance of the plant. This CDF is based on the summation of the estimated CDF from each initiator class. If truncation of accident sequences and cutsets is applied, truncation limits are set so that the overall model results are not impacted significantly and that important accident sequences are not eliminated. Therefore, the truncation limit can vary for each accident sequence. Consequently, the truncation value is selected so that the accident sequence CDF before and after truncation only differs by less than one significant figure.

Interpretation of results analysis entails examining and understanding the results of the PRA and identifying the important contributors sorted by initiating events, accident sequences, equipment failures and human errors. Methods such as importance measure calculations (e.g., Fussel-Vesely, risk achievement, risk reduction, and Birnbaum) are used to identify the contributions of various events to the model estimation of core damage frequency for both individual sequences and the model as a total. Sources of uncertainty are identified and their impact on the results analyzed. The sensitivity of the model results to model boundary conditions and other key assumptions is evaluated using sensitivity analyses to look at key assumptions both individually or in logical combinations. The combinations analyzed are chosen to fully account for interactions among the variables.

Level 2 PRA --

Plant damage state analysis groups similar core damage scenarios together to allow a practical assessment of the severe accident progression and containment response resulting from the full spectrum of core damage accidents identified in the Level 1 analysis. The plant damage state analysis defines the attributes of the core damage scenarios that represent important boundary conditions to the assessment of severe accidents progression and containment response that ultimately affect the resulting source term. The attributes address the dependencies between the containment systems modeled in the Level 2 analysis with the core damage accident sequence models to fully account for mutual dependencies. Core damage scenarios with similar attributes are grouped together to allow for efficient evaluation of the Level 2 response.

Severe accident progression analysis models the different series of events that challenge containment integrity for the core damage scenarios represented in the plant damage states. The accident progressions account for interactions among severe accident phenomena and system and human responses to identify credible containment failure modes including failure to isolate the containment. The timing of major accident events and the subsequent loadings produced on the containment are evaluated against the capacity of the containment to withstand the potential challenges. The containment performance during the severe accident is characterized by the timing (e.g., early versus late), size (e.g., catastrophic versus bypass), and location of any containment failures. The code(s ) used to perform the analysis are validated and verified for both technical integrity and suitability. Calculations are performed by personnel qualified to perform the types of analyses of interest and well trained in the use of the code(s).

Source term analysis characterizes the radiological release to the environment resulting from each severe accident sequence leading to containment failure or bypass. The characterization includes the time, elevation, and energy of the release and the amount, form, and size of the radioactive material that is released to the environment. The source term analysis is sufficient to determine whether a large early release (significant, unmitigated releases from containment in a time frame prior to effective evacuation of the close-in population such that there is a potential for early health effects) or large late release occurs (significant, unmitigated release from containment in a time frame that allows effective evacuation of the close-in population such that early fatalities are unlikely).

Quantification integrates the accident progression models and source term evaluation to provide estimates of the frequency of radionuclide releases that could be expected following the identified core damage accidents. This quantitative evaluation reflects the different magnitudes and timing of radionuclide releases and specifically allows for identification of the LERF and the probability of a large late release.

Interpretation of results analysis entails examining results from importance measure calculations (e.g., Fussel-Vesely, risk achievement, risk reduction, and Birnbaum) to identify the contributions of various events to the model estimation of LERF and large late release probability for both individual sequences and the model as a total. Sources of uncertainty are identified and their impact o the results analyzed. The sensitivity of the model results to model boundary conditions and other key assumptions is evaluated using sensitivity analyses to look at key assumptions both individually or in logical combinations. The combinations analyzed are chosen to fully account for interactions among the variables.

Internal Floods --

Flood identification analysis identifies those plant areas where flooding could pose significant risk. Flooding areas are defined on the basis of physical barriers, mitigation features, and propagation pathways. For each flooding area, flood sources due to equipment (e.g., piping, valves, pumps), internal (e;.g., tanks) and external (e.g., rivers) water sources are identified along with the affected SSCs. Flooding mechanisms are examined which include failure modes of components, human induced mechanisms, and other water releasing events. Flooding types (e.g., leak, rupture, spray) and flood sizes are determined. Plant walkdowns are performed to verify the accuracy of the information.

Flood evaluation analysis identifies the potential flooding scenarios for each flood source by identifying flood propagation paths of water from the flood source to its accumulation point (e.g., pipe and cable penetrations, doors, stairwells, failure of doors or walls). Plant design features or operator actions that have the ability to terminate the flood are identified. Credit given for flood isolation is justified. The susceptibility of each SSC in a flood area to flood-induced mechanisms is examined (e.g., submerge, spray, pipe whip, and jet impingement). Flood scenarios are developed by examining the potential for propagation and giving credit for flood mitigation. Flood scenarios can be eliminated on the basis of screening criteria. The screening criteria used are well defined and justified.

Quantification analysis provides an estimation of the CDF of the plant due to internal floods. Flooding induced initiating events that represent the design, operation and experience of the plant are identified and their frequencies quantified. The Level 1 models are modified and the internal flood accident sequences quantified: (1) modify accident sequence models to address flooding phenomena, (2) perform necessary calculations to determine success criteria for flooding mitigation, (3) perform parameter estimation analysis to include flooding as a failure mode, (4) perform human reliability analysis to account for PSFs due to flooding, and (5) quantify internal flood accident sequence CDF. Modification of the Level 1 models are performed consistent with the characteristics for Level 1 elements for transients and LOCAs. In addition, sources of uncertainty are identified and their impact o the results analyzed. The sensitivity of the model results to model boundary conditions and other key assumptions is evaluated using sensitivity analyses to look at key assumptions both individually or in logical combinations. The combinations analyzed are chosen to fully account for interactions among the variables.

Internal Fire --

Screening analysis identifies fire areas where fires could pose a significant risk. Fire areas which are not risk significant can be "screened out" from further consideration in the PRA analysis. Both qualitative and quantitative screening criteria can be used. The former address whether an unsuppressed fire in the area poses a nuclear safety challenge; the latter are compared against a bounding assessment of the fire-induced core damage frequency for the area. The potential for fires involving multiple areas should be addressed. Assumptions used in the screening analysis should be verified through appropriate plant walkdowns. Key screening analysis assumptions and results, e.g., the area-specific conditional core damage probabilities (assuming fire-induced loss of all equipment in the area), should be documented.

Fire initiation analysis determines the frequency and physical characteristics of the detailed (within-area) fire scenarios analyzed for the unscreened fire areas. The analysis needs to identify a range of scenarios which will be used to represent all possible scenarios in the area. The possibility of seismically-induced fires should be considered. The scenario frequencies should reflect plant-specific experience, and should be quantified in a manner that is consistent with their use in the subsequent fire damage analysis (discussed below). The physical characterization of each scenario should also be in terms that will support the fire damage analysis (especially with respect to fire modeling).

Fire damage analysis determines the conditional probability that sets of potentially risk-significant components (including cables) will be damaged in a particular mode, given a specified fire scenario. The analysis needs to address components whose failure will cause an initiating event, affect the plant's ability to mitigate an initiating event, or affect potentially risk significant equipment (e.g., through suppression system actuation). Damage from heat, smoke, and exposure to suppressants should be considered. If fire models are used to predict fire-induced damage, compartment-specific features (e.g., ventilation, geometry) and target-specific features (e.g., cable location relative to the fire) should be addressed. The fire suppression analysis should account for the scenario-specific time required to detect, respond to, and extinguish the fire. The models and data used to analyze fire growth, fire suppression, and fire-induced component damage should be consistent with experience from actual nuclear power plant fire experience as well as experiments.

Plant response analysis involves the modification of appropriate plant transient and LOCA PRA models to determine the conditional core damage probability, given damage to the set(s) of components defined in the fire damage analysis. All potentially significant fire-induced initiating events, including such "special" events as loss of plant support systems, and interactions between multiple nuclear units during a fire event, should be addressed. The analysis should address the availability of non-fire affected equipment (including control) and any required manual actions. For fire scenarios involving control room abandonment, the analysis should address the circuit interactions raised in NUREG/CR-5088, including the possibility of fire-induced damage prior to transfer to the alternate shutdown panel(s). The human reliability analysis of operator actions should address fire effects on operators (e.g., heat, smoke, loss of lighting, effect on instrumentation) and fire-specific operational issues (e.g., fire response operating procedures, training on these procedures, potential complications in coordinating activities). In addition, sources of uncertainty are identified and their impact o the results analyzed. The sensitivity of the model results to model boundary conditions and other key assumptions is evaluated using sensitivity analyses to look at key assumptions both individually or in logical combinations. The combinations analyzed are chosen to fully account for interactions among the variables.

External Hazards --

Screening and bounding analysis identifies external events other than earthquake that may challenge plant operations and require successful mitigation by plant equipment and personnel to prevent core damage from occurring. The term "screening out" is used here for the process whereby an external event is excluded from further consideration in the PRA analysis. There are two fundamental screening criteria embedded in the requirements here, as follows: An event can be screened out either (i) if it meets the certain design criteria, or (ii) if it can be shown using an analysis that the mean value of the design-basis hazard used in the plant design is less than 10^-5/year, and that the conditional core-damage probability is less than 10^-1, given the occurrence of the design-basis hazard. An external event that cannot be screened out using either of these criteria is subjected to the detailed-analysis.

Hazard Analysis characterizes non-screened external events and seismic events, generally, as frequencies of occurrence of different sizes of events (e.g., earthquakes with various peak ground accelerations, hurricanes with various maximum wind speeds) at the site. The external events are site specific and include both aleatory and epistemic uncertainties.

Fragility Analysis characterizes conditional probability of failure of important structures, components, and systems whose failure may lead to unacceptable damage to the plant (e.g., core damage) given occurrence of an external event. For important SSCs, the fragility analysis is realistic and plant-specific. The fragility analysis is based on extensive plant-walkdowns reflecting as-built, as-operated conditions.

Level 1 Model Modification assures that the system models include all important external-event caused initiating events that can lead to core damage or large early release. The system model includes external-event induced SSC failures, non-external-event induced failures (random failures), and human errors. The system analysis is well coordinated with the fragility analysis and is based on plant walkdowns. The results of the external event hazard analysis, fragility analysis, and system models are assembled to estimate frequencies of core damage and large early release. Uncertainties in each step are propagated through the process and displayed in the final results. The quantification process is capable of conducting necessary sensitivity analysis and to identify dominant sequences and contributors.

Documentation

Traceability and defensibility provides the necessary information such that the results can easily be reproduced and justified. The sources of information used in the PRA are both referenced and retrievable. The methodology used to perform each aspect of the work is described either through documenting the actual process or through reference to existing methodology documents. Assumptions⁽¹⁾ made in performing the analyses are identified and documented along with their justification to the extent that the context of the assumption is understood. The results (e.g., products and outcomes) from the various analyses are documented.

4.  PEER REVIEW PROCESS

A peer review process can be used to identify weaknesses in the PRA and the importance of the weaknesses to the confidence in the PRA results. An acceptable peer review needs to be performed by qualified personnel, needs to be performed according to an established process that compares the PRA against desired characteristics and attributes, and needs to document the results including both strengths and weaknesses of the PRA.

The desired characteristics and attributes for an acceptable peer review of a PRA are described below and summarized in Table 5.

Table 5  Summary of Desired Characteristics and Attributes of a Peer Review

The peer review process includes a documented procedure to direct the team in evaluating the acceptability of a PRA. The review process compares the PRA against the desired PRA characteristics and attributes. In addition to reviewing the methods utilized in the PRA, the peer review also determines if the application of those methods were done correctly. The PRA models are compared against the plant design and procedures to validate that they reflect the as-built and as-operated plant. Key assumptions are reviewed to determine if they are appropriate and if they have a significant impact on the PRA results. The PRA results are checked for fidelity with the model structure and also for consistency with the results from PRAs for similar plants. Finally, the peer review process examines the procedures or guidelines in place for updating the PRA to reflect changes in plant design, operation, or experience.

Documentation provides the necessary information such that the peer review process and the findings are both traceable and defensible. A description of the qualifications of the peer review team members and the peer review process are documented. The results of the peer review for each technical element and the PRA update process are described including those areas where the PRA do not meet or exceed the desired characteristics and attributes used in the review process. This includes an assessment of the importance of any identified deficiencies on the PRA results and potential uses and how these deficiencies were addressed and resolved.

5.  PRA TECHNICAL ACCEPTABILITY

The technical acceptability of the PRA can be determined by performing a peer review against a defined set of elements and characteristics specifying the scope and risk characterization. Applications can differ in the weight given to PRA results in the decision-making process. The weight given will depend on the scope of the PRA as well as its technical quality. For a given scope, the technical quality will determine the degree of confidence the decision-maker can have in the results and their role in the decision-making.

This role of the PRA is determined initially by its ability to produce the results required of the decision, and secondly by the degree of coverage of the risk contributors included in the risk metrics used in the decision. Given the role has been defined, the next step is to determine the technical acceptability of the PRA to support the results used, identify the differences, determine the importance of the differences, and determine an acceptable resolution for the important differences. The characteristics and attributes of this process are described below and summarized in Table 6.

Table 6  Summary of Characteristics and Attributes of an Acceptable Use of a PRA in Risk-Informed Applications

Determination of the adequacy of PRA identifies differences between the existing PRA and the above defined PRA scope, elements, and technical attributes and the significance of these differences. It may be determined that the scope of the existing PRA does not provide the required risk information, (for example because it only addresses internal events at full power, and the decision algorithm involves risk from all modes of operation and all initiating events); or it does not have the needed elements and technical attributes for the specific application. For the important differences, a process for resolution is determined (as discussed below).

Resolution of Differences identifies the process for resolution of identified important differences between the standard and the PRA. The resolution process either includes updating the PRA to include the important missing scope, elements and attributes, or performing compensatory measures. These measures involve accounting for deficiencies by an expert panel (see below).

6.  EXPERT PANEL

As discussed above, not meeting specific attributes of an element that is important to the decision under consideration does not necessarily invalidate the use of the PRA model. The results will either have to be supplemented by engineering judgement, or compensated for by including conservatisms, or limitations in the implementation of the decision. This process can be performed with the use of an expert panel.

If an expert panel approach is elected, then there are certain characteristics and attributes that the expert panel needs to meet to be an acceptable alternative. With respect to the PRA, the primary responsibility of the expert panel is to establish the role that PRA results play in the decision, commensurate with the level of confidence in those PRA results. This requires establishing an appreciation of, and compensation for, the limitations of the model, which can be identified by comparison with the desired requirements for technical acceptability. PRA technical acceptability, as discussed above, may be achieved by performing a PRA that meets the desired characteristics and attributes defined for each technical element for the defined scope and level of analysis.

The desired characteristics and attributes to define an acceptable expert panel that are needed to support the identified applications are described below and summarized in Table 7.

Table 7  Summary of Desired Characteristics and Attributes of an Expert Panel to Use PRA Results

The decision-making process is based on a written, systematic approach and shown to be appropriate for the decisions the panel is needed to render. The necessary technical information is made available to the panel and is examined to allow the applicable issues to be raised. The issues are disposed of using a systematic and defensible process, and documentation of findings made by the panel are traceable and reviewable. Any evaluation of the risk significance of issues appropriately consider probabilistic information, traditional engineering evaluations, sensitivity studies, operational experience, engineering judgment, and current regulatory requirements.

The technical information bases provides the necessary information for the panel to arrive at a defensible decision. This information is derived from various sources, including, for example, simplified or detailed engineering analyses, specific plant-operational expertise, and expert opinion, and shown to be adequate for the scope of the analysis. Therefore, the used technical information is sufficient to allow analysis (e.g., quantification) of both success and failure scenarios to (1) identify the roles played by the SSCs, and (2) establish the safety significance of the SSCs; and to identify causal models to be used to establish the effects of any proposed changes.

Incorporation of non-PRA modeled items involves evaluating the safety significance items not modeled in the PRA but affected by a proposed application. This systematic evaluation consists of searching for items that might contribute to initiating event occurrence, identifying mitigating system items that were not modeled in the PRA because their failure was not expected to dominate system failure in the baseline configuration, and recognizing items in systems that do not play a direct role in accident mitigation but do interface with accident mitigating systems.

Identification of limitations specifies those aspects in the PRA that decrease the level of confidence in the results, and consequently, to be addressed by the expert panel process. These deficiencies may exist because (1) an item was not modeled in the PRA, (2) an item was inappropriately modeled, or (3) lack of technology to adequately model in the PRA. The process used by the expert panel to resolve the deficiency is based the type of deficiency identified and includes (1) modeling the item in the PRA or accounting for the effects of the item by other means (e.g., using surrogate components), (2) revising the PRA model to appropriately model the item, or (3) soliciting and using expert opinion to resolve items involving a lack of technology. When a decision made by the panel that does not follow straightforwardly from the PRA, a technical basis is provided that shows how the PRA information and the supplementary information validly combine to support the finding. Further, no findings by the panel can contradict the PRA in a fundamental way.

Documentation provides the necessary information such that the expert panel process and its findings are both traceable and defensible. The documentation includes a description of the qualifications of each expert panel member, the written procedures employed by the panel, and a report of any decisions made by the panel including the basis for the conclusions.

ATTACHMENT 2

PRA QUALITY IN RISK-INFORMED REGULATION

• Risk Insights in Decision-Making
• Implementation of Decisions

In making any risk-informed regulatory decision, the staff's goal is to ensure that an appropriate level of safety is maintained, and, in particular, if any risk increases do occur as a result of a design or operational change, they are small as discussed in Regulatory Guide 1.174. A PRA provides only one part of the information used to make such a decision. Since, as discussed below, the importance of the PRA results will vary from decision to decision, the quality of the PRA must be judged in the context of the decision-making process, and on the way the PRA results are used to justify the decision. The quality of the PRA, coupled with an understanding of the sources of uncertainty and how they impact the results is what determines the confidence we can have in the results it generates. The less confident the decision-maker is in the results, the more he has to rely on compensatory measures to ensure that safety is maintained. These measures include an increased reliance on the more traditional approaches such as relying on defense-in-depth or adequate safety margins, which will restrict the degree of implementation of the application. Another approach is to institute performance monitoring to make sure that any plant changes do not result in unexpected degradation of performance. There will, therefore, generally be a trade-off between the benefit to be obtained from the application, in terms of relaxation of requirements for example, and the quality of the risk information. The better the quality of PRA information, the more benefit that should be expected.

Risk Insights in Decision-Making

In making a regulatory decision risk insights are integrated with defense-in-depth and safety margins considerations. The degree to which the risk insights play a role, and therefore the need for detailed staff review, is application dependent.

Quantitative risk results from PRA calculations are typically the most useful and complete characterization of risk, but they are generally supplemented by qualitative risk insights and traditional engineering analysis. Qualitative risk insights include generic results which have been learned from the numerous PRAs that have been performed in the past decades, and from operational experience. For example, if one is deciding which motor operated valves in a plant can be subject to less frequent testing, the plant-specific PRA results can be compared with results from similar plants. This type of comparison can give support to the licensee's analysis, and reduce the reliance of the staff review on the quality of the licensee PRA. However, as a general rule, applications that impact large numbers of SSCs will benefit from having a PRA of high technical quality.

Traditional engineering analysis provides insight into available margins and defense in depth. In the example of the operational assessment of steam generator tubes discussed below, it is traditional engineering analysis that provides assurance of meeting the structural integrity and leakage criteria. With few exceptions, these assessments are performed without any quantification of risk.

In general, a risk-informed application will require some quantitative risk calculations using PRA methods. In some cases, the use of PRA will be extensive and will be crucial to the success of the application. There are some proposals for real-time use of the PRA and associated risk management software as a tool to assess plant configuration. The more ambitious proposals involve the use of "risk meters." For example, the NRC and industry are cooperating on the risk-informed standard technical specification (RI-STS) project. One element of this project is to replace the traditional limiting conditions for operation (LCO) action statements with a PRA based approach. When a licensee encounters an LCO, rather than shutting down the plant, they would be authorized to use the plant PRA to determine an appropriate configuration which represents an acceptable level of risk. Applications of this type require a detailed PRA model that is capable of evaluating the risk associated with specific plant configurations. Since the configuration specific risk could be affected by any of the elements of the model, this requires that the model has to be of relatively high quality.

There are some applications which, because of the nature of the proposed change, have a limited impact on risk, and this is reflected in the impact on the elements of the risk model. Such an application would lead the staff to conduct a more limited review of the risk estimates, and therefore to place less emphasis on the quality of the PRA than might otherwise have been the case. The staff would also tend to focus its PRA review in specific areas of the PRA.

An example of this was the issue of BWR incremental power uprates. The staff examined the early submittals to determine whether there was reason for concern about undue increases in risk. Five potential areas of impact on risk were postulated and evaluated in the context of two plant applications. The staff concluded that the risk implications of the BWR power uprates was limited by the nature of the application. The staff also concluded that the one area requiring review in some cases is the possibility that increased power levels would result in less time for operator action during an accident. This is an example of how the extent of analysis required to support an application can be circumscribed in advance by examining the inherent risk limitation of the application.

Another example is risk-informed inservice inspection (RI-ISI). In this application, risk significance was used as one criterion for selecting pipe segments to be periodically examined for cracking. The staff spent a great deal of effort reviewing the topical reports and pilot plant submittals. Much of this effort was focused on the quantitative estimates of risk. During the review, however, it became clear that this level of emphasis of PRA results and PRA quality was not necessary. Since ISI involves examination of a sample of pipe segments to identify the onset of cracking problems, the success of the program was very tolerant to errors in the risk significance of the selected segments. Therefore, the staff review of plant specific submittals referencing the RI-ISI topical will include only a limited scope review of PRA quality.

Implementation of Decisions

The implementation of the decision is a function of the confidence we can have in the results of the analysis. One important factor that can be considered when determining the degree of implementation of the change, is the ability to monitor the performance to limit the potential risk. In many applications, the potential risk can be limited by defining specific measures and criteria which must be monitored subsequent to approval. When relying on performance monitoring, the staff must have assurance that the measures truly represent the potential for risk increase, and that the criteria are set at reasonable limits. Moreover, one must be sure that degrading performance can be detected in a timely fashion, long before a significant public health issue results. The impact of the monitoring can be fed back into the analysis to demonstrate how it supports the decision.

An example of this is the management of steam generator tube degradation. The NRC staff has been working with industry to approve licensee use of NEW-97-06, a guidance document for determining what tubes can be left in service and how frequently steam generators need to be inspected. The guidance in NEW-97-06 includes guidance for licensees to perform an operational assessment prior to restart from an outage. Any tubes which exceed certain limits must be repaired or removed from service. The licensee must determine whether the tubes left in service will meet structural strength and leakage criteria at the end of the cycle. If not, the licensee must take compensatory action, such as a mid-cycle inspection. At the end of the cycle, the licensee must perform condition monitoring, in which the actual condition is examined to determine whether the actual performance met the criteria. Any unfavorable deviation of the actual tube behavior from the predicted performance must be accounted for in subsequent operational assessment.

In this example, performance monitoring (condition monitoring) is relied upon to assure that any deviations from acceptance criteria are detected promptly. Moreover, the results are used to improve the analysis techniques to limit potential deviations in future cycles. The NRC staff has decided that the performance monitoring in NEW-97-06 is of sufficient quality and timeliness to assure acceptable risk from steam generator tube failure. Consequently, the staff concentrates more of its review and inspection effort on the results of the condition monitoring, rather than on the far more complex and time consuming review of pre-cycle predictions.

Finally, when implementing a decision, the licensee may choose to compensate for lack of confidence in the analysis by restricting the degree of implementation. This has been the technique used in several applications involving SSC categorization into low or high safety significance. In general unless there is compelling evidence that the SSC is low safety significant it is maintained as high safety significant. This requires a reasonable understanding of the limitations of the PRA. Another example of risk limitation is the placing of restrictions on the application. For example, risk-informed technical specification allowed outage time changes are accompanied by implementation of a configuration risk management program, which requires licensees to examine their plant configuration before voluntarily entering the approved condition.

The NRC review of an application will take all these factors into consideration. The review of PRA quality in particular will focus on those aspects that impact the results used in the decision, and on the degree of confidence required in those results.

1. Assumptions include those decisions and judgments that were made in the course of the analysis.