| |
Search Options | |||
| Index | Site Map | FAQ | Facility Info | Reading Rm | New | Help | Glossary | Contact Us | |
|||
POLICY ISSUE
INFORMATION
SECY-00-0146
June 28, 2000
| FOR: | The Commissioners |
| FROM: | William D. Travers Executive Director for Operations |
| SUBJECT: | STATUS OF RISK-BASED PERFORMANCE INDICATOR DEVELOPMENT AND RELATED INITIATIVES |
To provide the Commission with a status report on the development of risk-based performance indicators (RBPIs) and related initiatives in support of the Reactor Oversight Process (ROP).
The current ROP utilizes performance indicators that were developed based on generic risk insights. These performance indicators provide a measure of plant performance in selected areas and utilize generic performance thresholds. They are described in SECY 99-007, "Recommendations for Reactor Oversight Process Improvement." When SECY 99-007 was developed, it was recognized that improved performance indicators could be developed to enhance the ROP. RBPIs are intended to provide improved indicators to the ROP.
RBPIs are currently under development and the outcome of this development will determine the extent to which they are employed.
A white paper (Attachment 1 
)
on the development of RBPIs was issued for comment in March 2000. The purpose
of this white paper was to provide an overview of the current efforts to develop
RBPIs. The development of the RBPI white paper was closely coordinated with
the Office of Nuclear Reactor Regulation (NRR) and the Regions. In addition,
a public meeting with external stakeholders was held to obtain their comments.
Attendees included the Nuclear Energy Institute (NEI), Institute of Nuclear
Power Operations (INPO), Union of Concerned Scientists, and Public Citizen.
RBPIs provide performance measures that are related as explicitly as practical to risk-significant elements of plant operation. That is, they provide performance measures whose impact on core damage frequency (CDF) and large, early release frequency (LERF) can be established through a risk model or risk logic. In developing RBPIs, "performance" refers to activities in design, procurement, construction, operation, and maintenance that support achievement of the objectives of the cornerstones of safety in the ROP. Although the indicators are "risk-based" they will be used as an input to the "risk-informed" ROP decision making process.
The RBPIs could provide the following benefits to the ROP:
Development of RBPIs will be accomplished in phases. Phase-1 of the development will include reliability and availability indicators for full power mode, shutdown modes, internal events, fire events, and industry-wide performance trends. Phase-1 will include the initiating events, mitigating systems, and the containment portion of the barrier integrity cornerstones of safety. Additional phases of the RBPI development will include an integrated indicator, improvements to the Phase-1 RBPIs, consideration of other external events (e.g., seismic and wind), and follow-on work to improve existing indicators in response to the ROP implementation.
The RBPI development activities have been closely coordinated with NRR and the Regions. A number of meetings were held to review the general approach and concepts. The RBPI white paper reflects those review comments.
The ACRS was briefed on the RBPI white paper on April 5, 2000. The ACRS issued a letter on April 23, 2000, that documented the results of their evaluation
of the RBPI development program (Attachment 2 ). The ACRS concluded that the RBPI development is very important for
the successful transition to risk-informed regulation and recommended that the agency assign a high priority and adequate resources to this program. The ACRS also recommended that the staff should work with the industry to ensure that licensee reporting of reliability data becomes an industry self-imposed requirement of the Equipment Performance and Information Exchange (EPIX) database.
A public meeting with external stakeholders was held on April 28, 2000, to discuss their comments on the overall concept and technical approach outlined in the RBPI development white paper. A summary of their comments discussed at the meeting and the written comments provided by the external stakeholders are presented in Attachment 3. The external stakeholder comments were focused primarily on policy and implementation issues, which will be addressed following the Phase-1 effort to assess the technical feasibility of RBPI development. The key issues raised by external stakeholders were:
Industry-wide performance trends are part of the industry-wide reactor safety performance "measures" discussed in the draft NRC Strategic Plan for FY 2000-2005. Currently, industry trends are monitored using a set of seven performance indicators (NUREG-1187, "Performance Indicators for Operating Commercial Nuclear Power Reactors") and trends from the ASP program. In the future, performance indicators and inspection findings from the ROP and trends from the ASP program could be used to monitor industry trends. However, in addition to the above, performance indicators from the RBPI program could also be used since these would provide a broader coverage of risk and industry-wide trends, including risk-significant trends on performance elements that are difficult to trend on a plant-specific basis. These trends could also provide feedback to the ROP to assess the effectiveness of its oversight activities.
The NRC staff is developing Phase-1 RBPIs using the overall concept described in the RBPI white paper. The preliminary results of this effort will be published in the summer of 2000. We plan to brief the Commission on study findings in December 2000 following interactions with internal and external stakeholders.
The EPIX database is an industry initiative that collects information on equipment performance for a number of applications. EPIX includes information on reliability and availability of certain components that are critical to the development of many potential RBPIs. We have been working with INPO on this activity in response to the SRM dated June 13, 1997, on SECY-97-101 directing the staff to work with industry on a voluntary alternative to the proposed Reliability and Availability Data Rule. Verification and validation of the EPIX data is a crucial element of the RBPI development. We are working with INPO and NEI on these issues.
| /RA by Frank J. Miraglia Acting For/
William D. Travers |
| Contact: | Hossein G. Hamzehee, RES/DRAA/OERAB (301) 415-6228 |
At the meeting on April 28, 2000, five specific questions were raised. The questions and the corresponding responses/comments are summarized below:
Are concepts in the RBPI white paper consistent with the philosophy and direction?
Are RBPI characteristics appropriate?
Are RBPI development steps appropriate?
Are the proposed RBPI elements (data sources, RBPIs, industry trends) appropriate for RROP?
Are proposed Phase-1 products appropriate?
Some of the comments were related to the RBPI process and general communication issues as summarized below:
[ Letter - Comments on Risk-Based Performance Indicators ]
Mr. Thomas L. King, Director
Division of Risk Analysis and Applications
Office of Nuclear Regulatory Research
Mail Code 10 E5O
U.S Nuclear Regulatory Commission
Washington, DC 20555-0001
Dear Mr. King:
Thank you for the opportunity to participate in the public meeting held on April 28, 2000 to discuss the NRC Research White Paper, "Development of Risk-Based Performance Indicators: Program Overview." This letter reiterates our oral comments at that meeting and the issues we believe need to be addressed in the ongoing effort to develop meaningful and useful performance indicators for use in the Reactor Oversight Process (ROP).
NEI believes it is appropriate for NRC to pursue improvements to the ROP performance indicators. However, it is important to keep in mind the intended purpose of these indicators. PIs perform an important, but limited role in the ROP. They are meant to be a set of indicators which, in conjunction with inspection and the SDP process, allow the NRC to assess plant performance and determine where to allocate inspection resources.
Performance indicators in the ROP are not meant to be the full set of systems, structures, and components whose reliability and unavailability will allow one to calculate an integrated core damage frequency using plant specific Standardized Plant Analysis Risk (SPAR) models. That is not the purpose of PIs, although it appears to be a major objective of NRC Research in developing Risk-Based Performance Indicators (RBPI). Seeking to "measure safety through a comprehensive set of indicators is a significant departure from the current purpose of PIs and should be dealt with as a policy issue.
The plant specific SPAR models used by the NRC have not been validated by utilities. These models should be reviewed and modified as necessary prior to attempting to implement plant specific thresholds.
The issue of the applicability and use of the Equipment Performance and Information Exchange System (EPIX) data base to populate Research's RBPIs is also an issue which needs to be discussed at a management level above the working level group. In fact, the NEI Data Review Group (which includes NRC, NEI, INPO and utility membership) is studying the data demands on industry and the cost/benefit of additional data collection. The group will be reporting to the NEI Nuclear Strategic Issues Advisory Committee (NSIAC - a group consisting of all utilities Chief Nuclear Officers) later this year.
We are concerned that NRC Research is proceeding with pilot plant evaluations of new RBPIs using the EPIX data base without the participation of NRR and NEI's Plant Safety Assessment Task Force. The process of developing the PIs in the ROP was in large measure successful due to the public interaction and deliberative process employed to identify potential indicators, determine their value in the context of plant assessment, refine definitions prior to piloting, careful data collection, and finally acceptance.
The White paper suggests that PIs may be developed to attempt to measure cross-cutting issues, such as PIs at the component level and measurement of human performance. We believe these efforts will be as unsuccessful in the future as they have proven to be in the past.
In addition, the paper suggests the development of an "integrated" indicator. The ROP purposefully avoided the development of an integrated or overall "score" for the program for two reasons: (1) it places inappropriate attention on a single number (derived through subjective decisionmaking in the case of SALP; in this case, a subjective system to weight different indicators) and (2) a single integrated indicator is not actionable, i.e., what specific action should be taken based on this artificial, made-up number?
We believe that the efforts of Research as described in the White Paper should be reoriented to address the key problems with the current set of indicators and to propose new indicators in those areas currently lacking indicators. For example:
We also believe it will be useful for NRC to pursue whether there is a more appropriate criteria for the "green-white" threshold, which in most cases is now based on the concept of outliers from normal and satisfactory industry performance (i.e., a 95th percentile threshold.) While a more risk based threshold should be explored, it must be capable of being easily explained to all stakeholders.
It is not clear what level of resources it would take to gather the vastly expanded number of RBPIs. This increased regulatory burden needs to be balanced against the gain of reduced inspection activity. Is the NRC truly receptive to reducing the baseline inspection program based on additional PIs?
If additional PIs are added to the ROP, what will be the impact on the Regulatory Action Matrix? It would seem that there should be an adjustment in the number of white indicators which cause one to move from one column to another.
Two characteristics of useful indicators which are not mentioned in the white paper, but which are essential if they are to be used in the ROP are: (1) easy to understand and collect data, and (2) minimize potential for unintended consequences, e.g., causing an operator or manager to take inappropriate action in order to avoid crossing a P1 threshold.
Please call me at 202-739-8078 or email me at sdf@nei.org if you have any questions.
Sincerely
Stephen D. Floyd
| Christine Todd Whitman Governor |
State of New Jersey Department of Environmental Protection Division of Environmental Safety, Health and Analytical Programs Radiation Protection Programs P0 Box 415 Trenton, New Jersey 08625-0415 Phone: (609) 984-5636 Fax: (609) 633-2210 |
Robert C. Shinn Jr. Commissioner |
June 8, 2000
Chief, Operating Experience Risk Analysis Branch
Division of Risk Analysis and Applications
Office of Nuclear Regulatory Research
U.S. Nuclear Regulatory Commission
Washington, DC 20555-0001
Subject: Risk-Based Performance Indicator White Paper
Dear Mr. Baranowsky:
We have reviewed the U.S. Nuclear Regulatory Commission (NRC) April 14, 2000 white paper, which outlines the NRC's preliminary overview and approach to incorporate risk-based performance indicator (RBPI) in the recently implemented NRC commercial nuclear power plant oversight program. We appreciate the NRC's openness in inviting comment on the white paper, and hope that we will have continuing interaction as you consider our comments.
The use of risk informed performance indicators would be an improvement over the current performance indicators because you are establishing a scientific basis for the performance indicators. We have previously commented to the NRC on the revised oversight process for nuclear power plants and we focused on the lack of a connection between the performance indicators and risk. We first learned of the Office of Nuclear Regulatory Research's efforts during the "lessons learned" meeting in January 2000. We support your efforts. The connection between performance indicators and risk, including the basis for each color threshold is necessary if the overall oversight program is to have credibility.
Your white paper makes it clear, however, that the NRC does not intend to focus on the current performance indicators but examines the process for developing new risk based performance indicators. The exact characterization of the effort is that the development of RBPI's is a "possible enhancement to the Revised Reactor Oversight Process". Though we support your efforts to bring some logic and risk basis to the performance indicators, it is puzzling that there is no effort to examine the nsk basis of the current performance indicators. It appears that the addition of the RBPIs, as described in Figure ES-i will simply add a new layer.
An already complicated system will be made more complicated without addressing the original complications. We suggest that the basis for the current performance indicators be reviewed and justified by the Office of Nuclear Regulatory Research.
Tn the process of developing the RBPIs, NRC will be looking at IPEs, IPEEEs, and existing PRA studies to develop more plant specific indicators. Similar plants will be grouped together in an effort to reflect risk sigmficant differences in plant designs. This seems like an improvement, but there are concerns with the accuracy of the PRAs on an individual plant basis.
They were never intended for the development of performance indicators. How will the Office of Nuclear Regulatory Research ascertain that the PRAs reflect actual conditions at the plant? In grouping plants of "similar" design, there is a question of how similar. In using the PRA data, the NRC will have to also determine the uncertainty introduced by the PRA data itself and then add the uncertainty introduced by grouping plants of similar design.
According to page 10 of the white paper, the SPAR model will then evaluate plant-specific baseline values. So another level of uncertainty wilL be added by taking information from similar plants and then reapplying it to particular plants. The levels of uncertainty seem to be multiplying as the methodology for developing RBPIs is sequentially applied.
The description of the "integrated indicator" which combines the risk significance of changes occurring in all monitored performance areas seems to be so far removed from the individual indicators that its significance is questionable. The combined uncertainty could be so great that the integrated indicator would have no relevance to risk.
While the white paper is very careful to say that use of the RBPIs is only a possible enhancement, and no implementation decisions have been made, there is some speculation about how the RBPIs could be used. In the first full paragraph on page ii, you state that the greater coverage of risk significant performance afforded by the RBPIs will allow for concomitant changes to inspections in those areas covered by the RBPIs and the explicit identification of risk significant areas that the inspection program must cover. This statement can be taken two ways. Either it means that less inspection will be required and the current inspection program will be trimmed down or that new ~n~ections will be developed to cover any new risk significant findings keeping the overall inspection effort constant. Our position is that the total number of inspections in the baseline inspection program is too little now and the overall level should not be reduced but increased. With our lack of confidence in the current oversight program's ability to capture performance, current or declining, we do not support reduced inspections or inspectors.
On page iv, you state that RBPI's will be developed in all cornerstones except in the areas of emergency preparedness, radiation and security. What is your plan for developing a risk basis for the performance indicators in those areas?
Finally, we would like to participate in public meetings on this new initiative. The summary of expected accomplishments on page 16 contains no indication of the allocation of time to each phase. Can you provide specific dates when products will be available for review and comment? This will help our staff better plan to participate in this process. We need at least 30 days notice prior to the meeting in order to have approval to travel
| Regards,
Jill Lipoti, Ph.D. |
| cc: | Paul Lohaus, NRC Hubert Miller, Region 1 Administrator, NRC Kent Tosch, NJ DEP Dennis Zannoni, NJ DEP |
New Jersey is an Equal Opportunity Employer
Recycled Paper
|
Privacy Policy |
Site Disclaimer |