| |
Search Options | |||
| Index | Site Map | FAQ | Facility Info | Reading Rm | New | Help | Glossary | Contact Us | |
|||
SECY-00-0049
February 24, 2000
| FOR: | The Commissioners |
| FROM: | William D. Travers /RA/ Executive Director for Operations |
| SUBJECT: | RESULTS OF THE REVISED REACTOR OVERSIGHT PROCESS PILOT PROGRAM |
This Commission paper provides the results and lessons learned from the 6-month pilot program conducted on the revised reactor oversight process (RROP). In addition to a discussion and evaluation of the pilot program lessons learned, this paper requests Commission approval to implement the RROP at all power reactors. Finally, this paper provides the staff response to the Commission comments documented in the Staff Requirements Memorandum (SRM) on SECY-99-007 and SECY-99-007A dated June 18, 1999.
A 6-month pilot program of the RROP was conducted at two sites per region from May to November 1999. The purpose of the pilot program was to apply the RROP and identify lessons learned so that the various processes and procedures could be refined and revised as necessary prior to a Commission decision on the initial implementation of the RROP at all power reactors. Pilot program criteria were established to evaluate the results of implementing each of the components of the RROP at the pilot plants.
In addition to evaluating the new process against these pilot program criteria, the staff employed a number of methods to obtain internal and external stakeholder feedback during the pilot program. This feedback was considered by the staff, along with the other pilot program results and lessons learned, and pertinent oversight processes and procedures were revised as appropriate.
The effort undertaken by the staff to implement the RROP at the pilot plants highlighted the challenges inherent in developing a risk-informed regulatory oversight process. Due to its nature, the uncertainties associated with risk analysis make it difficult to establish objective, risk-informed thresholds for both performance indicators and inspection findings. However, the pilot program demonstrated that these new risk-informed tools, used by a knowledgeable and experienced inspection staff, result in an oversight process that is more objective and predictable than the current oversight process.
Based on the results of the 6-month pilot program, the staff has concluded that the cornerstones of safety concept and the associated framework is sound. Pilot program feedback received by the staff, from both internal and external stakeholders, indicates that further experience with the process is needed. Implementing the RROP at all sites will enable the staff to acquire further experience and provide it the opportunity to identify additional lessons learned and gain greater confidence in the efficacy of the RROP.
The pilot program increased staff confidence that the combination of PIs and inspection findings can provide adequate indications of licensee safety performance. The appropriate implementation of these processes should provide reasonable assurance that safe plant operation is maintained. Most internal and external stakeholders agreed that the RROP provides assessments of licensee performance, with corresponding NRC actions, in a manner that is more objective, understandable, and predictable than the current oversight process. Additional data is needed following initial implementation to confirm the capability of the RROP to identify declining safety performance trends in a timely manner, adjust thresholds if needed, and determine any increased efficiency resulting from the new oversight processes. Based on industry and agency feedback, the regulatory burden associated with the RROP appears appropriate.
Certain significant aspects of the RROP (e.g., event response and the annual assessment process) were not able to be fully exercised during the pilot program. The staff has determined that the continued development and implementation of these, and other aspects of the RROP, will not adversely impact the initial implementation of the RROP. As described in this Commission paper, these longer term issues will be addressed by a program self-assessment to be conducted by the staff during the first year of implementation.
Other than these noted exceptions, the procedures and guidance documents required to support initial implementation of the RROP at all power plants have been sufficiently exercised during the pilot program. Most pilot program lessons learned have been incorporated into the oversight process procedures as appropriate and these procedures are expected to be issued in time to support initial implementation. A few significant issues remain to be resolved to support initial implementation. These issues, and planned staff action to ensure that they are adequately resolved, are described in detail in this Commission paper. The NRC will continue to conduct staff training and public workshops to ensure that all stakeholders have a sufficient understanding of the RROP prior to initial implementation.
With the premise that these outstanding issues will be successfully resolved, the staff concludes that the RROP will be ready for the proposed initial implementation on April 2, 2000, for all commercial operating power plants.
On January 8, 1999, the staff issued SECY-99-007, "Recommendations for Reactor Oversight Process Improvements," forwarding the staff's recommendations for a revised reactor oversight process for commercial nuclear power plants. These recommendations consisted of a framework for regulatory oversight that established seven cornerstones of safety. Fundamental to this concept was that licensee performance that met the objectives and key attributes of each of these cornerstones would provide reasonable assurance that public health and safety was maintained.
In the RROP, licensee performance within each cornerstone is measured by a combination of performance indicators (PIs) and inspection results. PIs were developed for each of the cornerstones to provide an objective indication of licensee performance. A risk-informed baseline inspection program was developed to both independently verify the PIs and to inspect those aspects of licensee performance not adequately covered by a PI. The risk-informed baseline inspection program established the minimum inspection effort that all licensees would receive, regardless of their performance.
Risk-informed thresholds were developed for both the PIs and inspection findings to establish performance bands. These performance bands provide for increased regulatory action as licensee performance degrades, as indicated by crossing more risk significant thresholds. A key aspect of using performance thresholds is that it establishes a level of licensee performance that does not warrant additional NRC involvement beyond the baseline inspection program.
The assessment process was redesigned to be more streamlined and objective by using the PIs and inspection findings as assessment inputs and applying an Action Matrix to determine the appropriate follow-up to indications of degrading licensee performance. The enforcement process was also revised to be better integrated and consistent with the inspection program and assessment process.
On January 20, 1999, the staff briefed the Commission on the scope and concepts of the RROP as described in SECY-99-007. On March 22, 1999, the staff issued SECY-99-007A, "Recommendations for Reactor Oversight Process Improvements (Follow-up to SECY-99-007)," which forwarded to the Commission additional information on the concepts for the RROP, and presented the staff's plans for a 6-month pilot of the revised oversight processes at two sites per region. On June 18, 1999, the SRM on SECY-99-007 and SECY-99-007A was issued which approved the scope and concepts for the RROP, approved the staff plan for the pilot program, and provided additional issues and Commission comments for the staff to consider when evaluating the pilot program results.
The 6-month pilot program for the RROP was conducted at two sites per region from May 30, 1999, to November 27, 1999. The pilot program was conducted in accordance with the guidelines and procedures forwarded by memorandum from the Director, Office of Nuclear Reactor Regulation (NRR) to the four Regional Administrators (RAs), dated May 20, 1999. The sites participating in the pilot program were:
| Region I | Region II | Region III | Region IV |
| Salem/Hope Creek | Shearon Harris | Prairie Island | Fort Calhoun |
| FitzPatrick | Sequoyah | Quad Cities | Cooper |
The purpose of the pilot program was to apply the RROP and identify lessons learned so that the various processes and procedures could be refined and revised as necessary prior to initial implementation. The objectives of the pilot program were: (1) to exercise the various components of the RROP to evaluate whether or not they could function efficiently, (2) to identify significant process and procedure problems and make appropriate changes prior to initial implementation, and (3) to the extent possible, evaluate the effectiveness of the new process. Pilot program criteria were established to evaluate the results of implementing the RROP at the pilot plants.
In addition to evaluating the new process against these pilot program criteria, the staff employed a number of methods to obtain internal and external stakeholder feedback and comments during the pilot program. The evaluation and disposition of this feedback is discussed in detail in this Commission paper under the pilot results for each component of the RROP.
Internal feedback and comments from NRC staff were obtained using various methods. Weekly teleconferences were held with regional management and biweekly teleconferences with the pilot program resident inspectors to solicit feedback. Monthly counterpart meetings were held with the regional Division Directors and Executive Forum meetings were periodically conducted with the four Deputy Regional Administrators to solicit feedback and comments on the RROP. Inspection procedure and oversight process feedback forms were developed and used during the pilot program for regional staff to document questions and concerns on the various components of the RROP. Comments from these feedback forms were utilized by the staff in making needed modifications to procedures as the pilot program progressed. Finally, an internal stakeholder survey of the RAs and staff who participated in the pilot program was conducted at the end of the pilot to gather additional insights to be considered while evaluating the pilot program lessons learned.
The results of the surveys indicated that the regional staff and RAs considered the revised program to be an improved oversight approach. Among the areas commented on by the RAs as needing additional monitoring or improvement were the licensee's corrective action program, cross-cutting issues, the significance determination process (SDP), the resources needed to implement the new program, and communication and internal program management issues leading toward acceptance and implementation of the revised process. Among the concerns expressed by the regional staff were a need for clarification of definitions of some PIs and their thresholds; the limited opportunity to fully utilize the SDP during the pilot program; and disagreement as to the ability of the revised process to allow identification of declining safety performance before significant reductions in safety margins occurred.
The staff is mindful of this latter concern and recognizes that a lack of experience in dealing with problem plants using the RROP has contributed to the above noted skepticism. In developing the RROP, the staff has used a graded approach to establish risk-informed thresholds, which are intended to provide reasonable assurance that safety margins are being maintained, and allow sufficient time for both the NRC and licensees to address noted performance deficiencies before there is an undue risk to public health and safety. The staff believes the appropriate fundamentals are in place and expects that this concern will be alleviated over time. The results of the regional survey are discussed in more detail in Attachment 1.
Public comment was solicited on the RROP and the results of the pilot program by a Federal Register notice (FRN) dated July 26, 1999. The FRN established a public comment period that ended on December 31, 1999, and included a questionnaire to focus public comment on specific topics. This questionnaire requested comment and feedback on the RROP's ability to meet the four agency outcome measures, and also requested feedback and comments on topics such as the role of positive inspection findings in the RROP and the need to develop overall assessment ratings for power plants. A summary of these comments can be found in Attachment 1. The State of New Jersey had significant involvement in the pilot program and had many specific comments. Attachment 1 contains a special section which addresses these comments.
To keep local public stakeholders informed of the new oversight process, public meetings were held in the vicinity of each pilot plant. Public meetings were first held at the beginning of the pilot program, and then a series of Public Roundtable meetings were conducted at the end of the pilot program. These meetings were designed to both explain the new program, and then solicit feedback from the public on their views of the RROP. A summary of the results of these public meetings is included in Attachment 1.
Finally, a pilot program evaluation panel (PPEP) was established by the agency in accordance with Federal Advisory Committees Act (FACA) requirements to serve as an independent advisory committee to the agency. This panel was a cross-disciplinary group of managers and industry experts representing many different nuclear power interests, including the Union of Concerned Scientists, the Nuclear Energy Institute, pilot plant licensee management, and the Illinois Department of Nuclear Safety, in addition to NRC Headquarters and regional management. The purpose of the PPEP was to independently evaluate the results of the pilot program and draw conclusions regarding required process changes and the readiness for initial implementation. The results of the PPEP review are included as Attachment 2 to this Commission paper.
Staff response to the PPEP conclusions and recommendations are included as Attachment 3. Overall the PPEP made several recommendations for action prior to initial implementation, including: (1) develop the process for handling inaccuracies in performance indicator data reported and resolve how the requirements of 10 CFR 50.9 will apply, (2) ensure that the thresholds for the different significance determination processes are consistent in safety impact, and (3) develop a clear and formally documented procedure for handling those rare instances when deviation from the Action Matrix is warranted. As described in this Commission paper, these PPEP recommended actions have been considered and addressed by the staff in conjunction with the pilot program results and lessons learned.
Culminating the feedback activities, the staff conducted a public lessons learned workshop from January 10-13, 2000. The purpose of the workshop was to bring internal and external stakeholders together to identify lessons learned and approaches to resolving key issues of concern. The workshop was successful in enabling the staff to achieve a good level of concensus on those issues requiring action prior to initial implementation, longer-term resolution, and continued monitoring during initial implementation.
The following discussion provides the pilot program criteria results and lessons learned, significant stakeholder feedback on the RROP, and process changes resulting from the pilot effort. The discussion also addresses the status of remaining work to support initial implementation of the RROP, a discussion of the staff's plans to transition from the current oversight process, and the staff's recommendation for the initial implementation of the RROP at all nuclear power plants.
In addition to exercising each component of the RROP, the pilot program also provided for a practical evaluation of the concepts for the new regulatory oversight framework and the cornerstones of safety. The results of the pilot program indicated that the cornerstone concept was fundamentally sound and that the attributes of the seven cornerstones established a framework that could provide reasonable assurance that public health and safety will be maintained. The establishment of performance bands for performance indicators and inspection findings allowed agency and industry resources to be focused on those issues with the most risk significance.
The results of reporting PIs during the pilot program are described in detail in Attachment 4. During the pilot program, all PIs were able to be reported on time, within the required 14 days for each of the pilot plants. However, several problems were noted during the pilot program with accurately reporting the PIs in accordance with the industry guideline document, NEI 99-02, "Regulatory Assessment Performance Indicator Guideline." Most of these errors were minor in nature and were largely attributed to the difficulty in collecting and reporting historical data and problems with ambiguous definitions and clarifying instructions. There were, however, some pilot plants where errors in reporting were substantial and continued for some time. Because these problems could occur at other plants, the staff will be looking for similar problems during initial implementation and will promptly take the appropriate corrective actions. Many changes were made to the industry guideline document during the pilot program and the accuracy of the data submitted for the PIs improved significantly by the end of the pilot.
Significant stakeholder feedback and comments were received on PI reporting, including: (1) several PI definitions, calculation methods, and descriptions need to be clarified (e.g., Security Equipment Performance Index), (2) several PI thresholds do not appear appropriate (e.g., Reactor Coolant System Specific Activity), (3) additional guidance for PI data reporting needs to be provided (e.g., when a PI should be considered invalid), (4) the process for resolving PI interpretation issues needs to be formalized, (5) the process for changing, adding, or deleting PIs needs to be developed, (6) the PI reporting period should be extended beyond 14 days from the end of the quarter to ensure accurate data reporting, and (7) the definitions for PIs need to be made more consistent across the industry (e.g., NEI 99-02, World Association of Nuclear Operators (WANO), Maintenance Rule).
Several changes were made to the NEI 99-02 PI reporting guidance, both during the pilot program and in response to feedback received afterward, including: (1) clarifying the guidance for reporting unplanned power changes, (2) clarifying how fault exposure time will be accounted for in the Safety System Unavailability PI, (3) deleting the Containment Leakage PI, and (4) extending the reporting period from 14 to 21 days.
Several items remain to be addressed prior to the proposed initial implementation of the RROP on April 2, 2000, including: (1) develop and implement a more formal process for resolving PI interpretation issues, (2) reassess the guidance, definitions, and thresholds for several PIs based on the January 2000 historical data submittal, including the Security Equipment Performance Index PI, (3) develop guidance for identifying invalid PIs, such as those in which the NRC has lost confidence that the licensee is reporting accurately, and (4) issue a Temporary Instruction for inspecting licensee processes for collecting and reporting PI data.
Additional performance indicator issues have been identified by the staff for resolution, but are not significant enough to impact initial implementation of the RROP. The staff will work to resolve these issues, and others identified during the first year of implementation. Resolution of these issues will be discussed in the staff's report to the Commission on the self-assessment of the first year of implementing the RROP, due in June 2001.
These longer term issues include: (1) improving the guidance, definition, and thresholds for several PIs, such as Reactor Coolant System Specific Activity, (2) developing the formal process for changing, adding, or deleting performance indicators, and (3) working to develop more consistency between NEI 99-02 PI definitions and WANO indicator definitions.
In addition, the staff continues to work with the public and industry on other improvements to the indicators that will: (1) will provide more meaningful data, thus enhancing NRC effectiveness and efficiency as well as public confidence, and (2) will be easier to understand and simpler to implement, thereby reducing unnecessary regulatory burden, while continuing to maintain safety. The staff will report to the Commission on these additional changes at the end of the first year of implementation.
The NRC's revised inspection program for commercial nuclear power reactors consists of several elements: risk-informed baseline inspections, supplemental inspections for indicated declines in performance, and when necessary, inspections to address generic safety issues. In addition to these elements, the inspection program includes guidance for responding to events and for overseeing plants in extended shutdowns to address performance issues.
As described in Attachment 5, the pilot results demonstrated that baseline inspections could be planned and an inspection schedule issued in a timely manner. Most of the baseline inspection procedures were clearly written, were appropriately risk-informed, and inspectors were able to satisfy the inspection objectives by completing the inspection procedures. In most cases, the scope and frequency of the baseline inspection procedures were adequate to meet the key attributes identified for each cornerstone. Inspection reports were issued, and the plant issues matrices (PIMs) were able to be updated in a timely manner to document inspection findings. The pilot data could not support definitive conclusions on inspection resource requirements. At least a year's experience with the RROP at all power plants will be needed before the staff can provide more definitive information on the resource implications.
The supplemental inspection program was exercised twice during the pilot program in response to single white inputs, with generally positive stakeholder feedback received on its use. While the entire baseline inspection program and some elements of the supplemental inspection procedures were exercised during the pilot program, the event response and extended shutdown procedures were still under development by the end of the pilot program. As described later, a feasibility review was performed near the end of the pilot to evaluate the event response and other oversight procedures that were not exercised during the pilot program.
Significant stakeholder feedback and comments received on the inspection program include: (1) the resource estimates for many of the individual inspection procedures were too low, (2) the scope and frequency of many inspections should account for site-specific differences, (3) the program should more clearly define the role of cross-cutting issues, such as human performance and problem identification and resolution, in the oversight process and determine the proper documentation of cross-cutting issues in inspection reports, and (4) the threshold for documenting inspection observations and findings needs to be clarified to avoid documenting insignificant issues, yet allow the documentation of issues that could potentially lead to more significant programmatic problems.
The staff evaluated all of the comments received and revised most of the inspection program documents and procedures to incorporate this feedback. In some cases, inspectable areas were combined into other procedures to put the inspection objectives into a better context and provide the appropriate emphasis. Adjustments were made to the scope, frequency, and resource estimates for some of the inspection procedures as they were revised. Using pilot program lessons learned, a resource planning meeting was held in mid-February 2000 with the regional Division Directors and the program office to further adjust resource estimates. Based on comments received, the guidance for inspection reports was changed to allow inspectors to document pertinent observations that relate to important cross-cutting areas, but that do not readily lend themselves to evaluation through the significance determination process.
To support initial implementation of the RROP, the staff is developing a process for responding to those occasions when the results of a PI verification inspection determine that the NRC has lost confidence in a licensee's ability to report PI data accurately or completely. This could include supplemental inspection above the baseline to compensate for the missing data and insights intended to be provided by the PIs in question. As previously discussed, the staff is developing an inspection Temporary Instruction to verify licensees' processes for reporting PIs.
Additional experience with implementing specific portions of the inspection program will be required during the first year or more following initial implementation to collect and evaluate lessons learned. The experience is needed to gain further insights into the supplemental and event response portions of the inspection program and into the extended shutdown oversight process because they were not fully executed as part of the pilot program. The staff will collect additional feedback and lessons learned on how cross-cutting issues are addressed by the inspection program. Changes to the program will be evaluated and incorporated to address any lessons learned during the first year of initial implementation.
Also during the first year of initial implementation, the staff will continue to work on improvements to the content of the inspection program to ensure that it provides adequate information to assess the key attributes of the cornerstones. The staff will continue validating and resolving issues, including feedback on the time estimates for completing a number of the baseline inspections. A full year of implementation beyond the pilot program is planned to obtain reliable data on which to estimate the amount of resources needed to accomplish individual inspections as well as execute the overall inspection program.
The results of the pilot application of the SDP are described in detail in Attachment 6. In summary, three inspection findings identified during the pilot program were characterized as "white" (low to moderate risk significance) by the SDP and received preliminary concurrence by the SDP Oversight Panel. However, two of these issues could not be completely resolved within the pilot program timeliness criteria of 120 days from issue discovery. The time involved with resolving the technical issue, documenting and issuing the inspection report, and allowing due-process for the licensee made it difficult to resolve these issues in a timely manner. An independent review of the plant-specific reactor safety SDP characterizations of inspection findings did not identify any risk significant inspection findings that were inappropriately characterized as "green" (very low risk significance).
Significant stakeholder comments and feedback on the pilot application of the SDP recommended that the staff: (1) determine whether the plant-specific reactor
safety SDP characterization should consider the entire plant configuration including equipment out of service for routine maintenance or testing (conditional
core damage probability [CCDP]), or be based on an increase in core damage frequency (
CDF) which already considers routine maintenance and testing
unavailability, (2) improve the consistency of SDP entry conditions for inspection findings, (3) improve the guidance for conducting plant-specific reactor
safety SDP Phase 3 analyses and consider an appeal process for licensees, (4) reduce the complexity and improve the usability of the fire protection SDP, and
(5) improve the degree to which the safeguards SDP is risk-informed.
Early in the pilot program, concerns were raised regarding the use of the change in annualized core damage frequency (CDF) as the appropriate risk measure
for assigning significance to inspection findings within the reactor safety cornerstones, as opposed to the use of CCDP. Based on further review and evaluation
during the pilot program, the staff concluded that CDF is the appropriate risk measure for inspection findings to support the assessment and enforcement
processes. A complete discussion of the staff's approach and its basis is given in Attachment 6. In addition, the staff considers CCDP to be a potentially useful
risk metric to help risk-inform our decisions to initiate rapid inspection responses to reactor events or degraded conditions, as discussed further in Attachment 7.
When the pilot program started, it was noted that SDPs for containment, fire protection, and shutdown activities needed to be developed. The staff completed the initial development of the fire protection SDP, which was exercised during the pilot period and lessons learned were collected. These lessons learned and other stakeholder comments will be used to refine this SDP prior to initial implementation. The staff is also working with its stakeholders to develop a screening tool for inspection findings that affect shutdown safety and the development of an SDP for inspection findings that affect containment barrier integrity, prior to initial implementation. During the pilot program, several inspection findings involving shutdown conditions or the containment were forwarded to NRC Headquarters for significance characterization by risk analysts.
The plant-specific reactor safety SDP is a three step process that is intended to provide inspectors with a relatively simple, conservative process to assign a risk characterization to inspection findings in the initiating events, mitigation systems, and barrier integrity cornerstones. This presents unique challenges in establishing a process that is not overly complex or purely risk-based, but maintains a risk-informed perspective with an appropriate degree of conservatism. The reactor safety SDP process allows risk analysts to directly apply licensee's PRA insights in establishing the risk significance of inspection findings. Therefore, the current SDP process includes some element of risk-based decision making. Several issues exist within the plant-specific reactor safety SDP that challenge the staff in just this manner.
The first of these issues is fundamental to the use of any probabilistic methodology to inform a decision process. This issue is the adequacy (i.e., accuracy) of the assumptions that define the model of the plant and its response to various initiating events. The plant-specific reactor safety SDP helps to address such uncertainty issues by revealing the major assumptions that combine within the probabilistic framework to render each significance determination. This offers the opportunity for inspectors, NRC management, and other stakeholders to either accept or challenge these assumptions. It further allows the decision-maker to examine the sensitivity of the result to the various assumptions that are made. It is through this process of applying a probabilistic model that decision-makers gain risk insights.
The plant-specific reactor safety SDP was developed to address the significance of an issue as to its impact on CDF and large early release frequency (LERF). The potential impact of external events as evaluated by licensee individual plant examination for external events (IPEEEs) are not currently reflected in these SDPs. For some plants where the influence of external events on overall risk is notable, this will introduce non-conservatism into the SDP analysis for initiating event frequency. For initial implementation at all plants, the staff will develop a screening tool to help identify those issues that should receive further evaluation specifically to account for the increased risk contribution from external initiating events. This tool will be evaluated during initial implementation. Further efforts may be required to refine the SDP to account for the impact of external events.
During the pilot program, comparing plant-specific reactor safety SDP Phase 2 worksheets to licensee PRAs identified that the SDPs underestimated risk in some instances due to the omission of certain core damage sequences associated with support system initiating events such as failure of AC/DC control power or component cooling water. The information needed to complete the development of the SDP Phase 2 worksheets, which provide plant-specific information necessary to characterize risk changes, needs to be collected during a site visit. The staff plans to visit each plant site to discuss the accuracy of the plant-specific SDP Phase 2 worksheets with the licensee. The objective of these visits is to ensure that the individual plant examination (IPE) based SDP worksheets are modified as needed to include any plant changes since the IPE was written or any new risk insights generated by the licensee's most current risk analysis. These site visits, which are currently in progress, will not be completed until approximately a month or so after the proposed initial implementation date. The staff believes that initial implementation of the RROP can proceed while the plant-specific reactor safety SDPs are completed. Until the Phase 2 worksheets are finalized for a plant, which takes several weeks following the site visit, the Phase 1 screening tool will be used to identify potentially risk-significant inspection findings for a Phase 3 analysis by a risk analyst. Based on the results of the pilot program, it is expected that only a handful of Phase 3 analyses will be required for those sites without finalized Phase 2 worksheets in place when initial implementation commences.
It is expected that there will also be a number of questions regarding the appropriate use and limitations of the SDP during the early stages of initial implementation. To alleviate the potential resource impact on the Senior Reactor Analyst (SRA) workload following initial implementation, the Operations Support Team in the Probabilistic Safety Assessment Branch of NRR will assist the NRC regional offices in conducting the necessary risk assessments.
Additionally, to support initial implementation, the staff is working to clarify the guidance to: (1) ensure that the entry conditions for each SDP are consistent, (2) ensure that inspection findings characterized by the same color have similar degrees of importance both within each cornerstone and throughout all cornerstone SDPs, and (3) better define the plant-specific reactor safety SDP Phase 3 process to ensure it can be completed in a timely manner.
The staff will continue to evaluate the efficacy of the various SDPs and will continue to evaluate NRC risk analyst staffing needs throughout the first year of implementation. It is anticipated that additional risk analyst expertise will be needed to augment the regional SRAs on a long-term basis. The staff is working with the regions and the Office of Human Resources to identify an appropriate approach to augment risk analyst expertise in the regions.
The results of piloting the new enforcement policy for the RROP are documented in Attachment 8. During the pilot program, all compliance issues assessed as "green" (very low risk significance) were documented as non-cited violations in issued inspection reports. Regulatory conferences were held with the licensees and Notices of Violation were issued for those issues identified as "white" (low to moderate risk significance) during the pilot program. These enforcement outcomes were consistent with the SDP and enforcement policy guidance.
Significant stakeholder comments and feedback on enforcement included: (1) determine the applicability of 10 CFR 50.9 to any errors and inaccuracies associated with the reporting of PI data, and (2) agency oversight panels such as those for SDP, Maintenance Rule and 10 CFR 50.59 need to be better integrated.
Prior to initial implementation, the staff will resolve the concern of how the requirements of 10 CFR 50.9 will be applied to PI data reporting. Specifically, the staff needs to establish the thresholds to determine the degree of enforcement action that will be taken. The resolution to this concern will likely have an impact on the level of licensee management overview that is given to PI data prior to submittal to the agency, and could have an impact on the timeliness of PI data reporting. In establishing a 21-day period for PI data reporting, this issue was taken into consideration.
Based on the experience gained during the pilot program, a better integration with current panels discussing regional issues is needed. As a result, NRR and the Office of Enforcement have started to integrate the panels into the existing weekly regional panel schedule and an individual in the region and in headquarters will be assigned responsibility for each issue. This integration will ensure that panels are appropriately scheduled, that documentation needed for discussion at the panels is available and that personnel are not required at different meetings being held simultaneously.
The pilot program demonstrated that the assessment process and Action Matrix could be used to take appropriate actions in response to indications of licensee performance. During the pilot program, several different PIs and three inspection findings were reported as crossing the green/white threshold. The NRC response dictated by the Action Matrix was appropriate to address these performance issues, and follow-up action was taken in a consistent manner. However, since the pilot program was only able to exercise the first two columns of the Action Matrix, a feasibility review was performed near the end of the pilot to allow for further evaluation of additional aspects of the assessment process using actual, non-pilot plant events. It was noted early in the pilot program that the originally proposed time line for completing the mid-cycle and end-of-cycle reviews could not be implemented in a practical manner by the regional staff. The time line for conducting these assessment meetings and issuing the assessment letters (with an updated inspection plan) was therefore extended from four to eight weeks from the end of the quarter. Assessment process pilot results and lessons learned are described in more detail in Attachment 9.
Significant stakeholder comments and feedback received on the assessment process include: (1) there needs to be clear program guidance for those instances when deviation from the Action Matrix is necessary, and (2) there needs to be a methodology for addressing "substantial" cross-cutting issues in the assessment process.
In response to this feedback, the assessment procedure was revised to provide additional guidance on the process for approving and documenting deviations from the Action Matrix which are expected to occur in a few rare instances. The treatment of cross-cutting issues in the assessment process was discussed at length with various stakeholders, and no clear consensus could be reached. One opinion was that the NRC should not assess cross-cutting issues in the absence of other issues outside of the licensee response band, as stated in the original tenets of the framework. Another opinion was that the NRC should inspect and assess cross-cutting issues which, individually, do not rise to the high level of risk associated with the SDP but which, collectively, could reveal important insights into the licensee's programs.
For initial implementation, stakeholders agreed during the lessons learned workshop that the assessment process should be modified to allow regional offices to provide a qualitative assessment of the effectiveness of the licensee's corrective action program and a discussion of any other substantial cross-cutting issues in the mid-cycle and annual assessment letters. However, NRC action would not be taken for these cross-cutting concerns absent a performance indicator or inspection finding that has at least entered the "white" (low to moderate risk significance) band. During the first year of implementation, the staff will establish a working group to continue to evaluate the role of cross-cutting issues in the assessment process.
Certain aspects of the assessment process, such as the end-of-cycle assessment and agency action review meeting, were not exercised during the pilot program. However, these portions of the assessment process will be exercised at the pilot plants during May 2000. Lessons learned from the pilot plants will be incorporated into the procedure guidance prior to conducting these portions of the assessment process at all plants in May 2001.
During the pilot program, the staff generally demonstrated that the agency's information management systems can support the RROP, though some key milestones still need to be met to support the proposed initial implementation in April 2000. PIs were successfully submitted by each of the pilot plants and received by the NRC, were first made available on the external Web in July 1999, and were successfully updated monthly throughout the pilot program. Inspection findings, including the PIM and the inspection reports, were first made available on the external Web in mid-October 1999, and were also updated throughout the remainder of the pilot. The initial posting of inspection findings on the Internet was delayed during the pilot program due in part to the inherent time delay in issuing inspection reports and updating the PIM, and the need for the program office to provide additional guidance and review PIM entries to ensure quality and consistency.
New Regulatory Information Tracking System (RITS) codes were established and successfully used throughout the pilot for time reporting and budget purposes. Portions of the Reactor Program System (RPS) and its associated modules were modified and successfully used during the pilot. In spite of these successes, the agency's data submittal and processing systems were not able to be fully tested during the pilot since the systems to be used for initial implementation will differ from those used during the pilot program. Additional detail regarding the pilot of the NRC's information management systems can be found in Attachment 10.
Significant stakeholder comments on the agency's systems for handling oversight process information included: (1) the posting of assessment information on the Internet has increased the availability of more objective and scrutable information on licensee performance, and (2) all inspection reports, not just those with issues that resulted in findings in the PIM, should be posted on the assessment Web page to provide a more balanced perspective of licensee performance.
The staff has identified several key information management system issues and milestones that need to be met to support the proposed initial implementation in April 2000, including: (1) the protocol for the quarterly PI submittals beginning in April 2000 still needs to be established and included in program documents, (2) trial runs of the first quarterly submittals need to be performed prior to the actual submittals to identify and resolve any problems ahead of time, (3) the overall design of the Web page needs to be revised to include the addition of all inspection reports, assessment letters, and inspection plans, (4) the new RITS activity codes need to be made available and their proper use verified, and (5) several minor modifications and enhancements to RPS need to be completed and the development of a new module in RPS to capture the PI data, compute the PIs, and post the information to the Web needs to be completed.
Certain aspects of the RROP, such as the event response procedures, were developed during the pilot program and were not fully implemented in the field. Therefore, the staff conducted a feasibility review near the end of the pilot program to exercise the event response procedures and those portions of the RROP designed to address significant licensee performance issues. This feasibility review included a table top test of the event response process using risk significant non-pilot plant events. The exercise demonstrated the ability of the process to:
Additionally, it was documented that cross-cutting issues such as human performance and corrective action program performance could be used effectively to adjust agency response within the appropriate Action Matrix column. Details of the results and conclusions of the feasibility review are discussed more fully in Attachment 7.
The staff has developed a plan to transition all operating nuclear power plants from the current oversight processes to the RROP. Pending Commission approval, initial implementation of the RROP for all power plants is scheduled to occur on April 2, 2000. During the first 12 months of initial implementation, the staff will continue to perform program self-assessments to collect additional lessons learned and gain insights from the new oversight process. Prior to initial implementation, the staff will develop a more detailed transition plan to describe the change management strategies and program oversight necessary to support initial implementation. As part of this transition plan, criteria will be established by the staff to evaluate the results of initial implementation and identify additional lessons learned. The results of the initial implementation self-assessment, lessons learned, and oversight process changes are expected to be presented to the Commission in June 2001. The schedule of activities to support the transition of all power plants to the RROP is included as Attachment 11.
The nine pilot plants have remained in the RROP following the completion of the pilot program in November 1999. For the non-pilot plants, the last Plant Performance Reviews (PPRs) under the current assessment process will be performed in late February and early March 2000. These PPRs will be conducted in a manner to help transition these plants to the RROP by assessing licensee performance using the historical PI data submittal from January 2000 in addition to the inspection record. The PPR letters to the non-pilot plant licensees will forward an inspection plan that implements the revised inspection program starting in April 2000. Those non-pilot plants whose performance warrants agency-level attention will be evaluated under the Senior Management Meeting (SMM) process following the PPRs. With the exception of D.C. Cook, the non-pilot plants will then join the pilot plants in the new inspection and assessment cycles starting in April 2000. The staff intends to keep the D.C. Cook plant under the current oversight process (it is currently under the Inspection Manual Chapter 0350 oversight process) until licensee corrective actions have been completed to the extent necessary to support the transition to the RROP. The staff has recently met with D.C. Cook management to discuss the transition of the plant to the RROP. Additional detail on the staff's plan to coordinate the transition to the RROP with the current oversight processes is described in Attachment 12.
A number of training sessions have been scheduled by the Technical Training Division to prepare NRC staff for initial implementation, with several of these training sessions having already been completed. Additionally, public workshops will be conducted in each region from late February through March 2000 to provide an overview of the RROP to the public and industry. Staff preparations for initial implementation of the RROP are discussed in more detail in Attachment 13.
All procedures and program documents for the new oversight processes are expected to be issued to support the proposed initial implementation date of April 2, 2000. Most of these procedures were distributed for regional and program office review in January 2000. Comments are currently being reviewed and incorporated, with all procedures and program documents, with some minor exceptions, expected to be ready for issuance by early March 2000.
Commission paper SECY-99-007 described several key policy issues that needed to be more fully considered and addressed to support the implementation of the RROP. These key policy issues, and their status, are as follows.
Changes to the reactor oversight process need to be closely coordinated with the agency initiative to risk-inform 10 CFR Part 50. The RROP increases focus on certain risk-significant requirements and decreases focus on certain other less risk significant requirements. This could result in situations where very low significance findings, even if numerous, would be evaluated and treated with little to no follow-up by the agency. Similarly, the RROP may identify risk significant conditions, not covered by current regulations, that warrant some level of agency follow-up beyond the baseline inspection program. Risk-informing 10 CFR Part 50 may provide further insights into aspects of our current regulations that may impact the inspection program. The staff will continue to monitor activities in this area and ensure that the RROP is consistent with the results of this effort.
The staff's approach to event assessment and response needs to be consistent with the fundamental concepts of the RROP. As described in Attachment 5, the agency event response procedures have been reviewed and revised to be consistent with the risk-informed approach of the RROP.
The RROP may have a long-term impact on the N+1 policy for resident inspector staffing due to the potential for improved efficiency and changes in the division of responsibility for implementing the new processes. The staff's proposal for addressing the N+1 resident inspector staffing policy under the RROP was forwarded to the Commission by SECY-99-227, "N+1 Resident Inspector Staffing Policy," dated September 13, 1999. By SRM dated January 11, 2000, the Commission approved the staff's recommendation to modify the policy to require N resident inspectors at dual and triple-unit plants, but maintain N+1 resident inspector staffing at single unit plants. This change in resident inspector staffing policy does directly reflect a reduction in inspection resources applied to a power plant, but is intended to provide the regions with more flexibility in assigning resources as required by the RROP. The staff will continue to evaluate the resource requirements of the RROP during the first year of implementation. The staff will provide recommendations on resident inspector staffing as part of the report on initial implementation of the RROP, due to the Commission in June 2001.
The regional and Headquarters organizational structures may need to be changed to support the cornerstone framework and the RROP. While sufficient data were not developed during the pilot program to draw any conclusions, the staff will continue to assess this issue during the first year of initial implementation to determine whether any agency organizational structure and staffing changes should be made to better support the RROP. Two areas where the staff has focused their attention for regional resources for performing the baseline inspections are in engineering area and fire protection. The baseline inspection program calls for engineering inspection in part because this area does not lend itself to performance monitoring through performance indicators and experience indicates that this is an area that we should continue to monitor through the inspection program. Headquarters and regional staff have reviewed the skills necessary for these inspections and have developed a plan to ensure the regions have the necessary expertise to conduct these inspections. Fire protection triennial inspections require a breadth of fire protection knowledge that does not currently exist in all regions. The staff has developed a plan to provide the regions with training in this area to support the baseline inspection program. Finally, the staff continues to evaluate the adequacy of risk expertise in the region and review the options for training additional people in the mechanics of probabilistic risk assessments.
In addition to these four previously identified issues, the staff has recognized several other agency coordination issues related to the development of the RROP. The staff has recognized that the implementation of the RROP will have an impact on the conduct of the agency's allegation program. Four options for modifying the allegation program were forwarded by the staff in Commission paper SECY-99-273, "Impact of Changes to the Inspection Program for Reactors on Implementing the Allegation Program," dated November 23, 1999. As documented in the SRM on SECY-99-273 dated January 27, 2000, the Commission approved the staff's proposal to solicit stakeholder input prior to a final Commission decision on one of the options. The staff will continue to work closely with the Agency Allegations Advisor to ensure that changes to the allegation program are closely integrated and consistent with the RROP.
The staff is working closely with the Office of Nuclear Regulatory Research (RES) to investigate the feasibility of risk-based performance indicators of component, train, and system performance. This effort is intended to enhance the current set of indicators by providing indicators in areas where there are currently none, such as equipment reliability, shutdown operations, and containment performance. It may also include the use of PI thresholds that are more plant-specific. RES expects to have preliminary results by mid-summer of 2000 that will allow the staff to better define its needs in this area. In addition, NRR and RES are working jointly to develop indicators of industry trends, including risk-significant trends of performance elements that are difficult to measure on a plant-specific basis. Such indicators may be useful in assessing the effectiveness of the RROP and in ensuring the plant safety is being maintained.
The staff is also working to apply many of the RROP concepts developed and lessons learned identified to the oversight of nuclear fuel cycle facilities. The Office of Nuclear Material Safety and Safeguards (NMSS) has forwarded its proposal for evaluating and proposing revisions to the oversight process in Commission paper SECY-99-188, "Evaluation and Proposed Revision of the Nuclear Fuel Cycle Facility Safety Inspection Program," dated July 21, 1999. NRR staff will continue to work closely with NMSS to share information and support its work to develop a more risk-informed process for the regulatory oversight of nuclear fuel cycle facilities.
As described throughout this Commission paper, the staff has worked closely with all stakeholders to resolve pilot program comments and lessons learned. For the most part, this effort has resulted in a solution for each issue that appropriately considers the majority of stakeholder feedback or concern. However, three issues emerged that have a significant impact on the implementation of the RROP, and for which the staff has not been able to resolve in a manner that adequately addresses the majority of stakeholder concerns. The staff committed to present these issues, the differing points of view, and the staff proposal for resolution to the Commission for consideration. These issues are as follows:
Based on the results of the 6-month pilot program, the staff has concluded that the cornerstones of safety concept and the associated framework is sound. Pilot program feedback received by the staff, from both internal and external stakeholders, indicates that further experience with the process is needed. Implementing the RROP at all sites will enable the staff to acquire further experience and provide it the opportunity to identify additional lessons learned and gain greater confidence in the efficacy of the RROP.
The pilot program increased staff confidence that the combination of PIs and inspection findings can provide appropriate indications of licensee performance. The graded approach used in establishing risk-informed thresholds are intended to assure that safety margins are being maintained and that sufficient time would exist for both the NRC and licensees to address noted performance deficiencies before there was an undue risk to public health and safety. The appropriate implementation of these processes should provide reasonable assurance that safe plant operation is maintained. Initial implementation of the RROP at all sites will enable the staff to confirm the capability of the RROP to identify declining safety performance trends in a timely manner, and to determine if threshold adjustments are warranted. During the pilot program, no significant aspects of licensee performance related to the cornerstones of safety were identified that were not adequately covered through the combination of PIs and inspection. Additional experience and insights gathered through the implementation of the RROP at all sites will permit more extensive execution of all aspects of the process and generate greater confidence in the appropriateness of the RROP.
Most internal and external stakeholders agreed that the RROP provides assessments of licensee performance, with corresponding NRC actions, in a manner that is more objective, understandable, and predictable than the current oversight process. The submittal of PIs by the licensees provided performance data that were more timely and relevant. Information was more readily accessible to the public through the posting of the PIs and inspection findings on the NRC's Internet Web page. The bases for NRC actions were more easily understood through the use of the assessment Action Matrix.
Sufficient data were not able to be generated during the pilot program to accurately quantify any efficiency changes associated with the RROP. Pilot program results indicated a notable shift in resource expenditure, with more time required for inspection preparation, and less time required for inspection documentation. Overall program resources, including inspection, assessment, enforcement, inspection preparation and documentation time, were about the same, and in some cases slightly more than those required to implement the current oversight process. However, several startup costs associated with the pilot program influenced these results, including the increased effort to prepare for, conduct, and document an inspection for the first time. More substantial data are required before a more accurate evaluation of resource requirements can be accomplished. The staff intends to address the resource implications of the RROP more fully in its report to the Commission on the first year of initial implementation, currently planned for June 2001.
Based on industry and agency feedback, the regulatory burden associated with the RROP appears appropriate. More licensee resources are required to support the data collection and reporting associated with PIs. However, this increase in burden has been more than offset by the changes to the inspection, assessment, and enforcement processes, which have allowed licensees to focus their resources more efficiently on those issues with the greatest safety significance.
As described in Attachment 2, the PPEP concluded that the RROP framework provides a more objective, clear, and risk-informed approach to the oversight of nuclear reactors. The PPEP recommended that the agency proceed to industry-wide initial implementation of the RROP. The PPEP identified several areas that need refinement before industry-wide initial implementation, which have been addressed in this Commission paper.
With some exceptions as noted earlier, the procedures and guidance documents required to support the RROP have been exercised sufficiently during the pilot program to support their use at all power plants. Pilot program lessons learned have been incorporated into the oversight process procedures as appropriate, and these procedures are expected to be issued in time to support initial implementation. Staff training and public workshops will also be completed to support implementation of the RROP. To support initial implementation, the following issues must be resolved by the staff:
Pending the successful completion of these issues, the staff concludes that the RROP is ready for the proposed initial implementation on April 2, 2000, for all commercial operating power plants (except for D. C. Cook as discussed earlier).
Significant stakeholder feedback will need to continue throughout the initial implementation of the RROP at all power plants to capture additional lessons learned and insights on the implementation of the new processes. Additionally, program self-assessments will continue during the first year of initial implementation to gather the additional information necessary to appropriately refine the oversight process.
The development of the RROP and the cornerstones of safety framework have resulted in an oversight process that is more focused on safety significant issues, with a better defined and consistent set of inspection requirements. Additional data needs to be obtained during the first year of implementation to determine any resource changes associated with the RROP. Therefore, it would be premature to make any resource reduction decisions at this time beyond those already documented in the fiscal year (FY) 2000 and FY 2001 budgets. The RROP can be implemented at all operating commercial nuclear power reactors, effective April 2000, with the resources already budgeted for FYs 2000 and 2001. Oversight process resource requirements will be re-evaluated following the first year of initial implementation, and are expected to be reported to the Commission in June 2001.
The SRM for SECY-99-007 and SECY-99-007A, dated June 18, 1999, identified 11 specific issues that the Commission requested the staff to consider along with the RROP pilot program results. These specific issues have been considered by the staff along with the lessons learned from the pilot program in determining oversight process changes or improvements. Some of these issues were discussed in a memorandum from the Executive Director for Operations to the Commission dated August 12, 1999. Many of these issues were incorporated into earlier discussions in this Commission paper on oversight process changes. All of these issues, and how they were addressed by the staff, are discussed in detail in Attachment 14. In particular, the Commission directed the staff to consider the following issues.
The Office of the General Counsel has reviewed this Commission paper and has no legal objections to its content.
The Office of the Chief Information Officer has reviewed this Commission paper for information technology and information management implications and has no objections.
The Office of the Chief Financial Officer has reviewed this Commission paper for resource implications and has no objections.
That the Commission:
| 1. | Approve the initial implementation of the RROP at all operating commercial nuclear power plants (except for the D.C. Cook plant), effective April 2,
2000, as described in the staff's transition plan. | |
| 2. | Approve the termination of the SALP process. | |
| 3. | Specifically consider the three issues discussed in this Commission paper in the section titled "Issues of Note." | |
| 4. | Note: | |
| a. | Unless directed otherwise, the staff will continue with those efforts necessary to support initial implementation (e.g., procedure development and
staff training) as outlined in the transition plan. | |
| b. | By March 28, 2000, the staff will notify the Commission of the status of all outstanding RROP development items identified within this paper as
needed to support initial implementation. | |
| c. | The proposed plan for initial implementation of the RROP is contingent upon the staff receiving a response from the Commission by March 31,
2000. | |
| d. | The Commission provided direction to the staff in SECY-99-007A SRM dated June 18,1999, to commence implementation of the RROP in April
2000. The staff has expended substantial effort to have all necessary elements of the RROP in place to support this implementation date. Of
particular note is the need to develop and issue inspection plans for all reactor plants so that the baseline inspection program can commence in
April 2000. Therefore, consistent with Commission direction, the staff intends to issue these inspection plans to all licensees and initiate the new
baseline inspection program to support the proposed April 2, 2000 start date. | |
| e. | The staff recommends that the D.C. Cook plant not be included in the initial implementation of the RROP at this time. This is due to the licensee's continued efforts to implement corrective actions to address significant performance concerns and the increased agency focus to monitor and evaluate these licensee efforts. Regulatory oversight of the D.C. Cook plant would continue under the current inspection and assessment processes. The staff will develop a separate plan to transition the D.C. Cook plant to the RROP when licensee corrective actions have been completed to the extent necessary to support the transition to the new oversight process. | |
| /RA by Carl J. Paperiello Acting For/
William D. Travers |
| Contact: | William M. Dean, NRR 301-415-1257 |
[For attachments, see PDF 
version of SECY-00-0049]
| Attachments: | 1. Communication and Stakeholder Feedback |
| 2. PPEP Final Report | |
| 3. Staff Response to the PPEP Final Report | |
| 4. Performance Indicators | |
| 5. Inspection Program | |
| 6. Significance Determination Process | |
| 7. Feasibility Review | |
| 8. Enforcement Process | |
| 9. Assessment Process | |
| 10. Information Management Systems | |
| 11. Transition Schedule | |
| 12. Coordination with Current Oversight Processes | |
| 13. Staff Training for Initial Implementation | |
| 14. Staff Response to SECY-99-007A SRM |
|
Privacy Policy |
Site Disclaimer |