| |
Search Options | |||
| Index | Site Map | FAQ | Facility Info | Reading Rm | New | Help | Glossary | Contact Us | |
|||
|
POLICY ISSUE SECY-08-0046 April 2, 2008
The purpose of this paper is to present the results of the staff’s annual self-assessment of the Reactor Oversight Process (ROP) for Calendar Year (CY) 2007. The results of the CY 2007 self-assessment indicated that the ROP met its program goals and achieved its intended outcomes. The staff of the U.S. Nuclear Regulatory Commission (NRC) found the ROP objective, risk informed, understandable, and predictable, and the ROP met the agency goals of ensuring safety, openness, and effectiveness as listed in the NRC’s Strategic Plan for Fiscal Years (FY) 2004 - 2009. NRC staff maintained its focus on stakeholder involvement and continued to improve various aspects of the ROP. The staff implemented several ROP improvements in CY 2007 to address issues raised by the Commission, recommended by independent reviews, and obtained from internal and external stakeholder feedback. The NRC inspection and assessment program independently verified that nuclear power plants were operated safely and securely. During the year the staff made several improvements to the ROP including the timeliness of significance determination process (SDP) results, implementing enhancements to more fully address safety culture and oversight of licensees with performance problems, realigning inspection resources to improve effectiveness, and making changes to some performance indicators (PIs) to better identify declining safety performance. However, the staff recognizes the need for further enhancements to the ROP and will continue to actively solicit input from the NRC’s internal and external stakeholders. For example, the staff plans to explore ways in which substantive cross-cutting issues, traditional enforcement actions, and other insights could be used more effectively in the ROP. On February 24, 2000, the staff issued SECY-00-0049, “Results of the Revised Reactor Oversight Process Pilot Program.” The resulting Staff Requirements Memorandum (SRM), dated March 28, 2000, approved initial implementation of the ROP as recommended by the staff. The initial implementation of the ROP began on April 2, 2000. SECY-01-0114, “Results of the Initial Implementation of the New Reactor Oversight Process,” dated June 25, 2001, noted the staff’s intention to perform an annual self-assessment of the ROP. Accordingly, the staff has issued an ROP self-assessment Commission paper each year before the Agency Action Review Meeting (AARM) and has briefed the Commission on the self-assessment results following the AARM. This paper provides the results of the ROP self-assessment for CY 2007. The staff performed the CY 2007 self-assessment in accordance with Inspection Manual Chapter (IMC) 0307, “Reactor Oversight Process Self-Assessment Program.” The ROP self‑assessment program evaluates the overall effectiveness of the ROP through its success in meeting its pre-established goals and intended outcomes. In accordance with IMC 0307, this self-assessment does not include security and safeguards except where specifically noted. The annual Report to Congress on the Security Inspection Program, however, provides an assessment of the security program. The staff plans to reincorporate the security cornerstone in the ROP self-assessment process in CY 2008. In response to the staff’s briefing on the results of the AARM on May 31, 2007, the Commission directed the staff to take the actions specified in SRM M070531, dated June 14, 2007. In summary, these actions included providing a paper that describes the Baseline Risk Index for Initiating Events (BRIIE) and plans for its use as a new industrywide indicator; expanding the resident inspector (RI) demographics in the next self-assessment report on the ROP; continuing to look for leading PIs and ways to modify or improve the existing indicators; and considering ways to promote senior resident inspectors (SRIs) while still retaining them within the RI program. This paper and its enclosures address each of these items, along with previous commitments and other direction from the Commission. The staff uses program evaluations and performance metrics to determine the effectiveness of the ROP in meeting its seven program goals and intended outcomes. The seven goals include the four program-specific goals of being objective, risk informed, understandable, and predictable, as well as the three applicable performance goals listed in the NRC’s Strategic Plan for FY 2004 - 2009 (ensuring safety, openness, and effectiveness). The staff plans to revise IMC 0307 to reflect the recently issued Strategic Plan for FY 2008 – 2013 (including security), and will perform subsequent annual self-assessments in accordance with the revised guidance. The following intended outcomes of the ROP help form its basis and are incorporated into the various ROP processes:
During the eighth year of ROP implementation (CY 2007), the staff conducted numerous activities and obtained data from many diverse sources to ensure that it performed a comprehensive and robust self-assessment. Data sources included the ROP performance metrics described in IMC 0307, recommendations from independent evaluations, comments from external stakeholders in response to a Federal Register notice, insights from internal stakeholders through the ROP internal feedback process, and feedback received from stakeholders at various meetings, workshops, and conferences. The staff also applied the direction and insight provided by the Commission through several SRMs. The staff analyzed this information to gain insights regarding the effectiveness of the ROP in fulfilling its program goals and intended outcomes. The staff evaluated the key program areas of PIs, inspection, SDP, and assessment, as discussed in the following paragraphs. In addition, the staff assessment included ROP communication activities, ROP self-assessment and independent evaluations, ROP resources, and RI demographics and staffing. As noted in the pertinent sections of this paper, the staff has also included several enclosures with additional detail to support the staff’s assessment and conclusions. ROP Program Area Evaluations The staff performed evaluations in each of
the four key program areas of the ROP - PI, inspection, SDP, and assessment. Enclosure 1
PI Program - The staff continued to improve the PI program in CY 2007 to provide more meaningful indication of declining plant performance and to identify outliers. The Mitigating Systems Performance Index (MSPI) provided a significant input to the ROP Action Matrix; of the 16 new greater-than-green PIs in CY 2007, 10 were from MSPI. The staff and industry are reviewing the lessons learned from the first 2 years of MSPI implementation and will evaluate possible changes. The staff and industry jointly developed the Unplanned Scrams with Complications PI to replace the Unplanned Scrams with Loss of Normal Heat Removal PI. Further, the Personnel Screening Program and Fitness-for-Duty/Personnel Reliability PIs were discontinued because they provided minimal input and reasonable confidence exists through the security baseline inspection program. The staff plans to interact with the industry to explore other PIs that might provide more meaningful input. The staff is also reviewing the effectiveness of the Safety System Functional Failure PI, which had been an excellent indicator of poor and declining performance before the ROP, but has not been since ROP implementation. The staff will continue its efforts to improve the Emergency Preparedness PIs, the Reactor Coolant Leakage PI, and the other PIs, depending on available resources. Two of the eight PI metrics did not meet the established criteria. Specifically, the metric for whether the PIs provide for a timely indication of declining safety performance was missed based on three distinct sites crossing multiple thresholds in a given quarter. Additionally, the metric for the perception as to whether the PI program provides insights to help ensure plant safety was missed as identified by the external survey. As noted above, several significant efforts are currently underway or have recently been completed to improve the PI program. Inspection Program - NRC inspectors independently verified that plants were operated safely, appropriately identified performance issues, and evaluated the adequacy of licensee corrective actions to address the noted performance issues. The staff completed its second ROP realignment review to ensure the most effective overall application of inspection resources. Based on this review, changes were made to about 60% of the baseline inspection procedures, including the restructuring of the engineering inspection process. Through implementation of a new approach for modifications inspections and change in the frequency of the Component Design Basis Inspections (CDBIs), the revised program will consist of one major engineering inspection each year over a 3-year cycle (e.g., modifications, CDBI, fire protection). The staff successfully integrated the operating experience information into the baseline inspection program using the Operating Experience Smart Sample process. The staff conducted an accelerated program to inspect and assess material control and accounting programs at all nuclear power plants and wet storage sites, and completed all the remaining first cycle force-on-force inspections at reactor sites. The regions completed the required baseline inspection program for CY 2007, and all but one of the inspection program metrics were met. The timeliness of temporary instructions (TIs) metric was missed based on the untimely completion of one TI at one site. SDP - During this assessment period, the SDP remained an effective tool for determining the safety significance of identified performance issues. Most notably, the program met the SDP timeliness goal for the second consecutive year. The staff developed several significant enhancements to the SDP guidance, including revamping the initial screening and characterization of findings process, improving the SDP appeal process, and revising the Public Radiation Safety SDP to improve its objectivity. Additionally, the staff revised and made publicly available the Risk Assessment Standardization Project handbook to provide enhanced risk analysis methods and guidance regarding the assessment of licensee probabilistic risk assessment quality. The staff also continued the Security Findings Review Panel (SFRP) for all security findings to ensure regulatory consistency, and developed a comprehensive SFRP database for knowledge management and inspector use. One SDP performance metric - the perception that the SDP provides an objective and understandable regulatory response to performance issues - failed to meet program expectations. To address this concern, the staff has revised the Public Radiation Safety SDP to make it more objective and plans to perform a similar review for the Emergency Preparedness and Security SDPs. Assessment Program - The staff has made several enhancements to the ROP program guidance to more fully address safety culture and the oversight of licensees with performance problems. The staff is compiling lessons learned from the initial 18-month implementation of the enhanced ROP and plans to further enhance the ROP safety culture guidance documents. A key contributor to this effort will be the lessons-learned report resulting from the initial implementation of the revised IP 95003, “Supplemental Inspection for Repetitive Degraded Cornerstones, Multiple Degraded Cornerstones, Multiple Yellow Inputs, or One Red Input,” at the Palo Verde site in 2007. The staff also revised the assessment program to add guidance and clarity regarding when licensee senior management should be requested to meet with the Commission to discuss actions being taken to improve performance and to enhance the NRC’s communication with the public. The staff evaluated the three ROP Action Matrix deviations from CY 2007 for potential program changes and deemed that further evaluation and experience would be needed to conclude whether program changes would be necessary for the one new deviation and had already implemented changes for the two that were renewed. During CY 2007, the staff noted an increase in the number of sites in columns 3 and 4 of the ROP Action Matrix. Although a similar decline in licensee performance was not evident in the current industry trends program (ITP) results, the staff plans to monitor and assess this potential issue in CY 2008. Two of the eight assessment metrics did not meet their established criteria. Specifically, the metric for whether degradations in plant performance were gradual and allow adequate agency engagement of the licensee was missed because there were five occurrences where plants moved more than one column to the right in the Action Matrix. Additionally, the metric for whether the response to performance issues was timely was missed because the elapsed time between issuance of an assessment letter and the respective supplemental inspection exit meeting date had increased over previous years. The staff will further assess the data and engage with internal and external stakeholders to better understand the root causes of these issues. ROP Communication Activities The staff continued to seek and implement improvements to the ROP based on feedback and insights from all stakeholders. The staff used a variety of communication vehicles to ensure that all stakeholders have access to ROP information and results and have an opportunity to participate in the process and provide feedback. The staff continued to conduct monthly public meetings with external stakeholders and conducted a survey of external stakeholders to actively solicit and analyze stakeholder feedback regarding the effectiveness of the ROP. In addition, the staff began issuing public security inspection report cover letters and initiated actions to further assess the level of openness and transparency associated with the security cornerstone. The staff also continued the ongoing internal feedback process, held biweekly telephone conferences as well as frequent meetings with internal stakeholders, and visited each region to give inspection staff and management the opportunity to discuss ROP implementation and provide feedback. In addition, the staff continued to maintain the ROP Web pages to ensure that they remain useful tools for communicating accurate and timely information to all stakeholders. The responses from the survey of external
stakeholders were similar in content to those in previous years, but the number
of responses declined significantly. The
agency received only 7 responses for the CY 2007 survey, down
significantly from 16 in CY 2006 and 21 in CY 2005. Specifically, utility representatives
provided four of the seven responses, while public representatives submitted
two and a State agency provided one. Overall, the utility responses were generally positive, whereas the two
public respondents were less positive and raised specific concerns about the
effectiveness of the ROP. Enclosure 3
ROP Self-Assessment Metrics and Independent Evaluations The objectives and details of the ROP
self-assessment program appear in IMC 0307. This paper, supplemented by the annual report of performance metrics,
provides the results of the staff’s self-assessment for CY 2007.
The staff performed its
annual self-assessment of performance metrics for CY 2007 in accordance with
the recent revision to IMC 0307. Based
on the NRC staff’s review, most of the 48 performance metrics for the ROP met
the established criteria. All 16 metrics
in the “Overall ROP” area met the established criteria; however, two PI program
metrics, one inspection program metric, one SDP metric, and two assessment
program metrics did not. The staff
discusses its corrective actions to address these issues in the metric report
as well as in the program area evaluations in Enclosure 1
In addition to the ROP self-assessment program, several independent evaluations have been performed in the past few years, most notably by the U.S. Government Accountability Office (GAO), Office of Management and Budget, the Office of the Inspector General, and the Davis‑Besse Lessons Learned Task Force (DBLLTF). These evaluations generally provided favorable results, but they also suggested potential areas of improvement. The staff addresses several recommendations from these independent evaluations in the enclosures to this paper. GAO completed an independent evaluation of
the ROP and issued its report on September 27, 2006 (ADAMS Accession No.
ML062720030). The report, entitled
“Nuclear Regulatory Commission: Oversight of Nuclear Power Plant Safety Has Improved, but Refinements
Are Needed (GAO-06-1029),” included three recommendations. The NRC formally responded to the GAO report
on November 27, 2006 (ADAMS Accession No. ML062910527), and provided an
update in its annual status report to GAO on March 12, 2007 (ADAMS Accession
No. ML070400008). The staff is addressing
the GAO recommendations as discussed in further detail in Enclosure 2
The staff also received and evaluated
feedback from licensees as part of the regulatory impact process. The regulatory impact process was established
in 1991, based on Commission direction to develop a process for obtaining
feedback from licensees and reporting the feedback to the Commission. Over the past year, the staff received
feedback from 68 reactor licensees on 139 issues. Of the comments received, 86 percent were
favorable, and 14 percent were unfavorable. The comments fell into two main categories - formal communication with
licensees and inspector performance. Enclosure 4
The NRC also collects and monitors industry-wide data to assess whether the nuclear industry is maintaining the safety performance of operating plants. The NRC also uses these industry-level indicators to provide feedback to improve the ROP. In CY 2006, the staff completed the development of the BRIIE, a PI that monitors risk-significant initiating events and assigns an importance value to each initiating event according to its relative contribution to industry core damage frequency. In SECY-07-184, “Industry Trends Program for Operating Power Reactors - Baseline Risk Index for Initiating Events,” dated October 22, 2007, the staff requested Commission approval to implement the BRIIE as part of the ITP. The Commission approved the staff’s request in its SRM of December 21, 2007; however, the Commission directed the staff to develop a public communication strategy to explain the meaning of the BRIIE, its underlying concept, and its intended use before making the information publicly available. The staff is in the process of incorporating the BRIIE into the ITP and will provide initial results in the Commission ITP paper that will be issued in early 2009. The staff has reported the FY 2007 results of the ITP to the Commission in an annual paper that complements this paper. The results of the ITP will also be reviewed at the AARM. ROP Resources Overall
staff effort in FY 2007, as reflected in expended hours, increased 2.3 percent
compared with FY 2006. Baseline
inspection hours increased in 2007 primarily because of increased direct
inspection effort, with a corresponding increase in baseline inspection
preparation and documentation. Staff
inspection hours charged to inspection procedure (IP) 71111.21, “Component Design
Bases Inspection;” IP 71152, “Identification and Resolution of Problems;” and
IP 71153, “Follow-up of Events and Notices of Enforcement Discretion,” account
for the bulk of the increase. The staff
plans to evaluate the baseline inspection expenditures in CY 2008 to further
understand this increase. As in the 2006
inspection cycle, all four regions completed the required baseline inspections
in CY 2007. Resources spent for plant-specific
inspections in FY 2007 noticeably decreased overall compared with FY
2006. However, inspection resources in
this area are expected to significantly increase in FY 2008 because of the
increased number of sites in columns 3 and 4 of the ROP Action Matrix and an
increase in special inspections. An
increase in effort related to generic safety inspections reflects the growing
activity in this area. The generic
safety inspections are typically one-time inspections of specific safety
issues, and the effort involved can vary significantly from year to year. Enclosure 5
RI Demographics and Site Staffing As
directed in an SRM dated April 8, 1998, the staff developed measures to monitor
and trend RI demographics and report the results to the Commission on an annual
basis. The staff also developed a site
staffing metric in response to a DBLLTF recommendation, which is included with
the annual analysis. The data from 2003
to 2007 indicate that the experience levels of both RIs and SRIs have remained
high. The staff turnover rate for the
RIs and SRIs increased nationwide from 2006 creating a complex human resource
allocation problem for the regions. Although all four regions met the 90-percent site staffing metric in
2007, two of the four regions had several sites that individually were below
90-percent site staffing. The staff
plans to closely monitor resident demographics and site staffing in 2008
because of anticipated continuing influences on the program as a result of the
expansion of the nuclear industry and internal growth to support the Office of
New Reactors. In addition, a task force
is currently assessing RI program retention issues and barriers for entering
the program. The task force plans to
provide recommendations and potential solutions to senior NRC management. Enclosure 6
Prior Commitments - The staff made four commitments in the CY 2006 ROP self-assessment to improve the efficiency and effectiveness of the ROP. The following summarizes the four actions taken by the staff to address these commitments: (1) The staff continued to
monitor MSPI implementation and incorporated additional improvements to the PI
program to better identify those plants with declining performance, as
described in Enclosure 1
(2) The staff implemented
the ROP realignment process and adjusted inspection resources accordingly, as
described in Enclosure 1
(3) The staff continued to
monitor implementation of the safety culture enhancements and addressed related
GAO recommendations in this area, as described in Enclosure 1
(4) The staff implemented
adjustments/changes to the process related to the point at which licensee
senior management will be requested to meet with the Commission to discuss
actions being taken to improve performance, as described in Enclosure 1
New Commitments - As described in this paper, the staff plans the following four significant actions or activities to improve the efficiency and effectiveness of the ROP in CY 2008: (1) The staff will complete its
lessons-learned review of the MSPI and, based on its recommendations and
discussion with industry, will make any necessary changes to improve the PI
program. The staff will further address
additional improvements to the PI program to better identify those plants with
declining performance.
(2) The staff will explore ways to ensure
site coverage and continuity within the resident program and ensure that
vacancies in the RI program are filled in a timely manner with experienced
individuals.
(3) The staff will continue
to monitor SDP timeliness and develop additional improvements to streamline the
SDP program with the inspection program.
(4) The staff will further enhance ROP
inspection and assessment guidance based on the lessons-learned evaluation of
the safety culture enhancements. Additionally, the staff will further assess the causes of the increase
in the number of sites in columns 3 and 4 of the ROP Action Matrix.
The staff will include the status of these
commitments and other program improvements noted in this paper in the CY 2008
ROP self-assessment.
The self-assessment results for CY 2007
indicate that the ROP provided effective safety oversight, as demonstrated by
meeting the seven program goals and achieving its intended outcomes. The staff continues to experience challenges
in certain areas and recognizes the need for further improvement. The ROP was successful in being objective,
risk informed, understandable, predictable, and in ensuring safety, openness,
and effectiveness. The NRC has
appropriately focused agency resources on performance issues in CY 2007, and
plants continue to receive a level of oversight commensurate with their
performance. The staff continues to
improve various aspects of the ROP as a result of stakeholder participation,
feedback, and lessons learned. Based on
its CY 2007 self-assessment, the staff intends to focus on the commitments
discussed above.
NRC
headquarters and regional resources are needed to conduct the periodic
assessment and realignment of ROP inspection procedures, revision and
maintenance of the NRC Inspection Manual, ROP annual program assessment,
mid-cycle and end-of-cycle licensee performance assessment, and all ROP
management and oversight activities. The
staff estimates that approximately 57 full-time equivalent (FTE) staff members
and $815,000 will be needed for FY 2008, and 57 FTE and $675,000 will be
needed for FY 2009 to conduct these NRR-funded activities. No resources beyond those already included in
the current budget requests for FY 2008 and FY 2009 are needed for these
activities.
The Office of the General Counsel has
reviewed this Commission paper and has no legal objections to its content. The Office of the Chief Financial Officer has
reviewed this Commission paper for resource implications and has no objections.
|
|
Privacy Policy |
Site Disclaimer |
