| |
Search Options | |||
| Index | Site Map | FAQ | Facility Info | Reading Rm | New | Help | Glossary | Contact Us | |
|||
|
POLICY ISSUE SECY-07-0069 April 6, 2007
To present the results of the staff's annual self-assessment of the Reactor Oversight Process (ROP) for calendar year (CY) 2006. The CY 2006 self-assessment results indicate that the ROP has been successful in meeting its program goals and achieving its intended outcomes. The ROP was deemed to be objective, risk-informed, understandable, and predictable, and the ROP met the agency goals of ensuring safety, openness, and effectiveness. The U.S. Nuclear Regulatory Commission (NRC) staff maintained its focus on stakeholder involvement and continued to improve various aspects of the ROP as a result of feedback and lessons learned. The staff implemented several ROP improvements in CY 2006 to address issues raised by the Commission, recommended by independent reviews, and obtained from internal and external stakeholder feedback. Most notably, the staff made significant enhancements to the assessment and inspection programs in CY 2006 to more fully address safety culture. The staff continues to improve the performance indicator (PI) program to better identify declining plant performance in a timely manner, but the staff recognizes the need for further improvement. The inspection program independently verified that plants were operated safely, appropriately identified performance issues, and ensured the adequacy of licensee corrective actions to address the noted performance issues. Further improvements in the significance determination process (SDP) resulted in the SDP timeliness goal being met in CY 2006 for the first time since ROP implementation. The assessment program ensured the staff and licensees took necessary actions to address identified performance issues. The staff will continue to actively solicit input from the NRC's internal and external stakeholders and further improve the ROP based on stakeholder feedback and lessons learned. On February 24, 2000, the staff issued SECY-00-0049, "Results of the Revised Reactor Oversight Process Pilot Program." The resulting staff requirements memorandum (SRM), issued on March 28, 2000, approved initial implementation of the ROP as recommended by the staff. The initial implementation of the ROP began on April 2, 2000. SECY-01-0114, "Results of the Initial Implementation of the New Reactor Oversight Process," issued June 25, 2001, noted the staff's intention to perform an annual self-assessment of the ROP. Accordingly, the staff has issued an ROP self-assessment Commission paper each year before the Agency Action Review Meeting (AARM) and has briefed the Commission on the self-assessment results following the AARM. This paper provides the results of the ROP self-assessment for CY 2006. The staff performed the CY 2006 self-assessment in accordance with Inspection Manual Chapter (IMC) 0307, "Reactor Oversight Process Self-Assessment Program." The ROP self-assessment program evaluates the overall effectiveness of the ROP through its success in meeting its preestablished goals and intended outcomes. In accordance with IMC 0307 and as noted in SECY-05-0070, "Reactor Oversight Process Self-Assessment for Calendar Year 2004," security and safeguards activities are no longer included in this self-assessment except where specifically noted. On January 9, 2007 the staff issued SECY-07-0008, "Evaluation of Revised Security Oversight Process for Nuclear Power Plants" that evaluated the implementation of the security ROP during the 2006 ROP inspection cycle. In response to the staff's briefing on the results of the AARM on May 16, 2006, the Commission directed the staff to take the actions specified in SRM M060516B, dated June 14, 2006. The Commission directed that, in addition to efforts described in the ROP self-assessment, the staff should continue working with stakeholders to improve the PI program to better identify those plants with declining performance. The Commission also directed the staff to continue improving the timeliness and efficiency of the SDP, and reconsider when licensee senior management should be requested to meet with the Commission to discuss actions being taken to improve performance. The staff uses program evaluations and performance metrics to determine the effectiveness (i.e., success) of the ROP in meeting its seven program goals and intended outcomes. The seven goals include the four program-specific goals of being objective, risk-informed, understandable, and predictable, as well as the three applicable performance goals listed in the NRC's Strategic Plan (ensuring safety, openness, and effectiveness). The intended outcomes of the ROP, which help form its basis and are incorporated into the various ROP processes, include:
During the seventh year of ROP implementation (CY 2006), the staff conducted numerous activities and obtained data from many diverse sources to ensure that it performed a comprehensive and robust self-assessment. Data sources included the ROP performance metrics described in IMC 0307, recommendations from independent evaluations, comments from external stakeholders in response to a Federal Register notice, insights from internal stakeholders based on the biennial survey and ROP internal feedback process, and feedback received from stakeholders at various meetings, workshops, and conferences. The staff also applied the direction and insight provided by the Commission through several SRMs. The staff analyzed this information to gain insights regarding the effectiveness of the ROP in fulfilling its program goals and intended outcomes. The staff evaluated the key program areas of PIs, inspection, SDP, and assessment, as discussed in the following paragraphs. In addition, the staff assessment included ROP communication activities, ROP self-assessment and independent evaluations, ROP resources, and resident inspector (RI) demographics and staffing. As noted in the pertinent sections of this paper, the staff has also included several enclosures with additional detail to support the staff's assessment and conclusions. ROP Program Area Evaluations The staff performed evaluations in each of the four key program areas of the ROP: the PI
program, inspection program, SDP, and assessment program. The results are summarized
below and are discussed in more detail in Enclosure 1 PI Program -- The staff and many stakeholders remain concerned that the current set of PIs and thresholds do not provide adequate information to identify outliers and detect declining plant performance. The Mitigating Systems Performance Index (MSPI) was implemented in April 2006, and the staff continues to monitor MSPI implementation and address implementation issues through the monthly public ROP meetings and through the ROP PI frequently asked question (FAQ) process. The staff is reviewing and revising several PIs, including unplanned scrams with loss of normal heat removal, reactor coolant system leakage, safety system functional failures, and PIs in the emergency preparedness cornerstone. As a result of the internal and external survey responses, two of the PI self-assessment metrics were not met: whether the PI program provides useful insights to help ensure plant safety and whether the PI program identifies performance outliers in an objective and predictable manner. As noted above and in last year's self-assessment, the staff recognizes the need to improve the PI program to better identify outliers and provide more meaningful indications of declining plant performance. Inspection Program -- NRC inspectors independently verified that plants were operated safely, appropriately identified performance issues, and ensured the adequacy of licensee corrective actions to address the noted performance issues. The staff completed its development of the ROP realignment process, that involves a biennial evaluation of the baseline inspection procedures as described in Appendix B to IMC 0307 to more efficiently allocate inspection resources, and will implement the realignment review in CY 2007. In CY 2006 the staff closed the remaining recommendations from the December 2004 Office of the Inspector General (OIG) audit of the baseline inspection program and the Davis-Besse Lessons Learned Task Force (DBLLTF). The staff continues to perform the engineering design inspections and will re-evaluate these inspections for potential improvements based on lessons learned. The staff also made substantive changes to the inspection and assessment program documents to more fully incorporate safety culture in CY 2006. The regions completed the required baseline inspection program for CY 2006, and all inspection program performance metrics met their criteria. Significance Determination Process -- During this assessment period, the remaining objectives of the SDP Improvement Plan were completed, and the program improvements resulted in the SDP timeliness goal being met for the first time. The staff instituted several significant enhancements including implementing the Phase 2 Pre-Solved Tables, issuing the Risk Assessment Standardization Project (RASP) Handbook, and providing additional guidance regarding the quality of licensee probabilistic risk assessments (PRAs). One SDP performance metric failed to meet program expectations; stakeholders did not perceive the SDP to yield an appropriate and consistent regulatory response across all ROP cornerstones. The staff believes that relative parity has been achieved among the cornerstones, but will continue to review findings to determine the need for adjustments, particularly in the emergency preparedness and public radiation safety cornerstones. Assessment Program -- Staff implementation of the assessment program ensured that staff and licensees took necessary actions to address performance issues and adjusted resources to focus on significant performance issues. The staff made significant enhancements to the program guidance to more fully address safety culture and implemented the revised program in July 2006. The staff is compiling lessons learned during the initial 18-month implementation phase of the enhanced ROP and will present the evaluation to the Commission in the CY 2007 ROP self-assessment. The staff evaluated and recommended when licensee senior management should be requested to meet with the Commission to discuss actions being taken to improve performance. The staff evaluated the three Action Matrix deviations from CY 2006 for potential program changes, but none were deemed necessary. All performance metrics in the assessment program met their criteria in CY 2006. ROP Communication Activities The staff continued to emphasize stakeholder involvement and open communication regarding the ROP. The staff used a variety of communication methods to ensure that all stakeholders could access ROP information and were given an opportunity to participate in the process and provide feedback. As discussed below, the staff sought and implemented improvements to the ROP based on feedback and insights from all stakeholders. External Stakeholder Interface -- The staff conducted monthly public working-level meetings with the Nuclear Energy Institute (NEI), the industry, and interested stakeholders to discuss ongoing refinements to the ROP. The staff conducted a number of public meetings prior to implementing the safety culture enhancements. The staff also conducted public meetings in the vicinity of each operating reactor to discuss the results of the NRC's annual assessment of the licensee's performance. These meetings provided interested stakeholders an opportunity to engage NRC on plant performance and the role of the agency in ensuring safe plant operations. The staff also sponsored three breakout sessions at the Regulatory Information Conference (RIC) in March 2006 on the inspection program, the assessment program, and safety culture, and discussed additional ROP topics during the regional breakout sessions. The staff also issued its annual external survey through the Federal Register in October 2006 to evaluate ROP effectiveness and gather stakeholder insights. The staff maintained and enhanced the NRC's Web pages to communicate current ROP-related information and results. These outreach efforts have resulted in valuable feedback and ROP improvements. Internal Stakeholder Interface -- The Office of Nuclear Reactor Regulation (NRR) staff continued to conduct biweekly conference calls with regional division- and branch-level management to discuss current issues associated with the ROP. In addition, the NRR staff met periodically with regional managers to discuss more complex ROP topics and issues. The NRR staff participated in each region's inspector counterpart meeting so that regional staff and management could provide feedback on ROP implementation. The NRR staff also administered its biennial internal survey in October 2006 to evaluate program effectiveness and gather direct feedback from the staff responsible for implementing the ROP. The ROP Digital City Web page was updated frequently to include recent and useful information specifically for our internal stakeholders. The staff issued several editions of the inspector newsletter to share inspection tips and lessons learned. In addition, the staff initiated the Operating Experience Smart Sample pilot program to further integrate operating experience with the ROP. The staff continued its efforts to improve the initial and continuing inspector training programs in order to produce and maintain well-qualified, competent inspectors. The ROP feedback process allows the NRC staff to identify concerns or issues and recommend improvements related to ROP policies, procedures, or guidance. Over the past few years, the staff has steadily improved the timeliness in resolving feedback issues. In CY 2006, the staff further enhanced the ROP feedback process to improve the timeliness, efficiency, and effectiveness of feedback resolution. Specifically, the NRR Work Planning and Control Center now assigns feedback review to the staff to account for resource usage and track the timeliness of the response. Stakeholder Survey Results -- The staff conducted both internal and external surveys on the ROP in CY 2006. The responses from the survey of external stakeholders were similar in number, distribution, and content to previous years. Specifically, 8 of the 16 responses were from utilities or their representatives, 3 were from State or other government entities, and 5 were from the public. Overall, the responses from utilities were generally positive, whereas responses from the public were less positive and raised specific concerns about the effectiveness of the ROP. The responses to the internal survey were similar in distribution to previous years, but the participation and number of comments increased in 2006. The responses were generally positive and showed increased satisfaction when compared with the previous internal survey in CY 2004. Enclosure 3 ROP Web Page Enhancements -- The staff continued to maintain the ROP Web pages to ensure that they communicate accurate and timely ROP information. Based on stakeholder feedback, the staff revamped the external ROP Web page in fall 2006. The more notable enhancements include --
ROP Self-Assessment Metrics and Independent Evaluations The objectives and details of the ROP self-assessment program are contained in IMC 0307. This paper, supplemented by the annual report of performance metrics, provides the results of the staff's self-assessment for CY 2006. In addition to the ROP self-assessment program, several independent evaluations have been performed in the past few years, most notably by the Government Accountability Office (GAO), Office of Management and Budget (OMB), the OIG, and the DBLLTF. These evaluations generally provided favorable results, but also suggested potential areas of improvement. The staff addresses several recommendations from these independent evaluations in the enclosures to this paper. ROP Performance Metrics -- The staff's annual report of self-assessment performance metrics was completed for CY 2006 and is publically available through ADAMS (reference ADAMS
Accession No. ML070720085). The ROP met 50 of its 53 performance metrics. The only
exceptions were two PI metrics and one SDP metric, all related to stakeholder feedback. The
staff discusses its corrective actions in the metric report as well as the program area
evaluations in Enclosure 1 Independent Evaluations -- The GAO completed an independent evaluation of the ROP and
issued its report on September 27, 2006 (reference ADAMS Accession No. ML062720030).
The report, entitled "Nuclear Regulatory Commission: Oversight of Nuclear Power Plant Safety
Has Improved, but Refinements Are Needed (GAO-06-1029)," included three
recommendations. The NRC formally responded to the GAO report on November 27, 2006
(reference ADAMS Accession No. ML062910527). The staff is addressing the GAO
recommendations as discussed in further detail in Enclosure 1 The staff also resolved and closed all recommendations from the OIG audit of the baseline inspection program (OIG-05-A-06, issued December 22, 2004). In addition, the staff addressed all items from the previous audits of the Reactor Program System (OIG-05-A-11, issued April 13, 2005) and SDP (OIG-02-A-15, issued August 21, 2002). The staff improved the ROP as a result of these recommendations. A number of enhancements were also made to the ROP based on recommendations from the DBLLTF. The staff completed the last few effectiveness reviews in CY 2006, and the staff considers all DBLLTF action items to be closed. All of the independent evaluations of the ROP, along with the staff's response and resultant program improvements, are available on the recently developed ROP Web page entitled "ROP Program Evaluations and Stakeholder Feedback." Regulatory Impact -- The staff also received and evaluated feedback from licensees as part of
the regulatory impact process. The regulatory impact process was established in 1991, based
on Commission direction to develop a process for obtaining feedback from licensees and
reporting the feedback to the Commission. Over the past year, the staff received feedback
from 68 reactor licensees on 191 issues. Of the comments received, 84 percent were
favorable, and 16 percent were unfavorable. The comments fell into three main categories:
formal communication with licensees, inspector performance, and security and safeguards
activities. Enclosure 4 Industry Performance Trends -- The NRC collects and monitors industrywide data to assess whether the nuclear industry is maintaining the safety performance of operating plants. The NRC also uses these industry-level indicators to provide feedback to improve the ROP. In CY 2006, the staff completed the development of the baseline risk index for initiating events (BRIIE), a performance indicator that monitors risk-significant initiating events and assigns an importance value to each initiating event according to its relative contribution to industry core damage frequency. The staff is in the process of incorporating BRIIE into the Industry Trends Program (ITP), and data collection started in January 2007. The staff will report the initial results of this new ITP indicator in a Commission paper on the fiscal year (FY) 2007 Industry Trends Program (issued in early CY 2008). The FY 2006 results of the ITP have been reported to the Commission in an annual paper that complements this paper. The results of the ITP will also be reviewed at the AARM. ROP Resources Overall staff effort in CY 2006 was essentially unchanged compared with CY 2005, although the
distribution of effort varied among the various elements of the ROP. As in the CY 2005
inspection cycle, the staff completed all required CY 2006 baseline inspections with the
allocated regional resources. A decrease in supplemental inspections in 2006 was offset by an
increase in plant-specific inspections. The plant-specific effort includes inspection activity at
sites with approved deviations for additional inspection, and inspection activity at Browns Ferry
Unit 1. Enclosure 5 Resident Inspector (RI) Demographics and Site Staffing As directed in an SRM dated April 8, 1998, the staff developed measures to monitor and trend RI demographics and report the results to the Commission on an annual basis. The staff also developed a site staffing metric in response to a DBLLTF recommendation, now included with the annual analysis. The 2006 data indicate that the experience levels of both RIs and Senior Resident Inspectors (SRIs) are relatively high, the RI and SRI staffing levels are generally good, and the staffing turnover rate was not excessive. However, the staff plans to closely monitor resident demographics and site staffing in 2007 due to anticipated influences on the program as a result of the projected expansion of the nuclear industry and our own growth to support the formation of the New Reactors Office. Detailed analyses of the 2006 RI demographics and site staffing are included as metrics O-14 and O-15, respectively, in the annual ROP performance metric report (reference ADAMS Accession No. ML070720085). Prior Commitments -- The staff made five commitments in the CY 2005 ROP self-assessment to improve the efficiency and effectiveness of the ROP. The following summarizes the staff's actions to address these commitments:
New Commitments -- As described in this paper, the staff plans the following significant actions or activities to improve the efficiency and effectiveness of the ROP in CY 2007:
The staff will include the status of these commitments and other program improvements noted in this paper in the CY 2007 ROP self-assessment. The self-assessment results for CY 2006 indicate that the ROP provided effective safety oversight as demonstrated by meeting the seven program goals and achieving its intended outcomes. The staff continues to experience challenges in certain areas and recognizes the need for further improvement. The ROP was successful in being objective, risk-informed, understandable, and predictable, and ensuring safety, openness, and effectiveness. The NRC has appropriately monitored operating nuclear power plant activities and focused agency resources on performance issues in CY 2006, and plants continue to receive a level of oversight commensurate with their performance. The staff continues to emphasize stakeholder involvement and improve various aspects of the ROP as a result of feedback and lessons learned. Based on its CY 2006 self-assessment, the staff intends to focus on the action presented in the commitments to the Commission discussed above. The staff estimates that approximately 57 full-time equivalent (FTE) staff members and $680,000 will be needed for FY 2007, and 60 FTE and $860,000 will be needed for FY 2008. These resources include regional, Office of Nuclear Reactor Regulation, Office of Nuclear Regulatory Research, and Office of Nuclear Security and Incident Response efforts for ROP refinement, management, and performance assessment activities within the scope of the current budget requests. No resources beyond those already included in the current budget requests for FY 2007 and FY 2008 are needed for these activities. The Office of the General Counsel has reviewed this Commission paper and has no legal objections to its content. The Office of the Chief Financial Officer has reviewed this Commission paper for resource implications and has no objections.
|
|
Privacy Policy |
Site Disclaimer |
.