ROP Program Evaluations and Stakeholder Feedback
On this page:This page includes links to files in non-HTML format. See Plugins, Viewers, and Other Tools for more information.
Annual Self-Assessments
On February 24, 2000, the staff issued SECY-00-0049, "Results of the Revised Reactor Oversight Process Pilot Program". The resulting staff requirements memorandum (SRM), issued on March 28, 2000, approved initial implementation of the ROP as recommended by the staff. The initial implementation of the ROP began on April 2, 2000. In a followup SRM issued on May 17, 2000, the Commission directed the staff to report on the ROP results after the first year of implementation. The staff did so and documented the results in SECY-01-0114, "Results of the Initial Implementation of the New Reactor Oversight Process", issued June 25, 2001. SECY-01-0114 also noted the staff's intention to continue to perform an annual self-assessment of the ROP. Accordingly, the staff has issued an ROP self-assessment Commission paper each year before the Agency Action Review Meeting (AARM) and has briefed the Commission on the self-assessment results following the AARM.
The objectives and details of the ROP self-assessment program are contained in IMC 0307, "Reactor Oversight Process Self-Assessment Program". The annual ROP self-assessments are summarized in the table below.
Time Period | Commission Paper | Commission Meeting | Commission SRM | Metric Report |
---|---|---|---|---|
CY 2011 | SECY-12-0055 | June 1, 2012 | CY 2011 | |
CY 2010 | SECY-11-0054 | May 27, 2011 | Meeting SRM M110527 | CY 2010 |
CY 2009 | SECY-10-0042 | May 27, 2010 | Meeting SRM M100527 | CY 2009 |
CY 2008 | SECY-09-0054 | May 14, 2009 | Meeting SRM M090514 | CY 2008 |
CY 2007 | SECY-08-0046 | June 4, 2008 | Meeting SRM M080604 | CY 2007 |
CY 2006 | SECY-07-0069 | May 31, 2007 | Meeting SRM M070531 | CY 2006 |
CY 2005 | SECY-06-0074 | May 16, 2006 | Meeting SRM M060516B | CY 2005 |
CY 2004 | SECY-05-0070 | May 25, 2005 | Meeting SRM M050525B | CY 2004 |
CY 2003 | SECY-04-0053 | May 4, 2004 | Meeting SRM M040504B | SECY-04-0053, Attachment 3 |
CY 2002 | SECY-03-0062 | May 15, 2003 | Meeting SRM M030515 | SECY-03-0062, Attachment 3 |
CY 2001 | SECY-02-0062 | May 1, 2002 | Meeting SRM M020501 | SECY-02-0062, Attachment 2 |
Initial Implementation | SECY-01-0114 | July 19, 2001 | Meeting SRM M010719B | N/A |
Pilot Program | SECY-00-0049 | March 7, 2000 | SRM 1 / SRM 2 | N/A |
Independent Evaluations
In addition to the ROP self-assessment program, several independent evaluations have been performed since the inception of the ROP to analyze its effectiveness and recommend improvements. Over the past several years, the Government Accountability Office (GAO), the Office of Management and Budget (OMB), the Office of the Inspector General (OIG), the Advisory Committee for Reactor Safeguards (ACRS), the Davis-Besse Lessons Learned Task Force (DBLLTF), and the SDP Task Group have all performed evaluations of the ROP. Early on, two evaluation panels performed detailed reviews of the pilot program and initial implementation of the ROP. These evaluations have generally provided favorable results, but have also suggested potential areas of improvement. Several recommendations from these independent evaluations have been incorporated into the ROP as detailed in the annual self-assessments above and staff responses below.
Entity | Final Report | Staff Response/Program Improvements |
---|---|---|
Independent Evaluation of the Reactor Oversight and Incident Response Program | 12/31/2008 | Action Plan 6/09 |
NRC Cross-cutting review and Palo Verde Lessons Learned | 12/2008 | Draft dispositions current as of April 17, 2008 |
GAO report on Nuclear Reactor Oversight | GAO-06-1029 09/2006 | Response 11/06 |
OIG audit of the Reactor Program System | OIG-05-A-11 04/2005 | Status 11/05
(closes items 5 and 10) Status 02/06 (closes remaining items) |
OIG audit of the Baseline Inspection Program | OIG-05-A-06 12/2004 | Status
09/05 (closes items 1, 5, 6, 9, and 10) Status 12/05 (closes items 7 and 8) Status 06/06 (closes items 3 and 4) |
OIG audit of the Significance Determination Process | OIG-02-A-15 08/2002 | Status updates in annual SECYs, tracked internally in NRR Director’s Quarterly Status Report. |
SDP Task Group | Task Group Report 12/2002 | Status updates in annual SECYs, tracked internally in NRR Director’s Quarterly Status Report. |
Advisory Committee for Reactor Safeguards | Latest Letter 03/2003 | Latest response 04/2003 |
Davis-Besse Lessons Learned Task Force | DBLLTF final report 09/2002 | Final
Report Status/Closure Significant Improvements |
OMB Program Assessment Rating Tool Evaluation | Final Report & Status of Performance Measures and Follow-up Actions 09/2003 | Final Report & Status of Performance Measures and Follow-up Actions |
Initial Implementation Evaluation Panel | IIEP final report 05/2001 | SECY-01-0014, Attachment 5 |
Pilot Program Evaluation Panel | PPEP final report 12/1999 | SECY-00-0049, Attachment 3 |
Stakeholder Feedback
External Surveys/Feedback
The staff biennially publishes a survey in a Federal Register notice to obtain external stakeholder input regarding ROP effectiveness. The frequency of the survey changed from annually to biennially in CY 2007. The surveys request responses to questions corresponding to specific ROP performance metrics, as defined in IMC 0307, and also provide an opportunity for open comments on the ROP that may not be directly captured by the specific questions. In an effort to actively solicit feedback on the implementation of the ROP, the staff (1) mails surveys directly to hundreds of stakeholders, (2) places a direct link to the survey information on both the ROP Web page and the �Documents for Comment� page of the NRC�s external Web site, and (3) issues a press release and posted it on the NRC�s external Web site. Since 2002, the NRC acknowledged receipt of each survey response by correspondence indicating that the staff has considered and generally addressed the comments during the annual ROP self-assessment, as appropriate. However, the NRC received feedback in the CY 2004 survey indicating that it was unresponsive to survey comments and decided to consolidate the comments by question and provide a comprehensive response to each question in addition to the acknowledgment letters. The staff received positive feedback on the consolidated response from several stakeholders in the CY 2005 survey, and plans to continue this practice.
External feedback is also solicited through monthly public meetings, special meetings/workshops to discuss specific issues, public meetings in the vicinity of each operating reactor to discuss the results of the NRC's annual assessment of the licensee�s performance, and at the annual Regulatory Information Conference.
Solicitation Date | Responses Received | Staff Evaluation | Staff Response | |
---|---|---|---|---|
CY2009/2008 | FRN 09/25/2009 | 5 | SECY-10-0042 | Consolidated Response |
CY2007 | FRN 10/11/2007 | 7 | SECY-08-0046, Enclosure 3 | Consolidated Response |
CY2006 | FRN 10/10/2006 | 16 | SECY-07-0069, Enclosure 3 | Consolidated Response |
CY2005 | FRN 10/21/2005 | 21 | SECY-06-0074, Enclosure 6 | Consolidated Response |
CY2004 | FRN 10/25/2004 | 21 | SECY-05-0070, Attachment 6 | Consolidated Response |
CY2003 | FRN 11/05/2003 | 18 | SECY-04-0053, Attachment 4 | Letters |
CY2002 | FRN 11/22/2002 | 19 | SECY-03-0062, Attachment 4 | Letters |
CY2001 | FRN 11/21/2001 | 13 | SECY-02-0062, Attachment 5 | N/A |
First Year | FRN 01/10/2001 | 14 | SECY-01-0014, Attachment 2 | N/A |
Pilot Program | FRN 07/26/1999 | 21 | SECY-00-0049, Attachment 1 | N/A |
Pre-ROP #2 | FRN 01/22/1999 | 28 | SECY-99-007A, Attachment 1 | N/A |
Pre-ROP #1 | FRN | 26 | SECY-99-007, Attachment 7 | N/A |
Internal Surveys/Feedback
Internal surveys are conducted on a biennial basis to solicit and analyze stakeholder feedback from NRC staff regarding the effectiveness of the ROP. All respondent replies are anonymous and each specific question has five possible answers/ratings. Many of the questions correspond to specific ROP performance metrics as defined in IMC 0307. The final each section of the survey provides space to expound or make additional comments. There have been four internal surveys to date as noted in the following table.
In addition, Office of Nuclear Reactor Regulation (NRR) staff conducts biweekly conference calls with regional division- and branch-level management to discuss current issues associated with the ROP, and meets periodically with regional managers to discuss more complex ROP topics and issues. NRR staff also participates in each regions’ resident counterparts meeting and makes site visits to give regional staff and management the opportunity to discuss ROP implementation and provide feedback. The internal feedback process, as described in Inspection Manual Chapter (IMC) 0801, “Reactor Oversight Process Feedback Program,” also provides a useful means for the NRC staff to identify concerns or issues and recommend improvements related to ROP implementation.
Solicitation Date | Responses Received | Staff Evaluation/Response | |
---|---|---|---|
CY 2008 & 2007 | 11/2008 | 159 | SECY-09-0054, Enclosure 2 |
CY 2006 & 2005 | 11/2006 | 266 | SECY-07-0069, Enclosure 3 |
CY 2004 & 2003 | 11/2004 | 209 | SECY-05-0070, Attachment 6 |
CY 2002 & 2001 | 12/2002 | 236 | SECY-03-0062, Attachment 5 |
First Year | 03/2001 | 234 | SECY-01-0014, Attachment 1 |
Pilot Program | 11/1999 | 102 | SECY-00-0049, Attachment 1 |