Search Options
Index | Site Map | FAQ | Facility Info | Reading Rm | New | Help | Glossary | Contact Us

SECY-98-059

March 26,1998

PURPOSE:

The purpose of this paper is to propose options for assessing the performance and competency of licensee management and delineate associated policy issues for Commission consideration and feedback. The staff seeks guidance on the Commission's preferred options before expending significant additional resources necessary to fully develop all available options.

BACKGROUND:

In its May 30, 1997, report entitled, "Nuclear Regulation: Preventing Problem Plants Requires More Effective NRC Action," the General Accounting Office (GAO) stated that NRC does not have an effective process for ensuring that licensees maintain competent management in their nuclear plants. Although NRC's regulations do not explicitly require the evaluation of plant management before a license to operate a nuclear plant can be issued, NRC must determine if the prospective licensee is technically and financially qualified to engage in the activities authorized by the operating license. The GAO recommended that the assessment of management's competency and performance be a mandatory component of NRC's inspection process. The NRC staff responded to GAO's recommendation by stating that, it has been, and remains to be, the NRC staff's practice to conduct performance-based inspections in all areas of facility operation and design and on the basis of the inspection results, to draw conclusions about the effectiveness of the licensee's management.

In its December 30, 1996, report regarding improvements to the Senior Management Meeting (SMM) process, the Arthur Andersen Consulting Firm recommended that the NRC staff assess leading indicators, such as management and operational effectiveness, on an ongoing basis. As a result, the Commission issued a Staff Requirements Memorandum (SRM) dated March 14, 1997, asking the staff to specifically consider the extent to which objective performance information could be used in the SMM decision process, with new performance indicators being phased in as appropriate. In SECY-97-072, "Staff Action Plan to Improve the Senior Management Meeting Process," dated April 2, 1997, the staff responded to the Commission that a development process had been initiated to assess leading indicators, such as management and operational effectiveness, on an ongoing basis.

The NRC currently conducts limited scope evaluations of management performance in response to specific operational events or adverse human or program performance trends. When evaluations of management have been conducted in response to specific events, they have typically been conducted as elements of Special Inspection Teams, Augmented Inspection Teams, and Incident Investigation Teams; if there is evidence of declining performance, Diagnostic Evaluations are conducted. The NRC also evaluates compliance of licensee programs against licensee Quality Assurance programs, which are developed to satisfy 10 CFR Part 50, Appendix B, Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants.

As part of the NRC's effort to improve the SMM process, the staff is investigating the development of management performance assessment tools for improving the current plant performance evaluation methodology. The Office of Nuclear Regulatory Research (RES) and AEOD have initiated development of a process to assess management performance on an ongoing basis. A key milestone in this effort included a week-long workshop in August 1997 involving NRC staff and experts from the research community. The general consensus of the workshop was that Management and Organizational (M&O) factors do influence human and hardware performance. The experts in attendance not only identified important M&O factors but also identified those with the highest potential impact on risk.

In December 1997, AEOD conducted an Organizational Effectiveness Workshop with representatives from RES and NRR to develop characteristics, measures, and indicators to assess management performance based on insights from existing NRC inspection programs, ongoing NRC research, and industry evaluation techniques. As a result of this workshop and a detailed study of inspection information contained in the NRC Plant Issues Matrix (PIM) data base for eight nuclear facilities, the staff determined that it was possible to infer conclusions about management performance from current inspection activities. However, it was concluded that a management assessment based on current inspection program data would not likely result in a leading indicator of plant performance and would not result in a comprehensive and direct assessment of licensee management performance.

DISCUSSION:

The current NRC inspection program assesses compliance with existing regulations and develops performance insights by observing the conduct of operations, material condition of the plant, performance of licensee personnel, quality of engineering work, and the licensee's performance in problem identification and resolution. The NRC inspection program also involves evaluation of operational events to identify root causes such as human error, design deficiencies, and weak administrative controls. The NRC then assesses overall plant performance and infers licensee management performance based on a comprehensive review of inspection findings, license amendments, event reports, enforcement history, and performance indicators.

In response to the GAO and Arthur Andersen recommendations mentioned above, the staff proposes five options for assessing performance and competency of licensee management.^(1),(2)

These options are summarized as follows:

1. Continue to conduct performance-based inspections in all areas of facility operation and design; however, do not attempt to infer or articulate conclusions regarding the performance or competency of licensee management.
   
   
2. Infer licensee management performance from the results of the current performance-based inspection program of overall plant performance. Enhance guidance to improve the quality and consistency of the management performance assessment.
   
   
3. Assess the performance of licensee management through targeted operational performance inspections using specific inspection procedures, trained staff, and contractors.
   
   
4. Assess the performance of licensee management by evaluating and documenting management performance attributes as part of the routine inspection program. Implement the necessary regulations. Revise the inspection, staff training, and qualification programs accordingly.
   
   
5. Assess the competency of licensee management by evaluating management competency attributes using specific inspection procedures, trained staff, and contractors. Implement the necessary regulations. Revise the inspection, staff training, and qualification programs accordingly.

To help determine whether the NRC should modify its current practice with respect to assessing licensee management performance, the potential benefits derived from insights into M&O factors were considered in concert with those findings typically available from existing inspection programs. In all options, the staff would continue to assess the role of management in event follow-up. In options two through five, the staff would develop improved guidance for defining and assessing management performance or competency. Currently, an ongoing RES program to develop the technical basis needed to define the relationship between M&O factors and equipment and human performance would support implementation of options two, three, or four. The options, along with the implementation and policy issues associated with each option, are discussed below.

Option: The NRC would continue its practice of conducting performance-based inspections in all areas of facility operation and design as required by the current inspection program. The NRC would continue inspecting for compliance with existing regulations by observing the conduct of operations, the material condition of the plant, the performance of licensee personnel, the quality of engineering work, and the licensee's performance in problem identification and resolution. In addition, the NRC would continue to examine operational events to identify root causes such as human error, design deficiencies, and administrative controls. The process for assessing plant performance would continue to be based on inspection findings, enforcement actions, operational events, and performance indicators. However, contrary to current practice, the NRC would no longer infer or articulate conclusions regarding management. This option would not require additional inspection effort beyond that required for implementing the current inspection program.

Implementation: Minimal guidance development and inspection training would be required to implement this option.

Policy Issue: This option would entail a Commission decision directing the staff to discontinue its practice of inferring management performance from the results of performance-based inspections.

Option: The NRC would continue its practice of conducting performance-based inspections by observing the conduct of operations, the material condition of the plant, the performance of licensee personnel, the quality of engineering work, and the licensee's performance in problem identification and resolution. The NRC would continue to examine operational events to identify root causes such as human error, design deficiencies, and administrative controls. The process for assessing plant performance would continue to be based on inspection findings, enforcement actions, operational events, and performance indicators using newly developed tools such as the Plant Performance Template and the Trending Methodology.

Management performance would be inferred from examination of inspection results and operational events. In the past, this has been accomplished through informal qualitative judgment. Guidance currently being developed by RES would allow for a more systematic method of inferring management performance. It is estimated that this option would require the same inspection effort and resources as those required for implementing the current inspection program.

Implementation: The NRC staff is currently exploring improvements to the process for assessing plant performance. This effort is known as the integrated review of assessment process (IRAP). The new process would involve annual meetings at the regional level to assess overall plant performance. Following the assessment, the licensee would be given the opportunity to respond to NRC's assessment findings. Subsequently, the assessment findings would be made public.

Currently the management performance implications of individual inspection findings are not documented in the PIM. In the future, in order to be compatible with the scrutability requirements of the IRAP, the staff will be required to identify a mechanism for periodically disclosing the managment assessment for public scrutiny.

This option is currently in place. Improvements to the process could be accomplished in FY 1999.

Policy Issue: Although this option is not a change in current policy, the Commission may choose to explicitly direct the staff to infer management performance from the data currently documented in the inspection program.

Option: NRC inspectors would continue to evaluate, respond to, and document failures to comply with existing regulations, much as they do under option 2 today. However, based on guidance currently under development by RES, inspection procedures would be developed to perform targeted operational performance inspections for specific indications of management performance. Inspectors would receive training on the types of issues identified in the inspection procedures and their relationship to management performance.

The assessment of management performance would be based primarily on the results of those targeted inspection efforts. Insights from operational events would also be factored into the assessment. It is estimated that this option would require reallocation of inspection resources to management performance issues.

Implementation: In this option, management performance findings would be documented in the PIM, and would become public at the time the PIM is updated. The overall assessment of management performance would be based on the number and severity of such findings. This option would require significant training for NRC staff.

This option could be implemented by FY 2000.

Policy Issue: This option would expand what is currently being done in the assessment process by collecting information related specifically to management performance. This option would require a change in current policy and would require the Commission to direct the staff to document information related to the assessment of management performance as part of NRC's formal inspection program.

Option: The NRC would directly assess the performance of licensee management by defining the attributes of management that are related to safety performance, developing assessment tools and applying them for assessing management performance at the licensees' facilities. The possible types of attributes that would be evaluated include: safety culture, communications, degree to which the organization is a learning organization. The NRC would develop acceptable levels of the various attributes of management performance based on available research. The necessary regulations and inspection procedures would also be developed. Inspectors would receive training on the types of issues identified in the revised regulations and inspection modules and their relationship to management performance.

In this option, the assessment would be based on the results of specific management performance inspections. The results would be documented in the PIM. Insights from operational events would be factored into the assessment. The NRC would take enforcement action based on failure to meet management performance standards. This option would require reallocation of inspection resources to management performance issues.

Implementation: If this option is chosen, significant additional study will be required to assess implementation options and their associated impacts. The possible implementation options would be (1) the NRC would periodically perform management performance inspections, (2) the NRC would perform management performance inspections on a for-cause basis, (3) the NRC would require the licensees to perform periodic assessments, or (4) the NRC would require licensees to conduct assessments on a for-cause basis. Regulations and Agency guidelines would provide consistency in evaluating management performance, provide a regulatory basis for developing questionnaires and inspection protocols, and ensure that the management issues assessed have been identified as important to risk. This option would require significant training for NRC staff.

This option could be implemented by FY 2001.

Policy Issue: This option would require the Commission to issue new regulations and would constitute a significant departure from current regulatory approaches under the NRC's statutory authority.

Option: The NRC would directly assess the competency of licensee management. The NRC would define the knowledge, skills, and attributes associated with competent safety management, and develop appropriate evaluation methods. The NRC would define levels of acceptable management competency based on available research on the topic. The NRC would implement the necessary regulations to enforce compliance with those standards. Inspection procedures to assess competency would be developed and implemented into the NRC inspection program.

In considering this option several questions, such as those noted below, need to be explored:

• Which management positions would be included in the assessment ?
  
  
• Would individual managers be assessed or would the assessment focus on management teams ?
  
  
• Would there be formal qualifications or licensing requirements for licensee managers ?

The NRC would take enforcement action based on failure to meet the management competency standards. The assessment of licensee performance would include a direct assessment of management competency. This option would require reallocation of inspection resources to management competency issues.

Implementation: If this option is chosen, significant additional study will be required to assess implementation options and their associated impacts. The possible implementation options would be (1) the NRC would periodically perform management competency inspections, (2) the NRC would perform management competency inspections on a for-cause basis, (3) the NRC would require the licensees to perform periodic competency assessments, (4) the NRC would require licensees to conduct competency assessments on a for-cause basis, (5) the NRC could require competency assessments by a third party, or (6) the NRC could license individual managers.

This option would likely require the development of a special corps of inspectors and the use of specialized contractors to implement the methods.

This option could be implemented in FY 2001.

Policy Issue: This option would require the Commission to issue new regulations and would constitute a significant departure from current regulatory approaches under the NRC's statutory authority.

RESOURCES AND IMPACTS:

The resources and impacts associated with each of the options are listed in Table 1 (Attached). These include the need for policy or rule change; the costs associated with development of the rule, necessary guidance, modifications to the inspection program and additional inspector training; impact on staff qualifications; reallocation of inspection resources (reduction in plant performance inspection to compensate for new inspections of management performance or competency); probability of successful implementation; use as a leading indicator; and scrutability of findings. Each of these factors was quantitatively (where FTE or contract costs could be estimated) or qualitatively assessed on a relative basis for each option.

Inspection of Table 1 shows that options that depend on rulemaking have the greatest impact on staff qualifications and require the most significant reallocation of inspection resources. On the other hand, those options will have better scrutability of findings because of documentation requirements and clarity of NRC expectations. The probability of successful implementation for these options is not as high as the other options because of the uncertainties associated with the rulemaking process and development of a defensible technical basis and regulatory analysis.

Based on Commission guidance, the earlier response to GAO will be revised as necessary.

REQUESTED COMMISSION ACTION:

The staff requests that the Commission select one of the options listed in this paper and provide guidance for proceeding further. Subsequent to the Commission's action, the staff will develop another Commission paper to address in more detail a plan of action for implementation of the option identified by the Commission.

COORDINATION:

The Office of the General Counsel (OGC) has reviewed this paper and has no legal objections, although OGC notes that there are significant legal issues that would need to be addressed if option four or five were pursued. The Office of the Chief Financial Officer has reviewed this paper for resource implications and has no objections.

RESOURCES:

Currently, the FY 98 and 99 budgets support development of guidance for options one, two, or three. Options four and five are not budgeted.

ATTACHMENT

Table 1

RESOURCES AND IMPACTS ASSOCIATED WITH ASSESSING PERFORMANCE AND COMPETENCY OF LICENSEE MANAGEMENT

(1) Currently funded in FY-98 and FY-99
(2) Reallocation of resources from plant performance inspection to management assessment
(3) Beyond the ability of plant performance assessment alone to indicate declining performance
(4) These options will require significant use of contractor support

1. Management Performance Assessment: assessing characteristics (e.g. culture, organizational learning, communications) of an organization to determine if the implementation of the organization's procedures and practices ensures equipment and human reliability, regulatory compliance, and safe plant performance.

2. Management Competency Assessment: assessing the characteristics, attributes, knowledge and skills of managers to determine if they have the necessary ability to develop and implement procedures and practices to ensure equipment and human reliability, regulatory compliance, and safe plant performance.