Index | Site Map | FAQ | Facility Info | Reading Rm | New | Help | Glossary | Contact Us

SECY-97-192

August 21, 1997

PURPOSE:

The purpose of this Commission Paper is to respond to staff requirements memorandum (SRM) M970129A, dated February 14, 1997, in which the Commission requested the staff to conduct a detailed peer review of the Arthur Andersen methodology, compare the results of the application of that methodology to the results of the last senior management meeting (SMM), evaluate and describe the reasons for any differences, and to address whether the Arthur Andersen methodology is more appropriate at the screening meetings or the SMMs themselves.

In addition, this paper responds to a separate request in SRM M970129A that the staff consider the advisability of reissuing adverse trend letters at each SMM for which they are left in effect, as well as, issuing trending letters between SMMs (or some other defined evaluation interval) in warranted situations. The Commission asked that the staff provide the results of its evaluation and provide any recommended changes for approval.

BACKGROUND ON THE ARTHUR ANDERSEN METHODOLOGY:

In an SRM dated June 28, 1996, the Commission directed the staff to assess the SMM process and evaluate the development of indicators that can provide bases for judging whether a plant should be placed on or deleted from the Watch List. In response to this request, the staff contracted with Arthur Andersen Consulting to perform an independent assessment of the SMM process. The contractor's report was issued in December 1996.

Following the January 1997 periodic briefing for the Commission on operating reactors and fuel facilities, the Commission issued SRM M970129A. In the SRM, the Commission requested that the staff initiate the actions described in the Purpose section, above. The staff's actions in response to SRM M970129A related to the Arthur Andersen methodology are described below.

DISCUSSION OF THE PEER REVIEW OF THE ARTHUR ANDERSEN METHODOLOGY:

The trend plots presented in the Arthur Andersen report were intended to illustrate that quantitative data could be used in a systematic way to indicate trends in plant performance. Arthur Andersen Consulting recognized the approximate nature of the method, and did not intend that their trend plots be used to evaluate the performance of operating reactors. The Arthur Andersen report recommended that NRC formalize a methodology based on a thorough review of available performance indicators. Since issuance of SRM M970129A, the staff has pursued a program to develop a validated NRC Trending Methodology. On April 2, 1997, the staff forwarded its plans to address the recommendations made by the Arthur Andersen Company in SECY-97-072, "Staff Action Plan to Improve the Senior Management Meeting Process," and the staff subsequently briefed the Commission. As indicated in SECY-97-072, several of the improvement actions had already been initiated.

In response to the briefing on staff plans, the Commission issued SRM M970424B, dated June 24, 1997. In it, the Commission approved the staff's plans to improve the SMM process but directed the staff to critically evaluate the proposed approach, assumptions, etc. In addition, SRM M970424B directed that the Arthur Andersen algorithm, or any other performance algorithm developed by the staff, should be peer reviewed, benchmarked, and trial-tested before being relied upon for making assessments or comparisons.

The staff has factored the direction provided by the Commission in the February and subsequent SRMs into current plans and activities. For example, the NRC Trending Methodology will be peer reviewed in detail by the Advisory Committee for Reactor Safeguards (ACRS) and will be published for industry and public review and comment.

The staff is moving ahead with development of the improved NRC Trending Methodology and plans to validate the proposed methodology as part of the trial testing and peer review activities as directed by the June 24, 1997, SRM (M970424B). Consistent with this additional guidance, the staff has not gone back to compare the trend plots that were presented for illustrative purposes by Arthur Andersen with the results of the January 1997 SMM since the staff expects there to be substantial differences between the initial and final trending methodologies. Additionally, the staff proposes to consider whether the NRC Trending Methodology is best utilized in the screening meetings or the SMM as it develops plans to implement the final, validated methodology in the SMM process.

For record purposes, the staff considers the activities requested by SRM M970129A to be subsumed by actions associated with implementing Commission direction in response to SRM M970424B. Future periodic update reports to the Commission on progress of the peer review and analyses of the NRC Trending Methodology will be made in response to SRM M970424B.

BACKGROUND ON USE OF TRENDING LETTERS:

In 1992, the Executive Director for Operations directed the Office of Policy Planning (OPP) and NRR to conduct a broad review of the reactor inspection program. The results of this review were summarized in OPP-92-01O, "Assessment of the Reactor Inspection Program." One of the issues identified by this review was that senior licensee managers indicated they would have initiated robust actions to remedy poor performance if they had known the NRC was considering their plant as a potential candidate for the Watch List. After being briefed on the results of the assessment, the Commission issued SRM M9304232 in which it suggested that the staff begin notifying the utility corporate president or board of directors when a plant has been identified at a SMM as trending towards placement on the Watch List. Following the June 1993 SMM, the Commission issued its first trending letter.

During the past several years, the Commission and the staff have evaluated the issue of the advisability of continuing the use of trending letters. In an SRM dated February 14, 1994, the Commission requested feedback from the staff on the use of trending letters and a recommendation as to whether the practice should be continued, modified, or terminated. During a periodic briefing for the Commission on operating reactors and fuel facilities in June 1994, the Commission expressed the concern that another performance category may have been inadvertently created with the use of "followup trending letters⁽¹⁾." The Commission was also concerned about how long a plant might be considered to be trending. The staff indicated that more experience was needed before a final recommendation could be made.

After gaining additional experience, the staff responded to the February 1994 SRM through issuance of SECY-95-085, "Recommendations on the Senior Management Meeting Process for the Continued Use of Trending Letters and for Plants that Remain on the Problem Plant List for an Extended Period." In SECY-95-085, the staff concluded that, based on the objectives of trending letters, the experience in implementing them, and the results obtained, the use of trending letters had achieved favorable results.

In the SRM that responded to SECY-95-085, the Commission directed the staff to use the trending letter only once, upon the detection of a declining trend, and discontinue the routine use of followup trending letters. The staff implemented changes in the SMM process to reflect the Commission guidance on use of trending letters. This change is reflected in the staff's current practices, as described in Management Directive 8.14, "Senior Management Meeting."

Following the January 29, 1997, Commission briefing on operating reactors, the Commission directed the staff (in SRM M970129A) to consider the advisability of reissuing adverse trending letters at each SMM, as well as issuing trending letters between SMMs (or some other defined evaluation interval) in warranted situations. The results of the staff's evaluation are discussed below.

DISCUSSION OF THE USE OF TRENDING LETTERS:

The staff has reviewed its current and past practices regarding the issuance of trending letters. As indicated above, the original intent of issuing trending letters was to provide an early warning to licensees identified by the NRC as trending towards placement on the Watch List.

Our experience indicates that the use of trending letters has been effective. For those plants with trending letters currently in effect, licensees have begun to take actions to address the issues identified. Similarly, licensees that received trending letters previously have responded positively and initiated corrective actions. In almost all instances, licensees that were issued trending letters reversed the negative trends, thereby preventing placement on the Watch List.

While the staff believes that the practice of issuing trending letters should be continued, there are no compelling arguments that would indicate that it is better to provide a single trending letter, as is current practice, or to reissue trending letters after each SMM they are left in effect. In addition, the use of trending letters will likely be impacted by the overall direction the NRC chooses as a result of the ongoing integrated review of the NRC assessment process. Therefore, in the interim, the staff recommends that the current practice be maintained.

The staff also considered the advisability of issuing trending letters between SMMs in warranted situations. In concept, issuing trending letters between SMMs, would allow the agency to provide earlier notification to licensees regarding adverse performance trends. In deciding whether to issue trending letters between SMMs, the following points must be considered:

• The NRC currently has a variety of tools that are used to communicate the concerns of NRC managers regarding plant performance to utility managers. These include, but are not limited to, inspection reports, management meetings and enforcement conferences. The tools ensure that the agency can take action to discuss emerging issues and trends with licensee's to ensure the earliest possible recognition and corrective action.
  
  
• Historically, changes in plant performance have been gradual. The likelihood that a plant will experience a sudden adverse trend in licensee's performance that would warrant SMM action in the intervening 4-6 months before the next SMM is remote.
  
  
• The issuance of trending letters is intended to be a SMM action. To issue trending letters between SMMs the staff would need to revise its existing assessment processes to identify possible candidates for this expedited attention based on routine/ongoing performance insights and provide opportunity for agency senior manager review, deliberation, and consensus before a decision is made. This process would be redundant to existing processes (e.g., plant performance review, SMM screening, SMM) and would result in an increased administrative burden on management and staff without commensurate benefit.
  
  
• Changes to make the process less burdensome, such as permitting issuance of the trending letter by the regional administrator without agency wide involvement achieved through processes like the SMM, would potentially result in greater inconsistency between regions and would potentially diminish the perceived impact of the letter.

Based on the above, the staff does not believe that issuance of trending letters between SMMs is advisable.

As mentioned above, the staff is conducting an integrated review of its various assessment processes. The results of this review along with ongoing improvements to the SMM process may have implications for the future use of trending letters. The staff will make additional recommendations regarding the specific practice of issuing trending letters as part of the changes that will be proposed to the Commission for approval.

SECY NOTE: TO BE MADE PUBLICLY AVAILABLE IN 5 WORKING DAYS FROM THE DATE OF THIS PAPER

1. At that time, trending letters were reissued to plants for each SMM they were left in effect. These are referred to as "followup trending letters."