IN RESPONSE, PLEASE
REFER TO: M970919C

October 24, 1997

MEMORANDUM TO: L. Joseph Callan
Executive Director for Operations
FROM: John C. Hoyle, Secretary /s/
SUBJECT: STAFF REQUIREMENTS - BRIEFING ON IMPROVEMENTS IN SENIOR MANAGEMENT ASSESSMENT PROCESS FOR OPERATING REACTORS, 1:30 P.M., FRIDAY, SEPTEMBER 19, 1997, COMMISSIONERS' CONFERENCE ROOM, ONE WHITE FLINT NORTH, ROCKVILLE, MARYLAND (OPEN TO PUBLIC ATTENDANCE)

The Commission was briefed by the NRC staff on the proposed improvements in the Senior Management Assessment Process for operating reactors.

The staff should focus additional attention on the decision-making process at the SMM. Prior to the next SMM, the staff should provide the Commission a paper describing how it intends to use information, such as performance trends and economic plots, to reach decisions at the SMM. The staff should also describe its plans for responding to the ACRS letter regarding the template and the SMM process in general.

(EDO) (SECY Suspense:  12/15/97)

In developing and evaluating changes to the Senior Management Meeting (SMM) process, the staff should consider what benefit the process and the watch list provide in terms of public health and safety. The staff should clearly indicate whether the SMM is the dominant driver of NRC's inspection, assessment, enforcement and regulatory processes. This information should be provided to the Commission prior to the next SMM.

(EDO) (SECY Suspense:  12/15/97)

In addition, the staff should provide its plans for additional early stakeholder interaction on assessing management effectiveness and utilizing economic indicators. Finally, the staff should provide an appropriate feedback mechanism to enable the staff to assess the reliability and validity of the improved SMM process.

(EDO) (SECY Suspense:  2/98)

In addition, the staff should provide a series of graphs that provide both false alarm and detection probabilities as a function of the observation period and various decision criteria. The graphs can be used to provide a basis for selecting appropriate decision criteria and duration period. The performance model/algorithm should be peer-reviewed by a broad audience of qualified individuals or organizations, including staff, and not limited to just ACRS.

(EDO) (SECY Suspense:  2/98)

The Commission would also be interested in knowing if the trend and regression models can be benchmarked against something other than decisions emanating out of past SMM processes. The staff should consider and evaluate the use of proprietary indicators in its assessment strategy, and inform the Commission of its conclusions. In this evaluation, the staff should provide a discussion of WANO's performance assessment process, including the significance of their "figure of merit," and the means by which it is derived and the performance indicators that are utilized, as compared to NRC's performance indicators and assessment processes.

(EDO) (SECY Suspense:  2/98)

The staff should inform the Commission of its plans regarding INPO's request that the NRC take a hard look at the INPO/WANO indicators. In its response, the staff should also discuss its past activities, experiences, and conclusions regarding its attempts to obtain, evaluate, and correlate proprietary performance data and indicators, such as those maintained by INPO, to the results of NRC's assessments, and of any current arrangements between INPO and NRC regarding the sharing of, or access to, performance data.

(EDO) (SECY Suspense:  2/98)

The staff should provide information regarding the resource commitments expended to-date on the improvement of the current SMM process, an estimate of the anticipated resources needed to complete this effort, and an estimate of the resources needed to maintain and refine the SMM process once it becomes final and is in-place. The resource commitment information should include data from both the headquarters and regional offices. The staff should continue to ensure intra- and inter-office involvement in the development of the SMM process. The degree of involvement by the regional offices should be provided to the Commission.

(EDO) (SECY Suspense:  2/98)

cc: Chairman Jackson
Commissioner Dicus
Commissioner Diaz
Commissioner McGaffigan
OGC
CFO
CIO
OCA
OIG
Office Directors, Regions, ACRS, ACNW, ASLBP (via E-Mail)
PDR - Advance
DCS - P1-17