Search Options
Index | Site Map | FAQ | Facility Info | Reading Rm | New | Help | Glossary | Contact Us

SECY-98-123

June 1, 1998

BACKGROUND

In response to a draft circular from the Office of Management and Budget dated August 3, 1994, the ACRS and ACNW each reviewed its planned activities and developed performance measures and assessment standards. These were provided to the Commission in a February 14, 1995 memorandum. Subsequently, as part of the agency's strategic assessment, an issue paper was developed on independent oversight (COMSECY-96-028, Strategic Assessment Issue Paper: Independent Oversight [DSI-19]). In a staff requirements memorandum (SRM) of August 21, 1996, associated with that COMSECY, the Commission requested each Committee "to produce a set of criteria, for Commission consideration, under which the performance of the committee would be evaluated in the future. The committee should then periodically review itself against these criteria and provide the results of this evaluation to the Commission."

The Executive Director of the ACRS/ACNW Office responded to this direction in a memorandum dated December 23, 1996. Subsequently, the agency announced the development of an agency-wide Strategic Plan that included a requirement for each office to prepare an Operating Plan. Because the requirements of the Operating Plan overlapped those of the SRM, the Executive Director of the ACRS/ACNW Office and members of his staff met with representatives of each Commissioner's office to discuss performance criteria that would meet both the requirements of the SRM and the Strategic Plan initiative.

It was agreed that the ACRS and the ACNW would each prepare an Operating Plan that would include self-assessment measurements and other suggestions contained in the SRM.

OPERATING PLANS

Both Committees have identified Priority Issues for 1998-1999. The Commission has received these Priority Issues and has commented on the ACNW priorities. The Operating Plans of both Committees list Planned Accomplishments for each issue, together with milestones and target dates for their completion. These Operating Plans are living documents that will be updated on a quarterly basis to reflect the actual activities of each Committee and the date each activity was accomplished. The first quarterly updates are now in progress. In addition, the Committees developed procedures for assessing performance that are described below under "Self-Assessment Measures."

SURVEYS AND RESULTS

Key measures of the performance of an NRC advisory committee include the timeliness, effectiveness, and quality of its advice. The last two attributes can be judged, to some degree, by monitoring the actions taken by NRC in response to the advice. However, it is sometimes difficult to evaluate the impact of the Committees' advice on their stakeholders. Therefore, a survey was designed to collect the current opinions of the stakeholders of the ACRS and ACNW and identify areas for improvement. Those surveyed included Commissioners and their staff members; the Executive Director for Operations; the Offices of Public Affairs, the General Counsel, and the Secretary of the Commission; Division Directors in all NRC Offices; Regional Administrators; and selected representatives of the industry and the public, including the states and other Federal agencies.

The survey was designed to measure opinions in three areas: written documents, presentations, and the capabilities of the Committees, their staffs, and their consultants. The overall results of the survey affirmed the important role of the Committees in contributing to the NRC's mission, with a large majority of the respondents agreeing that the Committees add value to the regulatory process. Areas for improvement were also identified. Some of the key results of the survey are noted below:

• A majority agreed that the Committees provide a forum for the public.
  
  
• The technical capability and professionalism of members and staff for both Committees were ranked high. Overall, consultants for both Committees also ranked high; however, certain ACRS consultants were rated somewhat lower.
  
  
• The majority of NRC staff and others participating in Committee reviews reported that the value added by the ACRS and ACNW review was worth the resources they expended for preparation and participation.
  
  
• The majority of those making presentations are satisfied with the facilities, the equipment, and the scheduling of presentations.
  
  
• Committee letters and reports on specific topics are read and used extensively by respondents, although other Committee products are used less frequently.

Complete reports of the results of each survey are included as Attachments 1 and 2 to this paper.

As a result of the survey, several areas for improving the performance of the Committees have been identified. The Committees will be examining ways to improve the value of their written products and the contribution of consultants and will implement appropriate changes.

SELF-ASSESSMENT MEASURES

The Committees have instituted procedures for reviewing their activities and monitoring their performance on a quarterly basis through an updated Operating Plan. In addition, letters and reports will be carefully evaluated to determine the quality, effectiveness, and timeliness of the Committees' advice. Letters and reports will be assessed to determine if the Committees' advice was technically sound, clear and concise, relevant, balanced and unbiased; if it reflected state-of-the-art knowledge; and if it addressed a risk-significant issue. The letters and reports will also be assessed to determine whether they were responsive to Commission needs; were considered in Commission and staff decisions; were incorporated into NRC policies, programs, and regulations; and were forward-looking. The assessments will be based on the following:

1. Documented evidence that the advice was accepted or adopted
2. Solicited feedback from stakeholders
3. Unsolicited feedback

Input for item 2 above will be solicited from members of the Commissioners' staff and the EDO, cognizant NRC staff members, outside groups, and the appropriate media representatives. More in-depth performance evaluations will be made by the Committees every 12 to 18 months.

SUMMARY

The ACRS and ACNW have, in response to the Commission's request, established goals for assessing each Committee's performance and have developed procedures for measuring the achievement of those goals. The ACRS/ACNW Office has surveyed stakeholders, identified areas for improvement, and is considering changes to increase the efficiency and effectiveness of both Committees. The results of the assessments so far have shown that both Committees appear to be cost-effective in terms of adding value to the regulatory process without unnecessarily burdening the NRC staff. The Committees will continue to monitor the efficacy of their operations and report periodically to the Commission.