Download WordPerfect File [TR97J07A.WP5]

DEFENSE NUCLEAR FACILITIES SAFETY BOARD

January 7, 1997

1. Purpose

   This report documents a review conducted by the Defense Nuclear Facilities Safety Board (Board) staff of the integrated Department of Energy (DOE)/laboratory "Feedback and Improvement" core function at the Department of Energy Albuquerque Operations Office (DOE-AL), Kirtland Area Office (KAO), and SNL. The staff review evaluated this core function at SNL's defense nuclear facilities within the context of SNL's institutional Integrated Safety Management System, with a primary focus on the ongoing Pilot Environment, Safety, and Health (ES&H) Oversight Program (ES&H Pilot). The review was performed during December 10-12, 1996, by Board staff members J. Preston and J. Deplitch, and outside expert R. Lewis (System Planning Corporation).

2. Summary

   Midway through the 2-year SNL ES&H Pilot, DOE-AL and KAO do not have adequate assurance of overall ES&H performance at SNL, although such assurance exists for SNL's defense nuclear facilities at Tech Area V. The intent of the ES&H Pilot is to demonstrate that DOE's overall assurance can be derived from three integrated mechanisms: (1) SNL line management and internal independent ES&H assessment results, plus teaming of DOE subject matter experts (SMEs) with SNL personnel in these assessments; (2) day-to-day on-site monitoring by KAO Facility Representatives (FRs); and (3) SNL's reporting on performance objectives and indicators. DOE-AL was not scheduled to conduct an annual on-site oversight appraisal as part of the SNL ES&H Pilot, in contrast with the pilots at the two University of California weapons laboratories.

   However, the SNL ES&H institutional self-assessment program is still in the very early stages of development, and the cooperative teaming with DOE SMEs is not taking place to the extent planned. The KAO FRs are not routinely provided necessary access to internal SNL ES&H assessment results, and the KAO FR program is currently experiencing some internal difficulties. Finally, the still-evolving performance objectives and indicators are defined at a very high level; thus they do not provide sufficiently detailed information to support DOE oversight.

   The defense nuclear facilities in Tech Area V have undergone annual ES&H assessments, management surveillances, and regularly scheduled operational assessments by the cognizant line management organization, Division 9000. The KAO FRs for Tech Area V appear to be effectively integrated into the day-to-day operations. The results of the SNL surveillances and assessments have been documented, and deficiencies, issues, and comments appear to have been logged and tracked to closure. However, the ES&H issues and lessons learned at Tech Area V do not appear to have been reviewed institutionally or disseminated throughout SNL.

3. Background

   The aggregate of contractor self-assessment and DOE oversight activities constitutes a vital part of the "Feedback and Improvement" core function of integrated safety management. The conduct of the ES&H Pilot that was a primary focus of this review was directed by the Deputy Secretary of Energy, Charles B. Curtis (in a memorandum dated August 31, 1995), primarily in response to the "Galvin Report." This initiative was intended to determine whether DOE's on-site appraisal activities (and their associated "burden") could be greatly reduced via increased dependence on the results of the weapons laboratories' self-assessment programs.

   The duration of the SNL ES&H Pilot is 2 years (fiscal years 1996 and 1997). Both pilots at the University of California weapons laboratories had a duration of at least 1 year (fiscal year 1996) and included an annual 2-week on-site ES&H appraisal led by the cognizant DOE Operations Office. The SNL ES&H Pilot, as proposed, was intended to "push the envelope" by not requiring the on-site DOE appraisal.

   Within the DOE complex, the principles of a self-assessment program are derived primarily from DOE G 414.1-1, Implementation Guide for Use with Independent and Management Assessment Requirements of 10 CFR Part 830.120 and DOE 5700.6C, Quality Assurance, dated August 1996. This guide addresses independent and management assessments at the process, system, and program levels. Based on this guide, the associated DOE Order, and the Board staff's review experience, criteria for review of self-assessment programs have been identified and were applied during the present review (see the attachment to this report).

   Criteria for judging the overall adequacy of integrated DOE and contractor "Feedback and Improvement" programs are still evolving within DOE. The ES&H Pilot used the performance measures that are currently incorporated in the laboratories' contracts with DOE as a starting point for assessing the effectiveness of the contractor's overall ES&H program.

4. Discussion/Observations

   As noted above, the SNL ES&H Pilot is based on the integration of three mechanisms: (1) SNL ES&H self-assessments (with DOE SME teaming), (2) KAO FR monitoring, and (3) performance objectives and indicators.

   SNL ES&H Self-Assessment Program. The staff made the following observations with regard to the SNL ES&H self-assessment program:

   • The SNL institutional self-assessment program is at the early stages of implementation. SNL has not yet defined or implemented an institutional-level self-assessment program. One year into the ES&H Pilot, SNL lacks an adequate organizational process for reviewing, assimilating, and communicating deficiencies and lessons learned throughout the site and does not routinely determine root causes of deficiencies.

     A detailed project management approach is being followed to put a self-assessment program (called the ES&H Oversight Project) in place. The scheduled start for most tasks was October 1, 1996, a year after the start of the SNL ES&H Pilot (and suspension of periodic DOE on-site oversight appraisals). Completion of the ES&H Oversight Project is scheduled for September 30, 1997; however, this seems optimistic.

     Progress to date has been measurable, but not noteworthy. The implementation of the program has been slowed not only by the size and diversity of SNL facilities, but also because efforts are being made to ensure that the self-assessment program will be consonant with the philosophy and requirements of SNL's Recommendation 95-2 compliant Integrated Safety Management System. SNL's current schedule for completion of their institutional self-assessment program focuses only on initial implementation and does not seem to reflect sufficient "shake-down" time.

     The Board staff was briefed on a recent reorganization of an ES&H support division. The creation of the 7500 ES&H Center was intended to consolidate the support function so as to begin promoting some institutional-level consistency. This Center has begun working to improve the integration of ES&H professionals into the various line management organizations through the creation of dedicated teams. These teams will provide day-to-day ES&H support, while being linked to the Center to facilitate improved dissemination of issues and lessons learned. The ES&H Center may improve institutional self-assessment at SNL; however, much work is needed.

     
     
     
   • Adequate DOE teaming in existing SNL ES&H self-assessments is not apparent. SNL management does not seem to have fully accepted the concept of DOE SMEs teaming on those self-assessments being conducted. The participation of DOE SMEs was actually limited to two or three assessments, which were conducted as very structured interactions characterized as "prototypes." The anticipated widespread DOE access to and participation in line management and independent self-assessments has not been granted by SNL management, despite considerable pressure from DOE-AL/KAO. SNL's resistance has reportedly been based, in part, on a few experiences in which DOE SMEs used their access to conduct independent evaluations, rather than fully participating in the team assessment. In any case, the lack of teaming opportunities has limited accomplishment of the primary objective of this element of the SNL ES&H Pilot—for DOE SMEs to develop confidence in SNL's self-assessment ability by direct observation.
     
     
   • The Tech Area V defense nuclear facilities appear to have an effective line management self-assessment/DOE oversight system in place. Self-assessments are taking place at Tech Area V in the form of scheduled, completed, and documented line management and ES&H functional area assessments. The assigned DOE FR who was interviewed was knowledgeable, well integrated with the facilities, and regarded as an asset to the facility management. Noted deficiencies and action items were logged and tracked to closure. Although not all of the practices were formalized and the results did not seem to have been disseminated to the rest of SNL, the practices reviewed at Tech Area V could serve as a model for the development of line management self-assessment programs for other facilities at the site.
     
     
   • Management surveillance (walkthroughs) appeared to be conducted regularly and effectively at the defense nuclear facilities in Tech Area V. Management surveillance is required and described by an ES&H standard operating procedure in the SNL ES&H Manual. All line managers, from the SNL President to department managers, are required to conduct these surveillances, accompanied by an ES&H SME. Reports for surveillance at defense nuclear facilities in Tech Area V were found for all levels of management, from the department manager to the SNL Executive Vice President, at the appropriate surveillance intervals. Additionally, operational supervisors performed scheduled assessments on a monthly basis, maintained a notebook of all completed assessments, and maintained a log of all noted deficiencies and observations to track them and ensure closure. Recommendations from a variety of SNL safety committee reviews are maintained in an action items database to ensure closure.
     
     
   • The SNL "Feedback" program is operating at a very basic level. SNL uses various vehicles to communicate to its population the most significant lessons learned (such as those associated with severe accidents on site or at other laboratories); however, less significant events are not always well disseminated. The effectiveness of the vehicles used is as yet unmeasured. The integration of lessons learned into activities, analyses, and future self-assessments is also not fully developed.

   Facility Representatives/Kirtland Area Office Monitoring. The review included interviews with the KAO operations manager (FR program manager), the FR supervisor, two FRs, and the KAO health physicist. The staff made the following observations with regard to the FR/KAO monitoring:

   • The assessment system, the quality of the assessments, the findings database, and the presently assigned FRs appeared adequate. Assessments were scheduled on an annual calendar and conducted as scheduled, and the findings were entered into the Kirtland Issue Management System database for tracking and analysis. This database allows manipulation of the data to support a variety of analysis objectives. A Conduct of Operations Attributes Checklist had recently been developed to assist the FRs in conducting assessments. This checklist includes numerous subtopics for investigation, each with definable criteria, recommended sample size, and expectations, as appropriate.
     
     
   • There appears to be some resistance within SNL to the FRs' role. Although day-to-day monitoring by the KAO FRs is a defined element of the SNL ES&H Pilot, SNL personnel seemed surprised and somewhat dismayed that the KAO FR on-site presence had increased. While the Tech Area V FR interviewed seemed well integrated with the facilities he oversees, it was reported to the staff that such acceptance is not necessarily universal.

     Not all FRs are given visual access to line management self-assessment results; no one from DOE is ever allowed to take away a copy of these assessments. SNL justifies this by saying they feel people would not be as cooperative during assessments if they knew the results were to be shared outside their division. SNL representatives expressed concern over release of assessment results to DOE before the entire senior management chain (including the Executive Vice President) has been fully informed.

   • There is a lack of staffing/management stability in the FR program. There have been four different FR supervisors in the past 2 years. Several of the FRs were newly designated in the past year and are in various stages of training and qualification. Three of the most experienced FRs will be lost in January 1997. The KAO FRs have been selected for high-visibility ancillary responsibilities, such as lead for KAO Recommendation 95-2 implementation and manager of Price Anderson Amendment Act activities, as well as education and training. These opportunities, while career-enhancing, take the FRs away from day-to-day monitoring activities.

   Performance Objectives and Indicators. The initial set of objectives and indicators was not sufficiently developed to assure DOE of adequate SNL performance. Objectives were aimed at general criteria, such as fatalities and accidents. Indicators, such as injury rate, lost workdays, and regulation compliance, were not detailed enough to identify and evaluate safe practices and activities, or to be useful as a real feedback and improvement aid. As an interim result of the ES&H Pilot, DOE-AL/KAO and SNL have learned that the existing performance measures are generally too high-level to be useful for either self-assessment or oversight. Much additional work is needed to identify indicators that will predict performance problems.

   Discussions with Senior DOE-AL/KAO Management. The problems with the SNL institutional self-assessment program and with the execution of the ES&H Pilot are known to DOE-AL and KAO. These problems led to a meeting between top KAO and SNL management just prior to the Board staff's review visit, during which it was agreed that DOE needed an additional mechanism to provide assurance of SNL ES&H performance. There is some indication that, as a result of this DOE/SNL meeting, a 2-week on-site DOE appraisal may be added to the SNL ES&H Pilot. This possibility was revealed during a Board staff discussion with DOE-AL, KAO, and the SNL Executive Vice President on the last day of the review.

5. Future Staff Actions

   The Board staff will perform follow-up reviews to monitor the SNL ES&H Pilot, as well as the effective implementation of SNL's Integrated Safety Management System and institutional self-assessment program. The staff will incorporate the findings of its SNL reviews into future reviews of the integrated DOE/contractor "Feedback and Improvement" programs within the DOE defense nuclear complex.

Attachment
Self-Assessment Program Review Criteria

• Is there a process/protocol for assessments?
  
  
• Are there site-wide standards of performance that are useful for both line and cross-site independent appraisal groups?
  
  
• Is the conduct of assessments adequate and thorough, including objectives or focus and specific criteria?
  
  
• Are results documented, with weaknesses identified?
  
  
• Are root causes evaluated for those weaknesses that are identified?
  
  
• Is there status feedback to management, the responsible office/agency, and stakeholders?
  
  
• Are weaknesses tracked and closed?
  
  
• Are facility-specific and site-wide results evaluated to identify trends?
  
  
• Is the effectiveness of the assessment program reviewed by both line and independent management (using metrics)?
  
  
• Are lessons learned evaluated and disseminated?