[DNFSB LETTERHEAD]

October 2, 2001

The Honorable Jessie Hill Roberson
Assistant Secretary for
Environment Management
Department of Energy
1000 Independence Avenue, SW
Washington, DC 20585-o 104

Dear Ms. Roberson:

The Hanford Tank Farms contractor's Integrated Safety Management (ISM) system was declared implemented in June 2000, although a number of areas needing improvement were noted by the review team. One of these was the program for self-assessment, feedback, and improvement. In February 2001, a comprehensive self-assessment by CH2MHill Hanford Group (CHG) found similar deficiencies in feedback and improvement. This has been followed by a number of other reviews and/or assessments which resulted in similar findings. Foremost among these is an independent assessment completed in April 2001 by the Department of Energy (DOE) Office of Environment, Safety, and Health (EH-2). To date, the corrective action plan (CAP) for the EH-2 assessment has not been put in place. A recent Defense Nuclear Facilities Safety Board (Board) staff review revealed that aspects of CHG's feedback and improvement program remain weak. The enclosed report prepared by the Board's staff summarizes these issues and is provided for your consideration in evaluating the CAP for the EH-2 assessment.

Additionally, CHG's revised program will likely be significantly different from the ISM system that was verified. Considering the problems identified in CHG's feedback and improvement programs and the magnitude of the completed and planned modifications, DOE may deem it necessary to complete a focused ISM System Verification Review for the revised program. Such action would be consistent with the guidance contained in DOE G 450.4-lB, Integrated Safety Management System Guide, in order to provide confidence that the revised ISM system is adequate. The Board would like to be kept informed of CHG's progress in making these improvements and DOE's review of their adequacy.

Sincerely,

John T. Conway
Chairman

c: Dr. Harry Boston
Mr. Mark B. Whitaker, Jr.

Enclosure

DEFENSE NUCLEAR FACILITIES SAFETY BOARD

Staff Issue Report

September 4, 2001

This report documents a review of the feedback and improvement programs of CH2M Hill Hanford Group (CHG). This review was conducted by M. T. Sautman of the staff of the Defense Nuclear Facilities Safety Board (Board) and outside expert R. Lewis. The review examined the implementation of the contractor assessment, lessons learned, tracking and trending, and post-job review programs.

Feedback and Improvement Programs. At the time of this review, CHG was in the early stages of a major effort to improve its feedback and improvement programs. The Independent Performance Evaluation (IPE) performed last winter and other contractor and Department of Energy (DOE) evaluations of CHG's feedback and improvement programs had revealed a number of significant weaknesses in the programs' content and implementation. Many of the processes developed in response to the IPE had not yet been fully defined, and given the magnitude of the changes envisioned, it will be several months before the new processes will be formalized and implemented. Complicating matters, CHG is attempting to develop a rigorous corrective action plan to address the IPE findings and measure the effectiveness of the corrective actions while simultaneously trying to address major weaknesses in its programs for corrective action management and tracking/trending. It was encouraging that the significance of the weaknesses had been recognized by CHG management, who displayed a high degree of commitment to improving the programs. In July, CHG issued its Performance Enhancement Plan (PEP), which outlines the initiatives to be implemented to address the IPE findings, the expected results, and measurement standards.

Corrective Action Management--CHG's IPE revealed that issues often remain unresolved and that similar findings are repeatedly identified. It also noted that corrective actions identified as a result of assessments are slow to be implemented, are not directed toward resolving causal factors and generic implications, are not being adequately verified as having been completed, and do not include features to monitor for desired effects. The Quality Assurance organization performs an independent verification of the corrective actions for issues with the highest two (of five) risk rankings and reviews the effectiveness of corrective actions for issues with the highest risk ranking. Considering the weaknesses identified in the IPE, more emphasis is needed on reviewing the closure of corrective actions and determining their effectiveness. Although current indicators show a dramatic decrease in the number of delinquent corrective actions, this apparent improvement coincided with a push by management to grant extensions to completion dates for corrective actions. During discussions with management, it was stated that this granting of extensions was encouraged to minimize overdue actions, that there were no criteria for determining whether an extension should be granted, and that requests for extensions were never rejected. This weakness is recognized by CHG, which plans to start tracking extensions in the future to address the problem. The Management Plan in the PEP outlines several actions aimed at addressing these past weaknesses with closure of corrective actions.

CHG recently instituted the use of Problem Evaluation Requests (PERs), a zero-threshold problem-reporting system. PERs are used for compliance as well as other problems, which are ranked as to their significance. In the past, deficiencies outside of the compliance arena were generally excluded from the deficiency evaluation process. After a large initial flux of PERs, the number of PERs being generated daily has dropped off to a more manageable level. During the remainder of 2001, PERs will be integrated into nearly every program that identifies deficiencies within tank farms. CHG is also chartering a Corrective Action Review Board, comprising key senior managers, to evaluate the scope and depth of the analysis of significant PERs, as well as the effectiveness of their proposed resolutions.

Assessment Program--DOE P 450.5, Line Environment, Safety, and Health Oversight, and the Quality Assurance rule discuss the requirements for an effective contractor self-assessment program. CHG's Assessment Program procedure provides guidance on the planning and conduct of independent, management, and worker assessments. However, there are few requirements defining the scope and number of independent assessments to be performed. CHG's IPE revealed that CHG has not completed implementation of an Independent Assessment Program. At this time, CHG does not really have an independent assessment organization. While there is a small assessment group, its mission is to perform readiness reviews and compile an assessment schedule rather than to conduct assessments.

The Board's staff reviewed the assessment schedules and assessment reports generated during the last 6 months. Most of the assessments were compliance-driven, focusing on radiological protection and quality assurance. Other than the IPE performed a few months ago, there were few independent assessments of other programs. Most notably absent were assessments of the ongoing construction projects in support of high-level waste feed delivery with regards to topics other than quality assurance. In addition, few of the assessment reports contained any findings, and the findings that were included were often administrative issues that were quickly fixed. This raises questions about the thoroughness of the assessments and the assessors' training. CHG recognizes this issue, and plans to provide more training to its assessors and possibly certify some of them. The PEP identifies other initiatives for upgrading the assessment program, such as establishing an independent assessment program.

CHG's IPE also revealed that the Management Assessment Program has not been implemented consistently. Currently, management assessments consist of highly focused reviews of a particular topic, such as the implementation of a specific procedure requirement. On a monthly basis, the Senior Management Oversight Program (SMOP) requires managers to observe operations during an assigned shift, focusing on a particular topic (e.g., lock and tag) and documenting their observations in a log book. SMOP participants include a wide range of technical and nontechnical senior managers (e.g., Vice Presidents of Operations and Human Resources), some of whom have little or no background in conduct of operations. An additional program, the Management Oversight Program (MOP), requires line managers to observe a particular activity (e.g., a pre-job meeting) and then complete a short narrative and checklist. The MOP and SMOP observations are screened to determine whether any PERs need to be generated. In addition, the dozens of MOP reports are consolidated into a single report on a monthly basis.

A review of management assessment reports revealed that few contained any findings. For example, of the 82 MOP assessments conducted for single-shell tanks in April 2001, only 4 resulted in formal corrective actions; 17 others resulted in some form of on-the-spot correction. It was the staffs observation that the MOP reports tend to be quite limited in scope, informal, and fairly shallow. Other than combining them into a single report, there appeared to be no effort to assess the collective significance of the observations. As a result, the issues were being addressed in an isolated manner rather than being used as data points for evaluating the overall effectiveness of pre-job meetings or work control processes. Considering the limited scope of the independent assessments, the informal and shallow nature of the MOP and SMOP reviews, and the lack of a systematic analysis of the results of those reviews, it is not apparent that many programs and topics are subject to sufficient independent or management review. The greatest benefit of the MOP and SMOP appears to be increased management presence in the field.

A recent evaluation of CHG's assessment program conducted by the Office of River Protection led to similar conclusions about the MOP. The overall evaluation was that the assessment program had not achieved sufficient integration of environment, safety, health, and quality elements to effectively focus attention on potential areas of vulnerability. The assessment program was also found to lack sufficient focus on Integrated Safety Management (ISM), independence from responsible managers, and oversight for subcontractor activities. Low assessment completion rates were also noted for Nuclear Operations and other organizations.

Post-Job Reviews and Lessons Learned Programs--CHG procedures require that a post-work review be conducted on all field work, and the activity job hazards analysis (AJHA) post-job review (PJR) form completed for all work packages requiring enhanced planning. However, the staffs review of several dozen completed work packages showed no documentation of the performance of any post-work review for most packages. Nearly all the AJHA PJR forms had been checked off with no comments. The only exceptions were for high-visibility/high-risk activities that triggered an As Low As Reasonably Achievable (ALARA) PJR. These reviews were used to discuss all aspects of the activity (including work planning and nonradiological operations), evaluate the effectiveness of controls, and identify lessons learned. While the content of the reviews was good, the use of an ALARA PJR to carry out this function may not be the most appropriate mechanism. For example, the use of a form specifically designed for the job's radiological aspects to assess an entire job essentially force-fits the PJR into a form not designed to address other hazardous situations. In addition, an ALARA PJR is triggered only when certain radiological criteria are met. Thus, an in-depth PJR may not be performed for activities with significant nonradiological problems or lessons learned. Furthermore, construction activities that are performed by a subcontractor do not generate any PJRs unless they trigger an ALARA PJR.

It is unfortunate that the majority of the workforce is not taking the time to provide feedback since CHG is improving the way this feedback is disseminated. Copies of all AJHA PJR forms and formal lessons learned are easy to access on the local intranet. Linking PJRs to PERs would also ensure that identified issues are addressed. Thus efforts to encourage work planners to use this information in developing work packages need to continue. While there is anecdotal evidence that ALARA PJRs are referenced for upcoming major tasks that are highly similar to past tasks, it is not clear how often lessons learned are used for day-to-day work activities.

Summary. The staff believes CHG is making progress in addressing some deficiencies, but much work remains to be performed. Although the IPE was an encouraging exercise, CHG's routine independent assessment programs have several significant deficiencies. A large number of management assessments are being performed, but they do not appear to be identifying the significant issues that have required attention in the recent past (e.g., work control, hazard identification). A major source of feedback is untapped since a majority of the workforce is not taking the time to provide feedback in PJRs.

If fully implemented, the PEP should address some of the issues identified in this report. However, the PEP will not address them all, and several of the initiatives to resolve the widespread feedback and improvement issues are vague. These factors will limit the effectiveness of the corrective actions. In addition, implementation of the PEP should lead to a major modification to CHG's ISM System, which will require the completion of a revised system description. Considering the problems identified with CHG's feedback and improvement programs and the magnitude of the completed and planned modifications, DOE may deem it necessary to complete a focused ISM System Verification Review for the revised program, consistent with the guidance contained in DOE G 450.4-lB, Integrated Safety Management System Guide, in order to be confident that the revised ISM System is adequate.