[DNFSB LETTERHEAD]

March 25, 2002

The Honorable Everet H. Beckner
Deputy Administrator for Defense Programs
National Nuclear Security Administration
U. S. Department of Energy
1000 Independence Avenue, SW
Washington, DC 20585-0104

Dear Dr. Beckner:

On October 2, 2001, the Defense Nuclear Facilities Safety Board (Board) sent a letter to your office concerning the numerous incidents involving inadequate procedural, compliance at the Pantex Plant during calendar year 2001. Personnel from your office, as well as the Amarillo Site Office (ASO), and the Pantex Plant contractor, BWXT, have discussed this issue with the Board and acknowledged the importance of addressing it aggressively.

The Board's staff recently completed a review of the corrective action program by which BWXT is attempting to improve formal conduct of operations and procedural compliance at the Pantex Plant. While the staff found that progress has been made, it was clear that further action is required. The enclosed report is provided for your information and use, as appropriate.

The Board remains intensely interested in the resolution of this issue and will continue to follow BWXT's and ASO's progress closely.

Sincerely,

John T. Conway
Chairman

c: Mr. David E. Beck
Mr. Daniel E. Glenn
Mr. Mark B. Whitaker, Jr.

Enclosure

DEFENSE NUCLEAR FACILITIES SAFETY BOARD

Staff Issue Report

February 22, 2002

Background. On October 2, 2001, the Board sent a letter to the Acting Deputy Administrator for Defense Programs of the National Nuclear Security Administration (NNSA), U.S. Department of Energy (DOE) concerning numerous incidents involving inadequate procedural compliance at the Pantex Plant. Reference was made to occurrence reports for more than 50 such incidents at Pantex since January 2001, including missed steps, steps performed out of sequence, violations of procedural requirements, and use of wrong procedures. At least 10 of these incidents involved work directly on nuclear explosives. Based on the NNSA response, the Board directed its staff to gather more information relative to this issue.

The Board has stressed the importance of procedural compliance on numerous occasions. This position was specifically articulated in the Board's fourth annual report: "The health and safety of the public and workers rest on a properly trained workforce accomplishing tasks in a formal, deliberate fashion in accordance with reviewed and approved procedures. " Absent strict procedural compliance, the Production Technicians (PTs) are thrust into the role of determining which steps in a procedure must be performed as written to implement safety requirements, a role for which they are neither trained nor qualified. At the Pantex Plant, procedural compliance takes on added significance because the nature of the potential hazards and to the heavy reliance on administrative (procedurally implemented) controls in the site's authorization bases.

Summary. The Board's staff discussed in detail the corrective actions being taken to address procedural compliance issues with representatives of both the Amarillo Site Office (ASO) and the Pantex Plant contractor (BWXT). The staff also spent a significant amount of time observing nuclear explosive operations in Pantex bays and cells. BWXT has developed a series of initiatives to address the procedural compliance issue, and it was apparent that progress has been made in the area. However, the Board's staff identified procedural violations in four of six weapon programs observed, as well as further procedural violations during a maintenance operation.

It appears that the single most effective action taken by BWXT to improve procedural compliance is to require PTs to stamp off each step in a procedure as it is completed. However, several violations of this requirement were observed by the Board's staff. Most of the violations could be traced, in part, to poor procedures, and it is not clear that BWXT's efforts to improve procedures will be successful. Further, it appears that BWXT's self-assessment teams are not providing feedback adequate to guide effective corrective actions.

The procedural compliance problems that continue to be identified by BWXT's internal reviews and assessments, together with the observations of the Board's staff presented in this report, indicate that the issue identified in the Board's original letter has not been addressed effectively by ASO and BWXT. Many of the corrective actions being taken lack sufficient direction, rely on inadequately trained personnel, or are being completed hastily, rather than in a deliberate manner.

Discussion. ASO and BWXT have attempted to address the issue of procedural compliance at the Pantex Plant on three separate fronts: nuclear explosive operations, maintenance, and material moves. This report focuses on procedural compliance during nuclear explosive operations. Observations of the Board's staff on procedural compliance during maintenance activities and material moves is provided in Attachments 1 and 2.

Analysis of and Response to the Issue by ASO/BWXT--The Board's staff noted that neither ASO nor BWXT has performed a formal, comprehensive root-cause analysis of the issue. Rather, the root-cause codes entered into the DOE's Occurrence Reporting and Processing System for each individual event have been binned and counted as the means of identifying focus areas for corrective action. As a result, it is possible that additional root and contributory causes remain unidentified.

Management of BWXT's Operations Division has initiated a series of actions directed at the root causes identified in the occurrence reports. These actions have included: (1) instituting the new required method of placekeeping noted above (requiring PTs to stamp off each step in a procedure as it is completed); (2) scheduling senior Operations Division personnel to observe operations, as well as performing a series of external assessments of conduct of operations; creating eight new positions--Nuclear Safety Officers (NSOs)--whose responsibilities will include observing and reporting on conduct of operations; and (3) initiating improvements in the level of conduct of operations taught to PTs during training. Upon further investigation, however, the Board's staff identified flaws in each of these corrective actions that may preclude success, as delineated below.

Field Observations: Effectiveness of BWXT's Response--As noted, requiring the PTs to stamp off each step appears to be the single most effective action taken to enhance procedural compliance. However, this requirement has not been entirely effective. The Board's staff observed several instances of steps performed out of sequence and missed steps:

• In one program, a review of the last five completed procedures revealed that all five included steps that had been stamped off as complete even though they had not yet been performed.



• In another program, a step in a hoisting operation was performed out of sequence. The PTs subsequently tried to invoke another procedure that allowed the alternate sequence, but did not use it in the reader-worker-checker manner required of a procedure classified as "critical use. "



• In another program, a data recording step was missed, and two quality-related steps were not performed because the PTs assumed they had already been done.



• In another program, the procedure was constructed such that a series of steps was required to be performed more than once. Stampoff was not consistently required for these steps after the initial cycle, leaving no documentation that the steps had been accomplished more than once.



• The Board's staff observed that requirements in general plant procedures, which do not require stampoff (e.g., P7-0040, Combustible Material Controls), were violated in several of the facilities visited by members of the Board's staff. For one program in particular, essentially none of the requirements of P7-0040 were being enforced.

The pattern of procedural violations observed by the Board's staff is significant, and calls into question the reliability of all administrative controls invoked by procedure. Fortunately, none of the procedural violations observed by the staff involved a critical safety step, although the failures observed in implementing P7-0040 in this case represent a violation of a Technical Safety Requirement-level administrative control.

The Board's staff also observed that several of the procedural violations involving procedures for weapon operations occurred in sequences of steps that did not appear to be constructed in a logical fashion. In effect, the sequence of steps in the procedure made the probability of a procedural violation more likely.

Effectiveness of BWXT's Assessments--The Board's staff identified two shortcomings in the programs BWXT has instituted to identify procedural compliance issues:

• No formal guidance is available to focus the attention of senior managers in the Operations Division on procedural compliance or conduct of operations as they perform their scheduled observations of PTs at work. The initiative appears to be designed to obtain one review of each focus area in DOE Order 5480.19, Conduct of Operations Requirements for DOE Facilities, during the course of the year.



• A review of the schedule of future NSO observations did not reveal a focus on procedural compliance.

Dedicated observations of weapon program work by members of the Quality Assurance Independent Assessment Group (30 hours per week) did appear to be focused on procedural compliance. However, this activity is being phased out as the NSOs assume their responsibilities.

Effectiveness of B BWXT's Feedback and Improvement--The staff observed that feedback on identified procedural violations does not appear to be consistent:

• None of the procedural violations observed by the Board's staff and discussed in this report have been recorded in either the Occurrence Reporting and Processing System or the Non-Compliance Tracking System.



• After the Board's staff provided feedback from observation of the step performed out of sequence in the hoisting operation, BWXT's senior management resisted for 2 days categorizing the observed sequence of events as a procedural violation.



• Corrective actions to address an external assessment conducted by an Assess, Improve, Modernize Team in September-October 2001 have not yet been formulated. A baseline assessment of conduct of operations performed by the (not yet qualified) NSOs appeared to focus on the administrative aspects of conduct of operations, and no observations of procedural compliance were recorded.



• Improvements in training of PTs in conduct of operations remain to be implemented.

Future BWXT's Corrective Actions--In the longer term, BWXT's management recognizes the need to address inadequacies in the nuclear explosive operating procedures and has instituted a year-long program to upgrade all of those procedures. However, there is a large number of procedures: BWXT personnel estimated the number at roughly 500. Given the chronic shortage of skilled procedure writers and the historically ineffective process for procedure validation at the Pantex Plant, it is not clear that this program will produce the necessary improvements in the quality of procedures. Ultimately, ASO and BWXT plan to address this issue by instituting interactive electronic procedures (IEPs) for all nuclear weapon programs. However, a recent change in program direction may result in additional delays in initiating the IEP program.

Attachment 1

Procedural Compliance During Maintenance at the Pantex Plant

The key corrective action taken to date appears to be increased field presence by maintenance management personnel. The division manager has established a self-assessment goal for all maintenance supervisors, engineers, and planners (about 48 people), requiring one field observation per shift. This action has increased the level of direct feedback available to maintenance craft workers, as well as provided a conduit for craft workers wishing to highlight recurring errors in procedures. However, correction of procedural inadequacies continues to be a problem: 30 percent of roughly 3800 maintenance procedures have been reviewed for errors since December, and two-thirds of those reviewed required changes.

The large numbers of procedures being reviewed and changed in a short period of time, together with the questionable qualifications and training of the reviewers, raise some question as to whether the actions being taken to upgrade the maintenance procedures will be successful. Of particular concern is the adequacy of the feedback provided by BWXT maintenance managers. Based on a review of the database of self-assessment comments, it appears that few of these managers have experience in an environment of strict procedural compliance and conduct of operations. The documented guidance provided to personnel performing the self-assessments is inadequate to offset the inexperience of the managers involved. For example, no direct observations of the procedural compliance of craft workers appear to have been recorded; the division is only tracking inadequacies in work packages and procedures and procedure change packages. Under these conditions, it is not clear that adequate standards of procedural compliance will be set and maintained. In fact, it is possible that inadequate standards of compliance will be reinforced.

The above conclusions were verified while the Board's staff was observing a maintenance procedure undergoing self-assessment. A procedure violation was identified, and the assessor stepped out of his independent role and became a supervisor. As supervisor, he went beyond established plant directives to order restoration of the system, instead of providing a second check to the craft worker once he had restored the system. It also appeared that the potential feedback to be derived from the situation was not recorded on the self-assessment form.

Both BWXT's Facility Reliability Department and ASO's facility representatives intend to conduct detailed reviews of compliance with maintenance procedures in July 2002. However, these spot reviews will not have the impact of continuous, brutally honest feedback on procedural compliance, and effective corrective actions.

Attachment 2

Procedural Compliance During Material Moves at the Pantex Plant

BWXT management determined that the root cause of failure to comply with the procedures for material moves was that the system was too complex. To address this problem, BWXT has consolidated the Explosive Tracking Center and the Nuclear Material Control Center to form a centralized dispatch center. In addition, the dispatch center directs movements of nuclear explosives, radioactive materials, and bulk fuels, and tracks individual facility inventories against allowable limits. The transactions associated with material moves are now the responsibility of the material handler conducting the move, who coordinates all associated activities by means of a single phone call to the dispatch center. The material movement procedure has been rewritten to reflect this simplification. Operator aids have been distributed to allow rapid identification of movement requirements for different materials.

At present, deconflicting movements of different materials is a difficult task that is accomplished manually. BWXT intends to advance to a fully automated (computerized), rule-based material tracking system by August 2002. This is an aggressive timeline, however, especially given the fact BWXT personnel were unable to answer questions raised by the Board's staff with regard to whether such a system would have to be safety-class or safety-significant (since it implements an administrative control in a Technical Safety Requirement), what procurement specifications would be required to meet such a classification; and what software quality assurance requirements would then have to be invoked.

During observation of a material move, the Board's staff noted that PTs (who complete much of the paperwork associated with preparing material for a move) did not have in their possession the operator aids associated with the new material movement system. BWXT has experienced seven material move incidents (one reportable as an occurrence) since the Board issued its original letter on the procedural compliance issue at Pantex.