[DOE LETTERHEAD]
September 4, 2001
The Honorable John T. Conway
Chairman
Defense Nuclear Facilities Safety Board
625 Indiana Avenue, NW, Suite 700
Washington, DC 20004-2901
Dear Mr. Chairman:
Thank you for your letter expressing the Defense Nuclear Facilities Safety Board's (DNFSB) concern about the approval of alternate methodologies in preparing documented safety analyses. I believe the Department of Energy (DOE) has been responsive to your concerns regarding the use and approval of alternate methodologies.
Specifically, the DOE Safety Management Functions, Responsibilities, and Authorities Manual (FRAM, DOE M 411.1-1B) was revised on May 22, 2001, and is now being revised in response to the planned organizational changes that were announced on July 26, 2001. Section 9.3.1 of the FRAM now requires Secretarial Officer review and approval, with Office of Environment, Safety and Health (EH) concurrence (or comment to the National Nuclear Security Administration), of alternate methodologies. This level of approval ensures that the approach to identify the appropriate safety structures, systems and components to mitigate or control hazards will be subject to a rigorous review at an early stage of development.
Additionally, DOE committed to review your concern regarding the 95 percent statistical methodology and to develop further 10 CFR Part 830 guidance on the review and approval of alternate methodologies. Enclosure 1 to this letter provides a report of our review of the 95 percent statistical method. Enclosure 2 provides the status of the actions we have taken, or are taking, to respond to your general concerns and specific issues raised in your April 10, 2001, letter.
I am confident these actions ensure the safety analyses for DOE nuclear facilities and provide the level of protection that is required by DOE nuclear safety policy and regulation. Richard Black (EH-53) will continue to keep your staff apprised of our progress in addressing this important nuclear safety issue of mutual interest.
Sincerely,
Spencer Abraham
2 Enclosures
Enclosure 1
Status of EH Action 1: The DOE Functions, Responsibilities, and Authorities Manual (FR4M) was revised to include a function and responsibility for the Office of Environment, Safety and Health (EH) to review and comment on the National Nuclear Security Administration (NNSA) applications and concur in the use of alternate methodologies for non-NNSA applications. The revised FRAM was submitted for review and comment through the Field Management Council. EH provided the revised FRAM to Defense Nuclear Facilities Safety Board (DNFSB) staff and has worked with your staff to resolve its comments. Both FMC and DNFSB staff comments were resolved and the revised FRAM has been published.
Status of EH Action 2: The interim Implementation Guide for use with 10 Code of Federal Regulations (CFR) Part 830, Subpart B "Safety Basis and Documented Safety Analysis," contains a provision that describes the process in which the Department of Energy (DOE) line organizations review and approve alternate methodologies with EH concurrence/comment. This Implementation Guide should be published as a final guide in the next month and is currently being reviewed by the DNFSB staff.
Status of EH Action 3: EH, along with DOE line organizations and DNFSB staff, reviewed the 95 percent statistical methodology at a workshop at Savannah River on March 15, 2001. Based on issues and concerns raised at the workshop as noted below, it is agreed that further work is required on the methodology before it can be considered for approval as a safe harbor in the development of a documented safety analysis under Part 830, Subpart B.
The following are our responses to the three additional issues identified in your April 10 letter:
DNFSB Issue 1: Provide merits of the proposed statistical methodology. EH and DOE line management have been reviewing the proposed methodology. The review to date includes a workshop discussion and an EH evaluation. The detailed EH findings are contained in the attached Office of Nuclear and Facility Safety Policy (EH-53) report. The report shows that the proposed methodology will require significant additional work before final judgment can be made. The methodology does not currently satisfy DOE's Nuclear Safety Management requirements in 10 CFR 830 for a document safety analysis.
DNFSB Issue 2: Indicate path forward. The path forward for completing the EH-led review and providing corresponding guidance consists of several actions as indicated in the EH-53 report. Currently the developer of the proposed statistical methodology is considering two options for the proposed methodology. The first option is to develop a revised topical report that addresses the shortcomings identified in the EH-53 report and the attachment to the Board letter of April 10, 200l. DOE will then carefully review this topical report and provide guidance before the methodology can be approved as a safe harbor for DOE documented safety analyses. The second option for the developer is to withdraw the proposal. We are awaiting a decision.
DNFSB Issue 3: Assure that the methodology will be approved prior to its use. On April 18, 2001, EH-53 transmitted its evaluation and report on the proposed methodology to EM-5. On May 3, 2001, the Acting Assistant Secretary for Environmental Management issued a memorandum forbidding the use of the 95 percent methodology for EM applications until further notice. Finally, as noted above, the FRAM was revised on May 22, 2001, to require review and approval of alternate methodologies.
Enclosure 2
OFFICE OF NUCLEAR AND FACILITY SAFETY POLICY (ONFSP) TECHNICAL
REPORT AND SUMMARY
PROPOSED 95 PERCENT STATISTICAL METHODOLOGY
The nuclear safety management rule (10 CFR Part 830) requires a contractor to develop a DSA for Category 1, 2, and 3 hazard nuclear facilities and have it approved by the Department of Energy (DOE). Table 2 to Appendix A of 10 CFR Part 830 establishes methodologies to develop a DSA that have been approved by DOE (the so-called safe harbors). A contractor may choose to use a different methodology, but its use must be pre-approved by DOE. Under the provisions of DOE G 421 .X-X and the Functions, Responsibilities, and Authorities Manual (FRAM), alternate methodologies to those of the 10 CFR Part 830 safe harbors for nuclear facility safety bases will require line management approval, with the Office of Environment, Safety and Health (EH) concurrence (or comment to the National Nuclear Security Administration [NNSA]).
Westinghouse Safety Management Solutions (WSMS) has proposed to use a so-called 95 percent statistical methodology in lieu of DOE-STD-3009 which is the established safe harbor for nonreactor nuclear facilities.
It was asserted that the WSMS proposed application of the methodology was made strictly to support the potential application of the methodology at SRS for the selection of Safety Class structures, systems, and components (SSCs). However, ONFSP noted that the general approach is very similar to that used at the DOE Richland Operations Office in the preparation of the DSAs (or Safety Analysis Reports) for the Hanford Tank Farm and the Cold Vacuum Drying Facility (CVDF).
That proposed application stimulated a letter from Chairman Conway of the DNFSB to the former Assistant Secretary of EH, Dr. David Michaels, dated November 1, 2000. The DNFSB letter expressed concern that the local approval authority could override DOE standards and guides for facility safety documents. Specifically, Chairman Conway commented:
"This methodology reduces the conservatism in the current DOE recommended approach by using a probabilistic combination of uncertainties or errors in calculating unmitigated consequences."
Unmitigated consequence calculations are done for the purpose of classifying safety systems important to public safety as Safety Class. Dr. Michaels replied to Chairman Conway's letter on December 20, 2000, saying that EH would propose, in a revision to the FRAM, that EH approval would be necessary for deviations from 10 CFR 830 safe harbor methodologies. Dr. Michaels also indicated that EH would work with the line organizations to assess and resolve any deficiencies in the statistical methodology. DOE has made the proposed revision to the FRAM that is consistent with a corresponding discussion of DOE approval in the 10 CFR 830 Implementation Guides.
In light of potential concerns regarding the proposed methodology raised by DOE and Defense Nuclear Facilities Safety Board (DNFSB) personnel, a workshop was held at the Savannah River Site (SRS) on March 15, 2001, for a presentation on the methodology and a general discussion on potential issues and concerns.
The workshop was arranged by the DOE Savannah River Operations Office (SR) and was attended by about 50 people from Westinghouse Savannah River Company, WSMS, DOE SR, and DOE Headquarters personnel. DNFSB staff (Bamdad Burns and Wayne Andrews) were also in attendance. In addition to the presentations made by WSMS on the methodology, a presentation was made on the application of the methodology for the Hanford Tank Farms by representatives from Office of River Protection and the DOE Richland Operations Office.
Prior to the workshop, a draft paper titled "Statistical Methodology in Safety Analysis," dated March 2001 was distributed by WSMS to provide preliminary information on the proposed application. At the workshop, presentations and a demonstration were made by WSMS on the methodology and its application. Briefly the methodology involves:
This is in contrast to the method described in Appendix A to DOE-STD-3009, which specifies the use of reasonably conservative values of each parameter. The draft paper suggests that use of the statistical methodology reduces calculated doses by a factor up to 600, depending on the specific circumstances. Although it was not shown during the workshop or in the draft paper, the presenters committed to provide the details of how these reduction factors were obtained. They also indicated that the factors would need to be redone based on newer information and with consideration of the questions that were asked during the workshop.
At the conclusion of the workshop, representatives from ONFSP presented their reaction and recommendation regarding this methodology. The following paragraphs expand upon these. While not a complete list of technical issues that the methodology needs to address, some of the more significant issues identified to date are provided below.
Inconsistent with approved methodology (3009).
It is quite clear that the methodology is not consistent with that of DOE-STD-3009, especially in the context of the proposed use in classifying Safety Class SSCs. Appendix A to DOE- STD- 3009 is quite explicit on the method to be used. It was made explicit in order to achieve uniformity throughout the complex with regard to Safety Class SSCs. The central difference is that STD-3009 specifies "that calculations be based on reasonably conservative estimates of the various input parameters," rather than on statistical distributions of limited data. One of the parameters (leakpath factor) must be set to unity per STD-3009. The proposed methodology would permit use of a statistical distribution of this factor.
Unwarranted extrapolations of limited experimental data.
DOE-STD-3010, which was referenced as being the primary source for accident parameters for this methodology, specifically cautions against using distributions of very limited experimental data. In order to achieve some sort of uniformity and repeatability, a well-defined protocol and criteria needs to be established that would be used to obtain and qualify a statistical distribution for use in the methodology. This protocol and the criteria need to address the uncertainties associated with individual data points, confidence levels associated with a set of data, the amount of data needed to define a distribution, and so forth. There is no indication that the proposed methodology has addressed the issue of uncertain data.
Lack of compatibility of the methodology with the Evaluation Guideline (EG).
The use of the proposed methodology is to compare a calculated accident dose with the EG for designation of a Safety SSC as Safety Class. The EG for Safety Class SSCs of 25 rem was selected in the context and understanding of the conservatism inherent in the methods of calculating unmitigated accident doses described in DOE-STD-3009. The proposed methodology, it is claimed, would reduce calculated doses by up to a factor of 600. That being the case, the proposed methodology needs to address the issue of whether the EG needs to be adjusted to provide the same level of assurance of safety for the public as would use of DOE- STD-3009 methodology.
Path forward.
It is apparent that the draft paper "Statistical Methodology in Safety Analysis," dated March 2001, does not include all the development work done to date. It is also apparent that further work remains to be done to provide complete documentation of the methodology sufficient for review. It has been recommended to Westinghouse that a topical report be developed that would provide the basis for a review for acceptance of this methodology as an additional safe harbor for the 10 CFR 830 nuclear safety basis requirements in a generic sense.
It is possible that the proposed methodology has some useful application in helping to understand the relative importance of various alternative accident mitigation strategies. Its role, in relation to prevention strategies, needs to be described more fully. Also, the limitations on use of the methodology need further development.
At the conclusion of the workshop, the presenters developed a list of "needs" for additional information that included the development of a side-by-side comparison of the margins in the DOE-STD-3009 methodology and the proposed statistical methodology; development of a protocol; characterization of benefits; description of the process as related to hazard analysis, accident analysis, control selection, and Technical Safety Requirement development; and a peer review, among others. These "needs" and the issues discussed in this report must be addressed in order for this proposed methodology to be approved by DOE as an additional safe harbor for the preparation of DSAs to meet the requirements of 10 CFR 830, Subpart B.