April 24, 2006; Board letter establishing a 30-day reporting requirement to address comments regarding the Draft Repackaging Prioritization Methodology relative to implementation plan 2005-1

[DNFSB LETTERHEAD]

April 24, 2006

The Honorable Samuel W. Bodman

Secretary of Energy

1000 Independence Avenue, SW

Washington, DC 20585-1000

Dear Secretary Bodman:

The Defense Nuclear Facilities Safety Board (Board) received the Department of Energy’s (DOE) letter dated March 30, 2006, enclosing the DOE repackaging prioritization methodology, which is a deliverable in the Implementation Plan (IP) for the Board’s Recommendation 2005-1, Nuclear Material Packaging. DOE committed to developing a repackaging prioritization methodology in response to sub-recommendation (3) of Recommendation 2005-1: “Prioritize implementation of the improved nuclear material packaging requirement consistent with the hazards of the different material types and the risk posed by the existing package configurations and condition.” Section 5.0 of the IP describes the resolution of this sub-recommendation as follows: “Based on the Department nuclear material risk profile, the Department will ensure that the highest priority items, as determined by the complex-wide risk ranking methodology will be qualified or repackaged first at all sites.”

The DOE letter describes DOE’s commitments under the IP for each site to prioritize repackaging or qualification of existing packaging based on risk, and for each site to development a resource-loaded schedule and funding plan for implementing the DOE Nuclear Materials Packaging Manual, without emphasizing the subsequent commitment to issue a “DOE-wide schedule for 2005-1 implementation.” The Board believes that such a DOE-wide schedule cannot meet the commitment made in Section 5.3 of the IP unless a complex-wide risk ranking methodology aimed at estimating the overall nuclear material risk profile is developed. As transmitted to the Board, the repackaging prioritization methodology cannot be used to develop an accurate nuclear material risk profile for the complex and thus does not meet the IP commitment. This deficiency was previously noted by the Board’s staff in its initial review of the draft repackaging prioritization methodology.

The Board’s staff has reviewed the repackaging prioritization methodology and found that none of its earlier comments on the draft document, provided in enclosure 1 to this letter, are addressed in the final version. In addition to those comments, the Board’s staff provided a detailed discussion, presented in enclosure 2 to this letter, which elaborated on the technical bases and justification for the comments. Comments 1-6 address technical deficiencies in the repackaging prioritization methodology that adversely affect DOE’s ability to fulfill the IP commitments cited above. The Board’s staff also provided suggestions for addressing specific deficiencies in the risk ranking methodology that, if adopted, would improve DOE’s ability to identify the highest-priority items at a complex-wide level.

The most serious deficiency in the risk ranking methodology involves DOE’s adoption of two inconsistent models for estimating package failure probabilities. Specifically, one of the allowed models does not account for the vulnerability of the package to known failure mechanisms, and would therefore be unlikely, from a nuclear safety point of view, to identify the highest-risk packages accurately. The other model does attempt to account for the vulnerability of the package, but is presented with no supporting data for the values used in predictions for package failure probabilities. Use of two different models for determining package failure probabilities could result in identical packages being ranked in a different order at different sites. A single model that consistently accounted for hazards posed by different material types and risks posed by the existing package configurations and conditions would allow for meaningful comparisons of the risks posed by packages across the complex.

The Board is aware that some information needed for accurate estimation of the integrity of existing packaging may not currently exist. However, considerable information does exist and should be incorporated into the package failure probability model at this time. For cases in which this information is lacking, values for such parameters can be judiciously selected. The Board believes that it would be appropriate to formulate the package failure probability model as a best estimate based on available information, and to refine the model as additional information is obtained through surveillance or repackaging efforts.

As noted in comment 8 in enclosure 1, some of the same issues discussed in this letter were independently identified by DOE’s technical review board for Recommendation 2005-1 before the repackaging prioritization methodology was finalized. It appears that DOE’s internal process for comment resolution failed to resolve the DOE technical review board’s key substantive comments.

Therefore, pursuant to 42 U.S.C. § 2286b(d), the Board requests that, within 30 days of receipt of this letter, DOE provide a response to the enclosed comments. This response should specifically address comments 1-6. Comment 7 need not be addressed at this time, as it also applies to draft DOE Manual M 441.1, Nuclear Materials Packaging Manual, and can better be addressed after the Board has had the opportunity to review that document. DOE should also outline in the response the steps to be taken to ensure more effective and consistent resolution of the technical review board’s comments.

Sincerely,

A. J. Eggenberger

Chairman

c: The Honorable C. Russell H. Shearer

Mr. Richard M. Stark

Mr. Mark B. Whitaker, Jr.

Enclosures

Enclosure 1

Comments on DOE’S Draft Repackaging Prioritization Methodology for Recommendation 2005-1, Nuclear Material Packaging

1. The vulnerability of package components should be reincorporated into the option 1 model. The vulnerability parameter from the original LA-UR-05-3864 model has been removed from the model adopted as option 1. By not attempting to account for the vulnerability of the package, option 1 currently assigns the same failure probability index to a specific material form, regardless of the type, number, or robustness of the containers. A model that does not attempt to account for the vulnerability of the package to known failure mechanisms is not likely to estimate failure probabilities accurately. Assuming container vulnerability is fully reincorporated in a future revision of the model, the following sub-comments apply:

(a) The vulnerability indices for unknown inner containers may not be appropriate for the characteristics of the population. The assignment of maximum vulnerability for unknown containers results in assigning packages with inner containers known to be highly vulnerable a lower failure probability than packages consisting of unknown inner containers within the same outer container. This may not be appropriate if a large number of packages having initially unknown inner containers are eventually shown, on average, to contain much more robust containers than the assumed worst-case scenario.

(b) A listing of standardized container vulnerability indices for package configurations that are present in the complex is not provided. There can be no expectation of consistent choices for container vulnerability indices across sites without an agreed-upon list. The packaging information collected under the first Implementation Plan commitment could be used as the basis for providing expert judgments to form this list.

(c) The use of zero values for minimum vulnerability indices creates inconsistencies in the predicted results. The assignment of zero value vulnerability indices to a barrier having otherwise maximum indices results mathematically in a degenerate total container vulnerability vector for packages having unknown barriers. For example, three nested slip-lid cans would have the same vulnerability index as a single such can with unknown inner containers.

(d) The fifth reactivity parameter for radiation-induced challenges to the package is not utilized. Recent experience with package failure has reinforced the importance of this challenge to the packaging. Assignment of values reflecting the true radiolytic potential of the material, rather than a placeholder value of 1, might better account for potential radiation damage to polymer-based packaging.

2. The reactivity indices provided for option 1 in LA-UR-05-3864 do not reflect known differences in reactivity among elements. For example, a highly reactive material, such as plutonium metal, is currently assigned the same reactivity indices as a considerably less reactive material of the same form, such as uranium metal. There may be a need for additional expert judgments regarding other material forms to account for differences in reactivity among elements.

3. The assumed linear effect of package age on failure probability may warrant further refinement. There may be other time-to-failure relationships that agree better with recent survey and package failure data. For example, a survey of the literature suggests nonrandom failures of components that wear or degrade over time may exhibit a more than linearly increasing failure rate over time. Appropriate consideration of age strongly impacts the accuracy of package failure predictions.

4. The value of allowing for the use of alternative package failure probability models is unclear. Having two options for determining relative package failure probability could result in identical packages being ranked in a different order at different sites. In principle, a single methodology is preferable because it facilitates meaningful comparisons of the risk posed by packages across the complex. Having a single methodology would provide an important tool for the Department of Energy (DOE) to ensure that the highest-priority items are qualified or repackaged first at all sites, as stated in Section 5.3 of the Implementation Plan.

5. No evidence is presented to support the option 2 model. Without data to support the judgments made on individual values used for the parameters or the model itself, there is no way to assess the validity of the option 2 model for predicting package failure probabilities.

6. Some parameters and numerical indices used in option 2 appear inconsistent. While the values chosen appear to be generally reasonable and attempt to account for the robustness of the package, the values assigned for unknown conditions do not appear to be consistent with respect to the parameters for known conditions.

7. The threshold dose consequence in the repackaging document appears to be inconsistent with the threshold being proposed in the draft packaging manual. The chart in Appendix C of the draft repackaging document illustrates the threshold for repackaging as a potential dose consequence of 5 rem committed effective dose equivalent or greater, using the methodology of Los Alamos National Laboratory for calculating the dose to workers. This approach yields considerably different results from the threshold the staff understands to be proposed in the draft manual, which is based on the methodology in 49 Code of Federal Regulations (CFR) 173 and does not use airborne respirable material calculations. This inconsistency results in excluding packages with sufficient quantities of material to be within the scope of the manual from the repackaging prioritization process.

8. DOE’s review process for Recommendation 2005-1 deliverables needs improvement. Many of the problems identified by the staff ought to have been identified by the technical review board and resolved before the draft document was transmitted to the Board. A subsequent staff review of the comments of the technical review board revealed that in fact the technical review board had identified some of these problems. Although most of the technical review board’s comments of an editorial nature were addressed, the more significant comments were not resolved. The comment resolution process needs to be improved and better integrated for future deliverables.

Enclosure 2

Discussion of Comments of the Board’s Staff on DOE’s Draft Repackaging Prioritization Methodology for Recommendation 2005-1, Nuclear Material Packaging

DOE’s IP for Recommendation 2005-1 includes deliverable 5.3-1, “Provide draft of repackaging risk prioritization methodology to the Board for review and comment.” This commitment was made by DOE in response to sub-recommendation (3) of Recommendation 2005-1: “Prioritize implementation of the improved nuclear material packaging requirement consistent with the hazards of the different material types and the risk posed by the existing package configurations and condition.” Section 4.0 of DOE’s IP for Recommendation 2005-1 summarizes the purpose of the risk ranking methodology as follows: “The application of this methodology will allow DOE to focus resources on the highest risk materials and packages and will accelerate the reduction in risk to nuclear material handlers.” Section 5.3 of the IP describes the resolution of this issue as follows: “Based on the Department nuclear material risk profile, the Department will ensure that the highest priority items, as determined by the complex-wide risk ranking methodology will be qualified or repackaged first at all sites.” DOE’s Responsible Manager for Recommendation 2005-1 transmitted the draft repackaging prioritization methodology to the Board in a letter dated January 30, 2006. This enclosure provides the technical bases of the comments transmitted to DOE, which were intended to improve the ability of the risk ranking methodology to meet the above commitments.

Draft Repackaging Risk Prioritization Methodology. The draft repackaging risk prioritization methodology is based on a model developed by Los Alamos National Laboratory (LANL) entitled Risk Ranking of LANL Nuclear Material Storage Containers for Repackaging Prioritization (LA-UR-05-3864). DOE’s draft document includes two alternative models for the package failure index, which is a measure of the relative probability of package failure. Use of either risk ranking model is limited to establishing priorities for repackaging of nuclear material; the models are not to be used for safety basis purposes.

Both models share a common method for the calculation of dose consequence based on the amount of radionuclides released as airborne respirable material, using the five-factor formula from DOE-HDBK-3010-94, Airborne Release Fractions/Rates and Respirable Fractions for Nonreactor Nuclear Facilities. This amount of material is then converted to a worker dose using an assumed exposure time and appropriate dose conversion factor. The LANL model (option 1) defines relative risk as the product of the package failure index and dose consequence. The package failure index is calculated from the product of the square of the reactivity index and package age. The reactivity index is a qualitative measure of the propensity of specific material forms to challenge the package integrity through (1) corrosion, (2) internal pressurization, (3) pyrophoric reactions, (4) oxidation expansion of metals, and (5) radiation damage.

The alternative model (option 2) assesses the package failure index using numerical indices that are assigned to various generic material and container characteristics and used for qualitative assessment of various challenges to the package. The indices are summed to determine a numerical ranking for container robustness, the inverse of which is multiplied by the age of the package to determine the package failure index.

Comments on the Draft Document. The comments developed by the Board’s staff can be categorized into the following areas:

· Suggested improvements to the package failure probability model (option 1)

– Incorporating the vulnerability of the package into the model

– Accounting for material type in the reactivity indices

– Increasing the effect of package age on failure probability

· Issues regarding the alternative package failure probability model (option 2)

· Misapplication of threshold dose consequence in the repackaging document

· Need for improvement in the review process for Recommendation 2005-1 deliverables

Each of these areas is discussed in detail below.

Suggested Improvements to the Package Failure Probability Model (Option 1).

Several aspects of the LANL model could be improved. The Recommendation 2005-1 working group, with input from the technical review board, should investigate for potential improvements to the current model in three principal areas: (1) accounting for the vulnerability of the packaging configuration, (2) accounting for the effects of different material types in the reactivity indices, and (3) increasing the effect of package age on failure probability. A preliminary evaluation by the Board’s staff suggests that these changes could result in more reasonable predictions for package failure. The three areas of suggested improvement to the option 1 model are discussed separately below.

Incorporating the Vulnerability of the Package into the Model―The original LANL model, as presented in LA-UR-05-3864, was based on the product of dose consequence, material reactivity, package age, and package vulnerability. The current LANL model, as adopted in option 1, calculates risk based only on the first three parameters. LANL chose not to incorporate container vulnerability in the current model because of the lack of specific container information on many of its items. However, by not attempting to account for the vulnerability of the package, the option 1 model assigns the same failure probability index to a specific material form regardless of the type, number, or robustness of the containers. For example, a package consisting of plutonium oxide stored in a plastic bottle, which is known to decompose from ionizing radiation, would be assigned the same failure probability index as the identical material stored in a stainless steel container.

The first commitment completed under the IP for Recommendation 2005-1 was issuance of a request to the sites to provide DOE with specific information on material packaging. LANL also has been collecting surveillance data on packages in its inventory as part of its packaging risk reduction commitments under the Board’s Recommendations 94-1 and 2000-1. The members of the Recommendation 2005-1 working group and technical review board should be able to use this information as the basis for providing expert judgments on container vulnerabilities. A model that does not attempt to account for the vulnerability of the package to known failure mechanisms is unlikely to yield accurate estimates of the failure probabilities of packages and the resulting risk posed to workers.

The draft repackaging risk prioritization document includes a figure from LA-UR-05-3864 that is intended to offer some support for verification of the option 1 model. LANL points out that this plot of dose consequence versus failure probability index demonstrates that the model correctly assigns the highest-risk decade to the failed packages that were the subject of the August 5, 2003, Type B accident investigation. However, these items were assigned the highest material reactivity after the accident and were among the materials with the highest dose consequence in the inventory. Since these represent two of the three parameters LANL uses to assess risk, this result provides only limited verification of the model. To adequately demonstrate verification of the model, results based on data obtained after the model has been developed need to be consistent with model predictions.

Recent data demonstrate that the option 1 model does not assign a particularly high failure probability index to the breached package that resulted in the December 19, 2005, occurrence of vault worker contamination at LANL. Nor does the model assign high failure probability indices to the three packages that LANL’s surveillance program recently discovered to have inner containers that were completely corroded, bulged, or severely degraded. Because risk is defined as the product of dose consequence and failure probability, packages with low dose consequences and high failure probabilities may not end up being assigned to the highest-risk decades. For packages known to have failed by a mechanism intended to be accounted for in the model, there is an expectation that the model would assign a high failure probability index even if it did not assign a high risk. The option 1 model does not meet this expectation for the recent package failures.

The original LANL model attempted to account for package vulnerability by considering only the three most robust barriers of a set of nested package containers. Each of these three barriers was represented by a vector whose indices varied from zero to 3, corresponding qualitatively to the vulnerability of the barrier to (1) corrosion, (2) internal pressurization, (3) pyrophoric reactions, (4) oxidation expansion of metals, and (5) radiation damage. For packages consisting of fewer than three barriers or packages with unknown barriers, each missing or unknown barrier was assigned the highest vulnerability indices. The total container vulnerability vector was determined by vector multiplication of the indices for each of the three barriers. Although the LANL document listed some container vulnerability indices for an example package, it did not include a table of standardized container vulnerability indices for package configurations that are present in the complex.

An analysis of the container indices used in the original LANL model suggests that use of zero values for minimum vulnerability indices can create inconsistencies in the predicted results, particularly when the vulnerability of a total package consisting of multiple containers or barriers is calculated. For example, a package consisting of three taped stainless steel slip-lid cans each assigned vulnerability indices of (0, 0, 3, 3, 0) has the same total container vulnerability indices of (0, 0, 27, 27, 0) as a package consisting of a single taped stainless steel slip-lid can, since multiplying by the two missing barrier indices of (3, 3, 3, 3, 3) yields the same result as multiplying by the additional two cans. This inconsistency could be eliminated by assigning only nonzero indices for container vulnerability.

Another aspect of container vulnerability in the option 1 model that may require refinement is the assignment of maximum vulnerability indices to packages with unknown inner containers. As a result, packages with inner containers known to be highly vulnerable to decomposition and accelerated degradation of the outer container (e.g., a plastic bottle containing plutonium within an outer steel can) would be assigned lower failure probabilities than packages consisting of unknown inner containers within the same outer container. This method may be appropriate for assigning risk to a large population of packages having only a small number of unknown containers or a larger number of unknown containers believed to have a high percentage of vulnerable inner containers. However, it may not be appropriate if a large number of packages having initially unknown inner containers are eventually shown, on average, to contain much more robust containers than the assumed worst-case scenario.

Overall, the option 1 model might benefit from judicious selection of container vulnerability indices and assignment of numerical values for unknown containers. The staff’s preliminary evaluation of incorporating nonzero container vulnerability indices into the model and assuming high but not maximum values for unknown barrier vulnerability suggests that predictions for package failure derived with this approach are more reasonable than predictions obtained with the current model.

Accounting for Material Type in the Reactivity Indices―The draft risk ranking methodology document refers to LA-UR-05-3864 for details on material reactivity indices. These indices were determined by the expert judgment of LANL personnel and an independent review panel. It appears there may be a need for additional expert judgments regarding other material forms that account for differences in reactivity among elements. For example, a highly reactive material, such as plutonium metal, is assigned the same reactivity indices as a considerably less reactive material of the same form, such as uranium metal. Furthermore, the option 1 model does not make use of the fifth reactivity parameter in the original LANL model. If container vulnerability indices were incorporated back into the model, this fifth reactivity parameter could be used to account for additional challenges to the packaging posed by ionizing radiation.

Increasing the Effect of Package Age on Failure Probability―LANL has assumed that the relative probability of package failure increases linearly with age. The source of this assumption appears to be data presented in an earlier report entitled A Risk-Based Prioritization Methodology for Legacy Fissile Material Disposition at LANL (LA-UR-00-5111). The author of this report attempted to establish a relationship that would adequately predict time to failure of packages using data from inspection of inner containers. When all of the available data for inner containers noted to have failed inspection were compiled as a function of age of the item, an approximately linear trend was observed. In this study, containers that failed inspection were assumed to be representative of containers that could eventually fail. Even though the linear age relationship concluded from the limited data was tenuous, the author did not provide support for such a relationship based on known constitutive relationships or generally accepted principles of component failure.

The treatment of the data in that report may not be fully justified. There are, as noted in LA-UR-05-3864, at least five independent mechanisms for failure of containers: (1) corrosion of the container wall from residual water and any chlorides present in the material; (2) over pressurization of the container by internal gas generation from volatilization of residual water and/or polymers; (3) breach of the container from expanding gas caused by a pyrophoric reaction between oxygen and metal fines; (4) rupture of the container from oxygen in-leakage, resulting in oxidation and volume expansion of metal items; and (5) loss of integrity by radiolytic embrittlement in plastic containers. The kinetics of these reactions would not be expected to be the same, and therefore the corresponding container failure times would be expected to vary depending on which mechanism of failure was rate-controlling. A set of containers found to have failed inspection because of degradation by one of several different possible failure mechanisms would not necessarily be a homogeneous population. The ideal method for determining the variation of failure probability with package age would have been to stratify the data into homogeneous populations based on failure mechanism.

However, because there may be insufficient data to stratify in this manner, a more defensible relationship for time to failure could perhaps be estimated from commonly accepted statistical analyses of failure probability. In general, the failure rate for nonrandom failures of components without initial quality control problems is known to increase monotonically with age. For example, components that wear out over time, such as automobile parts, are known to exhibit low failure rates after the infant mortality period in the early years of operation, followed by rapidly increasing failure rates during the later years. For the case of nuclear material packages, as the container degrades by one of the above mechanisms, the likelihood of failure will probably be very low initially, since it takes time for substantial degradation of the packaging to occur. Depending on the kinetics of the rate-controlling mechanism(s) of failure, it will take a certain amount of time before degradation of the packaging is extensive enough to cause failure. The failure rate for a population of similar packages will begin to increase as a greater number of packages degrade sufficiently to approach incipient failure. The probability of failure might be expected to increase substantially beyond this time. Therefore, it may be reasonable to assume that the relative probability of package failure increases more than linearly with age.

One method that is commonly used to model failure rates empirically is the Weibull distribution. The change in failure rate (or probability of failure) with time is a function of a shape parameter, which can be varied to account for constant, linearly increasing, or power law relationships with time. Using different versions of LANL models based on linear age and the square of age (i.e., power law relationship with an exponent of 2), the Board’s staff evaluated predictions of failure probability for the packages involved in the Type B accident at LANL, the recent breached package at LANL, and the recently discovered inner container failures. Results showed that for all of these cases, the models based on the square of age yielded consistently high failure probability indices for all of the actual failed packages. In contrast with these results, the option 1 model based on linear age predicted high failure probabilities for the Type B packages, but lower failure probability indices for the recently failed containers. Additionally, hypothetical robust packages of the same material and age as the failed containers, modeled using age squared, resulted in much lower failure probabilities than those of the failed packages. Therefore, incorporating container vulnerability into the model and using age squared instead of linear age improves the consistency of package failure predictions between the limited set of package failures and various hypothetical robust packages that would be expected to have lower failure probability indices. DOE may be able to develop other, more accurate time-to-failure relationships, which likewise could be evaluated using recent survey data or additional container failures.

Issues Regarding the Alternative Package Failure Probability Model (Option 2). In principle, a single methodology for package failure probability is preferable because it facilitates meaningful comparisons of the risk posed by packages across the complex. Having two different methodologies may result in ranking packages differently from one site to another. As previously discussed, there are inconsistencies and weaknesses in the option 1 model. The most serious weakness of the option 1 model is the absence of any parameters that account for the vulnerability of the package to failure. The option 2 model does attempt to account for the robustness of the package using parameters and numerical indices that appear to be generally reasonable; however, the values assigned for unknown parameters do not appear to be applied consistently with respect to known parameters. No data have been presented to support the judgments made about individual values used for the parameters or the model itself, so there is no way to assess the validity of the option 2 model. Combining the best aspects of each approach into one model for calculating failure probabilities may represent an improvement over using two separate methodologies.

Misapplication of Threshold Dose Consequence in the Repackaging Document.

DOE had earlier notified the Board that the packaging and storage criteria document will be in the form of a manual supporting 10 Code of Federal Regulations 835, Occupational Radiation Protection. DOE’s Recommendation 2005-1 working group has developed a draft methodology for defining nuclear material thresholds to be associated with specific packaging criteria based upon the material’s potential radiological consequence. This methodology uses the same calculation employed by the Department of Transportation in determining the A2 values presented in the table in 49 CFR 173.435, General Requirements for Shipments and Packagings. These values are used to determine the need for the more robust Type B (instead of Type A) packaging for transport of radionuclides. The A2 values are based on worker doses from an uptake scenario using a net intake factor of 10^-6. Currently, the working group has set the de minimis quantity (threshold to be in the scope of the draft manual requirements) to be equal to a worker committed effective dose equivalent (CEDE) of 5 rem.

DOE’s draft repackaging prioritization methodology states that all packages exceeding the threshold defined on the plot of dose versus failure probability index are deemed to need repackaging. This plot illustrates the threshold as those packages determined to have a dose consequence of 5 rem CEDE or greater, using the LANL methodology for calculating the dose to workers. LANL’s dose methodology yields significantly different results than those obtained with the method proposed in the draft manual. As a result, packages with sufficient quantities of material to be in the scope of the draft manual may be excluded from the repackaging prioritization process. The threshold dose values in the repackaging prioritization methodology ought to either be clarified to convey the definition used in the draft manual or removed.

Need for Improvement in the Review Process for Recommendation 2005-1 Deliverables. The process for completing a deliverable, such as this draft repackaging risk prioritization document, consists of three steps. First, the Recommendation 2005-1 working group drafts the document and sends it to the technical review board. Second, the technical review board provides comments on the draft document. Third, the working group and technical review board resolve the comments, and a final draft version of the document is sent to the Board. Any comments that cannot be resolved by the working group and technical review board are to be resolved by the Responsible Manager for Recommendation 2005-1. The Board’s staff has observed that the technical review board members are highly effective at identifying weaknesses and suggesting critical clarifications in the draft work products.

Many of the problems identified by the Board’s staff ought to have been identified by the technical review board and resolved before the draft document was transmitted to the Board. The Board’s staff conducted a subsequent review, which revealed that some of the issues raised by the staff had in fact been identified by the technical review board. Although most of the technical review board’s comments of an editorial nature were addressed, the more significant comments were not resolved in the draft risk ranking methodology that was transmitted to the Board. This situation calls into question the thoroughness of the comment resolution process. The document development and review processes of the Recommendation 2005-1 working group need to be improved and better integrated before future draft deliverables arc transmitted to the Board.