ISRP comments on U.S. Army Corps' response to: Preliminary review of the United States Army Corps of Engineers' Bonneville Decision Document Juvenile Fish Passage Recommendation October 2001

supplement to document ISRP 2001-11

Document history

Dec 18, 2001 - The ISRP released report 2001-11 (Preliminary review of the United States Army Corps of Engineers' Bonneville Decision Document Juvenile Fish Passage Recommendation October 2001), accepting public comments through January 31, 2002.

Jan 31, 2002 - Public comment period closes, Council received comments from the U.S. Army Corps of Engineers

Mar 1, 2002 - This is the ISRP response to those comments

Letter of ISRP comments

Date: March 1, 2002

To: Doug Marker, Northwest Power Planning Council

From: ISRP; Primary Reviewers, C. C. Coutant, D. Goodman, L. McDonald, P. Mundy, and R. Whitney

Subject: Corps of Engineers Draft Decision Document for Bonneville Dam. ISRP comments on the letter from Davis G. Moriuchi, Deputy District Engineer for Project Management, Portland District, Corps of Engineers, which responds to the December 18, 2001 ISRP review of the Draft Decision Document for Bonneville Dam.

In a February 7, 2002 letter, Davis Moriuchi, on behalf of the Corps of Engineers, Portland District, responded to the ISRP's preliminary comments of December 18, 2001 on the subject of the Corp's Draft Decision Document for Bonneville Dam. We provide the following in response to the Corps' proposed modifications to the Draft Decision Document as described in the letter from Mr. Moriuchi.

1. We believe it would be most helpful for the ISRP and the Corps to continue in a "response loop", during which the ISRP would provide additional review of future drafts to assist the Corps in development of a final Decision Document.

2. The letter from Davis Moriuchi was a very helpful and constructive response to comments and suggestions from the ISRP. The ISRP will be interested in reviewing future draft text when it is ready. There are two significant issues that we believe should be addressed more fully in the next draft.

A. The "risk analysis" as it is applied in the Draft Decision Document does not adequately convey the uncertainties inherent in comparing alternatives for juvenile passage improvement. While the dependence upon "point estimates", certainly simplifies the judgments required in choosing alternatives, the approach fails to realistically incorporate the uncertainties that unavoidably are present. We understand the willingness of the participants to forego dealing with that problem. However, we believe the Decision Document needs to withstand the scrutiny of others who are not privy to the insider thinking that went into the decisions on levels of "risk". We believe the process needs to be described fully enough on paper that uninitiated readers would be able to follow and perhaps even duplicate it.

The Corps' response on this issue expresses a concern that incorporating uncertainties will result in overlap of ratings assigned to the different configurations. Yes, we expect that overlap is possible (in fact it may be very likely) and are convinced that if overlap occurs then it realistically reflects what ought to be expected of apparatus whose performance depends upon behavioral responses of fishes. Such overlap is a possible outcome of probability sampling in a statistical sense, which is a process assumed in estimating survival of juvenile salmon in tests of prototype devices. To imply otherwise would be misleading. Nevertheless, decisions need to be made, and on the basis of imperfect information. The Decision Document ought to document the level of uncertainty associated with the evaluation of each option, and it should explain how the uncertainty was taken into account in the selection of a preferred alternative.

As an example of a possible approach that would provide some quantification and be more visible than the "risk analysis" employed, we suggest that it might be useful to make a list of assumptions required to develop the survival estimates for each alternative, i.e. passage route, survival rate, injury rate, and other criteria. Then for each assumption a rating might be assigned, such as: 1) sound assumption with good evidence to support it, 2) fair evidence, 3) weak evidence. For alternatives that have not yet been tested, a fourth rating "no evidence on survival as yet, but indications are favorable", or something of that sort might be used. The alternatives might then be displayed in a table for side-by-side comparison, with rankings that would reflect the total number of assumptions required for each and the relative confidence that experts have in the survival estimates for each. Such an approach might also be helpful in identifying specific needs for further study. To be sure, such an approach would likely resolve itself into a bottom line where some level of "best professional judgment" would be required in analysis of the rankings, but the basis for it would be provided in the table for all to see, and some text could justify the judgment.

B. SIMPAS appears to be a useful model for this application. However, care should be exercised that dependence upon the model does not inhibit flexibility in the approach to estimating survival in the various alternatives. As employed for one project, i.e. Bonneville Dam, the model should not be so complex as to prevent modifications that would be useful for application that would include a number of probable scenarios of flow and powerhouse operations.

A sensitivity analysis ought to be performed with SIMPAS for the purpose of identifying the degree of change that might be expected in total survival past the project given a range of expected improvements in survival that might result from the alternatives being considered for adoption. A number of scenarios of flow and powerhouse operation should be considered. A sensitivity analysis of SIMPAS might include random or systematic sampling from a hydrograph plus random or systematic sampling from the ranges of other input values.