Coastal Services Center

---

---

Quick Links Statistical Design Reference Sites Monitoring Parameters and Methods Timing, Frequency, and Duration Summary References

Performance Assessment: Development of the Monitoring Program

Monitoring fish use at Kunz Marsh in Coos Bay, Oregon. Courtesy Craig Cornu, South Slough NERR. The monitoring program provides direct feedback on the development of a restored system with respect to performance criteria. Monitoring parameters such as water quality, vegetation cover, and fish use are measured to determine success of the system. The following are examples of field-sampling methods that could be selected for each parameter: Water Quality - In situ sampling (temperature, DO, salinity, turbidity); Sample collection for lab analysis (nutrients, contaminants, coliforms) Vegetation Cover - Transect and quadrat sampling; Aerial photography Fish Use - Otter- and beam-trawls; Beach seines; Drop nets The selection of appropriate reference or control sites in the vicinity of a restoration project is critical to the identification of trends in the monitoring data. For example, climatic variability can affect vegetation growth patterns. Reference areas allow scientists to be able to separate changes due to natural variability from those due to restoration efforts.

Assessing Performance

Before project implementation, the goals and objectives for a restoration project are developed along with specific performance criteria expected to be met on a proposed timeline. The performance criteria are measurable or otherwise observable aspects of the restored system used to assess a restoration project's progress. Most performance criteria are controlling factors or measures of ecological response, such as water temperatures that fall within a certain range or the density of a particular plant species. These criteria are based on a thorough knowledge of the system being restored. To determine how well and when the project objectives are met, there must be a way of measuring progress and assessing performance. A well-designed monitoring program performs this function.

A monitoring program does not need to be complex and expensive to be effective. A well-designed, systematic program that targets key parameters tied to goals and performance criteria can produce concise and informative results. The amount of monitoring required is dependent on the goals and performance criteria for the project, as well as on the type of ecological system under restoration. Factors that contribute to a successful monitoring program include the following:

• consideration of statistical design
• selection of appropriate reference sites
• selection of the most useful monitoring parameters and methods
• timing, frequency, and duration of sampling.

Statistical Design

The monitoring study design needs to include statistical considerations, such as location of sampling sites and the number of replicate samples to be collected. Many scientists view restoration projects as experiments that can be set up to test hypotheses. In this case, performance goals and criteria could be written as statements of testable hypotheses. Although rigorous experimental designs that evaluate one or more null hypotheses are appropriate for some restoration efforts, less rigorous analyses are more appropriate for supplying evidence for the development of the ecosystem.

For restoration projects, the analysis of the results should be driven by an understanding of the ecosystem rather than by statistics. For example, statistical tests might indicate that a system has not changed over time based on the data collected; however, the variance could be very high (due to variation in nature), resulting in a non-significant difference. In reality, the system could be functioning very differently than it was previously. Statistical test results, if used, should be tempered by critical and objective assessment to avoid overlooking actual ecological change.
> Return to top

Figure 1. An example of restoration and reference sites for a salt marsh restoration project in the South Slough National Estuarine Research Reserve. Courtesy: Craig Cornu.

Reference Sites

Reference sites typically represent undisturbed habitat, with characteristics desired of the restoration site when it will have been successfully brought back to a state of health and function. The restoration site is compared with the reference site(s) and should become increasingly similar to it over time. Reference sites provide information about the natural range of values for the parameters used in the monitoring program and show the annual variation in these parameters. Without a reference site, natural development of the restored site is difficult to ascertain due to the annual variation. In addition, reference sites can also document the physical and biological boundaries of the system (e.g., hydrology regimes, tidal elevations of vegetation), information which can be used to bound the design of the restoration project. Whenever possible, reference sites are located within the region where restoration takes place to maximize the comparability and to allow evaluation of natural variations within the system (Figure 1). For example, it is best if the sites have similar climates, soils, plant and animal species; similar human influence; and similar functions. Including several reference sites in the monitoring program can further delineate the natural range of values and annual variation in the parameters that are monitored. However, it is often difficult to find appropriate reference sites, particularly in urban areas. As more of our coastal areas are impaired, the need to establish, protect, and study reference systems increases.
> Return to top

Figure 2. Diagram showing fish monitoring method in the created Chehalis Slough in Washington. Courtesy: Blake Feist in Simenstad and others, 2001.

Selection of Monitoring Parameters and Methods

The selection of scientifically-based, easily measurable monitoring parameters has a large influence on the ability to track restoration project performance over time. Performance criteria and monitoring parameters should be carefully selected to ensure that as a tool for managers, they will be highly relevant and sensitive indicators of the progress of the system.

An important criterion for selecting monitoring parameters is that they respond to the restoration effort and, if relevant, any remaining impacts affecting the site. Further, at least three parameters should be selected and if possible they should include physical, chemical, and biological measures (NRC 1992). Too few parameters may provide insufficient information to evaluate performance or may provide information that is difficult to interpret. With an increase in the number of parameters, both the robustness of the monitoring assessment and the confidence in conclusions will increase. However, as the number of parameters increase the costs also increase accordingly. Sample categories for measurements may include water quality, hydrology, soils and sediment, vegetative health, habitat mapping, wildlife, and fisheries. For example, Figure 2 shows a diagram of an inlet/outlet fyke net for sampling juvenile salmonid use (a biological measure) of a restored slough.

Three basic questions to ask when selecting methods for monitoring are as follows:

1. Does the method efficiently provide accurate data on the parameters?
2. Is the method repeatable?
3. Is the method feasible within time and cost constraints?

Any method used for sampling a parameter should be applied according to a documented protocol. This documentation often consists of peer-reviewed technical articles in which the method is well described. Under the Estuary Restoration Act, NOAA is developing standard monitoring protocols for estuary habitat restoration projects. Also, see Innovative and Successful Monitoring and Adaptive Management Approaches for examples of programs that use standard monitoring protocols.

It is highly desirable to choose sampling methods that provide for collection of data on more than one parameter. For example, a sediment core sample can provide information on rhizome development, hydrology, and invertebrate communities. Some of the information from the core (e.g., odor, texture) can be taken directly in the field, whereas information such as particle size would be assessed through use of appropriate laboratory methods. Collecting concordant data is efficient and allows for robust analysis.

Additionally, the project manager should be aware of available information collected by others. Consultation with agency personnel, local universities, consultants, citizen environmental groups, and landowners in the area can reveal information of this type. Ongoing monitoring programs provide useful data, such as state hunting and fishing reports, U.S. Geological Survey hydrological data and topographic maps, Audubon Society bird counts, Natural Resources Conservation Service (NRCS) soils maps, U.S. Weather Service data, and air quality data. Many agencies and volunteer groups want to see their data used and are willing to cooperate with restoration programs, but a systematic and equitable method of data transfer should be planned.

A fundamental decision in choosing monitoring methods is whether the monitoring must show highly quantifiable results or whether the program only needs to illustrate general changes. Erwin (1990) suggested that quantitative methods should be used when there is uncertainty associated with the restoration technique or when success criteria are related to obtaining specific thresholds. Conversely, in situations where there is more certainty of success, and where performance is not tied to specific quantitative criteria, qualitative evaluations are appropriate. A combination of quantitative and qualitative methods can also be employed effectively in the same monitoring program. Table 1 lists examples of qualitative and quantitative evaluation techniques used for wetlands.

Table 1. An example of qualitative and quantitative evaluation techniques used for wetlands

Qualitative measures	Quantitative measures
Plan view map showing observation points and estimated vegetation coverage	Scaled vegetation map quantifying coverage areas
Vegetation (species list and qualitative abundance estimates)	Vegetation density, cover, and biomass
Fixed point panoramic photographs	Elevation
Rainfall and water level data	Water quality and soil properties data
Wildlife utilization observations	Wildlife counts
Fish and macroinvertebrate (species list and qualitative abundance estimates)	Fish and macroinvertebrate (density and distribution)

> Return to top

Timing, Frequency, and Duration

Monitoring programs should be carried out according to a systematic schedule. The monitoring program should be designed prior to conducting any baseline studies, so that the pre- and post-restoration sampling and analysis methods are the same. Baseline studies are used to understand pre-project conditions and for comparison to post-restoration data. Restoration performance monitoring should commence as soon as the major restorative actions have taken place and the system is left to develop more or less on its own.

The plan should include a start date, the time of the year during which field studies will take place, the frequency of field studies, and the end date for the program. Timing, frequency, and duration are dependent on the system type and complexity, and uncertainty. In addition, controversy over the project can force a higher degree of scrutiny and may increase the level of monitoring effort. Seasonality must be taken into consideration. For example, if a particular plant species of interest is conspicuous only during spring flowering, sampling must be conducted during its period of bloom. Well-timed sampling minimizes the number of sampling efforts and thereby reduces the cost of the program. Because weather varies from year to year, it is wise to "bracket" the season. For example, sampling temperature four times during the midsummer would be better than a single sampling in the middle of the season.

Monitoring of restoration sites and associated reference sites can be performed in two ways: 1) by concentrating all tasks during a single site visit, or 2) by carrying out one task or a similar set of tasks at several sites in a single day. The latter strategy is preferable, because it minimizes seasonal effects and variability in conditions from day to day, and repeating the same task on the same day may be more efficient. However, it is not always practical if sampling sites are far apart or difficult to access. Sampling of specific parameters in reference areas should take place during the same time of year as sampling in restored areas.

Frequency of sampling can vary within a year as well as among years. In general, new systems change rapidly and should be monitored more often than older systems. This is especially true for systems in which success is highly uncertain. By sampling more often, deviations from the expected trajectory of development, if detected early, may be corrected more easily than those allowed to progress further. As the system becomes established, it is generally less vulnerable to disturbances. Hence, monitoring can be less frequent.

The duration of the monitoring program is a controversial issue. In general, the program should extend beyond the period of most rapid change and into the period of stabilization so that there is reasonable assurance that the system has met its performance criteria, will meet them, or will not likely meet them. A growing body of evidence on constructed systems shows that most aquatic systems do not reach stability in less than 5 years and may take decades or centuries to develop. New, constructed ecosystems that start with no vegetation and for which hydrology must be established take a longer time to develop than systems in which only minor adjustments of existing aquatic habitats are necessary.

The Chehalis Slough mitigation project is an example of a long-term, post-restoration monitoring program in a new system with a high degree of uncertainty regarding functional performance. In 1990, the U.S. Army Corps of Engineers (USACE), in conjunction with the local sponsor (Port of Grays Harbor), constructed a tidal slough adjacent to the Chehalis River in Grays Harbor, Washington. The slough was intended to serve as mitigation for loss of juvenile salmonid habitat caused by navigation channel improvements. Because the slough was essentially dug out of upland habitat, it represented an entirely new ecosystem for the site. The monitoring program, which focuses on vegetation, fish prey, and fish use of the system, was conducted annually in spring and summer during the first 2 years. Vegetation was monitored annually for 3 years, and fish annually for 2 years, then again in Years 5 and 10 (Figure 3). Sedimentation, site stability, and retention of large organic debris were also monitored in Year 10. USACE committed to post-construction monitoring over 50 years to ensure that the mitigation effectively fulfilled its objectives, but the frequency of monitoring was not specified beyond the initial 10-year period.

An attenuated frequency of sampling from an annual basis initially, to every 2 to 4 years later, is considered adequate and appropriate for documenting major changes in the system. If the restoration is not going to succeed, it will often become apparent in 1 to 3 years. If the site will eventually develop into a functioning system but may not meet expectations in the long term, it will become apparent in later years. This strategy for attenuating sampling allows for adaptive management of the system, while minimizing monitoring effort and cost.
> Return to top

Summary

In summary, the goals for most restoration projects center on structural and/or functional parameter(s) indicative of target system conditions. These parameters change through time, and can be tracked through monitoring programs that are designed to measure progress toward specific objectives. Statistics can be employed to determine sampling frequency and replication, and to compare project site progress with reference site conditions. The monitoring parameters selected should be scientifically-based, accurate, economical, and feasible to measure. The data should be collected at a frequency to provide enough data to make adaptive management decisions and be sufficiently specific to provide reasonable assurance that the system is or is not going to meet the goals of the project. This requires a commitment to monitoring, often long after the construction work is completed. As more projects employ long-term monitoring and the results are disseminated through the literature, we may expect to see improvements in restoration methods that benefit our coastal resources.
> Return to top

References

Adamus, P.R. 1983. A method for wetland functional assessment. U.S. Department of Transportation, Federal Highway Administration, Office of Research, Environmental Division. Washington, D.C.

Boesch, D.F., and others. 1994. "Scientific Assessment of Coastal Wetland Loss, Restoration and Management in Louisiana." Journal of Coastal Research. (Special Issue Number 20) Page 103.

Boumans, R.M.J., D.M. Burdick, and M. Dionne. 2002. "Modeling Habitat Change in Salt Marshes After Tidal Restoration" Restoration Ecology. Volume 10, Number 3. Pages 543 to 555. http://www.blacksci.co.uk.

Calloway J.C., and others. 2001. Assessment and Monitoring. In Handbook for Restoring Tidal Wetlands, ed. J.B. Zedler. Pages 271 to 335.

EPA. 1991a. Biological criteria: research and regulation. Proceedings of a Symposium. EPA-440/5-91-005, U.S. Environmental Protection Agency, Office of Water. Washington, D.C.

EPA. 1991b. Biological criteria: guide to technical literature. EPA-440/5-91-004, U.S. Environmental Protection Agency, Office of Water. Washington, D.C.

Erwin, K.L. 1990. Wetland Evaluation for Wetland Creation and Restoration. Pages 420 to 458. Island Press. Washington, D.C.

Fonseca, M.S., W.J. Kenworthy, and G.W. Thayer. 1998. Guidelines for the Conservation and Restoration of Seagrasses in the United States and Adjacent Waters. National Oceanic and Atmospheric Administration, Coastal Ocean Office. Silver Spring, MD.

Horner, R.R., and K.J. Raedeke. 1989. Guide for wetland mitigation project monitoring. Washington State Department of Transportation. Olympia, WA.

Hunsaker, C.T., and D.E. Carpenter. 1990. Environmental monitoring and assessment program: Ecological indicators. U.S. Environmental Protection Agency. Washington, D.C.

Kentula, M.E., and others. 1992. An Approach to Improving Decision Making in Wetland Restoration and Creation. U.S. Environmental Protection Agency. Corvallis, OR.

Kusler, J.A., and Kentula, M.E. 1990. Wetland Creation and Restoration, the Status of the Science. Island Press. Washington, DC.

National Research Council (Committee on Mitigating Wetland Losses, Board on Environmental Studies and Toxicology, Water Science and Technology Board). 2001. Compensating for wetland losses under the Clean Water Act. National Academy Press. Washington, D.C. http://www.nap.edu/books/0309074320/html.

National Research Council. 1992. Restoration of Aquatic Ecosystems. National Academy Press. Washington, DC.

Neckles, H. and M. Dionne. 1999. Regional Standards to Identify and Evaluate Tidal Wetland Restoration in the Gulf of Maine. http://www.pwrc.usgs.gov/resshow/neckles/Gpac.pdf.

PERL (Pacific Estuarine Research Laboratory). 1990. A Manual for Assessing Restored and Natural Coastal Wetlands with Examples from Southern California. La Jolla, CA.

Simenstad, C.A., and others. 2001. Decadal Development of a Created Slough in the Chehalis River Estuary: Year 2000 Results. Report to US Army Corps of Engineers, Seattle District. Available online at: http://www.nws.usace.army.mil/publicmenu/
DOCUMENTS/Chehalis_Created_Slough_2000_Monitoring_.pdf.

Simenstad, C.A., and others. 1991. Estuarine Habitat Assessment Protocol. Fisheries Research Institute, University of Washington. Seattle, WA.

Steyer, G.D., and R.E. Stewart Jr. 1992. Monitoring Program for Coastal Wetlands Planning, Protection, and Restoration Act Projects.

U.S. Fish and Wildlife Service. 1980. Habitat Evaluation Procedures. U.S. Department of the Interior, Fish and Wildlife Service, Division of Ecological Services. Washington, DC.

USGS (U.S. Geological Survey). 2002. Species profiles: life histories and environmental requirements of coastal fishes and invertebrates. Available URL: http://www.nwrc.usgs.gov/publications/specintro.htm.

Wilber, P., G., and others. 2000. "Goal Setting and Success Criteria for Coastal Habitat Restoration. Ecological Engineering." Volume 15, Number 3 and 4 (Special Issue). Pages 165 to 405.

Yoccuz, N.G. 1991. "Commentary: Use, Overuse, and Misuse of Significance Tests in Evolutionary Biology and Ecology." Bulletin of the Ecological Society of America. Volume 72. Pages 106 to 111.

Zedler, J.B. 2000. "Restoration of Biodiversity to Coastal and Inland Wetlands." In Biodiversity in Wetlands: Assessment, Function and Conservation, Vol 1. Backhuys. Leiden, NL. Pages 311 to 330.

Additional information and citations are available in:

Diefenderfer, H.L., and R.M. Thom. 2003. Systematic Approach to Coastal Ecosystem Restoration. Prepared for NOAA Coastal Services Center, Charleston, SC., by Battelle Marine Sciences Laboratory. Sequim, WA. PNWD-3237.
> Return to top