|
Approach:
We reviewed seven
vegetation monitoring
manuals produced by federal agencies
to determine which
elements would be appropriate for
evaluating ES&R treatment success. The following are descriptions of program
design elements found within the reviewed monitoring manuals
that are suitable for use in evaluating ES&R treatment
success.
Objectives
All seven manuals included objectives as an
important element of a monitoring program. Several
manuals also provide good descriptions of how to formulate
objectives.
As described in Measuring and Monitoring Plant
Populations (Elzinga
et. al. 1998), both management and sampling objectives should
be written for any project. Management objectives are
“clearly articulated descriptions of a measurable standard,
desired state, threshold value, amount of change, or trend
that you are striving to achieve for a particular plant
population or habitat characteristics.”
Well-defined
management objectives in a monitoring program perform two
functions: first, they establish a standard to measure the
degree of success; and second, they determine the
appropriate indicators to measure. A standard protocol can
then be followed for the measurement of each indicator;
thus, data-collection activities are directly related to
management objectives. Sampling objectives should be paired
with each management objective and specify the desired
confidence level, confidence interval width (precision),
level of type II error, or detectable change for the
sampling effort.
Stratification
Stratification is
described in five of the seven monitoring manuals reviewed.
Stratification is the partitioning of treatment areas to
reduce variation and increase precision of sampling efforts. Areas that may respond differently to ES&R treatments such
as different soil types or ecological sites are good
candidates for strata.
Rules for stratification of treatment
areas into monitoring units should be created during the
planning stage of an ES&R project. Stratification can be
undertaken concurrently with or after identification of
treatment areas. In many cases, stratification is completed
as a byproduct of treatment planning such as assigning
different seed mixes to sites with different characteristics
or potentials.
Background information on the treatment area
is essential for stratification (Herrick et al., 2005b;
Lutes et al., 2006; USDI NPS, 2003). A variety of GIS data
are useful for delineating monitoring units, including
digital elevation models (DEMs), fire perimeters, proposed
and actual treatment areas, soils (if available), roads, and
land-use information. Using GIS software, such as ArcGIS,
monitoring units can be derived based on the available
information and the specifics of the project. If shapefiles
for ecological sites are available, then these files may be
the preferred initial strata. If only soils are available,
then the site can be divided initially into soil strata
separately to reduce variation and increase monitoring
efficiency. If shapefiles are available only for soil
surveys, but
soil-to-ecological site correlations are known, then
differing soils that correlate to the same ecological site
may be combined into the same strata. Additionally, slope
classes can be generated from DEMs when seedings will occur
over a large range of slopes. Areas that are not likely to
be seeded due to topography can be excluded from the
monitoring unit using DEMs. Using this information,
stratification of the area for both treatments and
monitoring can be accomplished using a defined set of
variables such as slope, aspect, elevation, treatment
type, minimum size, soil type, or ecological site.
Descriptions of
monitoring units should be included in monitoring plans so
that the scope of inference is known. For example:
Similar methods of
stratification used on different projects will facilitate
comparisons among those projects and aid region-wide
assessments of ES&R treatment effectiveness. Additional
information on stratification can be found in the Fire
Monitoring Handbook (USDI NPS, 2003), Fire Effects
Monitoring and Inventory Protocol (Lutes et al., 2006), and
the Monitoring Manual for Grassland, Shrubland, and Savanna
Ecosystems (Herrick et al., 2005b).
Control Plots
Control plots are
locations within a proposed treatment area that are
established prior to and avoided when treatments are
implemented. They provide important information on
natural recovery that can be used to determine whether or
not treatments were necessary in the first place. This
is especially useful given limited resources for
implementing large projects in severe fire years. The Fire Monitoring Handbook states that control
areas should be used when attributing a particular effect to
the applied treatment. The
goal is to show that the treatment caused the observed
change.
Prior to applying
treatments, a minimum of three control plots
should be randomly placed within each monitoring unit.
Control plots should not be placed in adjacent untreated
areas because they were not proposed to be treated and are
therefore different from those areas that were proposed to
be treated. Since adjacent areas are
most likely different from the area being treated, they make poor controls.
Control plots may not
be practical in all situations such as in cases where life
or property are threatened. These situations often
occur when the areas to be treated are on slopes above
developments and it is not possible to leave some areas
untreated. However, controls should be used whenever possible
because they provide the best measure of natural
regeneration and ES&R treatment success.
Random Sampling
Random sampling ensures
that monitoring data are unbiased and representative of the
monitoring unit. While this may be time-consuming, it is essential for defensible monitoring data
and success determinations. Without random sampling, criticism
may include that the samples only came from
areas where the treatments were effective, or that data were
biased by the site-selection process. In addition,
data that are not derived from random sampling are only
valid for the plot at which they were collected and cannot be
used to infer to the rest of the treatment area.
There are several
different methods of random sampling that can be used to
monitor ES&R treatments: simple, systematic, restricted, or two-stage
random sampling. One of these methods should be
used to enable statistical inference over as much of the
treated area as possible. To generate random samples,
there are several options, including grids placed over maps
or GIS random-point generators (see links page).
Rejection criteria should
be defined and procedures established if random plot
locations occur in areas that cannot be seeded, such as
roads, rocky outcrops, steep slopes, streams, or other
features. In some cases it may be possible to relocate
the plot nearby in a random direction. Or, it may be
necessary to move to the next randomly generated point.
Data Quality
After collecting
monitoring data, it is helpful to determine how well it can
assess ES&R treatment success. Parameters that should
be examined are confidence intervals, sample-size estimates, precision or minimum
detectable change, and power (if appropriate). These
parameters can then be considered when determining treatment success
either by comparison to quantitative objectives or when comparing
treatment to control data.
Confidence intervals are
the intervals surrounding a sample mean that we know with a
level of certainty contain the value of the true parameter. Confidence intervals are very useful for graphical analysis
of treatment success (Di Stefano, 2004). Comparing means and
confidence intervals of control and treatment plots side by
side or examining the confidence interval of the difference
are useful methods of viewing monitoring data. Overlapping
confidence intervals often mean that two means cannot be
proven to be different, but the confidence intervals also
need to be evaluated. For instance, very wide confidence
intervals (greater than 50 percent of the mean) may not be
considered adequate to perform any comparisons. Often,
additional sampling would be needed to decrease the width of
the confidence intervals.
The number
of samples required to estimate a parameter to a desired
level of precision (confidence interval half-width) or to detect a certain magnitude of
change (minimum detectable change, or MDC) can also be used
to assess data quality. This number can be estimated
using sample-size equations which use the standard deviation
of an initial sample or known variance to estimate the
number of samples required to achieve a desired precison or
MDC. There are several different sample size equations
depending on which comparison you will be performing. The
number of plots required will depend on the variability of
the plant community and the type of analysis you are
performing. While the number of plots
required will vary depending on the location, three plots are needed to generate an
estimate of variability and should be considered an absolute
minimum number of both control and treatment plots for any
monitoring unit. Five plots are usually better.
Power is an estimate of
the chance you have made a type II error (concluding there
was no change when there actually was a change). Power is
applicable when comparing treatment plots to control or
reference plots, but not when comparing treatment plots to a
defined standard.
Knowing the level of data quality you have attained will aid
in using limited information to determine treatment success. In general, attaining high data quality on a landscape scale
is difficult. For example, sample-size requirements are
easier to obtain in plant communities that are uniform and
difficult to achieve in communities that are highly
variable. Additionally, sample-size requirements will rarely
be achieved at the species level when monitoring large
post-fire treatment areas. Because of this, it is
useful to
estimate sample size based on life form. For example, sample
size requirements calculated for all perennial seeded
grasses will be lower than for each species of perennial
seeded grass individually. Additionally, the
intensity of sampling and resulting data quality will often
be limited by budget and time constraints.
Statistical Analysis
Use of the previous five
common elements (objectives, stratification, control plots,
random sampling, and data quality) will facilitate evaluation of the success of
post-fire treatments in a defensible statistical analysis. The simplest forms of analysis are to graphically
compare treatment plots with control plots or to treatment plots
to quantitative objectives. Alternatively,
treatment data can be compared to a control or reference
plot using a t-test. There are many statistical software
packages that will perform these calculations, or they can
be accomplished by hand. At the project level this will
result in a determination of treatment success for a
monitoring unit or project. An MS Excel
spreadsheet has been created to help with statistical
analysis and sample size estimation and can be downloaded
at:
ESR_Monitoring_Equations.xls
| 
