roy troll border art
nefsc banner
Technical Memoranda Reference Documents Classic Publications Contract/Grant Reports
CMER Publications Series Information Links and Contacts Annual/Biennial Lists
Web Manager Email Search Publications Publications Home Site Map
CONTENTS
Executive Summary
Introduction
Design Considerations
Data Sources
Unlikely Cells
Missing Cells: Imputation and Pilot Coverage
Estimation of Bycatch Ratios
Estimation of Total Discards
Meta-Analysis
Accuracy Analyses
Overlap Analyses
Optimization Tool
Sources of Uncertainty/
Discussion
Summary and Conclusions
Acknowledgments
References
Northeast Fisheries Science Center Reference Document 06-22

The Analytic Component to the Standardized Bycatch Reporting Methodology Omnibus Amendment:
Sampling Design, and Estimation of Precision
and Accuracy

NOTE: This is the ORIGINAL VERSION -- please click here for updated Second Edition (May 2007; CRD 07-09)

by S.E. Wigley, P.J. Rago, K.A. Sosebee, and D.L. Palka

National Marine Fisheries Service, 166 Water St, Woods Hole MA 02543

Print publication date September 2006; web version posted November 16, 2006

Citation: Wigley SE, Rago PJ, Sosebee KA, Palka DL. 2006. The Analytic Component to the Standardized Bycatch Reporting Methodology Omnibus Amendment: Sampling Design, and Estimation of Precision and Accuracy. US Dep. Commer., Northeast Fish. Sci. Cent. Ref. Doc. 06-22; 135 p.

get acrobat reader Download complete PDF/print version

Executive Summary

Standardized Bycatch Reporting Methodology (SBRM) can be viewed as the combination of sampling design, data collection procedures, and analyses used to estimate bycatch in multiple fisheries.  The SBRM provides a structured approach for evaluating the efficacy of the allocation of observer days to multiple fisheries to monitor a large number of species under the different Fishery Management Plans (FMPs), the Marine Mammal Protection Act, and the Endangered Species Act.  In this report, we examine 45 fleets and 60 species/species groups to encompass all federal FMP-managed species in the Northeast.  A comprehensive summarization of 2004 data collected by the Northeast Fisheries Observer Program (NEFOP) is presented, as well as estimation of  precision of bycatch for fish, turtles, marine mammals, and sea birds by using three methods and two discard ratios, an evaluation of these different methods, and the estimation of sea days required to achieve the desired level of precision.  A combined ratio method using a discard-to-kept weight ratio was selected to evaluate the monitoring of bycatch over a diverse range of species and fleets in the Northeast region.  We recognize that research on discard estimation is ongoing and future work may lead to improvements of this method.  The number of sea days necessary to achieve a 30% coefficient of variation (CV) for all identified fisheries exceeds 33,000 days.  Application of additional criteria for evaluating the efficacy of the monitoring program may lead to reductions in the total number of days necessary.  In particular, criteria related to the magnitude of total discards relative to landings may reduce the number of days necessary to estimate infrequent discard events of fish.


INTRODUCTION

This report presents the analyses needed to support the Standardized Bycatch Reporting Methodology (SBRM) required for the Omnibus amendment to the Northeast fisheries management plans.  These analyses include: 1) a comprehensive summarization of 2004 data collected by the Northeast Fisheries Observer Program (NEFOP); 2) estimation of bycatch precision for fish and protected species using three methods and two discard ratios; 3) evaluation of these different methods; and 4) estimation of sea days required to achieve the desired level of precision.  Subsequent SRBM-related analyses can account for the overlapping nature of multiple species caught by a fishery, develop species-specific imputation methods, and expand the optimization tool currently used to allocate sea day coverage to account for all monitoring objectives.  These secondary analyses are briefly described in this report and can be undertaken sequentially in the future, but are not the primary focus for this report.

The methods used in this report generally follow those recommended by the National Working Group on Bycatch (NWGB) (NMFS 2004) and further developed the work by Rago et al. (2005) and Fogarty and Gabriel (2005) for the New England multispecies groundfish fishery.  These methods reflect a design-based rather than a model-based approach, and directly link the data collection monitoring program with the evaluation analyses.  In Rago et al. (2005), three fleets and twelve species were examined; in this report, it was necessary to examine 45 fleets and 60 species/species groups to encompass all federally managed species in the Northeast.

The Northeast Fisheries Science Center (NEFSC) administers the Northeast Fisheries Observer Program (NEFOP).  The NEFOP observer data is a key component to the SBRM in the Northeast region.  The SBRM can be viewed as the combination of sampling design, data collection procedures and analyses used to estimate bycatch in multiple fisheries.  The SBRM provides a structured approach for evaluating the efficacy of the allocation observer days to multiple fisheries to monitor a large number of species under a 14 different Fishery Management Plans (FMPs), the Marine Mammal Protection Act, and the Endangered Species Act.  The SBRM is not intended to be the definitive document on the estimation methods nor is it a compendium of discard rates and totals.  (Note: Discard rates are not included in this report.)  Instead, the SBRM is intended to support the application of multiple bycatch estimation methods that can be used in specific stock assessments.  The SBRM provides a general structure for defining fisheries into homogeneous groups and allocating observer coverage based on prior information and the expected improvement in overall performance of the program.  The general structure helps identify gaps in existing coverage, similarities among groups that allow for realistic imputation, and the tradeoffs associated with coverage levels for different species.  The allocation process, while guided by a concept of optimization, explicitly recognizes that many different factors affect the realized allocation of observer days to specific fisheries.  Moreover, the optimization model allows for continuous improvement in allocation as new information on the results of the previous year’s data are obtained.

Throughout this report we will use the term “bycatch” synonymously with “discard”.  In basic terms, bycatch is defined as living organisms that are captured by fishing gear and returned to the water.  We do not define bycatch as the capture and retention of non-target species nor do we account for potential survival of organisms returned to the water.  Most importantly, we do not base any of our analyses on the potential mortality associated with unobserved encounters with fishing gear.  Our omission of these mortality sources does not confirm or deny their potential importance; rather, it explicitly recognizes that such events cannot be observed even when an observer is present on a given trip and therefore there is no basis for extrapolation to unobserved sampling units (i.e., trips).  Thus our definition of bycatch is restrictive when compared to the definition given in NMFS (2004).


DESIGN CONSIDERATIONS

Sampling Unit, Response Variables, and Precision Goals

Among the most important decisions in the preparation of the SBRM are the definition of the sampling unit, the definition of the quantity to be measured for each sampling unit (in statistical terms this is known at the response variable), and the desired level of precision for this value.  The sampling unit is an object on which a measurement is taken (Cochran 1963, Mendenhall et al. 1971).  The sampling unit for the SBRM is the vessel trip.  For the purpose of the SBRM, the response variable for each trip is the total bycatch for a single species or a group of species.  A bycatch ratio can be derived by dividing the total bycatch by some measure of fishing effort.  If all trips have similar attributes (e.g., vessel power, gear, duration, etc) then an average bycatch per trip may be an acceptable ratio.  Otherwise, the bycatch rate can be expressed as the ratio of total discards to vessel days absent, vessel days fished, or total kept weight of species.  Total kept weight is, in this sense, a surrogate for effective fishing power.  For finfish and shellfish, the numerator of the bycatch ratio is defined as the total weight of the species or species group discarded.  The denominator of the bycatch ratio is either weight of all species kept or fishing effort.  Owing to difficulties in interpreting quantitative measures of fishing effort found in the Vessel Trip Reports (VTRs), effort is approximated by days absent.  For turtles, marine mammals, and sea birds, the numerator in the bycatch ratio is the total number of individuals discarded.  Bycatch rates for these species are expressed as numbers per unit of fishing effort or numbers per species of kept pounds.

The NWGB advocated evaluating bycatch programs on the basis of aggregated species, but this will not guarantee that programs will be adequate for individual species (NMFS 2004).  To address this dilemma, this study estimates not only bycatch ratios and associated precision (relative standard error) for species complexes relevant to the FMPs (e.g., multi-species groundfish, summer flounder-scup-black sea bass, etc.), but also bycatch ratios and precision for each individual species.  Stock areas will not be considered in the analyses.  Conceptually, the problem of stock area is similar to that of estimating age-specific discard estimates: the full variability of those estimates is the product of the uncertainty of the species-specific discard estimates and the sampling distribution of the age-length key, an issue of fine-scale detail that is beyond the scope of the broad SBRM.  Parenthetically, the sampling design underlying SBRM is designed to support robust post-stratification, sufficient estimation of stock area, and age-specific estimates of discards.

Although marine mammals and sea birds are not required for evaluation by the Magnuson-Steven Act (NOAA Fisheries 1996), marine mammals and sea birds are included in these analyses to illustrate the comprehensive nature of the NEFOP and the SBRM. 

The aggregate species approach will illustrate the overall effectiveness of the SBRM. The individual species approach will show the tradeoffs for varying levels of precision.  With respect to the precision targets, the NWGB determined that a 20-30% coefficient of variation (CV) for the bycatch estimate is a useful goal.  They stated:

“Protected species
For marine mammals and other protected species, including sea birds and sea turtles, the recommended precision goal is a 20-30% CV for estimates of bycatch for each species/stock taken by the a fishery.

Fishery Resources
For fishery resources, excluding protected species, caught as bycatch in a fishery, the recommended precision goal is a 20-30% CV for estimates of total discards (aggregated over all species) for the fishery; or if total catch can not be divided into discards and retained catch then the goal is a 20-30% CV for estimates of total catch.” (NMFS 2004)

As the NWGB pointed out, “Ideally, standards of precision would be based on the benefits and costs of increasing precision” (NMFS 2004).  They also noted that under some circumstances, attaining the precision goal alone would not be an efficient use of the public resources.

In the evaluation of precision of discard estimates, a 30% CV was selected to derive the number of sea days that would be necessary to sufficiently monitor the bycatch of species groups within a fleet sector.  Selection of the higher value is predicated upon stratification of species and fisheries at a finer level than the NWGB recommended.  In this report, the term “CV” is defined as the ratio of the standard error of the estimate divided by the “estimate”. The “estimate” can be total discard or mean discard rate.  Our use of CV is equivalent to the term “proportional standard error”; for the sake of consistency with the NWGB (NMFS 2004) we use CV throughout this report.

The NWGB recommended the precision goals for a ‘fishery’.  In the Northeast region, a fishery may comprise several gear types, e.g., the groundfish fishery is comprised of otter trawls, gillnets, and longlines.  Thus, in order to define a fishery, gear type and mesh size are used as two key components in defining fleets within a fishery. 

Definition of Strata – Fishery Identification

To monitor the diverse fisheries off the Northeast coast of the US with at-sea observers, it is necessary to stratify the trips into fleet sectors with similar characteristics.  For this report, fleet sectors are defined as strata within a survey design.

Commercial fishing trips are partitioned into fleet sectors using six classification variables:  calendar quarter, geographical region, gear type, mesh size, access area, and trip category.   Some fleets were further stratified due to FMP requirements (e.g., quota-monitoring in the US/Canada Resource Sharing Area, B-days, etc.).  These classification variables are selected because they are generally known before a trip occurs.  Using these criteria it is possible to generate a list of candidate vessels for each stratum, which simultaneously enables a random selection process and reduces the number of repeat trips on vessels.  This is a critical aspect for both strata definition and sample selection.  One cannot base a sampling design on the outcome of a sample observation.  For example, in this exercise, it is not possible to select a sampling design that specifically improves the precision of cod discards, since that objective is dependent on the realization of the actual sample.  However, it is possible to select samples that will improve the probability of obtaining improved discard estimates by estimating the expected proportion of trips that catch species groups of interest.

Calendar quarter was considered the most feasible temporal unit to capture seasonal variations in fishing activity and bycatch rates over the full range of fisheries.  Although some management regulations operate at a finer scale (e.g., weekly), quarterly data can be further subdivided if finer resolution is needed.

Additionally, trips are classified into two broad geographical regions – New England (NE) and Mid-Atlantic (MA) – based upon the port of departure: ports located from Maine to Connecticut were grouped together to form the NE region and ports located in states from New York southward comprised the MA region.  While data from both VTRs and NEFOP are summarized by port landed, allocation of sea day coverage is necessarily based upon port of departure since an observer must physically board the vessel.  A review of the NEFOP and VTR databases for 2004 revealed few instances (less than 2% of trips) where the change of port of landing from port of departure resulted in a change in region (i.e., NE to MA or vice versa).  It should also be pointed out that the basis for classifying trips is the region/port of departure since areas fished are not always predetermined.  The majority (over 93%) of 2004 NEFOP trips both originated and fished in the same region, and exhibited the same general pattern observed in the VTR data; however, the proportion of trips that do not do so can be accounted for in the sea day allocation.

Percentage of 2004 observed trips that departed and fished in the NE and MA regions.
Region/port of departure Area Fished
New England (Subarea 5) Mid-Atlantic (Subarea 6)
New England 72.4% 6.3%
Mid-Atlantic 0.2% 21.1%

Percentage of 2004 VTRs that departed and fished in the NE and MA regions.
Region/port of departure Area Fished
New England (Subarea 5) Mid-Atlantic (Subarea 6)
New England 60.1% 3.8%
Mid-Atlantic 0.8% 35.3%

In these analyses, fourteen general gear types were considered: longline, otter trawl, scallop trawl, shrimp trawl, gillnets, scallop dredge, mid-water (paired and single) trawl, fish pots/traps, purse seine, hand line, Scottish seine, clam dredge, crab pots, and lobster pots.  Although the northern shrimp and the lobster fisheries are managed under Atlantic States Marine Fisheries Commission (ASMFC), these fisheries have bycatch of fish and protected species managed by the New England Fishery Management Council (NEFMC), and the Mid-Atlantic Fishery Management Council (MAFMC); therefore these gear types are included to the extent possible.

Mesh size groups were formed for the otter trawl and gillnet gear types.  For otter trawl, two mesh groups were formed: small (less than 5.5 inches) and large (5.5 inches and greater).  For gillnet, three mesh groups were formed: small (less than 5.5 inches), large (from 5.5 to 7.99 inches), and extra large (8 inches and greater).

Trips that used either scallop trawl or scallop dredge were further classified into two access areas (open or closed) as well as two trip categories (general or limited)[1].  Trips using other gear types were not further classified.

Due to the mixture of species caught during a trip, it is not sufficient to classify trips with regard to target species because discard of target and non-target species may occur. 

A total of sixty individual species or species groups are examined in these analyses.  These species/species groups comprise the fourteen FMPs of the NEFMC and MAFMC, an all species combined group, and five protected species groups (Table 1).  The fisheries encompassing these sixty species/species groups required forty-five different fleet sectors to account for all regional, gear, mesh, and quota-monitoring status combinations (Table 2a).


DATA SOURCES

The sampling unit used in these analyses is the trip.  Trip characteristics are recorded in both the NEFOP and fishing VTR data sets.  These databases are used to define the size of the sample and the size of the strata, respectively.  Data from each source are retrieved and prepared separately before they are combined.

Fishing Vessel Trip Report (VTR) Data

Beginning in June 1994, the Northeast Region’s data collection system was changed from a voluntary to a mandatory reporting system for fishermen and dealers holding federal permits[2] who catch and buy/sell species regulated by the NEFMC and/or the MAFMC.  The mandatory reporting system consists of two components: 1) dealer reporting and 2) vessel trip reporting.  Each component contains information needed for fishery management and stock assessment analyses.  The dealer reports contain total landings by market category, while the VTRs contain information on area fished, kept and discarded portions of the catch, fishing effort and the gear type and mesh size used.  Ideally, these data collection systems would record equivalent total landings.  In practice, a variety of problems, especially incomplete or delayed reporting of VTR, generally results in a slight underestimation of landings.  These disparities are discussed below.

The VTR data have been routinely used in management analyses and peer reviewed stock assessments.  Details on example applications of the VTR to stock assessments may be found in a large number of reports of the Stock Assessment Review Committee (SARC)[3].

In these analyses, the 2004 VTR (commercial) data are used to: 1) define the sampling frame of the commercial fishing trips; 2) expand bycatch rates to total discards (Note: total discards are not presented in this report); and 3) evaluate the accuracy of the observer data with respect to area fished, kept pounds, and trip length.  The VTR data are the only synoptic data source for vessel activity, area fished, and fishing effort for commercial fisheries.  The Vessel Monitoring System (VMS) data and the Days-At-Sea data systems cover only portions of the fisheries and therefore are limited in use for this type of analysis.

The VTR data can be used as a basis for defining the sampling frame, since all federally permitted vessels are required to file a VTR for each fishing trip (see NMFS-NERO http://www.nero.noaa.gov/ro/fso/vtr_inst.pdf).  These self-reported data constitute the basis of the fishing activity of the commercial fleets.  VTR trip data are collapsed into fleets as defined above.  For each fleet sector the number of trips, the average number of days absent per trip, and the kept weight of species are calculated.

The limitations of self-reported catch data are well known (e.g., Walsh et al. 2002, NMFS 2004).  Limitations of the initial VTR data sets were described by the SARC in 1996 (NEFSC 1996).  Since then, many of these limitations have been addressed.  In particular, subsequent peer reviews through numerous SARCs and a review by the National Research Council (1998) have identified the strengths, weaknesses, and appropriate uses of the VTR data from the Northeast. 

Measures to ensure the validity of the VTR database include routine auditing procedures, standardized data entry protocols, and compliance reviews (pers. comm., Greg Power, Chief, Fisheries Information Section, Northeast Regional Office, NMFS).

The VTR data are converted to round (live) weight using Commercial Fisheries Database System (CFDBS) conversion factors for the species.  Days absent and total species kept on a trip are also calculated.  The VTR trips are collapsed into strata as defined above.  For each fleet sector, the number of trips is calculated.  Trips participating in the US/Canada access area, B-day category, and other quota-monitored programs could not be identified in the VTR data.  These trips have been grouped by the other stratification variables and have not been partitioned separately.

The validity of VTR data as a basis for a sampling frame is generally supported by comparisons with total landings data from dealer records. All dealers that buy and sell species regulated by federal FMPs are required to report 100% of the landings.  These data are generally thought to constitute a near census of landings.  A comparison of species landings from VTR and dealer records for calendar year 2004 reveals some discrepancies, by species group, between these two sources (see text table below).  Overall, there is a 1.4% difference between dealer and VTR; however, this low percent difference is driven by a -10% difference for herring.  If herring is removed from the total, there is a 2.7% difference between the total kept species weight.

The large percent difference for monkfish may be attributed to the misreporting of monkfish product in the VTR.  If the incorrect product grade is reported (“monk” vs “monkt” representing whole monkfish and monkfish tails, respectively) then an underestimation of monkfish landings in the VTR may result.  Large percent differences for bluefish and spiny dogfish may be attributed to the inability to partition out the mandatory reporting landings (reflective of the VTR data) from the state landings data.  This is an issue unique to 2004, when mandatory dealer electronic reporting was implemented.  Additionally, total landings of bluefish and spiny dogfish represent a small fraction of the total landings, and these percentage differences are considered to be negligible.  Ideally, it would be preferable to use total kept species weight and days absent from dealer data to expand bycatch rates and in the variance calculations of total discards; however, the VTR data are the only source for gear and mesh information, two key components of fisheries.  (1994-present dealer data do not contain gear and mesh information.)

Species Group
VTR Landings
(mt, live)
Dealer Landings
(mt, live)
Difference
(mt, live)
Percent
Difference
Bluefish 2,357 3,423 1,067 31.2%

Herring

94,223 85,456 -8,766 -10.3%
Salmon - -    
Red crab 1,733 2,041 307 15.1%
Scallop 242,550 243,736 1,187 0.5%
Mackerel/Squid/Butterfish 97,400 97,083 -317 -0.3%
Monkfish 14,643 21,185 6,543 30.9%
NE Multi-species (Large mesh) 35,101 41,414 6,313 15.2%
NE Multi-species (Small mesh) 8,883 9,277 394 4.2%
Skate Complex
(7 species of skates)
13,054 16,073 3,020 18.8%
Dogfish, spiny 600 983 382 38.9%
Fluke/Scup/Black Sea Bass 11,732 13,887 2,155 15.5%
Surf Clam/Ocean Quahog* 295,381 295,381 0 0.0%
Tilefish 1,229 1,216 -13 -1.0%
Total 819,486 831,156 11,670 1.4%
         
Total minus herring 725,264 745,700 20,436 2.7%
* Surf clam and ocean quahog single source (VTR is the source for the Dealer data).

Measures of fishing effort may be in terms of trips, days absent, or days fished.  Days fished is the finest level of effort, representing the time the gear is actually deployed in the water (e.g., trawl duration, soak time for fixed gears, etc.), while days absent represents a coarser level of effort, generally measuring the time a vessel is away from port.  The lowest resolution of effort is the trip, which may encompass varying levels of days fished, days absent, and fishing power.  

The above comparisons of dealer and VTR-based landings estimates suggest that some of the expansion factors for estimating total discards, and the weighting factors for d/k ratios will be underestimated slightly.  Further work on factors underlying these disparities is needed and will be addressed in a subsequent phase of this project.

Northeast Fisheries Observer Program (NEFOP) Data

The NEFOP is a multi-purpose program that collects a broad range of data on all species that are encountered during a fishing trip, as well as gear characteristics data, economic information and biological samples.  The NEFOP employs trained, sea-going observers to collect these data, which also include weight, by species and disposition (retained and discarded), of the entire catch. 

Standard sampling protocols have been established and are utilized throughout the various fisheries[4].  For most gear types, observers use a ‘complete’ sampling protocol that includes obtaining species weights for both kept and discarded portions of all species in the catch on every haul.

In addition to the ‘complete’ sampling protocol, there is a ‘limited’ sampling protocol that is used on some gillnet trips where specific information for marine mammals is collected.  In a ‘limited’ sampling scenario, only kept species weights are obtained (no discard weights) since the observer must watch the gillnet gear during haul-back to observe if marine mammals roll out of the gear before the gear returns to the deck. 

Due to these two sampling protocols for data collection, two data sets were formed using the 2004 NEFOP data: one data set for fish that utilized the ‘complete’ sampling protocols and another for turtles, marine mammals, and birds that utilized both the ‘complete’ and ‘limited’ sampling protocols.

For the fish dataset, only observed hauls in which all discarded species were recorded are used.  In the majority of trips, all hauls are observed.  However, for some gear types – particularly scallop dredge, where fishing activity occurs continuously and a single observer cannot observe all hauls – it was necessary to expand discard species weights by the ratio of the number of total hauls to the number of observed hauls to account for all hauls in the trip.  The expanded discard weight was used in the subsequent discard-to-days-absent analysis (but not in the discard-to-kept analysis) because days absent is a ‘trip’ level variable representing the entire trip, not just the observed portion of the trip.  Observer training trips were excluded from the fish data set but were utilized for the protected species set, because it was assumed that training trips were capturing protected species information even though all discarded fish information might not be collected.   For the protected species data set, all on-watch hauls are included in the data set, regardless if discarded fish species were recorded.  Since all hauls are used in this data set, it was not necessary to adjust the discard weight to account for non-observed hauls. 

Quota-monitoring observed trips were included by gear type in the protected species set, but were partitioned out into separate strata for the fish set because of the total allowable catch limits associated with these access area programs.  There were limitations in carrying estimates for these strata forward due to the inability to identify all quota-monitoring trips in the VTR data.

Species hail weight can be reported in round or dressed weights; if kept hail weights are reported as ‘dressed’, then the hail weight is converted to round (live) weight using CFDBS conversion factors for the species.  All discard hail weights are assumed to be round (live) weight. Turtles, marine mammals, and sea birds are recorded as numbers of individuals.

The NEFOP trip data are collapsed into strata as defined above.  For each fleet sector, the number of observed trips, number of observed hauls, average trip length (in days), kept weight of all species in the trip, discarded weight of all (combined) species in the trip, and the discard weight of each species are calculated. 

A summary of the number of 2004 observed trips and sea days and 2004 commercial VTR trips and sea days by fleet sector and calendar quarter is presented in Table 2a-b.   There is a broad range of observer coverage by gear type in 2004; eleven of the fourteen gear types had observer coverage.  Lobster pot, crab pot, and clam dredge gear types were not covered in 2004.  Regionally sparse coverage occurred for longline, shrimp trawl, fish pots, and handline.  There are some gear types with very low industry activity and/or strong seasonal activity patterns, such as Scottish seine and purse seine, respectively. 

For the fleets examined in the analyses, there were a total of 3,587 observed trips and 126,498 VTR trips resulting in approximately 3% overall coverage.  Finer scale coverage rates vary among fleet and quarter.  The highest percentage coverage (45%) occurred in the Mid-Atlantic closed-area scallop dredge fleet.  It should be noted that percentage coverage is only one measure for monitoring adequacy, and that precision of discard rates, along with overall discard magnitude relative to population size, are the preferred measures for monitoring adequacy.


UNLIKELY CELLS

In the matrix of fleet by species/species group, there are some species and gears that are infeasible combinations (for example: scallops in longline gear, surf clam in gillnet gear, etc.).  With the assistance of the Plan Development Teams and Fishery Management Action Teams, cells have been identified as “unlikely” based on review of the previous 16 years of observer data, general knowledge of gear, fish distribution and abundance patterns.  Unlikely cells are indicated in the matrix as gray-shaded cells.  For some protected species, there was insufficient information with which to determine whether or not a cell was unlikely.  Although all analyses were conducted for all cells in the matrix, often the amount of coverage necessary to achieve a given level of precision for unlikely cells would exceed funding resources.  When evaluating coverage, the unlikely cells can be removed to provide a more realistic estimate of necessary coverage.  It is important to note that as fishing patterns or species abundance and/or distribution change, these gray-shaded cells may shift to reflect dynamic changes.

The occurrence of trips with zero discards is summarized in Table 3 and Table 4 for fish and protected species, respectively.  Generally, the unlikely gray-shaded cells correspond to trips where 100% of the trips had zero discards for the species.  Two exceptions are notable: in the scallop dredge fleets, trips are discarding squid-butterfish-mackerel and surfclam-ocean quahog. 


MISSING CELLS

The absence of observer coverage in feasible combinations of stratification variables (i.e., cells) causes problems in two ways.  First, if the cells are ignored, the basis for comparing the average bycatch ratio will vary by fishery, species and species group.  In this situation the inferences about the overall efficacy of a program are restricted to the set of cells with observer data.  Second, if the cells are included, it is necessary to make some assumption about the mean and variance of the discard rate for these cells.  This process is known as imputation, and it relies on information from the known part of the survey to impute information about the unknown.  Imputation of missing cells is routinely used in survey estimation, but it can be controversial because of the expert judgment required.  Use of imputed values to compute an overall estimate of the CV of a bycatch rate will lead to a conditional estimate.  “Conditional” in this context implies that the estimate depends on the set of rules/decisions used for imputation.  

As part of the feedback process for improving the sampling design, it is necessary to use imputed values as a basis for allocating coverage.  Imputation procedures have been developed for Northeast groundfish (Rago et al. 2005) using a multi-tier imputation procedure for three gear types.  Due to the diverse species and large geographic range of the present study, a detailed imputation procedure would be needed to account for the seasonal variability of the species over the geographic range.  Further work will continue to expand the imputation described in Rago et al. (2005) to provide appropriate means and variances by stratum for various species and species complexes and gear types.  Until that work is complete, we use a simple imputation approach using data from adjoining strata.  In this simple imputation, only the temporal stratification (calendar quarter) was relaxed to half year, recognizing that seasonal variation can occur for some species (Table 2a-b).  In the case of shrimp trawl, given the northern shrimp fishery is a seasonal fishery comprising half the year, the quarterly data were applied annually.  Data from adjoining cells were pooled to impute estimates for cells with zero or one trip.  However, simple imputation could not be applied to fleets where observer coverage was low or missing throughout the year (i.e. too few data to support the simple imputation approach).  In these cases, imputed values were not used, and the fleet was designated as a fleet in need of pilot coverage.  If some data were available, then some estimates were derived; however, the sea days needed to achieve a 30% CV were estimated based on pilot coverage levels (details are described below).

Pilot coverage is defined as a minimum level of coverage to acquire bycatch information with which to calculate variance estimates that in turn can be used to further define the level of sampling needed.  Based on Evaluating Bycatch: A National Approach to Standardized Bycatch Monitoring Programs (NMFS 2004), pilot coverage can range between 0.5 and 2%.  In this study, 2% of the annual VTR trips for a fleet, with a minimum of 12 trips per year (3 trips per quarter) and a maximum of 400 trips per year (100 trips per quarter) was used for pilot coverage.  The fleets that needed pilot coverage are indicated in Table 2a-b.  

Based on 2004 observer coverage, four scenarios were developed to determine when to use imputation or pilot coverage: 

  1. if observer coverage exists in all 4 quarters with sufficient sample sizes to generate quarterly CVs, then no imputation or pilot coverage was used;
  2. if observer coverage exists in 3 quarters with sufficient sample sizes to generate a CV, then the missing quarter was imputed using half-year estimates;
  3. if observer coverage exists in 1 or 2 quarters with sufficient sample sizes to generate a CV and the other 2 or 3 quarters had zero or 1 trips, then there were insufficient data to apply simple imputation, so pilot coverage was used; and
  4. if no observer coverage exists in all 4 quarters, then pilot coverage was used.

ESTIMATION OF BYCATCH RATES

There are many different methods for estimating bycatch rates.  Design-based estimators are often used for finfish bycatch (e.g., Pikitch et al. 1998, Stratoudakis et al. 1999, Rochet et al. 2002) while model-based estimators are more commonly used for predicting less frequent bycatch events (e.g., Walsh et al. 2002, Perkins and Edwards 1996).  Ratio estimators represent a simple form of model-based estimation within a sampling design.  Studies that have compared the use of ratio estimators with other simple and proportional probability estimators have reported mixed results.  Diamond (2003) found that ratio estimators overestimated discards compared to simple means based estimators.  Allen et al. (2001), however, found that ratio estimators performed better but that the appropriate covariate varied among species.  Discard estimation is a very active area of fisheries and statistical research.  Within the last year a number of very promising approaches (Miller and Skalski 2006, Kaiser 2006) that combine design and model-based estimation have been proposed.  These estimators will be examined in the future.  However, we anticipate that the sampling design proposed in this document is sufficiently robust to support many of the newly proposed methods.

For the purpose of the SBRM we examined a number of design-based approaches that have been advocated in the literature and tested the assumptions of each.  Bycatch rates were expressed as:

  1. the ratio of total weight of one or more species discarded to total weight of one or more species kept,
  2. the ratio of total weight of one or more species discarded to days absent,
  3. discards per trip.

The basic difference between method (2) and (3) is that “days absent” is assumed to contain more information about fishing effort than the sampling unit “trip”.  For the ratio estimators (1) and (2) we examined the effects of pooling ratios over strata, using the “separate” and “combined”  approaches given in Cochran (1963, p. 197-173).  Details of the separate and combined estimators follow a brief introduction to ratio estimators.    Overall, we examined two different ratio estimators (discard/kept [d/k] vs discard/days absent [d/da]) for two different pooling strategies (separate vs combined).  In addition, the discard per trip estimator (3) was applied individually to the datasets for d/k and d/da.  The only differences between the two data sets were slight variations in the number of cases available in each stratum.  Thus a total of six different estimators were applied to the set of 45 fleets and 60 species/species groups. 

Ratio Estimators

Bycatch rates for each fleet, quarter and species/species groups (stratum) were estimated using two ratios: discard to all species kept (d/k) and discard to days absent (d/da), Eq. 1a and 1b, respectively.

(1a) and  (1b) 

where Rjh is the bycatch rate of species group j in stratum h; dijh is the discards (for fish, weight in pounds; protected species, in numbers of animals) for species group j within trip i in stratum h; kih is the kept weight, in pounds, of all species within trip i in stratum h; daih is the days absent of trip i in stratum h.

The approximate variance of the estimate of Rjh is obtained from a first order Taylor series expansion about the mean. The computational formula for these quantities can be expressed as:

(2a)

and

(2b)

where dijh is the total discard weight of species group j in trip i within stratum h; kih is the total kept weight of all species in trip i within stratum hnh is the sample size (number of observed trips) in stratum h, and h is the mean kept landings of all species within the stratum.

Note that in this formulation of the variance, the finite population correction factor (fpc) – i.e., one minus the sampling fraction within the stratum – has been omitted.  This has been done to improve readability.  The fpc is included, however, in Eqs. 12, 16, and 19 for the total variance of the bycatch ratios.

The CV for the bycatch ratio for species group j in stratum h is defined as

(3) 

The number of trips necessary to achieve a 30% CV for species group j in stratum h is defined as

(4) 

where nh is the number of observed trips in stratum h, Nh is the number of VTR trips in stratum h, sD is the standard error of the total discard of species group j in stratum h,  is the discard ratio of species group j in stratum h.

The number of sea days () necessary to achieve a 30% CV for species group j in stratum h is defined as

(5)

where  is the average trip length of observed trips in stratum h.

Ratio assumptions

Eq. 2a and 2b are the computational formulas for a more general expression of the variance of a ratio (R=y/x) estimate which incorporates the covariance of the relationship between the numerator y and denominator x.  The correlation (ρ) between the numerator and denominator is simply the covariance divided by the product of the standard errors of the numerator and denominator.  The ratio estimator of a total Y can be written as the Y=(y/x)X where X is the total value of the covariate.  The approximate variance of Y based on a ratio estimator can be written as

(5.1)    

where Sy and Sx are the standard errors of y and x.  Note that increases in the correlation coefficient (ρ) will decrease the variance of the total.  Increases in ρ imply a higher degree of association between the numerator and denominator and imply that the variance will decrease when the ratio model is appropriate.  When ρ approaches zero, the benefits of ratio estimation decrease and the variance may actually increase because the squared ratio estimate (the second term within the parentheses on the right hand side of Eq. 5.1 could increase the variance of the total).

In general, the ratio estimate has a bias of order 1/n (Cochran 1963).  For moderate and large sample sizes, the bias is negligible.  In this study, approximately three quarters of the strata have sample sizes of 30 or smaller.  To evaluate the impact of bias in this study, the significance of correlation between sample size and ρ (the correlation of the ratio estimate, rho) was examined.

The correlation of the ratio estimate is defined as

(6)   

(7)   

(8)  

(9)  

where xij is days absent or kept pounds for species j in trip i; yij is discarded pounds of species j on trip i; nh is number of observed trips in stratum h;  r 2  is squared correlation coefficient for species j.

Results of the correlation analyses are summarized in Table 5 for the ratio of discards by species group to total kept.  Overall the correlation coefficients were low, but the exceptions are important and notable.  Correlations exceeded 0.47 in the New England large mesh trawl fishery for monkfish, and in the large and small mesh multispecies groundfish species.  Associations for small mesh trawls in New England were also strong for squid, mackerel, and butterfish and small mesh multispecies.  Correlations for skate discard rates were above 0.32 in the NE and MA large mesh trawl fisheries, above 0.48 in the NE and MA extra large mesh gill net fisheries, and above 0.2 in four of the six scallop dredge fisheries.  

Linearity assumptions

The ratio estimator assumes that a zero intercept regression is an appropriate model of the relationship between discard and kept (or days absent).  The putative linear relationship between discarded and kept components of observed trips was examined by gear type and species group.  For illustration purposes, two example plots of discard and kept are given using two different scales: nominal scale and fourth root transformation[5].  These two illustrative plots (Figure 1a-b) reveal that the fourth root transformation facilitates the depiction of information and does not obscure the underlying pattern of increasing variance and a zero intercept.  Thus, using a fourth root transformation, examples of the comparison between discard and kept (or days absent) are illustrated by thirteen fish species groups in otter trawl and gillnet gears by mesh sizes (presented in Figure 1c-zz) and by five protected species groups for longline, otter trawl, gillnet and scallop dredge (Figure 2a-j).  Departures from linearity are often controlled by large numbers of trips with zero discards.  When trips with zero discards are removed, improvement in linearity occurs. Examples of these are given for large-mesh groundfish discarded in the otter trawl and gillnet fleets (Figure 3a-d).

Rho and sample size analyses (using power = 0.80, alpha = 0.10; alternative hypothesis = ‘not equal’ and null value = 0) indicated that a low percentage of fleets and species groups had linear relationships using a ratio estimator (d/k or d/da).


ESTIMATION OF TOTAL DISCARDS

Three methods were examined to estimate total discards, precision, and coverage necessary to achieve a 30% CV for fleets and species/species groups:  1) separate ratio method; 2) combined ratio method, and 3) simple expansion method (mean discard per trip).  Cochran (1963) discusses these three methods in greater detail; we attempt to follow Cochran’s notation to facilitate comparisons.  Each method utilized quarterly estimates of bycatch rates (d/k and d/da) and associated CV, and the number of sea days necessary to achieve a 30% CV.  In these analyses, stratum is defined as fleet and species group.  We note that significant improvements in discard estimation may be possible through a variety of species-specific refinements.  These might be accomplished via use of additional covariates, post stratification, or other model-based approaches.

Method 1. Separate Ratio Method

Total discarded pounds of species j using Method 1:

(10a)     and   (10b)

where

(11a)         and   (11b) 

where 1,j is the total discarded pounds for species j; Kh is the VTR total kept pounds in stratum h; DAh is the VTR total days absent in stratum h; rs,jh is the separate ratio for species j in stratum h; djih is discards of species j from trip i in stratum h; kih is kept pounds of all species on trip i in stratum hdaih = days absent from trip i in stratum h.

Variance of 1,j

(12a)  

and

(12b)  

where 1,j is the total discarded pounds for species j; Kh is the VTR total kept pounds in stratum h; DAh is the VTR total days absent in stratum h; rs,jh is the separate ratio for species j in stratum h; djih is discards of species j from trip i in stratum h; kih is kept pounds of all species on trip i in stratum hdaih = days absent from trip i in stratum hNh is the number of VTR trips in stratum h; nh is the number of observed trips in stratum h.

Coefficient of variation of 1,j  is defined as:

(13)  

Method 2.  Combined Ratio Method

The combined ratio method is based on a ratio estimate pooled over all strata and trips within strata. Total discarded pounds for species j:

(14a)       and     (14b)   

where

(15a)         and     (15b)   

where 2,j is total discarded pounds for species j; Kh is VTR total kept pounds in stratum h; DAh is VTR total days absent in stratum h; rc,j is the combined ratio of species j in stratum h; djih is discards of species j from trip i in stratum h; kih is kept pounds of all species on trip i in stratum h; daih is days absent from trip i in stratum h, Nh is the number of VTR trips in stratum h; and nh is the number of observed trips in stratum h.

Variance of 2,j for species j

(16a) 

and

(16b)  

where 2,j is total discarded pounds for species j; Kh is VTR total kept pounds in stratum h; DAh is VTR total days absent in stratum h; rc,jh is the combined ratio of species j in stratum h; djih is discards of species j from trip i in stratum h; kih is kept pounds of all species on trip i in stratum h; daih is days absent from trip i in stratum h; Nh is the number of VTR trips in stratum h; and nh is the number of observed trips in stratum h.

Coefficient of variation of 2,j

(17)   

Method 3.  Simple Expansion Method: Mean Discard per Trip

Total discarded pounds for species j using Method 3:

(18)        

where djih is discards of species j from trip i in stratum h; Nh is the number of VTR trips in stratum h; nh is the number of observed trips in stratum h.

Note: 3 will differ between discard/da and d/kall sets due to expansion of discards to account for non-observed hauls in the d/da set.

Variance of 3,j for total discarded pounds using Method 3 for species j

(19)    

where 3,j is total discarded pounds for species j; djih is discards of species j from trip i in stratum hNh is the number of VTR trips in stratum h; nh is the number of observed trips in stratum h.

Coefficient of variation of 3,j

(20)   

The number of observer sea days (S30) necessary to achieve a 30% CV for a fleet and species/species group is defined as

(21)   

if a quarterly sea day estimate was not available (due to no observer coverage or the CV could not be estimated due to a bycatch rate of zero)  then the quarterly sea days were estimated by pilot coverage

(22) 

where  is 2% of the VTR trips in stratum h and quarter q, and 3 <= hq <= 100 trips; is the average trip length of VTR trips in stratum h and quarter q.

The composite number of sea days and trips necessary to achieve a 30% CV is independent of the three methods to estimate total discards.


META-ANALYSIS

A meta-analysis of the sixty species groups and thirty-nine fleets (excluding five quota-monitoring fleets and the Scottish seine fleet in the Mid-Atlantic) was conducted to compare estimates of total discards and the precision of the three methods and two bycatch ratio estimators. 

Total discards derived from each method and ratio estimator were compared to each other by plotting all combinations within a single plot for each major gear type and region.  The comparisons of total discard for four major gear types (longline, otter trawl, scallop dredge, and gillnet) and region are presented in Figures 4a-g.   The comparisons of standard error (SE) of total discard and the CV of total discards for the four major gear types by region are presented in Figures 5a-n.  For Figures 4 and 5, the symbol within each subplot represents a species/species group and mesh size, the line represents a regression through the data points, and the ellipse is the 68% confidence region.  

Generally, there is close relationship between all methods and ratio estimators for longline, otter trawl and scallop dredge for total discards (Figures 4a-g).  For longline and scallop dredge gear the estimated total discards were strongly correlated among estimators (Figures 4a,d,e).  Differences between the “combined” and “separate” estimators of  total discards in the trawl fisheries were negligible, but differences between d/k and d/da-based estimates were more pronounced (Figures 4b,c), especially for high values of discard.

There is some departure between methods and ratio estimators for gillnets in the Mid-Atlantic (Figure 4f) but not in New England (Figure 4g). This may be attributed to the use of days absent with a fixed gear fishery.  Some fleets ‘tend’ their nets while the gear in the water, thus days absent is correlated with soak time; this may not be true for fleets who do not tend. 

For measures of uncertainty of the estimate, there was general agreement among the three methods and two ratio estimators (Figures 5a-g).  Confidence ellipse for longline, gillnet, and scallop dredge were stronger than for otter trawl; however, otter trawl associations were tight.   In general, results in Figures 5h-n suggested a greater degree of dispersion among methods 1 to 3 when days absent was used as a measure of fishing effort.  Since days absent does not account for variations in steam time vs. fishing time nor for the effects of soak time for fixed gear, it was judged to be less useful than estimators based on a discard to kept ratio.  In particular, estimators based on the separate ratio method were more variable than those based on the combined ratio method.

Closer examination of the comparison of precision from the combined ratio method and the simple expansion method are presented in Figures 6a-g for four major gear types (longline, otter trawl, gillnet, and scallop dredge).  In these figures, the identity line and a reference line representing a 30% CV are given; the symbol represents a species/species group and mesh size.  There is general symmetry above and below the identity line, except for MA otter trawl where coverage is low and precision estimates are higher consequentially leading to higher coverage.

The meta-analyses indicate that generally there was little difference between the two bycatch ratios (d/da and d/k) for most species in most fleets, with the exception of gillnets where the d/da provided lower estimates of variation of total discards compared with d/k ratios.  Generally there was little difference between the three methods, but the ratio estimators tended to give higher CVs of the total than the simple expansion method.  A relatively large fraction of the overall estimates for species, gear, and mesh size had CVs less than 30%, irrespective of which method was used.

The tables presenting precision (Table 6), ranking of total discards (Table 7), and the sea days and trips necessary to achieve a 30% CV (Table 8, Table 9, and Table 10) are based upon the combined ratio method (Method 2) and the discard to kept ratio.  

The precision of the total discards by fleet and species is presented in Table 6 (see Appendix Table I for individual species).  Cells with adequate precision (at or below 30% CV) are identified with bold font.  Note that when a CV is reported for a fleet where pilot coverage is needed, the CV is based upon the available, limited observer coverage. 

For all species combined, CVs were estimated for 28 fleets.  Nineteen of these fleets (68%) had CVs less than or equal to 0.30 (Table 6).  For tilefish, 3 of the 4 fleets where discarded tilefish occurred were above 30% CV.  Of the 600 cells in the fleet by species matrix, 30% of the cells had CV less than or equal to 30%.  Caution should be used in evaluating the matrix in this manner, as this percentage does not include the cells where no discarding occurred (CV = null), nor does it incorporate the unlikely (gray-shaded) cells.  Additionally, the relative magnitude of the discard should be considered when evaluating the precision.  There are cases – for example, large-mesh NE multi-species in the mid-water trawls – where the magnitude of the total catch, rather than the precision of the estimate, is the most important factor.  It is not possible at this time to compile a complete list of all cases.

To provide insight into which species are discarded in each fleet, the total discard of each species group was ranked (highest pounds = 1, lowest pound = n) within a fleet.  The rank indicates the relative magnitude of the discarded species group within a fleet.  Ranking of total discard weight within a fleet for fish species group is presented in Table 7a, and ranking of total number of incidental takes of turtles, marine mammals, and sea birds within a fleet is presented in Table 7b (see Appendix Table II for individual species).  In the gillnet fleets, spiny dogfish are discarded the most (rank = 1 for all gillnet fleets), while in the scallop dredge fleets, scallops and skates are the two species most heavily discarded.  Although protected species are not often encountered, dolphins/porpoise occur more often in otter trawl fleets than other protected species, while sea birds and turtles are encountered more frequently than other protected species in the gillnet and scallop fleets.  Total discard weight for fish species and total numbers of incidental takes were also ranked within species group (Tables 7c and 7d, respectively; see Appendix Table III for individual species).  Compared to other fleets, the New England large-mesh otter trawl fleet discards the most dogfish and NE multispecies.  The open access, limited scallop dredge fleets discard the most scallops and monkfish.  Turtles are taken most often in the MA scallop trawl fleets.

The sea days and trips necessary to achieve a 30% CV for each species group and fleet are presented in Table 8 and Table 9, respectively (see Appendix Table IV and Appendix Table V for individual species).  The sea days and trips are additive across fleets within species groups (i.e., column sums); however, the days and trips are not additive across species group within fleets (i.e., row sums).  Fine-tuning of the unlikely (gray-shaded) cells may be necessary before making a final determination of the number of sea days and trips needed to monitor bycatch in the Northeast region, due to exceptions to the 30% CV standard, resource limitations, and relative magnitude of discards.  For example, the need for 6,058 observer days to estimate surf clam discards in the large mesh New England otter trawl fishery is driven by imprecise estimates of small numbers.  Such an allocation of observer days would be wasteful with respect to surf clam discards, and would over-sample by a factor of about 10 the estimated days necessary to obtain a CV of 30% for large-mesh groundfish species.

To determine the number of sea days needed to achieve a 30% CV within a fleet, the maximum number of sea days for all species groups in the study (i.e., the maximum number of days within a row) is used.  This ensures that all other species groups will have a 30% CV or less.  Based upon this approach, Tables 10a-b present the number of sea days and trips needed for each fleet for: 1) all 20 species groups considered in this study; 2) 15 species groups[6] (all of the fish species groups plus the turtles); 3) the 20 species groups filtering out the unlikely (gray-shaded) cells; and 4) the 15 species groups filtering out the unlikely cells.   In Tables 10a-b, the total number of sea days and trips needed to achieve a 30% CV for each of these four scenarios is attained by summing each column.  These totals range from 33,602 to 38,882 days; for comparative purposes, approximately 8,000 observer sea days were utilized by the NEFOP in 2004.

Given this fourfold disparity between the projected number of sea days needed to meet the CV objective and the maximum number of observer days expended in the history of the NEFOP, it is possible that further reductions in the number of sea days will be necessary.   This could be accomplished by applying a series of ‘filters’ in Table 8.   These potential filters are explained in detail in Chapter 6 of the Omnibus SBRM Amendment.  Briefly, these filters are based on considerations such as: 1) importance of discard with respect to the stock assessment or resource status for a given species; 2) elimination of cells in which the CV is below 30% at current levels of observer coverage; 3) elimination of cells in which discards are a minor component of the total discards for that species group; and 4) elimination of cells in which discards are a minor component of the total landings for that species group.


ACCURACY ANALYSES

Several tests were conducted to evaluate the potential sources of bias in the 2004 data.  We compared several measures of performance for vessels with and without observers present.  Bias can arise if the observed trips within a stratum are not representative of the other vessels within the stratum.  Such bias could arise if the vessels with observers on board consistently catch more or less than other vessels, if the average trip durations are different, or if observed vessels fish in different areas than the rest of the fleet.  Each of these hypotheses was tested by comparing observable properties in strata having data from vessels with and without observers. 

All vessels are required to report the total trip landings, the number of days absent from port, and the primary statistical area fished.  Average catches (kept pounds) by species groups for observed and total trips compared favorably (Figure 7) and followed an expected linear relationship.If the observed and unobserved trips within a stratum measure the same underlying process, one would expect no statistical difference in the average catches (and the standard deviations) between the VTR and observer data sets.  An examination of the distribution of these differences (Figure 8 and Figure 9) by species group indicates no evidence of systematic bias and general symmetry in the pattern of positive and negative differences[7].

The mean difference of species pounds were generally small relative to total trip pounds, and the average catch rates between the two data sets were not significantly different from zero in 12 of the 14 comparisons (Table 11).  Additionally, a paired t-test of the stratum-specific standard deviations of pounds kept showed significant differences from six of the 14 comparisons.  A strong correlation was detected in trip duration between observed and unobserved trips (Figure 10), with observed trips averaging about a quarter-day longer (Figure 11, Table 11).  However, the difference in stratum-specific standard deviations of trip length was significantly different from zero (p = 0.002).  Some skewing of the differences in mean trip duration is evident, with observed trips being slightly longer.

Two measures of spatial coherence were also examined.  Within stratum h (fleet and quarter) the expected number of observer trips by statistical area j  (Ejh) as the product of the proportion of VTR trips in Statistical Area j and stratum h  (Vjh) and the number of observed trips in stratum nh.   Thus, Ejh = Vjh * nh.   These expectations can then be compared to the actual frequencies (Ojh) of observed trips by statistical area.  Results of these analyses indicate that the spatial distribution of fishing effort for trips with observers on board closely matches the spatial distribution of trips for the stratum as a whole (Table 12).  It was possible to compute chi-square statistics for 86 strata.  The null hypothesis of observer proportions equal to VTR proportions was rejected (P<0.05) in 38 of the 86 comparisons.  This analysis used training and quota-monitoring trips, which have a disproportionately higher rate of observer coverage than other observed trips; this may explain the significant differences for otter fleets.  Murawski et al. (2005) compared the spatial distribution of 2003 otter trawl fishing effort for vessels with Vessel Monitoring Systems (VMS) with the distribution of fishing effort from 2003 observed trips.  Qualitatively, the spatial distributions match very well with high concentrations of effort near the boundaries of existing closed areas on Georges Bank and within the Gulf of Maine.  Moreover, the effort concentration profiles deduced from VMS data coincide almost exactly with the profiles derived from the observed trips.  Overall, these comparisons suggested strong coherency between these two independent measures of fishing locations; there is no evidence of bias in the observer data.  


OVERLAP ANALYSES

Fishing trips in a given stratum may catch species from more than one species group.  The degree of overlap among species groups has important implications for the efficacy of sampling within strata.  Accounting for the magnitude of overlap can circumvent this potential inefficiency.  The overlap approach developed and described by Rago et al. (2005) for New England groundfish can be expanded and applied to the species groups and fleets considered in this study.


OPTIMIZATION TOOL

The optimization model described by Rago et al. (2005) can be expanded to encompass more species groups and gear types.  For the optimization model to be useful, it will take extensive analyses to ensure that the assumptions necessary to set up the model are sensible.  Even then, the optimization model is simply a tool to help guide the allocation process.

The most important aspect of using the optimization model is that it explicitly incorporates a regular feedback mechanism for continuously improving the performance of the bycatch monitoring.  The optimization approach should be viewed as a set of Quality Assurance/Quality Control (QA/QC) measures that provide a formal way of updating and improving the sampling design as new information is obtained.  It interacts with the formal sampling design by using updated estimates of variances and overall patterns of fishing effort to improve, via reallocation of observer coverage, the overall performance of the sampling program.  Overall performance is measured as a composite of the precision of discard estimates.  Developing a composite measure of performance requires development of weighting factors for each species group and fishery.  As the dimensionality of the bycatch allocation process is very high (species groups x strata), the definition of an acceptable set of weighting factors will be challenging.

The optimization model also incorporates explicitly external constraints that affect the allocation of observer effort.  While the most important constraint is the total budget for observers, the prescribed percent coverage for regulatory programs (e.g., US-Canada resource sharing areas, B days, and scallop vessels in closed areas) has substantial impacts on the overall performance of the program.  The optimization model provides at least one measure of the impacts of externally imposed constraints.

The use of observer data for single species stock assessments and the sea day allocation are presented in Figure 12.  This overview illustrates the ‘feedback’ loop, and the use of observer data in the stock assessment process and in the sea day allocation process.  The stock assessments analyses benefit from the sea day allocation process through improved monitoring of bycatch.


SOURCES OF UNCERTAINTY / DISCUSSION

The difficulties of discard estimation are well known and have been described extensively in the literature (e.g., Rochet et al. 2002, Diamond 2002, Rago et al. 2005, Kaiser 2006).  In this report we have used a design-based approach to organize the basic concepts of inferring the behavior of a population from the properties of a sample.  The design-based approach should be viewed as a first approximation of the overall efficacy of an observer sampling program.  As additional information is obtained, more refined estimators of discards for individual or groups of species can be devised.  The design approach does not preclude such development; instead, it facilitates such development by ensuring that the sampling is robust to uncertainties in the prosecution of fisheries.  Allocation of observer effort to fisheries and quarters protects against unforeseen changes in seasonal effort patterns, shifts to new fisheries (e.g., trawlers to general category scallopers), or effects of closed areas.  Moreover, the design-based approach can help smooth out the allocation process over time, thereby reducing potential labor problems.  A design-based approach for biological sampling has proven to be an excellent technique for monitoring the biological attributes of landings.  An extension of this concept to observer coverage has similar advantages.

Some critical areas of concern include the following:

  1. Measures of overlap
  2. Influence of zero observations
  3. Influence of extremely high variation on measures of central tendency
  4. Alternative predictive variables
  5. Development of aggregate measures of performance/efficacy for the observer program
  6. Relationship between design and model based estimators
  7. Influence of over-stratification on bias of estimation
  8. Lack of persistence in fishing behavior over years
  9. Influence of fishing regulations on vessel behavior
  10. Imprecise estimation of location from VTR
  11. Utility of aggregate species measures of discard
  12. Improving correspondence between VTR and Dealer data
  13. Incorporation of more advanced statistical estimators that explicitly treat zero observations and over-dispersion
  14. Development of appropriate criteria to filter the importance of fisheries and species combinations for the estimation of adequate sampling coverage.

The statistical theory applicable to the estimation of fisheries bycatch is evolving rapidly, and significant advances are anticipated. Several promising methods, recently published or now under development, are expected to advance the reliability of discard estimation.  Field testing of these newer methods for multiple geographical regions and fisheries will take time.  Meanwhile, the sampling design developed in this report – and more importantly, the underlying data collected by NERO and NEFSC – should retain enough flexibility to accommodate/support many of these newer methods.


SUMMARY AND CONCLUSIONS

We stratified fisheries in the Northeast region into 45 fleets and examined discard rates of 60 species/species groups of fish, turtles, marine mammals and sea birds using 2004 NEFOP and VTR data.  Although several species and gear combinations were identified as unlikely, these were included in the analyses.  Since the emphasis of this study is to evaluate the precision and accuracy of the bycatch monitoring program, the discard rates and total discard weight are not presented.   

Two ratio estimators were used: discard to days absent and discard to kept pounds of all species.  Three computational methods were employed to derive these ratio estimates: a separate ratio method, a combined ratio method, and a simple expansion method.  In general, estimation of total discards was comparable for each ratio estimator and method.

We examined precision of all six estimates for each fleet and species/species group combination.  Again, precision levels were comparable for each estimator and method.  In the end, we selected the combined ratio method using the discard to kept pounds; data for kept pounds are more verifiable than data for days absent, and the combined ratio method better utilized information associated with kept pounds.

A 30% CV was selected as a target level of precision based upon the recommendation of the NWGB.  The number of observed sea days (and trips) necessary to achieve a 30% CV for species was derived for each fleet and species/species group combination.  The total estimated number of sea days necessary to achieve a 30% CV exceeded 33,000 days.

Analyses were performed to evaluate potential sources of bias in the 2004 NEFOP data.  In general, there was no evidence of a systematic bias in amount of kept pounds, trip duration or area fished between the NEFOP and VTR data.


ACKNOWLEDGMENTS

We wish to thank the Protected Species Branch staff of the NEFSC for their guidance on the use protected species data.  We thank Fred Serchuk for his helpful comments and support during the development of this report.


REFERENCES

Allen M, Kilpatrick D, Armstrong M, Briggs R, Perez N, Course G. 2001. Evaluation of sampling methods to quantify discarded fish using data collected during discards project EC 95/-94 by Northern Ireland, England, and Spain.  Fish Res 49:241-254.

Cochran WL.  1963.  Sampling Techniques.  New York (NY): J Wiley and Sons; 448 p.

Diamond S.  2003.  Estimation of bycatch in shrimp trawl fisheries: a comparison of estimation methods using field data and simulated data.  Fish Bull (US) 101:484-500.

Fogarty MJ, Gabriel WL.  2005.  Relative Precision of Discard Rate Estimators for the Northeast Groundfish Complex.  Unpublished document available from Northeast Fisheries Science Center, Woods Hole MA 02543.

Green RH. 1979.  Sampling design and statistical methods for environmental biologists.  New York (NY): J Wiley and Sons; 257 p.

Kaiser MK.  2006.  Development of an estimator of discards for the Northeast Observer Program.  In: MRAG Americas. Final Report – Task 1.  Northeast Fisheries Bycatch Analysis, Contact Number EA 1330-04-RQ-0129, MRAG Americas, Tampa FL.  June 2006, 94 p.

Miller TJ, Skalski JR.  2006.  Integrating design- and model-based inference to estimate length and age composition in North Pacific longline catches.  Can J Fish Aquat Sci 63:1092-1114.

Mendenhall W, Ott L, Schaeffer RL. 1971. Elementary Survey Sampling. Belmont (CA): Duxbury Press.  247 p,

Murawski S, Wigley S, Fogarty M, Rago P, Mountain D. (2005).  Effort distribution and catch patterns adjacent to temperate MPAs.  ICES J Mar Sci 62(6):1150-1165.

NMFS (National Marine Fisheries Service). 2004.  Evaluating bycatch: a national approach to standardized bycatch monitoring programs.  US Dep Comm NOAA Tech Memo NMFS-F/SPO-66, 108 p.

On-line version  http://www.nmfs.noaa.gov/by_catch/SPO_final_rev_12204.pdf

NMFS-NERO (National Marine Fisheries Service) Northeast Regional Office.  http://www.nero.noaa.gov/ro/fso/vtr_inst.pdf 

National Research Council (NRC) 1998. Review of Northeast Fishery Stock Assessments. Washington (DC): National Academy Press, 128 p.

NEFSC (Northeast Fisheries Science Center). 1996. Analysis of the 1994 fishing vessel logbook data. In: 22nd Northeast Regional Stock Assessment Workshop: Stock Assessment Review Committee consensus summary of assessments.  Woods Hole (MA): NEFSC Reference Doc. 96-13, 242 p.

New England Fishery Management Council (NEFMC).  2006.  Framework Adjustment 42 to the Northeast Multispecies Fishery Management Plan.  36 p.

NOAA (National Oceanic and Atmospheric Administration). 1996. Magnuson-Stevens Fishery Management and Conservation Act amended through 11 October 1996.  US Dep Comm NOAA Tech Memo NMFS-F/SPO-23.  121 p.

Perkins PC, Edwards EF. 1996.  A mixture model for estimating discarded bycatch from data with many zero observations: tuna discards in the eastern tropical Pacific Ocean.  Fish Bull (US) 94:330-340.

Pikitch EK, Wallace JR, Babcock EA, Erickson DL, Saelens M, Oddsson G. 1998, Pacific Halibut bycatch in the Washington, Oregon and California groundfish and shrimp trawl fisheries.  N Am J Fish Manage 18:569-586.

Rago PJ, Wigley SE, Fogarty MJ.  2005.  NEFSC bycatch estimation methodology: allocation, precision and accuracy.  US Dep Comm Northeast Fish Sci Cent Ref Doc 05-09; 44 p.  On-line version:  http://www.nefsc.noaa.gov/nefsc/publications/crd/crd0509/

Rochet M-J, Peronnet I, Trenkel VM.  2002.  An analysis of discards from the French trawler fleet in the Celtic Sea.  ICES J Mar Sci 59:538-552.

Stratoudakis Y, Fryer RJ, Cook RM, Pierce GJ.  1999.  Fish discarded from Scottish demersal vessels: Estimators of total discards and annual estimates for targeted gadoids.  ICES J Mar Sci 56:592-605.

Walsh WA, Kleiber P, McCracken M.  2002.  Comparison of logbook reports of incidental blue shark catch rates by Hawaii-based longline vessels to fishery observer data by application of a generalized additive model.  Fish Res 58:79-94.


Footnotes

[1] See http://www.nefmc.org/scallops/index.html for further information on the sea scallop FMP.

[2] with the exception of those vessels that hold only federal lobster permits.

[3] Reports prepared since 2000 may be found at http://www.nefsc.noaa.gov/nefsc/saw/.  Earlier reports are available by contacting saw_reports@noaa.gov.

[4] On-vessel sampling of large-volume fisheries can be difficult. Subsampling protocols were under development for the purse seine and mid-water pair trawl fisheries during 2004; thus results for species groups from these fleets should be considered preliminary.  Sampling protocols have since been established for these large-volume fisheries; the standardized sampling protocols for all fisheries with observer coverage are given in the Northeast Fisheries Observer Program Manual.  

[5] The fourth root transformation approximates a natural logarithm transformation without the difficulty of adding a constant (Green 1979).

[6] Magnuson-Stevens Act covers these 15 species groups.

[7] From mid-November 2004 through October 2005, Northeast multispecies regulations included a pilot program that prohibited discards of legal-sized groundfish and required fishermen to take specific actions when the catch of these species exceeded very low limits. There is evidence that compliance with these regulations was influenced by the presence of an observer (NEFMC 2006). Investigation of whether this effect also influenced discards was not attempted in this paper since the program was in effect for just over one month in 2004, a small number of vessels participated during this period, and the trips cannot be (directly) identified in the VTR data.


List of Acronyms and Abbreviations
ASMFC
Atlantic States Marine Fisheries Commission
CFDBS
Commercial Fisheries Database System
CV
coefficient of variation
d/da
discard/days absent
d/k
discard/kept
ESA
Endangered Species Act
FMP
Fishery Management Plan
fpc
finite population correction factor
MA
Mid-Atlantic
MAFMC
Mid-Atlantic Fishery Management Council
MMPA
Marine Mammal Protection Act
NE
New England
NEFMC
New England Fishery Management Council
NEFOP
Northeast Fisheries Observer Program
NEFSC
Northeast Fisheries Science Center
NERO
(NMFS) Northeast Regional Office
NMFS
National Marine Fisheries Service
NWGB
National Working Group on Bycatch
SARC
Stock Assessment Review Committee
SBRM
Standardized Bycatch Reporting Methodology
SE standard error
VMS
Vessel Monitoring System
VTR
Vessel Trip Report