Speaker IV: Debra Duarte
Title: Review of Methodologies for Detecting an Observer Effect in Commercial Fisheries Data
Abstract: Observers are deployed on commercial fishing trips to collect representative samples of discard behavior. However, some fishermen change their fishing habits when an observer is onboard. If the extent of this “observer effect” is substantial, the observed data will not be representative of unobserved trips, potentially biasing the estimation of discards. This can impact catch monitoring, stock assessments, and fishery management. Further, the increased variance in discard estimation can lead to higher observer coverage requirements to achieve precision targets. The purpose of this study was to examine the power and error rate of several published methods for detecting an observer effect using trip metrics such as landings and trip duration.
The simplest methods (t-test and F-test for difference of means and variances) were unable to reliably detect bias of less than 30% and could not distinguish between an observer effect and a deployment effect (non-random allocation of observer coverage within a stratum). A generalized linear mixed effect model (GLMM) was also not reliable at detecting low levels of bias but was not confounded by deployment effects and was relatively robust to changing coverage rates, except at the lowest coverage levels (e.g., 5%). The most complicated tests involved comparing differences between subsequent trips for observed-unobserved and unobserved-unobserved pairs. These were able to detect smaller observer effects (15-20%) and were not confounded by deployment effects, but were least reliable at the highest coverage rates (>60%), producing both high false positive and false negative rates. Sensitivity tests also showed differing detection accuracy as the distribution of the metric of interest changed. Thus the optimal test for detecting an observer effect will depend on the metric of interest, the coverage rate, and whether a deployment effect exists. Results should be considered carefully when declaring that an observer effect is or is not occurring