1. Evaluating and refining the
ORCS Working Group approach to data-poor fisheries
stock status and catch limit estimation
Christopher Free1, Olaf Jensen1, John Wiedenmann1, Jonathan Deroba2
1 Rutgers University, 2 NOAA-NEFSC
Mid-Atlantic ORCS Workshop
November 30, 2016| JC NERR

2. Step 1. Estimate stock status using Table of Attributes (TOA)
Risk of overexploitation #
Attribute
Low (1)
Moderate (2)
High (3) 1
Status of assessed stocks
<10% overfished
10-25% overfished
>25% overfished 2
Refuge availability
<50% of habitat accessible
50-75% of habitat accessible
>75% of habitat accessible 3
Behavior affecting capture
Low susceptibility
Average susceptibility
High susceptibility 4
Morphology affecting capture 5
Discard rate
Discards <10% of catch
Discards 10-25% of catch
Discards >25% of catch 6
Targeting intensity
Not targeted
Occasionally targeted
Actively targeted 7
M compared to dominant species
Higher mortality rate
Equivalent mortality rates
Lower mortality rate 8
Occurrence in catch
Sporadic in catch
Common in catch
Frequent in catch 9
Value
<US$1 / pound (5-year mean)
US$ / lb (5-year mean)
>US$2.25 / lb (5-year mean) 10
Recent trends in catch
Increasing last 5 years (score=1.5)
Stable last 5 years (score=1.5)
Decreasing last 5 years 11
Habitat loss
No time in threatened habitats
Part time in threatened habitats
Full time in threatened habitats 12
Recent trends in effort
Decreasing last 10 years
Stable last 5 years
Increasing last 5 years 13
Recent trends in abundance 14
Regulatory effectiveness
Most of resource is protected
Much of resource is protected
None of resource is protected

3. Step 1. Estimate stock status using Table of Attributes (TOA)
Risk of overexploitation #
Attribute
Low (1)
Moderate (2)
High (3) 1
Status of assessed stocks
<10% overfished
10-25% overfished
>25% overfished 2
Refuge availability
<50% of habitat accessible
50-75% of habitat accessible
>75% of habitat accessible 3
Behavior affecting capture
Low susceptibility
Average susceptibility
High susceptibility 4
Morphology affecting capture 5
Discard rate
Discards <10% of catch
Discards 10-25% of catch
Discards >25% of catch 6
Targeting intensity
Not targeted
Occasionally targeted
Actively targeted 7
M compared to dominant species
Higher mortality rate
Equivalent mortality rates
Lower mortality rate 8
Occurrence in catch
Sporadic in catch
Common in catch
Frequent in catch 9
Value
<US$1 / pound (5-year mean)
US$ / lb (5-year mean)
>US$2.25 / lb (5-year mean) 10
Recent trends in catch
Increasing last 5 years (score=1.5)
Stable last 5 years (score=1.5)
Decreasing last 5 years 11
Habitat loss
No time in threatened habitats
Part time in threatened habitats
Full time in threatened habitats 12
Recent trends in effort
Decreasing last 10 years
Stable last 5 years
Increasing last 5 years 13
Recent trends in abundance 14
Regulatory effectiveness
Most of resource is protected
Much of resource is protected
None of resource is protected
Average TOA score = 2.25
Moderate risk of overexploitation
Black sea bass
True status: moderate

4. = Step 2. Estimate OFL using historic catch statistic Overfishing
limit (OFL)
2.0 * historic catch statistic low risk
1.0 * historic catch statistic moderate risk
0.5 * historic catch statistic high risk =
Historic catch statistic = e.g., mean, median, 75th percentile of XX-yr catch history
Overfishing limit = 1.0 x 3129 kg = 3129 kg
True OFL: 3903 kg

5. Objectives Evaluate the performance of the ORCS approach
Refine the ORCS approach to improve performance
Weight attributes by importance
Account for interactions between attributes
Account for non-linearity in attribute behavior
Empirically identify the best catch statistics and scalars
Develop a web tool for implementing the refined approach
Apply the refined approach to data-poor stocks in the Mid-Atlantic

6. We evaluate the ORCS approach by applying it to 152 data-rich stocks with traditional stock assessments
RAM Legacy Stock Assessment Database (Ricard et al. 2012)

7. How well does the current approach predict stock status?
We use Boosted Regression Trees (BRT) to:
Weight attributes by their predictive power
Account for interactions between attributes
Account for non-linearity in attribute behavior
Olaf’s a good method would…
Answer? Poorly.

8. What do we need to do to refine the approach?
Weight attributes by their importance
Account for interactions between attributes
Account for non-linearity in attribute behavior
And we need to accommodate missing values.
How do we do this?
Boosted classification trees
combine machine learning and classification and have been celebrated for their predictive power.

9. The boosted classification tree model

10. How well does the refined approach predict stock status?
Training dataset
123 stocks, (80% of data)

11. How well does the refined approach predict stock status?
Low risk
Mod risk
High risk
Test dataset:
29 stocks (20% of data)
Answer? Hmm…

12. How does the refined approach compare to other catch-only methods?
Applied to 29 stocks in test dataset.
Method
Kappa
Accuracy 1
Refined ORCS approach
0.270
0.621 2
SSP-20132
0.223
0.483 3
SSP-20022 4
CMSY3
0.210
0.500 5
mPRM4
0.127
0.517 6
Original ORCS approach
-0.043
1 10 underexploited, 15 fully exploited, and 4 overexploited stocks
2 SSP = Stock status plot methods from Froese & Kesner-Reyes 2002 and Kleisner et al. 2013
3 CMSY = catch-MSY modified from Martell & Froese 2013
4 mPRM = modified panel regression model from Costello et al. 2012

13. Remember Step 2? Estimate the overfishing limit by scaling a historic catch statistic based on stock status
Low risk stocks:
90th percentile, whole time series
Moderate risk stocks:
25th percentile, last 10 years
High risk stocks:
10th percentile, whole time series

14. How well does the refined approach predict the OFL?
log(OFLknown)
log(OFLpredicted)
OFLpredicted / OFLknown
Risk of overexploitation

15. How well does the refined approach predict the OFL?
log(OFLknown)
log(OFLpredicted)
OFLpredicted / OFLknown
Risk of overexploitation

16. How well does the refined approach predict the OFL?
log(OFLknown)
log(OFLpredicted)
OFLpredicted / OFLknown
Risk of overexploitation
Answer? Usefully!

17. https://cfree.shinyapps.io/refined_orcs_approach/
Web tool for implementing the approach

18. https://cfree.shinyapps.io/refined_orcs_approach/
Web tool for implementing the approach

19. Conclusions Any questions?
The current ORCS approach is a poor predictor of stock status and should not be used for tactical management decisions.
The refined ORCS approach is a better predictor of stock status but still struggles with high risk stocks.
With conservative catch scalars, the refined ORCS approach estimates catch limits that prevent overfishing in accordance with U.S. legal mandates.
Web tool:
Any questions?

21. Han 2015

24. (generally good shape)
Assessed fisheries
(generally good shape)
Unassessed fisheries
(generally bad shape)
Underexploited
Fully exploited
Overexploited
Costello et al Science