Presentation is loading. Please wait.

Presentation is loading. Please wait.

PICES/NPRB Indicators Workshop Points from Background Papers and June 1 Deliberations.

Similar presentations


Presentation on theme: "PICES/NPRB Indicators Workshop Points from Background Papers and June 1 Deliberations."— Presentation transcript:

1 PICES/NPRB Indicators Workshop Points from Background Papers and June 1 Deliberations

2 SOURCES OF COMMENTS SAFE ANNEX AND PICES REPORT SAFE ANNEX AND PICES REPORT ALL THE BACKGROUND PAPERS** ALL THE BACKGROUND PAPERS** PRESENTATIONS PRESENTATIONS DISCUSSION BETWEEN PAPERS** DISCUSSION BETWEEN PAPERS** CONTEXT OF EXPERIENCE SEVERAL PLACES (ICES, EU, CANADA, FAO etc) CONTEXT OF EXPERIENCE SEVERAL PLACES (ICES, EU, CANADA, FAO etc)

3 STRUCTURE OF COMMENTS OVERALL MESSAGES OVERALL MESSAGES COMMENTS ON MAJOR REPORTS COMMENTS ON MAJOR REPORTS FROM PAPERS AND TALKS FROM PAPERS AND TALKS What is missing What is missing What is present but vague What is present but vague Present but needs further discussion Present but needs further discussion (Gradient across topics) MY OWN IDEAS OF USEFUL WAYS AHEAD

4 MESSAGE 1 YOU ARE IN PRETTY GOOD SHAPE!!! YOU ARE IN PRETTY GOOD SHAPE!!! A LOT OF WORK ON OBJECTIVES A LOT OF WORK ON OBJECTIVES APPRECIATION OF NEED FOR SPECIFICITY APPRECIATION OF NEED FOR SPECIFICITY OBJECTIVES FROM MANY SOURCES ARE CONVERGING OBJECTIVES FROM MANY SOURCES ARE CONVERGING SOME SOCIAL-ECONOMIC OBJECTIVES SOME SOCIAL-ECONOMIC OBJECTIVES INDICATORS BEING MATCHED TO OBJECTIVES INDICATORS BEING MATCHED TO OBJECTIVES Acknowledge two modes of use Acknowledge two modes of use NOT IN DESPERATION MODE NOT IN DESPERATION MODE

5 The Major Reports Both very good Both very good Different audiences match different content Different audiences match different content Both have enough detail to allow user preconceptions to guide selection of content Both have enough detail to allow user preconceptions to guide selection of content Do not try to put all the detail that any user will want. Aim for a Guidebook not an Encyclopaedia. (motivate and guide) Do not try to put all the detail that any user will want. Aim for a Guidebook not an Encyclopaedia. (motivate and guide) Make YOUR big messages clearer. Make YOUR big messages clearer.

6 Papers & Talks – Missing DPSIR structure – proving useful in DPSIR structure – proving useful in Organizing dialogue and reduction in number Organizing dialogue and reduction in number Matching Indicators to use in the overall processes Matching Indicators to use in the overall processes Risk management framework Risk management framework Overall absence of risk language in papers Overall absence of risk language in papers Beths talk has one good way forward Beths talk has one good way forward Need more focus on displaying uncertainty Need more focus on displaying uncertainty

7 Missing or Under-represented Of the suites of indicators: Of the suites of indicators: Spatial content (missing everywhere) Spatial content (missing everywhere) Size-based – under-represented relative to their performance elsewhere (esp. ICES) Size-based – under-represented relative to their performance elsewhere (esp. ICES) FORMAL SELECTION PROCESSES FORMAL SELECTION PROCESSES Needs to be more that a beauty contest Needs to be more that a beauty contest Jason and Beth had some impt. points Jason and Beth had some impt. points

8 PRESENT BUT VAGUE How to test performance of indicators during selection process How to test performance of indicators during selection process NOT the same for indictors used in AUDIT function and indicators used in CONTROL function NOT the same for indictors used in AUDIT function and indicators used in CONTROL function AUDIT – Targets primary, limits secondary AUDIT – Targets primary, limits secondary CONTROL – Limits primary, Targets 2ndary CONTROL – Limits primary, Targets 2ndary METHODS EXIST FOR TESTNG BOTH METHODS EXIST FOR TESTNG BOTH

9 Present but Vague Where do we get the reference points? Where do we get the reference points? Differentiate Indicator [say, SSB] from Reference Points [Bmsy, B 35% etc] Differentiate Indicator [say, SSB] from Reference Points [Bmsy, B 35% etc] Reversibility of impact? Responsiveness to management at all? Reversibility of impact? Responsiveness to management at all? The classic three stage model should have ONE biological (or socio-economic) fixed point and the rest is making uncertainty explicit – NOT two biological fixed points.

10 More Critical Thinking, Perhaps? When and how to use absolute vs relative indicators When and how to use absolute vs relative indicators Experience with IUCN decline criterion and marine species Experience with IUCN decline criterion and marine species Reference Points for different regimes Reference Points for different regimes Pop Size NO; uses of populations YES Pop Size NO; uses of populations YES Especially if likelihood of prompt detection of regime change is low Especially if likelihood of prompt detection of regime change is low

11 More Critical Thinking, Perhaps? If you like traffic light-y presentation, then biological calibration of cut-points is a crucial research topic If you like traffic light-y presentation, then biological calibration of cut-points is a crucial research topic What to do with tough decisions and multiple indicators What to do with tough decisions and multiple indicators EU experience with just B and F EU experience with just B and F US legislation on over-fishing and over-fished wont transfer readily to ecosystem metrics US legislation on over-fishing and over-fished wont transfer readily to ecosystem metrics

12 What other field works with Indicators in similar contexts? PSYCHOMETRICS PSYCHOMETRICS SIMILARITIES SIMILARITIES Fundamental underlying processes matter but are NOT accessible to direct measurement Fundamental underlying processes matter but are NOT accessible to direct measurement INDIRECT indices proliferate quickly and (if flexible) easily INDIRECT indices proliferate quickly and (if flexible) easily Normal is not a fixed point on ANY scale Normal is not a fixed point on ANY scale A LOT hinges on decisions A LOT hinges on decisions Abuse and/or mis-interpretation is easy Abuse and/or mis-interpretation is easy

13 Selection – Control function SIGNAL DETECTION THEORY SIGNAL DETECTION THEORY From Human Factors Research From Human Factors Research 70 year history (WWII was first flowering) 70 year history (WWII was first flowering) REQUIRES REQUIRES Reconstruction of history of values of indicator(s) Reconstruction of history of values of indicator(s) RECONSTRUCTION OF HISTORY OF WHAT GOOD DECISIONS WOULD HAVE BEEN!!! RECONSTRUCTION OF HISTORY OF WHAT GOOD DECISIONS WOULD HAVE BEEN!!! If we cant do that retrospectively how can we support decision- making into the future If we cant do that retrospectively how can we support decision- making into the future

14 SIGNAL DETECTION THEORY History of indicator gives direct record of what the decision would have been, had that indicator been used History of indicator gives direct record of what the decision would have been, had that indicator been used History of what good decisions would have been gives standard for whether the decision IN HINDSIGHT would have been right or wrong History of what good decisions would have been gives standard for whether the decision IN HINDSIGHT would have been right or wrong (Piet and Rice – Lower or not lower quota) (Piet and Rice – Lower or not lower quota)

15 SIGNAL DETECTION THEORY Four possible Outcomes: Four possible Outcomes: HIT (something should have been done and Indicator said DO IT) HIT (something should have been done and Indicator said DO IT) TRUE NEGATIVE (no mgmt response needed and indicator said status quo OK) TRUE NEGATIVE (no mgmt response needed and indicator said status quo OK) MISS (something should have been done but indicator did not say action was needed) MISS (something should have been done but indicator did not say action was needed) FALSE ALARM (nothing needed to be done but indicator called for management intervention) FALSE ALARM (nothing needed to be done but indicator called for management intervention)

16 SIGNAL DETECTION THEORY Results presented as 2 X 2 tables Results presented as 2 X 2 tables Perfect indicator has no Misses and False Alarms Perfect indicator has no Misses and False Alarms COSTS of Misses and False Alarms not the same!!!! (nor equal to different perspectives) COSTS of Misses and False Alarms not the same!!!! (nor equal to different perspectives) Simple method to choose decision point on indicator (reference point) to Simple method to choose decision point on indicator (reference point) to Minimize overall error rate Minimize overall error rate Control ratio of Misses and False Alarms (medical) Control ratio of Misses and False Alarms (medical) Easy to compare performance of Indicators Easy to compare performance of Indicators

17 AUDIT FUNCTION- PSYCHOMETRIC DIAGNOSTICS History of over a century History of over a century Many mistakes (and advocacy abuses), many lessons already learned Many mistakes (and advocacy abuses), many lessons already learned Uses are numerous Uses are numerous Career aptitude testing Career aptitude testing Legal competency for actions Legal competency for actions Personality disorders and counselling Personality disorders and counselling Extensive validation testing and codification of professional standards Extensive validation testing and codification of professional standards

18 PSYCHOMETRIC DIAGNOSTICS Reminder : Dealing with all the same problems: Reminder : Dealing with all the same problems: Fundamental underlying processes matter but are NOT accessible to direct measurement Fundamental underlying processes matter but are NOT accessible to direct measurement INDIRECT indices proliferate quickly and (if flexible) easily INDIRECT indices proliferate quickly and (if flexible) easily Normal is not a fixed point on ANY scale Normal is not a fixed point on ANY scale A LOT hinges on decisions A LOT hinges on decisions Abuse and/or mis-interpretation is easy Abuse and/or mis-interpretation is easy

19 PSYCHOMETRIC DIAGNOSTICS General Approach General Approach Have large battery of questions (= suites of indicators) – Binet, MMPI, Rorschach etc Have large battery of questions (= suites of indicators) – Binet, MMPI, Rorschach etc Test a large populace on the battery Test a large populace on the battery Test sets of individuals KNOWN with confidence to have specific disorders. Test sets of individuals KNOWN with confidence to have specific disorders. WHAT COMBINATION OF QUESTIONS IN WHAT WEIGHTING MAXIMALLY GROUPS AS DISTINCT THE KNOWN PATHOLOGIES WHILE LEAVING MOST OF POPULACE IN A CENTRAL CLOUD WHAT COMBINATION OF QUESTIONS IN WHAT WEIGHTING MAXIMALLY GROUPS AS DISTINCT THE KNOWN PATHOLOGIES WHILE LEAVING MOST OF POPULACE IN A CENTRAL CLOUD NOT TRIED IN ECOLOGY (that I know of) NOT TRIED IN ECOLOGY (that I know of)

20 The Classic three stage model HEALTHY CAUTIOUS CRITICAL HARVEST RATE BIOMASS LRP EN TH ? WHERE IS SC? CONVENTIONAL FISHERIES MANAGEMENT REBUILDING ENDANGERED SPECIES TOOLS

21 ONE Fixed Point in 3-stage model Some govt responsibility – law or policy Some govt responsibility – law or policy Serious or Irreversible harm (CBD & PA) Serious or Irreversible harm (CBD & PA) 1. Best biological estimate of that point (ICES is B lim – damaged productivity) 1. Best biological estimate of that point (ICES is B lim – damaged productivity) 2. Estimating current status relative to that point has uncertainty, so buffer is needed- B pa 2. Estimating current status relative to that point has uncertainty, so buffer is needed- B pa Point where PROBABILITY that true stock may be at the limit is > 0.05 Point where PROBABILITY that true stock may be at the limit is > 0.05 Risk averse management relative to makes whole system precautionary Risk averse management relative to makes whole system precautionary Gets F pairs from F which implies equilibrium B lim Gets F pairs from F which implies equilibrium B lim

22 The Issue of PREDICTABILITY Scenario Exploration [use your favourite term] in Climate Change and Marine Ecosystem Dynamics have an important difference: Scenario Exploration [use your favourite term] in Climate Change and Marine Ecosystem Dynamics have an important difference: CC - NO EXPECTATION OF ACCURACY ON TIME SCALES >30 DAYS OR 30 DAYS OR < 30 YEARS ECOSYSTEM – 3-7(10?) YEAR PATTERN IS CORE OF DECISION SUPPORT ECOSYSTEM – 3-7(10?) YEAR PATTERN IS CORE OF DECISION SUPPORT CC – key decisions are long-term strategic CC – key decisions are long-term strategic Ecosystem medium-term tactical Ecosystem medium-term tactical

23 What to Predict? Dont try to capture the inter-annual flutter Dont try to capture the inter-annual flutter How does probability of a an extreme event (good or bad) vary with natural or anthropogenic forcers? How does probability of a an extreme event (good or bad) vary with natural or anthropogenic forcers? Multi-factor non-parametric probability density estimation methods DO show inflections in plot of P(extreme event) as f (specified forcers) Multi-factor non-parametric probability density estimation methods DO show inflections in plot of P(extreme event) as f (specified forcers) Easy to use and interpret Easy to use and interpret DO require decisions about what is extreme DO require decisions about what is extreme


Download ppt "PICES/NPRB Indicators Workshop Points from Background Papers and June 1 Deliberations."

Similar presentations


Ads by Google