1. Effectiveness Monitoring: What’s Working to Restore Puget Sound? Leska Fore, Puget Sound Partnership Constance Sullivan, Friday Harbor Labs Ken Dzinbal, Puget Sound Partnership Scott Collyard, Dept. of Ecology & Many Contributors West Sound Watersheds Council & West Central LIO January 21, 2015

2. Measures of biological or physical parameters to determine whether actions were effective in creating a specific outcome.-- Redman et al., 2013 What do we mean by “Effectiveness Monitoring”?

3. Measures of biological or physical parameters to determine whether actions were effective in creating a specific outcome.-- Redman et al., 2013 What do we mean by “Effectiveness Monitoring”? Other fields call this: Program Evaluation

4. 1Context for Effectiveness Monitoring 2Statistical Meta-analysis 3Communication TOPICS

5. Action 1Context for Effectiveness Monitoring Ecosystem Pressure Animals Habitat

6. Action 1Context for Effectiveness Monitoring Ecosystem Pressure Animals Habitat Did the trees live?

7. Action 1Context for Effectiveness Monitoring Ecosystem Pressure Animals Habitat Did the trees live? Did temperature go down?

8. Action 1Context for Effectiveness Monitoring Ecosystem Pressure Animals Habitat Did the trees live? Did temperature go down? Are there more salmon?

9. Action 1Context for Effectiveness Monitoring Ecosystem Pressure Animals Habitat Did the trees live? Did temperature go down? Are there more salmon? Did salmon population increase?

10. Action 1Context for Effectiveness Monitoring Ecosystem Pressure Animals Habitat Did the trees live? Did temperature go down? Are there more salmon? Did salmon population increase? Easy to measure Harder to measure

11. 1Context for Effectiveness Monitoring 2Statistical Meta-analysis 3Communication TOPICS

12. “System-wide learning” “Shared measures” “Collaborative learning” 2 Statistical Meta-Analysis An analysis approach that supports:

13. 2 Statistical Meta-Analysis Many restoration projects across Puget Sound… …how do we compare without using the same methods?

14. Widely used in education and medical research Change statistics are standardized across projects Measures effectiveness of actions Before and After Statistics Temp After – Temp Before (Pooled Variance) 0.5 Change = = Cohen’s d Change statistic (Cohen’s d) is difference of the means divided by the standard deviation 2 Statistical Meta-Analysis

15. BeforeActionAfter 198620012010 Musselwatch begins BAN on TBT 2 Statistical Meta-Analysis Effectiveness of Tributyltin BAN

16. * Effect size +/- 90% CI ACTION: Ban Tributyltin (anti-fouling paint) EFFECTIVE: Reduced toxics in Mussels 16 Tributyltin (Mussel)

17. * Effect size +/- 90% CI 17 Tributyltin (Mussel) Interpretation Large effect size >0.8 Medium 0.2 – 0.8 Small effect < 0.2 ACTION: Ban Tributyltin (anti-fouling paint) EFFECTIVE: Reduced toxics in Mussels

18. 18 Not effective/ Effect Size Effective/ Worsening Improving Fecal bacteria reduced ACTIONS: Samish Basin - Inspections, farm plans EFFECTIVE: Reduced fecals, opened shellfish beds

19. 19 Variety of Actions evaluated for Effectiveness in Puget Sound

20. Vetting the Results 2 Statistical Meta-Analysis PSEMP Steering Committee PSEMP Toxics Work Group Dale Norton Rob Duff George Onwumere Scott Collyard Lincoln Loehr Jim West Sandie O’Neill Scott Redman Tracy Collier Mark Meyers Lyndal Johnson Kate O’Laughlin Josh Latterell Jason Toft Maggie Bell-McKinnon Scott Collyard Debby Sargeant Mindy Fohn Stuart Whitford Rick Haley Michael See Pascale Warren

21. Effectiveness Monitoring & the Action Agenda Nearshore habitat restoration increased benthic taxa richness B2. Protect and restore nearshore and estuary ecosystems Pesticide partial ban not effective; Eagle Harbor cap improved fish health; TBT ban was effective C1. Prevent, reduce, and control the sources of toxic contaminants entering Puget Sound Lake treatments reduced Phosphorus A2. Protect and restore upland, freshwater, and riparian ecosystems PIC programs in Liberty Bay reduced fecals in bays and streams C7. Ensure abundant, healthy shellfish for ecosystem health and for commercial, subsistence, and recreational harvest consistent with ecosystem protection

22. 1Context for Effectiveness Monitoring 2Statistical Meta-analysis 3Communication TOPICS

23. 3 Communication Science Decisions

24. 3 Communication Science Decisions Who is your audience?

25. 3 Communication Intended audience for results of Effectiveness Monitoring Project Sponsors Project Implementers Funders What do they need to know? Actions Outcomes Costs Challenges

26. Actions  Outcomes  Costs Fact Sheet + Narrative Samish Bay: Fecals & Shellfish King Co: Reforestation Nisqually: Estuary restoration www.psp.wa.gov/effectiveaction.php 3 Communication

28. 28

29. 29

30. 1Surprising amount of data to measure effectiveness and compare costs. 2Info buried in tech reports. 3Don’t wait for standardization – use change statistics now. 4Lots of interest in what’s working to restore Puget Sound. CONCLUSIONS

31. Coming up next… Effectiveness of Estuary Restoration Tidal wetlands Floodgates Channel formation Species use