1. Implications of Model Specification and Temporal Revisit Designs on Trend Detection Leigh Ann Starcevich (OSU) Kathryn M. Irvine (USGS) Andrea M. Heard (UCR, NPS)

2. Outline Question of interest Question of interest Case study Case study Trend models Trend models Simulation results Simulation results

3. Question of interest Trend estimation and testing Trend estimation and testing Components of variation impact power to detect trend Components of variation impact power to detect trend Three approaches to trend estimation and testing suggested by: Three approaches to trend estimation and testing suggested by: Urquhart, Birkes, and Overton (1993) Urquhart, Birkes, and Overton (1993) Piepho and Ogutu (2002) Piepho and Ogutu (2002) Kincaid, Larsen, and Urquhart (2004) Kincaid, Larsen, and Urquhart (2004) Which approach has highest power for trend detection for a given Type I error level? Which approach has highest power for trend detection for a given Type I error level?

4. Motivation SIEN lake chemistry monitoring SIEN lake chemistry monitoring Trend and status Trend and status Annual effort is limited in vast landscape Annual effort is limited in vast landscape ~800 lakes in network ~800 lakes in network

5. Sierra Nevada Network Lakes Sequoia-Kings Canyon NP: ~ 860,000 acres Yosemite NP: ~ 760,000 acres

6. Trend model Linear mixed model used to estimate trend and components of variance Linear mixed model used to estimate trend and components of variance Trend models contain both fixed and random effects Trend models contain both fixed and random effects Fixed effects describe the mean Fixed effects describe the mean Random effects describe the variance structure Random effects describe the variance structure

7. Variance components Site-to-site variation (σ a 2 ) Site-to-site variation (σ a 2 ) Does not affect the trend estimate or SE when same sites visited annually (Piepho and Ogutu, 2002) Does not affect the trend estimate or SE when same sites visited annually (Piepho and Ogutu, 2002) Year-to-year variation (σ b 2 ) Year-to-year variation (σ b 2 ) Visiting additional sites will not increase power to detect trend when year-to-year variation is high Visiting additional sites will not increase power to detect trend when year-to-year variation is high Site-by-year variation (σ c 2 ) Site-by-year variation (σ c 2 ) Requires within-year visits to a site (not estimable in SIEN lakes survey) Requires within-year visits to a site (not estimable in SIEN lakes survey) Random slope variation (σ t 2 ) Random slope variation (σ t 2 ) Might indicate subpopulations with different trends Might indicate subpopulations with different trends Residual error variation (σ e 2 ) Residual error variation (σ e 2 )

8. Power to detect trend Affected by: Affected by: Type I error level (α) Type I error level (α) Trend magnitude (β 1 ) Trend magnitude (β 1 ) Variance composition Variance composition Associated with a particular hypothesis test Associated with a particular hypothesis test One-sided vs. two-sided alternative hypothesis? One-sided vs. two-sided alternative hypothesis? Reflects monitoring goals Reflects monitoring goals We examine: We examine:

9. Revisit designs Panel designs allow more sites to be visited over time Panel designs allow more sites to be visited over time When all sites are not visited annually, data are purposefully unbalanced When all sites are not visited annually, data are purposefully unbalanced Serially-alternating augmented designs considered Serially-alternating augmented designs considered Connected across time for powerful trend tests Connected across time for powerful trend tests Incorporates more sites for status estimates Incorporates more sites for status estimates

10. [1-0] 12345678 XXXXXXXX Notation of McDonald (2003) Notation of McDonald (2003) Urquhart and Kincaid (1999) showed that [1-0] is best for trend estimation Urquhart and Kincaid (1999) showed that [1-0] is best for trend estimation

11. [(1-0),(1-3)] 12345678 XXXXXXXX XX XX XX XX

12. [(1-0),(1-8)]123456789 XXXXXXXXX X X X X X X X X X

13. Estimation of fixed effects Generalized least squares (GLS) used to estimate fixed effects Generalized least squares (GLS) used to estimate fixed effects where

14. Estimation of RE variance components ANOVA Type III when data are balanced ANOVA Type III when data are balanced REML for unbalanced data REML for unbalanced data Piepho & Ogutu (2002) Piepho & Ogutu (2002); Spilke, et al. (2005) ANOVA Type III and REML provide the same estimates when data are balanced ANOVA Type III and REML provide the same estimates when data are balanced Satterthwaite degrees of freedom with Geisbrecht-Burns approximation Satterthwaite degrees of freedom with Geisbrecht-Burns approximation

15. Approach 1: Urquhart, et al. Variance components obtained from model without fixed trend slope Variance components obtained from model without fixed trend slope Construct Φ(θ)=Var(Y) from variance components Construct Φ(θ)=Var(Y) from variance components Estimate β and SE(β) with GLS Estimate β and SE(β) with GLS This approach assumed the variance components were known This approach assumed the variance components were known Did not address estimation Did not address estimation We use REML We use REML

16. Approach 2: Piepho and Ogutu (2002) Extension of VanLeeuwen, et al (1996) Extension of VanLeeuwen, et al (1996) Random slope effect incorporated Random slope effect incorporated Trend and variance component estimation Trend and variance component estimation Trend testing conducted using a synthetic F test Trend testing conducted using a synthetic F test Wald F-test not invariant to location shifts Wald F-test not invariant to location shifts P&O relaxed assumption of independence between random site effect and random slope for invariant Wald F-test P&O relaxed assumption of independence between random site effect and random slope for invariant Wald F-test

17. Approach 3: Kincaid, et al. (2004) Two models used Two models used Trend model omits RE for year Trend model omits RE for year Variance components model omits fixed linear trend Variance components model omits fixed linear trend RE’s for site, year, interaction RE’s for site, year, interaction This paper focused on status estimation This paper focused on status estimation Trend approach mentioned incidentally Trend approach mentioned incidentally

18. Desirable properties Trend test Trend test Powerful Powerful Nominal test size Nominal test size Trend model Trend model Ability to accurately estimate trend Ability to accurately estimate trend Nominal CI coverage for trend Nominal CI coverage for trend Variance component estimation Variance component estimation

19. SIEN Lake Chemistry Pilot data: Seven lakes study & Western lakes study Pilot data: Seven lakes study & Western lakes study Three outcomes chosen for study Three outcomes chosen for study  Ca: high random site variability  Cl: high random slope variability  NO3: high year-to-year and residual error variation  residual error variation   Indicator 3: high year-to-year variation  Indicator 4: high residual error variation

20. Monte Carlo power simulation Simulate population of lakes from estimated fixed effects and variance components obtained from the case study data Simulate population of lakes from estimated fixed effects and variance components obtained from the case study data 1000 populations generated 1000 populations generated 3 independent random samples selected from each population 3 independent random samples selected from each population Generate known trend Generate known trend Annual decline of 1% or 4% Annual decline of 1% or 4% 10 years 10 years Impose revisit design  1/3 of effort to annual panel Impose revisit design  1/3 of effort to annual panel Simulation power is proportion of times that null hypothesis is correctly rejected at the α = 0.10 level Simulation power is proportion of times that null hypothesis is correctly rejected at the α = 0.10 level

21. Test size Large Var. Comp. Revisit design s = 10s = 60 Approach 123123 [1-0]0.3960.1040.5120.1620.0960.600 [(1-0),(1-3)]0.2280.1100.4000.1040.0960.514 [(1-0),(1-8)]0.2700.1420.3960.1400.1240.464 [1-0]0.5960.0980.6020.3940.1080.570 [(1-0),(1-3)]0.3600.0940.4080.2140.0840.428 [(1-0),(1-8)]0.3500.0960.4400.2120.0960.384 [1-0]0.1000.0940.5520.0580.0820.792 [(1-0),(1-3)]0.0940.1100.5560.0780.1260.808 [(1-0),(1-8)]0.0840.1040.4900.0640.0940.768 [1-0]0.3220.0880.3740.2920.0980.392 [(1-0),(1-3)]0.2100.0940.3020.1740.1140.288 [(1-0),(1-8)]0.1760.0800.2920.1100.0700.238

22. Simulation results Power approximations are too high when test size exceeds nominal rate Power approximations are too high when test size exceeds nominal rate Estimates of β 1 are generally unbiased Estimates of β 1 are generally unbiased Bias of β 1 most sensitive to revisit design, not trend approach Bias of β 1 most sensitive to revisit design, not trend approach Observed that bias of SE(β 1 ) was less severe as revisit cycle length increased Observed that bias of SE(β 1 ) was less severe as revisit cycle length increased

23. Rel. Bias of SE(β 1 ): σ a 2 large, p =-1%

24. Rel. Bias of SE(β 1 ): σ t 2 large, p= -1%

25. Rel. Bias of SE(β 1 ): σ b 2 large, p= -1%

26. Rel. Bias of SE(β 1 ): σ e 2 large, p= -1%

27. Discussion Approach 2 has most stable test size Approach 2 has most stable test size When σ e 2 high, Approach 2 overestimates σ a 2, σ t 2, σ b 2, and SE(β 1 ) When σ e 2 high, Approach 2 overestimates σ a 2, σ t 2, σ b 2, and SE(β 1 ) Poor indicator for monitoring Poor indicator for monitoring Simulation power is almost always lower than power approximations assuming large-sample theory Simulation power is almost always lower than power approximations assuming large-sample theory

28. Conclusions Trend test size should be assessed when examining power to detect trend Trend test size should be assessed when examining power to detect trend Including a random slope effect that is correlated with the random site effect in the mixed model approach provides nearly-nominal trend tests Including a random slope effect that is correlated with the random site effect in the mixed model approach provides nearly-nominal trend tests Examining variance components is useful for choosing monitoring indicators and revisit designs Examining variance components is useful for choosing monitoring indicators and revisit designs

29. Ongoing work Determine if bias of variance components estimates may be reduced Determine if bias of variance components estimates may be reduced Incorporate autocorrelation estimation Incorporate autocorrelation estimation Examine relationship between revisit cycle length and SE(β 1 ) Examine relationship between revisit cycle length and SE(β 1 )

30. Acknowledgements NPS Vital Signs Monitoring Agreement NPS Vital Signs Monitoring Agreement Linda Mutch Linda Mutch James Sickman, John Melack, and Dave Clow James Sickman, John Melack, and Dave Clow Kirk Steinhorst Kirk Steinhorst N. Scott Urquhart N. Scott Urquhart Tom Kincaid Tom Kincaid

31. Questions?