1. KSU Monitoring Designs # 1 WE PROBABLY COULD HAVE MORE FUN TALKING ABOUT THESE TRAFFIC STOPPERS

2. KSU Monitoring Designs # 2 WHO CLEARLY HAVE THE RIGHT OF WAY! BUT…

3. KSU Monitoring Designs # 3 DESIGNING MONITORING SURVEYS OVER TIME (PANEL SURVEYS) POWER, VARIANCE and RELATED TOPICS N. Scott Urquhart Senior Research Scientist Department of Statistics Colorado State University Fort Collins, CO 80527-1877

4. KSU Monitoring Designs # 4 OUTLINEOUTLINE  Anatomy Of Sampling Studies Of Ecological Responses Through Time  Collaborator = Tony Olsen, EPA, WED  http://www.oregonstate.edu/instruct/st571/urquhart/anatomy/index.htm http://www.oregonstate.edu/instruct/st571/urquhart/anatomy/index.htm   Urquhart, N.S. (1981). Anatomy of a study. HortScience 16:621-627.  Elaboration on  Survey Designs – GRTS – Work of Don Stevens  Temporal Designs  Power to detect trend – joint with Tom Kincade  Uses components of variance  Current work = estimating variance  Work of Sarah Williams, finishing MS this month

5. KSU Monitoring Designs # 5 A CONTEXT  “EMAP-TYPE SITUATIONS” EMAP = US EPA’S Environmental Monitoring and Assessment Program “EMAP-TYPEEMAPEnvironmental Monitoring and Assessment Program“EMAP-TYPEEMAPEnvironmental Monitoring and Assessment Program  Estimate status, changes, and trends in selected indicators of our nation’s ecological resources on a regional scale with known confidence.  Estimate status, changes, and trends in the extent and geographic coverage of our nation’s ecological resources on a regional scale with known confidence.  Describe associations between indicators of anthropogenic stress and indicators of condition.

6. KSU Monitoring Designs # 6 WHO MUST COMMUNICATE  Ecologists & Other Biologists  Statisticians  Geographers  Geographic Information Specialists  Information Managers  Quality Assurance Personnel  Managers, at Various Levels

7. KSU Monitoring Designs # 7 “SAMPLING”“SAMPLING”  A WORD OF MANY MEANINGS  A statistician often associates it with survey sampling  An ecologist may associate it with the selection of local sites or material  A laboratory scientist may associate it with the selection of material to be analyzed from the material supplied  Common general meaning, varied specific meanings

8. KSU Monitoring Designs # 8 THE SPECIAL NEED  Communication Demands a Distinction Between  The local process of evaluating a response, and  The statistical selection of a sampling unit, for example, A lake A lake A point on a steam A point on a steam A point in vegetation A point in vegetation  The terms  Response design  Sampling design or survey design  Can be used to make this distinction

9. KSU Monitoring Designs # 9 BASIC ROLES  Survey Design Tells Us Where To Go to Collect Sample Information or Material  Response Design Tells Us What To Do Once We Get There  But These Two Components Exist in a Broader Context

10. KSU Monitoring Designs # 10 AN IMPORTANT DISTINCTION  Monitoring Strategy  Conceptual  Impacted by objectives  Addressable without regard to the inference strategy  Inference Strategy  Places to evaluate the response  Relation between points evaluated and the population Ie, the basis for inference Ie, the basis for inference

11. KSU Monitoring Designs # 11 SAMPLING STUDIES OF ECOLOGICAL RESPONSES THROUGH TIME HAVE  Monitoring Strategy  Universe model  Statistical population  Domain design  Response design  Inference Strategy  Survey design  Temporal design  Quality assurance design These components exist regardless of the inference strategy These components exist for any monitoring strategy

12. KSU Monitoring Designs # 12 The UNIVERSE MODEL  Reality (Universe): Ecological Entity Within a Defined Geographic Area to be Monitored  Model of the Universe:  Development of monitoring approach requires construction of a model for the universe  Elements Of The Universe Model: Set of Entities Composing the Entire Universe of Concern

13. KSU Monitoring Designs # 13 The UNIVERSE MODEL  Population Description And Its Sampling Require Definition Of the “Units” in the Population  Discrete units: Lakes may be viewed this way Lakes may be viewed this way Individual trees can be viewed this way, too Individual trees can be viewed this way, too  Continuous structure in space of some dimension: 2-SPACE: Forests or Agroecosystems 2-SPACE: Forests or Agroecosystems 1-SPACE: Streams 1-SPACE: Streams 3-SPACE: Groundwater 3-SPACE: Groundwater

14. KSU Monitoring Designs # 14 A CONTINUOUS MODEL FOR STREAMS Strahler Orders Third Order First Orders First Order First Orders Second Order Second Order

15. KSU Monitoring Designs # 15 The STATISTICAL POPULATION  The Collection of Units (as modeled) Over Some Region of Definition  Spatial  Temporal  Spatial and Temporal  Population Definition Could Include Features Which Depend on Response Values  EX: acid sensitive streams at upper elevations

16. KSU Monitoring Designs # 16 The DOMAIN Design  Specifies Subpopulations or “Domains” of Special Interest  May Specify Meaningful Comparisons Between Domains  Similar to “planned comparisons” in experimental design situations  Domain design may depend in response values EX: Warm Versus Cold Water Lakes EX: Warm Versus Cold Water Lakes

17. KSU Monitoring Designs # 17 The RESPONSE DESIGN  The Response Design Specifies  The process of obtaining a response At an individual element (site) At an individual element (site) Of the resource Of the resource During a single monitoring period During a single monitoring period  Response: What Will Be Determined on an Element Needs to be responsive to the objectives of the monitoring activity Needs to be responsive to the objectives of the monitoring activity

18. KSU Monitoring Designs # 18 The INFERENCE STRATEGY  Is The Basis For Scientific Inference  Provides The Connection Between Objectives and the Monitoring Strategy  Monitoring Strategy Usually Must Rely On Obtaining Information on a Subset Of All Possible Elements in the Universe  Specifies Which Elements of the Universe Will Have Responses Determined on Them  Can Be Based on Either  Judgment selection of units Inferential validity rests on knowledge of relation between the universe and the units evaluated Inferential validity rests on knowledge of relation between the universe and the units evaluated –Why do a study if you know this much about the population?  Probability selection of units The focus here The focus here

19. KSU Monitoring Designs # 19 The SURVEY Design  Probability Based Survey Designs are Considered Here May Be Somewhat Limited To Sedentary Resources  Positive Features -- As An Observational Study  Permit clear statistical inference to well-defined populations  Measurements often can be made in natural settings, giving to greater realism to results

20. KSU Monitoring Designs # 20 The SURVEY DESIGN - CONTINUED  Disadvantages  Limited control over predictor variables  Restricts causative inference  Usually will produce inaccessible sampling points Good - for inference Good - for inference Bad - for logistics Bad - for logistics

21. KSU Monitoring Designs # 21 The TEMPORAL Design  The TEMPORAL DESIGN specifies the pattern of revisits to sites selected by the Survey Design  Sampled population units are partitioned into one (degenerate case) or more PANELS.  Each population unit in the same panel has the same temporal pattern of revisits.  Panel definition could be probabilistic or systematic  Several temporal designs follow after a brief discussion of the rest of the Anatomy, and a bit on site selection.

22. KSU Monitoring Designs # 22 QUALITY ASSURANCE DESIGN  Defines Those Activities Intended to Provide Data of Known Quality: Blind duplicates Blind duplicates Accepted chemical standards, etc Accepted chemical standards, etc  Can Provide Valid Estimates of the Variance Of Pure Measurement Error

23. KSU Monitoring Designs # 23 ON SITE SELECTION  Systematically Selected Sites  Good for means & totals, but do not support design-based estimate of variance  Probably OK for large areas like national forests,  Systematic designs can systematically miss things that have a natural layout. EX: Triangular grid (deliberately skewed) in early EMAP got fowled up with EX: Triangular grid (deliberately skewed) in early EMAP got fowled up with –Coastline in the Northeast –The canal network in Florida –Lakes east of the Cascade Mountain Range in Oregon  How to select spatially balanced, but random sites?

24. KSU Monitoring Designs # 24 GENERALIZED RANDOM TESSELLATION STRATIFIED (GRTS) DESIGN  Due to Don Stevens – see references  Allows  A continuous population model  Variable density sampling by defined areas  Accommodates an “imperfect frame” = reality  Sequential addition of points while maintaining spatial balance  Differing measurements Lots of points for inexpensive measures Lots of points for inexpensive measures A subset for more expensive measures A subset for more expensive measures A further subset for very expensive measures A further subset for very expensive measures Implemented in Southern California Bight Implemented in Southern California Bight

25. KSU Monitoring Designs # 25 GENERALIZED RANDOM TESSELLATION STRATIFIED (GRTS) DESIGN  Two GIS-based implementations  EMAP R code operates on ARC “Shape” files, and returns points there Begin at http://www.epa.gov/nheerl/arm/ Begin at http://www.epa.gov/nheerl/arm/ http://www.epa.gov/nheerl/arm/designpages/monitdesign/monitoring_design_info.htm http://www.epa.gov/nheerl/arm/designpages/monitdesign/monitoring_design_info.htm http://www.epa.gov/nheerl/arm/designpages/monitdesign/monitoring_design_info.htm http://www.epa.gov/nheerl/arm/documents/design_doc/psurvey.design_2.2.1.zip http://www.epa.gov/nheerl/arm/documents/design_doc/psurvey.design_2.2.1.zip  STARMAP – Dave Theobald  RRQRR operates completely in ArcGIS http://www.nrel.colostate.edu/projects/starmap/rrqrr_index.htm http://www.nrel.colostate.edu/projects/starmap/rrqrr_index.htm http://www.nrel.colostate.edu/projects/starmap/rrqrr_index.htm  Both Allow Variable (spatial) Sampling Rates  Generally much better than stratification  (We can talk about this more if you want)

26. KSU Monitoring Designs # 26 Urquhart, N.S. and T.M Kincaid (1999). Designs for detecting trend from repeated surveys of ecological resources. Journal of Agricultural, Biological and Environmental Statistics 4: 404 - 414. Initially presented at the invited conference Environmental Monitoring Surveys Over Time, held at the University the Washington, Seattle, in 1998 THE FOLLOWING MATERIAL WAS ADAPTED FROM

27. KSU Monitoring Designs # 27 MOTIVATING SITUATION  In 1986 Oregon Department of Fisheries and Wildlife Sought a “One Time” Probability Sampling Design To Survey Coastal Salmon. They Used It In 1990.  It showed earlier estimates of salmon returns to spawn to have been grossly overstated.  Consequence: continue to repeat an available design.  How Good Is The Repeated Use Of Such a Design For Estimating Trend?

28. KSU Monitoring Designs # 28 CONCLUSIONSCONCLUSIONS  General: Power for Trend Detection  Planned revisits are far superior to obtaining revisits from random “hits”  Year Variance: Power Deteriorates Fast as Increases  Site Variance:  No problem with revisit designs.  Without revisits it increases residual variance.  Sampling Rate: Power Increases with Sampling Rate (No surprise!)

29. KSU Monitoring Designs # 29 EVALUATION CONTEXT  General Perspective  Finite population sampling  But model assisted A generalization of the “error analysis” perspective of samplers A generalization of the “error analysis” perspective of samplers But recognizing realities of natural resource sampling But recognizing realities of natural resource sampling  Specific Perspective  Finite population, like of stream segments.  Response exists continuously in time, or at least for reoccurring blocks of time.  Take independent samples at different points in time (during an “index window”)

30. KSU Monitoring Designs # 30 EVALUATION CONTEXT (CONTINUED)  Model:  Sites (or stream segments) = a random effect  Years = a random effect, but may contain trend  Residual = a random effect Specific evaluation time Specific evaluation time Variation introduced by collection protocol Variation introduced by collection protocol Crew effect, if present Crew effect, if present –(often present for large surveys) “Measurement error” - broadly interpreted “Measurement error” - broadly interpreted

31. KSU Monitoring Designs # 31 PANEL PLANS = “TEMPORAL DESIGNS”  Sampled Population Units are Partitioned into One (Degenerate Case) or More Panels  Each population unit in the same panel has the same temporal pattern of revisits.  Panel definition could be probabilistic or systematic  Specific Plans  Always revisit  Never revisit  repeated surveys  Random revisits and other plans

32. KSU Monitoring Designs # 32 TEMPORAL DESIGN #1: ALWAYS REVISIT = ONE PANEL (This is Wayne Fuller’s “PURE PANEL”)

33. KSU Monitoring Designs # 33 TEMPORAL DESIGN #2: NEVER REVISIT = NEW PANEL EACH YEAR (INDEPENDENT SURVEYS IN A LARGE POPULATION)

34. KSU Monitoring Designs # 34 TEMPORAL DESIGN #3: ROTATING PANEL like NASS

35. KSU Monitoring Designs # 35 TEMPORAL DESIGN #3: ROTATING PANEL  A Rotating Panel Design Is The Temporal Design Used By The National Agricultural Statistical Service (US - “NASS”)  This Temporal Design Is “Connected” In The Experimental Design Sense  It is fairly well suited for estimation “status,”  But not nearly particularly powerful for detecting trend over intermediate time spans

36. KSU Monitoring Designs # 36 TEMPORAL DESIGN: SERIALLY ALTERNATING (ORIGINAL EMAP)  This Temporal Design Is “Unconnected” in the Experimental Design Sense.

37. KSU Monitoring Designs # 37 TEMPORAL DESIGN #5: AUGMENTED SERIALLY ALTERNATING (CURRENTLY USED BY EMAP FOR SURFACE WATERS)  This Temporal Design Is “Connected” in the Experimental Design Sense.

38. KSU Monitoring Designs # 38 TEMPORAL DESIGN #6: RANDOM PANELS

39. KSU Monitoring Designs # 39 STATISTICAL MODEL  Consider A Finite Population Of Sites  {S 1, S 2, …, S N }  and a Time Series Of Response Values At Each Site:  A finite population of time series  Time is continuous, but suppose Only a sample can be observed in any year, and Only a sample can be observed in any year, and Only during an index window of, say, 10% of a year Only during an index window of, say, 10% of a year

40. KSU Monitoring Designs # 40 STATISTICAL MODEL -- II

41. KSU Monitoring Designs # 41

42. KSU Monitoring Designs # 42 STATISTICAL MODEL -- III

43. KSU Monitoring Designs # 43 STATISTICAL MODEL -- IV  If P Indexes Panels, Then  Sites are nested in panels: p( i ) and  Years of visit are indicated by panel with n pj > 0 or n pj = 0 for panels visited or not visited in year j  The vector of cell means ( of “visited” cells) has a covariance matrix 

44. KSU Monitoring Designs # 44 STATISTICAL MODEL -- V  Now Let X Denote a Regressor Matrix Containing a Column Of 1’s and a Column of the Numbers of the Time Periods Corresponding to the Filled Cells. The Second Elements of Contain an Estimate Of Trend and its Standard Error.

45. KSU Monitoring Designs # 45  Ability of a Panel Plan to Detect Trend Can Be Expressed As Power.  We Will Evaluate Power in Terms of Ratios of Variance Components:  and of TOWARD POWER

46. KSU Monitoring Designs # 46 A SIMULATION STUDY TO MAKE POWER COMPARISONS   n = 60  N = 60, 240, 600, 1200, 10,000  ==> Sampling rates of 100%, 25%, 10%, 5%, ~ 0%  ==> Sampling rates of 100%, 25%, 10%, 5%, ~ 0%

47. KSU Monitoring Designs # 47 ALWAYS REVISIT, or EMAP-LIKE

48. KSU Monitoring Designs # 48 0.000, 0.075, 0.15, 0.30 ALWAYS REVISIT, or EMAP-LIKE

49. KSU Monitoring Designs # 49 ALWAYS REVISIT, or EMAP-LIKE

50. KSU Monitoring Designs # 50 NEVER REVISITALWAYS REVISIT, or EMAP-LIKE

51. KSU Monitoring Designs # 51 RANDOM REVISITALWAYS REVISIT, or EMAP-LIKE NEVER REVISIT

52. KSU Monitoring Designs # 52 NEVER REVISITALWAYS REVISIT, or EMAP-LIKE RANDOM REVISIT

53. KSU Monitoring Designs # 53 ROTATING PANEL

54. KSU Monitoring Designs # 54 CONCLUSIONSCONCLUSIONS  General: Power for Trend Detection  Planned revisits are far superior to obtaining revisits from random “hits”  Year Variance: Power Deteriorates Fast as Increases  Site Variance:  No problem with revisit designs.  Without revisits it increases residual variance.  Sampling Rate: Power Increases with Sampling Rate (No surprise!)

55. KSU Monitoring Designs # 55 CURRENT WORK   Stevens D.L. Jr and A.R. Olsen (2003). Variance estimation for spatially balanced samples of environmental resources. Environmetrics 14: 593-610.   Proposed a local estimator for variance.   I have been using some variance component estimators. How do these two approaches relate? Should one be used rather than the other?   MS Student – Sarah Williams   Use local estimator for things like status measures Because it includes some site variance   Use components of variance for trend studies Revisits to sites remove most of the effect of that component   Currently investigating variance component of trend   And its impact on trend detection

56. KSU Monitoring Designs # 56 This research is funded by U.S.EPA – Science To Achieve Results (STAR) Program Cooperative Agreement # CR - 829095 The work reported here today was developed under the STAR Research Assistance Agreement CR-829095 awarded by the U.S. Environmental Protection Agency (EPA) to Colorado State University. This presentation has not been formally reviewed by EPA. The views expressed here are solely those of presenter and STARMAP, the Program he represented. EPA does not endorse any products or commercial services mentioned in this presentation. FUNDING ACKNOWLEDGEMENT

57. KSU Monitoring Designs # 57 DISTRIBUTION OF SIMULATED POWER: 10 YEARS SITE VARIANCE = 1.875; YEAR VARIANCE: 0.30 0.10 0.075 RATE = 25% RATE = 5% RATE = 10%

58. KSU Monitoring Designs # 58 DISTRIBUTION OF SIMULATED POWER: 20 YEARS SITE VARIANCE = 1.875; YEAR VARIANCE: 0.30 0.10 0.075 RATE = 25% RATE = 5% RATE = 10%

59. KSU Monitoring Designs # 59 DISTRIBUTION OF SIMULATED POWER: 10 YEARS SITE VARIANCE = 2.50; YEAR VARIANCE: 0.30 0.10 0.075 RATE = 25% RATE = 5% RATE = 10%

60. KSU Monitoring Designs # 60 DISTRIBUTION OF SIMULATED POWER: 20 YEARS SITE VARIANCE = 2.50; YEAR VARIANCE: 0.30 0.10 0.075 RATE = 25% RATE = 5% RATE = 10%