Applying False Discovery Rate (FDR) Control in Detecting Future Climate Changes ZongBo Shang SIParCS Program, IMAGe, NCAR August 4, 2009

North American Regional Climate Change Assessment Program (NARCCAP) Predicted Changes in Future Winter Temperature ( °C) Note: This figure shows the difference between the mean of future (2040 – 2069 ) winter temperature vs. current (1970 – 1999) winter temperature.

Can We Trust What We See? Note: Those two figures show the means of 10 replicate random fields that are generated from the same Matèrn semi-variogram model, but with different random seeds.

What’s the Problem with Pointwise Two-sample t Tests?

False Discovery Rate (FDR) Control FDR controls the expected proportion of incorrectly rejected null hypotheses (type I errors) among the rejected null hypotheses. Less conservative than Bonferroni procedures, with greater power than Familywise Error Rate (FWER) control, at a cost of increasing the likelihood of obtaining type I errors. Applications of FDR in Genes Expression and Microarray Applications of FDR in Functional Magnetic Resonance Imaging

Definition of False Discovery Rate Declared non- significant (fail to reject) Declared significant (reject) Total True null hypotheses UVm₀ Non-true null hypotheses TSm-m₀ m-RRm Let Q = V / (V + S) define the proportion of errors committed by falsely rejecting null hypotheses. Notice Q is an unobservable random variable. Define the FDR to be the expectation of Q:

False Discovery Rates for Spatial Signals Testing on clusters rather than individual locations Procedure 1: Weighted Benjamini & Hochberg (BH) procedure Procedure 2: Weighted two-stage procedure Procedure 3: Hierarchical testing procedure – Testing stage: control FDR on clusters – Trimming stage: control FDR on selected points Reference: Benjamini, Y. and Heller, R False discovery rates for spatial signals. Journal of the American Statistical Association. 102:

Simulation Studies 1. Random Fields 2. Random Field Block 3. Random Field Gradient

Simulation Study I: Two Random Fields Note: Those two figures show the means of 10 replicate random fields that are generated from the same Matèrn semi-variogram model, but with different random seeds.

Pre-defined Clusters

Simulation Study 1: Pointwise vs. False Discover Rate Control

9 Repeats on Simulation Study I

Simulation Study II: Pre-defined Block Trend

Simulation Study II: Average of 10 Replicates Random Field (Matèrn, σ = 0.4) + Block Trends

Simulation Study II: Pointwise vs. False Discover Rate Control

9 Repeats on Simulation Study II

Study III: Pre-defined Gradient Trend

Study III: Average of 10 Replicates Random Field (Matèrn, σ = 2) + Gradient Trends

Simulation Study III: Pointwise vs. False Discover Rate Control

9 Repeats on Simulation Study III

Applying FDR Control for Detecting Future Climate Changes Download climate datasets from NARCCAP program Calculate seasonal average Construct clusters from EPA Eco-regions Conduct two-sample t test on temperature/precipitation Pointwise p-values and corresponding z scores Build semi-variogram model to estimate spatial autocorrelation Calculate z score and p-value by cluster Reject clusters based on FDR control

GIS: Vector Dataset, Lambert Equal-Area Projection

61 regions rejected at q=0.25 level 56 regions rejected at q=0.1 level 54 regions rejected at q=0.05 level 51 regions rejected at q=0.01 level H 0 : Future Winter Temperature Increase by 3 ˚C

H 0 : Winter Temperature ↑ 1 ˚CH 0 : Winter Temperature ↑ 2 ˚CH 0 : Winter Temperature ↑ 3 ˚C H 0 : Winter Temperature ↑ 4 ˚CH 0 : Winter Temperature ↑ 6 ˚CH 0 : Winter Temperature ↑ 5 ˚C FDR Tests on Winter Temperature

H 0 : Winter Prec ↓ 20 Kg/ m²H 0 : ↓ 10 Kg/ m²H 0 : ↑ 10 Kg/ m²H 0 : ↑ 20 Kg/ m² H 0 : ↑ 50 Kg/ m²H 0 : Winter Prec ↑ 30 Kg/ m²H 0 : ↑ 75 Kg/ m²H 0 : ↑ 100 Kg/ m² FDR Tests on Winter Precipitation

Acknowledgement Dr. Steve Sain, IMAGe, NCAR Drs. Douglas Nychka, Tim Hoar, IMAGe, NCAR Dr. Armin Schwartzman, Harvard University University of Wyoming SIParCS, IMAGe, NCAR NARCCAP