Climate Forecasting Unit CFU R common diagnostics CFU_load CFU_season CFU_clim CFU_anoCFU_anocrossvalid CFU_plotclimCFU_plotano CFU_smoothing CFU_trend.

Slides:



Advertisements
Similar presentations
The Live Access Server (Access to observational data) Jonathan Callahan (University of Washington) Steve Hankin (NOAA/PMEL – PI) Roland Schweitzer, Kevin.
Advertisements

Part II – TIME SERIES ANALYSIS C3 Exponential Smoothing Methods © Angel A. Juan & Carles Serrat - UPC 2007/2008.
1 Sylvia Murphy National Center for Atmospheric Research File Handling with CMPS.
Climate Specific Tools: The cdutil Package. cdutil - overview The cdutil Package contains a collection of sub- packages useful to deal with Climate Data.
STA305 week 31 Assessing Model Adequacy A number of assumptions were made about the model, and these need to be verified in order to use the model for.
Climate Predictability Tool (CPT)
Maximum Covariance Analysis Canonical Correlation Analysis.
Objectives (BPS chapter 24)
Statistical tools in Climatology René Garreaud
July 1, 2008Lecture 17 - Regression Testing1 Testing Relationships between Variables Statistics Lecture 17.
Copyright ©2011 Brooks/Cole, Cengage Learning Testing Hypotheses about Means Chapter 13.
By Hrishikesh Gadre Session II Department of Mechanical Engineering Louisiana State University Engineering Equation Solver Tutorials.
Resampling techniques Why resampling? Jacknife Cross-validation Bootstrap Examples of application of bootstrap.
Statistics: Data Analysis and Presentation Fr Clinic II.
Linear Regression Models Based on Chapter 3 of Hastie, Tibshirani and Friedman Slides by David Madigan.
Data Freshman Clinic II. Overview n Populations and Samples n Presentation n Tables and Figures n Central Tendency n Variability n Confidence Intervals.
Test statistic: Group Comparison Jobayer Hossain Larry Holmes, Jr Research Statistics, Lecture 5 October 30,2008.
Statistics: Data Presentation & Analysis Fr Clinic I.
Climate Variability and Prediction in the Little Colorado River Basin Matt Switanek 1 1 Department of Hydrology and Water Resources University of Arizona.
FORTRAN Short Course Week 4 Kate Thayer-Calder March 10, 2009.
Quantitative Business Analysis for Decision Making Simple Linear Regression.
Hydrologic Statistics
1 Doing Statistics for Business Doing Statistics for Business Data, Inference, and Decision Making Marilyn K. Pelosi Theresa M. Sandifer Chapter 11 Regression.
Simple linear regression and correlation analysis
Climate Predictability Tool (CPT) Ousmane Ndiaye and Simon J. Mason International Research Institute for Climate and Society The Earth.
Topic 14: Inference in Multiple Regression. Outline Review multiple linear regression Inference of regression coefficients –Application to book example.
Barcelona, 06 May 2015 s2dverification Seasonal to decadal forecast verification in R Overview Nicolau Manubens.
TEMIS user workshop, Frascati, 8-9 October 2007 Long range transport of air pollution service: Part 1: Trajectories Bart Dils, M. De Mazière, J. van Geffen,
EUROBRISA Workshop – Beyond seasonal forecastingBarcelona, 14 December 2010 INSTITUT CATALÀ DE CIÈNCIES DEL CLIMA Beyond seasonal forecasting F. J. Doblas-Reyes,
Various topics Petter Mostad Overview Epidemiology Study types / data types Econometrics Time series data More about sampling –Estimation.
RAMS Evaluation and Visualization Utilities (REVU) Post Processing.
Contrasts & Inference - EEG & MEG Himn Sabir 1. Topics 1 st level analysis 2 nd level analysis Space-Time SPMs Time-frequency analysis Conclusion 2.
Week 11 Introduction A time series is an ordered sequence of observations. The ordering of the observations is usually through time, but may also be taken.
1 Peter Fox GIS for Science ERTH 4750 (98271) Week 10, Friday, April 6, 2012 Lab:
1 An Introduction to SPSS for Windows Jie Chen Ph.D. 6/4/20161.
The climate and climate variability of the wind power resource in the Great Lakes region of the United States Sharon Zhong 1 *, Xiuping Li 1, Xindi Bian.
1 Motivation Motivation SST analysis products at NCDC SST analysis products at NCDC  Extended Reconstruction SST (ERSST) v.3b  Daily Optimum Interpolation.
Lecture 4: Statistics Review II Date: 9/5/02  Hypothesis tests: power  Estimation: likelihood, moment estimation, least square  Statistical properties.
PROCESS MODELLING AND MODEL ANALYSIS © CAPE Centre, The University of Queensland Hungarian Academy of Sciences Statistical Model Calibration and Validation.
ANOVA: Analysis of Variance.
Aquarius Level-3 Binning and Mapping Fred Patt. Definitions Projection - any process which transforms a spatially organized data set from one coordinate.
Climate Predictability Tool (CPT) Ousmane Ndiaye and Simon J. Mason International Research Institute for Climate and Society The Earth.
This supervised learning technique uses Bayes’ rule but is different in philosophy from the well known work of Aitken, Taroni, et al. Bayes’ rule: Pr is.
7 - 1 Chapter 7: Data Analysis in the Service of Modeling The Art of Modeling with Spreadsheets S.G. Powell and K.R. Baker © John Wiley and Sons, Inc.
Mr. Magdi Morsi Statistician Department of Research and Studies, MOH
Non-Parametric Methods in Forest Models James D. Arney, Ph.D. Forest Biometrics Research Institute FBRI Annual Meeting December 4, 2012.
July, 2009 WRF-Var Tutorial Syed RH Rizvi 0 WRFDA Background Error Estimation Syed RH Rizvi National Center For Atmospheric Research NCAR/ESSL/MMM, Boulder,
Ozone time series and trends Various groups compute trends in different ways. One goal of the workshop is to be able to compare time series and trends.
Arrays. Review of last lecture Overview of computer languages Overview of IDL Strength of IDL: (1) Interactive/compiled modes; (2) Array-oriented; (3)
WCRP Extremes Workshop Sept 2010 Detecting human influence on extreme daily temperature at regional scales Photo: F. Zwiers (Long-tailed Jaeger)
Chris Ferro Climate Analysis Group Department of Meteorology University of Reading Extremes in a Varied Climate 1.Significance of distributional changes.
Mapdraw ats315. Today’s goal: Drawing a map in the graphics window! Drawing a map in the graphics window!
The ENSEMBLES high- resolution gridded daily observed dataset Malcolm Haylock, Phil Jones, Climatic Research Unit, UK WP5.1 team: KNMI, MeteoSwiss, Oxford.
Diagnostic verification and extremes: 1 st Breakout Discussed the need for toolkit to build beyond current capabilities (e.g., NCEP) Identified (and began.
Review of BUSA3322 Mary M. Whiteside. Methodologies Two sample tests Analysis of variance Chi square tests Simple regression Multiple regression Time.
Statistical Fundamentals: Using Microsoft Excel for Univariate and Bivariate Analysis Alfred P. Rovai Charts Overview PowerPoint Prepared by Alfred P.
Biostatistics Regression and Correlation Methods Class #10 April 4, 2000.
16-1 PC-HYSPLIT WORKSHOP Workshop Agenda Introduction to HYSPLIT Introduction.ppt Model Overview Model_Overview.ppt Meteorological Data Meteorological_Data.ppt.
Example 16.2a Moving Averages | 16.1a | 16.2 | 16.3 | 16.4 | 16.5 | 16.6 | 16.7 | 16.7a | 16.7b16.1a a16.7b DOW.XLS.
Lab 4 Multiple Linear Regression. Meaning  An extension of simple linear regression  It models the mean of a response variable as a linear function.
Predicting Future. Two Approaches to Predition n Extrapolation: Use past experiences for predicting future. One looks for patterns over time. n Predictive.
Combining Models Foundations of Algorithms and Machine Learning (CS60020), IIT KGP, 2017: Indrajit Bhattacharya.
of Temperature in the San Francisco Bay Area
Skillful Arctic climate predictions
of Temperature in the San Francisco Bay Area
Unfolding Problem: A Machine Learning Approach
CSC 558 – Data Analytics II, Spring, 2018
Comparison of different combinations of ensemble-based and variational data assimilation approaches for deterministic NWP Mark Buehner Data Assimilation.
The general linear model and Statistical Parametric Mapping
Chapter 8 Supplement Forecasting.
Presentation transcript:

Climate Forecasting Unit CFU R common diagnostics CFU_load CFU_season CFU_clim CFU_anoCFU_anocrossvalid CFU_plotclimCFU_plotano CFU_smoothing CFU_trend CFU_animvsltime

Climate Forecasting Unit CFU R common diagnostics CFU_load Minimum set of arguments : 1) var 2) exp 3) obs (can be obs=NULL) 4) sdates You can request area-averages, longitudinal or latitudinal averages, 2d fields You can define any region by sending masks You can request a subset of leadtimes You can work on a subdomain by providing lat/lon borders

Climate Forecasting Unit CFU_season CFU R common diagnostics This function computes averages over extended season. It can be used to compute annual means for exemple.

Climate Forecasting Unit CFU R common diagnostics CFU_clim This function computes per-pair climatologies, one climatology per member or one for all the members together. If you have only one start date, your climatology should be computed as a simple annual cycle not with CFU_clim. If you don’t have observations, you don’t need the per-pair method. Your clim is clim=CFU_mean1dim(exp, 3)

Climate Forecasting Unit CFU R common diagnostics CFU_anocrossvalid This function computes anomalies using the cross- validation method, i.e. for each startdate, the climatology is computed using all the other startdates. It also uses the per-pair method.

Climate Forecasting Unit CFU R common diagnostics CFU_trend This function provides not only the linear trend but also the linearly detrended data.

Climate Forecasting Unit CFU R common diagnostics CFU_load CFU_clim CFU_anoCFU_anocrossvalid CFU_plotclimCFU_plotanoCFU_animvsltime mod = array(dim=c(nexp, nmemb, nsdates, nltimes) to mod = array(dim=c(nexp, nmemb, nsdates, nltimes, nlat, nlon) obs = array(dim=c(nobs, nmemb, nsdates, nltimes) to obs = array(dim=c(nobs, nmemb, nsdates, nltimes, nlat, nlon) Those functions work only with the common diagnostic structure.

Climate Forecasting Unit CFU R common diagnostics CFU_season CFU_smoothing CFU_trend For those functions, the input structure is free. Input matrix can have any number of dimensions and the dimension along which the trend, smoothing or season has to be computed should be specified. Default parameters : common diagnostic structure, leadtime dimensions for CFU_season/CFU_smoothing, nsdates for CFU_trend You can use them on any time series

Climate Forecasting Unit CFU R common diagnostics CFU_load CFU_season CFU_clim CFU_anoCFU_anocrossvalid CFU_plotclimCFU_plotano CFU_smoothing CFU_trend CFU_animvsltime

Climate Forecasting Unit CFU R common diagnostics CFU_anoCFU_anocrossvalid CFU_spread CFU_corr CFU_RMS CFU_trend CFU_ratioRMS CFU_ratioSDRMSCFU_RMSSS CFU_consist_trend CFU_animvsltimeCFU_plotvsltimeCFU_plotequimap

Climate Forecasting Unit CFU R common diagnostics CFU_spread CFU_corr CFU_RMS CFU_trend CFU_ratioRMS CFU_ratioSDRMSCFU_RMSSS For those functions, the input structure is free. Default : common diagnostic structure Scores are computed for each experimental dataset versus each observational dataset in your input matrix.

Climate Forecasting Unit CFU R common diagnostics CFU_anoCFU_anocrossvalid CFU_consist_trend CFU_animvsltimeCFU_plotvsltime Those functions expect the common diagnostic structure

Climate Forecasting Unit CFU R common diagnostics CFU_plotequimap For this function, (lat,lon) expected and a second matrix of flags=T/F with the same dimensions is expected for significance level It has many functionalities to make nice plots for publication. Color levels (square or smoothed), contours, dots …, continents can be filled in grey or show as black lines. Colorbar can be drawn or not…. It can be used in a multipanel after splitting the space with layout

Climate Forecasting Unit CFU R common diagnostics CFU_spread CFU_corr CFU_RMS CFU_trend CFU_ratioRMS CFU_ratioSDRMSCFU_RMSSS CFU_consist_trend Confidence intervals or significance levels or both are systematically provided.

Climate Forecasting Unit CFU R common diagnostics CFU_corr CFU_RMSCFU_ratioRMS CFU_ratioSDRMSCFU_RMSSS For those functions, there are issues about the temporal dependance of the data for confidence intervals/significance levels. For non-parametric tests, a window of dependence has to be defined, for parametric ones, a number of independant data has to be defined. Those functions currently use parametric tests with a number of independant data defined following the classical formula from Von Storch and Zwiers (2001). This might change depending on the literature. Call to CFU_eno

Climate Forecasting Unit CFU R common diagnostics CFU_spread CFU_corr CFU_RMS CFU_trend CFU_ratioRMS CFU_ratioSDRMSCFU_RMSSS CFU_consist_trend bootstrap one sided T-test Fisher transform chi2 one-sided Fisher test two-sided Fisher test one-sided Fisher test T- distribution

Climate Forecasting Unit CFU R common diagnostics CFU_anoCFU_anocrossvalid CFU_spread CFU_corr CFU_RMS CFU_trend CFU_ratioRMS CFU_ratioSDRMSCFU_RMSSS CFU_consist_trend CFU_animvsltimeCFU_plotvsltimeCFU_plotequimap

Climate Forecasting Unit CFU R common diagnostics CFU_eno CFU_mean1dim CFU_meanlistdim CFU_insertdim CFU_colorbar For those functions, the input structure is free. This function makes a colorbar if you send the levels and colors. Useful for multipanels after calling layout

Climate Forecasting Unit CFU R common diagnostics ~]$ R  source(‘/cfu/pub/scripts/R/common_diagnostics.txt’) [1] List of functions : [1] [1] CFU_load [1] CFU_season [1] CFU_clim [1] CFU_ano [1] CFU_ano_crossvalid [1] CFU_smoothing [1] CFU_plotano [1] CFU_plotclim [1] CFU_spread [1] CFU_plotvsltime [1] CFU_corr [1] CFU_RMS [1] CFU_RMSSS

Climate Forecasting Unit CFU R common diagnostics [1] CFU_ratioRMS [1] CFU_ratioSDRMS [1] CFU_trend [1] CFU_consist_trend [1] CFU_plotequimap [1] CFU_colorbar [1] CFU_animvsltime [1] CFU_eno [1] CFU_enlarge [1] CFU_insertdim [1] CFU_mean1dim [1] CFU_meanlistdim [1] CFU_inilistdims [1] [1] For more information about any function, type info_cd('function name')  info_cd(‘CFU_load’)

Climate Forecasting Unit CFU R common diagnostics [1] [1] Description [1] ~~~~~~~~~~~~~ [1] [1] Load experimental data and corresponding observed ones in 2 matrix with similar structures [1] If loading EC-Earth experiments, PUT FIRST THE EXPERIMENT ID WITH THE LARGEST NUMBER [1] OF MEMBERS & if possible, THE LARGEST NUMBER OF LEADTIMES. If not possible, fill up the nleatime argument. [1] [1] Inputs [1] ~~~~~~~~ [1] [1] - var= 'tas','prlr','tos','g500','g200','ta50','psl','hflsd','hfssd','rls','rss','rsds','uas','vas'

Climate Forecasting Unit [1] - exp=c('ecmwf','ukmo','cerfacs','ifm','DePreSysAsimDec','DePreSysNoAsimDec','DePr eSysAsimSeas','ECMWF_S3Seas','ECMWF_S4Seas','ECMWF_S4SeasQWeCI','hadcm 3dec','miroc4dec','miroc5dec','mri-cgcm3dec','cancm4dec1','cancm4dec2','cnrm- cm5dec','knmidec','mpimdec','gfdldec','cmcc- cmdec','hadcm3his','miroc4his','miroc5his','mri-cgcm3his','cancm4his','cnrm- cm5his','knmihis','i00k','b013','b014','yve2'...) [1] - obs=c('ERA40','NCEP','ERAint','GHCN','ERSST','HADISST','GPCP','GPCC','CRU','DS9 4','OAFlux','DFS4.3','NCDCglo','NCDCland','NCDCoc','GISSglo','GISSland','GISSoc','H adCRUT3glo','HadSST2oc','CRUTEM3land') [1] - sdates=c('YYYYMMDD','YYYYMMDD') [1] - lonmin, lonmax, latmin, latmax : domain border 0 <= lonmin,lonmax <= 360 [1] default : world [1] - nleadtime : optional argument needed only if the first exp does not have the largest number of leadtimes. [1] default : number of leadtimes of the first experiment. CFU R common diagnostics

Climate Forecasting Unit [1] - leadtimemin : output only the leadtimes from leadtimemin. default = 1 [1] - leadtimemax : output only the leadtimes before leadtimemax. default = nleadtime [1] - output = 'areave' / 'lon' / 'lat' / 'lonlat' [1] 1) Time series of area-averaged variables over the specified domain [1] 2) Time series of meridional averages as a function of longitudes [1] 3) Time series of zonal averages as a function of latitudes [1] 4) Time series of 2d fields [1] default : 'areave' [1] - method = 'bilinear' / 'bicubic' / 'conservative' / 'distance-weighted' [1] Method of interpolation for 'lon' / 'lat' / 'lonlat' output options [1] default : 'conservative' [1] - grid = to choose the output grid [1] possible options : rNXxNY or tTRgrid, ex: r96x72, t106grid [1] default : model grid, argument need to be filled if various exp on various grids CFU R common diagnostics

Climate Forecasting Unit [1] - maskmod=list(mask[lon,lat]) = 1/0 : kept/removed grid cell over the entire model domains [1] Warning : list() compulsory even if 1 model !!! [1] default : 1 everywhere [1] - maskobs=list(mask[lon,lat]) = 1/0 : kept/removed grid cell over the entire [1] observed domains, only necessary for 'areave' output option [1] Warning : list() compulsory even if 1 dataset !!! [1] default : 1 everywhere [1] CFU R common diagnostics

Climate Forecasting Unit [1] Outputs [1] ~~~~~~~~~ [1] [1] $mod = model outputs [1] $obs = observations [1] $lat = latitudes of the model grid [1] $lon = longitudes of the model grid [1] [1] 2 matrix with dimensions [1] c(nmod/nexp or nobs, nmemb/nparam, nsdates, nltime) if output = 'areave' [1] c(nmod/nexp or nobs, nmemb/nparam, nsdates, nltime, nlat ) if = 'lat' [1] c(nmod/nexp or nobs, nmemb/nparam, nsdates, nltime, nlon ) if = 'lon' [1] c(nmod/nexp or nobs, nmemb/nparam, nsdates, nltime, nlat, nlon) = 'lonlat' [1] [1] Author [1] ~~~~~~~~ [1] [1] CFUers March 2011 CFU R common diagnostics