Gap filling using a Bayesian-regularized neural network B.H. Braswell University of New Hampshire.

Slides:

Advertisements

Similar presentations

Neural Networks and Kernel Methods

Advertisements

Why gap filling isn’t always easy Andrew Richardson University of New Hampshire Jena Gap Filling Workshop September 2006.

Page 1 of 50 Optimization of Artificial Neural Networks in Remote Sensing Data Analysis Tiegeng Ren Dept. of Natural Resource Science in URI (401)

Biointelligence Laboratory, Seoul National University

Intelligent Environments1 Computer Science and Engineering University of Texas at Arlington.

Artificial Intelligence Lecture 2 Dr. Bo Yuan, Professor Department of Computer Science and Engineering Shanghai Jiaotong University

Machine Learning Neural Networks

Neural Networks I CMPUT 466/551 Nilanjan Ray. Outline Projection Pursuit Regression Neural Network –Background –Vanilla Neural Networks –Back-propagation.

Multiple Criteria for Evaluating Land Cover Classification Algorithms Summary of a paper by R.S. DeFries and Jonathan Cheung-Wai Chan April, 2000 Remote.

Lecture 14 – Neural Networks

Pattern Recognition and Machine Learning

Chapter 5 NEURAL NETWORKS

November 2, 2010Neural Networks Lecture 14: Radial Basis Functions 1 Cascade Correlation Weights to each new hidden node are trained to maximize the covariance.

Data mining and statistical learning - lecture 11 Neural networks - a model class providing a joint framework for prediction and classification  Relationship.

Study of Sparse Online Gaussian Process for Regression EE645 Final Project May 2005 Eric Saint Georges.

Neural networks - Lecture 111 Recurrent neural networks (II) Time series processing –Networks with delayed input layer –Elman network Cellular networks.

Raw data Hz HH data submitted for synthesis Flux calculation, raw data filtering Additional filtering for footprint or instrument malfunctioning.

Gaussian process modelling

CSSE463: Image Recognition Day 21 Upcoming schedule: Upcoming schedule: Exam covers material through SVMs Exam covers material through SVMs.

Efficient Direct Density Ratio Estimation for Non-stationarity Adaptation and Outlier Detection Takafumi Kanamori Shohei Hido NIPS 2008.

Biointelligence Laboratory, Seoul National University

Applications of Bayesian sensitivity and uncertainty analysis to the statistical analysis of computer simulators for carbon dynamics Marc Kennedy Clive.

ANNs (Artificial Neural Networks). THE PERCEPTRON.

1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 23 Nov 2, 2005 Nanjing University of Science & Technology.

11 CSE 4705 Artificial Intelligence Jinbo Bi Department of Computer Science & Engineering

Outline 1-D regression Least-squares Regression Non-iterative Least-squares Regression Basis Functions Overfitting Validation 2.

Using Neural Networks to Predict Claim Duration in the Presence of Right Censoring and Covariates David Speights Senior Research Statistician HNC Insurance.

Neural Networks - Berrin Yanıkoğlu1 Applications and Examples From Mitchell Chp. 4.

Long term weather and flux data: treatment of discontinuous data. Bart Kruijt, Wilma Jans, Cor Jacobs, Eddy Moors Loobos.

NEURAL NETWORKS FOR DATA MINING

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 16: NEURAL NETWORKS Objectives: Feedforward.

Classification / Regression Neural Networks 2

Comparison of Bayesian Neural Networks with TMVA classifiers Richa Sharma, Vipin Bhatnagar Panjab University, Chandigarh India-CMS March, 2009 Meeting,

Gap filling of eddy fluxes with artificial neural networks

17 May 2007RSS Kent Local Group1 Quantifying uncertainty in the UK carbon flux Tony O’Hagan CTCD, Sheffield.

CS Inductive Bias1 Inductive Bias: How to generalize on novel data.

CSSE463: Image Recognition Day 14 Lab due Weds, 3:25. Lab due Weds, 3:25. My solutions assume that you don't threshold the shapes.ppt image. My solutions.

Dropout as a Bayesian Approximation

Recursive Calibration of Ecosystem Models Using Sequential Data Assimilation Mingshi Chen¹, Shuguang Liu¹, Larry L. Tieszen², and David Y. Hollinger 3.

Reservoir Uncertainty Assessment Using Machine Learning Techniques Authors: Jincong He Department of Energy Resources Engineering AbstractIntroduction.

Neural Networks Presented by M. Abbasi Course lecturer: Dr.Tohidkhah.

1 Statistics & R, TiP, 2011/12 Neural Networks  Technique for discrimination & regression problems  More mathematical theoretical foundation  Works.

Uncertainty in eddy covariance data and its relevance to gap filling David HollingerAndrew Richardson USDA Forest ServiceUniversity of New Hampshire Durham,

Machine Learning 5. Parametric Methods.

Towards a robust, generalizable non-linear regression gap filling algorithm (NLR_EM) Ankur R Desai – National Center for Atmospheric Research (NCAR) Boulder,

Supervised Machine Learning: Classification Techniques Chaleece Sandberg Chris Bradley Kyle Walsh.

Machine Learning: A Brief Introduction Fu Chang Institute of Information Science Academia Sinica ext. 1819

BACKPROPAGATION (CONTINUED) Hidden unit transfer function usually sigmoid (s-shaped), a smooth curve. Limits the output (activation) unit between 0..1.

RiskTeam/ Zürich, 6 July 1998 Andreas S. Weigend, Data Mining Group, Information Systems Department, Stern School of Business, NYU 2: 1 Nonlinear Models.

Neural networks (2) Reminder Avoiding overfitting Deep neural network Brief summary of supervised learning methods.

Data Mining: Concepts and Techniques1 Prediction Prediction vs. classification Classification predicts categorical class label Prediction predicts continuous-valued.

Ch 1. Introduction Pattern Recognition and Machine Learning, C. M. Bishop, Updated by J.-H. Eom (2 nd round revision) Summarized by K.-I.

Neural Networks - Berrin Yanıkoğlu1 MLP & Backpropagation Issues.

Prepared by Fayes Salma.  Introduction: Financial Tasks  Data Mining process  Methods in Financial Data mining o Neural Network o Decision Tree  Trading.

Neural Networks: An Introduction and Overview

Big data classification using neural network

CEE 6410 Water Resources Systems Analysis

LECTURE 28: NEURAL NETWORKS

Biointelligence Laboratory, Seoul National University

CSSE463: Image Recognition Day 18

LECTURE 28: NEURAL NETWORKS

Neural Networks II Chen Gao Virginia Tech ECE-5424G / CS-5824

CSSE463: Image Recognition Day 18

CSSE463: Image Recognition Day 18

Neural Networks II Chen Gao Virginia Tech ECE-5424G / CS-5824

Multiple features Linear Regression with multiple variables

Neural Networks: An Introduction and Overview

Multiple features Linear Regression with multiple variables

Uncertainty Propagation

Presentation transcript:

Gap filling using a Bayesian-regularized neural network B.H. Braswell University of New Hampshire

MacKay DJC (1992) A practical Bayesian framework for backpropagation networks. Neural Computation, 4, Bishop C (1995) Neural Networks for Pattern Recognition, New York: Oxford University Press. Nabney I (2002) NETLAB: algorithms for pattern recognition. In: Advances in Pattern Recognition, New York: Springer-Verlag. Proper Credit

Two-layer ANN is a nonlinear regression

e.g., tanh() usually nonlinear usually linear

Neural networks are efficient with respect to number of estimated parameters Polynomial of order M: N p ~ d M Consider a problem with d input variables Neural net with M hidden nodes: N p ~ d∙M

✦ Early stopping ✦ Regularization ✦ Bayesian methods Avoiding the problem of overfitting:

✦ Early stopping ✦ Regularization ✦ Bayesian methods

Avoiding the problem of overfitting: ✦ Early stopping ✦ Regularization ✦ Bayesian methods

1

2

3

4

5

6

Hagen SC, Braswell BH, Frolking, Richardson A, Hollinger D, Linder E (2006) Statistical uncertainty of eddy flux based estimates of gross ecosystem carbon exchange at Howland Forest, Maine. Journal of Geophysical Research, 111. Braswell BH, Hagen SC, Frolking SE, Salas WE (2003) A multivariable approach for mapping subpixel land cover distributions using MISR and MODIS: An application in the Brazilian Amazon. Remote Sensing of Environment, 87: Previous Work

ANN Regression for Land Cover Estimation Band1 Band2 Band3 Band4 Forest Fraction Cleared Fraction Secondary Fraction Training data supplied by classified ETM imagery

ANN Regression for Land Cover Estimation

ANN Estimation of GEE and Resp, with Monte Carlo simulation of Total Prediction uncertainty ClimFlux

Weekly GEE from Howland Forest, ME based on NEE ANN Estimation of GEE and Resp, with Monte Carlo simulation of Total Prediction uncertainty

Some demonstrations of the MacKay/Bishop ANN regression with 1 input and 1 output

Noise=

Noise=0.10 Linear Regression

Noise=0.10 ANN Regression

Noise=0.02 ANN Regression

Noise=0.20 ANN Regression

Noise=0.20 ANN Regression

Noise=0.10 ANN Regression

Noise=0.05 ANN Regression

Noise=0.05 ANN Regression

Noise=0.05 ANN Regression

Issues associated with multidimensional problems ✦ Sufficient sampling of the the input space ✦ Data normalization (column mean zero and standard deviation one) ✦ Processing time ✦ Algorithm parameter choices

Our gap-filling algorithm 1.Assemble meteorological and flux data in an Nxd table 2.Create five additional columns for sin() and cos() of time of day and day of year, and potential PAR 3.Standardize all columns 4.First iteration: Identify columns with no gaps; use these to fill all the others, one at a time. 5.Create an additional column, NEE(t-1), flux lagged by one time interval 6.Second iteration: Remove filled points from the NEE time series, refill with all other columns

Room for Improvement 1.Don’t extrapolate wildly, revert to time-based filling in areas with low sampling density, especially at the beginning and end of the record 2.Carefully evaluate the sensitivity to internal settings (e.g., alpha, beta, Nnodes) 3.Stepwise analysis for relative importance of driver variables 4.Migrate to C or other faster environment 5.Include uncertainty estimates in the output 6.At least, clean up the code and make it available to others in the project, and/or broader community