Non-Parametric Density Functions Christoph F. Eick 12/24/08.

Slides:

Advertisements

Similar presentations

Principles of Density Estimation

Advertisements

Overview of SPM p <0.05 Statistical parametric map (SPM)

Aggregating local image descriptors into compact codes

Christoph F. Eick Questions and Topics Review Dec. 10, Compare AGNES /Hierarchical clustering with K-means; what are the main differences? 2. K-means.

Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) ETHEM ALPAYDIN © The MIT Press, 2010

Notes Sample vs distribution “m” vs “µ” and “s” vs “σ” Bias/Variance Bias: Measures how much the learnt model is wrong disregarding noise Variance: Measures.

INTRODUCTION TO MACHINE LEARNING Bayesian Estimation.

1 Non-Linear and Smooth Regression Non-linear parametric models: There is a known functional form y=  x,  derived from known theory or from previous.

INC 551 Artificial Intelligence Lecture 11 Machine Learning (Continue)

INTRODUCTION TO Machine Learning 3rd Edition

K Means Clustering , Nearest Cluster and Gaussian Mixture

Lecture 3 Nonparametric density estimation and classification

Regression. So far, we've been looking at classification problems, in which the y values are either 0 or 1. Now we'll briefly consider the case where.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

MACHINE LEARNING 9. Nonparametric Methods. Introduction Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Statistical analysis and modeling of neural data Lecture 4 Bijan Pesaran 17 Sept, 2007.

Pattern Recognition. Introduction. Definitions.. Recognition process. Recognition process relates input signal to the stored concepts about the object.

1 An Introduction to Nonparametric Regression Ning Li March 15 th, 2004 Biostatistics 277.

Ch. Eick: Support Vector Machines: The Main Ideas Reading Material Support Vector Machines: 1.Textbook 2. First 3 columns of Smola/Schönkopf article on.

1 Lazy Learning – Nearest Neighbor Lantz Ch 3 Wk 2, Part 1.

Buckingham Pi Theorem This example is the same as example 7.2 in the textbook except that we assume the pipe is a smooth pipe. Using Buckingham Pi theorem,

Chapter 11: Estimation Estimation Defined Confidence Levels

CHAPTER 7: Clustering Eick: K-Means and EM (modified Alpaydin transparencies and new transparencies added) Last updated: February 25, 2014.

Topic9: Density-based Clustering

Image Modeling & Segmentation Aly Farag and Asem Ali Lecture #2.

Week 21 Stochastic Process - Introduction Stochastic processes are processes that proceed randomly in time. Rather than consider fixed random variables.

Data Mining in Finance, 1999 March 8, 1999 Extracting Risk-Neutral Densities from Option Prices using Mixture Binomial Trees Christian Pirkner Andreas.

Lecture3 – Overview of Supervised Learning Rice ELEC 697 Farinaz Koushanfar Fall 2006.

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture.

Data Mining Anomaly/Outlier Detection Lecture Notes for Chapter 10 Introduction to Data Mining by Tan, Steinbach, Kumar.

Nonparametric Density Estimation Riu Baring CIS 8526 Machine Learning Temple University Fall 2007 Christopher M. Bishop, Pattern Recognition and Machine.

EM Mixture models: datapoint has nonzero probs to belong to multiple k distributions So Hidden Var in each datapoint: e.g for k=2 hypothesis h about parameters.

Sampling Theory and Some Important Sampling Distributions.

1 Chapter 8: Model Inference and Averaging Presented by Hui Fang.

Kernel Methods Arie Nakhmani. Outline Kernel Smoothers Kernel Density Estimators Kernel Density Classifiers.

Cost Modeling of Spatial Query Operators Using Nonparametric Regression Songtao Jiang Department of Computer Science University of Vermont October 10,

Eick: kNN kNN: A Non-parametric Classification and Prediction Technique Goals of this set of transparencies: 1.Introduce kNN---a popular non-parameric.

CHAPTER 8: Nonparametric Methods Alpaydin transparencies significantly modified, extended and changed by Ch. Eick Last updated: March 4, 2011.

Density Estimation in R Ha Le and Nikolaos Sarafianos COSC 7362 – Advanced Machine Learning Professor: Dr. Christoph F. Eick 1.

Part 3: Estimation of Parameters. Estimation of Parameters Most of the time, we have random samples but not the densities given. If the parametric form.

Lecture 7 Spring 2010 Dr. Jianjun Hu CSCE883 Machine Learning.

Stochastic Process - Introduction

Estimation and Confidence Intervals

More on Clustering in COSC 4335

Lecture #26 Thursday, November 17, 2016 Textbook: 14.1 and 14.3

Bayesian Generalized Product Partition Model

INTRODUCTION TO Machine Learning 3rd Edition

Ch8: Nonparametric Methods

3(+1) classifiers from the Bayesian world

Non-Parametric Density Functions

Nonparametric Density Estimation

Bias and Variance of the Estimator

Outline Parameter estimation – continued Non-parametric methods.

K Nearest Neighbor Classification

Statistical Learning Dong Liu Dept. EEIS, USTC.

Nearest-Neighbor Classifiers

More about Normal Distributions

CHAPTER 22: Inference about a Population Proportion

Where did we stop? The Bayes decision rule guarantees an optimal classification… … But it requires the knowledge of P(ci|x) (or p(x|ci) and P(ci)) We.

Calculating the Center of Mass of Solid Objects (using Calculus)

LECTURE 16: NONPARAMETRIC TECHNIQUES

Non-parametric Filters: Particle Filters

Nonparametric density estimation and classification

Mathematical Foundations of BME

Non-parametric Filters: Particle Filters

Summation Notation.

SVMs for Document Ranking

Presentation transcript:

Non-Parametric Density Functions Christoph F. Eick 12/24/08

Influence Functions The influence function of a data object is a function f B y : F d → R 0 + which is defined in terms of a basic influence function f B The influence function of a data object is a function f B y : F d → R 0 + which is defined in terms of a basic influence function f B Examples of basic influence functions are: Examples of basic influence functions are:

3 Example: Kernel Density Estimation D={x1,x2,x3,x4} f D Gaussian (x)= influence(x1) + influence(x2) + influence(x3) influence(x4)= =0.78 x1 x2 x3 x4 x y Remark: the density value of y would be larger than the one for x

Notations O={o 1,…,o n } is a dataset whose density has to be measured O={o 1,…,o n } is a dataset whose density has to be measured r is the dimensionality of O r is the dimensionality of O n is the number of objects in O n is the number of objects in O d: F r X F r → R 0 + is a distance function for the objects in O; d is assumed to be Euclidian distance unless specified otherwise d: F r X F r → R 0 + is a distance function for the objects in O; d is assumed to be Euclidian distance unless specified otherwise d k (x) is the distance of x its k-nearest neighbor in O d k (x) is the distance of x its k-nearest neighbor in O

Gaussian Kernel Density Functions Density functions f B : defined as the sum of the influence functions of all data points. Given N data objects, O={o 1,…, o n } the density function is defined as—normalized version: Density functions f B : defined as the sum of the influence functions of all data points. Given N data objects, O={o 1,…, o n } the density function is defined as—normalized version: Non-Normalized Version: Non-Normalized Version: Example: Gaussian Kernel Non-parametric Density Function f Example: Gaussian Kernel Non-parametric Density Function f Remark:  is called kernel width; called h in our textbook!

kNN-based Density Functions Example: kNN density function with variable kernel width—width is proportional to d k (x) in point x: the distance of the k th nearest neighbor in O to x; is usually difficult to normalize due to the variable kernel width; therefore, the integral over the density function does not add up to 1 and usually is . Parameter selection: Instead of the width , k has to be selected! or Alpaydin mentions the following variation: