Objectives: Normal Random Variables Support Regions Whitening Transformations Resources: DHS – Chap. 2 (Part 2) K.F. – Intro to PR X. Z. – PR Course S.B.

Slides:

Advertisements

Similar presentations

Component Analysis (Review)

Advertisements

ECE 8443 – Pattern Recognition LECTURE 05: MAXIMUM LIKELIHOOD ESTIMATION Objectives: Discrete Features Maximum Likelihood Resources: D.H.S: Chapter 3 (Part.

Face Recognition Ying Wu Electrical and Computer Engineering Northwestern University, Evanston, IL

LECTURE 11: BAYESIAN PARAMETER ESTIMATION

Pattern Classification Chapter 2 (Part 2)0 Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O.

Probability Review 1 CS479/679 Pattern Recognition Dr. George Bebis.

Prénom Nom Document Analysis: Parameter Estimation for Pattern Recognition Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Multivariate Distance and Similarity Robert F. Murphy Cytometry Development Workshop 2000.

Probability theory 2011 The multivariate normal distribution  Characterizing properties of the univariate normal distribution  Different definitions.

Descriptive statistics Experiment  Data  Sample Statistics Experiment  Data  Sample Statistics Sample mean Sample mean Sample variance Sample variance.

CS Pattern Recognition Review of Prerequisites in Math and Statistics Prepared by Li Yang Based on Appendix chapters of Pattern Recognition, 4.

Visual Recognition Tutorial1 Random variables, distributions, and probability density functions Discrete Random Variables Continuous Random Variables.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Probability theory 2008 Outline of lecture 5 The multivariate normal distribution  Characterizing properties of the univariate normal distribution  Different.

Lecture II-2: Probability Review

Modern Navigation Thomas Herring

Today Wrap up of probability Vectors, Matrices. Calculus

Computer vision: models, learning and inference Chapter 5 The Normal Distribution.

METU Informatics Institute Min 720 Pattern Classification with Bio-Medical Applications PART 2: Statistical Pattern Classification: Optimal Classification.

Probability of Error Feature vectors typically have dimensions greater than 50. Classification accuracy depends upon the dimensionality and the amount.

Principles of Pattern Recognition

1 Pattern Recognition: Statistical and Neural Lonnie C. Ludeman Lecture 13 Oct 14, 2005 Nanjing University of Science & Technology.

Speech Recognition Pattern Classification. 22 September 2015Veton Këpuska2 Pattern Classification  Introduction  Parametric classifiers  Semi-parametric.

ECE 8443 – Pattern Recognition LECTURE 03: GAUSSIAN CLASSIFIERS Objectives: Normal Distributions Whitening Transformations Linear Discriminants Resources.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 02: BAYESIAN DECISION THEORY Objectives: Bayes.

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 03: GAUSSIAN CLASSIFIERS Objectives: Whitening.

ECE 8443 – Pattern Recognition LECTURE 07: MAXIMUM LIKELIHOOD AND BAYESIAN ESTIMATION Objectives: Class-Conditional Density The Multivariate Case General.

Ch 2. Probability Distributions (1/2) Pattern Recognition and Machine Learning, C. M. Bishop, Summarized by Yung-Kyun Noh and Joo-kyung Kim Biointelligence.

Course 9 Texture. Definition: Texture is repeating patterns of local variations in image intensity, which is too fine to be distinguished. Texture evokes.

N– variate Gaussian. Some important characteristics: 1)The pdf of n jointly Gaussian R.V.’s is completely described by means, variances and covariances.

ECE 8443 – Pattern Recognition LECTURE 10: HETEROSCEDASTIC LINEAR DISCRIMINANT ANALYSIS AND INDEPENDENT COMPONENT ANALYSIS Objectives: Generalization of.

A Review of Some Fundamental Mathematical and Statistical Concepts UnB Mestrado em Ciências Contábeis Prof. Otávio Medeiros, MSc, PhD.

ECE 8443 – Pattern Recognition ECE 8423 – Adaptive Signal Processing Objectives: ML and Simple Regression Bias of the ML Estimate Variance of the ML Estimate.

Bayesian Decision Theory (Classification) 主講人：虞台文.

: Chapter 3: Maximum-Likelihood and Baysian Parameter Estimation 1 Montri Karnjanadecha ac.th/~montri.

ECE 8443 – Pattern Recognition LECTURE 08: DIMENSIONALITY, PRINCIPAL COMPONENTS ANALYSIS Objectives: Data Considerations Computational Complexity Overfitting.

1 Bayesian Decision Theory Shyh-Kang Jeng Department of Electrical Engineering/ Graduate Institute of Communication/ Graduate Institute of Networking and.

Discriminant Analysis

Geology 6600/7600 Signal Analysis 02 Sep 2015 © A.R. Lowry 2015 Last time: Signal Analysis is a set of tools used to extract information from sequences.

Introduction to Pattern Recognition (การรู้จํารูปแบบเบื้องต้น)

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 12: Advanced Discriminant Analysis Objectives:

ECE 8443 – Pattern Recognition LECTURE 04: PERFORMANCE BOUNDS Objectives: Typical Examples Performance Bounds ROC Curves Resources: D.H.S.: Chapter 2 (Part.

Principal Component Analysis (PCA)

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 04: GAUSSIAN CLASSIFIERS Objectives: Whitening.

Random Processes Gaussian and Gauss-Markov processes Power spectrum of random processes and white processes.

Geology 5670/6670 Inverse Theory 28 Jan 2015 © A.R. Lowry 2015 Read for Fri 30 Jan: Menke Ch 4 (69-88) Last time: Ordinary Least Squares: Uncertainty The.

Pattern Recognition Mathematic Review Hamid R. Rabiee Jafar Muhammadi Ali Jalali.

ECE 8443 – Pattern Recognition Objectives: Reestimation Equations Continuous Distributions Gaussian Mixture Models EM Derivation of Reestimation Resources:

Ch 2. Probability Distributions (1/2) Pattern Recognition and Machine Learning, C. M. Bishop, Summarized by Joo-kyung Kim Biointelligence Laboratory,

ECE 8443 – Pattern Recognition ECE 8527 – Introduction to Machine Learning and Pattern Recognition LECTURE 10: PRINCIPAL COMPONENTS ANALYSIS Objectives:

ECE 471/571 – Lecture 3 Discriminant Function and Normal Density 08/27/15.

Pattern Recognition Mathematic Review Hamid R. Rabiee Jafar Muhammadi Ali Jalali.

Geology 6600/7600 Signal Analysis 04 Sep 2014 © A.R. Lowry 2015 Last time: Signal Analysis is a set of tools used to extract information from sequences.

Lecture 2. Bayesian Decision Theory

LECTURE 11: Advanced Discriminant Analysis

LECTURE 04: DECISION SURFACES

LECTURE 10: DISCRIMINANT ANALYSIS

LECTURE 03: DECISION SURFACES

CH 5: Multivariate Methods

LECTURE 01: COURSE OVERVIEW

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

ECE 417 Lecture 4: Multivariate Gaussians

LECTURE 05: THRESHOLD DECODING

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

LECTURE 01: COURSE OVERVIEW

LECTURE 09: DISCRIMINANT ANALYSIS

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Multivariate Methods Berlin Chen

Multivariate Methods Berlin Chen, 2005 References:

LECTURE 11: Exam No. 1 Review

Presentation transcript:

Objectives: Normal Random Variables Support Regions Whitening Transformations Resources: DHS – Chap. 2 (Part 2) K.F. – Intro to PR X. Z. – PR Course S.B. – Statistical Norm. Java Applet URL:.../publications/courses/ece_8443/lectures/current/lecture_07.ppt.../publications/courses/ece_8443/lectures/current/lecture_07.ppt ECE 8443 – Pattern Recognition LECTURE 07: WHITENING TRANSFORMATIONS

A normal or Gaussian density is a powerful model for modeling continuous-valued feature vectors corrupted by noise due to its analytical tractability. Univariate normal distribution: 07: WHITENING TRANSFORMATIONS UNIVARIATE NORMAL DISTRIBUTION where the mean and covariance are defined by: and the entropy is given by:

07: WHITENING TRANSFORMATIONS MEAN AND VARIANCE A normal distribution is completely specified by its mean and variance: A normal distribution achieves the maximum entropy of all distributions having a given mean and variance. Central Limit Theorem: The sum of a large number of small, independent random variables will lead to a Gaussian distribution. The peak is at: 66% of the area is within one  ; 95% is within two  ; 99% is within three .

07: WHITENING TRANSFORMATIONS MULTIVARIATE NORMAL DISTRIBUTIONS A multivariate distribution is defined as: where  represents the mean (vector) and  represents the covariance (matrix). Note the exponent term is really a dot product or weighted Euclidean distance. The covariance is always symmetric and positive semidefinite. How does the shape vary as a function of the covariance?

07: WHITENING TRANSFORMATIONS SUPPORT REGIONS A support region is the obtained by the intersection of a Gaussian distribution with a plane. For a horizontal plane, this generates an ellipse whose points are of equal probability density. The shape of the support region is defined by the covariance matrix.

07: WHITENING TRANSFORMATIONS DERIVATION

07: WHITENING TRANSFORMATIONS IDENTITY COVARIANCE

07: WHITENING TRANSFORMATIONS UNEQUAL VARIANCES

07: WHITENING TRANSFORMATIONS OFF-DIAGONAL ELEMENTS

07: WHITENING TRANSFORMATIONS UNCONSTRAINED ELEMENTS

07: WHITENING TRANSFORMATIONS COORDINATE TRANSFORMATIONS Why is it convenient to convert an arbitrary distribution into a spherical one? (Hint: Euclidean distance) Consider the transformation: A w =   -1/2 where  is the matrix whose columns are the orthonormal eigenvectors of  and  is a diagonal matrix of eigenvalues (  =    t ). Note that  is unitary. What is the covariance of y=A w x? E[yy t ]= (A w x)(A w x) t =(   -1/2 x) (   -1/2 x) t =   -1/2 x x t  -1/2  t =   -1/2   -1/2  t =   -1/2    t  -1/2  t =   t  -1/2   -1/2 (  t ) = I

07: WHITENING TRANSFORMATIONS MAHALANOBIS DISTANCE The weighted Euclidean distance: is known as the Mahalanobis distance, and represents a statistically normalized distance calculation that results from our whitening transformation. Consider an example using our Java Applet.Java Applet