Debrup Chakraborty Non Parametric Methods Pattern Recognition and Machine Learning.

Slides:

Advertisements

Similar presentations

Principles of Density Estimation

Advertisements

Clustering k-mean clustering Genome 559: Introduction to Statistical and Computational Genomics Elhanan Borenstein.

K-means method for Signal Compression: Vector Quantization

1 Machine Learning: Lecture 10 Unsupervised Learning (Based on Chapter 9 of Nilsson, N., Introduction to Machine Learning, 1996)

Support Vector Machines and Margins

1 CS 391L: Machine Learning: Instance Based Learning Raymond J. Mooney University of Texas at Austin.

1 Machine Learning: Lecture 7 Instance-Based Learning (IBL) (Based on Chapter 8 of Mitchell T.., Machine Learning, 1997)

Lazy vs. Eager Learning Lazy vs. eager learning

Classification and Decision Boundaries

Pattern recognition Professor Aly A. Farag

Instance Based Learning

K nearest neighbor and Rocchio algorithm

MACHINE LEARNING 9. Nonparametric Methods. Introduction Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2 

CS292 Computational Vision and Language Pattern Recognition and Classification.

By Fernando Seoane, April 25 th, 2006 Demo for Non-Parametric Classification Euclidean Metric Classifier with Data Clustering.

Instance-Based Learning

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

These slides are based on Tom Mitchell’s book “Machine Learning” Lazy learning vs. eager learning Processing is delayed until a new instance must be classified.

Semi-Supervised Clustering Jieping Ye Department of Computer Science and Engineering Arizona State University

Aprendizagem baseada em instâncias (K vizinhos mais próximos)

KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.

Chapter 4 (part 2): Non-Parametric Classification

Nearest Neighbour Condensing and Editing David Claus February 27, 2004 Computer Vision Reading Group Oxford.

CS Instance Based Learning1 Instance Based Learning.

Methods in Medical Image Analysis Statistics of Pattern Recognition: Classification and Clustering Some content provided by Milos Hauskrecht, University.

嵌入式視覺 Pattern Recognition for Embedded Vision Template matching Statistical / Structural Pattern Recognition Neural networks.

Unsupervised Learning and Clustering k-means clustering Sum-of-Squared Errors Competitive Learning SOM Pre-processing and Post-processing techniques.

Data mining and machine learning A brief introduction.

COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.

Nearest Neighbor (NN) Rule & k-Nearest Neighbor (k-NN) Rule Non-parametric : Can be used with arbitrary distributions, No need to assume that the form.

Non-Parameter Estimation 主講人：虞台文. Contents Introduction Parzen Windows k n -Nearest-Neighbor Estimation Classification Techiques – The Nearest-Neighbor.

So Far……  Clustering basics, necessity for clustering, Usage in various fields : engineering and industrial fields  Properties : hierarchical, flat,

Visual Information Systems Recognition and Classification.

David Claus and Christoph F. Eick: Nearest Neighbor Editing and Condensing Techniques Nearest Neighbor Editing and Condensing Techniques 1.Nearest Neighbor.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John Wiley.

METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Part 6: Nearest and k-nearest Neighbor Classification.

Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.

Clustering Instructor: Max Welling ICS 178 Machine Learning & Data Mining.

Machine Learning Queens College Lecture 7: Clustering.

KNN & Naïve Bayes Hongning Wang Today’s lecture Instance-based classifiers – k nearest neighbors – Non-parametric learning algorithm Model-based.

6.S093 Visual Recognition through Machine Learning Competition Image by kirkh.deviantart.com Joseph Lim and Aditya Khosla Acknowledgment: Many slides from.

CS Machine Learning Instance Based Learning (Adapted from various sources)

Example Apply hierarchical clustering with d min to below data where c=3. Nearest neighbor clustering d min d max will form elongated clusters!

Eick: kNN kNN: A Non-parametric Classification and Prediction Technique Goals of this set of transparencies: 1.Introduce kNN---a popular non-parameric.

Fuzzy Pattern Recognition. Overview of Pattern Recognition Pattern Recognition Procedure Feature Extraction Feature Reduction Classification (supervised)

Given a set of data points as input Randomly assign each point to one of the k clusters Repeat until convergence – Calculate model of each of the k clusters.

Intro. ANN & Fuzzy Systems Lecture 20 Clustering (1)

Linear Models & Clustering Presented by Kwak, Nam-ju 1.

KNN & Naïve Bayes Hongning Wang

Fuzzy Logic in Pattern Recognition

Semi-Supervised Clustering

Nonparametric Density Estimation – k-nearest neighbor (kNN) 02/20/17

Non-Parameter Estimation

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

Instance Based Learning (Adapted from various sources)

Machine Learning Week 1.

K Nearest Neighbor Classification

Nearest-Neighbor Classifiers

Instance Based Learning

COSC 4335: Other Classification Techniques

Chap 8. Instance Based Learning

Nearest Neighbors CSC 576: Data Mining.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Hairong Qi, Gonzalez Family Professor

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

ECE – Pattern Recognition Lecture 10 – Nonparametric Density Estimation – k-nearest-neighbor (kNN) Hairong Qi, Gonzalez Family Professor Electrical.

Presentation transcript:

Debrup Chakraborty Non Parametric Methods Pattern Recognition and Machine Learning

Nearest Neighbor classification Given: Given a labeled sample of n feature vectors ( call X) A distance measure (say the Euclidian Distance) To find: The class label of a given feature vector x which is not in X

Nearest Neighbor classification (contd.) The NN rule: Find the point y in X which is nearest to x Assign the label of y to x

Nearest Neighbor classification (contd.) This rule allows us to partition the feature space into cells consisting of all points closer to a given training point x All points in such cells are labeled by the class of the training point. This partitioning is called a Voronoi Tesselation

Voronoi Cells in 2d Nearest Neighbor classification (contd.)

Nearest Neighbor classification Complexity of the NN rule Distance calculation Finding the minimum distance

Nearest Neighbor classification Nearest Neighbor Editing X= Data set, n= no of training points, j=0 Construct the full Voronoi diagram for X Do j=j+1, for each point x j in X find Voronoi neighbors of x j If any neighbor is not from the same class as x j then mark x j Until j==n Discard all points that are not marked.

k nearest neighbor classification Given: Given a labeled sample of N feature vectors ( call X) A distance measure (say the Euclidian Distance) An integer k (generally odd) To find: The class label of a given feature vector x which is not in X

k-NN classification (contd.) Algorithm: Find out the k nearest neighbors of x in X Call them Out of the k samples, let k i of them belong to class c i. Choose that c i to be the class of x for which k i is maximum

K-nn Classification Class 1Class 2Class 3 z

k-NN classification (contd.) Distance weighted nearest neighbor Training set Given an instance x to be classified Let be the nearest neighbors of x Return In case x=x i, return f(x i )

Remarks on k-NN classification The distance weighted kNN is robust to noisy training data and is quite effective when it is provided a sufficiently large set of training examples. One drawbak of kNN method is that, it defers all computation till a new querry point is presented. Various methods have been developed to index the training examples so that the nearest neighbor can be found with less search time. One such indexing method is kd-tree developed by Bently 1975 kNN is a lazy learner

Locally Weighted Regression In the linear regression problem, to find h(x) at a point x we would do the following: 1.Minimize 2.Output

Locally Weighted Regression In the llocally weighted regression problem we would do the following 1.Minimize 2.Output A standard choice of weights is is called the bandwidth parameter

Clustering Is different from Classification Classification is partitioning the feature space whereas Clustering is partitioning the data into “homogeneous groups” Clustering is Unsupervised!!

K-means Clustering Given: A data set Fix the number of clusters K Let represent the i-th cluster center (prototype) at the k-th iteration Let represent the j-th cluster at the k-th iteration

K-means Clustering Steps 1.Choose the initial cluster centers 2.At the k-th iterative step distribute the points in X in K cluster using: 3.Compute 4.If then the procedure has converged else repeat from 2.