1 Nearest Neighbor Learning Greg Grudic (Notes borrowed from Thomas G. Dietterich and Tom Mitchell) Intro AI.

Slides:

Advertisements

Similar presentations

1 Classification using instance-based learning. 3 March, 2000Advanced Knowledge Management2 Introduction (lazy vs. eager learning) Notion of similarity.

Advertisements

Data Mining Classification: Alternative Techniques

Data Mining Classification: Alternative Techniques

1 CS 391L: Machine Learning: Instance Based Learning Raymond J. Mooney University of Texas at Austin.

Indian Statistical Institute Kolkata

Instance Based Learning

1 Machine Learning: Lecture 7 Instance-Based Learning (IBL) (Based on Chapter 8 of Mitchell T.., Machine Learning, 1997)

Lazy vs. Eager Learning Lazy vs. eager learning

Lecture 6 K-Nearest Neighbor Classifier

Classification and Decision Boundaries

Navneet Goyal. Instance Based Learning  Rote Classifier  K- nearest neighbors (K-NN)  Case Based Resoning (CBR)

Instance Based Learning

Nearest Neighbor. Predicting Bankruptcy Nearest Neighbor Remember all your data When someone asks a question –Find the nearest old data point –Return.

Sparse vs. Ensemble Approaches to Supervised Learning

Instance based learning K-Nearest Neighbor Locally weighted regression Radial basis functions.

CS 590M Fall 2001: Security Issues in Data Mining Lecture 3: Classification.

By Fernando Seoane, April 25 th, 2006 Demo for Non-Parametric Classification Euclidean Metric Classifier with Data Clustering.

Greg GrudicIntro AI1 Introduction to Artificial Intelligence CSCI 3202 Fall 2007 Introduction to Classification Greg Grudic.

Instance Based Learning

Comparison of Instance-Based Techniques for Learning to Predict Changes in Stock Prices iCML Conference December 10, 2003 Presented by: David LeRoux.

Data Mining Classification: Alternative Techniques

These slides are based on Tom Mitchell’s book “Machine Learning” Lazy learning vs. eager learning Processing is delayed until a new instance must be classified.

An Intelligent & Incremental Approach to kNN using R-trees DJ Oneil & Esten Rye (G01)

CES 514 – Data Mining Lec 9 April 14 Mid-term k nearest neighbor.

Aprendizagem baseada em instâncias (K vizinhos mais próximos)

KNN, LVQ, SOM. Instance Based Learning K-Nearest Neighbor Algorithm (LVQ) Learning Vector Quantization (SOM) Self Organizing Maps.

INSTANCE-BASE LEARNING

1 Decision Trees Greg Grudic (Notes borrowed from Thomas G. Dietterich and Tom Mitchell) Modified by Longin Jan Latecki Some slides by Piyush Rai Intro.

Sparse vs. Ensemble Approaches to Supervised Learning

General Mining Issues a.j.m.m. (ton) weijters Overfitting Noise and Overfitting Quality of mined models (some figures are based on the ML-introduction.

Nearest Neighbor Classifiers other names: –instance-based learning –case-based learning (CBL) –non-parametric learning –model-free learning.

CS Instance Based Learning1 Instance Based Learning.

Module 04: Algorithms Topic 07: Instance-Based Learning

Learning: Nearest Neighbor Artificial Intelligence CMSC January 31, 2002.

Data Mining Joyeeta Dutta-Moscato July 10, Wherever we have large amounts of data, we have the need for building systems capable of learning information.

Machine Learning1 Machine Learning: Summary Greg Grudic CSCI-4830.

K Nearest Neighborhood (KNNs)

DATA MINING LECTURE 10 Classification k-nearest neighbor classifier Naïve Bayes Logistic Regression Support Vector Machines.

COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.

K Nearest Neighbors Classifier & Decision Trees

Overview of Supervised Learning Overview of Supervised Learning2 Outline Linear Regression and Nearest Neighbors method Statistical Decision.

1 CSCI 3202: Introduction to AI Decision Trees Greg Grudic (Notes borrowed from Thomas G. Dietterich and Tom Mitchell) Intro AIDecision Trees.

Chapter1: Introduction Chapter2: Overview of Supervised Learning

KNN Classifier.  Handed an instance you wish to classify  Look around the nearby region to see what other classes are around  Whichever is most common—make.

DATA MINING LECTURE 10b Classification k-nearest neighbor classifier

Meta-learning for Algorithm Recommendation Meta-learning for Algorithm Recommendation Background on Local Learning Background on Algorithm Assessment Algorithm.

Greg GrudicIntro AI1 Support Vector Machine (SVM) Classification Greg Grudic.

K-Nearest Neighbor Learning.

Eick: kNN kNN: A Non-parametric Classification and Prediction Technique Goals of this set of transparencies: 1.Introduce kNN---a popular non-parameric.

Debrup Chakraborty Non Parametric Methods Pattern Recognition and Machine Learning.

Kansas State University Department of Computing and Information Sciences CIS 890: Special Topics in Intelligent Systems Wednesday, November 15, 2000 Cecil.

Instance-Based Learning Evgueni Smirnov. Overview Instance-Based Learning Comparison of Eager and Instance-Based Learning Instance Distances for Instance-Based.

CS 8751 ML & KDDInstance Based Learning1 k-Nearest Neighbor Locally weighted regression Radial basis functions Case-based reasoning Lazy and eager learning.

Machine Learning Supervised Learning Classification and Regression K-Nearest Neighbor Classification Fisher’s Criteria & Linear Discriminant Analysis Perceptron:

Lecture 05: K-nearest neighbors

Classification Nearest Neighbor

Decision Trees Greg Grudic

K Nearest Neighbor Classification

Classification Nearest Neighbor

Nearest-Neighbor Classifiers

یادگیری بر پایه نمونه Instance Based Learning Instructor : Saeed Shiry

Instance Based Learning

COSC 4335: Other Classification Techniques

Chap 8. Instance Based Learning

Junheng, Shengming, Yunsheng 11/2/2018

Nearest Neighbors CSC 576: Data Mining.

Lecture 03: K-nearest neighbors

Data Mining Classification: Alternative Techniques

CSE4334/5334 Data Mining Lecture 7: Classification (4)

Presentation transcript:

1 Nearest Neighbor Learning Greg Grudic (Notes borrowed from Thomas G. Dietterich and Tom Mitchell) Intro AI

2 Nearest Neighbor Algorithm Given training data Define a distance metric between points in inputs space. Common measures are: –Euclidean (squared) –Weighted Euclidean Intro AI

3 K-Nearest Neighbor Model Given test point Find the K nearest training inputs to given the distance metric Denote these points as Intro AI

4 K-Nearest Neighbor Model Regression: Classification: Intro AI

5 K-Nearest Neighbor Model: Weighted by Distance Regression: Classification: Intro AI

6 Picking K (and ) Given training examples Use N fold cross validation –Search over K = (1,2,3,…,Kmax). Choose search size Kmax based on compute constraints –Calculated the average error for each K: Calculate predicted class for each training point (using all other points to build the model) Average over all training examples Pick K to minimize the cross validation error Intro AI

7 Class Decision Boundaries: The Voronoi Diagram Each line segment is equidistance between points in opposite classes. The more points, the more complex the boundaries. Intro AI

8 K-Nearest Neighbor Algorithm Characteristics Universal Approximator –Can model any many-to-one mapping arbitrarily well Curse of Dimensionality: Can be easily fooled in high dimensional spaces –Dimensionality reduction techniques are often used Model can be slow to evaluate for large training sets –kd-trees can help (low dimensional problems) –Cover Trees (high dimensional): –Selectively storing data points also helps Intro AI

9 kd-trees Intro AI