Learning.

Slides:

Advertisements

Similar presentations

Data Mining Lecture 9.

Advertisements

Computational Learning An intuitive approach. Human Learning Objects in world –Learning by exploration and who knows? Language –informal training, inputs.

Naïve-Bayes Classifiers Business Intelligence for Managers.

Decision Tree Approach in Data Mining

Data Mining Classification: Alternative Techniques

. Markov Chains as a Learning Tool. 2 Weather: raining today40% rain tomorrow 60% no rain tomorrow not raining today20% rain tomorrow 80% no rain tomorrow.

Navneet Goyal. Instance Based Learning  Rote Classifier  K- nearest neighbors (K-NN)  Case Based Resoning (CBR)

1 Pattern Recognition Pattern recognition is: 1. The name of the journal of the Pattern Recognition Society. 2. A research area in which patterns in data.

1 Chapter 10 Introduction to Machine Learning. 2 Chapter 10 Contents (1) l Training l Rote Learning l Concept Learning l Hypotheses l General to Specific.

Learning from Observations Chapter 18 Section 1 – 4.

CS 590M Fall 2001: Security Issues in Data Mining Lecture 3: Classification.

1 Pattern Recognition Pattern recognition is: 1. The name of the journal of the Pattern Recognition Society. 2. A research area in which patterns in data.

CSE803 Fall Pattern Recognition Concepts Chapter 4: Shapiro and Stockman How should objects be represented? Algorithms for recognition/matching.

Lazy Learning k-Nearest Neighbour Motivation: availability of large amounts of processing power improves our ability to tune k-NN classifiers.

Data Mining with Decision Trees Lutz Hamel Dept. of Computer Science and Statistics University of Rhode Island.

Three kinds of learning

These slides are based on Tom Mitchell’s book “Machine Learning” Lazy learning vs. eager learning Processing is delayed until a new instance must be classified.

Instance Based Learning IB1 and IBK Small section in chapter 20.

Aprendizagem baseada em instâncias (K vizinhos mais próximos)

1 MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING By Kaan Tariman M.S. in Computer Science CSCI 8810 Course Project.

Learning….in a rather broad sense: improvement of performance on the basis of experience Machine learning…… improve for task T with respect to performance.

© Prentice Hall1 DATA MINING Introductory and Advanced Topics Part II Margaret H. Dunham Department of Computer Science and Engineering Southern Methodist.

Memory-Based Learning Instance-Based Learning K-Nearest Neighbor.

Stockman CSE803 Fall Pattern Recognition Concepts Chapter 4: Shapiro and Stockman How should objects be represented? Algorithms for recognition/matching.

Nearest Neighbor Classifiers other names: –instance-based learning –case-based learning (CBL) –non-parametric learning –model-free learning.

CS Instance Based Learning1 Instance Based Learning.

MACHINE LEARNING. What is learning? A computer program learns if it improves its performance at some task through experience (T. Mitchell, 1997) A computer.

Review Rong Jin. Comparison of Different Classification Models  The goal of all classifiers Predicating class label y for an input x Estimate p(y|x)

SEEM Tutorial 2 Classification: Decision tree, Naïve Bayes & k-NN

Hidden Markov Models in Keystroke Dynamics Md Liakat Ali, John V. Monaco, and Charles C. Tappert Seidenberg School of CSIS, Pace University, White Plains,

 2003, G.Tecuci, Learning Agents Laboratory 1 Learning Agents Laboratory Computer Science Department George Mason University Prof. Gheorghe Tecuci 9 Instance-Based.

1 Pattern Recognition Pattern recognition is: 1. A research area in which patterns in data are found, recognized, discovered, …whatever. 2. A catchall.

CS690L Data Mining: Classification

1 Chapter 10 Introduction to Machine Learning. 2 Chapter 10 Contents (1) l Training l Rote Learning l Concept Learning l Hypotheses l General to Specific.

Decision Trees Binary output – easily extendible to multiple output classes. Takes a set of attributes for a given situation or object and outputs a yes/no.

Introduction to Machine Learning Fall 2013 Comp3710 Artificial Intelligence Computing Science Thompson Rivers University.

Classification Today: Basic Problem Decision Trees.

SUPPORT VECTOR MACHINES Presented by: Naman Fatehpuria Sumana Venkatesh.

CMPS 142/242 Review Section Fall 2011 Adapted from Lecture Slides.

Data Mining Practical Machine Learning Tools and Techniques

DECISION TREES An internal node represents a test on an attribute.

Ananya Das Christman CS311 Fall 2016

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

CS 9633 Machine Learning Concept Learning

Introduction to Machine Learning

Reading: Pedro Domingos: A Few Useful Things to Know about Machine Learning source: /cacm12.pdf reading.

Data Mining Lecture 11.

Machine Learning Week 1.

Pattern Recognition Pattern recognition is:

Data Mining Practical Machine Learning Tools and Techniques

Learning with Identification Trees

Hidden Markov Models Part 2: Algorithms

Prepared by: Mahmoud Rafeek Al-Farra

CS 478 Homework CS Homework.

MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING

MACHINE LEARNING TECHNIQUES IN IMAGE PROCESSING

Instance Space (X) X T BP SK x1 L - x2 N x3 H x4 x5 x6 x7 x8 x9.

Introduction to Machine Learning

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Decision trees One possible representation for hypotheses

A task of induction to find patterns

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Implementation of Learning Systems

Memory-Based Learning Instance-Based Learning K-Nearest Neighbor

A task of induction to find patterns

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

Data Mining CSCI 307, Spring 2019 Lecture 11

INFORMATION RETRIEVAL TECHNIQUES BY DR. ADNAN ABID

Presentation transcript:

Learning

What is learning?

Supervised Learning Training data that has been classified Examples Concept learning Decision trees Markov models Nearest neighbor Neural Nets (in coming weeks) Inductive Bias - limits imposed by assumptions! Especially what factors we choose as inputs

Rote Learning Store training data Limitation - does not extend beyond what has been seen Example: concept learning

Concept Learning Inductive learning with generalization Given training data: tuples <a1, a2, a3,…> Boolean value ai - can be any value ? Is used for a don’t care positive Null is used for don’t care negative

A hypothesis if a tuple that is true hg <?, ?, ….> - most general - always true hs <null, null, …> most specific always false hg >= hs Defines a partially ordered lattice

Training Method Use the lattice to generate the most general hypothesis Weakness Inconsistent data Data errors

Decision Trees ID3 algorithm Entropy: a measure of information p(I)log2 p(I) entropy of an element Entropy of the system of information: Sum - p(I)log2 p(I) P(I) is instances of I / total instances This is done over the outputs of the tree

Gain(S, A) = Entropy(S) - S ((|Sv| / |S|) * Entropy(Sv)) Gain is a measure of the effectiveness of a attribute Gain(S, A) = Entropy(S) - S ((|Sv| / |S|) * Entropy(Sv)) Sv Number of outputs with value v is attribute S is the number of elements in the outputs

ID3 Greedy algorithm Select the attribute by the largest gain Iterate until done

Markov Models Markov Chain is a set of states State transitions are probabilistic State xi goes to state xj with P(xj | xi) This can be extended to allow the probability to depend on a set of past states (Memory)

Example from the Text Given a set of words, Markov chain to generate similar words For each letter position of the words, compute probability Use a matrix of counts Count[from][to] Normalize rows by total count in row

Nearest Neighbor 1NN: Use vectors to represent entities Use distance measure between vectors to locate closest known entity Can be effected by noisy data

kNN - better Use k closest neighbors and vote

Other techniques Yet to cover! Evolutionary algorithms Neural nets