Machine Learning 12. Local Models.

Slides:

Advertisements

Similar presentations

Memristor in Learning Neural Networks

Advertisements

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

2806 Neural Computation Self-Organizing Maps Lecture Ari Visa.

Neural Networks Chapter 9 Joost N. Kok Universiteit Leiden.

Self Organization: Competitive Learning

Kohonen Self Organising Maps Michael J. Watts

INTRODUCTION TO Machine Learning 2nd Edition

Competitive Networks. Outline Hamming Network.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Radial Basis-Function Networks. Back-Propagation Stochastic Back-Propagation Algorithm Step by Step Example Radial Basis-Function Networks Gaussian response.

Radial Basis Functions

Slides are based on Negnevitsky, Pearson Education, Lecture 8 Artificial neural networks: Unsupervised learning n Introduction n Hebbian learning.

1 Chapter 11 Neural Networks. 2 Chapter 11 Contents (1) l Biological Neurons l Artificial Neurons l Perceptrons l Multilayer Neural Networks l Backpropagation.

Prediction Networks Prediction –Predict f(t) based on values of f(t – 1), f(t – 2),… –Two NN models: feedforward and recurrent A simple example (section.

MACHINE LEARNING 12. Multilayer Perceptrons. Neural Networks Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Hazırlayan NEURAL NETWORKS Radial Basis Function Networks I PROF. DR. YUSUF OYSAL.

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Aula 4 Radial Basis Function Networks

Hazırlayan NEURAL NETWORKS Radial Basis Function Networks II PROF. DR. YUSUF OYSAL.

Radial Basis Function Networks

Radial Basis Function Networks

Soft Computing Colloquium 2 Selection of neural network, Hybrid neural networks.

CS623: Introduction to Computing with Neural Nets (lecture-20) Pushpak Bhattacharyya Computer Science and Engineering Department IIT Bombay.

Presentation on Neural Networks.. Basics Of Neural Networks Neural networks refers to a connectionist model that simulates the biophysical information.

Chapter 4 Supervised learning: Multilayer Networks II.

Self Organizing Maps (SOM) Unsupervised Learning.

Intrusion Detection Using Hybrid Neural Networks Vishal Sevani ( )

Machine Learning Neural Networks (3). Understanding Supervised and Unsupervised Learning.

MACHINE LEARNING 8. Clustering. Motivation Based on E ALPAYDIN 2004 Introduction to Machine Learning © The MIT Press (V1.1) 2  Classification problem:

Neural Networks - Lecture 81 Unsupervised competitive learning Particularities of unsupervised learning Data clustering Neural networks for clustering.

UNSUPERVISED LEARNING NETWORKS

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture.

1 Adaptive Resonance Theory. 2 INTRODUCTION Adaptive resonance theory (ART) was developed by Carpenter and Grossberg[1987a] ART refers to the class of.

Radial Basis Function ANN, an alternative to back propagation, uses clustering of examples in the training set.

1 Lecture 6 Neural Network Training. 2 Neural Network Training Network training is basic to establishing the functional relationship between the inputs.

Unsupervised Learning Networks 主講人 : 虞台文. Content Introduction Important Unsupervised Learning NNs – Hamming Networks – Kohonen’s Self-Organizing Feature.

CHAPTER 14 Competitive Networks Ming-Feng Yeh.

Example Apply hierarchical clustering with d min to below data where c=3. Nearest neighbor clustering d min d max will form elongated clusters!

Supervised Learning – Network is presented with the input and the desired output. – Uses a set of inputs for which the desired outputs results / classes.

Machine Learning Supervised Learning Classification and Regression

Big data classification using neural network

Chapter 5 Unsupervised learning

Self-Organizing Network Model (SOM) Session 11

Multilayer Perceptrons

Neural Networks Winter-Spring 2014

第 3 章神经网络.

Unsupervised Learning Networks

Chapter 4 Supervised learning: Multilayer Networks II

CSE P573 Applications of Artificial Intelligence Neural Networks

CSE 473 Introduction to Artificial Intelligence Neural Networks

Unsupervised learning

Lecture 22 Clustering (3).

Neuro-Computing Lecture 4 Radial Basis Function Network

Artificial Intelligence Chapter 3 Neural Networks

CSE 573 Introduction to Artificial Intelligence Neural Networks

Artificial Intelligence Chapter 3 Neural Networks

INTRODUCTION TO Machine Learning

Introduction to Radial Basis Function Networks

CSCE833 Machine Learning Lecture 9 Linear Discriminant Analysis

Artificial Intelligence Chapter 3 Neural Networks

Artificial Intelligence Chapter 3 Neural Networks

Feature mapping: Self-organizing Maps

Radial Basis Functions: Alternative to Back Propagation

Prediction Networks Prediction A simple example (section 3.7.3)

Artificial Neural Networks

Artificial Intelligence Chapter 3 Neural Networks

Presentation transcript:

Machine Learning 12. Local Models

Introduction Clustering Use neural network for unsupervised learning Using K-means algorithm Iterative algorithm Batch learning Use neural network for unsupervised learning Online learning Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Online K-means Reconstruction Error For batch k-means, center update is Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Online k-means Reconstruction error for single instance Using gradient descent Move closest center to the direction of new sample Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Competitive Learning Based on 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Competitive Learning Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Using Dot product If All centers have the same norm Minimum Euclidian distance Correspond to maximum dot product Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Choosing maximum using NN Using Recurrent Networks Lateral inhibition Positive(excitatory) recurrent connection to itself Negative(inhibitory) recurrent connection to its neighbors Suitable weights and activation function converge to a maximum Singe output is 1 All other 0 Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

NN network Winner-take-all network Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Hebbian learning Update First term: increase weight if input and output are activated together (correlated) Second term: prevent unbounded weight growth Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Adaptive Resonance Theory Incremental; add a new cluster if not covered; vigilance, ρ (Carpenter and Grossberg, 1988) Based on Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Self-Organizing Maps Units have a neighborhood defined; mi is “between” mi-1 and mi+1, and are all updated together One-dim map: (Kohonen, 1990) Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Locally Receptive Units Divide the input space into local regions and learn simple (e.g. constant/linear) models in each patch Radial-basis func, mixture of experts Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Local vs Distributed Representation Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Radial-Basis Functions Locally-tuned units: Based on E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Training RBF Hybrid learning: Fully supervised RBF is differentiable First layer centers and spreads: Unsupervised k-means Second layer weights: Supervised gradient-descent Fully supervised RBF is differentiable Back propagation Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)

Rule-Based Knowledge Incorporation of prior knowledge (before training) Rule extraction (after training) (Tresp et al., 1997) Lecture Notes for E Alpaydın 2004 Introduction to Machine Learning © The MIT Press (V1.1)