Unsupervised Learning Networks 主講人 : 虞台文

Content Introduction Important Unsupervised Learning NNs – Hamming Networks – Kohonen’s Self-Organizing Feature Maps – Grossberg’s ART Networks – Counterpropagation Networks – Adaptive BAN – Neocognitron Conclusion

Unsupervised Learning Networks Introduction

What is Unsupervised Learning? Learning without a teacher. No feedback to indicate the desired outputs. The network must by itself discover the relationship of interest from the input data. – E.g., patterns, features, regularities, correlations, or categories. Translate the discovered relationship into output.

A Strange World

Supervised Learning IQ Height A B C

Supervised Learning IQ Height A B C Try Classification

The Probabilities of Populations IQ Height A B C

The Centroids of Clusters IQ Height A B C

The Centroids of Clusters IQ Height A B C Try Classification

Unsupervised Learning IQ Height

Unsupervised Learning IQ Height

Clustering Analysis IQ Height Categorize the input patterns into several classes based on the similarity among patterns.

Clustering Analysis IQ Height Categorize the input patterns into several classes based on the similarity among patterns. How many classes we may have?

Clustering Analysis IQ Height Categorize the input patterns into several classes based on the similarity among patterns. 2 clusters

Clustering Analysis IQ Height Categorize the input patterns into several classes based on the similarity among patterns. 3 clusters

Clustering Analysis IQ Height Categorize the input patterns into several classes based on the similarity among patterns. 4 clusters

Unsupervised Learning Networks The Hamming Networks

The Nearest Neighbor Classifier Suppose that we have p prototypes centered at x (1), x (2), …, x (p). Given pattern x, it is assigned to the class label of the i th prototype if Examples of distance measures include the Hamming distance and Euclidean distance.

The Nearest Neighbor Classifier x (1) x (2) x (3) x (4) The Stored Prototypes

The Nearest Neighbor Classifier x (1) x (2) x (3) x (4)  ?Class

The Hamming Networks Stored a set of classes represented by a set of binary prototypes. Given an incomplete binary input, find the class to which it belongs. Use Hamming distance as the distance measurement. Distance vs. Similarity.

The Hamming Net Similarity Measurement MAXNET Winner-Take-All x1x1 x2x2 xnxn

The Hamming Distance y = 1   x =  1   1 1 Hamming Distance = ?

y = 1   x =  1   1 1 The Hamming Distance Hamming Distance = 3

y = 1   The Hamming Distance   1  1 1 Sum=1 x =  1   1 1

The Hamming Distance

The Hamming Net Similarity Measurement MAXNET Winner-Take-All n1n1 n1n1 n n x1x1 x2x2 xm1xm1 xmxm n1n1 n1n1 n n y1y1 y2y2 yn1yn1 ynyn

The Hamming Net Similarity Measurement MAXNET Winner-Take-All n1n1 n1n1 n n x1x1 x2x2 xm1xm1 xmxm n1n1 n1n1 n n y1y1 y2y2 yn1yn1 ynyn W S =? W M =?

The Stored Patterns Similarity Measurement MAXNET Winner-Take-All n1n1 n1n1 n n x1x1 x2x2 xm1xm1 xmxm n1n1 n1n1 n n y1y1 y2y2 yn1yn1 ynyn W S =? W M =?

The Stored Patterns Similarity Measurement k x1x1 x2x2 xmxm... m/2

Weights for Stored Patterns Similarity Measurement n1n1 n1n1 n n x1x1 x2x2 xm1xm1 xmxm W S =?

Weights for Stored Patterns Similarity Measurement n1n1 n1n1 n n x1x1 x2x2 xm1xm1 xmxm W S =? m/2

The MAXNET Similarity Measurement MAXNET Winner-Take-All n1n1 n1n1 n n x1x1 x2x2 xm1xm1 xmxm n1n1 n1n1 n n y1y1 y2y2 yn1yn1 ynyn

Weights of MAXNET MAXNET Winner-Take-All n1n1 n1n1 n n y1y1 y2y2 yn1yn1 ynyn 1 1

Weights of MAXNET MAXNET Winner-Take-All n1n1 n1n1 n n y1y1 y2y2 yn1yn1 ynyn   0<  < 1/n 1 1

Updating Rule MAXNET Winner-Take-All n1n1 n1n1 n n   0<  < 1/n 1 1 s1s1 s2s2 s3s3 snsn

Updating Rule MAXNET Winner-Take-All n1n1 n1n1 n n   0<  < 1/n 1 1 s1s1 s2s2 s3s3 snsn

Analysis  Updating Rule Let If now

Analysis  Updating Rule Let If now

Example

Unsupervised Learning Networks The Self-organizing Feature Map

Feature Mapping Map high-dimensional input signals onto a lower- dimensional (usually 1 or 2D) structure. Similarity relations present in the original data are still present after the mapping. Dimensionality Reduction Topology-Preserving Map

Somatotopic Map Illustration: The “Homunculus” The relationship between body surfaces and the regions of the brain that control them.

Another Depiction of the Homunculus

Phonotopic maps

humppila

Self-Organizing Feature Map Developed by professor Kohonen. One of the most popular neural network models. Unsupervised learning. Competitive learning networks.

The Structure of SOM

Example

Local Excitation, Distal Inhibition

Topological Neighborhood SquareHex

Size Shrinkage

Learning Rule Similarity Matching Updating

Example