1. Default ARTMAP 2 Gregory Amis & Gail A. Carpenter International Joint Conference on Neural Networks (IJCNN) Orlando, Florida August, 2007

2. 2 Geometric example: Circle-in-Square x y 01 1 Task: Discriminate points INSIDE the circle from points OUTSIDE the circle Using training data like

3. 3 ARTMAP Geometry Long-term memory weight vector for coding node: feature 1 feature 2 01 0 1 Geometrically represented as a hyper-rectangle in the input feature space called a category box R w = ( u, v c ) v c = 1 − v v u

4. 4 T J = M – d(R J,a) –  |R J | Input signal to coding node feature 1 feature 2 Input signal to coding node J : Choice-by-difference rule Input signal T J is greatest for box closest to input a If point a is in multiple boxes the smallest box gets greatest signal a d(R1,a)d(R1,a) |R3||R3| R1R1 R3R3 R2R2 R4R4 d(R4,a)d(R4,a) # input features0.01 city-block distancebox size Measure of similarity between coding node weights and input pattern Determines which coding nodes learn and contribute to class prediction

5. 5 1 st training pair creates point box x y 01 1 0 a (1) No coding nodes/boxes exist First input:class training pair (a (1),IN) presented. R1R1 u 1 = v 1 = a (1) Recruit new coding node, Fast commit learning sets w 1 = A, u 1 = v 1 = a (1) creates point box R 1 at a

6. 6 u 1 new = u 1 old  a (2) = u 1 old v 1 new = v 1 old  a (2) = a (2) u 1 = v 1 = a (1) 2 nd pair expands existing box x y 01 1 0 a (2) Existing box R 1 matches a (2) well enough Second training pair (a (2),IN) presented. R1R1 Fast learning expands R 1 to include a (2) R1R1 R1  aR1  a

7. 7 Existing box R 1 matches a (3) well enough but it’s the wrong class Match tracking raises vigilance resets R 1 3 rd pair creates new OUT point box x y 01 1 0 a (3) New node recruited creating OUT point box R 2 Third training pair (a (3),OUT) presented. R1R1 R2R2

8. 8 4 th point extends OUT box x y 01 1 0 a (4) Fourth training pair (a (4),OUT) presented. Existing box R 1 matches a (4) well enough but it’s the wrong class Match tracking raises vigilance resets R 1 R 2 wins passes vigilance because |R 2  a|  |R 1  a| and learns R2R2 R1R1

9. 9 5 th pair creates new OUT point box x y 01 1 0 a (5) Fifth training pair (a (5),OUT) presented. R2R2 Existing box R 1 matches a (5) well enough but it’s the wrong class Match tracking resets R 1 R 2 wins, but fails vigilance |R 2  a| > |R 1  a| New node recruited creating OUT point box R 3 R3R3 R1R1

10. 10 6 th pair expands R 3 x y 01 1 0 Sixth training pair (a (6),OUT) presented. R2R2 R1R1 Existing box R 1 matches a (5) well enough but it’s the wrong class Match tracking resets R 1 R 2 wins, but fails vigilance R 3 wins, passes vigilance and learns R3R3 a (6) R3R3

11. 11 At this point… x y 01 1 0 Classification Pattern Winner-Take-All Distributed Activation R1R1 R2R2 R3R3

12. 12 7 th pair fails next input test! x y 01 1 0 a (7) Classification Pattern Winner-Take-All Distributed Activation Predicts a (7) is IN Correct! Still thinks a (7) is OUT Wrong! Seventh training pair (a (7),IN) presented. No matter how many times a (7) is presented, Default ARTMAP 1 will never correct this mistake! R1R1 R2R2 R3R3

13. 13 Default ARTMAP 2 learns on a (7) x y 01 1 0 Distributed Activation Predicts a (7) is IN Correct! Seventh training pair (a (7),IN) presented. Checks distributed next-input test: fails Match tracking raises vigilance resets R 1 a (7) New coding node recruited creates point box R 4 Passes next-input test! R4R4 R1R1 R2R2 R3R3