1. DESIGN OF A SELF- ORGANIZING LEARNING ARRAY SYSTEM Dr. Janusz Starzyk Tsun-Ho Liu Ohio University School of Electrical Engineering and Computer Science May 25-28 th, 2003 IEEE International Symposium on Circuits and Systems

2. May 25-28 th, 2003 School of Electrical Engineering and Computer Science2 Outline  Introduction  Self-Organizing Learning Array Structure  Neuron Structure and Self-Organizing Principles  Data Preprocessing  Software Simulation Result  Conclusion and Future Work

3. May 25-28 th, 2003 School of Electrical Engineering and Computer Science3 Introduction  Digital computers are good at:  Fast arithmetic calculation  Precise software execution  Artificial Neural Networks are good at:  Software free  Robust classification and pattern recognition  Recommendation of an action  Massive parallelism

4. May 25-28 th, 2003 School of Electrical Engineering and Computer Science4 Introduction (Cont’d)  Research Objective:  Less interconnection  Self-organizing  Local Learning  Nonspecific classification

5. May 25-28 th, 2003 School of Electrical Engineering and Computer Science5 Self-Organizing Learning Array Structure (Cont’d)  Feed forward organization and structure

6. May 25-28 th, 2003 School of Electrical Engineering and Computer Science6 Self-Organizing Learning Array Structure (Cont’d)  Initial Wiring

7. May 25-28 th, 2003 School of Electrical Engineering and Computer Science7 Neuron Structure and Self- Organizing Principles  Neuron Input - System clock

8. May 25-28 th, 2003 School of Electrical Engineering and Computer Science8 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron Input - Data input

9. May 25-28 th, 2003 School of Electrical Engineering and Computer Science9 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron Input - Threshold control input (TCI)

10. May 25-28 th, 2003 School of Electrical Engineering and Computer Science10 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron Input - Input information deficiency  Indication of how much the input space (corresponding to this selected TCI) has been learned  [0, 1]  1 is set initially at the first input layer  0 indicates this neuron has solved the problem 100%

11. May 25-28 th, 2003 School of Electrical Engineering and Computer Science11 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron inside  Transformation functions  Linear and nonlinear  Single input or multiple inputs  Information index calculation

12. May 25-28 th, 2003 School of Electrical Engineering and Computer Science12 Neuron Structure and Self- Organizing Principles (Cont’d)

13. May 25-28 th, 2003 School of Electrical Engineering and Computer Science13 Neuron Structure and Self- Organizing Principles (Cont’d)

14. May 25-28 th, 2003 School of Electrical Engineering and Computer Science14 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron output - System output

15. May 25-28 th, 2003 School of Electrical Engineering and Computer Science15 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron output - Output Clock

16. May 25-28 th, 2003 School of Electrical Engineering and Computer Science16 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron output - Output information deficiency  of TCO = Input information deficiency  of TCOT = Input information deficiency * local information deficiency (pass threshold)  of TCOTI = Input information deficiency * local information deficiency (does not pass threshold)

17. May 25-28 th, 2003 School of Electrical Engineering and Computer Science17 Data Preprocessing  Missing data recovery  All features are independent  Some features are dependent  Ref: [Liu] & [Starzyk & Zhu]  Symbolic values assignment  Number of numerical feature = 1  Number of numerical features > 1

18. May 25-28 th, 2003 School of Electrical Engineering and Computer Science18 Symbolic value – numerical feature =1 1) 2) 3) 4)

19. May 25-28 th, 2003 School of Electrical Engineering and Computer Science19 Symbolic value – numerical feature =1  Symbolic value – numerical feature =1 X s = [1.0 3.0 3.0 3.5 3.5 8.5 8.5 9.0 9.0 9.0] T

20. May 25-28 th, 2003 School of Electrical Engineering and Computer Science20 Data Preprocessing (Cont’d) 1) 2) 3) 4) 5)

21. May 25-28 th, 2003 School of Electrical Engineering and Computer Science21 Data Preprocessing (Cont’d)  Symbolic value – numerical feature > 1 X s = [1.0 2.85 2.85 3.274 3.274 7.241 7.241 7.884 7.88 7.884] T

22. May 25-28 th, 2003 School of Electrical Engineering and Computer Science22 Software Simulation Result

23. May 25-28 th, 2003 School of Electrical Engineering and Computer Science23 Software Simulation Result (Cont’d) FSS Naïve Bayes0.1405 NBTree0.1410 C4.5-auto0.1446 IDTM (Decision table)0.1446 HOODG / SOLAR0.1482 C4.5 rules0.1494 OC10.1504 C4.50.1554 Voted ID3 (0.6)0.1564 CN20.1600 Naïve-Bayes0.1612 Voted ID3 (0.8)0.1647 T20.1687 1R0.1954 Nearest-neighbor (3)0.2035 Nearest-neighbor (1)0.2142 PeblsCrashed

24. May 25-28 th, 2003 School of Electrical Engineering and Computer Science24 Conclusion and Future Work  Conclusion  Local learning  Self-organizing  Data preprocessing  Future work  VHDL simulation  FPGA machine  VLSI design

25. May 25-28 th, 2003 School of Electrical Engineering and Computer Science25 Reference  Information & Computer Science (ICS), University of California at Irvine (UCI). (1995, December), Machine Learning Repository, Available FTP: Hostname: ftp.ics.uci.edu Directory: /pub/machine-learning- databases/  Liu T. H. (2002), Thesis, Future Hardware Realization of Self- Organizing Learning Array and Its Software Simulation. School of Electrical Engineering and Computer Science, Ohio University.  Starzyk A. J. and Zhu Z. (2002), Software Simulation of a Self- Organizing Learning Array. Int. Conf. on Artificial Intelligence and Soft Computing.