Presentation is loading. Please wait.

Presentation is loading. Please wait.

Future Hardware Realization of Self-Organizing Learning Array and Its Software Simulation Adviser: Dr. Janusz Starzyk Student: Tsun-Ho Liu Ohio University.

Similar presentations


Presentation on theme: "Future Hardware Realization of Self-Organizing Learning Array and Its Software Simulation Adviser: Dr. Janusz Starzyk Student: Tsun-Ho Liu Ohio University."— Presentation transcript:

1 Future Hardware Realization of Self-Organizing Learning Array and Its Software Simulation Adviser: Dr. Janusz Starzyk Student: Tsun-Ho Liu Ohio University School of Electrical Engineering and Computer Science November 8 th, 2002

2 School of Electrical Engineering and Computer Science2 Outline  Introduction  Overview of Biological Neural Network  Self-Organizing Learning Array Structure  Neuron Structure and Self-Organizing Principles  Final Classification  Data Preprocessing  Software Simulation Result  Conclusion and Future Work

3 November 8th, 2002School of Electrical Engineering and Computer Science3 Introduction  Digital computers are good at:  Fast arithmetic calculation  Precise software execution  Digital computers bed at:  Interacting with data from environment  Adapting to different conditions

4 November 8th, 2002School of Electrical Engineering and Computer Science4 Introduction (Cont’d)  Advantage of Artificial Neural Networks:  Software free  Robust classification and pattern recognition  Recommendation of an action  Massive parallelism

5 November 8th, 2002School of Electrical Engineering and Computer Science5 Introduction (Cont’d)  Research Objective:  Less interconnection  Self-organizing  Local Learning  Nonspecific classification

6 November 8th, 2002School of Electrical Engineering and Computer Science6 Overview of Biological Neural Network  What are biological neurons?  Receiving information  Integrating information  Transmitting information  No homogeneous organization  Different shapes

7 November 8th, 2002School of Electrical Engineering and Computer Science7 Overview of Biological Neural Network (Cont’d)  Structure and function of a neuron (Fraser, 1998, September)

8 November 8th, 2002School of Electrical Engineering and Computer Science8 Overview of Biological Neural Network (Cont’d)  Neurons Communicate Modified (Fraser, 1998, September)

9 November 8th, 2002School of Electrical Engineering and Computer Science9 Overview of Biological Neural Network (Cont’d)  Categories of neurons:  Long-axon cells  Carrying information for long distance  Further away connections  Short-axon cells  Integrating and processing information  Local connections

10 November 8th, 2002School of Electrical Engineering and Computer Science10 Self-Organizing Learning Array Structure  Three sub-components  Input  Process layer  Output

11 November 8th, 2002School of Electrical Engineering and Computer Science11 Self-Organizing Learning Array Structure (Cont’d)  Structure

12 November 8th, 2002School of Electrical Engineering and Computer Science12 Self-Organizing Learning Array Structure (Cont’d)  Feed forward neural network organization

13 November 8th, 2002School of Electrical Engineering and Computer Science13 Self-Organizing Learning Array Structure (Cont’d)  Initial Wiring

14 November 8th, 2002School of Electrical Engineering and Computer Science14 Neuron Structure and Self- Organizing Principles  Neuron’s input and output

15 November 8th, 2002School of Electrical Engineering and Computer Science15 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron Input  System clock  Data input  Threshold control input (TCI)  Input information deficiency

16 November 8th, 2002School of Electrical Engineering and Computer Science16 Neuron Structure and Self- Organizing Principles (Cont’d)  System clock

17 November 8th, 2002School of Electrical Engineering and Computer Science17 Neuron Structure and Self- Organizing Principles (Cont’d)  Data input

18 November 8th, 2002School of Electrical Engineering and Computer Science18 Neuron Structure and Self- Organizing Principles (Cont’d)  Threshold control input (TCI)

19 November 8th, 2002School of Electrical Engineering and Computer Science19 Neuron Structure and Self- Organizing Principles (Cont’d)  Input information deficiency  Indication of how much the input space (corresponding to this selected TCI) has been learned  [0, 1]  1 is set initially at the first input layer  0 indicates this neuron has solved the problem 100%

20 November 8th, 2002School of Electrical Engineering and Computer Science20 Neuron Structure and Self- Organizing Principles (Cont’d)  Termination of learning  Input information deficiency <= the chosen information deficiency threshold (IDT)  Try to learn from other subspace  If none, stop learning.

21 November 8th, 2002School of Electrical Engineering and Computer Science21 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron inside  Transformation functions  Linear and nonlinear  Single input or multiple inputs  Information index calculation

22 November 8th, 2002School of Electrical Engineering and Computer Science22 Neuron Structure and Self- Organizing Principles (Cont’d)

23 November 8th, 2002School of Electrical Engineering and Computer Science23 Neuron Structure and Self- Organizing Principles (Cont’d)

24 November 8th, 2002School of Electrical Engineering and Computer Science24 Neuron Structure and Self- Organizing Principles (Cont’d)  After optimum information index is obtained  Save:  Selected data inputs (or single input)  Selected threshold control input (TCI)  Selected transformation function  Selected threshold value  Probabilities of correct classification for different classes (passed threshold and does not passed threshold)

25 November 8th, 2002School of Electrical Engineering and Computer Science25 Neuron Structure and Self- Organizing Principles (Cont’d)  Neuron output  System output  Output clocks  Threshold control output (TCO)  Threshold control output–threshold (TCOT)  Threshold control output–threshold–inverted (TCOTI)  Output information deficiencies

26 November 8th, 2002School of Electrical Engineering and Computer Science26 Neuron Structure and Self- Organizing Principles (Cont’d)  System output

27 November 8th, 2002School of Electrical Engineering and Computer Science27 Neuron Structure and Self- Organizing Principles (Cont’d)  Output Clock

28 November 8th, 2002School of Electrical Engineering and Computer Science28 Neuron Structure and Self- Organizing Principles (Cont’d)  Output information deficiency  of TCO = Input information deficiency  of TCOT = Input information deficiency * local information deficiency (pass threshold)  of TCOTI = Input information deficiency * local information deficiency (does not pass threshold)

29 November 8th, 2002School of Electrical Engineering and Computer Science29 Final Classification  Voting flags  Select-output-passed-threshold (SOT)  Set if output information deficiency of TCOT <= the voting threshold  Select-output-passed-threshold-inverse (SOTI)  Set if output information deficiency of TCOTI <= the voting threshold

30 November 8th, 2002School of Electrical Engineering and Computer Science30 Final Classification (Cont’d) Testing Data Selected TCI Selected Transformation Function Pass Selected Threshold? END VOTE! SOT=1?SOTI=1? YESNO YES NO

31 November 8th, 2002School of Electrical Engineering and Computer Science31 Final Classification (Cont’d) Neuron Number 12345 Class 1000.0380.1090.12 Class 20.8330.030.9620.8910.88 Class 30.1670.97000 Class 1Class 2Class 3 0.1910.9800.971

32 November 8th, 2002School of Electrical Engineering and Computer Science32 Further discussion Confidence interval

33 November 8th, 2002School of Electrical Engineering and Computer Science33 Further discussion Confidence interval Neuron 1Neuron 2Neuron 3Neuron 4Neuron 5Neuron 6 Number of Samples 1205105200300 P c for Class 11.000.600.710.080.120.25 Low Limit for Class 1 0.20650.53170.40570.0070.08200.2044 High Limit for Class 1 1.000.66460.89780.51800.17230.3020 Calculated Mean 0.76760.59910.67930.18350.12370.2516 P c for Class 20.000.400.290.920.880.75 Low Limit for Class 2 0.000.33540.10220.48200.82770.6980 High Limit for Class 2 0.79350.46830.59430.99300.91800.7956 Calculated Mean 0.23240.40090.32070.81650.87630.7484

34 November 8th, 2002School of Electrical Engineering and Computer Science34 Further discussion Confidence interval  Condition A = Only P c  Condition B = Mean of classes probabilities Condition ACondition B Weight of class10.99990.8851 Weight of class20.96050.9454

35 November 8th, 2002School of Electrical Engineering and Computer Science35 Data Preprocessing  Missing data recovery  All features are independent  Some features are dependent  Symbolic values assignment  Number of numerical feature = 1  Number of numerical features > 1

36 November 8th, 2002School of Electrical Engineering and Computer Science36 Missing data –independent features Original WeightRecovered Weight 0.28000.4818 1.51001.4995 1.29451.1037 0.69950.7079 0.68800.8776 1.10000.9907 0.57800.9907 0.90700.7645 0.96150.8210 1.29600.9341 (ICS, UCI, 1995, December )

37 November 8th, 2002School of Electrical Engineering and Computer Science37 Missing data –independent features

38 November 8th, 2002School of Electrical Engineering and Computer Science38 Missing data – dependent features  Row 1 = Row 2 * 1.03 + Row 4 * 1.02 #1#2#3#4#5#6#7#8 ?5699.75708.95905.64636.74555.45865.55724.2 ?1827.01727.01923.01101.01231.01983.01837.0 2749.02843.02213.02938.02302.02943.02837.02744.0 ?3743.03853.03848.03434.03223.03748.03757.0 4360.04321.04996.04858.04324.04211.04983.04372.0 5700.05495.05323.05959.05483.05321.05848.05748.0 6210.06723.06232.06835.06859.06948.06382.06223.0 5682.85718.34788.45263.35891.16101.04912.056746 7410.07239.07221.07122.07473.07484.07223.07434.0 8318.08372.08843.08235.08243.08873.08332.08321.0 9328.09323.09122.09422.09483.09566.09222.09332.0

39 November 8th, 2002School of Electrical Engineering and Computer Science39 Missing data – dependent features #1#2#3#4#5#6#7#8 5772.15699.75708.95905.64636.74555.45865.55724.2 1919.91827.01727.01923.01101.01231.01983.01837.0 2749.02843.02213.02938.02302.02943.02837.02744.0 3720.23743.03853.03848.03434.03223.03748.03757.0 4360.04321.04996.04858.04324.04211.04983.04372.0 5700.05495.05323.05959.05483.05321.05848.05748.0 6210.06723.06232.06835.06859.06948.06382.06223.0 5682.85718.34788.45263.35891.16101.04912.056746 7410.07239.07221.07122.07473.07484.07223.07434.0 8318.08372.08843.08235.08243.08873.08332.08321.0 9328.09323.09122.09422.09483.09566.09222.09332.0

40 November 8th, 2002School of Electrical Engineering and Computer Science40 Symbolic value – numerical feature =1 1) 2) 3) 4)

41 November 8th, 2002School of Electrical Engineering and Computer Science41 Symbolic value – numerical feature =1  Symbolic value – numerical feature =1 X s = [1.0 3.0 3.0 3.5 3.5 8.5 8.5 9.0 9.0 9.0] T

42 November 8th, 2002School of Electrical Engineering and Computer Science42 Data Preprocessing (Cont’d) 1) 2) 3) 4) 5)

43 November 8th, 2002School of Electrical Engineering and Computer Science43 Data Preprocessing (Cont’d)  Symbolic value – numerical feature > 1 X s = [1.0 2.85 2.85 3.274 3.274 7.241 7.241 7.884 7.88 7.884] T

44 November 8th, 2002School of Electrical Engineering and Computer Science44 Software Simulation Result Class 1Class2Class 3Class 4Class 5 Number of points 503429190682542

45 November 8th, 2002School of Electrical Engineering and Computer Science45 Software Simulation Result (Cont’d)

46 November 8th, 2002School of Electrical Engineering and Computer Science46 Software Simulation Result (Cont’d)

47 November 8th, 2002School of Electrical Engineering and Computer Science47 Software Simulation Result (Cont’d)

48 November 8th, 2002School of Electrical Engineering and Computer Science48 Software Simulation Result (Cont’d)

49 November 8th, 2002School of Electrical Engineering and Computer Science49 Software Simulation Result (Cont’d) DataClassifie d as Class 1Class 2Class 3Class 4Class 5 Data from Class 1 0.817100.00400.17690.0020 Data from Class 2 00.99770.002300 Data from Class 3 000.926300.0737 Data from Class 4 0.14810.010300.84160 Data from Class 5 00001

50 November 8th, 2002School of Electrical Engineering and Computer Science50 Credit Card Approval Cal50.131 SOLAR0.1333 Itrule0.137 Discrim0.141 Logdisc0.141 DIPOL920.141 CART0.145 RBF0.145 CASTLE0.148 NaiveBay0.151 IndCART0.152 Backprop0.154 C4.50.155 SMART0.158 Baytree0.171 k-NN0.181 NewID0.181 AC 2 0.181 LVQ0.197 ALLOC800.201 CN20.204 Quadisc0.207 Default0.440 KohonenFailed

51 November 8th, 2002School of Electrical Engineering and Computer Science51 Adult Income FSS Naïve Bayes0.1405 NBTree0.1410 C4.5-auto0.1446 IDTM (Decision table)0.1446 HOODG / SOLAR0.1482 C4.5 rules0.1494 OC10.1504 C4.50.1554 Voted ID3 (0.6)0.1564 CN20.1600 Naïve-Bayes0.1612 Voted ID3 (0.8)0.1647 T20.1687 1R0.1954 Nearest-neighbor (3)0.2035 Nearest-neighbor (1)0.2142 PeblsCrashed

52 November 8th, 2002School of Electrical Engineering and Computer Science52 Conclusion and Future Work  Conclusion  Local learning  Self-organizing  Data preprocessing  Future work  VHDL simulation  FPGA machine  VLSI design

53 November 8th, 2002School of Electrical Engineering and Computer Science53 Reference  Fraser, N. (1998, September), The Biological Neuron, Avaiable http://vv.carleton.ca/~neil/neural/neuron-a.html http://vv.carleton.ca/~neil/neural/neuron-a.html  Information & Computer Science (ICS), University of California at Irvine (UCI). (1995, December), Machine Learning Repository, Available FTP: Hostname: ftp.ics.uci.edu Directory: /pub/machine-learning- databases/

54 November 8th, 2002School of Electrical Engineering and Computer Science54 Thank You  Thank you for your attention  Question


Download ppt "Future Hardware Realization of Self-Organizing Learning Array and Its Software Simulation Adviser: Dr. Janusz Starzyk Student: Tsun-Ho Liu Ohio University."

Similar presentations


Ads by Google