ICCINC' Janusz Starzyk, Yongtao Guo and Zhineng Zhu Ohio University, Athens, OH 45701, U.S.A. 6 th International Conference on Computational Intelligence and Neural Computing Cary, NC, September 30 th, 2003

ICCINC' OUTLINE  Neural Networks  Traditional Hardware Implementation  Principle of Self-Organizing Learning  Advantages & Simulation Algorithm  Hardware Architecture  Hardware/software Codesign  Routing and Interface  PCB SOLAR  Future Work  Conclusion

ICCINC' Traditional ANN Hardware –Limited routing resource. –Quadratic relationship between the routing and the number of neuron makes classical ANNs wire dominated. input output information flow hidden Interconnect is 70% of chip area

ICCINC' Biological Neural Networks Biological Neural Networks Cell body From IFC’s webpage Dowling, 1998, p. 17

ICCINC' What is SOLAR?What is SOLAR? New Biologically Inspired Learning Network Organization Basic Fabric: A fixed lattice of distributed, parallel processing units (neurons) Self-organization: N eurons chose inputs adaptively from routing channels. N eurons chose inputs adaptively from routing channels. Neurons are adaptively self re-configured. Neurons are adaptively self re-configured. Neurons send output signals to the routing channels. Neurons send output signals to the routing channels. Number of neurons results automatically from problem complexity. Number of neurons results automatically from problem complexity. Self Organizing Learning Array SOLAR

ICCINC' Self Organizing Learning Array SOLAR-Organization  Neurons organized in a cell array  Sparse randomized connections  Local self-organization  Data driven  Entropy based learning  Regular structure  Suitable for large scale circuit implementation

ICCINC' Neuron’s Simulation Structure Neuron Inputs Neuron Inputs –System clock –Data input –Control input TCI –Information deficiency ID Other Neurons This neuron System clock Nearest neighbor neuron Remote neurons TCITCI IDID Neuron Outputs -Data output -Control output -Information deficiency

ICCINC' Self-Organizing Process

ICCINC' Self-organizing Principle Information index Neuron self-organizes by maximizing the information index

ICCINC' Self-organizing Principle Output information deficiency. Information deficiency (helps to organize SOLAR learning) The learning array grows by adding more neurons until input information deficiency of a subsequent neuron falls below threshold

Self-organizing Process Matlab Simulation Initial interconnection Learning process

ICCINC' Software Simulation Training Data SOLAR & other Algorithms Credit card approval data (ftp:cs.uci.edu) SOLAR & other Classifiers (Simulation) MethodMiss Detection Probability MethodMiss Detection Probability CAL5.131Naivebay.151 DIPOL92.141CASTLE.148 Logdisc.141ALLOC SMART.158CART.145 C NewID.181 IndCART.152CN2.204 Bprop.154LVQ.197 RBF.145Quadisc.207 Baytree.171Default.440 ITule.137k-NN.181 AC2.181SOLAR.135

ICCINC' Structure of a single neuron RPU: reconfigurable processing unit CU: control unit DPE: dynamic probability estimator EBE: entropy based evaluator DSRU: dynamic self-reconfiguration memory. NI/NO: Data input/output CI/CO: Control input/output

ICCINC' Routing Structure –CSU:configurable switching unit –BRU: bidirectional routing unit

ICCINC' Configurable Switching Unit (CSU) CSU is used to realize flexible connections among neurons –Butterfly structure –CSU can take any number of inputs Even number of inputs Odd number of inputs

ICCINC' Configurable Switching Unit(cont’d) Random connections of neurons with branching ratio of 50% for 3*6 and 6*15 neurons array Routing resources used 62.7%Routing resources used 85.3%

ICCINC' Branching Ratio of 10%Branching Ratio of 90% Random connections of 4*7 neurons array with branching ratio of 10% and 90% Configurable Switching Unit(cont’d)

ICCINC' HW/SW Codesign Partition of System Co-simulation  Neuron’s architecture  System initialization, organization and management  Interface JTAG Programming Software run in PC PCI Bus Hardware Board Virtex XCV800FPGA dynamic configuration

ICCINC' Software Model In Behavioural VHDL Hardware Model In Structural VHDL SW/HW Co-simulation A software process –Written in behavioral VHDL A hardware process –Written in RTL VHDL which is synthesizable HW/SW communication –FSM and FIFOs

ICCINC' Hardware Architecture

ICCINC' Software Architecture System Design Data I/O API PCI FUNC Kernel Driver Ctrl I/O API Sys Func Hardware Access Function Data I/O matDIME_DMARead.dll matDIME_DMAWrite.dll matviDIME_ReadRegister.dll matviDIME_WriteRegister.dll … Ctrl I/O matCloseDIMEBoard.dll matConfigDIMEBoard.dll matOpenDIMEBoard.dll … PCI BUS

ICCINC' PCB Design Single SOLAR PCB contains 2x2 VIRTEX XCV1000 chips

ICCINC' SOLAR PCB Design Boards Interface BoardSOLAR Board

ICCINC' Neurons Prototyping Problem: Neurons need to be carefully placed - otherwise some resources are lost. Neurons memory needs to be optimized for best resource utilization.

ICCINC' Future Work - System SOLAR

ICCINC' SOLAR is different from traditional neural networks …  Expandable modular architecture  Dynamically reconfigurable hardware structure  Interconnection number grows linearly with the number of neurons  Data-driven self-organizing learning hardware  Learning and organization is based on local information

ICCINC' Why to focus on networks of neurons?  Increases computational speed  Improves fault tolerance  Constraints us to use distributed solutions  Brain does it 

ICCINC' Can we set milestones in developing intelligent networks of neurons? How to represent a distributed cognition? How to model machine will to learn and act? How to introduce association between patterns? How a machine shell implement temporal learning? How machine shell block repetitive information from being processed over and over again? How machine shell evaluate its state with respect to set objectives and plan its actions? How to implement elements of reinforcement learning in distributed networks?

ICCINC' Questions