1. Parallel Implementation of a Biologically Realistic NeoCortical Simulator E.Courtenay Wilson

2. Overview Biological Topology Biological Topology Computational Topology Computational Topology Parallel Implementation Parallel Implementation Results Results Conclusions Conclusions Future Work Future Work

3. Biological Topology Columns Columns  High connectivity within columns.  Less connectivity across columns

4. Biological Topology Neurons Neurons  Excitatory  Interneurons (inhibitory)

5. Biological Topology Channels Channels  Potassium Family  M, A, AHP Channels  Suppressing behavior on parent cell

6. Biological Topology Synapses Synapses  Analog converter of binary spike event.  Contextual filters.

7. Computational Topology Object Oriented Design Object Oriented Design Multi-Compartment Model. Multi-Compartment Model.  Encapsulation of biological functionality.  Containers and hierarchical ownership.

8. Computational Topology Cell and Compartment model Cell and Compartment model  Cell is container for Compartment objects  Compartment is container for: Synapses, Channels, Spike Shapes dendrite Synapse Axon Soma dendrite Cell

9. Computational Topology Synapses Synapses  Encapsulation of Short Term Dynamics and Hebbian Learning  Quantities are determined by connectivity within system  Triggered State Machine Pre-Synaptic incoming spike event (Short Term Dynamics)Pre-Synaptic incoming spike event (Short Term Dynamics) Post-Synaptic outgoing spike event (Hebbian Learning)Post-Synaptic outgoing spike event (Hebbian Learning)

10. Messages Messages  Used for communicating data and information across cells  Direct correlation between number of Synapses and Number of Messages  Packaged in Message Passing Interface (MPI) format for inter-nodal communication.  Stored in a memory pool for faster performance Computational Topology

11. Parallel Implementation Message Bus Object Message Bus Object  Encapsulates all inter-nodal communication.  Encapsulation allows easy swap out for different hardware/software architectures Message Bus File I/O Stimulus Reports Brain Cell/Comp Container Interconnection Network Message Bus (other node)

12. Cell/Compartment Communication: Cell/Compartment Communication:  Local list of SendingTo & ReceivingFrom Cell/Compartment  When Action Potential is generated a Message is generated, envelope consists of SendingTo address Parallel Implementation

13. Parallel Implementation : Global Communication Flow Messages Message Bus Comp 1 spike Interconnection Network Message Bus (other node) Cell 2 Cell n Synapse 2 Synapse 1 Hebbian Learning

14. Parallel Implementation : Cell/Compartment Communication Flow Comp 1 Message Bus Cell n Comp 2 Synapse 1 Synapse 2 Synapse 3 Synapse 4 Synapse 5 Short Term Dynamics

15. Scaling: Scaling:  Limiting factor is memory and interconnect speed.  Depends on connectivity - greater the connectivity the greater the number of synapses and messages.  With 4GB of RAM/node, ~10 Million synapses per node is possible. Parallel Implementation

16. Myrinet: Myrinet:  Clos network with cut-through routing  High Bandwidth (1.92Gbits/sec)  Low Latency ( 9micro seconds)  Low latency reduces minimizes the effect of fine grain parallelization in a large network. Parallel Implementation

17. Clos network with 128 hosts Parallel Implementation

18. Myrinet switch line card Parallel Implementation

19. Myrinet PC card Parallel Implementation

20. Results : Performance : Memory Use Connectivity is the deciding factor Connectivity is the deciding factor  Higher the connections, the more Synapses: the more memory used.  With 4GB of RAM, max # synapses/node = ~10 million

21. Ethernet latency is bottleneck Ethernet latency is bottleneck Used N-Squared connection scheme Used N-Squared connection scheme Simulation Simulation  Average of 1 million synapses per node  0.1 second simulation  2.3 hrs total  1.66 hrs synchronizing nodes Results : Performance : Ethernet Cell 1 Cell 2 Cell 4 Cell 3

22. ~123,000 Cells Across 24 nodes ~123,000 Cells Across 24 nodes  varying connectivity (0.01%, 0.1%, 1.0%) Results : Performance : Ethernet

23. Results : Auditory Processing: “About” Auditory Signal (decibels) Cell Spike Response (mVolts)

24. Results : Auditory Processing: “About” Density Graph

25. Results : Synaptic Dynamics : Short Term Depression FacilitationDepression & Facilitation None

26. Results : Synaptic Dynamics : Hebbian Learning Negative Hebbian Positive Hebbian Positive & Negative Hebbian None

27. Results : Membrane Dynamics : Channels No Channels Bursting Accommodating Classic Accommodating

28. Results : Membrane Dynamics : Channels Classic Non-Accommodating Classic Stuttering Delayed Non- Accommodating

29. Conclusions 45 Objects total 45 Objects total Over 27,000 lines of code Over 27,000 lines of code Runs sequentially and in parallel Runs sequentially and in parallel Generic objects allow us to model other parts of brain Generic objects allow us to model other parts of brain Encapsulation allows us to test different paradigms more easily Encapsulation allows us to test different paradigms more easily

30. Future Work Biological Neural Network for novel auditory classification Biological Neural Network for novel auditory classification Multiple Compartment Model Multiple Compartment Model Fuzzy Clustering and Distribution of Cells Fuzzy Clustering and Distribution of Cells