Parallel Artificial Neural Networks Ian Wesley-Smith Frameworks Division Center for Computation and Technology Louisiana State University

Basics of ANNs Vague model of biological neural network No authoritative definition for ANNs –Group of small computational components (neurons) networked together

Strengths of ANNs –Inherently Non-Linear –Learn Supervised Input-Output Mapping –Simple enough to implement in hardware Good at: –Optimization problems (traveling salesman) –Pattern classification (handwriting analysis)

Examples of ANNs Digital Signal Processing (DSP) Optical Character Recognition (OCR) Sales Forecasting Industrial Process Control SONAR/RADAR Medical Assessment Games

Examples of ANNs Robot Army

Components of Neurons Inputs –Vector Weights –Matrix [inputs x outputs] Output (activation) Function –Threshold –Sigmoid

Example Neuron x1x1 x2x2 xnxn  f(x) Weights Inputs Output Function A Single Neuron

Perceptrons Simplest ANN Single Layer Single Neuron –Can be more Simple pattern classifiers –Only classify linearly separable sets Learning with the delta-rule Output function is threshold

Sample Data

Perceptrons Computation x1x1 x2x2 xnxn  f(x) Weights Inputs Output Function A Single Neuron

Perceptrons Delta-Learning

Methodology Implicit parallelism Neurons are independent of one another Calculations are relatively simple Process large datasets faster Implementation –Serial to Parallel Implementation –PETSc Portable Extensible Toolkit for Scientific Computing –Run Details AMD Dual Opteron 2 Processor Run Varying Sized Data Sets ( million)

Error Function

Other Types of ANNs ● Multi-layered Perceptron ● Capable of solving more complex problems (XOR) ● Backpropagation Network ● Speech analysis ● Hopfield Network ● Pattern association ● Optimization problems ● Kohonen Feature Map ● Learning most closely related to biological learning

Results Parallel ANN was functional Parallel implementation performs slower than serial –This is expected Possible Reasons –Single Neuron Problem –PETSc/MPI Overhead

Future Work Implement more advanced (recurrent) networks Hand code MPI instead of relying on PETSc Test in larger environments –32 processors minimum

Acknowledgments Yaakoub Y. El-Khamra Dr. Gabrielle Allen Dr. Ed Seidel Kathy Traxler Louisiana State University –Center for Computation and Technology –Computer Science Department