Artificial Neural Network Theory and Application Ashish Venugopal Sriram Gollapalli Ulas Bardak

Presentation Overview Artificial Neural Networks NASA profiles as a Pattern Classification Task Implementing Pattern Classification on the Silicon Recognition Neuron Board

ANN General Overview Inspired by biological neuron models Decision are distributed throughout system Components interacts at several stages Components work in unison to solve one problem ANN well suited to problems that don’t have closed form algorithmic solution Black box model – hard to interpret

ANNs Applied Driving a car. Task involves determining the direction the road is curving based on an image Image pixels are input Curve angle is output Learned non-linear relationship between pixels and roads hard to understand the model

Theory Input layer, Hidden layer, Output Layer Input – one “neuron” for each pixel Hidden – set of neurons to store what was learned. Output layer – one neuron for each of 5 turn directions – ie straight, slight left, hard right.. But what is a neuron?

Neurons Math terms: Binary decision makers English terms: take in some input, decide whether is “high” or “low”, and pass this value on. Similar to neurons in the brain propagating signals.

How is the model learned? Once you have training examples Pair (feature set, classification) Run the back propagation algorithms Learns connection weights between neurons, as well as within neuron transfer weights. Network topology changes affects learning behavior!

Unsupervised Networks So far we have used labeled training examples to learn relationship between features and response variable. What if we are just exploring patterns that exist in the features? Unsupervised learning / clustering…

Unsupervised Goal Find prototypical points within data set that can be used to approximate distribution of the whole data set. Compression / Visualization Typical methods Define a measure of similarity between data points Select initial prototypes Move prototypes to best fit the data – minimize error

Kohonen Net Unsupervised technique Each output neuron represents a prototype point, input layer is presented with data

Self Organizing Map Each prototype is pulled towards that data, and brings its closest prototypes a little closer as well… SOM Demo

How is it relevant to our problem? We will go through: Definition of what we are facing How we can use neural nets How we can improve on this method

LIDAR in Space Data from LITE project Send a laser beam down from a shuttle and get reflection readings. Data gathered for 53 hours in GB of data

Input

Problem There is too much data to send and process Each pixel is a number to be transmitted. Each reading (one column) contains 3000 pixels. Need a way to transmit the information in a more compact fashion.

Applying Neural Nets Some readings are very similar to each other. Define classes that will contain multiple readings. Define a representative for the class which is close enough to all the class members.

Applying Neural Nets (cont.) In order to get class Train Kohonen Nets on data for specific number of classes. Once trained, just pass in a new reading (3000 data points) and get the class id. Just transfer the class id. Go from 3000x3000 numbers to be transferred to 3000 numbers

Example

Result – 8 classes

Result – 16 Classes

Result – 64 Classes

Result – 128 Classes

Result – 256 Classes

Problems Takes a while to train 256 classes took 44 hours on a dual P3 1GHz computer with 1.5GB of RAM. Not optimal Classifying by the complete reading is wasteful. Need one number for each reading

Improving NN sky

Improving NN (cont.) Easier version: Look at blocks of 50x50 pixels instead of just 1 pixel. Classify as sky, etc. Can preprocess using kNN, basic filters to get rid of noise in data. Classes can be formed from sequence of 50 pixel blocks.

Improving NN (cont.) Harder version: Use variable sized mesh In parts with just the sky use a big block. In detailed parts use smaller blocks. Something to think about…

Muren Board Silicon Recognition, Inc. 2 ZISC078 Chips (156 neurons in parallel) 1 Mb of memory Up to 1 million recognitions/second

ZISC ZISC, the cornerstone of the Muren board, completely overcomes the three major limitations of software-based pattern recognition applications: ZISC fully handles non-linearity and fuzziness ZISC is a fully massively parallel processing system. Recognition speed is consistent, regardless of the number of stored patterns ZISC does not use any algorithms for programming the solution, ZISC is taught

RBF Space mapping

Example Source picture (circuit board) Result (after processing) (shows good and bad points)

Process Learning Classifying Training Testing with new data

NASA Data

Further investigation Can we write directly to the board? Ability to preprocess data?

Questions?