1. NVIS: An Interactive Visualization Tool for Neural Networks Matt Streeter Prof. Matthew O. Ward by Prof. Sergio A. Alvarez advised by and

2. What is a Neural Network? Weighted, directed graph, organized into layers Set of neurons (nodes) and synapses (edges), with signals transmitted between neurons via synapses Valuable tool for pattern recognition and function approximation

3. Why create a visualization tool for neural networks? Understand how neural networks work, gain insight into problem being solved Understand how genetic algorithm evolves networks Other tools exist, but do not show neuron activations or genealogical relationships

4. Feedforward network visualization Synapse strength represented by length and brightness of colored bars (linear scale). Blue lines indicate positive weights; red lines indicate negative Diameter of white circles represents neuron’s output or activation Each weight acts as a slider

5. Compact matrix representation Purpose is to allow many networks to be displayed on the screen at once One matrix for each level of weights Row x, column y of matrix n represents weight from node y of layer n to node x of layer n+1 (same colors)

6. Generations & family trees Row of compact matrix for each generation, ordered by fitness User can select any network in the population history Separate window shows family tree of selected network

7. Interactive environment Set evolution strategy, network architecture, and training set Graph representation and family tree available for any network in population history Load/save networks Real-time fitness graph

8. Designing networks By dragging weights, user can design a network to solve a problem, or refine a network that has already been trained Real-time display of fitness score; easy to see importance of particular weight Not a practical way to find a network to solve a problem

9. Understanding genetic drift Genetic drift is tendency for members of artificial populations to all be alike Initial diversity in generations 0-2, rapidly lost in generations 3-5 Best (leftmost) network in generation 3 is parent of best network in generation 4, grandparent of best 8 in generation 5, and ancestor of all later networks (not shown)

10. Changing weights & local optima Error backpropagation algorithm performs gradient-based search (local optimum) Weight dragged while backprop is running will either “snap back” to original optimum, or all weights will shift to new optimum Can estimate the length of a local optimum with respect to each axis in weight-space

11. Extracting domain knowledge Positive weights in all but first layer; effect of input nodes therefore directly related to incident weights Higher crime rates (C) tend to reduce value of house; higher number of rooms (R) tends to increase value Analysis could be applied to problem domains where no a priori knowledge exists

12. Future work Graph representation does not scale well Implement a variety of evolutionary algorithms (breeding & selection schemes) Depict network architectures other than feedforward User evaluation

13. For more information... Visit http://www.wpi.edu/~mjs/mqp See technical report WPI-CS-TR-00-11, available at: http://www.cs.wpi.edu/Resources/techreports/index.html Questions?