1. Neural Networks Slides by Megan Vasta

2. Neural Networks Biological approach to AI Developed in 1943 Comprised of one or more layers of neurons Several types, we’ll focus on feed-forward networks

3. Neurons http://faculty.washington.edu/chudler/color/pic1an.gif http://research.yale.edu/ysm/images/78.2/articles-neural-neuron.jpg Biologica l Artificial

4. Neural Network Neurons Receives n-inputs Multiplies each input by its weight Applies activation function to the sum of results Outputs result http://www-cse.uta.edu/~cook/ai1/lectures/figures/neuron.jpg

5. Activation Functions Controls when unit is “active” or “inactive” Threshold function outputs 1 when input is positive and 0 otherwise Sigmoid function = 1 / (1 + e -x )

6. Neural Network Layers Each layer receives its inputs from the previous layer and forwards its outputs to the next layer http://smig.usgs.gov/SMIG/features_0902/tualatin_ann.fig3.gif

7. Simple Example Application Smart Sweepers Red mine sweepers are best generated so far Hit ‘F’ key to switch between displays Every generation changes neural net weights by a genetic algorithm

8. Neural Network Learning Back-Propagation Algorithm: function BACK-PROP-LEARNING(examples, network) returns a neural network inputs: examples, a set of examples, each with input vector x and output vector y network, a multilayer network with L layers, weights W j,i, activation function g repeat for each e in examples do for each node j in the input layer do a j  x j [e] for l = 2 to M do in i   j W j,i a j a i   g(in i ) for each node i in the output layer do  j  g ’ (in j )  i W ji  i for l = M – 1 to 1 do for each node j in layer l do  j  g’(in j )  i W j,i  i for each node i in layer l + 1 do W j,i  W j,i +  x a j x  i until some stopping criterion is satisfied return NEURAL-NET-HYPOTHESIS(network) [Russell, Norvig] Fig. 20.25 Pg. 746

9. Back-Propagation Illustration ARTIFICIAL NEURAL NETWORKS Colin Fahey's Guide (Book CD)

10. Neural Networks and Game AI simplify coding of complex state machines or rules-based systems potential for AI to adapt as the game is played “I am very confident about it as well, because I understand what the neural net is doing. I haven't just created a big mysterious black box, I can map out the internal workings of it.” Jeff Hannan, creator of AI for Colin McRae Rally 2.0 http://www.generation5.org/content/2001/hannan.asp

11. Colin McRae Rally 2.0 Used standard feedforward multilayer perceptron neural network Constructed with the simple aim of keeping the car to the racing line http://www.generation5.org/content/2001/hannan.asp

12. BATTLECRUISER: 3000AD 1998 space simulation game 'I am a Gammulan Criminal up against a Terran EarthCOM ship. My ship is 50% damaged but my weapon systems are fully functional and my goal is still unresolved' what do I do? http://www.gameai.com/games.html http://www.3000ad.com/shots/bc3k.shtml

13. Black and White Uses neural networks to teach the creature behaviors “Tickling” increase certain weights Hitting reduces certain weights

14. Usefulness in Games Control – motor controller, for example, control of a race car or airplane Threat Assessment – input number of enemy ground units, aerial units, etc. Attack or Flee – input health, distance to enemy, class of enemy, etc. Anticipation – Predicting player’s next move, input previous moves, output prediction http://www.onlamp.com/pub/a/onlamp/2004/09/30/AIforGameDev.html

15. Influence Maps Refresher From section 7.1 Creates a spatial database of game world info Static or dynamic Openness Layer Occupancy Layer Static Cover Layer Desirability Layer

16. Neural Networks and Influence Maps Usually influence map layers are combined using a weighted sum Neural network can be used instead Finding correct weight for each layer usually results in trial and error Choosing relevant layers can be difficult Potential loss of important information if factors cancel each other

17. Computational Complexity Could lead to a very large number of calculations Influence Map Layer 1 Influence Map Layer 2 Hidden Units Output Units Input Units

18. Optimizations Only analyze relevant portion of map Reduce grid resolution Train network during development, not in-game

19. Design Details Need an input for each cell in each layer of the influence map Need one output for each cell on map Hidden units are arbitrary, usually 10-20 with some guess and test to prune it

20. Different Decisions and Personalities One network for each decision Implemented as one network with a different array of weights for each decision Different personalities can have multiple arrays of weights for each decision

21. Training Datasets from gaming sessions of human vs. human are best Must decide whether training will occur in-game, during development, or both Learning during play provides for adaptations against individual players

22. Conclusions Neural networks provide ability to provide more human-like AI Takes rough approximation and hard-coded reactions out of AI design (i.e. Rules and FSMs) Still require a lot of fine-tuning during development

23. References Four Cool Ways to Use Neural Networks in Games Interview with Jeff Hannan, creator of AI for Colin McRae Rally 2.0Interview with Jeff Hannan Interview with Derek Smart, creator of AI for Battlecruiser: 3000ADInterview with Derek Smart Neural Netware, a tutorial on neural networksNeural Netware Sweetser, Penny. “Strategic Decision-Making with Neural Networks and Influence Maps”, AI Game Programming Wisdom 2, Section 7.7 (439 – 46) Russell, Stuart and Norvig, Peter. Artificial Intelligence: A Modern Approach, Section 20.5 (736 – 48)