Exploring Machine Learning in Computer Games Presented by: Matthew Hayden Thurs, 25 th March 2010

Introduction - Games Games are important …for fun …for learning (coincidentally?) What are games? …(arguably) simulations of reality There are rules and restrictions …don't have all the repercussions (i.e. Simulation) Cheap to play (good for us) Most importantly, fun! Computer games are big business

Introduction – Games (2) Computers play games (against humans) Board games …'brute force' calculation of the cost of a move …often 'zero-sum' minimax algorithm …estimate (sometimes solve) the next optimal move 'Noughts and Crosses' or 'Tic Tac Toe' is an example of a popular 'zero-sum' game that has been solved.

Classical Artificial Intelligence More complex/realistic simulations require heuristics for intelligence, including... Rule based systems (easy to exploit) Path finding algorithms (cannot adapt) Finite state automata (imperative fast, but stupid) Planning (more robust, declarative, but slow) Could completely remove all need Scripted sequences...but we want to make the game more interesting!

Classical Artificial Intelligence (2)

ML in Games – Path finding Path finding in dynamic landscapes Changes constantly adaptive algorithms Ant Colony Optimisation algorithm Finding optima quickly Evolving optimal racing lines (rally games) Genotype Decision trees Phenotype Ability driving round a track Evaluation Make n decisions a second Selection Faster more likely to be selected

ML in Games - Adaptation Games should adapt to be personalised Different players have their own styles of play Players get better, need smooth difficulty curve …we could fake it More complex AI to anticipate more behaviours Ramp up the difficulty more gradually …or we could use Machine Learning On-line training algorithms No fixed difficulty curve

ML in Games – Adaptation (2) Case study: Left 4 Dead – Dynamic AI Director Player-opponent interaction as well as context No fixed narrative, generated procedurally Collects data from user performance/physiology and construct a player model Procedural play lots of of games offline for validation Cleverly adapted the current models into a Machine Learning context Zombies! Many data points