1. Machine Learning A Quick look Sources:
Artificial Intelligence – Russell & Norvig
Artifical Intelligence - Luger
By: Héctor Muñoz-Avila

2. What Is Machine Learning?
“Logic is not the end of wisdom, it is just the beginning” --- Spock
time
Environment
Knowledge
Environment
System
System
Action1
Action2
Knowledge
changed

3. Learning: The Big Picture
Two forms of learning:
Supervised: the input and output of the learning component can be perceived (for example: friendly teacher)
Unsupervised: there is no hint about the correct answers of the learning component (for example to find clusters of data)

4. Offline vs. Online Learning
11-Apr-17
Offline vs. Online Learning
Online – during gameplay
Adapt to player tactics
Avoid repetition of mistakes
Requirements: computationally cheap, effective, robust, fast learning (Spronck 2004)
Offline - before the game is released
Devise new tactics
Discover exploits
Two different types of learning in games: online and offline:
Online learning takes place during gameplay against human players. The main applications for online learning are to adapt to player tactics and playing styles and avoid the repetition of mistakes. However, online learning has rarely been applied in commercial games because most game companies are not comfortable with shipping their games with an online learning technique included in the game. What if the NPC learns something stupid? What if the learning takes too long or is too computational expensive? Therefore we set a couple of requirements for online learning:
Computationally cheap: Should not disturb flow of gameplay.
Effective: Should not generate too many bad inferior opponents.
Robust: Should be able to deal with randomness incorporated in games. For instance you don’t want permanently unlearn a specific behavior because it performed badly at some point. The reason for the bad performance could be subscribed to chance.
Fast learning: Should lead to results quickly. Don’t want to bore the players with slow learning AI.
Offline learning can be applied before a games is being released. Typically, offline learning is used to explore new game tactics or to find exploits.

5. Classification (According to the language representation)
Symbolic
Version Space
Decision Trees
Explanation-Based Learning …
Sub-symbolic
Reinforcement Learning
Connectionist
Evolutionary

6. Classification (According to the language representation)
Symbolic
Version Space
Decision Trees
Explanation-Based Learning …
Sub-symbolic
Reinforcement Learning
Connectionist
Evolutionary

7. Version Space
Idea: Learn a concept from a group of instances, some positive and some negative
Example:
target: obj(Size,Color,Shape)
Size = {large, small}
Color = {red, white, blue}
Shape = {ball, brick, cube}
Instances: +:
obj(large,white,ball)
obj(small,blue,ball) −:
obj(small,red,brick)
obj(large,blue,cube)
Two extremes (temptative) solutions:
obj(X,Y,Z)
obj(large,white,ball) obj(small,blue,ball) …
too general
too specific
concept space
obj(large,Y,ball)
obj(small,Y,ball)
obj(X,Y,ball) …

8. How Version Space Works
If we consider only positives
If we consider positive and negatives + + + + + + + + + + + + + + − − − − + +
What is the role of the negative instances?
to help prevent over-generalizations

9. Classification (According to the language representation)
Symbolic
Version Space
Decision Trees
Explanation-Based Learning …
Sub-symbolic
Reinforcement Learning
Connectionist
Evolutionary

10. Explanation-Based learningC A B C A B C B A B A C ? B A ? C ? B A B C C B A B A C A C A A B C B C C B A A B C
Can we avoid making this error again?

11. Explanation-Based learning (2)C A B ? B A ? C ? A B C C B A B
Possible rule: If the initial state is this and the final state is this,
don’t do that A C
More sensible rule: don’t stack anything above a block, if the block has to be free in the final state

12. Classification (According to the language representation)
Symbolic
Version Space
Decision Trees
Explanation-Based Learning …
Sub-symbolic
Reinforcement Learning
Connectionist
Evolutionary

13. Motivation # 1: Analysis Tool
Suppose that a company have a data base of sales data, lots of sales data
How can that company’s CEO use this data to figure out an effective sales strategy
Safeway, Giant, etc cards: what is that for?

14. Motivation # 1: Analysis Tool (cont’d)
Sales data
induction
Decision Tree
Ex’ple
Bar
Fri
Hun
Pat
Type
Res
wait x1 no
yes
some
french x4
full
thai x5 x6 x7 x8 x9
x10
x11
“if buyer is male & and age between & married
then he buys sport magazines”

15. The Knowledge Base in Expert Systems
A knowledge base consists of a collection of IF-THEN rules:
if buyer is male & age between & married
then he buys sport magazines
if buyer is male & age between 18-30
then he buys PC games magazines
Knowledge bases of fielded expert systems contain hundreds and sometimes even thousands such rules. Frequently rules are contradictory and/or overlap

16. Main Drawback of Expert Systems: The Knowledge Acquisition Bottle-Neck
The main problem of expert systems is acquiring knowledge from human specialist is a difficult, cumbersome and long activity.
Name KB
#Rules
Const. time
(man/years)
Maint. time
MYCIN KA
500 10
N/A
XCON
2500 18 3
KB = Knowledge Base
KA = Knowledge Acquisition

17. Motivation # 2: Avoid Knowledge Acquisition Bottle-Neck
GASOIL is an expert system for designing gas/oil separation systems stationed of-shore
The design depends on multiple factors including:
proportions of gas, oil and water, flow rate, pressure, density, viscosity, temperature and others
To build that system by hand would had taken 10 person years
It took only 3 person-months by using inductive learning!
GASOIL saved BP millions of dollars

18. Motivation # 2 : Avoid Knowledge Acquisition Bottle-Neck
Name KB
#Rules
Const. time
(man/years)
Maint. time
(man/months)
MYCIN KA
500 10
N/A
XCON
2500 18 3
GASOIL
IDT
2800 1
0.1
BMT
KA (IDT)
30000+
9 (0.3)
2 (0.1)
KB = Knowledge Base
KA = Knowledge Acquisition
IDT = Induced Decision Trees

19. Example of a Decision Tree
Patrons?
waitEstimate? no
yes
0-10
>60
Full no
yes
none
some
Alternate?
Reservation?
Yes
30-60 no
yes
Hungry?
yes No
10-30
Alternate?
Yes no
Raining?
Fri/Sat? No
Yes
yes no No no
Bar?
Yes
yes

20. Definition of A Decision Tree
A decision tree is a tree where:
The leaves are labeled with classifications (if the classification is “yes” or “no”. The tree is called a boolean tree)
The non-leaves nodes are labeled with attributes
The arcs out of a node labeled with an attribute A are labeled with the possible values of the attribute A

21. Induction Databases: what are the data that matches this pattern?
Induction: what is the pattern that matches these data?
induction
Data
Ex’ple
Bar
Fri
Hun
Pat
Type
Res
wait x1 no
yes
some
french x4
full
thai x5 x6 x7 x8 x9
x10
x11
pattern

22. Induction of Decision Trees
Objective: find a concise decision tree that agrees with the examples
The guiding principle we are going to use is the Ockham’s razor principle: the most likely hypothesis is the simplest one that is consistent with the examples
Problem: finding the smallest decision tree is NP-complete
However, with simple heuristics we can find a small decision tree (approximations)

23. Induction of Decision Trees: Algorithm
Initially all examples are in the same group
Select the attribute that makes the most difference (i.e., for each of the values of the attribute most of the examples are either positive or negative)
Group the examples according to each value for the selected attribute
Repeat 1 within each group (recursive call)

24. Example Ex’ple Bar Fri Hun Pat Alt Type wait x1 no yes some French x4
full
Thai x5 x6
Italian x7
none
Burger x8 x9
x10
x11 No

25. IDT: Example
Lets compare two candidate attributes: Patrons and Type. Which is a better attribute?
Patrons?
none
X7(-),x11(-)
some
X1(+),x3(+),x6(+),x8(+)
full
X4(+),x12(+),
x2(-),x5(-),x9(-),x10(-)
Type?
french
X1(+),
x5(-)
italian
X6(+),
x10(-)
burger
X3(+),x12(+),
x7(-),x9(-)
X4(+),x12(+)
x2(-),x11(-)
thai

26. Example of a Decision Tree
Patrons?
waitEstimate? no
yes
0-10
>60
Full no
yes
none
some
Alternate?
Reservation?
Yes
30-60 no
yes
Hungry?
yes No
10-30
Alternate?
Yes no
Raining?
Fri/Sat? No
Yes
yes no No no
Bar?
Yes
yes

27. Decision Trees in Gaming
Black & White, developed by Lionhead Studios, and released in 2001 used ID3
Used to predict a player’s reaction to a certain creature’s action
In this model, a greater feedback value means the creature should attack

28. Decision Trees in Black & White
Example
Attributes
Target
Allegiance
Defense
Tribe
Feedback D1
Friendly
Weak
Celtic
-1.0 D2
Enemy
0.4 D3
Strong
Norse D4
-0.2 D5
Greek D6
Medium
0.2 D7
-0.4 D8
Aztec
0.0 D9

29. Decision Trees in Black & White
Allegiance
Friendly
Enemy
Defense
-1.0
Weak
Strong
Medium
0.4
-0.3
0.1
Note that this decision tree does not even use the tribe attribute

30. Decision Trees in Black & White
Now suppose we don’t want the entire decision tree, but we just want the 2 highest feedback values
We can create a Boolean expressions, such as ((Allegiance = Enemy) ^ (Defense = Weak)) v ((Allegiance = Enemy) ^ (Defense = Medium))

31. Classification (According to the language representation)
Symbolic
Version Space
Decision Trees
Explanation-Based Learning …
Sub-symbolic
Reinforcement Learning
Connectionist
Evolutionary