1. Learning: Introduction and Overview
Chapter also: AI 2nd ed, Rich and Knight, chapter 17
COMP151 April 13, 2007

2. What is Learning?
The act, process, or experience of gaining knowledge or skill.
Psychology. Behavioral modification especially through experience or conditioning. American Heritage Dictionary

3. What is Learning?
Learning denotes “changes in the system that are adaptive in the sense that they enable the system to do the same task or tasks drawn from the same population more efficiently and more effectively the next time” Herbert A. Simon, 83
Agents can improve their performance through diligent study of their own experiences Russell & Norvig

4. What is Learning?
Working definition for our discussions: Learning is self-modification that results in beneficial changes to behavior.
Can machines learn?
If you program a computer, has it learned anything?
If you add data to a database, is it learning?

5. Methods of Learning Rote learning: Learning by taking advice:
simple storage of computed information
Learning by taking advice:
similar to rote learning, but advice may need to be operationalized
Learning from problem-solving experience:
remembering effective structures and methods
Learning from examples:
usually involves a teacher who helps to classify things

6. Why Should Machines Learn?
Learning is essential for unknown environs
Everything in the environment cannot be anticipated
designer lacks omniscience
Learning is an alternative to explicit design
expose the agent to reality rather than trying to tell it about reality
lazy designer

7. Learning Agents

8. Learning Element Design of a learning element is affected by
Which components of the performance element are to be learned
What feedback is available to learn these components
What representation is used for the components

9. Performance Element Components
Mapping from conditions to actions:
Instructor says “brake”, agents learns condition-action rule for stopping
Means to infer properties of world from percepts
Agent studies pictures of buses to learn how to recognize a bus
Info about how world evolves and results of possible agent actions
Brake on a wet road and observe result

10. Performance Element Components
Info about desirability of world states
Agent receives no tip from a passenger of a rough ride and learns value of steadiness
Info about desirability of actions
Goals that describe classes of states that maximize agent’s utility

11. Types of Feedback Supervised learning: Unsupervised learning:
Agent is given correct answers for each example
Agent is learning a function from examples of its inputs and outputs
Unsupervised learning:
Agent must infer correct answers
Completely unsupervised learning is impractical, since agent has no context
Reinforcement learning:
Agent is given occasional rewards for correct
Typically involves subproblem of learning “how the world works”

12. Representation
The representation of learned knowledge affects the learning method/algorithm
Prop. Logic  inductive learning (chap 18)
FOL  inductive logic (chap 19)
Mathematical  statistical learning (chap 20)

13. Rote Learning: Example
Caching or memoization:
Storage of computed results to avoid recomputation later
Improves performance
Doesn’t improve effectiveness

14. Supervised Learning Example: Inductive Learning
Simplest form: learn a function from examples
f is the target function
An example is a pair (x, f(x))
Problem: find a hypothesis h
such that h ≈ f, given a training set of examples

15. Inductive Learning
Construct/adjust h to agree with f on training set (h is consistent if it agrees with f on all examples)
example, curve fitting:

16. Inductive Learning

17. Inductive Learning

18. Inductive Learning

19. Inductive Learning

20. Inductive Learning
Ockham’s razor: prefer the simplest hypothesis consistent with data
Overfitting: finding overly complex functions to account for noise or irrelevant data

21. Learning Decision Trees (LDT)
Decision trees branch on values of a set of input attributes, leading to answers at the leaves.
Construction (and optimization) of the tree is a learning problem.
Classification learning: learning a discrete function
Regression: learning a continuous function

22. DT Example Problem: Wait for a table at a restaurant? Attributes:
Alternate: is there an alternative restaurant nearby?
Bar: is there a comfortable bar area to wait in?
Fri/Sat: is today Friday or Saturday?
Hungry: are we hungry?
Patrons: number of people in the restaurant (None, Some, Full)
Price: price range ($, $$, $$$)
Raining: is it raining outside?
Reservation: have we made a reservation?
Type: kind of restaurant (French, Italian, Thai, Burger)
WaitEstimate: estimated waiting time (0-10, 10-30, 30-60, >60)

23. DT Example
This is a Boolean classification problem
Example decisions:

24. DT Example

25. Training and Test Sets
The agent must construct (learn) the DT from attribute  decisions pairs
This is the training set
Restaurant example: training set supplied by human to mimic own behavior
Assessing Performance:
Available example data is divided (randomly) among the training set and the test set.
Test set is then used to predict future performance

26. Knowledge in Learning
How can an agent make use of what it already knows, when learning new things? Hypothesis  Descriptions ╞ Classifications
Hypothesis is unknown: solve constraint from some hypothesis space
Adding background knowledge Background  Hypothesis  Descriptions ╞ Classifications

27. knowledge-based inductive learning
Cumulative Learning
Cumulative learning: agent uses and adds to its stock of knowledge
prior knowledge
knowledge-based inductive learning
hypothesis
observations

28. Statistical Learning Learning as uncertain reasoning from observations
Bayesian learning: computes probabilities for hypotheses and use that to make predictions
Neural networks: adjusts neuron activation functions based on example input/output pairs

29. Neural Networks input bits output bits percepts answers
operational mode
input bits
output bits
percepts
correct answers
training mode
net adjusts its internal parameters by comparing its outputs to the correct answers