Presentation is loading. Please wait.

Presentation is loading. Please wait.

Class 2 Please read chapter 2 for Tuesday’s class (Response due by 3pm on Monday) How was Piazza? Any Questions?

Published byCordelia Stanley Modified over 8 years ago

Similar presentations

Presentation on theme: "Class 2 Please read chapter 2 for Tuesday’s class (Response due by 3pm on Monday) How was Piazza? Any Questions?"— Presentation transcript:

1 Class 2 Please read chapter 2 for Tuesday’s class (Response due by 3pm on Monday) How was Piazza? Any Questions?

2 Reinforcement Learning: a problem, not an approach What if you learn perfectly for level 1 and go to level 2? Trial-and-error again? (Gabe)

3 Sequential Decision Making: RL Environment Agent Action StateReward Markov Decision Process (MDP) S: set of states in the world A: set of actions an agent can perform T: S  A  S (transition function) R: S   (environmental reward)  : S  A (policy) Q: S  A   (action-value function)

4

5 Reinforcement Learning (RL) Basis in psychology (dopamine) Very limited feedback Delayed Reward Very flexible Possibly very slow / high sample complexity Agent-centric TD-Gammon Aibo Learned Gait: Tesauro, 1995 Kohl & Stone, 2004 Aside: Self play with same algorithm or different ones? (Josh / Anthony)

6 Reinforcement Learning (RL) Basis in psychology (dopamine) Very limited feedback Delayed Reward Very flexible Possibly very slow / high sample complexity Agent-centric TD-Gammon Aibo Learned Gait Tesauro, 1995 Kohl & Stone, 2004

7 Tic-Tac-Toe example Minimax Dynamic programming Evolutionary approach – entire policy, what happens during game is ignored (Approximate) Value Functions Exploration/exploitation – Greed is often good

8 Tic-Tac-Toe example Aside: Non-TD methods for learning value functions? (Bei)

9 Symmetries in tic-tac-toe (David) Opponent modeling vs. optimal play (Anthony)

10 Chris: UCT UCB1

11 Consider the following learning problem. You are faced repeatedly with a choice among n different options, or actions. After each choice you receive a numerical reward chosen from a stationary probability distribution that depends on the action you selected. Your objective is to maximize the expected total reward over some time period, for example, over 1000 action selections. Each action selection is called a play.

12 First, what is optimal policy if payoffs are deterministic? Evolutionary Approach? Value-based approach? Explore vs. Exploit in this context?

Download ppt "Class 2 Please read chapter 2 for Tuesday’s class (Response due by 3pm on Monday) How was Piazza? Any Questions?"

Similar presentations

© Jude Shavlik 2006, David Page 2007 CS 760 – Machine Learning (UW-Madison)RL Lecture, Slide 1 Reinforcement Learning (RL) Consider an “agent” embedded.

© Jude Shavlik 2006, David Page 2007 CS 760 – Machine Learning (UW-Madison)RL Lecture, Slide 1 Reinforcement Learning (RL) Consider an “agent” embedded.
Markov Decision Process

Markov Decision Process
Value Iteration & Q-learning CS 5368 Song Cui. Outline Recap Value Iteration Q-learning.

Value Iteration & Q-learning CS 5368 Song Cui. Outline Recap Value Iteration Q-learning.
brings-uas-sensor-technology-to- smartphones/ brings-uas-sensor-technology-to-

brings-uas-sensor-technology-to- smartphones/ brings-uas-sensor-technology-to-
Questions?. Setting a reward function, with and without subgoals Difference between agent and environment AI for games, Roomba Markov Property – Broken.

Questions?. Setting a reward function, with and without subgoals Difference between agent and environment AI for games, Roomba Markov Property – Broken.
Extraction and Transfer of Knowledge in Reinforcement Learning A.LAZARIC Inria “30 minutes de Science” Seminars SequeL Inria Lille – Nord Europe December.

Extraction and Transfer of Knowledge in Reinforcement Learning A.LAZARIC Inria “30 minutes de Science” Seminars SequeL Inria Lille – Nord Europe December.
Reinforcement learning (Chapter 21)

Reinforcement learning (Chapter 21)
Probability CSE 473 – Autumn 2003 Henry Kautz. ExpectiMax.

Probability CSE 473 – Autumn 2003 Henry Kautz. ExpectiMax.
Markov Decision Processes

Markov Decision Processes
Lehrstuhl für Informatik 2 Gabriella Kókai: Maschine Learning Reinforcement Learning.

Lehrstuhl für Informatik 2 Gabriella Kókai: Maschine Learning Reinforcement Learning.
Reinforcement learning

Reinforcement learning
SA-1 1 Probabilistic Robotics Planning and Control: Markov Decision Processes.

SA-1 1 Probabilistic Robotics Planning and Control: Markov Decision Processes.
Reinforcement Learning

Reinforcement Learning
KI2 - 10 Kunstmatige Intelligentie / RuG Markov Decision Processes AIMA, Chapter 17.

KI Kunstmatige Intelligentie / RuG Markov Decision Processes AIMA, Chapter 17.
R. S. Sutton and A. G. Barto: Reinforcement Learning: An Introduction 1 Chapter 2: Evaluative Feedback pEvaluating actions vs. instructing by giving correct.

R. S. Sutton and A. G. Barto: Reinforcement Learning: An Introduction 1 Chapter 2: Evaluative Feedback pEvaluating actions vs. instructing by giving correct.
Outline MDP (brief) –Background –Learning MDP Q learning Game theory (brief) –Background Markov games (2-player) –Background –Learning Markov games Littman’s.

Outline MDP (brief) –Background –Learning MDP Q learning Game theory (brief) –Background Markov games (2-player) –Background –Learning Markov games Littman’s.
Reinforcement Learning

Reinforcement Learning
Reinforcement Learning Mitchell, Ch. 13 (see also Barto & Sutton book on-line)

Reinforcement Learning Mitchell, Ch. 13 (see also Barto & Sutton book on-line)
Chapter 1: Introduction

Chapter 1: Introduction
1 Hybrid Agent-Based Modeling: Architectures,Analyses and Applications (Stage One) Li, Hailin.

1 Hybrid Agent-Based Modeling: Architectures,Analyses and Applications (Stage One) Li, Hailin.

Similar presentations

Ads by Google