Presentation is loading. Please wait.

Presentation is loading. Please wait.

Reinforcement Learning (for real this time). Administrivia No noose is good noose (Crazy weather?)

Published by Modified over 9 years ago

Similar presentations

Presentation on theme: "Reinforcement Learning (for real this time). Administrivia No noose is good noose (Crazy weather?)"— Presentation transcript:

1 Reinforcement Learning (for real this time)

2 Administrivia No noose is good noose (Crazy weather?)

3 Historical perspective Last time P2 handout Discussion of P2 This time Questions/answers -- generics, etc. Q2 (for real this time!) Intro to RL

4 Meet Mack the Mouse* Mack lives a hard life as a psychology test subject Has to run around mazes all day, finding food and avoiding electric shocks Needs to know how to find cheese quickly, while getting shocked as little as possible Q: How can Mack learn to find his way around? * Mickey is still copyright ?

5 Start with an easy case V. simple maze: Whenever Mack goes left, he gets cheese Whenever he goes right, he gets shocked After reward/punishment, he’s reset back to start of maze Q: how can Mack learn to act well in this world?

6 Learning in the easy case Say there are two labels: “cheese” and “shock” Mack tries a bunch of trials in the world -- that generates a bunch of experiences: Now what?

7 But what to do? So we know that Mack can learn a mapping from actions to outcomes But what should Mack do in any given situation? What action should he take at any given time? Suppose Mack is the subject of a psychotropic drug study and has actually come to like shocks and hate cheese -- how does he act now?

8 Reward functions In general, we think of a reward function: R() tells us whether Mack thinks a particular outcome is good or bad Mack before drugs: R( cheese )=+1 R( shock )=-1 Mack after drugs: R( cheese )=-1 R( shock )=+1 Behavior always depends on rewards (utilities)

9 Maximizing reward So Mack wants to get the maximum possible reward (Whatever that means to him) For the one-shot case like this, this is fairly easy Now what about a harder case?

10 Reward over time In general: agent can be in a state s i at any time t Can choose an action a j to take in that state Rwd associated with a state: R(s i ) Or with a state/action transition: R(s i,a j ) Series of actions leads to series of rewards (s 1,a 1 ) → s 3 : R(s 3 ); (s 3,a 7 ) → s 14 : R(s 14 );...

11 Reward over time s1s1 s2s2 s3s3 s4s4 s5s5 s6s6 s4s4 s2s2 s7s7 s 11 s8s8 s9s9 s 10

12 Reward over time s1s1 s2s2 s3s3 s4s4 s5s5 s6s6 s4s4 s2s2 s7s7 s 11 s8s8 s9s9 s 10 V(s 1 )=R(s 1 )+R(s 4 )+R(s 11 )+R(s 10 )+...

13 Reward over time s1s1 s2s2 s3s3 s4s4 s5s5 s6s6 s4s4 s2s2 s7s7 s 11 s8s8 s9s9 s 10 V(s 1 )=R(s 1 )+R(s 2 )+R(s 6 )+...

Download ppt "Reinforcement Learning (for real this time). Administrivia No noose is good noose (Crazy weather?)"

Similar presentations

Reinforcement Learning

Reinforcement Learning
 It encourages managers to cheat and lie  Managers inflating results and lowballing targets  It penalizes for telling the truth.

 It encourages managers to cheat and lie  Managers inflating results and lowballing targets  It penalizes for telling the truth.
Eick: Reinforcement Learning. Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning.

Eick: Reinforcement Learning. Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning.
Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning Applications Summary.

Reinforcement Learning Introduction Passive Reinforcement Learning Temporal Difference Learning Active Reinforcement Learning Applications Summary.
Positive And Negative Reinforcers For Your Child Psychology 121.

Positive And Negative Reinforcers For Your Child Psychology 121.
Drugs By Marcus Fuller Phetslat Carlton Kris Lunsford – Barrett and Chase Ropp.

Drugs By Marcus Fuller Phetslat Carlton Kris Lunsford – Barrett and Chase Ropp.
Reinforcement learning (Chapter 21)

Reinforcement learning (Chapter 21)
Social Relations: Conflict and Peacemaking How do we relate to others?

Social Relations: Conflict and Peacemaking How do we relate to others?
Reinforcement Learning & Apprenticeship Learning Chenyi Chen.

Reinforcement Learning & Apprenticeship Learning Chenyi Chen.
Reinforcement Learning

Reinforcement Learning
Reinforcement learning

Reinforcement learning
Reinforcement Learning

Reinforcement Learning
Intro to Reinforcement Learning Learning how to get what you want...

Intro to Reinforcement Learning Learning how to get what you want...
Reinforcement Learning, Cont’d Useful refs: Sutton & Barto, Reinforcement Learning: An Introduction, MIT Press 1998.

Reinforcement Learning, Cont’d Useful refs: Sutton & Barto, Reinforcement Learning: An Introduction, MIT Press 1998.
Planning in MDPs S&B: Sec 3.6; Ch. 4. Administrivia Reminder: Final project proposal due this Friday If you haven’t talked to me yet, you still have the.

Planning in MDPs S&B: Sec 3.6; Ch. 4. Administrivia Reminder: Final project proposal due this Friday If you haven’t talked to me yet, you still have the.
1 Hybrid Agent-Based Modeling: Architectures,Analyses and Applications (Stage One) Li, Hailin.

1 Hybrid Agent-Based Modeling: Architectures,Analyses and Applications (Stage One) Li, Hailin.
Reinforcement Learning Reinforced. Administrivia I’m out of town next Tues and Wed Class cancelled Apr 4 -- work on projects! No office hrs Apr 4 or 5.

Reinforcement Learning Reinforced. Administrivia I’m out of town next Tues and Wed Class cancelled Apr 4 -- work on projects! No office hrs Apr 4 or 5.
Ai in game programming it university of copenhagen Reinforcement Learning [Intro] Marco Loog.

Ai in game programming it university of copenhagen Reinforcement Learning [Intro] Marco Loog.
Reinforcement Learning: Learning to get what you want... Sutton & Barto, Reinforcement Learning: An Introduction, MIT Press 1998.

Reinforcement Learning: Learning to get what you want... Sutton & Barto, Reinforcement Learning: An Introduction, MIT Press 1998.
Planning to learn. Progress report Last time: Transition functions & stochastic outcomes Markov chains MDPs defined Today: Exercise completed Value functions.

Planning to learn. Progress report Last time: Transition functions & stochastic outcomes Markov chains MDPs defined Today: Exercise completed Value functions.

Similar presentations

Ads by Google