Download presentation

Presentation is loading. Please wait.

Published byDonna Dean Modified about 1 year ago

1
Reinforcement Learning for the Soccer Dribbling Task Arthur Carvalho Renato Oliveira

2
Introduction RoboCup soccer simulation –Scoring “A Data Mining Approach to Solve the Goal Scoring Problem” –Passing “A New Passing Strategy Based on Q-Learning Algorithm in RoboCup” –Dribbling ?

3
Soccer Dribbling Task

4
Outline The soccer dribbling task as a RL problem RL solution Experiments Conclusion

5
The Soccer Dribbling Task as a RL Problem Coach –Setting positions Dribbler is placed in the center-left region together with the ball Adversary is placed in a random position –Manage the play Adversary wins when he gains possession or when the ball goes out of the field Dribbler wins when he crosses the field with the ball

6
The Soccer Dribbling Task as a RL Problem When an episode ends, the coach starts a new one RoboCup soccer simulator operates in discrete time steps Episodic reinforcement-learning framework

7
The Soccer Dribbling Task as a RL Problem Actions –HoldBall() –Dribble(α, k) Dribble(30, 5), Dribble(330, 5), Dribble(0, 5), Dribble(0, 10) The dribbler can kick the ball forward (strongly and weakly), diagonally upward, and diagonally downward.

8
The Soccer Dribbling Task as a RL Problem State VariableMeaning posY (dribbler) Vertical position of the dribbler ang(dribbler)Global angle of the dribbler ang(dribbler; adversary) The relative angle between the dribbler and the adversary ang(ball; adversary) The relative angle between the ball and the adversary dist(ball; adversary) Distance between the ball and the adversary

9
Outline The soccer dribbling task as a RL problem RL solution Experiments Conclusion

10
RL Solution

11
CMAC –Partitioning the state space into several receptive fields (hyper-rectangles) Each one is associated with a weight –Multiple partitions of the state space (layers) are usually used –The CMAC’s response to a given input is equal to the sum of the weights of the excited receptive fields

12
RL Solution

13
Outline The soccer dribbling task as a RL problem RL solution Experiments Conclusion

14
Experiments

15
Adversary –Fixed policy –It computes a near-optimal interception point (UvA Trilearn 2003 team) Two phases –Training –Testing

16
Experiments Training Phase: 5 independent runs, each one lasting 50,000 episodes 53%

17
Experiments Qualitatively –Rule #1

18
Experiments Qualitatively –Rule #2

19
Experiments

20
Outline The soccer dribbling task as a RL problem RL solution Experiments Conclusion

21
Dribble –Soccer dribbling task –Reinforcement learning solution Benchmark Start point for dribbling tasks in other sports games –E.g., hockey, basketball, and football

22
Thank you! Source code available at: Arthur Carvalho Renato Oliveira

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google