1. Bernhard Nebel, Albert-Ludwigs-Universität Freiburg How Much Intelligence Do You Need to Play Soccer? Soccer and Intelligence? Robotic Soccer and the RoboCup Competitions CS Freiburg: Hardware and Software Architecture Cooperative Sensing From Perception to Action Team Play: Dynamic Role Assignment Conclusions & Outlook

2. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?2 Soccer and Intelligence? Do you really need any intelligence in order to play soccer? Isn’t it just a reactive game?

3. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?3 Soccer and Intelligence! Reconstruction of environment from sensor data Spatial reasoning Being able to select the right motor skill (and parameterize it) in order kick the ball into the right direction Strategic thinking and acting Proactive in order to create opportunities Reactive in order to exploit opportunities  Insects do not play soccer

4. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?4 Why Robotic Soccer? Chess is “solved” Robotic soccer has a number of interesting properties (that makes it different from chess)  uncertainty in sensor interpretation and acting, a highly dynamic environment, and the need for cooperation Challenge for designing integrated systems that “close the loop” Solutions have relevance for other areas, in particular for multi-robot systems (e.g., teams of cleaning robots)

5. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?5 The RoboCup Initiative Alan Mackworth 1993: Integration of AI, Robotics, MAS, Real-Time Reasoning RoboCup [Kitano et al 97]:  Workshops & Competitions RoboCup leagues:  Simulation league  F180 (small) league  Aibo (Sony dog) league  F2000 league (4 players/team, 4x9 m field) Challenge:  Win against human world champion by 2050 Goal against RMIT Raiders (Melbourne) (RoboCup´99)

6. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?6 Competitions & Science Competitions push the development of science  direct comparisons  everybody gives the best  development of new, innovative solutions Possible problems:  exploitation of loop holes in the rules  and: concentration on competition How to build a successful system:  single conceptual pieces are very important (e.g.self- localization, team coordination)  combination and integration is important: “closing the loop” Results are communicated in workshops & symposia

7. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?7 Our Goals, Our Approach Demonstration of our self-localization techniques using laser scanners Exploration of multi-robot systems  Cooperative acting  Cooperative sensing Attractive area for students “Classical” Artificial Intelligence approach  explicit model of the world  deliberation on world model ... finding the right balance between reactivity and deliberation

8. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?8 CS Freiburg: Robot Hardware Hardware  Pioneer 1 robots (from ActivMedia)  Pioneer 2 controller boards  Vaio Picturebook (with Linux)  WaveLan radio-Ethernet  Kicker: custom-made ( SICK AG ) Sensors  Digital Sony camera with Firewire  SICK laser scanner  Internal odometrie  Kicker sensors (state/ball)

9. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?9 Player Architecture world modeling cooperation & strategy action execution action selection sensors communication actuators

10. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?10 Sensor-Interpretation Inputs:  Laser scanner data (360 range values for 180°, 1cm accuracy, 30 scans/sec)  Odometrie (translation, rotation since last measurement, 10 estimates/sec)  Vision color data (720x576 pixel, 25 frames/sec) Outputs:  Own pose (position and orientation) = self localization  Poses and velocities of other players (team mates and opponents)  Ball position and velocity

11. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?11 Self-Localization Estimation of own pose: location and orientation  Difficult, even if initial position is known (odometrie errors accumulate) Use sensors to correct pose estimation  GPS (works only in outdoor environments), beacons...  Range finding sensors (sonars, laser scanners,...)  Vision (landmarks or 3D reconstruction) Combine odometrie and other sensor measurements  Kalman filter or Markov localization

12. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?12 Our Approach: Line Extraction & Scan Matching Extract lines from scans Try all pairings of scan lines with model lines and test for geometric realizability  i.e. find rotation and translation to make it fit 2 hypotheses if 3 walls are visible, 4 if 2 walls. Initial orientation is known  fast, robust, accurate  global localization

13. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?13 Player Recognition Remove all points from the scan that correspond to points on the wall Cluster remaining points Consider point of gravity as middle point of player (perhaps with offset)

14. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?14 Why is Self-Localization & Player Recognition Important? Going to the kick-off positions - different approaches (Game against CMU Hammerheads, RoboCup 2001)

15. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?15 Another Reason: The 10 Seconds Rule We obey the 10 seconds rule and leave the goal area before 10 seconds have elapsed... finally we score a goal against NAIST´00 (Japan) (RoboCup 2000)

16. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?16 Ball Recognition in Real Time Color segmentation (watch for orange blobs)  previously: Newton Lab hardware system  now: Sony camera with Firewire output, vision processing on standard notebook, CMVision system (CMU) for color segmentation  now we see the ball up to 5-6 meters Size and distance estimation  using a case-based interpolation method instead of Tsai- calibration and analytic estimation - more accurate if z- coordinate is constant Identification of most plausible blob as the ball  using size, distance, and duration of observation

17. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?17 Multi-Robot Sensor-Integration: Cooperative Sensing All players send their estimates (own position, ball position, position of other players) together with a time stamp to the global sensor-integration module. Estimations are combined Friend-opponent-distinction: our players report their own positions Global ball position estimation:  Integrate measurements from different robots using a Kalman filter  but ignore “hallucinations” using a Markov localization scheme

18. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?18 Kalman Filter Update step (when new observation is made):  Integrate new measurement  Take into account sensor model  distance error grows with distance  angular error is small and constant  Leads to triangulation (stereo vision with a robot group)... Prediction step (predict next location where ball will be observed):  Project ball position into the future using a constant negative ball acceleration  Consider a certain projection error

19. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?19 False Positives Player 2 is hallucinating ?

20. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?20 Phantom Ball Elimination: Markov Localization Update step  Incorporate new ball sighting, use Bayes´ rule (cond. prob. has normal distribution wrt. distance): Prediction step (with unknown direction and speed - only 2 dimensional probability grid)  The occupancy probability of a cell z´ t flows into cells z close to z´ (cond. prob. has normal distribution wrt. distance):

21. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?21 Phantom Balls: Development of Probability Distribution At RoboCup 2000, 938 out of 118388 (0.8%) ball observations were ignored because of the Markov localization filter. At German Open 2001, 0.6% ball observations were ignored. Consider area with highest peak as possible ball area after 1st measurement (1)after 2nd measurement (2)after 3rd measurement (3)after 4th measurement (1)after 5th measurement (2)after 6th measurement (3)

22. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?22 Importance of Accurate Global Ball Position Estimation Our goalie cannot see the ball but continues to defend the goal against Golem (Italy) in the RoboCup 2000 final... since he knows the ball position from his team mates

23. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?23 A Positive Example Minho (Portugal) shoots at our goal from the other side of the field. Our goalie gets this information early on and can easily defend (RoboCup 2001)

24. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?24 From Perception to Action: Do the Right Thing! Reactive, behavior-based vs. deliberative approaches Soccer is a fast game and it does not require long (on- line) deliberation However, there exist “higher” levels beyond situation- action-rule execution:  different actions and an action selection mechanism  cooperation: placement and team play  opponent prediction and counter strategies Most teams use hybrid approaches, but some are almost completely reactive

25. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?25 How do we Structure the Capabilities of a Soccer Agent? Actions: State-free behaviors (controllers) Action selection:  Select the most appropriate action  Do this by goal back-chaining (extended behavior networks)  Once an action has been selected, commit to it (persistence to avoid oscillations) and quit if action will fail necessarily Control execution of actions  Evaluate and adapt continuously every 100 msec Strategic considerations are factored out

26. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?26 Field-Player Action Repertoire With ball  ShootPos : shoot to a position, often close to opponent goal  MoveTrickShoot : run into direction of opponent goal and in the last possible moment change direction  BumpShoot : shoot ball by bumping against it  TurnBall : turn with ball slowly into the direction of opponent goal  DribbleBall : run with ball into the direction of opponent goal Without Ball  ObserveBall, SearchBall, GotoPosition, WaitPass, GetBall, GotoBall...

27. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?27 What is Important When Implementing Basic Actions? An attack on the GMD Robots’ goal, which could have been carried out more skillfully (RoboCup 1999)

28. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?28 Field Player: Tricking the Goalie We score against CE Sharif (Iran) by tricking the goalie: Move in one direction, shoot in the other direction ( MoveTrickShoot) (RoboCup 2000)

29. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?29 Field Player: Dribbling Consider points that are closer to the opponent goal and that do not require a sharp turn Evaluate the straight lines to these points according to  distance to obstacles  angle to goal  angle to current orientation Choose best alternative

30. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?30 Dribbling: An Example One of the 9 goals against CMU Hammerheads (USA) (RoboCup 2001)

31. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?31 Field Player: ShootPos Find a good position (close to opponent goal) to shoot to Consider also rebound shots! Choose the shot that  does not collide with obstacles  comes as close as possible to the opponent goal  does require a minimal turn angle

32. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?32 Field Player: A Rebound Shot Rebound shot against CoPS Stuttgart (RoboCup 2000)

33. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?33 Action Selection: Extended Behavior Networks Behavior networks [Maes 90]  action representation with preconditions and consequences  activation from situation & goals (  reactive & goal-oriented) Extended behavior networks [Dorer 99]  fuzzy propositions  activation only from goals  goal-tracking and no input normalization (  decision theoretic planning)  used in magma Freiburg (runner up in the simulation league RoboCup ´99

34. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?34 Part of the Extended Behavior Network soccer goalcooperate ObserveBall SearchBall MoveTrick- Shoot ShootPos (goal)l GetBalll WaitPass have_ball active role close_to_ball ball_present have_ball good_p_pos

35. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?35 Example: Action Selection & Sequencing Attack against CMU Hammer- heads (RoboCup 2000 quarter final) Action Sequence: GetBall, TurnBall, DribbleBall, ShootPos(goal)

36. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?36 Acting Cooperatively Without coordination, one gets swarm behavior Avoidance of interference  do not attack your own team mates  do not get into the way of an attacking or defending robot  use competence areas on the field (still backup) Task decomposition and task (re-)allocation  the player which is closest to the ball should go to the ball  if one player cannot do his task, another should take over Joint execution: passing the ball  Use (dynamic) role assignment

37. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?37 Cooperation: Dynamic Role Assignment Each player has one of 4 roles:  goalie (fixed)  active player: with ball or close  supporter : other half of field  strategic player: defender Placement: each role has a preferred location, which depends on the situation:  ball position, position of team mates and opponents  defensive situation or attack active Role: strategic role: supporter role:

38. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?38 Dynamic Role Assignment Each player computes the utility for each role and sends it around (same as in ART Italy ) Utility depends on distance to preferred location (large distance has low utility) and on the role ( active > strategic > supporter ) Each player tries to maximize the group utility  under the assumption that all team members do that Roles are reassigned only when two players agree  this does not exclude that there are occurrences when two players have the same role (because there is no synchronization) Note that opinion about global position can differ (even with global world model)

39. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?39 Team Play: Role Switch Tight defense against CE Sharif (Teheran) (RoboCup 2000 semi final)

40. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?40 Role Switch by Communication green: strategic light blue: supporter white: active

41. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?41 Another Example for Role Switching Defense against Artisti Veneti (Italy) (RoboCup 2001) The roles active and strategic player are switched a couple of times

42. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?42 Joint Execution: A Pass... that was Unsuccessful A pass in the semi- final against the Italian ART Italy team (RoboCup 1999). This was based on standard plan: “if it is not possible to score directly, wait until supporter arrives, then make the pass”

43. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?43 Passing the Ball as Emergent Behavior A slightly unconventional pass in a game against Robosix (RoboCup 2001)

44. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?44 Reactive vs. Deliberative: RoboCup 2001 Final C S Freiburg against Osaka Trackies (Japan) CS Freiburg is most of the time slower and has to defend... but it is more robust. Finally, when 2 Trackies had been removed...

45. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?45 CS Freiburg: Performance Competition Results  Winner of RoboCup´98, RoboCup 2000 and RoboCup 2001  Winner of German Vision-RoboCup´98 & ‘99 and German Open 2001, runner up in 2002  2 draws & 3 lost games in 52 official games Goal Rates (goals/minute)

46. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?46 Conclusion & Outlook The kind of intelligence you need to play soccer is quite different from the one you need to play chess Robotic soccer is an attractive and interesting research challenge, in particular for the area of multi-robot systems Current & future research issues  Learning & adapting the skills to new environments and capabilities  Addressing a different problem in the domain: a robot referee  Address other multi-agent problems such as RoboCup-Rescue  Transfer to a new domain: table soccer

47. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?47 Table Soccer... is interesting because computer systems may be able to play this game against humans (we do not intend to wait until 2050) Two humans against one computer: unfair?

48. B. Nebel, Univ. Freiburg How Much Intelligence Do You Need to Play Soccer?48 Acknowledgements CS Freiburg Team 2001 (www.cs-freiburg.de)  Markus Dietl, Florian Diesch, Steffen Gutmann (Sony), Alexander Kleiner, Boris Szerbakowski (SICK), Thilo Weigel, Patrick Stiegeler Previous team members  Burkhard Dümler, Wolfgang Hatzack, Immanuel Herrmann, Kornel Marko, Klaus Müller, Livia Predoiu, Christian Reetz, Frank Rittinger, Maximilian Thiel, Augustinus Topor Institutions  SICK AG (Hardware and manpower)  DFG, MFG, und Univ. Freiburg (financial support)  Sony, Siemens