1. Computer Engineering Department INTRODUCTION TO ROBOTICS COE 484 Dr. Mayez Al-Mouhamed SPRING 2008 Chapter III – Behavior Programming

2. Plan Introduction to Soccer Game Behavior Design, “Soccer Behaviors for Humanoid Robots, By Jörg Stückler and Sven Behnke German Team 2005

3. Soccer Behaviors for Humanoid Robots Jörg Stückler and Sven Behnke (2006) and other papers* We describe the design of the behavior control software for the Humanoid League team NimbRo. The control software is based on a framework that supports a hierarchy of reactive behaviors: It is structured both as an agent hierarchy (joint – body part – player – team) and as a time hierarchy. The speed of sensors, behaviors, and actuators decreases when moving up in the hierarchy. The lowest levels of this framework contain position control of individual joints and kinematic interfaces for body parts. At the next level, basic skills like omnidirectional walking, kicking, and getting-up behaviors are implemented. These are used by soccer behaviors like: searching for the ball, approaching the ball, avoiding obstacles, and defending the goal. Finally, on the team level, the robots communicate via a wireless network to share information about the world state and to negotiate roles like attacker and defender. Participated in RoboCup 2006. The KidSize robots won the Penalty Kick competition and came in second in the overall Best Humanoid ranking, next only to the titleholder, Team Osaka. * All Summaries are taken from ”Hierarchical Reactive Control for a Team of Humanoid Soccer Robots”, same authors, 2007.

4. Soccer Behaviors for Humanoid Robots The framework forces the behavior engineers to define abstract sensors that are aggregated from faster, more basic sensors. One example for such an abstract sensor is the robot’s attitude that is computed from the readings of accelerometers and gyros. Abstract actuators give higher-level behaviors the possibility to configure lower layers in order to eventually influence the state of the world. One such abstract actuator is the desired walking speed, which configures the gait engine, described below, implemented in the lower control levels. The behaviors within one layer of the behavior framework are activated according to the current state of its sensors. Activation is indicated by an activation factor in the interval. Each active behavior can manipulate the actuators in its layer. If multiple behaviors try to manipulate the same actuator, the actuator is set to the weighted sum of desired values, where the activation factors are used as weights. To prevent conflicting behaviors from being active at the same time, behaviors can inhibit other behaviors. If an inhibiting behavior is not completely active, the inhibited behaviors share the remaining activation, such that the activation factors sum to one. The control hierarchy of our soccer robots is arranged in an agent hierarchy, where multiple joints (e.g. left knee) constitute a body part (e.g. left leg), multiple body parts constitute a player (e.g. field player), and multiple players constitute a team. The behavior framework manages all but the motor control loop within the Dynamixel actuators, which has been implemented by Robotis.

5. Fig. 4. Actuators, behaviors, and mutual inhibitions within the behavioral joint – body part – player hierarchy. Upper layer behaviors can configure lower layer behaviors by manipulating the upper layer actuators. The resulting values of the actuators depend on the activation factors and the inhibitory structure of the manipulating behaviors.

6. Fig. 5. Actuators, behaviors, and mutual inhibitions within the behavioral hierarchy. Upper layer behaviors can configure lower layer behaviors by manipulating the upper layer actuators. The resulting values of the actuators depend on the activation factors and the inhibitory structure of the manipulating behaviors.

7. IV. BASIC SKILLS Fundamental for playing soccer are: the abilities to walk and to kick. As body contact between the physical agents is unavoidable, the capability of getting up after a fall is also essential. For keeping a goal, the robot must be able to perform special motions. The basic skills are implemented on the body part layer. Fig. 4 illustrates the inhibitory structure of the basic skills and how the motor control layer is configured through actuators. A.Omni-directional Walking We use the leg interface to implement omni-directional walking: The robots are able to walk in every direction and to change their heading direction at the same time. The gait target vector (vx, vy, v-theta) can be changed continuously while the robot is walking. This makes it possible to correct for deviations in the actual walking direction and to account for changes in the environment by using visual feedback. Behaviors of the upper level can control the gait target vector with an actuator that enforces maximal speeds and accelerations.

8. IV. BASIC SKILLS (Cont) B. Kicking In addition to walking, we implemented kicking: An actuator allows behaviors in the upper level to trigger kicks with both the left and the right leg. After inhibiting the walking behavior and coming to a stop, the robot moves its weight to the non-kicking leg (see hip roll angle). Then, it shortens the kicking leg, swings it back and accelerates forward. The kicking leg reaches its maximal speed when it comes to the front of the robot. At this point the hip pitch joint and the knee both move the foot forward and the ball is kicked. The kicking movement continues with deceleration of the foot and slow motion back to the bipedal stand. C. Getting up after a Fall Since in soccer games physical contact between the robots is unavoidable, the walking patterns are disturbed and the robots might fall: Hence, they must be able to detect the fall, to recognize their posture on the ground, and to get back into an upright posture. After falling, the robot’s center of mass (COM) projection to the ground is outside the convex hull spanned by the foot-contact points. Additional support points like knees, elbows, and hands must be used in order to move the COM back inside the foot polygon. Using their attitude sensors, the robots detect a fall, classify the prone or supine posture and trigger the corresponding getting-up sequence.

9. D. Goalkeeper Motions The goalkeeper is capable of diving into both directions or to bend forward with spread arms. First, it moves its COM and turns its upper body towards the left while shortening the legs. As soon as it tips over its left foot, it starts straightening its body again. While doing so it is sliding on its hands and elbows. The fully extended robot covers the entire goal half. After the dive Franz gets up again, as described above. V - PLAYING SOCCER Soccer behaviors of the robots during a 2 vs. 2 soccer game, which build on the basic skills. A - World and Internal State Representation The soccer behaviors require knowledge of the current game situation: The visual perception supplies relative distance, angle, and perceptual confidence for the ball pBall = (dBall, theta-Ball, cBall), the own goal pOwnGoal, the opponent goal pOppGoal, and the nearest obstacle pObstacle. The confidence is a value in the interval [0, 1]. In the Attacking Player, the relative position and confidence of the opponent goal is used as the target to kick at. We refer to this target as the ball-target with relative position pBallTarget. The decision for the kicking leg is made in every time step, depending on the relative position of the ball and the line from ball to ball-target, which we denote as ball-to-target- line. If the robot has to approach the ball-to-target-line from the right, it kicks with the left leg, and vice versa. To avoid oscillations it is important that the decision may only be changed if the distance of the robot to the ball-to-target-line is large.

10. A - World and Internal State Representation (Cont) To kick the ball with the chosen leg, the robot has to position itself behind the ball with lateral and segittal offsets, delta-l and delta-s that depend on the distance between the legs and the length of the feet. The target position behind the ball, pBehindBall, is calculated by adding the lateral and sagittal offsets to the ball position orthogonal to and along the ball-to-target-line, respectively. To generate smoothly approaching trajectories to this position, the sagittal offset is increased by an amount a that is proportional to the angle between the robot’s heading direction and the ball-target. Fig. 11 illustrates two examples for positioning behind the ball with the left leg as kicking leg. Fig. 11. Two examples showing sequences of robot poses, target positions behind the ball (blue crosses), and ball positions while approaching the ball with the left leg as kicking leg.

11. A - World and Internal State Representation (Cont) When playing as Defensive Field Player, the own goal is used as ball-target, such that the position behind the ball is set to a position between ball and own goal: The distance kept to the ball depends on the distance to the own goal. A threshold for the minimal distance to the goal lets the robot stay out of its goal, as long as the ball is still far away. If the ball and the robot are near the goal, the robot keeps behind the ball at a minimum distance. The robot maintains additional hypotheses about the relative ball location that are used for searching the ball. If a kick is triggered, one hypothesis is set in front of the robot at a distance depending on kick strength. The confidence of the hypothesis is discounted by the time since the kick started. Its relative position is altered according to the motion model. Additionally, hypotheses are maintained for the perceptions of the ball by other players on the field. The confidence of these hypotheses depends on the self-localization and ball perception confidences of the other players and the self-localization confidence of the robot itself.

12. Summary (World representation): The visual perception supplies relative distance, angle, and perceptual confidence for the ball, the own goal, the opponent goal, the center angle of the largest goal area not covered by the goalie, and the nearest obstacle. In the offensive role: The relative position and confidence of the largest free area in the opponent goal is used as the target to kick at (ball-target), if it is perceived confidently. Else the opponent goal is used as ball-target. To kick the ball, the robot has to position with its kicking leg behind the ball. Thus, the corresponding relative target position of the robot, denoted as behind-ball-position, and the kicking leg are included in the game state. The decision for the kicking leg is made at every time step, depending on the relative position of the ball and the line from ball to ball-target. If the robot has to approach the ball-to- target-line from the right, it kicks with the left leg, and vice versa.

13. Summary (World representation) (Cont): To avoid oscillations, decisions are only made as long as the distance of the robot to the ball-to- target-line exceeds a threshold. When playing as defensive field player: The own goal is used as ball-target, such that the position behind the ball is set to a defensive position between ball and own goal. The components of the current game state are provided to the soccer behaviors through sensors. Fig. 6 illustrates the positioning of offensive and defensive field player with an example. The robot also maintains hypotheses about the relative ball location that are used for searching the ball. If a kick is triggered, one hypothesis is set to the predicted end position of the ball, as due to limitations in visibility range the visual perception could fail to detect the ball. Additionally, hypotheses are maintained for the perceptions of the ball by other players on the field. The confidences in these hypotheses depend on the self-localization and ball perception confidences of the other players and the self-localization confidence of the robot itself. The relative positions of the hypotheses are altered according to the motion model. Their confidences are discounted by the time since the last update.

14. Fig. 6. An illustration for the positioning of offensive and defensive field players (green circles with orientation indicator). The offensive field player positions on the line from ball to target goal (yellow) with sagittal and lateral offsets behind the ball (orange circle) on the blue cross, such that the ball lies in front of the kicking leg. The defensive field player takes a defensive, supporting pose by positioning on the direct line from ball to own goal in a certain distance to the ball.

15. B. Soccer Behaviors According to the current game situation, behaviors like searching the ball, positioning behind the ball, or avoiding obstacles are activated. These behaviors are implemented on the player level and use the actuator provided by the basic skills of the lower layer. For example they set the gait target vector or trigger a kick. Fig. 4 illustrates the inhibitory structure of the soccer behaviors and the actuator interface to the basic skills which reside on the lower layer. 1)Searching the Ball Exploring the environment for the ball is always active, but inhibited by behaviors that activate when the ball has been perceived with a certain confidence. If a ball hypothesis with confidence over a certain threshold exists, the robot walks towards the most confident hypothesis. Otherwise, it turns towards the most confident hypothesis for a short time. If the ball is still not visible, it starts walking around the center circle in a constant distance in order to inspect all parts of the field. Summary: If the ball is not perceived, the robot has to explore the field for it. In the case that a confident ball hypothesis exists, it gazes and walks towards it. If no confident hypotheses exist, it first stops walking and sweeps its upper trunk over a range of ± pi/2, because motion blur reduces the visibility range of balls during walking. If still no ball is in sight, it walks to the farer goal, and then turns to the other goal. After this, it repeats the search process.

16. B. Soccer Behaviors 2) Walking towards the Ball The robot walks straight towards the ball, if it perceives the ball. The own goal must be either not visible (cOwnGoal 0.5m) to avoid an own goal. This behavior controls the gait target velocity to keep the robot near the ball, e.g. if visual perception fails to detect the opponent goal. The behavior inhibits searching the ball. Summary: As it is possible to perceive the ball, while the opponent goal is occluded, the robot is kept close to the ball. The behavior is not activated, if the robot is close to the own goal.

17. B. Soccer Behaviors (Cont) 3) Positioning behind the Ball: If the ball and the ball-target are perceived, the robot positions itself behind the ball, facing towards the ball-target. The robot is positioning on pBehindBall by controlling the gait target velocity. Some details are: If the distance to the target position is large (dBehindBall > 0.5m), the robot rotates towards the target position, such that it can approach it by mainly combining forward walking with turning. If it is near the target position (dBehindBall =< 0.25m), the robot aligns itself towards the ball-target. For intermediate distances, the gait rotation is interpolated linearly between both alignment targets. The behavior also handles the case, when the ball is positioned between robot and target position. Here, the robot walks around the ball by walking towards the target position but avoiding the ball-to-target-line. When playing as defensive field player, the robot rotates towards the ball at any distance. It does not avoid the ball-to-target-line, because the ball-target is the own goal. This behavior inhibits walking towards the ball, such that the inhibited behavior may only be active, if the ball-target has not been perceived. It also inhibits searching the ball.

18. B. Soccer Behaviors (Cont) 3) Positioning behind the Ball: Summary: If ball and opponent goal are visible, the robot positions behind the ball, as given in the current game state. While the distance to the target position is large, the robot rotates towards the target position, such that it can approach it by mainly combining forward walking with turning. If it is near the target position, the robot aligns itself towards the ball-target. For intermediate distances, the gait rotation is interpolated linearly between both alignment targets. The behavior also handles the case when the ball is located between the robot and the behind-ball-position. Here, the robot walks around the ball by walking towards the target position but avoiding the ball-to-target-line. When playing as defensive field player, the robot rotates towards the ball at any distance without avoiding the ball-to-target-line. *All Summaries are taken from ”Hierarchical Reactive Control for a Team of Humanoid Soccer Robots”, same authors, 2007.

19. B. Soccer Behaviors (Cont) 4) Kicking the Ball towards the Target: This behavior is activated as soon as the target position pBehindBall has been reached with a certain precision in angle to the ball-target and in distance to the target position: If the precision conditions hold, a kick is triggered. Naturally, ball and ball-target must be perceived. To prevent own goals, the own goal must be either not visible or be seen in an angle that is larger than ± pi/4. If the ball comes into a kicking position by chance, the behavior initiates a kick with the corresponding leg. As the robot has to come to a complete stop before the kicking motion can be executed, the robot can cancel the kick, if the ball moves away in the meantime. This behavior inhibits searching the ball, walking towards the ball, and positioning behind the ball. Summary: This behavior is activated as soon as the behind-ball position has been reached with a certain precision in angle to the ball-target and in distance to the target position. The active behavior triggers a kick with the correct kicking leg. If the ball comes into a kicking position by chance, the behavior initiates a kick with the corresponding leg. As the robot has to come to a complete stop before the kicking motion can be executed, the robot can cancel the kick, if the ball moves away in the meantime.

20. B. Soccer Behaviors (Cont) 5) Dribbling the Ball towards the Target: If positioning behind the ball was not successful for a longer time, or the game started with a kick-off for the player, the robot activates dribbling the ball towards the ball-target for some time. Additional preconditions for activation are that the ball and ball-target are perceived and the angle towards the balltarget is smaller than ± pi/4. Dribbling is performed by steering towards the ball. The forward walking speed is inversely related to the angle to the ball. In combination with positioning behind the ball, the robot is kept behind the ball, facing the ball-target when dribbling. Dribbling inhibits searching the ball, walking towards the ball, and positioning behind the ball. As we want the decision for dribbling to be strict, it also inhibits kicking the ball towards the target. Summary: If positioning behind the ball was not successful for a longer time, or at kick-off, the robot dribbles the ball towards the ball-target for some time. Target-oriented dribbling is achieved by steering towards the ball, if it is still far away. If it is close, the robot aligns the ball in the center in front of its feet by lateral motion, and aligns its orientation towards the ball-target. The better this alignment is, the faster the robot moves in forward direction. At intermediate distances, the alignment targets are interpolated linearly. Dribbling inhibits kicking the ball.

21. B. Soccer Behaviors (Cont) 6) Avoiding Obstacles: After a fall, the robot needs valuable time to get back on its feet. The main reason for our robots to fall is physical contact with other robots. Hence, obstacle avoidance is an important feature. The visual perception supplies the behavior with the nearest obstacle. If it is detected (cObstacle > 0.1), in a certain distance (dObstacle < dObstacle_max), and in front of the robot (|Obstacle| < 1.2), obstacle avoidance is activated by a factor that interpolates linearly between a minimum and a maximum distance for the obstacle, dObstacle_min and dObstacle_max. The avoidance sets the gait target actuator to a constant and a variable part of the direction from obstacle to robot. The strength of the variable part depends on the distance to the obstacle, similar to the activation factor. If the ball is between obstacle and robot, the variable avoidance is weakened, such that the robot moves more aggressively behind the ball. A stuck situation is indicated by a resulting gait target vector that is small in length for a longer time. In this case, the robot may sidestep the obstacle, if the ball is not between the obstacle in the front and the robot and is perceived on one side of the obstacle (|Obstacle − Ball| > pi / 4 ). The action is cancelled, if either the preconditions for sidestepping do not hold anymore or some time has passed since sidestepping has been activated. The deactivation of sidestepping after some time is important, because the decision for the sidestep direction is made only once on activation.

22. B. Soccer Behaviors (Cont) 6) Avoiding Obstacles (Cont): Summary: After a fall, the robot needs valuable time to get back on its feet. The main reason for our robots to fall is physical contact with other robots. Hence, obstacle avoidance is an important feature. The visual perception supplies the behavior with the nearest obstacle. If it is detected closely in front of the robot, obstacle avoidance is activated. The avoidance sets the gait target actuator to a constant and a variable part of the direction from obstacle to robot, which norm is proportional to the distance to the obstacle. If the ball is between obstacle and robot, the variable avoidance is weakened, such that the robot moves more aggressively behind the ball. A stuck situation is indicated by a resulting gait target vector that is small in length for a longer time. In this case, the robot may sidestep the obstacle, if the ball is not between the obstacle in the front and the robot and is perceived on one side of the obstacle. The action is canceled, if either the preconditions for sidestepping do not hold anymore or a certain amount of time has elapsed since sidestepping has been activated.

23. B. Soccer Behaviors (Cont) 7) Controlling the Gaze Orientation: Although the robot has wide-angled views to the front and the rear, it cannot perceive objects on the sides. Thus, a gaze control behavior is always active and primarily keeps the ball within an angular range of ± pi/ 4 by twisting the upper trunk with the trunk yaw joint. If the ball is not visible or within range and the robot is localized, it aligns the upper body onto the line between the goals to keep the localization landmarks visible. This is achieved by keeping the angle to the line within the angular range of ± pi / 4. The twisting of the trunk is limited to ± pi/4 in each direction. Summary: A gaze control behavior keeps the ball within an angular range of ±pi / 4 in the front camera by rotating the upper trunk in yaw direction. If the ball is not visible or within range and the robot is localized, it aligns the upper body with the line between the goals to improve the visibility of the landmarks.

24. B. Soccer Behaviors (Cont) 8) Goalkeeping: The goalkeeper’s objective apparently is to keep the ball out of the own goal. While the ball is visible and is not within the kicking distance to the robot, the goalkeeping behavior is active. Otherwise, the robot behaves like a field player and tries to shoot the ball towards the opponent goal. Hence, goalkeeping inhibits positioning behind the ball, kicking the ball, and dribbling the ball towards the target. Walking towards the ball and searching for the ball is not activated when playing as a goalkeeper. The goalkeeper stands still until it reacts on the ball. When the ball is close to the robot it lets it react immediately. It uses the angle to the ball to determine the type of motion (diving left/right or bending forward). To achieve fast reaction on an approaching ball, the visual perception supplies the difference of the ball position between the last two images. The magnitude of this vector is interpreted as approaching speed. The goalkeeper does not react on small speeds. The type of the goalkeeper motion is determined by the intersection point of the moving ball direction and the goal line. At kick-off, the goalkeeper is placed in the goal. After a diving motion, it gets up and repositions itself in the goal while facing the opponent goal.

25. B. Soccer Behaviors (Cont) 8) Goalkeeping: Summary: The goalkeeper’s objective apparently is to keep the ball out of the own goal. It positions itself in the own goal on the line from own goal to ball, or, if the ball is not visible, on the line between the goals. Balls close to the robot let it react immediately and trigger a corresponding goalie motion to capture the ball. To achieve fast reaction on an approaching ball, the visual perception supplies an estimation of the ball velocity from two successive frames. The goalkeeper reacts on balls that have velocity larger than a fixed threshold. The type of the goalkeeper motion, i.e. bending down or diving to a side, is determined by the intersection point of the moving ball direction and the goal line.

26. C. Team Behaviors The importance of team behaviors is still low in the Humanoid League, as only two players per team have competed so far. In Bremen 2006, most teams assigned one player to keep the goal clear and used the other player as field player. In our team, the players share perceptions via wireless communication. The ball perceptions communicated by other players are used for search. For the soccer play with two field players, we implemented simple but effective role negotiation between the players. As soon as one of our players has control of the ball, the other player goes to a defensive position between the ball and the own goal. A player takes control of the ball, if it is close to the ball and perceived it with high confidence. It loses control, if the ball gets far away or has low confidence. The thresholds for taking and losing control implement hystheresis to prevent oscillations of the control state.

27. VIII. TACTICS AND TEAM BEHAVIORS The soccer behaviors so far make use of egocentric information obtained from visual perception. To implement team play and to act in an allocentric perspective, we introduced a next higher level in our behavior framework that abstracts from the reactive, egocentric soccer behaviors. On the basis of self-localization, tactical behaviors can configure the lower level soccer behaviors by setting ball-target and target pose in Fig. 6. An illustration for the positioning of offensive and defensive field players (green circles with orientation indicator): The offensive field player positions on the line from ball to target goal (yellow) with sagittal and lateral offsets behind the ball (orange circle) on the blue cross, such that the ball lies in front of the kicking leg. The defensive field player takes a defensive, supporting pose by positioning on the direct line from ball to own goal in a certain distance to the ball. Allocentric coordinates at a rate of 20.83Hz. To implement target-pose positioning, an additional behavior is necessary on the lower level that inhibits all other behaviors on its level. Additionally, the tactical behaviors can control, when the soccer behaviors are allowed to dribble or kick. For instance, dribbling can be preferred over kicking. In this way, special game tactics can be implemented. Also, the technical challenges of the RoboCup Humanoid League, i.e. obstacle avoidance, slalom dribbling around poles, and passing between two field players, could easily be implemented with this abstract interface.

28. VIII. TACTICS AND TEAM BEHAVIORS Another main concept on this level is the role of the player within the soccer team. A player can act either as offensive field player, as defensive field player, or as goalkeeper. If only one field player is on the field, it plays offensive. When the team consists of more than one field player, the field players negotiate roles by claiming ball control. While no player is in control of the ball, all players attack. If one of the players takes control, the other player switches to the defensive role. A player may claim ball control, if it is not already claimed, relative ball position and angle are within a certain threshold, and the player is positioned better behind the ball than the other field player. As soon as the relative ball position and angle exceed a larger threshold or the player detects a fall or performed a kick, it abandons ball control. To assess the positioning quality of the players, a positioning utility is calculated by each player and communicated via WLAN to the other. The positioning utility depends on the deviation from the relative target position behind the ball that is given in the current game situation. Another application of the role concept is goal clearance by the goalkeeper: the goalkeeper switches its role to field player, if the ball gets closer than a certain distance. In this case it starts negotiating roles with other field players like a standard field player.

29. VIII. TACTICS AND TEAM BEHAVIORS We implemented the following behaviors on this level, excluding special behaviors for the technical challenges: Role Assignment: The behavior implements default role negotiation and role switching as described above. Positioning: Positioning on a target pose is used for kick-off and for technical challenges. Ball Handling: This behavior contains default ball handling skills, i.e. dribbling the ball close to the goal instead. Final game of the TeenSize penalty kick tournament: NimbRo vs. Pal Technology. NimbRo won the exciting match 5:4. Fig. 8. Final game of the KidSize soccer tournament: NimbRo vs. Team-Osaka. NimbRo won the exciting match 8:6 of kicking, and dribbling the ball from the opponent’s field corner to the center of the opponent’s half. Dribble around Obstacles: If the ball lies in front of a close obstacle, the player dribbles the ball around the obstacle onto the side where the angle between goal and obstacle is largest. Dribble into Freespace: If an obstacle is located between the player and the ball- target in the opponent’s half, the player dribbles the ball onto the field side that is not occupied by the obstacle.

30. C. Team Behaviors The importance of team behaviors is still low in the Humanoid League, as only two players per team have competed so far. In Bremen 2006, most teams assigned one player to keep the goal clear and used the other player as field player. In our team, the players share perceptions via wireless communication. The ball perceptions communicated by other players are used for search. For the soccer play with two field players, we implemented simple but effective role negotiation between the players. As soon as one of our players has control of the ball, the other player goes to a defensive position between the ball and the own goal. A player takes control of the ball, if it is close to the ball and perceived it with high confidence. It loses control, if the ball gets far away or has low confidence. The thresholds for taking and losing control implement hystheresis to prevent oscillations of the control state.

31. VII. CONCLUSIONS The behavior control software for our 2006 KidSize robots, which successfully took part as team NimbRo at the RoboCup 2006 competitions. We implemented the control software in a framework that supports a hierarchy of reactive behaviors. A kinematic interface for body parts made it possible to abstract from individual joints when implementing basic skills like omnidirectional walking. These basic skills made it possible to abstract from body parts when implementing more complex soccer behaviors. At this player level, our humanoid robots are very similar to wheeled or four- legged soccer robots. Finally, at the team level, the players are coordinated through role negotiation. Playing soccer with humanoid robots is a complex task, and the development has only started. So far, there has been significant progress in the Humanoid League, which moved in its few years from remotely controlled robots to soccer games with fully autonomous humanoids which is the most dynamic RoboCupSoccer league. We expect to see the rapid progress continue as more teams join the league. Many research issues, however, must be resolved before the humanoid robots reach the level of play shown in other RoboCupSoccer leagues. For example, the humanoid robots must maintain their balance, even when disturbed.

32. VII. CONCLUSIONS (Cont) Currently, we are working on postural reflexes, which should minimize the number of falls [12]. In the next years, the speed of walking must be increased significantly. We work on automatic gait optimization to increase both speed and stability. At higher speeds, running will become necessary. The visual perception of the soccer world must become more robust against changes in lighting and other interferences. We continuously improve our computer vision software to make it more reliable. The 2006 competition has shown that most teams were able to kick penalties, but that soccer games are much richer and more interesting. In the team leader meeting after the competition, the majority voted for abandoning penalty kick as a separate competition. Instead, the KidSize teams will focus on soccer games. Unfortunately, most teams do not feel ready to increase the number of players to more than two per team. This limits the possibilities for team play. As the basic skills of the humanoid soccer robots improve every year, teams will be able to focus on the more complex soccer behaviors and on team play. This will make structured behavior engineering a key factor for success.

33. Behavior Programming for GT 2005

34. Motivation The Sony robots have four legs with 3 DOF each, and a head with 3 DOF. Instead of kicking with a single kicking device like in the middle or small sized league, this allows for a lot of different kicking skills using legs, body, or even head, which often require preparatory movements. Instead of moving on wheels many different styles of walking are used in different situations. With the introduction of Wireless LAN communication in the Sony League in 2002, cooperative strategies became more complex and consequently require adequately formulated high level behavior. For perception, the Sony robots need a set of perception behaviors, too. Because the field of vision, the image quality and the quality of the other sensors are very limited, information has to be collected over time, the movement of legs and head has to be coordinated with current vision needs and the perception process needs to be supported by methods like active vision.

35. Motivation (cont) Usually, the related behaviors have to be merged with the other movement skills. This huge set of abilities results in the need for a complex behavior control architecture that integrates many behavior patterns. It should be modular in the meaning that behavior patterns can be reused in different contexts. It has to support reactive and real-time decision making as well as long term deliberative behaviors. The set of behaviors needs to be easy to extend - adding new behaviors should not have side effects on other ones. It is found that C++ is not well suited for specifying agent behaviors. Especially extension and maintenance of complex behavior control systems may become a tedious and error prone task. High level behavior specification languages allow for a separation of the behavior design from the implementation of the agent platform.

36. Related Work According to the dynamics of soccer, the agents act with only very limited foresight. Most teams in RoboCup are using layered architectures, with comparatively reactive behaviors (basic skills) at the lowest level. Ordering different behaviors on layers allows to follow different goals in parallel. Behaviors on higher levels invoke or activate behaviors on lower levels. As long as the architecture has to manage only few basic behaviors, the separation of behaviors in two or three layers may be sufficient. It becomes very difficult to control more than a few basic behaviors without introducing further hierarchies, when their usage depends on a careful analysis of the situation, when they require complex preconditions to be achieved and when their performance needs a considerable amount of time.

37. Related Work (cont) There are other attempts to use behavior languages in order to simplify the process of behavior development. For example, GOLOG is an logic based robot control language. Funge developed the cognitive modeling language (CML) for the domain of computer games. Obst and Stolzenburg employ UML state charts for specifying multiagent systems. They follow a layered state machine approach with a fixed number of layers. They used UML because there exists a rich set of easy-to-adapt editors for editing state charts. State machines are well suited for behavior modelling. The decision which action is executed next depends not only on the environment but also on the last state. That allows to keep behaviors stable and to define hysteresis between two behaviors for avoiding oscillations when the sensor readings are noisy.

38. Agent Behavior with XABSL XABSL a flexible, open hierarchical behavior control architecture. It consists of state machines which manage the transitions to new behaviors according to the last state and the recent situation. In a flat architecture, the number of transitions between states increases very fast with the number of states. Therefore we use options to encapsulate a limited number of states and transitions according to their abstractness. The options form a rooted directed acyclic graph. XABSL is an XML based behavior language that allows to completely specify the behavior of autonomous agents. The development of a robot control includes the design of a hierarchy of options and the implementation of their internal state machines. XABSL supports both tasks using the advantages of XML technologies. The XABSL framework contains a variety of visualization and debugging tools. The runtime system XabslEngine executes the behaviors written in XABSL. XABSL allows the efficient integration of program parts from different groups. It is possible to develop a new robot control with about 50 different behaviors in only two weeks.

39. The Architecture behind XABSL an agent consists of a number of behavior modules called options. The options are ordered in a rooted directed acyclic graph, the option graph, which may be expanded into a tree. The terminal nodes of that graph are called basic behaviors which generate the actions of the agent and are associated with basic skills. The task of the option graph is to activate and parameterize one of the basic behaviors, which is then executed. Beginning from the root option, each active option has to activate and parameterize another option on a lower level in the graph or a basic behavior. Within options, the activation of behaviors on lower levels is done by state machines. Each state has a subsequent option or a subordinated basic behavior. There can be several states that have the same subsequent option or basic behavior. Each state has a decision tree with transitions to other states at the leaves. The decision is based on the world state and other sensory information and messages from other agents can also be used. As timing is often important, the time how long the state is already active and the time how long the option is already active can be taken into account. Additionally, each state can set special requests, that influence the information processing or determine how and where the robot should point its camera.

40. Option graph of a simplified Goalie Boxes denote options, ellipses denote basic behaviors. The edges show which other option or basic behavior can be activated from within an option. The internal state machine of the option ”goalie-playing”. Circles denote states, the circle with the two horizontal lines denotes the initial state. An edge between two states indicates that there is at least one transition from one state to the other. The dashed edges show which other option or basic behavior becomes activated when the corresponding state is active.

41. The decision tree of the state ”get-to-ball” Graphical notation: The leaves of the tree are transitions to other states. The dashed circle denotes a transition to the own state. Pseudo code of the decision tree. Note that both charts were generated automatically from the XML source.

42. Behavior Specification in XML The GermanTeam found that C++ is error prone and not very comfortable. The source code became very large and it was quite hard to extend the behaviors. Therefore the Extensible Agent Behavior Specification Language (XABSL) was developed to simplify the process of specifying behaviors: XABSL is an XML dialect specified in XML Schema. The reasons to use XML technologies instead of defining a new grammar from scratch were the big variety and quality of existing editing, validation and processing tools (many XML Editors are able to check if an XABSL document is valid at runtime), the possibility of easy transformation from and to other languages as well as the general flexibility of data represented in XML languages. Behaviors specified in XABSL can be easily visualized using XSLT and DotML. We have implemented language elements for options, their states, and their decision trees. Boolean logic (||, &&, !, ==, ! =, and >=) and simple arithmetic operators (+, −,,/ and %) can be used for conditional expressions. Custom arithmetic functions (e.g. distance−to(x, y)) that are not part of the language can be easily defined and used in instance documents.

43. Symbols are defined in XABSL instance documents to formalize the interaction with the software environment. Interaction means access to input functions and variables (e. g. from the world state) and to output functions (e. g. to set requests for other parts of the information processing). For each variable or function that shall be used for conditions a symbol has to be defined. This makes the XABSL framework independent from specific software environments and platforms. An example: The first symbol ”ball.x” simply refers to a variable in the world state of the agent, ”utility-for-dribbling” stands for a member function of an utility analyzer and ”goalie- should-jump-right” represents a complex predicate function that determines if a fast moving ball is headed to the right portion of the own goal. In options, these symbols then can be referenced.

44. The developer may decide whether to express complex conditions in XABSL by combining different input symbols with boolean and decimal operators or by implementing the condition as an analyzer function in C++ and referencing the function via a single input symbol. As the basic behaviors are written in C++, prototypes and parameter definitions have to be specified in an XABSL document so that states can reference them.

45. The figures 1 and 2 were generated automatically from the XML source shown above.

46. Software Environment An XABSL behavior specification can be distributed over many files. Different XML files for symbol definitions can be used, basic behavior definitions, predefined conditions, agents and options. This helps larger teams of behavior developers to work in parallel. It is easier to keep an overview over the whole agent and a version control system like CVS can be easily used. Tools for generating three different types of documents from an XABSL instance document set: – An Intermediate Code which is executed by the XabslEngine because on many embedded computing platforms (like Sony’s AIBO), XML parsers are not available due to resource and portability constraints. – Debug Symbols containing the names for all options, states, basic behaviors and symbols make it possible to implement platform and application dependent debugging tools for monitoring option and state activations as well as input and output symbols. – An extensive auto-generated HTML-documentation containing SVG-charts for each agent, option and state which helps the developers to understand what their behaviors do. See XABSL source file and the XABSL web site for the language reference, the XML schema files and examples.

47. The Runtime System XabslEngine For running the compiled behavior on a target agent platform, the runtime environment XabslEngine has been developed. The engine is meant to be platform and application independent and can be easily employed on other robotic platforms. This results in a variety of abstract helper classes that have to be adapted to the current software environment. The XabslEngine parses and executes the intermediate code. It links the symbols from the XML specification that were used in the options and states to the variables and functions of the agent platform. Therefore, for each used symbol an entity in the software environment has to be registered to the engine. The following example connects the C++ variable worldState.ballPosition.x to the XABSL symbol ”ball.x”: myEngine.registerDecimalInputSymbol("ball.x", &worldState.ballPosition.x); While options and their states are represented in XML, basic behaviors are written in C++. They have to be derived from a common base class and registered to the engine. The engine provides extensive debugging interfaces to be able to monitor the option and state activations, the values of the symbols and the parameters of options and basic behaviors. Instead of executing the engine from the root option, single options or basic behaviors can be tested separately. A complete documentation of the engine, along with the code, can be found at the XABSL web site [14].

48. The hierarchical constitution of XABSL allows it to make many both very short-term and reactive decisions and more deliberative and long-term decisions co-instantaneous. The lower behaviors in the option hierarchy that are responsible for ball handling react instantly on changes in the environment. The more high-level behaviors like waiting for a pass, positioning or role changes try to prevent frequent state changes to avoid oscillations. The GermanTeam implemented over 50 different options in Fukuoka. About 10 team members were involved in developing and tuning the behaviors. The modular approach of XABSL made it easy to extend or advance the behaviors. New options could easily be added to existing ones without having negative side effects. Better solutions of existing options could be developed in parallel and were easily to compare with the previous ones. To help behavior control developers, an example agent was implemented for the Ascii Robot Soccer environment. In this simple soccer simulation by Tucker Balch the field is displayed in a 78 characters long and 21 lines wide text terminal. A team of four ”>” players plays against a team of four ”<” players with an ”o” as the ball. All agents retrieve the full information about the world and the set of possible actions is very limited. This makes the implemented XABSL agent simple and easy to understand. The example implementation containing the XabslEngine and the visualization tools can also be downloaded from the XABSL web site.

49. Scenes from a video of a round robin game against the team Georgia Tech (4:1) in Fukuoka: a) Use of communication. The first forward (1) performs a bicycle kick directed to the opponent goal. The second forward (2) was told to wait in front of the opponent goal to be able to help if the kick fails. b) Positioning. The second forward (1) tries to dribble the ball into the opponent half. The defender (2) stays behind it to support the forward. The first forward (3) waits in the opponent half for a pass.

50. Application: XABSL is used for the Aibo German Team. Later on this approach was chosen for the participation of the GermanTeam in the RoboCup 2002 in Fukuoka [9]. In the competitions the GermanTeam won all its games except against the later finalists from CMU and UNSW. The strength of the team was based on a big set of different behavior patterns. For instance the players employed over 16 different kicks in different situations. Amongst them the bicycle kick is a good method for getting the ball behind the player without previously turning around the ball. (cf. Fig. 4a) All these kicks require different behaviors for approaching the ball. Some work better for bigger ball distances, some require to grab the ball with the both front legs. Varying ball handling behaviors were chosen depending on whether the ball was in the opponent half, in the own half, at the left border, at the right border or in front of the opponent goal. XABSL proved to be suitable for implementing and integrating all these different abilities. On higher levels, a set of team strategies based on communication was implemented.

51. As it is often disadvantageous when two players try to obtain the ball the robots negotiated which of them handles the ball and which stays behind or waits for a pass. The state based architecture of XABSL simplifies the developing of such strategies. Each robot sends its option and state activations to all other robots so that all players know what the others plan to do. However, since the wireless communication is not always reliable, all strategies have to be able to resort to non- communicative behavior, when necessary. Complex positioning strategies were also employed. Each player had to care for an area of responsibility which changed depending on the score, the number of own players and the distribution of opponent players on the field. Although XABSL is a state based architecture, continuous approaches can easily be integrated into the behaviors. A potential field was employed to determine an optimal dribbling direction. This direction was made available to the options by an input symbol. A Fuzzy Logic based basic behavior for approaching the ball was implemented. Several options used continuous utility models for state transitions.

52. 52 Behavior Programming Specific behavior description languages prove to be suitable replacements to native programming language like C++ when the number and complexity of behavior patterns of an agent increases. The XML based Extensible Agent Behavior Specification Language (XABSL) simplifies the process of specifying complex behaviors and supports the design of both very reactive and long term oriented behaviors. XABSL uses hierarchies of behavior modules called options that contain state machines for decision making. Introduce the architecture behind XABSL, the formalization of that architecture in XML and the software library XabslEngine that runs the formalized behavior on an agent platform. The GermanTeam employed XABSL in the RoboCup Sony Four Legged League competitions in Fukuoka.

53. 3.8 Behavior Control The module BehaviorControl is responsible for decision making based on the world state, the game control data received from the RoboCup Game Manager, the motion request that is currently being executed by the motion modules, and the team messages from other robots. It has no access to lower layers of information processing. It outputs the following: A motion request that specifies the next motion of the robot, a head motion request that specifies the mode how the robot’s head is moved, a LED request that sets the states of the LEDs, a sound request that selects a sound file to be played by the robot’s loudspeaker, a behavior team message that is sent to other players by wireless communication. This section describes in detail the implemented strategies and behaviors of the GermanTeam 2005. Those who are not familiar with the XABSL language should probably read appendix D first. The behaviors which the GermanTeam developed in XABSL for the RoboCup championships 2005 in Osaka are distributed among about 80 options. Figure 3.27 shows the option graph of the soccer related behaviors. In general, the lower behaviors in the option hierarchy such as ball handling or navigation, have to react instantly on changes in the environment and are therefore very short-term and reactive. The more high-level behaviors such as waiting for a pass, positioning, or role changes

54. The behaviors which the GermanTeam developed in XABSL for the RoboCup championships 2005 in Osaka are distributed among about 80 options. Figure 3.27 shows the option graph of the soccer related behaviors. In general, the lower behaviors in the option hierarchy such as ball handling or navigation, have to react instantly on changes in the environment and are therefore very short-term and reactive. The more high-level behaviors such as waiting for a pass, positioning, or role changes try to prevent frequent state changes to avoid oscillations and make more deliberative and longterm decisions. This section describes from bottom to top how the GermanTeam’s robots play soccer starting with basic capabilities and finishing with the high-level team strategies. An extensive automatically generated HTML documentation of these behaviors can be found at http://www.ki.informatik.hu-berlin.de/XABSL/examples/gt2005/. It is recommended to use this site as an additional source to this Section.

55. 3.8.1 Ball Handling 3.8.1.1 Approaching “go-to-ball” behavior is responsible for walking to the ball, i.e. for ball approaching and dribbling For the use in different contexts it provides following parameters: First, the body of the robot is always directed to the ball, restricted by the parameter “max-turn-speed”. The maximum walk speed is given by the parameter “max-speed”, making higher options responsible for slowing down near the ball. The “max-speed.y” parameter restricts the sideward component, allowing for sprinting with the “dash” walk type. For dribbling and the “turn kick”, “y-offset” specifies a y offset with that the robot shall arrive at the ball. If the robot is very close to the ball and if the ball is much to the left or right, the translation component is almost completely inhibited, making the robot only turn in order to avoid pushing the ball away with the front legs.

56. The “go-to-ball” basic behavior use option “approach-ball”: “search-auto”: when robot is far away from the ball. Here the head-control mode is set: the head of the robot look at the ball and (frequently, always after a certain time of consecutively perceived balls) shortly look around for landmarks and obstacles to improve self-localization. If robot gets closer to the ball than specified in the option parameter “look- at-ball-distance”, in state “search-for- ball” the head control mode is set to “search-for-ball”. This lets the head look at the ball only. To avoid frequent changes between these two states, there is a distance hystheresis of 5 cm between them. In both states, the option “approach- ball-set-walk-speed”, which controls the walk speed, is referenced. Figure 3.28: Option “approach-ball” controls the head movements while approaching the ball and handles collisions and ball losses.

57. “search-for-ball”: head scans are disadvantageous near the ball. The head control mode is set to “search-for-ball”: Head look at the ball only. If the ball was not seen for (1) 1.3 seconds in the “search-auto” state, (2) 400 ms in the “search- for-ball” state, then in state “ball-not-seen” the option “turn-for-ball” tries to redetect the ball. If the ball is seen again, the state “ball-just-found” remains active for 2 seconds, setting the head control mode “search-for-ball” in order to avoid further ball losses due to scanning around with “search-auto”.

58. Option “approach-ball-set-walk-speed” controls the speed of ball approaching. It is only used by option “approach-ball”. In state “fast”, the basic behavior “go-to-ball” is executed with a fixed speed of 350 cm per second. If the robot gets closer to the ball than specified in parameter “slow-down-distance” (minus 2,5 cm hysteresis), in state “slow” the speed given in “slow- speed” is passed to “go-to-ball”. From the “fast” state, if the ball is farer away than 1200 cm and if the angle to the ball is between plus and minus 7 degrees, state “dash” becomes active. There “go-to-ball” is executed with walk type “dash”, a faster but not omnidirectional walking gait. The ball approaching stops immediately after the ball was not seen for a certain time. In this case, “approach-ball” references the option “turn-for-ball” to redetect the ball.

59. Option “approach-ball-set-walk-speed” controls the speed of ball approaching. b) Option “turn-for-ball” tries to redetect a previously lost ball.

60. The ball approaching stops immediately after the ball was not seen for a certain time. In this case, “approach-ball” references the option “turn-for-ball” to redetect the ball. In the initial state “ball-not-seen”, the basic behavior “stand” is executed. Note that “stand” does not stop walking immediately but continuously slows down in order to avoid bumpy movements if the ball is redetected fast. As “turn-for-ball” can be activated from different contexts and situations, the time how long state “ball-not-seen” remains active depends on how long the ball was not seen and where it was seen last. The state remains active for at least 800 ms which are needed for “stand” to almost slow down. As long as the ball was seen 1.7 seconds before, “ball-not-seen” keeps active to give the head control a chance to make a complete scan around.

61. If the ball was seen in the last 5 seconds and in the near, it is very likely that the ball is at the side of the robot. Therefore, in state “back-up” the robot walks backward for1.5 seconds to redetect the ball. If that fails (or from “ball-not-seen” if all other conditions fail), the state “ball-not-seen-left” or “ball-not- seen-right” gets active, depending on whether the ball was previously seen left or right. The robot turns around using the “walk” basic behavior. The head control mode is set to “look-left” or “look-right”, letting the robot look into the direction of turning. Although the “turn-for-ball” option is not activated anymore from “approach-ball” when the ball is redetected, state “ball- seen” becomes active when the ball is seen again, turning the robot to the ball.

62. Figure 3.30: Option “approach-and-turn” dribbles the ball into given direction by pushing the ball with the chest or pulling it around with the front legs.

63. Figure 3.31: Option “grab-ball-with-head” grabs the ball with the head.

64. 3.32: Option “carry-ball-fast” grabs and walks with the ball while touching and guiding it with the head or the mouth. Optionally a kick can be performed at the end oft the option.

65. Figure 3.33: Option “turn-and-release” grabs the ball and pulls it around. The actual job of lifting the head in the right moment is done in the head control.

66. Figure 3.34: a) The kick selection table for the goalie when the desired kick direction is “forward” (in the sector between -15 and 15 degrees). If the current position of the ball is in the outer blue areas, the “left-paw” or “right-paw” kick is selected, in front of the robot (red area), kick “chest-strong”, and in a narrow range more distant in front of the robot (brown area) “forward-kick-hard”. b) Data recorded from kick experiments for the “left-paw” kick. The dots mark the position where the ball was perceived before the kick started. The lines out of the dots indicate in which direction and how far the ball was kicked in the experiment. All kick experiments in that the ball was kicked into the “forward” sector are highlighted blue. The area for “put-left” in a) was defined by taking these highlighted entries into account. c) The goalie kick selection table for the sector between 45 and 75 degrees. For the ball to the very left (purple area), “put-left” is selected, close to the robot (red area) “hook-left”, and in the brown area “head-left”. d) Kick experiments for the “head-left” kick, with those entries highlighted where the ball was kicked into the direction between 45 and 75 degrees.

67. Figure 3.35: Option “handle-ball” selects between different behaviors for different zones on the field.

68. Figure 3.36: a) Option “get-to-position-and- avoid-obstacles” walks to a position avoiding obstacles on the way there. b) On top of that, “get-to- ball-and-avoid-obstacles” walks to the ball.

69. Figure 3.37: Option “position-supporter- near-ball” positions the robot near the ball. The speed is controlled depending on the reliability of the ball position.

70. Figure 3.39: The options a) “new-playing- offensive-supporter” and b) “new-playing- defensivesupporter” decide where to position the robot.

71. Figure 3.40: Option “playing-supporter- msh” intercepts kicks from the own team and blocks kicks of the other team.

95. References [1] R. C. Arkin. Motor schema-based mobile robot navigation. TheInternational Journal of Robotics Research, 8(4), 1989. [2] R. C. Arkin. Behavior-Based Robotics. MIT Press, 1998. [3] B. Bakker and J. Schmidhuber. Hierarchical reinforcement learning based on subgoal discovery and subpolicy specialization. In Proceedings ofIAS-8, pages 438-445, 2004. [4] T. Balch. The ascii robot soccer homepage, 1995. http://www- 2.cs.cmu.edu/-trb/soccer/.http://www- [5] R. A. Brooks. A robust layered control system for a mobile robot. IEEE Journal of Robotics and Automation, 2(1):14-23, 1986. [6] R. A. Brooks. The behavior language; user's guide. Technical Report AIM- 1127, MIT Artificial Intelligence Lab, 1990. [7] R. A. Brooks. Intelligence without representation. Artificial Intelligence, 47:139-159, 1991. [8] T. Buchheim et al. Team description paper 2004 CoPS stuttgart. In RoboCup 2004: Robot Soccer World Cup VIII, LNAI. Springer, 2005. [9] M. Fujita and H. Kitano. Development of an autonomous quadruped robot for robot entertainment. Autonomous Robots, 5(1):7-18, 1998. [10] J. D. Funge. Artificial Intelligence For Computer Games: An Introduction. AK Peters, Ltd, 2004. [11] E. Gat. Integrating planning and reacting in a heterogeneous asynchronous architecture for controlling real-world mobile robots. In Proceedings AAAI-92, pages 809-815. MIT Press, 1992. [12] H. Kitano, M. Asada, Y Kuniyoshi, I. Noda, and E. Osawa. RoboCup: The robot world cup initiative. In Proceedings of Agents'97, pages 340- 347. ACM Press, 1997. [13] K. Konolige. COLBERT: A language for reactive control in Sapphira. In KI-97: Advances in Artificial Intelligence, number 1303 in LNAI, pages 31-52. Springer, 1997.

96. References [13] K. Konolige. COLBERT: A language for reactive control in Sapphira. In KI-97: Advances in Artificial Intelligence, number 1303 in LNAI, pages 31-52. Springer, 1997. [14] H. J. Levesque, R. Reiter, Y Lesperance, F. Lin, and R. B. Scherl. GOLOG: A logic programming language for dynamic domains. Journal of Logic Programming, 31(1-3):59-83, 1997. [15] M. Loetzsch, M. Risler, and M. Juingel. XABSL web site, 2006. http://www.informatik.hu- berlin.de/ki/XABSL. [16] M. Lotzsch. XABSL - a behavior engineering system for autonomousagents. Diploma thesis. Humboldt-Universitat zu Berlin, 2004. Available online: http://www.martin- loetzsch.de/papers/diploma-thesis.pdf. [17] M. Lotzsch, J. Bach, H.-D. Burkhard, and M. Jiingel. Designing agent behavior with the extensible agent behavior specification language XABSL. In RoboCup 2003: Robot Soccer World Cup VII, volume 3020 of LNAI, pages 114-124. Springer, 2004. [18] D. MacKenzie, R. Arkin, and J. Cameron. Multiagent mission specification and execution. Autonomous Robots, 4(1):29-52, 1997. [19] P. Maes. Situated agents can have goals. In Designing Autonomous Agents: Theory and Practice from Biology to Engineering and Back, pages 49-70. MIT Press, 1990. [20] D. McDermott. Pddl the planning domain definition language, 1998. [21] S. Rabin, editor. AI Game Programming Wisdom, volume 2. Charles River Media, Inc., 2002. [22] T. Rofer et al. Germanteam 2005. In RoboCup 2005: Robot Soccer World Cup IX, LNAI. Springer, 2006. To appear. [23] S. Russel and P. Norvig. Artificial Intelligence, a Modern Approach. Prentice Hall, 1995. [24] T. Ziemke. Rethinking grounding. In Understanding Representation in the Cognitive Sciences - Does Representation Need Reality?, pages 177-190. Plenum Press, 1999.

97. Summary R-type components are: –Instruction Memory (IM) –Register File (RF) –Arithmetic and Logic Unit (ALU) R-type requires 4 clocks (IM, RF, ALU, and WB) Control is generated based on instruction type (OPCODE) Prepare for next lecture –Examine the datapath components needed for Load/Store -type instruction –Interconnect the needed components –Build the datapath Load/Store –type Thank you. Any question…