Presentation is loading. Please wait.

Presentation is loading. Please wait.

Development of Humanoid Soccer Robots Dr Changjiu Zhou School of Electrical & Electronic Engineering Singapore Polytechnic

Similar presentations


Presentation on theme: "Development of Humanoid Soccer Robots Dr Changjiu Zhou School of Electrical & Electronic Engineering Singapore Polytechnic"— Presentation transcript:

1 www.robo-erectus.org Development of Humanoid Soccer Robots Dr Changjiu Zhou School of Electrical & Electronic Engineering Singapore Polytechnic zhoucj@sp.edu.sg www.robo-erectus.org Learning and Control of Biped Locomotion

2 www.robo-erectus.org Development of Humanoid Soccer Robots Introduction Biped Walking Cycles How to Control Biped Locomotion How to Plan/Learn Biped Gaits Biped learning by reinforcement Some Research Topics Outline

3 www.robo-erectus.org Development of Humanoid Soccer Robots Single Support Double Support Time Biped Gait (Frontal View) Biped Gait (Frontal Plane)

4 www.robo-erectus.org Development of Humanoid Soccer Robots Biped Gait (Sagittal Plane)

5 www.robo-erectus.org Development of Humanoid Soccer Robots Right Support Left Support Left-to-Right Transition Right-to-Left Transition Swing time completed Left foot touches down Right foot touches down Swing time completed Finite State Machine for Biped Walking Control

6 www.robo-erectus.org Development of Humanoid Soccer Robots In static walking, the biped has to move very slowly so that the dynamics can be ignored. The bipeds projected center of gravity (PCOG) must be within the supporting area. Single Support Double Support Static Walking

7 www.robo-erectus.org Development of Humanoid Soccer Robots In dynamic walking, the motion is fast and hence the dynamics cannot be negligible. In dynamic walking, we should look at the zero moment point (ZMP) rather than PCOG. The stability margin of dynamic walking is much harder to quantify. Dynamic Walking

8 www.robo-erectus.org Development of Humanoid Soccer Robots Unpowered DOF between the foot and ground This constraint limits the trajectory tracking approaches used commonly in manipulators research. Why is Biped Robotics Hard?

9 www.robo-erectus.org Development of Humanoid Soccer Robots Inverse kinematics model Feet position and ZMP (PCOG) Desired joint angles Biped Robot Biped Control: Model-based

10 www.robo-erectus.org Development of Humanoid Soccer Robots Except for certain massless leg models, most biped models are nonlinear and do not have analytical solutions. Massless leg model is the simplest model. The body of the robot is usually assumed to be point mass and can be viewed to be an inverted pendulum. When the leg inertia and other dynamics like that of the actuator, joint friction, etc. are included, the overall dynamic equations can be very nonlinear and complex. Biped Control: Model-based

11 www.robo-erectus.org Development of Humanoid Soccer Robots Example: Massless leg model The simplest biped model Some assumptions, e.g., From DAlemberts principle

12 www.robo-erectus.org Development of Humanoid Soccer Robots Since none of the humanoid robots match biological humanoids in terms of mobility, adaptability, and stability, many researchers try to examine biological bipeds so as to extract certain algorithms that are applicable to the robots. Reverse Engineering Biped Control: Biologically Inspired

13 www.robo-erectus.org Development of Humanoid Soccer Robots 1. Central Pattern Generators (CPG) 2. Passive Walking Two Main Research Areas Biped Control: Biologically Inspired

14 www.robo-erectus.org Development of Humanoid Soccer Robots ZMP-based Gait Planning Plan the hip and ankle trajectories according to walking constraints and ground constraints. Derive all joint trajectories by inverse kinematics.

15 www.robo-erectus.org Development of Humanoid Soccer Robots Example: Gait Planning for Walking on Slope - Plan gait using 3rd order Spine which guarantees the continuity of both 1st derivative and 2nd derivative.

16 www.robo-erectus.org Development of Humanoid Soccer Robots Example: Planning Results Consecutive walking gait along slope Joint angles

17 www.robo-erectus.org Development of Humanoid Soccer Robots IP-based Gait Planning The dynamic equation of the IP model L v 2wf If the angle is small, it can be simplify as a linear homogeneous 2nd order differential equation

18 www.robo-erectus.org Development of Humanoid Soccer Robots 3D Linear Pendulum Model

19 www.robo-erectus.org Development of Humanoid Soccer Robots Example: IP-based Gait Planning

20 www.robo-erectus.org Development of Humanoid Soccer Robots Biped Kicking Kicking constraints: –Kicking range –Friction –…

21 www.robo-erectus.org Development of Humanoid Soccer Robots Kicking Pattern

22 www.robo-erectus.org Development of Humanoid Soccer Robots A humanoid robot aims to select a good value for the swing leg parameters for each consecutive step so that it achieves stable walking. A reward function that correctly defines this objective is critical for the reinforcement learning. Unstable r = -1 (punishment) Supporting foot Stable r = 0 (reward) Biped Learning by Reinforcement (1)

23 www.robo-erectus.org The control objective of the gait synthesizing for biped dynamic balance can be described as To evaluate biped dynamic balance in the frontal plane, a penalty signal should be given if the biped robot falls down in the frontal plane Biped Learning by Reinforcement (2)

24 www.robo-erectus.org Development of Humanoid Soccer Robots Supporting foot Excellent Good OK Bad Very Bad Reinforcement Learning with Fuzzy Evaluative Feedback Biped Learning by Reinforcement (3)

25 www.robo-erectus.org Development of Humanoid Soccer Robots Both the AEN and ASN are initialized randomly. Learning starts from scratch. It needs a large number of trials for learning.  AEN - the action-state evaluation network  ASN - the action selection network  SAM - the stochastic action modifier The RL Agent

26 www.robo-erectus.org Development of Humanoid Soccer Robots Neural fuzzy networks are used to replace the neuron-like adaptive elements. The expert knowledge can be directly built into the FRL agent as a starting configuration. The ASN and/or AEN could house available expert knowledge to speed up its learning. The FRL Agent

27 www.robo-erectus.org Development of Humanoid Soccer Robots The numerical evaluative feedback is not the biological plausible. The fuzzy evaluative feedback is much closer to the learning environment in the real world. The fuzzy evaluative feedback is based on a form of continuous evaluation. The FRL Agent with Fuzzy Evaluative Feedback

28 www.robo-erectus.org Development of Humanoid Soccer Robots TypesAction Network (ASN) Critic Network (AEN) Evaluative Feedback RL agentneural numerical FRL agent (Type A) neuro-fuzzyneuralnumerical FRL agent (Type B) neuro-fuzzy numerical FRL agent (Type C) neuro-fuzzy Fuzzy Comparison of FRL Agents

29 www.robo-erectus.org Development of Humanoid Soccer Robots Information Available for Biped Gait Synthesizing The Description of the Information Case ANo expert knowledge is available. Only numerical reinforcement signal is used to train the gait synthesizer. Case BOnly the intuitive biped balancing knowledge is used as the initial configuration of the gait synthesizer. Case CBoth the intuitive biped balancing knowledge and walking evaluation knowledge are utilized. Case DBesides all the information used in case C, the fuzzy evaluative feedback, rather than numerical evaluative feedback, is included.

30 www.robo-erectus.org Development of Humanoid Soccer Robots The Gait Synthesizer Using Two Independent FRL Agents

31 www.robo-erectus.org Development of Humanoid Soccer Robots Ankle jointKnee joint Before and After Learning

32 www.robo-erectus.org Development of Humanoid Soccer Robots The ZMP trajectory after FRL (type C) Results (1)

33 www.robo-erectus.org Development of Humanoid Soccer Robots Results (2) Walk (Backward)

34 www.robo-erectus.org Development of Humanoid Soccer Robots Some Research Topics Online gait generating Online footprint planning Constraints –ZMP constraint for stable walking –Friction constraint for stable walking –… Current Challenges –Knee bending –Body shifting –… …

35 www.robo-erectus.org Development of Humanoid Soccer Robots References C. Zhou, Robot learning with GA-based fuzzy reinforcement learning agents, Information Sciences 145 (2002) 45-68. C. Zhou, Fuzzy-arithmetic-based Lyapunov synthesis to the design of stable fuzzy controllers: a computing with words approach, Int. J. Applied Mathematics and Computer Science 12(3) (2002) 101-111. C. Zhou and Q. Meng, Dynamic balance of a biped robot using fuzzy reinforcement learning agents, Fuzzy Sets and Systems 134(1) (2003) 169-187. C. Zhou, P.K. Yue, Z. Tang and Z. Sun, Development of Robo-Erectus: A soccer-playing humanoid robot, Proc. IEEE-RAS Intl. Conf. on Humanoid Robots, CD-ROM, 2003. Z. Tang, C. Zhou and Z. Sun, Gait synthesizing for humanoid penalty kicking, Dynamics of Continuous, Discrete and Impulsive Systems, Series B, (2003) 472-477. D. Maravall, C. Zhou and J. Alonso, Hybrid fuzzy control of inverted pendulum via vertical forces, Int. J. of Intelligent Systems, 2004 (in press).

36 www.robo-erectus.org Development of Humanoid Soccer Robots Acknowledgements Staff Member P.K. Yue, F.S. Choy, Nazeer Ahmed M.F. Ercan, Mike Wong, H. Li Research Associate Z. Tang (Tsinghua U.), J. Ni (Shanghai Jiao Tong U.) Technical Support Officer H.M. Tan, W. Ye Students P.P. Khing, H. W. Yin, H.F. Lu, H.X. Tan, J.X. Teo, Stephen Quah, H.M. Tan, Y.T. Tan

37 www.robo-erectus.org Development of Humanoid Soccer Robots Thanks! Dr Changjiu Zhou School of Electrical and Electronic Engineering Singapore Polytechnic zhoucj@sp.edu.sg www.robo-erectus.org


Download ppt "Development of Humanoid Soccer Robots Dr Changjiu Zhou School of Electrical & Electronic Engineering Singapore Polytechnic"

Similar presentations


Ads by Google