RoboSina from Scratch Mostafa Rafaie-Jokandan & Nima Kaviani By Mostafa Rafaie-Jokandan & Nima Kaviani RoboSina from Scratch Submitted as partial Fulfillment for the Requirements of the degree of Bachelor of Science In Software Engineering

Introduction Simulation System and Soccer Server Learning Algorithms used in RoboSina Prominent Aspects of RoboSina RoboSina Agent and Learning Suggestions and Conclusion

Introduction Why RoboCup? Where are we moving toward? An approach to Distributed Artificial Intelligence DAI vs CAI Where are we moving toward? Who were we in RoboSina?

Simulation System & Soccer Server What is a Simulation System? Why Simulation? Soccer Server

Soccer Server Match Rules Properties of the Simulated Pitch Controlled by Soccer Server Controlled by Human Properties of the Simulated Pitch A 105.0 * 68.0 field covered by flags Goal widths are twice a real one

Some of predefined parameters for Soccer Server

A Simulated Field

Models used in Soccer Server Movement Model Visual Model Aural Model

Visual Model (see ObjName Distance Direction DistChng DirChng BodyDir HeadDir) ObjName ::= (p ”Teamname” UniformNumber goalie) | (g [l|r]) | (b) | (f c) | (f [l|c|r] [t|b]) | (f p [l|r] [t|c|b]) | (f g [l|r] [t|b]) | (f [l|r|t|b] 0) | (f [t|b] [l|r] [10|20|30|40|50]) | (f [l|r] [t|b] [10|20|30]) | (l [l|r|t|b])

Visual Model Visual Formulas Noise Formulas prx = pxt − pxo pry = pyt − pyo vrx = vxt − vxo vry = vyt − vyo Direction = arctan(pry/prx) − ao erx = prx/Distance ery = pry/Distance DistChng = (vrx ∗ erx) + (vry ∗ ery) DirChng = [(−(vrx ∗ ery) + (vry ∗ erx))/Distance] ∗ (180/π) BodyDir = PlayerBodyDir − gentBodyDir − AgentHeadDir HeadDir = PlayerHeadDir − AgentBodyDir − AgentHeadDir Noise Formulas d’ = Quntize ( exp(quantize(log(d) . quantize – step )), 0.1) Quantize (V , Q) = ceiling(V/Q) * Q

Visual Model 90 45

Aural Model (hear Time Sender ”Message”)

Movement Model

Movement Noise Model

Actions for an Agent Catch Dash & Stamina Model Kick Tackle Turn Move Turn-Neck Point to Focus Say Change View Score

Heterogeneous Players Do you know how a body builder differs a normal human? Acting in an environment with heterogeneous creatures covers all possible properties Coach Mapping of DAI to CAI with restricted bandwidth in communication What was the role of Jose Morinio in a team if we didn’t have a coach?!!!

Learning Algorithms Definition Decision Trees Artificial Neural Networks

Learning Algorithms-Keywords Environment Property State Goal Learning Algorithm

Learning Algorithms-Keywords State2 Propertyi Goal State State1 State3

Decision Trees Definition ID3 Basis ID3 Formulas ID3 Algorithm What is C4.5

Decision Trees - Definition A powerful tool for Inductive Inference Proposing a model for discrete environments Resistance against Noisy Data

in case of having a weather Playing a Soccer Game in case of having a weather with conditions D ? ? D=( Outlook = Sunny , Wind = Weak, Humidity = High )

Decision Trees – ID3 Basis Occam’s Razor, mid of 14th century “It is vain to do with more what can be done with less… entities should not be multiplied beyond necessity”

Decision Trees – ID3 Formulas

Decision Trees – ID3 Algorithm

Decision Trees – C4.5 A developed software on the basis of ID3 It’s abilities Avoiding over fitting the data Reduced error pruning Rule post-pruning Handling continuous attributes Handling attribute with different costs Choosing an appropriate selection measure Handling training data with missing attribute values Improving computational efficiency

Neural Networks Definition A Mathematical Representation BackPropagate Basis BackPropagate Algorithm

Neural Networks-Definition A mathematical model of human’s neurons It’s abilities Association, Clustering, Classification Pattern Recognition Generalization, Reference Improvement Classification on the basis of Learning Algorithm Supervised Unsupervised => Competitive

Neural Networks- Math Representation

Neural Networks- BackPropagate Basis A supervised learning algorithm Works on the basis of revising Weight Values Training Example (x, t) where x represents the entry, and t is the expected output value ai(l) representes the output gerenrated by the network in the layer named l x = ai(0) m is the frequency for the execution of learning algorithm

Neural Networks- BackPropagate Algorithm

Prominent Aspects System Architecture Localizing an Agent Intercept Dribble Formation Defense System Play With Ball System

System Architecture Describe the structure of the system's components Functional Architecture Operational Architecture Implementation Architecture Brief view to Functional Architecture

System Architecture-Functional Architecture

Localizing an Agent What is localization? f

Localizing an Agent A new approach used It deals with points rather than edges Uses a swipe line to find active edges A point will be added if : It is a result of an intersection It is surrounded by both active edges of the other polygon Stops after reaching the end point of a polygon It is executed in O(m+n) for 2 polygons T(n) = 2T(n/2) + O (n) It is executed in O(nlogn) using divide and conquer for n polygons

Intercept As a skills As a strategy utility Intercept the ball in at any distance. As a strategy utility Determine ball owner Predict ball owner Determine who must be intercept ball. Need to communication to enhance its efficiency

Dribble Running with the ball from one point to another as quickly as possible Misleading obtrusive defender Holding ball if it is need .

Dribble One , Two

Dribble Mislead

Formation Usage New Works Determine Agent position relative to the ball position New Works New Floating Formation Adding A Free midfielder Implementing the main formation as 4-2-3-1 Implementing forwards press.

New Floating Formation 4 2 3 1 The 4-2-3-1 formation is the main formation.

New Floating Formation 4 4 2 The 4-2-4 formation is a fully offensive formation.

Defense System Skills Block Mark Strategy Offside line Press

Defense System- Block

Defense System- Mark

Defense System- Offside Line NO, offside!!!

NO, I lost the ball !!

Play With Ball System Coming soon !!! It will be published within early days.

RoboSina Agent & Learning Agent learns to Shoot Neural Network and training a shoot skill Using BP and Matlab Agent learns to Pass Decision Trees and training a pass skill Using ID3 and C4.5

Learning how to Shoot Problem Definition Mapping the environment to a discrete model 6 input entries 6 output entries Grabbing relevant properties Using trainer to collect training samples Choosing the proper target to shoot Making the appropriate Neural Network Modeling the network with matlab A 2 layer network tgsig as activation function in the first layer logsig as activation function in the second layer Applying training samples to the network Results

Failed (gx, gy) (bx, by) distcb dircb 1st Successful State 1st Successful State State 19 = 100111 State 24 = 110001 Last Successful State State 28 = 111001

Shoot-Results Harms Very time consuming method!! Algorithm Occurrence in 8 matches Success Num Success Percentage Computational 141 93 65.9% Learning 173 126 72.8% Harms Very time consuming method!!

Learning how to Pass Problem Definition Grabbing relevant properties Using trainer to collect training samples Making the appropriate Decision Tree using C4.5 to create the tree Creating pass.names file applying training samples to the tree Creating pass.data file Results

Pass-Results Harms Partial Observation causes problems Pass Probability Occurrence in 8 matches Success Num Success Percentage 85-100% 912 753 78.3% 70-85% 328 187 57.01% 55-70% 168 92 54.7% Harms Partial Observation causes problems Body direction of the receiver is ignored during the train

RoboSina Project Honors Championship of the 3rd US-Open RoboCup Competitions, May 2005, Atlanta, USA 2nd place of the 3rd Iranian Open RoboCup Competitions, April 2005, Tehran, Iran 5th place of RoboCup World Cup 2004, Lisbon, Portugal Championship of the 2nd US-Open RoboCup Competitions, May 2004, New-Orleans, USA Championship of the 2nd Iranian Open RoboCup Competitions, March 2004, Tehran, Iran 10th place RoboCup World Cup 2003, Padova, Italy

Conclusions & Suggestions

Acknowledgements

Thank you very much for being here Any Question? We are ready to Answer