Presentation is loading. Please wait.

Presentation is loading. Please wait.

Modelling CGFs for tactical air-to-air combat training Motivation-based behaviour and Machine Learning in a common architecture Jan Joris Roessingh, Ph.D.,

Similar presentations


Presentation on theme: "Modelling CGFs for tactical air-to-air combat training Motivation-based behaviour and Machine Learning in a common architecture Jan Joris Roessingh, Ph.D.,"— Presentation transcript:

1 Modelling CGFs for tactical air-to-air combat training Motivation-based behaviour and Machine Learning in a common architecture Jan Joris Roessingh, Ph.D., Maj. Roel Rijken, M.Sc. National Aerospace Laboratory (NLR),Royal Netherlands Air Force

2 Contents Part 1(Intelligent CGFs) Smart Bandits Requirements State-of-the-art in CGFs Architecture Cognitive models Part 2 (Machine Learning) Experiment RL (example) Pros & Cons ML Towards hybrid models (example) Conclusions Exploratory Team under IST panel

3 Project Goals Smart Bandits Development of intelligent Computer Generated Forces for tactical mission training of fighter pilots in the opponent role humanlike behaviour –capable of tactical reasoning intelligent decision making team work.. –Constrained by Situation Awareness Memory capacity.. Intelligent CGFs should be suitable for use in simulations at MoD

4 Application of intelligent CGFs in Embedded Training?

5 Current Scope Tactical mission training Tactics are techniques for using aircraft and weapons in a combined fashion with the purpose to gain advantage over / defeat the enemy Air-to-Air Beyond Visual Range (>10 NM) Offensive and Defensive Counter Air (picture) 1v1, 2v2, 4v4 engagements

6 Requirements Operational CGFs in the opponent role –should be autonomous –should exhibit credible behaviour –should contribute to training value of simulation Functional weapon system functions human functions more specific functions per mission phase –planning & briefing –targeting –executing the game plan –self-defence

7 Research facility: NLRs Fighter 4-Ship One station of the Fighter 4-Ship (Four networked F-16 simulations)

8 F-16 executes OCA mission (Offensive Counter Air) Su-27 executes DCA mission (Defensive Counter Air) FLOT F-16 Su-27

9 State-Of-the-Art in CGFs Scenario-management packages behaviour of CGFs is scripted pre-defined CGFs lack appropriate weapon and human models limited possibilities for the use of AI Agent Qualities Non-responsive behaviour Stimulus-Response (S-R) behaviour Delayed Response (DR) behaviour motivation-based behaviour combines S-R and DR behaviour + motivational states TAC-AIR SOAR cognitive architecture deals with observations, decisions and coordination Order of magnitude: tactical decision rules Machine Learning techniques

10 Agent Development Approach Multi Agent - 2v2 Smart Adversary Behaviour Reinforcement Learning Neural Networks Single Agent - 1v1 Situation Awareness Theory of Mind Evolutionary techniques Multi Agent - 4v4 Decision Making Machine Learning Techniques Cognitive (BDI) Models Tactical Scenarios (scripted)

11 Architecture Agent-models are functionally separated from simulation environment Human-like behaviour can be linked to CGFs Different agent models can run on different machines Simulator – CGF package

12 Cognitive Models (Team) Situation Awareness Naturalistic Decision Making Theory of Mind

13 Example : Situation Awareness Definition Mica Endsley (1988) three levels of SA: – the perception of the environment, – the comprehension and integration of information, and – the projection of information into future events. Translation to BDI framework Perceive: Observations/ Simple beliefs Understand: Complex beliefs Anticipate: Future beliefs Human constraints belief formation constrained by workload

14 Cognitive model for situation awareness: overview from Hoogendoorn, van Lambalgen & Treur, 2011

15 Example belief network for SA model from Hoogendoorn, van Lambalgen & Treur, 2011

16 Reinforcement Learning Experiment

17 Pros and Cons Machine Learning Pros Save development time (less knowledge elicitation required) Adaptation to environment and opponent Complex behaviour in complex domains New tactics and evaluation of human tactics Cons Learning speed Effectiveness (unpredictable behaviour) Computation time and memory requirements Adaptation to game randomness Increase development time (tweaking)

18 Hybrid models (Dynamic Scripting, Spronck et al., 2005) Reinforcement learning Scripts

19 Conclusions Cognitive modelling one of the fundamental techniques for motivation-based behaviour CGFs Machine Learning is powerful tool to: enhance and complement cognitive models reduce knowledge elicitation efforts Smart Bandits: combination of models, utilizing advantages of different approaches

20 Technical Activity Proposal (TAP) Machine Learning Techniques for Battlefield Agents Exploratory Team under the IST panel Some topics to be covered: Current applications of ML Potential applications in Defence (all Forces) Potential barriers for application Most appropriate ML techniques Systems engineering aspects of ML 3 meetings in 2012, 1st in Amsterdam, early 2012 Leading to a Research Task Group TAP available! Let us know whether you are interested to participate!


Download ppt "Modelling CGFs for tactical air-to-air combat training Motivation-based behaviour and Machine Learning in a common architecture Jan Joris Roessingh, Ph.D.,"

Similar presentations


Ads by Google