Presentation is loading. Please wait.

Presentation is loading. Please wait.

Modelling CGFs for tactical air-to-air combat training

Similar presentations

Presentation on theme: "Modelling CGFs for tactical air-to-air combat training"— Presentation transcript:

1 Modelling CGFs for tactical air-to-air combat training
Motivation-based behaviour and Machine Learning in a common architecture Jan Joris Roessingh, Ph.D., Maj. Roel Rijken, M.Sc. National Aerospace Laboratory (NLR), Royal Netherlands Air Force

2 Contents Part 1(Intelligent CGFs) Smart Bandits Requirements
State-of-the-art in CGFs Architecture Cognitive models Part 2 (Machine Learning) Experiment RL (example) Pros & Cons ML Towards hybrid models (example) Conclusions Exploratory Team under IST panel

3 Project Goals “Smart Bandits”
Development of intelligent Computer Generated Forces for tactical mission training of fighter pilots in the opponent role ‘humanlike’ behaviour capable of tactical reasoning intelligent decision making team work .. Constrained by Situation Awareness Memory capacity Intelligent CGFs should be suitable for use in simulations at MoD

4 Application of intelligent CGFs in Embedded Training?

5 Current Scope Tactical mission training
Tactics are techniques for using aircraft and weapons in a combined fashion with the purpose to gain advantage over / defeat the enemy Air-to-Air Beyond Visual Range (>10 NM) Offensive and Defensive Counter Air (picture) 1v1, 2v2, 4v4 engagements

6 Requirements Operational CGFs in the opponent role
should be autonomous should exhibit credible behaviour should contribute to training value of simulation Functional weapon system functions human functions more specific functions per mission phase planning & briefing targeting executing the game plan self-defence

7 Research facility: NLR’s “Fighter 4-Ship”
One station of the Fighter 4-Ship (Four networked F-16 simulations)

8 F-16 executes OCA mission (Offensive Counter Air) Su-27 executes DCA mission (Defensive Counter Air)

9 State-Of-the-Art in CGFs
Scenario-management packages behaviour of CGFs is “scripted” pre-defined CGFs lack appropriate weapon and human models limited possibilities for the use of AI Agent Qualities Non-responsive behaviour Stimulus-Response (S-R) behaviour Delayed Response (DR) behaviour motivation-based behaviour combines S-R and DR behaviour + ‘motivational states’ TAC-AIR SOAR cognitive architecture deals with observations, decisions and coordination Order of magnitude: tactical decision rules Machine Learning techniques

10 Agent Development Approach
Cognitive (BDI) Models Machine Learning Techniques Tactical Scenarios (scripted) Single Agent - 1v1 Situation Awareness Evolutionary techniques Theory of Mind Reinforcement Learning Multi Agent - 2v2 Decision Making Neural Networks Multi Agent - 4v4 Smart Adversary Behaviour

11 Architecture Simulator – CGF package Agent-models are functionally separated from simulation environment Human-like behaviour can be linked to CGFs Different agent models can run on different machines

12 Cognitive Models (Team) Situation Awareness Naturalistic Decision Making Theory of Mind

13 Example : Situation Awareness
Definition Mica Endsley (1988) three levels of SA: the perception of the environment, the comprehension and integration of information, and the projection of information into future events. Translation to “BDI” framework Perceive: Observations/ Simple beliefs Understand: Complex beliefs Anticipate: Future beliefs Human constraints belief formation constrained by workload

14 Cognitive model for situation awareness: overview
from Hoogendoorn, van Lambalgen & Treur, 2011

15 Example belief network for SA model
from Hoogendoorn, van Lambalgen & Treur, 2011

16 Reinforcement Learning Experiment

17 Pros and Cons Machine Learning
Save development time (less knowledge elicitation required) Adaptation to environment and opponent Complex behaviour in complex domains New tactics and evaluation of human tactics Cons Learning speed Effectiveness (unpredictable behaviour) Computation time and memory requirements Adaptation to game randomness Increase development time (tweaking)

18 Hybrid models (Dynamic Scripting, Spronck et al., 2005)
Reinforcement learning Scripts ‘Adaptive game AI with dynamic scripting’, Spronck et al., 2005

19 Conclusions Cognitive modelling one of the fundamental techniques for motivation-based behaviour CGFs Machine Learning is powerful tool to: enhance and complement cognitive models reduce knowledge elicitation efforts Smart Bandits: combination of models, utilizing advantages of different approaches

20 Let us know whether you are interested to participate!
Technical Activity Proposal (TAP) Machine Learning Techniques for Battlefield Agents Exploratory Team under the IST panel Some topics to be covered: Current applications of ML Potential applications in Defence (all Forces) Potential barriers for application Most appropriate ML techniques Systems engineering aspects of ML 3 meetings in 2012, 1st in Amsterdam, early 2012 Leading to a Research Task Group TAP available! Let us know whether you are interested to participate!

Download ppt "Modelling CGFs for tactical air-to-air combat training"

Similar presentations

Ads by Google