1. S ystems Analysis Laboratory Helsinki University of Technology Game Theoretic Validation of Air Combat Simulation Models Jirka Poropudas and Kai Virtanen Systems Analysis Laboratory Helsinki University of Technology jirka.poropudas@hut.fi, kai.virtanen@hut.fi

2. S ystems Analysis Laboratory Helsinki University of Technology Air combat is analyzed to compare e ffectiveness of tactics and ways for conducting missions as well as system performance Test flights are expensive and time consuming  constructive simulation Discrete event simulation models provide a c ontrolled and reproducible environment that may be complex and convoluted with many levels of sub- models Air combat simulation Air combat simulation model Aircraft, weapon systems, radars, other apparatus Pilot decision making and situation awareness Uncertainties Validation of the model?

3. S ystems Analysis Laboratory Helsinki University of Technology Existing validation and optimization approaches Simulation metamodels – Mappings from simulation input to output - Response surface methods, regression models, neural networks, etc. Validation methods – Real data, expert knowledge, statistical methods, sensitivity analysis Simulation-optimization methods – Ranking and selection, stochastic gradient approximation, metaheuristics, sample path optimization One-sided approaches  Action of the adversary is not taken into account The game theoretic approach!

4. S ystems Analysis Laboratory Helsinki University of Technology The game theoretic approach 1)Definition of the scenario –Aircraft, weapons, sensory and other systems –Initial geometry –Objectives  Measures of effectiveness (MOEs) –Available tactics and systems = Tactical alternatives 2)Simulation of the scenario using the simulation model –Input: tactical alternatives –Output: MOE estimates 3)Estimation of games from the simulation data using statistical techniques 4)Use of the games in validation

5. S ystems Analysis Laboratory Helsinki University of Technology Games in validation Goal: Confirming that the simulation model performs as intended Comparison of the scenario and properties of the game Symmetry – Symmetric scenarios => symmetric games Dependence between decision variables and payoffs – Dependence between tactical alternatives and MOEs Best responses and Nash equilibria – Explanation and interpretation based on the scenario Initiative – Making one’s decision before or after the adversary => Advantageous/disadvantageous? – Explanation and interpretation based on the scenario

6. S ystems Analysis Laboratory Helsinki University of Technology Validation example: Aggression level Two-on-two air combat scenario – Identical aircraft, air-to-air missiles, radars, data links, etc. – Symmetric initial geometry – Identical tactical alternatives - Aggression levels of pilots: Low, Medium, High – Objectives => MOEs - Blue kills, red kills, difference of kills Simulation using X-Brawler – Many versus many air combat simulation – Discrete event simulation methodology – Aircraft, weapons and other hardware models – Elements describing pilot decision making and situation awareness

7. S ystems Analysis Laboratory Helsinki University of Technology Validation results RED, min IV 1.50 II 0.33 high IV 1.50 II 0.34 medium III 1.21 III 1.20 I 0.18 low high mediumlow BLUE, max Payoff: Blue kills Dependence –Increasing aggressiveness => Increase of blue kills Best responses & Nash equilibria –Medium or high for blue, low for red –Medium and high leading to the same outcome => Possible shortcoming Expert knowledge: Increasing aggressiveness  Increasing causality rates MOE: blue kills  Low aggressiveness for red  High aggressiveness for blue

8. S ystems Analysis Laboratory Helsinki University of Technology Validation results Symmetry –MOE estimates approximately zero when the decisions coincide –E.g., low, high => best, worst AND high, low => worst, best Dependence –Increasing aggressiveness => Increasing causality rates for both sides –Medium and high for blue leading to the same outcome => Possible shortcoming Best responses & Nash equilibrium –Low for blue, low for red III 0.00 III 0.04 I -0.83 high III 0.02 III 0.01 I -0.81 medium IV 0.89 IV 0.86 II -0.08 low high mediumlow RED, min BLUE, max Payoff: Blue kills – Red kills Expert knowledge: Increasing aggressiveness => Increasing causality rates Symmetric scenario => Symmetric game

9. S ystems Analysis Laboratory Helsinki University of TechnologyConclusions Novel way to analyze air combat – Combination of discrete event simulation and game theory – Extension of one-sided validation and optimization approaches Validation – Properties of games  Air combat practices – Simulation data in an informative form – Comparison of tactical alternatives using games – Systematic means for analyzing air combat - Single simulation batch Other application areas involving game settings