1. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-1A l Humans smarter, meaner, more strategic, emotional, variable, etc. l Will superconflict solving deteriorate or improve? Superconflict n=8

2. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-2A Who are the bots and who are the humans?

3. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-3A l Phase I: Web experiment (2000) l Phase II: Classroom experiment (2001) Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR Jacco Hoekstra, Head of Training & Concept Validation department NLR Airborne Separation Assurance Validation with Multiple Humans-in-the-loop

4. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-4A Human interaction experiments l Before these experiments: - micro: human-in-the-loop only 1 or 2 crews - macro: traffic simulations with all a/c controlled by ‘bots’

5. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-5A Human interaction experiments l Goal HIE: - Investigate effect of multiple humans in the loop l Research issues: - Validations bots behaviour vs. human behaviour - micro: difference 1 on 1 - macro: complex geometries - Competition effect

6. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-6A Distributed simulation via Internet DirectXTMX as Game server

7. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-7A Webexperiment set-up l Freesim: downloadable flight simulator with ASAS as Game Client (MS DirectX protocol)

8. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-8A FreeSim flight simulator l Downloadable with on-line user manual l Two modes: –Stand-alone with basic training scenarios –Online with Traffic Manager as traffic server l Still available on website: http://www.nlr.nl/public/hosted-sites/ freeflight/fsmanual/freesim2.zip

9. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-9A FreeSim on-line l Two modes open (demos) and secured (experiments): - using unique e-mail address - valid session name - IP number for login l User UTC time synchronized via US Navy clock on web l System time asynchronous, data logging centralized l Performance data and events are sent to TMX by Freesim

10. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-10A Web experiment l Validation of the bots Competition effect l Two types of conflicts: –human-human –human-bot l Both in traffic and conflict: participants could not tell difference bot/human in normal (realistic) flight scenario l Competition effect not tested due to low number of participants (max 12 simultaneously)

11. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-11A Result: Conflict rate variation per a/c high l Conflict rate distribution can be odd-shaped and cause problems # a/c

12. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-12A Classroom experiment Set-up l 24 participants (nearly or just grad) of KLM pilot school l Game effect: goal is time or safety l Two types of scenarios: normal (realistic) & superconflicts

13. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-13A Classroom experiment results l Workload ratings for normal scenarios with bottlenecks: - higher than before (training?) - multi a/c conflicts driving factor workload

14. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-14A Classroom experiment results l Preventive effect in superconflict scenarios visible on ‘no conflict’ manoeuvres l Minor effect of strategy superconflict normal no conflict in conflict

15. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-15A Classroom experiment results: Intrusions in superconflicts N >= 12 l Number of intrusions (all scrapings) - Bots lower in small superconflicts and higher in larger superconflicts l Larger margin required between resozone and cdzone Average Δt = 17 sec about 2% intrusion

16. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-16A Classroom experiment results: number of conflicts alerts in superconflict l Number of conflict alerts (excl. scrapings) Pilots significantly better

17. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-17A Classroom experiment Conclusions l Some human effects: - going for the edge - playing chicken (based on observation of one-sided resolutions) l Humans slightly better at solving superconflicts than the bots (lower number of conflicts per a/c)

18. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-18A Overall conclusions l Bots seem able to simulate humans in normal traffic pattern behaviour l Humans smarter and meaner overall slightly better in superconflicts l Conflict rate and especially bottlenecks are driving factor for workload l Internet technically feasible but requires extra organisation. Classroom allows more control over e.g. training and nr of participants

19. Nationaal Lucht- en Ruimtevaartlaboratorium National Aerospace Laboratory NLR CXXX-19A http://www.nlr.nl/public/hosted-sites/freeflight Questions? No problem! We have right of way.