Presentation is loading. Please wait.

Presentation is loading. Please wait.

Presenting Uncertainty to Controllers and Pilots

Similar presentations


Presentation on theme: "Presenting Uncertainty to Controllers and Pilots"— Presentation transcript:

1 Presenting Uncertainty to Controllers and Pilots
EUROCONTROL CARE - ATM Innovative Concepts Public Dissemination Forum EUROCONTROL HQ, Brussels, March 2004 Presenting Uncertainty to Controllers and Pilots David Nicholls, RM Consultants Ltd, UK Patrizia Marti and Valentina Barsotti, Deep Blue s.r.l., Italy risk management consultants NEW LOGO

2 Uncertainty in ATM If there were no uncertainty, would we need controllers and pilots …?

3 OUTLINE 1. Background 2. Objectives, Scope and Methods
3. Uncertainty in Conflict Detection 4 Uncertainty in Multi Sector Planning 5. Transfer to Other Contexts 6. Lessons Learned

4 1. BACKGROUND

5 Proposals for main CARE study
Preliminary study (2001) RMC, University of Glasgow, pilot & HF consultants Where is there uncertainty in aviation? How do stakeholders think about and manage uncertainty? Where might there be benefit in presentation? What presentational forms are already used? Proposals for main CARE study

6 Types of uncertainty Variability
Inaccuracy of measurement, prediction, display Missing data, lack of knowledge, incomplete understanding Information that is ambiguous or hard to interpret Subjective uncertainty - degree of belief or trust

7 Some aviation examples
When will we get start clearance? Accuracy of TCAS Was that a real GPWS? What’s it doing now? Is the tailwind really only 10 kts? Land or go-around? Why are they sending me this way? How long will it take to climb? Flight Plan consistency Did he really understand? Shift handover in a hurry How big is the storm? ? ?

8 Current approach to uncertainty
A culture of eliminating or minimising uncertainty: more accurate and reliable tools standard operating procedures controllers/ pilots see their role as to reduce uncertainty we don’t have uncertainty in ATM we can only work with information that is certain

9 Why do more? Pilots and controllers base some decisions on incomplete or uncertain information, especially where prediction is involved New tools tend to increase the use of, and reliance on, predicted information Even where better accuracy/ reliability compensates for greater reliance, could we do even better by presenting uncertainty?

10 applicable guidance for ATM
State of the art Number of references on presenting uncertainty general psychology of uncertainty and decision making experimental studies for specific aviation tasks and tools applicable guidance for ATM Generic Specific

11 Some advantages/ disadvantages of presenting uncertainty
Potential advantages more comprehensive picture for decision-making better indication that system is straying outside normal limits improved situation awareness, attention, enjoyment - an antidote to the problems of automation? Potential disadvantages increased display clutter increased cognitive load increased overall uncertainty in system and potential for confusion, because there is more scope for individual judgment

12 2. OBJECTIVES, SCOPE AND METHODS

13 Objectives Develop, using case studies, principles for deciding what information about uncertainty is useful, how & to whom it should be presented Intended benefits: practical guidance for ATM in general contribute to the development of HMIs for case study tools

14 Case studies Conflict prediction (TP/ MTCD/ CORA)
Multi Sector Planning (TLS) The case studies relate to en-route ATC - not necessarily the most critical uncertainties in ATM Selected as examples of interest at the time, but lessons learned may be generalised TLS = Tactical Load Smoother MSP = Multi-Sector Planner TP = Trajectory Prediction MTCD = Medium Term Conflict Detection CORA = Conflict Resolution Assistant

15 Scope of study Develop and evaluate prototypes to a level that allows an informed decision on more formal development (e.g. full simulation) Presentation, not reduction at source Information about uncertainty, not uncertain information Only uncertainties that are to some extent predictable - i.e. bounded and foreseeable Not concerned with verbal uncertainty

16 Multi-Sector Planning
Overview of methods Conflict Prediction existing tools, close to operational use ‘traditional’, problem- solving design approach add-ons to existing HMIs, exploring issues around data, algorithms evaluation in ERC Human Factors lab Multi-Sector Planning new concept - free to explore radical options development based on CREA! method - extensive user involvement new roles, tools and ways of working evaluation from animated storyboards

17 Project organisation EUROCONTROL HQ EUROCONTROL Research Centre
RM Consultants Ltd Deep Blue s.r.l Graffica Controller input: ENAV, CANAC, NATS

18 3. UNCERTAINTY IN CONFLICT PREDICTION (TP/ MTCD/ CORA)

19 Development/ evaluation method for conflict prediction
Identify suitable tools and displays: VAW – uncertainty in climb/ descent PPD – uncertainty in conflict time and distance analysis of MTCD/ TP/ CORA documents, simulation reports, discussions with designers and users workshop at ERC to brainstorm and short-list tools and sketch displays Develop algorithms (simple distributions with real or subjective data Implement in eDEP platform Small-scale evaluation in ERC Human Factors lab

20 Uncertainty in climb/ descent
Time of initiating climb/ descent: triangular distribution Rate of climb/ descent: triangular distribution Variability about nominal climb/ descent: normal distribution, growing then decaying with time Smoothing and truncation

21 Vertical Aid Window (VAW)
Late, shallow descent Early, steep descent

22 Uncertainty in conflict time and distance
Along-track error: normal distribution, linear growth with time Cross-track error: uniform distribution, constant with time From these the MTCD algorithms calculate confidence bounds on when the CPA will occur and what the separation will be t = 1 t = 1 t = 5 t = 10

23 PPD without uncertainty
15 2 10 TIME to CPA (minutes) 3 5 User might decide to resolve conflict 3 before conflict 2 1 5 10 DISTANCE (nm) at CPA

24 PPD with uncertainty Ellipses show 95% confidence on time and distance
a long way off but more certain 15 2 a long way off and uncertain 10 TIME to CPA (minutes) 3 certain and close 5 User might decide to resolve conflict 2 before conflict 3. Greater uncertainty = less urgent, in this context. 1 5 10 DISTANCE (nm) at CPA Ellipses show 95% confidence on time and distance Colour saturation increases as bubble shrinks, to counter the impression that ‘big = important’

25 Experiment at ERC Two groups of 4 controllers, two days each
Controllers worked both PC and TC roles No pilots, no R/T High and low traffic samples Uncertainty displays selected ON or OFF in each run Subjective measures – questionnaires, debriefings NASA TLX workload Initial results available

26 Questionnaire results
Number of controllers responding Yes No Yes No helpful? increases SA? increases workload? matches perception of u/c would like one? VAW PPD Majority of negative views, except with regard to workload and colours of PPD More found PPD useful than VAW

27 TLX - Mean and Standard Deviation
TLX workload Uncertainty Display For high traffic, TLX* appears lower when uncertainty is displayed * TLX is a self-assessed, post-run measure (range 0-100) based on scores for mental and temporal demand, effort, performance and frustration. ON OFF 70 60 High traffic 50 TLX - Mean and Standard Deviation 40 Low traffic 30 20

28 Comments on controller views
About half of the controllers were unfamiliar with MTCD/ PPD. Often comments (and scores?) related to these tools in general , not to the uncertainty displays specifically There were few occasions in which a conflict was seen against the VAW uncertainty area, and hence not many opportunities to use this display ‘too many windows’

29 Controller suggestions
Better integration of the 3/ 4 dimensions Better integration with the whole-traffic picture - separate windows don’t help build the picture and distract from or clutter the main radar view ‘Younger controllers might like it’ Uncertainty displays could be useful (with training) for: longer term planning roles e.g. MSP, ground control and arrival/ departure planning (especially in a Collaborative Decision Making environment or with DMAN/AMAN)

30 4. UNCERTAINTY IN MULTI-SECTOR PLANNING

31 Deep Blue presentation ….

32 5. TRANSFER TO OTHER CONTEXTS

33 Transferability Case studies were for en-route ATC, and solutions must be context-specific But uncertainty presentation may be transferable to other contexts - especially where users have long look-ahead times and/ or major uncertainties in the information they use …..

34 Flight deck Pilots may benefit where there is:
increased use of predictive tools full or partial self-separation (‘Free Flight’) - pilots taking more responsibility for separation may need a shared appreciation of uncertainty with the controller …..? Flight deck environment differs from control room: much of pilots’ work is objective-oriented and immediate main concern is with own aircraft, not with many physical space for new displays is limited …..

35 Arrival/ departure management
ground & tower controllers and airlines involved in arrival/ departure management Uncertainty in ETA at KOKSI Pax/ Bags loaded Engine start At runway hold KOKSI Push back On stand Take off Inbound Turn-round Taxi Airborne

36 6. LESSONS LEARNED

37 What to present Culture of eliminating uncertainty is deeply ingrained, but ….. Results of MSP study, particular, indicate potential benefits of presentation in some contexts Deciding, a priori, what specific uncertainty information will be useful is difficult

38 Who needs uncertainty information?
Users with time to analyse decisions, and/or major uncertainties in information, such as: long term planners (e.g MSP) ground & tower controllers/ airlines? pilots in Free Flight? Less useful for those making immediate decisions from short term information

39 Presentation means … ‘Presentation’ is not limited to explicit display. It can include: making the uncertainty evident showing how the situation may develop - alternative futures providing ‘what-if’ interactivity prompting the user to consider the certainty of their own judgment

40 Design ‘rules’ - 1 Solutions must be scenario-specific, but some ‘rules’ may have wider application: provide ‘what-if’ interaction give information about causes - whether within or outside user control visualise time - distinguish current and future indicate where tools / data are unreliable or incomplete

41 Design ‘rules’ - 2 …. present uncertainty on demand
integrate with main traffic picture displays support correct inferences from new data (Bayesian updating, avoiding cognitive biases of anchoring/ recency) ensure a clear understanding of ‘uncertainty’ but don’t be too ‘mathematical’

42 Development process A CREA!-based process has been developed and demonstrated It uses real-life scenarios to introduce the idea of uncertainty and scenario-based, iterative, user-centered development. It appears more likely to lead to acceptable solutions than traditional ‘problem-solving’ The CREA! method has been found to be scaleable, adaptable and effective


Download ppt "Presenting Uncertainty to Controllers and Pilots"

Similar presentations


Ads by Google