Presentation on theme: "Implications of Adaptive vs. Adaptable UIs on Decision Making: Why “Automated Adaptiveness” is Not Always the Right Answer Christopher A. Miller Harry."— Presentation transcript:
Implications of Adaptive vs. Adaptable UIs on Decision Making: Why “Automated Adaptiveness” is Not Always the Right Answer Christopher A. Miller Harry B. Funk Robert P. Goldman John Meisner Peggy Wu Originally presented at 1 st Annual Conference on Augmented Cognition Las Vegas, NV – July 24-28, 2005 AAAI-SS on Interaction Challenges for Automated Assistants March, 2007
21 April Adaptive vs. Adaptable Opperman, 1994 “Adaptive Systems” are those that automatically adapt to user needs “Adaptable Systems” are those in which the user adapts system behavior to his/her needs. “Learning” vs. “Instructing”
21 April Adaptive Automation as “Meta-Automation” World State Displays, Controls & Systems State, Needs Adaptive Automation removes a human from an aspect of the control loop … with what results? State, Needs Adaptive Approaches
21 April Benefits of Adaptive Automation Greater performance speed Greater performance consistency Improved safety/survivability Reduced training time
21 April Disadvantages to Adaptive Automation High levels of automation (which remove operators from portions of the task/control loop) have been shown to … Increase human workload at critical times if not properly designed [1,2] Decrease human situation awareness [3,4] Produce human complacency (~ unwarranted trust)  Result in human skill degradation  Decrease human acceptance ; increase human dissatisfaction  Provide inferior human + machine system performance (relative to lower levels of automation)  1.Parasuraman and Riley, Wiener, Endsley and Kiris, Sarter and Woods, Parasuraman, Molloy and Singh, Rose, Miller and Hannen, Vicente, Layton, Smith, McCovy, 1994
21 April 2015 In any human + automation system, a given level of Competency will entail a tradeoff between Human Workload (Un-)Predictability to User Competency Unpredictability Worklo ad Automation Control (aka Adaptive) ‘Competency’ is appropriate behavior in context. Characterizing the Tradeoff Competency gets harder with increases in Precision of context divisions Flexibility of behavior Increasing adaptivenss means either increasing human workload or increasing unpredictability, or both. Competency Unpredictability Worklo ad Human Control (aka Adaptable) Competency Unpredictability Worklo ad Competency AdaptableAdaptive A spectrum of alternatives
21 April So maybe taking the human out of the loop, even to do the human’s bidding, isn’t the best idea … For user acceptance For overall human + machine system performance
21 April How do we want to interact with automation? Would you want/trust an assistant who always handed you ostensibly right information – with no input/instruction/clarification from you? Wouldn’t trust, coordination, SA and maybe performance be improved by explicit, meta-level intent coordination? All right, dear, I’ll pick up some milk. Turn left now. In order to go to the store, turn left now. OK, then turn left now OK, Car, I want to go to the store.
21 April Advantages of (more) Adaptable Approaches Increased awareness of the situation and of system performance Better tuning of trust and better automation reliance decisions (since operators instruct automation how to behave) Better skill retention (through allowing the user to perform tasks when needed or desired) Better balance of mental workload (if humans are better assessors of their needs than AugCog systems) Better human + machine performance (if/when users make better judgments than automation about what is needed to complement their abilities). Leaving the user in charge yields greater automation acceptance, AND greater sense of being primarily responsible—in turn yielding greater attention and concern for the situation and all aspects of system performance.
21 April It’s not Either/Or … Adaptable or Adaptive It’s a SPECTRUM… Find the right point on it!
21 April A Tale of Two Associates Pilot’s Associate ( ) Single Pilot Direct pilot interaction with associate meant added workload Design philosophy minimized direct pilot interaction with associate Moderate user acceptance The Pilot is ALWAYS in charge. The Effort required of the pilot to control the associate must be less than the effort saved by the associate Rotorcraft Pilot’s Associate ( ) Two Pilots 1/3 of human activity is crew coordination Design philosophy included some direct pilot interaction with associate Improved User Acceptance
21 April Crew Coordination & Task Awareness Display Four buttons to convey major, associate-inferred task contexts Single press overrides = “No, you’re wrong. That’s not what we’re doing” – Associate ‘gets out of the way’ Press and Hold scrolls through tasks at same level of hierarchy – E.g., Area Recon, Zone Recon, Attack in Force, Hasty Attack, Delay, Evade, Ingress, Egress, etc. MISSIONPILOTASSOCIATECOPILOT AREA HOVER MANUAL
21 April Subjective Workload (TLX) Ratings Workload levels consistently higher for Baseline (AMEP) than for RPA (CDAS) Significant differences for 4 of 6 TLX subscales (and close for the 5th)
21 April CIM Utility and Overrides 1 Of no Use Not Very Useful 2345 Extrem- ely Useful Of Use Of Con- siderable Use Page Selection Symbol Selection Window Location Pan & Zoom Usefulness Ratings Most crews said CIM Behaviors were ‘Of Use’ or “Of Considerable Use’ 1 Always Fre- quently 2345 Never Now & Then Sel- dom Page Selection Symbol Selection Window Location Pan & Zoom Pilot-reported Frequency of Overrides/Corrections 3.00 Crews ‘Seldom’ overrode CIM’s symbol selections, but ‘Now & Then’ overrode other behaviors
21 April CIM was seen as useful and provided perceived performance and workload advantages in spite of ‘Now and Then’ or ‘Frequently’ providing the wrong information. Why?
21 April Crew Coordination & Task Awareness Display 1 None Not Very Ext- remely Of Use Con- siderable Mission Task Pilot Task Co-Pilot Task Associate Task Usefulness Ratings 4.3 Perceived accuracy of LED Task displays was very high Comments (and other ratings) indicated these were very useful to pilots MISSIONPILOTASSOCIATECOPILOT AREA HOVER MANUAL
21 April “Playbook” Interfaces for Plan Delegation NUGGET NUGGET: Composable ‘plays’ derived from Task Model shared between human and automation software.APPLICATION: Plan specification to any level of specificity (‘calling plays’) Automatic plan completion within human intent Feedback on plan feasibility STATUS: STATUS: Prototypes for: UCAV pre-mission planning Tactical Robot Control Work on: Unmanned Vehicle Teams Heterogeneous Urban RSTA UCAR Sim scenario specification HTNP Analysis Event Handling Control Algorithms Instructions Feedback Provably correct plans Provably safe reactions System control Shared Task Model Playbook GUI Over- watch Obtain Aircraft Scan Ingress Egress X X X Destage Overwatchmeans to provide continuous relayed imagery of a fixed or moving point or area. Overwatch Sortie X Maneuver X Fly-to X Aircraft:_____________ Target Area:_________ Imagery Start:________ Imagery End:_________
21 April Conclusions Adaptation is good Not everything has to be (or SHOULD be) done by automation Who does the adapting is important He who does the adapting is Better aware of what’s been adapted Better aware of why it’s been adapted Probably better served by the adaptation Humans don’t like to be managed … though there are some times when they should be Adaptable and Adaptive techniques can be complementary, not competing