Issues in measuring sensory-motor control performance of human drivers: The case of cognitive load and steering control Johan Engström, Volvo Technology Corporation European Workshop on Advanced Predictive Sensory-motor Control Joudkrante, Lithuania, 2009-05-21

Vehicle & Load Structures Outline Multitasking in the vehicle Secondary tasks – visual and cognitive The primary driving task – visual control of steering Effects of secondary tasks on steering control Different effects of visual and cognitive tasks The ”lane keeping improvement” effect of cognitive load Possible explanation in terms of satisficing vs. optimising steering control Testing predictions implied by this hypothesis Vehicle & Load Structures 2018-12-01

Multitasking in the vehicle: Driving + secondary tasks Vehicle & Load Structures 2018-12-01

Secondary tasks: Visual vs. cognitive distraction Visual distraction Looking off road E.g. Visual time sharing when tuning the radio Cognitive distraction: Engaging in demanding cognitive (working memory) tasks E.g. Mobile phone conversation Most real-world tasks involve both components… Vehicle & Load Structures 2018-12-01

The primary driving task: Sensory-motor control in steering Vehicle & Load Structures 2018-12-01

The visual control of steering: Optical and retinal flow Wann and Wilke (2000) Straight driving, looking ahead Straight driving, looking to the left side Retinal flow not equal to optical flow Vehicle & Load Structures 2018-12-01

Using retinal flow patterns to guide steering: Look where you’re going resulting heading initial heading Wann and Wilke (2000) Underrsteering Going towards target Oversteering Vehicle & Load Structures 2018-12-01

Gaze angle can be used as a direct cue for steering through curves (Land, 1998) Fixate tangent point and adjust steering to keep gaze angle constant Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Combining retinal flow patterns and gaze direction: ”Spring” model (Wann and Wilkie, 2000) Stiffness Angular acceleration Reliance on cues Damping Main point: Foveal vision is essential for accurate steering! Vehicle & Load Structures 2018-12-01

Effects of secondary tasks on steering control Vehicle & Load Structures 2018-12-01

Effects of visual distraction on lateral control Visual time sharing Gaze angle Looking away Loosing visual input for steering control Heading error builds up Looking back Large steering wheel correction Speed reduction to compensate Steering wheel angle Increased lane position variance Lane position Speed Engström and Markkula (2006) Vehicle & Load Structures 2018-12-01

What about purely cognitive distraction? Large number of simulator and real-world driving studies found reduced lane keeping variance during cognitive load (Brookhuis et al., 1991; Östlund et al., 2004; Jamson and Merat, 2005; Engström et al., 2005; Mattes, Föhl and Schindhelm, 2007; Merat and Jamson, 2008). Engström, Johansson and Östlund (2005) Does talking on the mobile phone really improve steering control? SD lane position Cognitive task difficulty Vehicle & Load Structures 2018-12-01

Other effects of cognitive distraction: Gaze concentration Victor, Harbluk and Engström (2005) Normal driving Cognitive task Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Other effects of cognitive distraction: Increased steering activity (number of steering reversals > 2 deg. per minute) SW reversals/min Engström et al. (2005) Vehicle & Load Structures 2018-12-01

Summary: Effects of cognitive distraction related to lateral control Improved lane keeping (!?) Gaze concentration towards the road centre Increased number of micro steering corrections (<2 deg) How are these effects related? How can they be explained? Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Possible explanation Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Two key distinctions Satisficing vs. Optimising Top-down (endogenous) vs. Bottom-up (exogenous) attention selection Vehicle & Load Structures 2018-12-01

1. Satisficing vs. optimising Target value Comfort zone Optimising: Minimising performance error relative to a target state. Satisficing: Maintaining performance within acceptable boundaries. Vehicle & Load Structures 2018-12-01

Example cost functions of optimising and satisficing in lane keeping Lane position Lane centre Vehicle & Load Structures 2018-12-01

Example dynamics of satisficing and optimising X_dot Satisficing Comfort zone Optimising X Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Simulations Satisficing Optimising Vehicle & Load Structures 2018-12-01

2. Bottom-up and top-down attention selection attention bias 2. Bottom-up and top-down attention selection Top-down selection Cognitive task Other visual task Vehicle dynamics Steering Bottom-up selection Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Top-down attention bias Normal driving Steering easy and automated task, bottom-up-driven -> satisficing Top-down selection Other visual task Vehicle dynamics Steering Spare top-down attentional resources used for other visual tasks Bottom-up selection visual time sharing Lane keeping variance Distributed gaze Only intermittent steering Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Top-down attention bias Cognitive load Top-down selection Top-down attention allocated to cognitive task Cognitive task Other visual task No top-down-initiation of other visual tasks Steering Vehicle dynamics Gaze can be fully devoted to steering (attracted bottom-up) Bottom-up selection Reduced lane keeping variance Gaze concentration Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Testable predictions General: Improved lane keeping should only occur if the driver is satisficing in baseline condition Specific predictions: Improved lane keeping should not occur if the steering task is difficult (so that satisficing is not possible) Improved lane keeping effect should not occur if the driver is motivated to optimise lane keeping Support for prediction 1 Cognitive load has been demonstrated to impair performance on tracking tasks (Creem and Proffitt, 2001; Strayer and Drews. 2001). These tasks could be expected to be more difficult and/or less automated than normal driving Prediction 2: Tested experimentally… Vehicle & Load Structures 2018-12-01

Instruction to optimise steering (baseline) Top-down attention bias Top-down selection Top-down attention allocated to steering task and cognitive task Other visual task Optimising steering performance Steering Vehicle dynamics Bottom-up selection Reduced lane keeping variance Gaze concentration Increased steering wheel control input Vehicle & Load Structures 2018-12-01

Testing prediction 2: Experimental design Simulator study in fixed based simulator (at Saab Automobile, Trollhättan) Cognitive task: Count backwards with 7 48 subjects, split in 4 groups: Incentive for group 1 and 2: Two cinema tickets instead of one if meeting some (unspecified) lane keeping criterion Instruction to keep in the middle of the lane Yes No Cognitive task Group 1 Group 3 Group 2 Group 4 Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Prediction Lane keeping improvement effect of cognitive load should only occur when the driver is not motivated to optimise lane keeping = satisficing Interaction between cognitive load and instruction to optimise Vehicle & Load Structures 2018-12-01

Preliminary results: Lane keeping (HP-filtered SD Lane Position) No cognitive task Effect only for non-instructed subjects Due to satisficing in baseline condition Cognitive task Instructed to optimise lane keeping No instruction Vehicle & Load Structures 2018-12-01

Steering wheel reversal rate Cognitive task Same effect in both conditions Cognitive load less efficient optimising: more steering – same lane keeping performance No cognitive task No instruction Instructed to optimise lane keeping Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Still to be analysed… Eye movements Speed change Performance on cognitive task Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures Discussion Replicated earlier findings for non-instructed drivers: Reduced lane keeping performance Increased steering wheel activity Predicted effect of instructions found -> improved lane keeping only for non-instructed drivers – due to satisficing in baseline condition Cognitive load seems to induce less efficient steering while optimising (more effort in steering, same result on lane keeping) Cognitive task does not really improve steering ability-> the effect rather reflects ”involuntary” improvement from ”sloppy” baseline driving Vehicle & Load Structures 2018-12-01

Vehicle & Load Structures General conclusions Caution is needed when interpreting driving performance measurements – do we compare to a baseline with satisficing or optimising performance? In this case, changing instructions and/or driving task difficulty may cancel or perhaps even reverse the effect of cognitive load Implies re-interpretation of many existing studies on the effects of cell phone conversation on driving performance (e.g. Strayer and Drews, 2001) Vehicle & Load Structures 2018-12-01