Cooperative crash prevention using human behavior monitoring Susumu Ishihara*† and Mario Gerla† (*Shizuoka University / †UCLA) Danger ! ! !

I started living in LA Aug

I bought a used car on Sep ,000 mi – Corolla 2005

Right front wheel was slanted! 5 min. after leaving the car shop, the car was crashed!

What I thought then I have to concentrate on driving. I do not want to die on the road. I have to concentrate on driving. I do not want to die on the road. But if the driver of a car behind is sleepy, my car might be crashed even if I am careful. But if the driver of a car behind is sleepy, my car might be crashed even if I am careful. What can I do to protect myself? What can I do to protect myself?

The driver behind you may be…

and you may be

Dozens of people have seizures while driving each year, causing collisions.

Vehicular Collision avoidance Car2Car communication and Sensors (RADAR, LIDAR, Camera) are (/will be) used for vehicular collision avoidance. Car2Car communication and Sensors (RADAR, LIDAR, Camera) are (/will be) used for vehicular collision avoidance. – But the requirement of communication performance for collision avoidance is not clear. Sensors can be used for monitoring driver’s behavior Sensors can be used for monitoring driver’s behavior – Apple watch may detect that you are sleepy. Message Frequency? Message reliability? Transmission Range? 9

Cooperative crash prevention using human behavior monitoring Automated Vehicle Platoon / Normal Vehicles The driver is distracted The driver is sleepy How to react to the alarm? We need a longer distance to to other cars to prevent an accident. Alarm Zzz… Alarm Change the signal pattern to make a space V2V V2I 10

Issues Driver’s behavior sensing – When to send alarm? User interface to the danger driver and other drivers. How other vehicles/drivers react to the alarm? Credibility of alarm Privacy / Incentive to provide sensor data Feasibility - Is it useful in real environments? 11

Driver’s behavior sensing What to sense – Sleepiness / Excitement / Seizure – Chatting / Looking away / Eating – Others. (Natural born danger driver?) Sensors – Camera (incl. IR-camera) – Microphone – Heartbeat with wearable devices, seat – Steering wheel / Pedal – GPS, CAN(Controller Area Network) – Sensors of other car – Camera 12

Issues Driver’s behavior sensing – When to send alarm? User interface to the danger driver and other drivers. How other vehicles/drivers react to the alarm? Credibility of alarm Privacy / Incentive to provide sensor data Feasibility - Is it useful in real environments? 13

User Interface Be careful! The driver of the car behind you is sleepy. Yeah, … but me too. Vibration Blinking light 14

Issues Driver’s behavior sensing – When to send alarm? User interface to the danger driver and other drivers. How other vehicles/drivers react to the alarm? Credibility of alarm Privacy / Incentive to provide sensor data Feasibility - Is it useful in real environments? 15

How other vehicle react to the alarm? To avoid collisions, vehicles should move adaptively to the behavior of the danger vehicle. – Predicting the movement of the danger car. We need a lot of data of drivers to obtain the model of danger cars! How to collect the data? / incentive? Zzz… Where does the car go? F(driver_type, sleepiness, interface_type, speed, distance to other cars, …) 16

Driver Model Sensors To: Other cars Zzz Steering, Speed up / slow down Visual /Sound Information Alert: “Danger Car” Assistant Message e.g. “Slow down” Visual /Sound Information Steering, Speed up / slow down ! Alert Controller Interface “Wake up!” Interface 17

Issues Driver’s behavior sensing – When to send alarm? User interface to the danger driver and other drivers. How other vehicles/drivers react to the alarm? Credibility of alarm Privacy / Incentive to provide sensor data Feasibility - Is it useful in real environments? 18

Credibility of alarm False positive … – Makes the system unbelievable “He is sleepy.” “Really?” – May commit libel If many (incredible) alarm signals generated? – Need Filtering? If malicious drivers/cars generates fake alarm? If the alarm generating device is broken? If the wireless network is congested? X: I’m awake. “X is sleepy.” 19

Issues Driver’s behavior sensing – When to send alarm? User interface to the danger driver and other drivers. How other vehicles/drivers react to the alarm? Credibility of alarm Privacy / Incentive to provide sensor data Feasibility - Is it useful in real environments? 20

Privacy Do you provide your “sleepiness” information to others? – NO? – But if your car insurance is discounted by providing the information? – But if the system may produce false positive…? Before we start the service, we need to collect large amount of data about sleepiness, etc. of drivers. 21

Issues Driver’s behavior sensing – When to send alarm? User interface to the danger driver and other drivers. How other vehicles/drivers react to the alarm? Credibility of alarm Privacy / Incentive to provide sensor data Feasibility - Is it useful in real environments? 22

Feasibility Real-World Testbed – Important, but Expensive Simulators – Humans, Alarm, Vehicles, Road, Communication 23 Veins: Multiple driver models, vehicle movement + wireless comm. Scenargie: Multi-Agent simulator, GIS, wireless comm. Is it useful in real environments?

Conclusions Let’s use driver’s behavior info for ARAMING to other cars Issues – Driver’s behavior sensing /When to send alarm? – User interface to the danger driver and other drivers. – How other vehicles/drivers react to the alarm? – Credibility of alarm – Privacy / Incentive to provide sensor data – Feasibility - Is it useful in real environments? 24

Thank you! Susumu Ishihara

Diffusion Scenario First step – Stand alone – My car / smart phone alerts me – Owner of the system will be safe, if the he/she realizes the alert even if he/she is sleepy. Second step – Cooperation (with low penetration ratio) – Cars / smart phones of others alert me – Owner of the system (capable of receiving the alert) will be save even if other drivers are sleepy Third step – Every car has the system – We all are safe

Driver Behavior Databases Japan – 100 drivers, Total 30,000 km drive on real roads – Data Location, Speed, Acceleration Pedal (Accel. Brake) Steering – No data about driver’s face, eye, and body movement

Feasibility Is it useful in real environments? We need the system requirements – Reliability of sensing / Credibility of Alarm – Communication characteristics, etc. How to validate the system? – Test on real roads … Danger / High cost – Simulation Driver model – sleepy drivers, normal drivers Vehicle model – cars, trucks, motorcycle, bikes, ped. Road model – Intersection, Highway, etc. Communication model - New protocol needed? 29