1. presentation by: Alon Baruch
Deep Learning and Its Applications to Signal and Image Processing and Analysis
presentation by: Alon Baruch

2. A Sensor Fusion Method Based on an Integrated Neural Network and Kalman Filter for Vehicle Roll Angle Estimation
By: Leandro Vargas-Meléndez, Beatriz L. Boada, María Jesús L. Boada, Antonio Gauchía and Vicente Día

3. outline Introduction – “smart” vehicle safety.
9/17/2018
Introduction – “smart” vehicle safety.
Problem description – vehicle roll angle estimation.
Presented solution – NN + Kalman Filter architecture.
Validation of a simulation model.
Simulation and experimental Tests.
Conclusions.

4. Introduction “smart” vehicles – vehicle safety.
9/17/2018
“smart” vehicles – vehicle safety.
Electronic Stability Control (ESC)
Roll Stability Control (RSC)
Detecting and reducing loss of traction.
Applies the breaks to help “steer” the vehicle.

5. Lateral stability ex
9/17/2018

6. Lateral stability ex
9/17/2018

7. Introduction - Roll Stability Control
9/17/2018
State Estimation –
High accuracy Inertial Measurement Unit (IMU).
Physical vehicle model.
Low cost sensors – sensor fusion algorithms.
Dual-antenna GPS.
And more types of specially designed and expensive sensors such as suspension deflection sensors…

8. Problem definition
9/17/2018
To estimate the vehicle roll angle 𝜙 in order to control the vehicle angle and prevent a roll over.

9. The presented solution:
9/17/2018
To train a Neuron Network to identify different states of the vehicle in according with the roll angle and to use the Network output as a pseudo measurement in the Kalman Filter

10. Estimator architecture
9/17/2018

11. Neural Network architecture
9/17/2018

12. NN- training The training data set is comprised of:
9/17/2018
The training data set is comprised of:
𝑥= 𝑎 𝑦𝑚 , 𝑎 𝑥𝑚 , 𝜙 𝑚 , 𝜓 𝑚 𝑎𝑛𝑑 𝑑= 𝜙 𝑑
Obtained from a simulator (TruckSim), which is experimentally validated.
The training data contain different movement types:
Double Lane Change(DLC) – from 30 to 140 km/h
Lane Change (LC) – from 30 to 140 km/h
J-turn maneuver– from 30 to 140 km/h
With different road friction coefficient.

13. Kalman Filter Optimal estimator (linear case).
9/17/2018
Optimal estimator (linear case).
In the discrete state space system:
𝑥 𝑘 =𝐴 𝑥 𝑘−1 + 𝑤 𝑘
𝑦 𝑘 =𝐻 𝑥 𝑘 + 𝑣 𝑘
estimate the state vector 𝑥 given the measurement 𝑦.
𝐴 is the state evolution matrix.
𝐻 is the observation matrix.
𝑤 𝑘 , 𝑣 𝑘 are process uncertainty and measurement noise respectively assumed to be Gaussian, zero mean stationary and uncorrelated.
𝑤~𝑁 0,𝑄
𝑣~𝑁 0,𝑅

14. Side note: Kalman Filter equations
9/17/2018
Prediciton:
𝑥 𝑘+1 − =𝐴 𝑥 𝑘 +
𝑃 𝑘+1 − =𝐴 𝑃 𝑘 + 𝐴 𝑇 +𝑄
Correction:
𝐾 𝑘 = 𝑃 𝑘+1 − 𝐻 𝑇 𝐻 𝑃 𝑘+1 − 𝐻 𝑇 +𝑅 −1
𝑥 𝑘+1 + = 𝑥 𝑘+1 − + 𝐾 𝑘 𝑦−𝐻 𝑥 𝑘+1 −
𝑃 𝑘+1 + =(𝐼− 𝐾 𝑘 𝐻) 𝑃 𝑘+1 −

15. State space model The state vector: 𝑥 𝑠 = Δ 𝐹 𝑧𝑙 , 𝑎 𝑦 , 𝑎 𝑦 ,𝜙, 𝜙
9/17/2018
The state vector:
𝑥 𝑠 = Δ 𝐹 𝑧𝑙 , 𝑎 𝑦 , 𝑎 𝑦 ,𝜙, 𝜙
The measurement vector:
𝑦 𝑘 = 𝑎 𝑦 ,𝜙, 𝜙 ,Δ 𝐹 𝑧𝑙
Notice that the measurement 𝜙= 𝜙 𝑁𝑁 is the output of the NN

16. Experimental validation of the simulation
An experiment to adjust the model parameters
The target vehicle:
9/17/2018

17. Experimental validation of the simulation
9/17/2018
TruckSim is widely-used simulation software in the automotive industry.
The simulation parameters should be adjusted to the vehicle model.
Simulation parameter adjustment:

18. Validation results
9/17/2018
DLC maneuver at 70 km/h

19. Validation results
9/17/2018
Good Enough?
DLC maneuver at 70 km/h

20. Validation results 𝐸 𝑡 - MSE normalized error
9/17/2018
𝐸 𝑡 - MSE normalized error
𝐸 𝑚𝑎𝑥 - maximum MSE error

21. Simulation test
After the simulation was validated in respect to the real vehicle, the estimator can be tested in simulation
2 maneuvers were tested:
Sine sweep at 50/70 km/h
Slalom at 35 km/h
Comparison between the proposed architecture and a pseudo measurement taken by suspension deflection sensors:
Δ 𝑖𝑗 - suspension deflection at position 𝑖,𝑗 (back-forth,front-rear)
𝑚 𝑣 - vehicle weight.
𝑘 𝑡 - roll stiffness (tire stiffness)
9/17/2018

22. Simulation test 1 LKF – Linear Kalman Filter NN – Neuron Network
9/17/2018
LKF – Linear Kalman Filter
NN – Neuron Network
DEF – Suspension deflection
Measured - simulated
Slalom maneuver at 35 km/h with a friction coefficient of 0.3

23. Simulation test 2 LKF – Linear Kalman Filter NN – Neuron Network
9/17/2018
LKF – Linear Kalman Filter
NN – Neuron Network
DEF – Suspension deflection
Measured - simulated
Sweep maneuver at 70 km/h with a friction coefficient of 0.3

24. Simulation summary
9/17/2018
The normalized error of the proposed method is smaller.
But the maximum error isn’t smaller for half of the presented cases.

25. Experimental test Experimental testing of different maneuvers between:
9/17/2018
Experimental testing of different maneuvers between:
DEF-LKF NN
NN-LKF
2 maneuvers were tested:
Sine sweep at 50/70 km/h
Slalom at 35 km/h

26. Experimental test 1
9/17/2018
(a)-DLC at 70 km/h (b)-LC at 70 km/h

27. Experimental test 3
J-turn and Slalom maneuver
9/17/2018

28. Experimental test 3
9/17/2018

29. Results summary 100% better result for the norm error
80% better result for the maximum error
In the worst case the maximum difference between the estimators roll angle is 0.197° which is negligible.
Using just the NN isn’t enough
9/17/2018

30. Conclusion
The NN module estimate a pseudo measurement that is feed into a linear KF.
The proposed method only use an IMU and doesn’t require more sensors which are expensive and aren’t always available.
The estimator was validated by a simulation and an experiment, with different maneuvers, speed and friction coefficients.
The proposed estimator improved the overall results in 71% of the cases that were analyzed.
Using just the NN isn’t enough
9/17/2018

31. Appendix A1: KF equations
9/17/2018

32. Appendix A2: KF equations
9/17/2018
The lateral load transfer function:
Where ℎ 𝑓 , h r are the heights of the front and rear roll centers respectively,
𝑘 𝑓 , 𝑘 𝑟 - roll stiffness
𝑒 𝑓 , 𝑒 𝑟 - vehicle tracks
𝑙 𝑓 , 𝑙 𝑟 - distance from the Center Of Gravity (COG) to the front and rear axles.
Lateral acceleration measurement:
Assuming small 𝜙:

33. Appendix B: vehicle model parameters
9/17/2018

34. Appendix C: Experimental test 2
9/17/2018
Slalom maneuver at 45 km/h

35. Appendix D: Experimental test 3
J-turn and DLC maneuver
9/17/2018

36. Appendix D: Experimental test 3
9/17/2018