1. Performance and Capability of a Road Departure Crash Warning System Bruce Wilson, Ph.D. and Jonathan Koopmann Advanced Vehicle Safety Technology Division Volpe National Transportation Systems Center Cambridge, MA

2. 2 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann The Problem 1,200,000 annual Road-Departure Crashes in US = 3,300 per day = 137 per hour Severe collisions - collide with fixed objects or rollover 19 percent of collisions, 43 percent of fatalities Step Towards Solution Field Operational Test (FOT) of a Road Departure Crash Warning System (RDCW) Subjects - 78 Participants –Male, female –Younger (20-30), Middle-aged (40-50), Older (60-70)

3. 3 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Road Departure Crash Warning System (Device) Two subsystems – Lateral-Drift Warning (LDW) – Curve-Speed Warning (CSW) LDW monitors: – lane position, lateral speed – Available Maneuvering Room (AMR) – alerts when vehicle likely to depart lane or road CSW monitors: – speed, upcoming curvature – alerts when vehicle approaching upcoming curve too fast

4. 4 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Project Participants

5. 5 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Data Sources and Analyses ObjectiveSystem Characterization TestFOT Objective DataFOT Subjective Data Availability and Accuracy Analysis of system availability for lighting and road conditions Analysis of lighting and road availability Comparison between RDCW and independent measurement system LDW Alert Logic Imminent alert TTC analysis for various lateral drift scenarios Timing of auditory and haptic warnings Alert need analysis for different lateral drift conditions True and false positive rates by conditions Necessity of warnings, adverse weather performance CSW Alert Logic Imminent alert TTC analysis for various curve approach scenarios Timing of auditory and haptic warnings Alert need analysis under different curvature conditions Necessity of warnings, adverse weather performance DVI Distinguish information in all lighting conditions, hear warnings while driving, recognize warning direction; discern seat vibration and direction

6. 6 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann RDCW Characterization: Method Independent Measurement System (NIST- developed, “ground truth”) 1,300 km of public road driving Series of maneuvers Analyzed: –Availability –Measurement accuracy –Alert timing –False positives, false negatives –Survey responses

7. 7 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann System Availability and Accuracy Availability – RDCW able to issue an alert Accuracy – how accurately RDCW estimates lateral speed, road edge, lane markings, solid objects “Ground truth” – independent measurement system (IMS) provided second set of reference measurements

8. 8 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann FOT LDW Availability by Road Type Road TypeLeft onlyRight onlyLeft and Right None Unknown VDT (km)100 800 Row Percent5%3%47%45% Freeway VDT (km)1,3001,50043,40010,600 Row Percent2%3% 76% 19% Non-Freeway VDT (km)4,7003,60022,90032,400 Row Percent7%6%36% 51% Average Row Percent5%4%55%36%

9. 9 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann FOT LDW Availability by Lighting and Atmosphere LightingLeft onlyRight onlyLeft and Right None Day, dry VDT (km)4,3004,20050,70032,100 Row Percent5% 56% 35% Day, wet VDT (km)2003002,500 Row Percent4%5%46% Dark, dry VDT (km)1,40070013,8007,700 Row Percent6%3% 58% 32% Dark, wet VDT (km)001001,600 Row Percent1%0%4% 95% Average Row Percent5%4%55%36%

10. 10 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann FOT CSW Availability by Road Type Road TypeAvailableUnavailable Unknown VDT (km)1,600400 Row Percent79%21% Freeway VDT (km)56,300800 Row Percent 99% 1% Non-Freeway VDT (km)64,4004,100 Row Percent 94% 6% Average Row Percent 96% 4%

11. 11 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Test Measurement Overview Characterization test data collected on public roads RDCW sensors supplemented with NIST-developed Independent Measurement System (IMS) –four cameras, differential GPS Key performance measures –Available maneuvering room, Time to collision –LDW alert need and timing, Time to curvature point of interest –Required deceleration, CSW alert timing and need

12. 12 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann AMR and TTC Comparison

13. 13 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Lateral Drift Alerts For shoulders less than 1 meter wide, the RDCW overestimated the width by 0.7 meters For shoulders more than 2 meters wide, the RDCW underestimated the width by 1.3 meters RDCW shoulder-width estimation errors contributed to: –1 in 8 false-negative alerts (alert needed but not issued) –1 in 3 false-positive alerts (alert issued but not needed) The RDCW had the highest percentage of true-positive alerts when the shoulder was 1 to 2 meters wide

14. 14 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Lateral Drift Sensitivity Testing with Wide Shoulders (> 2 m) Higher sensitivity resulted in earlier alerts (higher TTC) Wide shoulders and under- estimates of AMR resulted in large TTCs for alerts

15. 15 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann LDW Alert Issuance and Validity Alert neededAlert issued

16. 16 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann TTC by Shoulder Width for Solid Lane Boundary

17. 17 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann TTC when Crossing Dashed Boundary, datum 1 m outside boundary Alert typically issued 0.4 m after crossing dashed boundary

18. 18 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann TTC when Drifting Toward Jersey Barrier or Traffic Barrel Occasional incorrect radar readings caused inconsistent alert timing

19. 19 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann LDW Video Analysis and Survey Responses 3,800 alerts analyzed using video and numerical data 62 percent were true positive In dry weather 70 percent of the alerts were true positive; decreasing to 39 percent in rainy weather On wet surfaces 44 percent of the alerts were true positive On a scale of 1 to 7, participants rated the LDW alert timing favorably – above 5

20. 20 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann LDW Alert Classification by Maneuver (without lane changes) ManeuverFalse PosTrue PosRow Totals CountGoing Straight6999991698 Row Percent41.17%58.83% CountOn Curve37611541530 Row Percent24.58% 75.42% CountPassing423476 Row Percent55.26%44.74% CountMerging222143 Row Percent51.16%48.84% CountEntering Ramp221436 Row Percent61.11%38.89% CountAll Groups116222223384 Total Percent34.34%65.66%

21. 21 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann LDW Alert Classification by Atmosphere and Road Surface AtmosphereFalse PosTrue Pos Rain256165 Column %22.40%7.50% Row % 60.81% 39.19% Dry8872036 Column %77.60%92.50% Row %30.35%69.65% Totals11432201 Total %34.18%65.82% Surface Moisture False PosTrue Pos Other356283 Column %30.64%12.74% Row % 55.71% 44.29% Dry8061939 Column %69.36%87.26% Row %29.36%70.64% Totals11622222 Total %34.34%65.66% Rain and wet roads caused disproportionate number of false alerts

22. 22 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann LDW Alert Classification by Pavement Marking Pavement Marking False PosTrue PosRow Totals Atypical17472246 Column %14.97%3.24% Row % 70.73% 29.27% Typical98821503138 Column %85.03%96.76% Row %31.49% 68.51% Totals116222223384 Total %34.34%65.66%100.00% Atypical pavement markings, e.g., lane merge, caused disproportionate number of false alerts

23. 23 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Survey Responses to LDW Alert Timing Overall, I thought the LDW auditory warnings were provided at the right time (not presented too early or too late). Overall, I thought the LDW seat vibration warnings were provided at the right time (not presented too early or too late).

24. 24 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Survey Responses to LDW Alert Need The LDW always provided a warning when I thought it should. I did not receive any unnecessary LDW warnings. I did not receive any false LDW warnings

25. 25 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Curve Speed Alerts 1 of 5 alerts missed for curves with a radius less than 100 meters 2 of 5 alerts missed for curves with a radius greater than 100 meters Of the alerts issued on curves whose radius was less than 100 meters, 94 percent were on time, neither too early nor too late On a scale of 1 to 7, participants rated the CSW alert timing favorably; at 5

26. 26 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Alert Timing by Sensitivity Setting Higher speeds and higher sensitivity settings resulted in earlier alerts

27. 27 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann CSW Alert Issuance and Validity by Curve Radius Alert neededAlert issued

28. 28 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Alert Issuance near Exit Ramps Turn signal resulted in most likely path (MLP) switching to exit ramp With turn signal, ¾ ramp passes had alert Without signal, 10/11 ramp passes did not have alert

29. 29 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Survey Responses to CSW Alert Timing Overall, I thought the CSW auditory warnings were provided at the right time (not too early or too late). Overall, I thought the CSW seat vibration warnings were provided at the right time (not too early or too late).

30. 30 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann RDCW Characterization: Survey Responses, Timing LDW ratings slightly higher than CSW

31. 31 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann DVI Readability Participants approved of DVI visual elements, graphics, and performance

32. 32 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Warning Audibility Alert volume and directionality convey warning information to drivers

33. 33 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Summary of LDW FOT Performance 1 of 3 alerts issued was a false positive, AMR frequently underestimated Odds of nighttime alert being a false positive are 1.8 odds for daytime alert Odds of rain alert being a false positive are 3.6 odds for driving under dry conditions Half the alerts issued in construction zones were false positive Despite this – participants rated LDW quite favorably

34. 34 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Summary of CSW Characterization Test Performance Inaccurate estimates of the distance to the curve, the curve radius, or both compromised performance For typical curves, system failed to alert in 1 of 4 cases of excessive speed Map errors created some false curves and false alerts System performed well near ramps

35. 35 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Summary of Survey Data High ratings for: LDW and CSW alert timing, LDW and CSW missed alerts, LDW false positive performance Lower ratings for CSW false positive performance Participants recognized LDW limitations under poor lighting, rain, and wet roads 7 of 8 participants found it easy to interpret seat vibration alerts 4 of 5 found LDW audible alerts easy to interpret 6 of 7 found visual alerts easy to interpret

36. 36 RDCW Performance and Capability, Bruce Wilson and Jonathan Koopmann Video example of an LDW Alert