1. The challenges facing an autonomous car’s risk assessment Nick Durston, Senior Consultant

2. The challenges facing an autonomous car’s risk assessment A compelling argument for the introduction of autonomous cars onto UK roads; Autonomy - “one who gives oneself one’s own law”; Resilience to dynamic air and ground environments 2

3. A compelling argument for the introduction of autonomous cars onto UK roads Economic -Increased productivity Environmental -Minimised fuel consumption and emissions Social -Increased mobility Safety -Substantial reduction in collisions, deaths and injuries 3

4. A compelling argument for the introduction of autonomous cars onto UK roads Extant legal & regulatory frameworks will present a substantial challenge. Significant public concern exists regarding the responsible use of, security and safety of autonomous cars. Automotive must prove that there is no inherent danger to the public through a rigorous development process. Aerospace has produced results acceptable to the public, by means of organisations working together to harmonise stakeholder requirements. 4

5. 5 Autonomy - “one who gives oneself one’s own law” Paris 1914 Concours de la Securité en Aéroplane -Lawrence Sperry & Emil Cachin Curtiss C-2 biplane

6. Autonomy - “one who gives oneself one’s own law” Autonomy: -Freedom from external control or influence Autonomous Aircraft: -ICAO: An unmanned aircraft that does not allow pilot intervention in the management of flight. -CAA: All UAS are required to perform deterministically; No UAS currently meet the definition of autonomous; UAS are either: Highly automated; High authority automated. 6

7. Autonomy - “one who gives oneself one’s own law” Highly automated: -Still require inputs from a human operator: Confirmation of a proposed action but can implement the action without further human interaction once the initial input has been provided. High authority automated: -Evaluate data, select a course of action & implement that action without the need for human input: Take actions & respond through evaluation of a given dataset that represents the current situation including the status of all the relevant systems, geographical data & environmental data. 7

8. Autonomy - “one who gives oneself one’s own law” ASTREA II project : -Identified autonomous behaviour as one of the critical technologies that will make civil UAS operations viable. -Focused on specific capabilities & functionality, giving full consideration to equivalence to manned aircraft. -Autonomy a human centric process. 8

9. Autonomy - “one who gives oneself one’s own law” Why is autonomy important for unmanned systems? -Autonomous systems can replicate & augment the system monitoring & contingency management functions that would otherwise be performed by the human operator. -Such systems can supplement overall system situational awareness & permit continued safe operation in the event of system degradation. 9

10. Autonomy - “one who gives oneself one’s own law” 10 PACT Level Computer autonomyPACT locus of authority Levels of Human Machine Interface 5bComputer monitored by pilotFullComputer does everything autonomously 5a Computer chooses action, performs it and informs human 4bComputer backed up by pilotAction unless revoked Computer chooses action and performs it unless human disapproves 4a Computer chooses action and performs it if human approves 3Pilot backed up by computer Advice, & if authorised, action Computer suggests options and proposes one of them 2Pilot assisted by computerAdviceComputer suggests options to human 1 Pilot assisted by computer only when required Advice only if requestedComputer suggest options and human selects 0PilotNone Whole task done by human except for actual operation

11. Autonomy - “one who gives oneself one’s own law” 11 SAE J0316 Level NameNarrative Definition Human driver monitors the driving environment 0No automation The full-time performance by the human driver of all aspects of the dynamic driving task, even when enhanced by warning or intervention systems 1Driver assistance The driving mode-specific execution by a driver assistance system of either steering or acceleration/ deceleration using information about the driving environment and with the expectation that the human driver perform all remaining aspects of the dynamic driving task 2Partial automation The driving mode-specific execution by one or more driver assistance systems of both steering or acceleration/ deceleration using information about the driving environment and with the expectation that the human driver perform all remaining aspects of the dynamic driving task Automated driving system monitors the driving environment 3 Conditional automation The driving mode-specific performance by an automated driving system of all aspects of the dynamic driving task with the expectation that the human driver will respond to a request to intervene 4High automation The driving mode-specific performance by an automated driving system of all aspects of the dynamic driving task, even if a human driver does not respond appropriately to a request to intervene 5Full automation The full-time performance by an automated driving system of all aspects of the dynamic driving task under all roadway and environmental conditions that can be managed by a human driver

12. Autonomy - “one who gives oneself one’s own law” Dynamic Driving/Flying Task: -Operational (physical inputs of control); -Tactical (response to driving/flying environment). -Does not include the: Strategic aspect (determining destination, waypoint and route). Driving Mode/Phase of Flight -A type of driving/flying scenario with characteristic dynamic driving/flying task requirements, e.g. high speed motorway cruising/initial climb. Request to Intervene -Notification to a human operator, by the autonomous driving/flight system, prompting initialisation or recommencement of the dynamic driving/flying task. 12

13. Autonomy - “one who gives oneself one’s own law” Common technological themes: -Sense & avoid; -Communications security & spectrum; -Autonomy, decision making & contingency management; -Operations & human systems interaction. 13

14. Resilience to dynamic air & ground environments 14 Human Error: -93% of road traffic accidents; -1.3 million fatalities & 50 million injuries (globally & annually). Appropriate & critical autonomous system analysis & response is required for any system replicating the human functions of driving a car in a complex ground environment.

15. Resilience to dynamic air & ground environments Airspace is heavily controlled and regulated & supported by systems: -TCAS; -PSR; -SSR. Vehicles operating on roads, generally do not have such external control and support systems: -Greater dependence on see & avoid. 15

16. Resilience to dynamic air & ground environments Aerospace industry design complies with a strict regulatory framework ensuring interoperability. Aerospace flight control systems are developed around the flight envelope of the aircraft & the rules of the air. Autonomous cars will initially need to operate in a mixed environment: -Need to understand the rules of the road (the Highway Code & the legal framework, the Road Traffic Act 1988): Signage; Road conditions; Weather conditions; Interaction with vehicles (both driver operated and autonomous). 16

17. Resilience to dynamic air & ground environments Aircraft are significantly more complex than cars, however they operate in a far simpler environment. The complexity of an autonomous car’s operating environment should not be underestimated. A strategy for the deployment of autonomous cars needs to be developed i.a.w SAE J3016. 17

18. Resilience to dynamic air & ground environments Autonomous cars will either: -Need to be sufficiently sophisticated, requiring decision making capability, with total independence & no reliance / support from roadside infrastructure. -Require a complete support network from roadside infrastructure deploying a vast array of sensors & situational awareness systems. 18

19. Resilience to dynamic air & ground environments Autonomous systems will need to be able to monitor the environment and conduct dynamic risk assessments. Distinguishing between: -Day & Night; -Weather conditions; -Changes in road infrastructure; -Regional differences & travelling abroad. As a minimum, the setting of system and software assurance levels & rigorous testing is required. 19

20. Resilience to dynamic air & ground environments Tesla Motors Autopilot: -Model S is equipped with: forward radar; forward-looking camera; 12 long-range ultrasonic sensors; high-precision digitally-controlled electric assist braking system. 20

21. Resilience to dynamic air & ground environments Tesla Motors Autopilot: -Software release designed to work in conjunction with the automated driving capabilities & has enabled real-time data feedback. -Permits a Model S to steer within a lane, change lanes by activation of the turn signal, & manage speed by using active, traffic-aware cruise control. -The driver is still responsible for & ultimately in control of, the car, however the human machine interface provides intuitive access to the information the car is using to inform its driver of its actions. 21

22. Resilience to dynamic air & ground environments Monitoring operating environments & conducting dynamic risk assessments: -Level of authority over navigational commands may differ during the mission. -Dependent upon any safety of flight risks to the UAV & the time available for the human operator to intervene effectively. -Such systems are highly dependent on two types of safety-critical data when determining responses to hazardous situations: Data sourced from on board sensors; High integrity data sets. 22

23. Resilience to dynamic air & ground environments A key factor in mitigating hazards presented to vehicles when operating in any domain is the immediacy of response afforded to the human operator. How will such systems be sufficiently sophisticated to differentiate between hazards and respond to them in a given timeframe, without degrading the safety performance benchmarked by manned vehicles? 23

24. Resilience to dynamic air & ground environments A key factor in mitigating hazards presented to vehicles when operating in any domain is the immediacy of response afforded to the human operator. How will such systems be sufficiently sophisticated to differentiate between hazards and respond to them in a given timeframe, without degrading the safety performance benchmarked by manned vehicles? 24

25. Resilience to dynamic air & ground environments Some accidents will be inevitable, since some situations will require an autonomous car to make a decision which could result: -In running over a pedestrian on the road; -A passer-by on the side; -Alternatively choosing whether to run over a group of pedestrians; -Sacrifice the occupants in the autonomous car by driving into an obstacle. Defining the algorithms responsible for guiding autonomous systems when confronted with moral dilemmas will present a major challenge. 25

26. Conclusions The absence of a clear definition of Autonomy presents a major challenge to the introduction of such technologies. The aerospace industry has responded by developing frameworks, driven by initiatives such as PACT. These frameworks have been adopted by other industries with interests in autonomous technologies, but have been revised appropriately for suitability of use. 26

27. Conclusions A successful reduction in road traffic accidents can be realised following the successful development & integration of dependable & resilient systems. However, these systems need to be able to monitor the environments in which they operate & conduct dynamic risk assessments. Successful reductions in accidents & human error are dependent on the on the amount of system authority given to specific tasks & responses to real time environmental situations. Appropriate & critical autonomous system analysis & response is required for any system replicating the vast human functions concerned with driving a car in such a complex ground environment. 27

28. Conclusions A key factor in mitigating hazards presented to vehicles when operating in any domain is the immediacy of response afforded to the human operator. How will autonomous cars make decisions during situations involving imminent unavoidable harm? The necessity for algorithms responsible for ethical decisions accentuates the requirement for new & revisions to extant legal & regulatory frameworks. 28

29. Conclusions The ultimate goal of increasing public confidence that the autonomous systems in operation are safe can only be achieved through demonstration of compliance to safety requirements, confirming transparency & equivalence to existing manned systems. 29

30. Thank you! 30 Contact us: enquiries@ospreycsl.co.uk www.ospreycsl.co.uk +44 1420 520200 Further Discussion & Questions