Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus Federico Ferrari PhD Defense October 18, 2013 — Zurich, Switzerland Computer Engineering and Networks Laboratory

Cyber-Physical Systems (CPSs) Tightly integrate physical processes, computation, and communication Safety-critical control loops – Sensors gather data from the environment – Actuators react according to a control law October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus2 Physical processes Computation Communication

Safety-critical CPS application Most of the existing CPS communication protocols operate in a best-effort manner Infrastructure controlMedical systemsEnvironmental monitoring and control … Dependability Gap in Current CPSs October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus3

October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus4 – Resource-constrained wireless embedded devices Communication Challenges in CPSs [Tmote Sky] Tight physical integration → Severe constraints

October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus – Resource-constrained wireless embedded devices – Multi-hop network topologies that vary over time 5 Communication Challenges in CPSs Tight physical integration → Severe constraints

October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus6 – Resource-constrained wireless embedded devices – Multi-hop network topologies that vary over time – Operate for consecutive months/years Communication Challenges in CPSs Tight physical integration → Severe constraints How to design efficient protocols that provide also delivery guarantees?

Looking for Inspiration: Safety-Critical Wired Embedded Systems Based on time-triggered, shared buses – Time-Triggered Protocol (TTP) [Kopetz et al., FTCS 1993] – FlexRay [FlexRay Consortium, 2005] Successfully employed in automotive, avionics October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus7 Can we apply similar networking designs to CPSs?

Our Wireless Bus Conjecture A time-triggered communication infrastructure for multi-hop low-power wireless networks – Common notion of time – Communicate as if connected by a shared bus October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus8 It is possible to enable dependable yet efficient communication in CPSs by employing a wireless bus

Multi-hop low-power wireless network October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus9 Building a Wireless Bus Dependability gap Safety-critical CPS application

Multi-hop low-power wireless network One-to-all communication Global time synchronization Glossy Chapter 2[IPSN 2011] Building a Wireless Bus October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus10

Low-Power Wireless Bus , , Multi-hop low-power wireless network One-to-all communication Global time synchronization Glossy Chapter 2[IPSN 2011] Time-triggered operation Adaptive scheduling LWB Chapter 3[SenSys 2012] Building a Wireless Bus October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus11

, , Low-Power Wireless Bus , , V IRTUS , , Multi-hop low-power wireless network One-to-all communication Global time synchronization Glossy Chapter 2[IPSN 2011] Time-triggered operation Adaptive scheduling LWB Chapter 3[SenSys 2012] Delivery guarantees Failure management V IRTUS Chapter 4[SRDS 2013] Building a Wireless Bus October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus12 Safety-critical CPS application

Multi-hop low-power wireless network One-to-all communication Global time synchronization Glossy Chapter 2[IPSN 2011] Time-triggered operation Adaptive scheduling LWB Chapter 3[SenSys 2012] Delivery guarantees Failure management V IRTUS Chapter 4[SRDS 2013] Safety-critical CPS application Fast and reliable flooding of messages Accurate global time synchronization Hide complexity of multi-hop networks October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus13 Glossy: Objectives

Challenges for Efficient Flooding How to relay packets efficiently and reliably? Avoid aggressive, uncoordinated broadcasts Typical approach: Coordinate packet transmissions – CF [Zhu et al., NSDI 2010] – RBP [Stann et al., SenSys 2006] – Maintain topology-dependent state October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus14 initiator

Glossy Flooding Architecture All receiving nodes relay packets synchronously – Simple, but radically different solution – No explicit routing – No topology-dependent state Key Glossy mechanisms – Start execution at the same time – Compensate for hardware variations – Ensure deterministic execution timing October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus15 initiator

Propagation in Glossy October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus16 Rx Proc. Tx Proc. Tx Proc. Tx Proc. Tx Rx Tx Proc. Tx Proc. Tx Proc. Tx Rx Proc. Tx Proc. Tx Proc. Tx Rx Proc. Tx Proc. Tx Proc. Tx Proc. Tx Proc. Tx Rx Proc. Tx Proc. Tx Proc. Tx c = 0 c = 1c = 2c = 3c = 4c = 5 t (In this example a node transmits at most twice) A relay counter c is set to 0 at the first transmission A node increments c before relaying the packet initiator Rx Proc. Tx Proc. Tx Proc. Tx Proc. Tx Rx Tx Proc. Tx Proc. Tx Proc. Tx Rx Proc. Tx Proc. Tx Proc. Tx Rx Proc. Tx Proc. Tx Proc. Tx Proc. Tx Proc. Tx Rx Proc. Tx Proc. Tx Proc. Tx c = 0 c = 1c = 2c = 3c = 4c = 5 t

Time synchronization in Glossy October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus17 Reference time Constant relay length Rx Proc. Tx Proc. Tx Proc. Tx Proc. Tx Rx Tx Proc. Tx Proc. Tx Proc. Tx Rx Proc. Tx Proc. Tx Proc. Tx Rx Proc. Tx Proc. Tx Proc. Tx Proc. Tx Proc. Tx Rx Proc. Tx Proc. Tx Proc. Tx c = 0 c = 1c = 2c = 3c = 4c = 5 t Estimate the relay length during propagation Compute a common reference time initiator

Glossy: Main Evaluation Findings A few ms to flood packets to hundreds of nodes Reliability > % in most scenarios Synchronization error < 1 µs even after 8 hops October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus18

Multi-hop low-power wireless network One-to-all communication Global time synchronization Glossy Chapter 2[IPSN 2011] Time-triggered operation Adaptive scheduling LWB Chapter 3[SenSys 2012] Delivery guarantees Failure management V IRTUS Chapter 4[SRDS 2013] One-to-all communication Global time synchronization Glossy Chapter 2[IPSN 2011] Time-triggered operation Adaptive scheduling LWB Chapter 3[SenSys 2012] Delivery guarantees Failure management V IRTUS Chapter 4[SRDS 2013] Safety-critical CPS application A concrete wireless bus that: Adapts to varying conditions and demands Efficiently supports a wide range of scenarios Delivers messages with high reliability October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus19 LWB: Objectives

LWB Design Principles Bizarre idea: broadcast-only communication! – Multi-hop wireless network → Shared bus Synchronized, time-triggered operation – Collision-free and efficient bus accesses Centralized scheduling – A host node orchestrates all communication October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus20

LWB operation is confined to rounds A round consists of non-overlapping slots Each slot corresponds to a distinct Glossy flood October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus21 Round period Tt n1n2n3 n1 Time-Triggered Operation in LWB

Centralized, Adaptive Scheduling Demand response scheduling at the host Example scheduling policy – Minimize energy while providing enough bandwidth – Ensure fair allocation of slots October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus22 Low-Power Wireless Bus Host ResponseDemand

Schedule: sent by the host H, also for time-sync Data: messages transmitted by senders S1, S2, etc. Requests: competed by senders to join LWB Tt H Schedule not allocated Requests S1 Data … S2 Data LWB Activity during a Round October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus23 Host: compute schedule

LWB Additional LWB Mechanisms October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus24 Host failover policy Support for nodes joining and disconnecting Optimizations for energy efficiency Prompt adaptation to traffic changes

LWB: Main Evaluation Findings (4 testbeds, 7 state-of-the-art protocols, 256 runs, 838 hours) The same LWB prototype: Is efficient under a wide range of traffic loads Supports mobile nodes with no performance loss Is minimally affected by interference or failures October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus25

90 nodes Varying senders 8 receivers Reliability and Energy Efficiency with Many-to-Many Communication October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus26 LWB outperforms state of the art Reliability Energy efficiency LWB outperforms state of the art Reliability Energy efficiency

Multi-hop low-power wireless network One-to-all communication Global time synchronization Glossy Chapter 2[IPSN 2011] Time-triggered operation Adaptive scheduling LWB Chapter 3[SenSys 2012] Delivery guarantees Failure management V IRTUS Chapter 4[SRDS 2013] One-to-all communication Global time synchronization Glossy Chapter 2[IPSN 2011] Time-triggered operation Adaptive scheduling LWB Chapter 3[SenSys 2012] Delivery guarantees Failure management V IRTUS Chapter 4[SRDS 2013] Safety-critical CPS application Provide guarantees on message delivery – In the face of communication failures – In the face of node crashes Keep overhead low compared with LWB October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus27 V IRTUS : Objectives

Key V IRTUS Mechanisms Guarantee virtually-synchronous executions – All nodes see the same events in the same order Delivered messages Joining and failing nodes Atomic multicast – Deliver messages reliably and with total order Group management – Share information on currently active nodes October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus28 (Formally proven)

New Interactions in V IRTUS View: set of active nodes, sent by the host H Ack: receivers R1, R2, etc. buffer received data and send the content of their buffers October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus29 Tt H Schedule not allocated Requests S1 Data … Host: compute schedule S2 Data H View R1 Ack R2 Ack … and update view

V IRTUS provides delivery guarantees while outperforming existing best-effort solutions 90 nodes 45 senders Varying receivers V IRTUS Efficiency October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus30

Conclusions Wireless bus: delivery guarantees and efficiency Novel solutions Narrows the current dependability gap in CPSs October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus31 Multi-hop low-power wireless network One-to-all communication Global time synchronization GlossyChapter 2[IPSN 2011] Time-triggered operation Adaptive scheduling LWBChapter 3[SenSys 2012] Delivery guarantees Failure management V IRTUS Chapter 4[SRDS 2013] Safety-critical CPS application ➤ Multi-hop broadcasts have become cheap! ➤ Efficient support for multiple traffic patterns ➤ First to provide virtual synchrony to CPSs

October 18, 2013Enabling Dependable Communication in Cyber-Physical Systems with a Wireless Bus32 Multi-hop low-power wireless network One-to-all communication Global time synchronization GlossyChapter 2[IPSN 2011] Time-triggered operation Adaptive scheduling LWBChapter 3[SenSys 2012] Delivery guarantees Failure management V IRTUS Chapter 4[SRDS 2013] Safety-critical CPS application