Alessandro Leoni, Felix Siegle September, 2017 Possible requirements for the SpaceFibre network layer End-to-End Determinism, FDIR, Network Discovery, Configuration & Status Alessandro Leoni, Felix Siegle September, 2017

Outline OSRA-NET requirements SpaceFibre determinism – Single master per Virtual Network SpaceFibre determinism – Multiple masters per Virtual Network Simulation results Network discovery and configuration Broadcast definitions Open discussion…

OSRA-NET Requirements OSRA-NET: set of high-level requirements for forthcoming satellite networks covering: Traffic classes (Synchronous, Asynchronous, Best Effort) FDIR Quality of Service / Reliability (At least one, at most one, exactly one) Redundancy …

SpaceFibre determinism – Single master Straightforward solution : only one master in each Virtual Network * Can exploit SpaceFibre built-in features Virtual channels are (virtually) isolated No contentions at the output ports of a switch *S. Parkes, A. F. Florit, A. G. Villafranca, C. McClements and D. McLaren, "SpaceFibre network and routing switch," 2017 IEEE Aerospace Conference, 2017. A failing node doesn’t affect the rest of the network

Multiple masters on the same Virtual Network Possible problems when using one master per Virtual Network: The network becomes very big >32 Virtual Networks sharing the same link >64 Virtual Networks overall The switch has limited capacities (output ports with few Virtual Channels)

Multiple masters on the same Virtual Network Need to multiplex multiple input virtual channels into the same output buffer Same problem of SpaceWire networks Possibly undefined blocking time SpFi Router VC 0 VC 0 VC 1 VC 1 VC 2 VC 2 VC 0 VC 1 VC 2 Now we have to find a rule to CUT OFF wormholes from failing nodes because we are SHARING the buffers

Multiple masters – Synchronous Time Critical Data Prevent that a node transmits outside of its time-slot Almost zero blocking time R0 B Dest TS=0 TS=1 TS=2 A B A produces data B produces data (faulty) C produces data Maximum estimated time Actual transmission time

Multiple masters – Synchronous Time Critical Data We want a Time-slots guardian with scheduling tables replicated inside the switch to: Be able to detect when a fault happens (Fault Detection) Prevent a faulty node from affecting other communications (Fault Isolation) 0 1 2 3 VC0 X X VC1 0 1 2 3 VC0 X X VC1 0 1 2 3 VC0 X X X X VC1 0 1 2 3 VC0 X X VC1 0 1 2 3 VC0 X X VC1

Multiple masters – Asynchronous Time Critical Data Wormhole timeout: close a wormhole when it is taking more time than allowed Round Robin is used to arbitrate input port. Worst-case blocking time Dest E Start timeout for D Kill D / Start timeout for E D produces data (faulty) Timeout Transmission time E produces data

OMNeT++ Network Simulator SpaceFibre/SpaceWire simulator Developed in the framework of the NPI at ESTEC Based on OMNeT++ (open-source, C++ library for discrete-event simulation) Noticeable features: Path and Logical Addressing Multicast/Group Adaptive Routing SpaceWire/SpaceFibre bridge for packets and time-codes (with some assumptions on the network level) Hardware-in-the-loop support through Ethernet (at packet level)

OMNeT++ Network Simulator Already existing nodes: SpaceFibre Codec SpaceWire Codec Switch RMAP Initiator/Target General purpose application to simulate traffic HIL Bridge Useful for: Extract network performance results Test a new protocol

Asynchronous traffic analysis atcNode0 and atcNode1 send asynchronous packets to spfiNode3 High priority Packets with predefined max length They share VN 1 Router4 uses wormhole timeout (8000ns) for them spfiNode2 sends BE data continuously to spfiNode3 Uses VN 2 Low priority VN2 VN1 VN1 Scenario 1: atcNode0 and atcNode1 behave correctly. atcNode1_max_latency = time_to_transmit_atcNode0_packet + delta Scenario 2: atcNode0 fails and sends infinite packet. atcNode1_max_latency = atcNode0_timeout + delta

Router input/output ports utilization Input port 2, vn 2 (spfiNode2, BE) Input port 1, vn 1 (atcNode1) Input port 0, vn 1 (atcNode0, failing) Output port 3, all vn 8000 ns Remaining data in the buffer

Synchronous traffic analysis stcNode0 and stcNode1 send synchronous packets to spfiNode3 High priority Packets with predefined max length They share VN 1, but on different time slots Router4 closes wormhole when timeslot changes spfiNode2 sends BE data continuously to spfiNode3 Uses VN 2 Low priority masterNode4 sends out broadcasts to advertise about the new timeslots VN2 VN1 VN1 ts:0 stcNode0 ts:1 stcNode1 ts:0 stcNode0 ts:1 stcNode1 ts:n/a

Router input/output ports utilization Broadcast msgs (timeslots) ts0 (stcNode0) ts1 (stcNode1) Input port 2, vn 2 (SpfiNode2, BE) Input port 1, vn 1 (stcNode1) Input port 0, vn 1 (stcNode0, failing) Output port 3, all vn ~2200 ns  latency depends only on the remaining data in the buffer

Required Routing Switch (and Node) Extensions: Network Discovery Network Discovery can be kept simple by requiring any node and routing switch to have a dedicated RMAP target reachable via VN0. Implementation complexity is not an argument, one could specify that only a very minimal set of RMAP functions must be supported (basically: 32-bit write and 32-bit read). We need to specify a memory map that is valid for all nodes and routing switches. A unique Device Class ID identifies the device. The Network Manager can look up the Device Class ID and simply consult a corresponding Electronic Data Sheet for more information. Address Range 0x00 – 0x3c Network Discovery Device Type, Device Class ID etc. 0x40 – 0x7c Device Status 0x80 – 0xbc Device Configuration 0x100 – 0x1fc SpFi-Device Specific Registers 0x200 – […] User Registers

Required Routing Switch (and Node) Extensions: Status and Configuration Similar to the SpaceFibre Port specification, we should also specify the basic status flags and configuration parameters for SpaceFibre routing switch implementations. At least the essential features should work the same across several implementations. Then, we should fix them to registers in the memory map, making it easy for the user to configure and monitor SpaceFibre nodes and routing switches without any knowledge about manufacturer-specific details! Address Range 0x00 – 0x3c Network Discovery Device Type, Device Class ID etc. 0x40 – 0x7c Device Status 0x80 – 0xbc Device Configuration 0x100 – 0x1fc SpFi-Device Specific Registers 0x200 – […] User Registers

Required Routing Switch (and Node) Extensions: Device-Specific Registers The next memory block allows an end node or routing switch to implement manufacturer-specific configuration and status registers that extend the basic set of functionality. The exact meaning of the bit fields is unknown. However, by reading the Device Class ID, the user can e.g. consult an Electronic Data Sheet that contains the required information. Address Range 0x00 – 0x3c Network Discovery Device Type, Device Class ID etc. 0x40 – 0x7c Device Status 0x80 – 0xbc Device Configuration 0x100 – 0x1fc SpFi-Device Specific Registers 0x200 – […] User Registers

Required Routing Switch (and Node) Extensions: User-Specific Registers The last memory block can be used to make unit-specific status and configuration registers available via VN0. For instance, if the SpaceFibre end point is within a payload instrument, the configuration of the instrument itself could be done through these registers. Since these registers are not SpaceFibre-related, their meaning is only known to the user and does not need to be included in Electronic Datasheets. (although one could create extended EDS that also include this information) Address Range 0x00 – 0x3c Network Discovery Device Type, Device Class ID etc. 0x40 – 0x7c Device Status 0x80 – 0xbc Device Configuration 0x100 – 0x1fc SpFi-Device Specific Registers 0x200 – […] User Registers

Required Routing Switch (and Node) Extensions: Definition of Broadcasts So far, only the broadcast mechanism is specified but not the meaning of a broadcast! We need to specify at least: A broadcast to transfer SpW time-codes through a SpFi network. A broadcast to update time and time-slot information throughout the network (redundant time masters?). Broadcasts that are generated in case of errors and that include an error code. Like the memory map, we have to specify a list of error codes for all common errors that can occur in a SpaceFibre node and routing switch. Broadcasts to execute basic FDIR actions in case the node or router is not reachable via RMAP  e.g. disable port, reset port, close wormhole… In addition, we could specify a “universal weapon” broadcast that allows simple 32-bit read or write operations. Then, any possible configuration and status field would be accessible in case of failures.

Open discussion… Is the “Single Master per Virtual Network” approach a real limitation (considering future architectures) ? If yes, what about the two methods proposed: Wormhole Timeout Can deal with Asynchronous traffic Worst-case assumption can lead to relatively high latency Timeslot guardian Very low latency (~0) even in worst-case assumption Network Discovery and Configuration Broadcasts definition They provide: - Fault Detection (Visibility) - Fault Isolation