Jennifer Rexford Fall 2014 (TTh 3:00-4:20 in CS 105) COS 561: Advanced Computer Networks Multipath TCP

Multipath Mobile user –WiFi and cellular at the same time High-end servers –Multiple Ethernet cards Data centers –Rich topologies with many paths Benefits of multipath –Higher throughput –Failover from one path to another –Seamless mobility 2

Bringing Multipath to the End Host Multiple addresses –One or more addresses at an end host –E.g., one per interface card Multiple paths –Sequence of links between sender and receiver –E.g., four-tuple of source and dest address and port Multiple subflows –Flow of TCP segments over an individual path –All associated with a single TCP connection 3

Review of TCP Protocol 4

Establishing a TCP Connection Three-way handshake to establish connection – Host A sends a SYN (open) to the host B – Host B returns a SYN acknowledgment (SYN ACK) – Host A sends an ACK to acknowledge the SYN ACK 5 SYN SYN ACK ACK Data A B Each host tells its Initial Sequence Number (ISN) to the other host.

Initial Sequence Number (ISN) Sequence number for the very first byte – E.g., Why not a de facto ISN of 0? Practical issue: reuse of port numbers – Port numbers must (eventually) get used again – … and an old packet may still be in flight – … and associated with the new connection Security issue: adversary injecting packets –Adversary may try to inject packets in a connection –… by guessing the Initial Sequence Number –… to send counterfeit packets to the receiving host –… e.g., counterfeit packets that reset the connection –Some firewalls change the ISN to further randomize 6

Step 1: A’s Initial SYN Packet 7 A’s portB’s port A’s Initial Sequence Number Acknowledgment Advertised window 20 Flags 0 ChecksumUrgent pointer Options (variable) Flags: SYN FIN RST PSH URG ACK A tells B it wants to open a connection…

Step 2: B’s SYN-ACK Packet 8 B’s portA’s port B’s Initial Sequence Number A’s ISN plus 1 Advertised window 20 Flags 0 ChecksumUrgent pointer Options (variable) Flags: SYN FIN RST PSH URG ACK B tells A it accepts, and is ready to hear the next byte… … upon receiving this packet, A can start sending data

Step 3: A’s ACK of the SYN-ACK 9 A’s portB’s port B’s ISN plus 1 Advertised window 20 Flags 0 ChecksumUrgent pointer Options (variable) Flags: SYN FIN RST PSH URG ACK A tells B it is okay to start sending Sequence number … upon receiving this packet, B can start sending data

Sequence Number 10 Host A Host B TCP Data ISN (initial sequence number) Sequence number = 1 st byte Byte 81

TCP Header 11 Source portDestination port Sequence number Acknowledgment Advertised window HdrLen Flags 0 ChecksumUrgent pointer Options (variable) Data Flags: SYN FIN RST PSH URG ACK

Receive Buffering: Flow Control Receive window size – Amount that can be sent without acknowledgment – Receiver must be able to store this amount of data Receiver tells the sender the window – Tells the sender the amount of free space left 12 Window Size Outstanding Un-ack’d data Data OK to send Data not OK to send yet Data ACK’d

TCP Header: Receive Window 13 Source portDestination port Sequence number Acknowledgment Advertised window HdrLen Flags 0 ChecksumUrgent pointer Options (variable) Data Flags: SYN FIN RST PSH URG ACK

Tearing Down the Connection Closing (each end of) the connection – Finish (FIN) to close and receive remaining bytes – And other host sends a FIN ACK to acknowledge – Reset (RST) to close and not receive remaining bytes 14 SYN SYN ACK ACK Data FIN ACK time A B FIN ACK

Extending TCP: TCP Options TCP header –Ten mandatory fields –Optional extension field (usually during handshake) Examples –Maximum segment size (MSS) –Window scaling –Support for Selected ACKs Unknown options –Ignored by receiving host Routers and TCP options –Should ignore them, passing them through unchanged 15 But, some middleboxes: (i) strip TCP options from some packets or (ii) drop packets with TCP options

Incremental Deployment Challenges 16

Keeping the Same Socket API Backwards compatibility with existing apps –Present the same socket API and expectations Establish the TCP connection in the same way –Create a socket to a single remote IP address/port –… and then add more subflows to the connection Work in all scenarios where regular TCP works –If a subflow fails, the connection should continue –… as long as some other subflow has connectivity 17

MPTCP in the Network Stack 18 From

Negotiating MTTCP Capability How do end-points know they both speak MPTCP? –During the 3-way SYN/SYN-ACK/ACK handshake What if middleboxes strip the TCP option? –On the SYN? On the SYN-ACK? 19

Negotiating MTTCP Capability Include capability on the ACK of the SYN-ACK? –What if the ACK is lost? –Carry on all subsequent packets What if the middlebox drops SYN packets with unfamiliar options? –Sender can retransmit lost SYN without the option –… and fall back to regular TCP behavior 20

Adding Subflows, Idealized How to associate a new subflow with the connection? –Use the source/destination IPs and ports How to start using the new subflow? –Simply start sending packets with new IP/port pairs –… and associate them with the existing connection How could two end-points learn about extra IP addresses for establishing new subflows? –Implicitly: one end-point establishes a new subflow, to already-known address(es) at the other end-point 21

Challenges: NAT Network Address Translators (NAT) –Problem: NAT changes the IP address and port number How to identify a connection? –Using a token established during connection set-up How to establish new subflows? –Allow one end-point to tell another about its addresses 22 NAT1 NAT2 WiFi LTE

Challenges: Security Security –Malicious parties creating subflows –To highjack (part of) the connection How to bootstrap security? –Include a random key during connection set-up –… and use it to verify authenticity of new subflows How to identify the connection on new subflows? –A token generated from the key How to authenticate the addition of subflows? –Exchanging nonces and computing message authentication codes using the keys 23

Establishing New Flows, Reality 24

Sequence Numbers Challenges across subflows –Out-of-order packets due to RTT differences –Access networks that rewrite sequence numbers –Middleboxes upset by discontinuous TCP byte stream –Need to retransmit lost packets on a different subflow Two levels of sequence numbers –Sequence numbers per subflow –Sequence numbers for the entire connection Enables –Efficient detection of loss on each subflow –Retransmission of lost packet on a different subflow 25

Sending the Second Sequence # Mapping of subflow bytes to data sequence space –To associate segments in a subflow –… with their position in the connection Encoding the information –Send the additional data in TCP options –Carry the additional data in the TCP payload Problems with using TCP payload –Flow control may keep receiver from sending a data ACK –Simply exclude the data-ACKs from flow control? –… but middleboxes won’t know to do this…  26

Sending the Second Sequence # So, encode in TCP options Encode as a mapping –Subflow sequence # to connection sequence #? But, middleboxes may rewrite subflow sequence # Instead, map as an offset –Offset from the subflow’s initial sequence number –… robust to middleboxes that shift the sequence numbers Middleboxes that modify contents (and length)? –Include checksum in the mapping –… and terminate the subflow if modifications occur 27

Receive Buffer Space Each TCP connection has a receive buffer –Buffer space to store incoming data –… until it is read by the application TCP flow control –Receiver advertises the available buffer space –… using the “receive window” Should each subflow have its own receive window? –Starvation of some subflows in a connection? –Fairness relative to other TCP connections? –Fragmentation of the available buffer space? Instead, use a common receive window 28

Use of Multipath TCP in iOS 7 Multipath TCP in iOS 7 –Primary TCP connection over WiFi –Backup TCP connection over cellular data Failover –If WiFi becomes unavailable… –… iOS 7 will use the cellular data connection For destinations controlled by Apple –E.g., Siri 29

Multipath Congestion Control Slides from Damon Wischik 30

Goal #1: Fair at Shared Bottlenecks 31 To be fair, Multipath TCP should take as much capacity as TCP at a bottleneck link, no matter how many paths it is using. A multipath TCP flow with two subflows Regular TCP

Use Efficient Paths 32 Each flow has a choice of a 1-hop and a 2-hop path. How should split its traffic? 12Mb/s

Use Efficient Paths 33 If each flow split its traffic 1:1... 8Mb/s 12Mb/s

Use Efficient Paths 34 If each flow split its traffic 2:1... 9Mb/s 12Mb/s

Use Efficient Paths 35 Better: Each connection on a one-hop path Each connection should send all traffic on the least- congested paths 12Mb/s

Use Efficient Paths 36 Better: Each connection on a one-hop path Each connection should send all traffic on the least- congested paths But keep some traffic on the alternate paths as a probe 12Mb/s

Discussion 37