COS 561: Advanced Computer Networks Multipath TCP Jennifer Rexford Fall 2017 (TTh 1:30-2:50 in CS 105) COS 561: Advanced Computer Networks http://www.cs.princeton.edu/courses/archive/fall17/cos561/

Multipath Mobile user High-end servers Data centers WiFi and cellular at the same time High-end servers Multiple Ethernet cards Data centers Rich topologies with many paths Benefits of multipath Higher throughput Failover from one path to another Seamless mobility

Multipath TCP Protocol

Working With Unmodified Apps Present the same socket API and expectations Identified by the “five tuple” (IP address, port #, protocol) From http://queue.acm.org/detail.cfm?id=2591369

Working With Unmodified Hosts Establish the TCP connection in the normal way Create a socket to a single remote IP address/port And then add more subflows, if possible A B SYN SYN ACK Each host tells its Initial Sequence Number (ISN) to the other host. ACK Data Data

Negotiating MPTCP Capability How do hosts know they both speak MPTCP? During the 3-way SYN/SYN-ACK/ACK handshake If SYN-ACK doesn’t contain MP_CAPABLE Don’t try to add any subflows!

Adding Subflows, Idealized How to associate a new subflow with the connection? Use a token generated from original subflow set-up How to start using the new subflow? Simply start sending packets with new IP/port pairs … and associate them with the existing connection How could two end-points learn about extra IP addresses for establishing new subflows? Implicitly: one end-point establishes a new subflow, to already-known address(es) at the other end-point

Sequence Numbers Challenges across subflows Out-of-order packets due to RTT differences Access networks that rewrite sequence numbers Middleboxes upset by discontinuous TCP byte stream Need to retransmit lost packets on a different subflow Two levels of sequence numbers Sequence numbers per subflow Sequence numbers for the entire connection Enables Efficient detection of loss on each subflow Retransmission of lost packet on a different subflow

Receive Buffer Space Each TCP connection has a receive buffer Buffer space to store incoming data … until it is read by the application TCP flow control Receiver advertises the available buffer space … using the “receive window” Should each subflow have its own receive window? Starvation of some subflows in a connection? Fairness relative to other TCP connections? Fragmentation of the available buffer space? Instead, use a common receive window

Fairness and Efficiency in Multipath Congestion Control Slides from Damon Wischik

Goal #1: Fairness at Shared Bottlenecks To be fair, Multipath TCP should take as much capacity as TCP at a bottleneck link, no matter how many paths it is using. A multipath TCP flow with two subflows Regular TCP This is the very first thing that comes to mind with multipath TCP, and it’s something that many other people have solved in different ways. This is just a warm-up... Design Goal 3 is a much “richer” generalization of this goal, which accommodates different topologies, different RTTs. So there’s no point giving an evaluation here.

Goal #2: Use Efficient Paths 12Mb/s 12Mb/s 12Mb/s Each flow has a choice of a 1-hop and a 2-hop path. How should split its traffic? I’m thinking of the paths as given. MPTCP has the choice of how to split its traffic over those given paths.

Use Efficient Paths If each flow split its traffic 1:1 ... 12Mb/s

Use Efficient Paths If each flow split its traffic 2:1 ... 12Mb/s

Use Efficient Paths 12Mb/s 12Mb/s Better: Each connection on a one-hop path Each connection should send all traffic on the least-congested paths 12Mb/s 12Mb/s 12Mb/s 12Mb/s

Use Efficient Paths Better: Each connection on a one-hop path Each connection should send all traffic on the least-congested paths But keep some traffic on the alternate paths as a probe 12Mb/s 12Mb/s 12Mb/s 12Mb/s 12Mb/s 12Mb/s

Goal #3: Be Fair Compared to TCP Least-congested paths may not be best! Due to differences in round-trip time Two paths WiFi: high loss, low RTT Cellular: low loss, high RTT Using the least-congested path Choose the cellular path, due to low loss But, the RTT is high So throughput is low!

Be Fair Compared to TCP To be fair, Multipath TCP should give a connection at least as much throughput as it would get with a single-path TCP on the best of its paths. Ensure incentive for deploying MPTCP A Multipath TCP should take no more capacity on any path (or collection of paths) than if it was a single-path TCP flow using the best of those paths. Do no harm!

Achieving These Goals Regular TCP MPTCP Maintain a congestion window w On an ACK, increase by 1/w (increase 1 per window) On a loss, decrease by w/2 MPTCP Maintain a congestion window per path wr On an ACK on path r, increase wr On a loss on path r, decrease by wr/2 How much to increase wr on an ACK?? If r is the only path at that bottleneck, increase by 1/wr

If Multiple Paths Share Bottleneck? Don’t take any more bandwidth on a link than the best of the TCP paths would But, where might the bottlenecks be? Multiple paths might share the same bottleneck So, consider all possible subsets of the paths Set R of paths Subset S of R that includes path r E.g., consider path 3 Suppose paths 1, 3, and 4 share a bottleneck … but, path 2 does not Then, we care about S = {1,3,4}

Achieving These Goals What is the best of these subflows achieving? Path s is achieving throughput of ws/RTTs So best path is getting maxs(ws/RTTs) What total bandwidth are these subflows getting? Across all subflows sharing that bottleneck Sum over s in S of ws/RTTs Consider the ratio of the two Increase by less if many subflows are sharing And pick the results for the set S with min ratio To account for the most paths sharing a bottleneck

Incremental Deployment Challenges of Middleboxes

Middleboxes In-network services, e.g., Interaction with TCP Firewall Network address translator Transparent proxy Intrusion detection system Interaction with TCP Change IP addresses and port numbers Change TCP initial sequence number Remove TCP options Dividing large block of data into smaller packets Expect to see all packets of the connection Etc.

Negotiating MPTCP Capability What if middleboxes strip the TCP option? On the SYN? On the SYN-ACK? Include capability on the ACK of the SYN-ACK? What if the ACK is lost? Carry on all subsequent packets What if the middlebox drops SYN packets with unfamiliar options? Sender can retransmit lost SYN without the option … and fall back to regular TCP behavior

Challenges: NAT Network Address Translators (NAT) NAT1 Problem: NAT changes the IP address and port number How to identify a connection? Using a token established during connection set-up How to establish new subflows? Allow one end-point to tell another about its addresses WiFi NAT1 NAT2 LTE

Challenges: Security Security How to bootstrap security? Malicious parties creating subflows To highjack (part of) the connection How to bootstrap security? Include a random key during connection set-up … and use it to verify authenticity of new subflows How to identify the connection on new subflows? A token generated from the key How to authenticate the addition of subflows? Exchanging nonces and computing message authentication codes using the keys

Use of Multipath TCP in iOS 7 Multipath TCP in iOS 7 (fall 2013) Primary TCP connection over WiFi Backup TCP connection over cellular data Failover If WiFi becomes unavailable… … iOS 7 will use the cellular data connection For destinations controlled by Apple E.g., Siri See https://support.apple.com/en-us/HT201373

Discussion

Backup Slides: Review of TCP Protocol

Establishing a TCP Connection SYN SYN ACK Each host tells its Initial Sequence Number (ISN) to the other host. ACK Data Data Three-way handshake to establish connection Host A sends a SYN (open) to the host B Host B returns a SYN acknowledgment (SYN ACK) Host A sends an ACK to acknowledge the SYN ACK

Initial Sequence Number (ISN) Sequence number for the very first byte E.g., Why not a de facto ISN of 0? Practical issue: reuse of port numbers Port numbers must (eventually) get used again … and an old packet may still be in flight … and associated with the new connection Security issue: adversary injecting packets Adversary may try to inject packets in a connection … by guessing the Initial Sequence Number … to send counterfeit packets to the receiving host … e.g., counterfeit packets that reset the connection Some firewalls change the ISN to further randomize

Step 1: A’s Initial SYN Packet A’s port B’s port A’s Initial Sequence Number Flags: SYN FIN RST PSH URG ACK Acknowledgment 20 Flags Advertised window Checksum Urgent pointer Options (variable) A tells B it wants to open a connection…

Step 2: B’s SYN-ACK Packet B’s port A’s port B’s Initial Sequence Number Flags: SYN FIN RST PSH URG ACK A’s ISN plus 1 20 Flags Advertised window Checksum Urgent pointer Options (variable) B tells A it accepts, and is ready to hear the next byte… … upon receiving this packet, A can start sending data

Step 3: A’s ACK of the SYN-ACK A’s port B’s port Sequence number Flags: SYN FIN RST PSH URG ACK B’s ISN plus 1 20 Flags Advertised window Checksum Urgent pointer Options (variable) A tells B it is okay to start sending … upon receiving this packet, B can start sending data

Sequence number = 1st byte Host A ISN (initial sequence number) Byte 81 Sequence number = 1st byte TCP Data TCP Data Host B

TCP Header Data Source port Destination port Sequence number Flags: SYN FIN RST PSH URG ACK Acknowledgment HdrLen Flags Advertised window Checksum Urgent pointer Options (variable) Data

Receive Buffering: Flow Control Receive window size Amount that can be sent without acknowledgment Receiver must be able to store this amount of data Receiver tells the sender the window Tells the sender the amount of free space left Window Size Data ACK’d Outstanding Un-ack’d data Data OK to send Data not OK to send yet

TCP Header: Receive Window Source port Destination port Sequence number Flags: SYN FIN RST PSH URG ACK Acknowledgment HdrLen Flags Advertised window Checksum Urgent pointer Options (variable) Data

Tearing Down the Connection B ACK SYN SYN ACK Data ACK FIN ACK FIN ACK A time Closing (each end of) the connection Finish (FIN) to close and receive remaining bytes And other host sends a FIN ACK to acknowledge Reset (RST) to close and not receive remaining bytes

Extending TCP: TCP Options TCP header Ten mandatory fields Optional extension field (usually during handshake) Examples Maximum segment size (MSS) Window scaling Support for Selected ACKs Unknown options Ignored by receiving host Routers and TCP options Should ignore them, passing them through unchanged But, some middleboxes: (i) strip TCP options from some packets or (ii) drop packets with TCP options