1. XORs in The Air: Practical Wireless Network Coding Daniel Courcy- daniel.courcy@hp.comdaniel.courcy@hp.com Sachin Katti, Hariharan Rahul, Wenjun Hu, Dina Katabi, Muriel Medard, Jon Crowcroft MIT CSAIL & University of Cambridge

2. Worcester Polytechnic Institute 2 Introduction - COPE New Architecture For Wireless Mesh Networks Routers Forward & Code (Mix) Packets Intelligent Mixing Improves Throughput Prior Work = Theoretical & Multicast This Study = Practical & Unicast

3. Worcester Polytechnic Institute 3 Review Mesh Networks –Way to Route Data –Allows For Continuous Connections & Reconfigurations Multicast –Transmission to Multiple Selected Recipients Unicast –Transmission to Single Selected Recipient

4. Worcester Polytechnic Institute 4 COPE Substantially Improves Throughput ‘Coding Shim’ Between IP and MAC Layers –Finds Coding Opportunities –Benefits by Sending Multiple Packets in Single Transmission

5. Worcester Polytechnic Institute 5 COPE: Simple Example Bob & Alice Current Approach = 4 Transmissions COPE = 3 Transmissions Thus Allowing For Increased ThroughputIncreased Throughput Coding+MAC

6. Worcester Polytechnic Institute 6 COPE: Even Bigger Savings Previous Example: Obvious Throughput Gains COPE Exploits Shared Nature of Wireless Medium Some Nodes Overhear Transmission –Store These Packets for Short Time –Sends Data Out Telling What It Has Heard This Data is Used For Opportunistic Coding

7. Worcester Polytechnic Institute 7 Opportunistic Coding Each Node Uses Knowledge of What It’s Neighbors Have (Which Packets) This Data Allows For Source to Send XOR’ed Packets Intelligently –In Other Words: It Knows who Will Be Able to Decode The Encoded Packet Allows For More Than Two Flows Allows For Multiple Packets to be Coded

8. Worcester Polytechnic Institute 8 COPE = 2 Key Principles (1) COPE Embraces Broadcast Nature of Wireless Channel –Typically Network Designers Use Point-to- Point –Adaptations are Made To Use Forwarding and Routing Techniques Native to Wired Networking –COPE Exploits Radio Broadcast (Does Not Attempt to Hide It)

9. Worcester Polytechnic Institute 9 COPE = 2 Key Principles (2) COPE Employs Network Coding –Packets Are Mixed Before Transmission –Prior Work Is Mainly Theoretical –COPE Addresses: Unicast Traffic Dynamic & Bursty Flows Other Practical Issues Regarding Implementation

10. Worcester Polytechnic Institute 10 Why is COPE Different? 1st System Architecture For Wireless Network Coding Implementation: 1st Deployment of Wireless Network Coding Performance Study with Results of Wireless Network Coding

11. Worcester Polytechnic Institute 11 Summarized Findings Network Coding Can Improve Wireless Throughput When Congested with Mainly UDP Flows Throughput Gains 3 - 4x For Mesh Networks Connected to Internet via AP Gains Depend on Total Download / Upload Traffic @ AP w/o Hidden Terminals TCP Throughput Increases about 38%

12. Worcester Polytechnic Institute 12 Background Famous Butterfly Example: All Links Can Send One Message Per Unit of Time Sources Want to Hit Both Receivers Coding (Again) Increases Overall Throughput

13. Worcester Polytechnic Institute 13 Background (cont.) Ahlswede et al. Pioneered Network Coding –Routers That Mix Information Allow Communication to Achieve Multicast Capacity Li et al. Found For Multicast Linear Codes Sufficient to Achieve Max Capacity Bounds

14. Worcester Polytechnic Institute 14 Background (cont.) All Previous Work Theoretical & Multicast Some Unicast Scenarios Show Better Throughput for Those Scenarios This Paper Seeks to ‘Bridge the Gap’ Between Network Coding, Practical Design Seeks to Provide Operational Protocol

15. Worcester Polytechnic Institute 15 COPE Overview Before Getting Into The Details Know These Terms:

16. Worcester Polytechnic Institute 16 COPE Overview 3 Main Techniques –Opportunistic Listening –Opportunistic Coding –Learning Neighbor State

17. Worcester Polytechnic Institute 17 COPE Overview: Opportunistic Listening Wireless is a Broadcast Medium Many Chances for Nodes to Overhear COPE Sets All Nodes as Promiscuous They Store Overheard Packets for Time (T) –Default T =.5 Seconds Also: Reception Reports Are Sent Out –These Are Tacked Onto Normal Output –Includes Seq. Number of Stored Packets

18. Worcester Polytechnic Institute 18 COPE Overview: Opportunistic Coding Which Packets do we Combine to Achieve Maximum Throughput? Simple Answer: Send as Many (Native Packets) as Possible While Ensuring Nexthop Has Enough Info to Decode

19. Worcester Polytechnic Institute 19 COPE Overview: Opportunistic Coding Always Seeking Largest N That Satisfies Above Rule

20. Worcester Polytechnic Institute 20 COPE Overview: Learning Neighbor State Each Node Announces Its Stored Packets In Reception Reports Sometimes Reports Don’t Get Through –Congestion or In Times of Light Traffic To Solve This Problem: Intelligent Guess –Estimation of Probability That Neighbor Has Packet Based On Delivery Probability

21. Worcester Polytechnic Institute 21 COPE’s Gains How Beneficial is COPE? –Throughput Improvement Depends On: Coding Opportunities Traffic Patterns

22. Worcester Polytechnic Institute 22 COPE’s Gains: Coding Gain Coding Gain: –# Transmissions w/o Coding to the Minimum # Transmissions w/ Coding Remember Alice & Bob?Alice & Bob –Coding Gain = 4/3 = 1.33

23. Worcester Polytechnic Institute 23 COPE’s Gains: Coding Gain Maximum Achievable Coding Gain? –For Arbitrary Topologies - Open Question Authors Prove: With Listening Certain Topologies Benefit

24. Worcester Polytechnic Institute 24 COPE’s Gain: Coding Gain Interesting to Note: –Previous Slide Talks About Theoretical Gain –In Practice Gains Are Lower Due To: Coding Opportunities Packet Header Overhead Medium Losses COPE Coding Gains Are Not Lost When Medium Is Fully Utilized (as Opposed to Opportunistic Routing)

25. Worcester Polytechnic Institute 25 COPE’s Gain: Coding+MAC Gain Interaction Between Coding & MAC –Beneficial Results Example Bob & AliceBob & Alice –MAC Divides Bandwidth Between 3 –w/o Coding Router Sends 2 x More –Makes Router a Bottleneck –COPE Allows Routers Queue to Drain FastDrain Fast –Coding + MAC Gain of Alice & Bob = 2

26. Worcester Polytechnic Institute 26 COPE’s Gain: Coding+MAC Gain Authors Prove:

27. Worcester Polytechnic Institute 27 Making It Work - Packet Coding Algorithm Packets Are Never Delayed –If There Is Nothing To Code With, Send Anyway Preference to XOR with Similar Lengths –Small Packet XOR with Large = Less Bandwidth –If One Must XOR Different Lengths - Pad Never Code Packets to Same Nexthop

28. Worcester Polytechnic Institute 28 Making It Work - Packet Coding Algorithm Searching For Appropriate Packets to Code is Efficient –FIFO Output Queue: De-queue, Small or Large?, Look at Appropriate Queues (Only Heads to Avoid Reordering) –Worst Case - Looks @ 2M Packets (M=# Neighbors) Packet Reordeing Bad (TCP Thinks Congestion) –Doesn’t Happen Much But If so They Are Put in Order Before Transport Layer

29. Worcester Polytechnic Institute 29 Making It Work - Packet Coding Algorithm Finally Relay Nodes All Estimate Probability That Neighbor Has Packet Prior to Sending P D Must Stay Higher Than Threshold G (G = 0.8 Default) If Equation Is Above G Each Nexthop Has Probability G of Being Able to Decode Next Packet

30. Worcester Polytechnic Institute 30 Making It Work - Packet Decoding Fairly Simple –Each Node Maintains a Packet Pool –Searches Hash Table Keyed on Packet ID –XORs Native Packets with Coded Packets –Gets Packet Meant For It (Node)

31. Worcester Polytechnic Institute 31 Making It Work - Pseudo-Broadcast 802.11 Has Two MAC Modes: –Unicast –Broadcast Unicast –Packets Are Ack-ed –Exponential Backoff Multicast –Un-Reliable –No Backoff

32. Worcester Polytechnic Institute 32 Making It Work - Pseudo-Broadcast Pseudo-Broadcast –Piggy Backs Unicast –Link-Layer Destination Set to One Intended Node –XOR Header Added –Other Nodes Can Overhear Transmission –If Receiving Node is Nexthop - Continue –Else Store Packet in Buffer –More Reliable Than Pure Broadcast –Packets Have Several Tries To Get To Destination –Snooping Nodes Get More Chances to Update Their Buffers

33. Worcester Polytechnic Institute 33 Making It Work - Hop-by-Hop ACKs and Retransmissions (Again) Encoded Packets Require All Nexthops to Acknowledge Receipt of Native Packet –Packets Headed Many Places & Only Link Layer Designated Hop Returns Synchronous ACK –COPE May Guess Node Has Enough Info to Decode When it Really Does Not

34. Worcester Polytechnic Institute 34 Making It Work - Hop-by-Hop ACKs and Retransmissions When a Node Sends an Encoded Packet It Schedules a Retransmission Event For Each Encoded Native Packet If Any Packet is Not Ack-ed within Some Threshold (Time) That Native Packet is Encoded and Re-Sent Later Nexthops Receive Packets and ACK Immediately Upon Decoding via Header (or Control Packets which are also Used for Reception Reports)

35. Worcester Polytechnic Institute 35 Making It Work - TCP Packet Reordering Asynchronous ACKs Can Cause Packet Reordering –TCP May See This As Congestion COPE Has Ordering Agent –For Each TCP Flow Ending @ Host Maintains Packet Buffer Records Last TCP Sequence Number Will Not Pass On Packets to Transport Layer Until No Hole Exists or Timer Times Out

36. Worcester Polytechnic Institute 36 Implementation Details: Packet Format COPE Inserts Variable Length Coding Header Only Shaded Fields Below Required

37. Worcester Polytechnic Institute 37 Implementation Details: Packet Format First Block: Metadata For Decoding –ENCODED_NUM: # Encoded –For Each Packet PKT_ID (Dest. IP & Seq. #) –MAC of Nexthop (For Each Native Packet) Reception Reports –REPORT_NUM: # of Reports –SRC_IP: Source of Reported Packets –Last_PKT: Last Packet Heard From Source –Bit Map of Recently Heard Packets

38. Worcester Polytechnic Institute 38 Implementation Details: Packet Format Asynchronous ACKs –Cumulative ACKs on Per Neighbor Basis –Local Sequence Numbers Established –ACK Headers Start With # of ACKs –Each ACK Starts with MAC of Neighbor –Next Each ACK Has Pointer to End of Cumulative ACKs –Finally, Bit Map Shows Missing Packets

39. Worcester Polytechnic Institute 39 Implementation Details: Control Flow

40. Worcester Polytechnic Institute 40 Experimental Results 20 Node Wireless Testbed Following Slides Will Show: –When Many Random UDP Flows: Throughput 3 - 4x Increase –Traffic Does Not Use Congestion Control: Throughput Improves - Exceeding Coding Gain –Mesh Network -> Internet via Gateway Throughput Improvement Between 5 - 70% –w/o Hidden Terminals TCP’s Gain Agrees With Expected Coding Gain

41. Worcester Polytechnic Institute 41 Experimental Results: Testbed 20 Node Wireless Network –Two Floors Connected by Open Lounge –Offices, Passages, Etc. –Paths Btw 1 & 6 Hops –Loss Rate Btw 0 - 30% –802.11a @ 6 Mb/s

42. Worcester Polytechnic Institute 42 Experimental Results: Testbed Nodes Ran Linux / Used Click Toolkit Runs as User Space Daemon Applications Interact With Daemon as Normally Would With Any Network Device Testbed Used Srcr Routing Protocol –Djikstra’s Shortest Path Algorithm Each Node Had 802.11 Card w/ Omni-Directional Antenna 802.11 Ad Hoc Mode w/ RTS / CTS Disabled udpgen & ttcp Used to Generate Traffic –Long-live Flows & Attempt to Match Internet Traffic

43. Worcester Polytechnic Institute 43 Metrics Network Throughput –End-to-End Throughput Throughput Gain –Ratio of Measured Network Throughputs With and Without COPE What Else Might Have Been Interesting?

44. Worcester Polytechnic Institute 44 COPE in Gadget Topologies Toy Topologies –Very Small Loss Rate & No Hidden Terminals (40 Different Runs) Long-Lived TCP Flows –Close To Expected (Minus Overhead)

45. Worcester Polytechnic Institute 45 COPE in Gadget Topologies Above Results Show That w/ Congestion Control Results Lean Towards Coding Gain Rather than Coding+MAC –When Many Long-Lived Flows (TCP) Bottleneck Senders Backoff (to Avoid Drop) This Leaves only Coding Gains

46. Worcester Polytechnic Institute 46 COPE in Gadget Topologies Repeat of Above w/ UDP Coding+MAC Gains (Better Than TCP)Coding+MAC Coding Allows Downstream Routers to Avoid Dropping Packets Already Having Consumed Bandwidth

47. Worcester Polytechnic Institute 47 COPE in an Ad Hoc Network Here it is: 20 Node Wireless Testbed TCP –TCP Flows Arrive w/ Poisson Process –Pick Sender & Receiver Randomly –Traffic Models Internet –No Significant Improvement (2-3%) –Hidden Terminals are Culprit Many Retransmissions Queues @ Bottlenecks Never Build Up –Therefore No Coding Gains (or Opportunities) Would TCP Do Better w/o Collisions?

48. Worcester Polytechnic Institute 48 COPE in an Ad Hoc Network Compressed Topology –Within Carrier Sense Range –Artificially Impose Original Loss Rates –Hidden Terminals = No More At Peak 38% Gain Over No Coding

49. Worcester Polytechnic Institute 49 COPE in an Ad Hoc Network UDP (Back to Large Scale Testbed) (Again) Random Sender / Receiver File Size Follows Internet Studies 500 Experiments…

50. Worcester Polytechnic Institute 50 COPE in an Ad Hoc Network Scare Coding Opp. At Low Demands Demand Up / Congestion Up / Gain Up

51. Worcester Polytechnic Institute 51 COPE in an Ad Hoc Network Low Demand - Reports Arrive to Late Demand Goes Up - Bottlenecks Form - Longer Wait Times - Nodes Get More Reports Demands Get Higher - High Loss Rates of Reception Reports - Guessing Relied Upon

52. Worcester Polytechnic Institute 52 COPE in an Ad Hoc Network @ Peak Gain Point (5.6 Mb/s) On Average 3 Packets Coded Together Packets Drained From Bottlenecks Faster Throughput Gains 3 - 4 x

53. Worcester Polytechnic Institute 53 COPE in a Mesh Access Network Growing Interest In Accessing Internet Via Multi-hop Network With One (or More) Gateways Nodes Divided Into 4 Sets (1 is Gateway) UDP Flows (Of Course) Fluctuate Upload / Download Traffic Gain Goes Up as Upload Traffic Up

54. Worcester Polytechnic Institute 54 COPE in Mesh Access Network

55. Worcester Polytechnic Institute 55 Fairness Channel From Source to Bottleneck Matters Capture Effect (If Alice’s Channel is Bad Then Bob Might Push More Traffic) If Alice Moves Slowly Away? –Coding Opp. Down, Throughput Down, Fairness Down w/o Coding Throughput Goes Up Coding Aligns Fairness & Efficiency

56. Worcester Polytechnic Institute 56 Fairness

57. Worcester Polytechnic Institute 57 Discussion Target: Stationary Wireless Mesh Networks Memory Needed to Store Packets –Need More Than Delay Bandwidth Product Need Omni-Directional Antenna Current Design Does Not Consider Power

58. Worcester Polytechnic Institute 58 Conclusions Coding is an Old Theme COPE Has Potential to Largely Increase Network Throughput COPE Assists Many Random UDP Flows Best No Congestion Control Is a Good Thing No Hidden Terminals Is Good As Well (Even for TCP) Mesh Networks Connected to Internet via AP - COPE Shows Gains From 5 - 70 % Many Extensions - Sensor Networks? Cellular?

59. Worcester Polytechnic Institute 59 Questions?