1. Advanced Computing and Information Systems laboratory IP over P2P: Enabling Self- configuring Virtual IP Networks for Grid Computing Arijit Ganguly, Abhishek Agrawal, P. Oscar Boykin, Renato Figueiredo University of Florida IPDPS 2006

2. Advanced Computing and Information Systems laboratory 2 What is the talk about? Convergence of Grid and P2P technologies 1 Context of network virtualization 1 On death, taxes, and the convergence of peer-to-peer and Grid Computing. Foster et al. IPTPS 2003

3. Advanced Computing and Information Systems laboratory 3 Outline Virtual networking and Grid Computing Related work Our approach – IP over P2P Experimental evaluation Conclusion and Future work

4. Advanced Computing and Information Systems laboratory 4 Internet access Internet Background - Virtual Private Networks Files, emails, compute cycles printers VPN gateway Rhodes, Greece User inside ACIS private network Install Cisco VPN client Connect to VPN gateway router NAT/Firewall Tunnel

5. Advanced Computing and Information Systems laboratory 5 Grid scenario Internet router NAT Firewall NAT/Firewall Purdue Florida Northwestern LSU Grid user SSH only Internet router Purdue Florida Northwestern LSU Grid user SSH only Issues: Network Address Translation Idiosyncrasies of heterogeneous access Traffic generated by untrusted code from Grid users – DoS attacks, viruses NAT Firewall NAT/Firewall

6. Advanced Computing and Information Systems laboratory 6 Virtual network of Grid resources Internet router NAT Firewall NAT/Firewall Purdue Florida Northwestern LSU Grid user SSH only Virtual Network

7. Advanced Computing and Information Systems laboratory 7 Virtual networking for Grids VNET (Northwestern University) Bridge a remote Virtual Machine (VM) to a client network VIOLIN (Purdue University) Virtualized network components Isolated from real physical network ViNe (University of Florida) Virtual IP network of Grid resources To be presented on Friday (Session 32) Common technology: Overlay tunneling What differentiates us: P2P routing

8. Advanced Computing and Information Systems laboratory 8 Motivations for P2P Scalability and Self-configurability Manual effort required to add a new node constant Independent of size of the network Resiliency Robust P2P routing Accessibility Ability to traverse NAT Hole punching 1 1 RFC 3489 - STUN - Simple traversal of User Datagram Protocol through Network Address Translators

9. Advanced Computing and Information Systems laboratory 9 Our approach – IP- over-P2P (IPOP) Isolation Virtual address space decoupled from Internet address space Self-configurability Automatic setup of routes and topologies Decentralized No global state No central points of failure VM mobility Decentralized NAT traversal No changes to NAT configuration No globally deployed STUN servers #affiliation condor_wow #transport udp #port 15000 #number of remote TAs 2 #list of TAs brunet.udp://planetlab-01.bu.edu:15000 brunet.udp://planetlab1.cs.purdue.edu:15000 #virtual interface tap0 #virtual IP address of tap0 172.16.1.5 #MAC address of tap0 CB:DF:E7:20:60:35

10. Advanced Computing and Information Systems laboratory 10 IPOP - Architecture Overview IP tunneling over P2P overlay networks UDP, TCP Virtual IP packet capture and injection through tap interface Builds upon Brunet P2P library

11. Advanced Computing and Information Systems laboratory 11 IPOP – Packet capture and routing application Node Y IPOP eth0 IPOP Node X eth0 Socket s = new Socket(“172.16.0.10:3000”); s.connect(); ServerSocket serv = new ServerSocket(“172.16.0.10,3000); serv.accept() X Y Extract IP from Ethernet Encapsulate IP inside P2P Extract IP from P2P Encapsulate in Ethernet tap0 (172.16.0.10) tap0 (172.16.0.9)

12. Advanced Computing and Information Systems laboratory 12 X Y Brunet P2P architecture Ring-structured overlay network topology Nodes ordered on 160-bit addresses Overlay link: Near: neighbor connections Far: connections across ring Near connection Far connection Multi-hop path between X and Y U V

13. Advanced Computing and Information Systems laboratory 13 Brunet P2P architecture (2) Routing Constant number of connections O(log 2 (n)) overlay hops O(log(n)) connections O(log(n)) overlay hops n connections 1-hop C# library, supports: Connection setup and maintenance NAT traversal

14. Advanced Computing and Information Systems laboratory 14 Network Address Translation (NAT) Host A NAT Public host 128.227.56.83 Sends packet: Src = 10.5.144.69:5000 Dst = 216.239.37.99:80 Sends packet: Src = 216.239.37.99:80 Dst = 128.227.56.83:5126 216.239.37.99 10.5.144.69 Translated: Src = 128.227.56.83:5126 Dst = 216.239.37.99:80 10.5.144.69:5000  128.227.56.83:5126 Outgoing packet to 128.227.56.83:5126 NAT Tables Sends packet: Src = 216.239.37.99:80 Dst = 10.5.144.69:5000 Applications on NATed hosts can learn their NAT assigned IP:port

15. Advanced Computing and Information Systems laboratory 15 NAT traversal – Behind NATs Src = R:A Dst = N:Y Src = N:Y Dst = M:X R:A  M:X N:Y  S:B Outgoing packet to M:X (hole punched) Src = S:B Dst = M:X Src = M:X Dst = N:Y Outgoing packet to N:Y (hole punched) Src = M:X Dst = S:B Exchange each other’s NAT assigned IP:port M:X N:Y NAT N 128.139.156.90 NAT M 128.227.56.83 S:B R:A Allow Dropped

16. Advanced Computing and Information Systems laboratory 16 Experiments Latency overhead and throughput of single overlay link LAN and WAN MPI application over IPOP Light Scattering Spectroscopy (LSS) Multi-hop routing experiments More than 100 node network on PlanetLab

17. Advanced Computing and Information Systems laboratory 17 Latency (single IPOP link) Two IPOP nodes separated by single overlay hop ACIS – ACIS for LAN ACIS – VIMS for WAN Ping times between two nodes 6ms-11ms overhead per packet for ICMP ping Relative overhead is smaller in Wide-Area ACIS: Florida VIMS: Virginia

18. Advanced Computing and Information Systems laboratory 18 Latency overhead - analysis Reasons for high LAN overhead: Double traversal of kernel stack C# runtime User-level overlay – context switches Other user-level overlays (VNET, Violin) report few-ms latency overheads

19. Advanced Computing and Information Systems laboratory 19 Throughput (single IPOP link) Two IPOP nodes separated by single overlay hop ACIS – ACIS for LAN ACIS – VIMS for WAN “ttcp” file transfer sizes (13.09 MB, 92.97 MB) 1.9MB/s LAN bandwidth (20% of physical 9.4 MB/s) 1.2MB/s WAN bandwidth (80% of physical 1.5 MB/s) ACIS: Florida VIMS: Virginia

20. Advanced Computing and Information Systems laboratory 20 Real Application – Parallel LSS MPI + NFS + SSH1 1 1 Support for Data-Intensive, Variable-Granularity Grid Applications via Distributed File System Virtualization - A Case Study of Light Scattering Spectroscopy. Figueiredo et al. CLADE 2004 Internet router VIMS LSU Firewall NAT NAT/Firewall Florida

21. Advanced Computing and Information Systems laboratory 21 Real Application – Parallel LSS With IPOP, could run “parallel LSS” unmodified No changes to NAT/Firewall rules Achieve parallel speedup

22. Advanced Computing and Information Systems laboratory 22 PlanetLab experiments Demonstrate ease of adding a new node and achieving IP routability in WAN environment 118 node TCP-based overlay on PlanetLab Connect two IPOP nodes in ACIS lab to PlanetLab network Measure ping times between nodes Average: 1617 ms; Std Dev: 2098 ms

23. Advanced Computing and Information Systems laboratory 23 Planetlab experiments (analysis) Issues: High-load (>10) on nodes in routing path Geographically unaware p2p routing Packets between machines in Florida routed through machines in California Improvements: Direct overlay link setup between communicating nodes No concerns of load and inefficient p2p routing

24. Advanced Computing and Information Systems laboratory 24 Conclusion Our contribution: Novel virtual IP network based on P2P overlay Scalable and Self-configurable Resilient NAT traversal Experiments showed feasibility of using P2P approach for virtual networking

25. Advanced Computing and Information Systems laboratory 25 Future work Overhead of TCP or UDP Raw sockets or Ethernet-based overlay edges Kernel level extensions Tap module with encapsulation and bridging Reduce context switches

26. Advanced Computing and Information Systems laboratory 26 Related Work Virtual Networking VIOLIN VNET ViNe (Session 32) Internet Indirection Infrastructure (i3) Support for mobility, multicast, anycast Decouples packet sending from receiving Based on Chord p2p protocol IPv6 tunneling IPv6 over UDP (Teredo protocol) IPv6 over P2P (P6P)

27. Advanced Computing and Information Systems laboratory 27 Acknowledgments In-VIGO team at UFL National Science Foundation Middleware Initiative (http://www.nsf-middleware.org)http://www.nsf-middleware.org Research Resources Program nCn center Resources Peter Dinda (Northwestern University) SURA/SCOOP IBM Shared University Research Questions?

28. Advanced Computing and Information Systems laboratory 28 Thank You