1. Copyright © 2002 Wensong Zhang. Page 1 Free Software Symposium 2002 Linux Virtual Server: Linux Server Clusters for Scalable Network Services Wensong Zhang China National Laboratory for Parallel & Distributed Processing Free Software Symposium 2002 October 22, 2002

2. Copyright © 2002 Wensong Zhang. Page 2 Free Software Symposium 2002 Agenda Introduction Introduction The Linux Virtual Server framework The Linux Virtual Server framework Building Scalable Network Services using LVS Building Scalable Network Services using LVS Future Work of LVS Future Work of LVS Characteristics of LVS Characteristics of LVS

3. Copyright © 2002 Wensong Zhang. Page 3 Free Software Symposium 2002 Introduction Internet Explosive Growth of the Internet Explosive Growth of the Internet –100% annual growth rate Sites receiving unprecedented workload Sites receiving unprecedented workload –Yahoo! 625 million views per day –AOL Web cache system receiving 5 billion requests per day

4. Copyright © 2002 Wensong Zhang. Page 4 Free Software Symposium 2002 Introduction The requirements of Network Services Incremental scalability Incremental scalability 24x7 availability 24x7 availability Manageability Manageability Cost-effectiveness Cost-effectiveness

5. Copyright © 2002 Wensong Zhang. Page 5 Free Software Symposium 2002 Introduction Possible Solutions Single server upgrade Single server upgrade –complicated, high cost, single point of failure,... Cluster of servers is becoming a viable architecture for SNS Cluster of servers is becoming a viable architecture for SNS –Redundancy --> HA –Divide-and-Conquer --> High-performance –High performance/cost ratio

6. Copyright © 2002 Wensong Zhang. Page 6 Free Software Symposium 2002 LVS Framework The Linux Virtual Server Project The goal of the Linux Virtual Server project The goal of the Linux Virtual Server project –to provide a basic framework for building highly scalable and highly available network services using a large cluster of commodity servers.

7. Copyright © 2002 Wensong Zhang. Page 7 Free Software Symposium 2002 LVS Framework Framework

8. Copyright © 2002 Wensong Zhang. Page 8 Free Software Symposium 2002 IPVS is an advanced Layer-4 switching LVS Framework IP Virtual Server Implemented in the Linux kernel Implemented in the Linux kernel Three IP load balancing techniques Three IP load balancing techniques –Virtual Server via NAT –Virtual Server via IP Tunneling –Virtual Server via Direct Routing Eight scheduling algorithms Eight scheduling algorithms

9. Copyright © 2002 Wensong Zhang. Page 9 Free Software Symposium 2002 LVS Framework VS/NAT

10. Copyright © 2002 Wensong Zhang. Page 10 Free Software Symposium 2002 LVS Framework VS/TUN

11. Copyright © 2002 Wensong Zhang. Page 11 Free Software Symposium 2002 LVS Framework VS/DR

12. Copyright © 2002 Wensong Zhang. Page 12 Free Software Symposium 2002 LVS Framework Comparison Note: those numbers are estimated based on the assumption that load balancer and backend servers have the same hardware configuration.

13. Copyright © 2002 Wensong Zhang. Page 13 Free Software Symposium 2002 LVS Framework Scheduling Algorithms Round-Robin Round-Robin Weighted Round-Robin Weighted Round-Robin Least-Connection Least-Connection Weighted Least-Connection Weighted Least-Connection

14. Copyright © 2002 Wensong Zhang. Page 14 Free Software Symposium 2002 LVS Framework Scheduling Algorithms (Continued) Locality-Based Least-Connection Locality-Based Least-Connection Locality-Based Least-Connection with Replication Locality-Based Least-Connection with Replication Source Hashing Source Hashing Destination Hashing Destination Hashing

15. Copyright © 2002 Wensong Zhang. Page 15 Free Software Symposium 2002 LVS Framework State Synchronization Conn Sync daemon (kernel thread) Conn Sync daemon (kernel thread) UDP Multicast UDP Multicast

16. Copyright © 2002 Wensong Zhang. Page 16 Free Software Symposium 2002 Layer-7 switching in user-space Layer-7 switching in user-space –high overhead of context switching and memory copying –limited scalability LVS Framework KTCPVS

17. Copyright © 2002 Wensong Zhang. Page 17 Free Software Symposium 2002 kernel thread kernel thread loadable scheduling module loadable scheduling module LVS Framework KTCPVS (Cont’d)

18. Copyright © 2002 Wensong Zhang. Page 18 Free Software Symposium 2002 LVS Framework Cluster Monitoring software Red Hat Cluster Server / Piranha Red Hat Cluster Server / Piranha –LVS + Piranha UltraMonkey UltraMonkey –LVS + lvs-gui + heartbeat + ldirectord Keepalived Keepalived Netparse Netparse etc. etc.

19. Copyright © 2002 Wensong Zhang. Page 19 Free Software Symposium 2002 Building SNS using LVS General Architecture 3 tiers 3 tiers –Load balancer –Server cluster –Shared storage

20. Copyright © 2002 Wensong Zhang. Page 20 Free Software Symposium 2002 Building SNS using LVS Building Blocks Load balancer Load balancer –Layer-4 switching, Layer-7 switching Server Server –HTTP, HTTPS, FTP, SMTP, POP3, IMAP4, most TCP and UDP services Shared storage Shared storage –database, network file system, distributed file system,...

21. Copyright © 2002 Wensong Zhang. Page 21 Free Software Symposium 2002 Building SNS using LVS High Availability Server failover Server failover –ping, service detection, etc Load balancer failover Load balancer failover –state synchronization –heartbeat

22. Copyright © 2002 Wensong Zhang. Page 22 Free Software Symposium 2002 Building SNS using LVS Web Cluster

23. Copyright © 2002 Wensong Zhang. Page 23 Free Software Symposium 2002 Building SNS using LVS Cache Cluster

24. Copyright © 2002 Wensong Zhang. Page 24 Free Software Symposium 2002 Building SNS using LVS Mail Cluster

25. Copyright © 2002 Wensong Zhang. Page 25 Free Software Symposium 2002 Building SNS using LVS Some Sites using LVS UK National JANET Web Cache (wwwcache.ja.net) UK National JANET Web Cache (wwwcache.ja.net) linux.com linux.com sourceforge.net sourceforge.net valinux.com valinux.com real.com real.com One of largest PC manufacturers One of largest PC manufacturers etc. etc.

26. Copyright © 2002 Wensong Zhang. Page 26 Free Software Symposium 2002 Future Works of LVS Future Works Adding more load balancing algorithms Adding more load balancing algorithms Developing more advanced/flexible cluster monitoring software Developing more advanced/flexible cluster monitoring software Making KTCPVS ready for production Making KTCPVS ready for production Exploring TCP handoff Exploring TCP handoff etc. etc.

27. Copyright © 2002 Wensong Zhang. Page 27 Free Software Symposium 2002 Characteristics of LVS Characteristics LVS extends Linux kernel to support three IP load balancing techniques LVS extends Linux kernel to support three IP load balancing techniques Eight scheduling algorithms Eight scheduling algorithms High scalability (up to 100 nodes) High scalability (up to 100 nodes) High availability High availability Supporting most TCP and UDP services, no modifications to either clients or servers Supporting most TCP and UDP services, no modifications to either clients or servers

28. Copyright © 2002 Wensong Zhang. Page 28 Free Software Symposium 2002 Characteristics of LVS Compared to Other Commercial Products More IP load balancing techniques More IP load balancing techniques Multiple scheduling algorithms Multiple scheduling algorithms State synchronization State synchronization A robust and stable code base, a large user and developer base. A robust and stable code base, a large user and developer base. Reliability proven in big real world applications Reliability proven in big real world applications Free to everyone Free to everyone

29. Copyright © 2002 Wensong Zhang. Page 29 Free Software Symposium 2002 Summary Introduction Introduction The Linux Virtual Server framework The Linux Virtual Server framework Building Scalable Network Services using LVS Building Scalable Network Services using LVS Future Work of LVS Future Work of LVS Characteristics of LVS Characteristics of LVS

30. Copyright © 2002 Wensong Zhang. Page 30 Free Software Symposium 2002 Call to Action Building scalable network services is complicated and expensive Building scalable network services is complicated and expensive LVS is here to help make your life easier LVS is here to help make your life easier LVS is proven stable, and is being deployed by more and more sites. LVS is proven stable, and is being deployed by more and more sites.

31. Copyright © 2002 Wensong Zhang. Page 31 Free Software Symposium 2002 Collateral The Linux Virtual Server Project The Linux Virtual Server Project(http://www.LinuxVirtualServer.org) wensong@linux-vs.org wensong@linux-vs.org