1. USEIMPROVEEVANGELIZE Solutions for High Availability and Disaster Recovery with MySQL ● Detlef Ulherr ● Sun Microsystems

2. 2 USEIMPROVEEVANGELIZE Why Is High Availability Important?

3. 3 USEIMPROVEEVANGELIZE Why is HA Important? ● Downtime is costly to the business ● Components or systems failures are real possibilities – Hardware, Software, Human Error, Disaster – Automated recovery is the goal ● With a single physical system, single points of failure have greater impacts – Network card dies, CPU misbehaves, Disk drive crashes... HA Clusters and Solaris automate the recovery process from inevitable failures, minimizing downtime and cost

4. 4 USEIMPROVEEVANGELIZE HA Architectures for MySQL

5. 5 USEIMPROVEEVANGELIZE HA Architectures for MySQL ● Shared nothing – MySQL replication – MySQL Cluster – Solaris Cluster / Open HA Cluster (planned) ● Shared disk – Solaris Cluster / Open HA Cluster – VCS, HACMP, MC Serviceguard and similar.

6. 6 USEIMPROVEEVANGELIZE HA Architectures for MySQL (cont.) ● Replicated Disk – Linux + DRBD – Solaris Cluster/Open HA cluster + AVS (planned) – Solaris Cluster/Open HA cluster + SRDF/Truecopy

7. 7 USEIMPROVEEVANGELIZE MySQL and Business Continuity

8. 8 USEIMPROVEEVANGELIZE MySQL and Business Continuity Building Blocks ● Local HA ● Infrastructure – Replication – Networks – Nameservice ● DR Framework

9. 9 USEIMPROVEEVANGELIZE MySQL and Business Continuity Building Blocks ● Most important: People and Processes – Training – Management – Periodic DR tests

10. 10 USEIMPROVEEVANGELIZE MySQL's Availability Features

11. 11 USEIMPROVEEVANGELIZE MySQL's Availability Features ● Replication – Asynchronous – Read mostly scale out ● MySQL Cluster – Synchronous – High write profile (Commit latency)

12. 12 USEIMPROVEEVANGELIZE When to use MySQL Cluster ● Your database design is good for horizontal partitioning. ● The cluster should provide HA for MySQL only. ● You can tolerate downtimes while adding data nodes. ● The memory is large enough for at least twice all the indexes.

13. 13 USEIMPROVEEVANGELIZE MySQL and Solaris Cluster / Open HA Cluster

14. 14 USEIMPROVEEVANGELIZE Open HA Cluster Benefits ● A shared disk HA is simpler than replication or MySQL Cluster ● Provides automated failover ● Can support larger databases than MySQL Cluster ● Can be extended with the rest of the application stack in the same Cluster. MySQL Cluster has no framework to integrate applications

15. 15 USEIMPROVEEVANGELIZE Open HA Cluster Benefits cont. ● The Open HA Cluster MySQL agent leverages all the rich infrastructure features of Open HA Cluster (strong membership, dependencies,...) ● Can support Campus and Metro Clusters up to 700 km ● Supports MySQL replication in all its flavors

16. 16 USEIMPROVEEVANGELIZE Open HA Cluster Stack Operating System Cluster Infrastructure Agents Applications Heartbeats Membership

17. 17 USEIMPROVEEVANGELIZE ● HA Failover Service ● Application failover ● Within nodes or container ● Containers failover ● Failover of volumes/filesystems ● Failover IP address Failover Service

18. 18 USEIMPROVEEVANGELIZE Scalable Service Software Load Balancing Global Network Service Provides Global IP address with failure protection Scalable Service

19. 19 USEIMPROVEEVANGELIZE Apache (Scalable) Global Network Service Provides Global IP address with failure protection MySQL (Failover) Example: Apache and MySQL

20. 20 USEIMPROVEEVANGELIZE Heartbeats Membership Quorum Disk Fencing ● Resource Group Manager ● Resource (application) dependencies ● Inter RG dependencies ● RG affinities ● HA Failover Service ● Application failover ● Within nodes or container ● Containers failover ● Failover IP address Global File Service Failover File service Scalable Service Software Load Balancing Global Network Service Provides Global IP address with failure protection Monitoring Solaris Cluster Architecture Quorum Server

21. 21 USEIMPROVEEVANGELIZE Open HA Cluster and Virtualisation ● Cluster control is in the global zone – Failover xVM – Failover Container – Zone Nodes ● Cluster control is in the virtualization entity – Cluster in LDOMS – Zone Cluster

22. 22 USEIMPROVEEVANGELIZE Cluster Agents

23. 23 USEIMPROVEEVANGELIZE Open HA Cluster Stack Operating System Cluster Infrastructure Agents Applications

24. 24 USEIMPROVEEVANGELIZE Cluster Agents (Data Services) ● Applications run on cluster unmodified (off-the-shelf if it complies to some rules) ● Cluster Agents are the “glue” layer between applications and cluster infrastructure ● Application cannot break into the high availability business on Solaris Cluster without an agent!

25. 25 USEIMPROVEEVANGELIZE MySQL (Shared Disk) MySQL Storage Log. Host Node1(:Zone1) Node2(:Zone2) Client

26. 26 USEIMPROVEEVANGELIZE MySQL (Replicated Disk) Global Zone MySQL Storage Log. Host Node1 Node2 Storage based replication Truecopy SRDF

27. 27 USEIMPROVEEVANGELIZE MySQL (Shared Disk) Master - Slave Node1 : Zone1 Node1 : Zone3 Node2 : Zone2 Node2 : Zone4 MySQL2 Storage Log. Host MySQL1 Storage Log. Host Mys Repl Client

28. 28 USEIMPROVEEVANGELIZE MySQL (Shared Disk) Failover Zone MySQL Storage Log. Host Node1 Node2 Zone1

29. 29 USEIMPROVEEVANGELIZE MySQL (Shared Disk) Zone Cluster Storage Log. Host Node1:Zone-cl Node2:Zone-cl MySQL

30. 30 USEIMPROVEEVANGELIZE MySQL (Shared Disk) Multiple Clusters MySQ L Stora ge Log. Host Node1 (Zone) Node2 (Zone) MySQ L Stora ge Log. Host Node3 (Zone) Node4 (Zone) MySQL Replication Cluster 1 Cluster 2

31. 31 USEIMPROVEEVANGELIZE MySQL (Shared Disk) replication outside a cluster MySQ L Stora ge Log. Host Node1 (Zone) Node2 (Zone) Node3 (Zone) MySQL Replication Cluster 1

32. 32 USEIMPROVEEVANGELIZE Geographic Clustering Multi-Cluster and Multi-Site capability Site 1 Admin. Client User Client Site 2 Optional heartbeat networks Optional storage network Storage

33. 33 USEIMPROVEEVANGELIZE MySQL and Open HA Cluster Future Configurations

34. 34 USEIMPROVEEVANGELIZE Solaris Cluster / OHAC Future Benefits ● Can give the option to use MySQL replication as a replacement for shared storage ● Can be extended to a full DR solution using MySQL replication as the replication protocol ● Can do loadbalancing between scalable slaves

35. 35 USEIMPROVEEVANGELIZE Solaris Cluster / OHAC Future Benefits cont. ● Can be extended to integrate the rest of the MySQL features like MySQL Cluster ● Can leverage the Shared Address resource type to do load balancing for SQL nodes

36. 36 USEIMPROVEEVANGELIZE Geographic Clustering Multi-Cluster and Multi-Site capability Site 1 Admin. Client User Client Site 2 Optional heartbeat networks Optional storage network Storage MySQL replication is available soon

37. 37 USEIMPROVEEVANGELIZE MySQL Scalable Slaves Shared Addr. MySQL replication MySQL- Slave Storage MySQL- Slave Storage MySQL- Master Stora ge Log. Host Loadbalancer Client

38. 38 USEIMPROVEEVANGELIZE MySQL Scalable Slaves Shared Addr. Node2(:zone2) Node1(:zone1) MySQL replication MySQL- Slave Storage MySQL- Slave Storage MySQL- Master Stora ge Log. Host

39. 39 USEIMPROVEEVANGELIZE MySQL Shared Nothing (future) Rolechanger Log. Host Node2 : Zone2 Slave Node1 : Zone 1 Master Quorum Server MySQL-PMySQL-S MySQL replication

40. 40 USEIMPROVEEVANGELIZE MySQL Cluster (Shared Nothing) Local/Global Zone (future) SQL-Node Data-Node SQL-Node logh hsp Mys query Node5,6,7 logh hsp Mys query Node5,6,7 logh hsp Mys query Node5,6,7 logh hsp Mys mgt Node 6,7... hsp Mys data Node1 hsp Mys data Node3 hsp Mys data Node4 hsp Mys data Node2 Shared Addr

41. 41 USEIMPROVEEVANGELIZE References Cluster References ● http://www.opensolaris.org/os/community/ha-clusters/ohac/ http://www.opensolaris.org/os/community/ha-clusters/ohac/ ● http://www.opensolaris.org/os/project/ha-mysql/ http://www.opensolaris.org/os/project/ha-mysql/ ● http://docs.sun.com/app/docs/prod/sun.cluster32#hic http://docs.sun.com/app/docs/prod/sun.cluster32#hic Blogs ● http://blogs.sun.com/sc/ http://blogs.sun.com/sc/ ● http://blogs.sun.com/krishs/entry/deployment_and_failover_study_of http://blogs.sun.com/krishs/entry/deployment_and_failover_study_of Blueprints ● http://wikis.sun.com/display/BluePrints/High+Availability+MySQL+Database+Replication+with+Solaris+Zone+Cluster http://wikis.sun.com/display/BluePrints/High+Availability+MySQL+Database+Replication+with+Solaris+Zone+Cluster

42. USEIMPROVEEVANGELIZE Thank you! Detlef Ulherr detlef.ulherr@sun.com “open” artwork and icons by chandan: http://blogs.sun.com/chandan http://blogs.sun.com/chandan