Presentation by Krishna MySQL Cluster Presentation by Krishna

What is MySQL Cluster ? Just like Hadoop, it is a technology/framework for distributed databases. ACID compliant. Horizontal scaling of (MySQL server + storage engine). In a nut shell it provides clustering for the MySQL DBMS. MySQL Cluster

Design principles High write scalability- auto sharding. 99.99 % availability- Shared nothing design. Real-Time responsiveness- in memory data base system. Low TCO and Open platform. MySQL Cluster

Implementation Implemented through NDB (Network database) storage Engine. NDB cluster- against shared storage. Nodes in MySQL Cluster are categorized into three categories namely: Data Nodes SQL Nodes Management Nodes MySQL Cluster

Architecture MySQL Cluster Image source

Data Node Implemented as a ndb or ndbmtd in the NDB cluster. Primary function is to process and retrieve information. Monitoring other nodes in the cluster and notifying the Management node. Perform recovery on restart. MySQL Cluster

SQL Node Runs as the mysqld program, provides application access to the data nodes by receiving all interaction requests as queries. Interaction layer between MySQL clients and the data nodes. Typically , SQL node is a MySQL server which uses the NDB cluster storage engine. MySQL Cluster

Management Node Manages all other nodes in the Cluster. Performs functions such as providing configuration details, starting and stopping the nodes, running back up etc.. Key point - a node of this type should be started first, before any other node. MySQL Cluster

Cluster Configuration Information regarding the number of nodes (processes), hosts , and their properties is necessary to set up the cluster. Local configuration file- residing on each data/API node. Global configuration file- the central one residing on one or more management nodes. MySQL Cluster

Configuration files # # config.ini # [NDB_MGMD DEFAULT] Portnumber=1186 [NDB_MGMD] NodeId=49 HostName=10.30.12.23 DataDir=C:/Users/kummadisingu/MySQL_Cluster/49/ Portnumber=1186 [NDBD] NodeId=1 HostName=10.30.12.28 DataDir=C:/Users/kummadisingu/MySQL_Cluster/1/ [MYSQLD] NodeId=54 HostName=10.30.12.30 # my.cnf # [mysqld] log-error=mysqld.54.err datadir="C:/Users/kummadisingu/MySQL_Cluster/54/" basedir="C:/Program Files/MySQL/MySQL Cluster 7.3/" port=3307 ndbcluster=on ndb-nodeid=54 ndb-connectstring=10.30.12.23:1186, MySQL Cluster

Auto-Sharding MySQL Cluster automatically partitions the tables across data nodes in the NDB cluster. By default, sharding is based on hashing of the primary key, which generally leads to a more even distribution of data and queries across the cluster Automatically creates node groups based on configuration parameters. MySQL Cluster

How it happens ? MySQL Cluster Image-source

Continued Operation of MySQL Cluster As long as each node group participating in the cluster has at least one node operating, the cluster has a complete copy of all data and remains viable. However, if both nodes from either node group fail the cluster has lost an entire partition and so can no longer provide access to a complete set of all cluster data F1~ fragment 1 of partition 1 MySQL Cluster Image-source

MySQL Cluster Replication Multiple clusters within a cluster – scenario when Geographical replication is in demand. Synchronous replication- happens only between data nodes and uses the two phase commit protocol. Asynchronous replication- replication between two or more clusters (multi master). MySQL Cluster

CAP theorem and MySQL Cluster A single MySQL Cluster prioritizes data consistency over availability when network partitions occur. A pair of asynchronously replicating MySQL Clusters prioritizes service availability over data consistency when network partitions occur. That’s Great! MySQL Cluster

Single MySQL Cluster - CP When a network partition occurs, live nodes in each partition regroup and decide what to do next: If there are not enough live nodes to serve all of the data stored – shutdown ~ degrade availability. After partition if the live nodes still holds all of the data- Continue to provide service. MySQL Cluster

Asynchronously replicating clusters - AP  Data consistency within each cluster is guaranteed as normal, but data consistency across the two clusters is not because of asynchronous replication. And yet, continue to accept read and write requests by implementing a special type of consistency mechanism known as …? Eventually Consistent MySQL Cluster

NoSQL and MySQL Cluster All the benefits of an ACID RDBMS + performance capabilities of Key/Value store = MySQL Cluster By pass the SQL layer and directly access the data nodes via the memcache API or various other NoSQL interfaces. MySQL Cluster

MySQL Cluster & Memcache API key-value interaction with NDB engine via the familiar memcached API. Extra caching layer~ very low latency. Image-source MySQL Cluster

Scheme-less storage in MySQL Cluster By default, every Key / Value is written to the same table with each Key / Value pair stored in a single row - thus allowing schema-less data storage Image-source MySQL Cluster

NoSQL + SQL Alternatively, we can define a key-prefix so that each value is linked to a pre-defined column in a specific table. Image-source MySQL Cluster

Additional Features Online scaling & up-gradation without temporary outage. Elastic in nature ~ compatible with the cloud computing framework. Unlike other distributed databases, MYSQL Cluster supports execution of complex join queries- preserving ACID properties. MySQL Cluster

Resilient to Failures Employs a self healing auto recovery mechanism with : Automatic transfer of control. Automatic restart and resynchronize. ~No Single point failures. MySQL Cluster

Queries: Two key points to remember For a table to be replicated in the cluster, it must use the NDB Cluster storage engine by specifying “ENGINE=NDBCLUSTER”: CREATE TABLE table_name (col_name column_definitions) ENGINE=NDBCLUSTER; ALTER TABLE table_name ENGINE=NDBCLUSTER; Every NDBCluster has a primary key. If no primary key is defined by the user when a table is created, the NDBCluster storage engine automatically generates a hidden one. MySQL Cluster

Limitations Partitioning. - Limited partitioning schemes. - Upper limit on the maximum number of partitions: ~ 8 * [ number of node groups ] Storage size. - Single MYSQL cluster has a storage limit of 3TB, which is very low. MySQL Cluster

Compare Unrestricted data size Batch processing Real time analytics Disk storage Real time analytics Mission critical applications Data Integrity MySQL Cluster

Conclusion MySQL’s answer to the NoSQL competitors in the big data web market. Certainly brings out the best of the both worlds. MySQL Cluster is already serving some of the most demanding web and mobile services on the planet MySQL Cluster

Customers MySQL Cluster

Available Versions MySQL Cluster GA (Generally available) MySQL Cluster Carrier Guide Edition (Community and Commercial versions) Includes MySQL Cluster Manager 24 hour online support. MySQL Cluster

MySQL Cluster