Cloud Data Replication with SQL Data Sync October 2017 Warner Chaves SQL MCM/ Data Platform MVP

Bio DBA and Consultant for 11 years Previously L3 DBA at HP in Costa Rica, now Principal Consultant at Pythian in Ottawa, Ontario. Microsoft Data Platform MVP. Twitter: @warchav Blog: Sqlturbo.com Email: warner@sqlturbo.com Company: Pythian.com

Agenda Objective: cover the capabilities of SQL Data Sync for Azure based data replication We will go over: Architecture Difference with SQL Server Replication Use Case Scenarios Demos

Pre-requisites Sql Server experience. Azure SQL Database experience.

What is SQL Data Sync? Microsoft Azure Service Built on top of Azure SQL Database Syncs data bi-directionally between Azure SQL Db or SQL Server Cloud based replication

Architecture Azure Region Sync Database Hub Database Sync Group Sync Direction Sync Database Hub Database Sync Group Member: SQL Server On-Premises

Sync Group The Sync Schema describes which data is being synchronized. The Sync Direction can be Hub to Member or Member to Hub, or both. The Sync Interval is how often synchronization occurs. The Conflict Resolution Policy is a group level policy, which can be Hub wins or Member wins.

Database Topology The Hub Database must be an Azure SQL Database. The member databases can be either SQL Databases or SQL Server databases (cloud or on-prem). The Sync Database contains the metadata and log for Data Sync. The Sync Database has to be an Azure SQL Database located in the same region as the Hub Database. Synchronization happens between Hub and member(s) (5 minutes minimum).

Implementation Tracking data changes: it is done with insert, update, and delete triggers. The changes are recorded in a side table in the user database. Synchronizing data: the Hub syncs with each member individually. Changes from the Hub are downloaded to the member and then changes from the member are uploaded to the Hub. Resolving conflicts: Data Sync provides two options for conflict resolution, Hub wins or Member wins. If you select Hub wins, the changes in the hub always overwrite changes in the member. If you select Member wins, the changes in the member overwrite changes in the hub. If there's more than one member, the final value depends on which member syncs first.

DEMO: Setting up Data Sync

What are the Use Cases?

Recommended Scenarios Hybrid Data Footprint: clients that have some data on-prem on SQL and some in Azure SQL Db and need to sync some of it. Distributed Database Application: when one application is split on multiple databases in the cloud but they all reference some common data. Global cloud based replication: keeping databases synced across Azure regions for a subset of their data.

Not Recommended For Disaster Recovery: since data sync works on an individual table level, DR is usually the whole database and there are more efficient ways to do that. Read-Scale: same as the DR scenario, plus the latency on Data Sync might not make it acceptable for read scalability. ETL: it is not an ETL solution since there are no transformation capabilities. Migration from on-prem to Azure: there are other tools for this as well.

Master-Slaves Configuration

Example: Primary in Azure, Slaves in Azure and On-Premises Azure Region Sync Direction Sync Database Member: SQL Server On-Premises Hub Database (Primary) Sync Group Slave Azure Database

Example: Primary On-Premises, Slaves in Azure Azure Region Member: SQL Server On-Premises (Primary) Sync Direction Sync Database Hub Database Sync Group Azure Region Slave Azure Database

Multi-Master Configuration

Example: Read-Write in Azure and On-Premises Azure Region Sync Direction Sync Database Hub Database Sync Group Member: SQL Server On-Premises

Example: Read-Write on both SQL Servers, Azure used for Hub to coordinate Azure Region Sync Direction Sync Database Hub Database Sync Group Member: SQL Server On-Premises Member: SQL Server On-Premises

Careful: watch out for circular references! Sync Group Sync Group Azure Region Azure Region Member: SQL Server On-Premises Sync Database Hub Database Sync Database Hub Database Member: SQL Server On-Premises

Multi-Master There is no complex conflict resolution, either Hub Wins or Member Wins. For bigger safety, recommended to add a column to differentiate origin to the schema. There is no circular reference detection, it’s up to you to setup a valid topology. Don’t Go Crazy!

SQL Replication vs SQL Data Sync SQL Server Replication SQL Data Sync Reads from transaction log Uses scheduled agents Complex conflict detection and handling Can be continuous Supports DDL SQL Server only for publishing Uses table triggers Runs as a cloud service Simple conflict detection and handling 5 minute minimum No DDL support Can publish from SQL Server or Azure SQL Db.

Service Cost The sync service is free right now until it goes GA You pay for the Sync database depending on tier. For Production use, start with at least S0. If you add Hubs in different regions, your cost will go up since you will need another Sync database for that Hub.

Monitoring You can use the PowerShell cmdlets to roll your own monitoring. You can integrate the Data Sync log with Azure Monitor Log Analytics through a scheduled runbook.

DEMO: Data Sync in Action

Recap Data replication scenarios are requiring a cloud component more often. SQL Data Sync comes as a “SQL replication light” alternative using Azure SQL Db. It’s not fit for all use cases, definitely analyze before moving forward. It enables some interesting scenarios in terms of geographic sync and distributed applications.

Questions?

Thanks!!