1. Cloud Data Replication with SQL Data Sync
October 2017
Warner Chaves
SQL MCM/ Data Platform MVP

2. Bio DBA and Consultant for 11 years
Previously L3 DBA at HP in Costa Rica, now Principal Consultant at Pythian in Ottawa, Ontario.
Microsoft Data Platform MVP.
Blog: Sqlturbo.com
Company: Pythian.com

3. Agenda
Objective: cover the capabilities of SQL Data Sync for Azure based data replication
We will go over:
Architecture
Difference with SQL Server Replication
Use Case Scenarios
Demos

4. Pre-requisites
Sql Server experience.
Azure SQL Database experience.

5. What is SQL Data Sync? Microsoft Azure Service
Built on top of Azure SQL Database
Syncs data bi-directionally between Azure SQL Db or SQL Server
Cloud based replication

6. Architecture Azure Region Sync Database Hub Database Sync Group
Sync Direction
Sync Database
Hub Database
Sync Group
Member: SQL Server On-Premises

7. Sync Group The Sync Schema describes which data is being synchronized.
The Sync Direction can be Hub to Member or Member to Hub, or both.
The Sync Interval is how often synchronization occurs.
The Conflict Resolution Policy is a group level policy, which can be Hub wins or Member wins.

8. Database Topology The Hub Database must be an Azure SQL Database.
The member databases can be either SQL Databases or SQL Server databases (cloud or on-prem).
The Sync Database contains the metadata and log for Data Sync. The Sync Database has to be an Azure SQL Database located in the same region as the Hub Database.
Synchronization happens between Hub and member(s) (5 minutes minimum).

9. Implementation
Tracking data changes: it is done with insert, update, and delete triggers. The changes are recorded in a side table in the user database.
Synchronizing data: the Hub syncs with each member individually. Changes from the Hub are downloaded to the member and then changes from the member are uploaded to the Hub.
Resolving conflicts: Data Sync provides two options for conflict resolution, Hub wins or Member wins.
If you select Hub wins, the changes in the hub always overwrite changes in the member.
If you select Member wins, the changes in the member overwrite changes in the hub. If there's more than one member, the final value depends on which member syncs first.

10. DEMO: Setting up Data Sync

11. What are the Use Cases?

12. Recommended Scenarios
Hybrid Data Footprint: clients that have some data on-prem on SQL and some in Azure SQL Db and need to sync some of it.
Distributed Database Application: when one application is split on multiple databases in the cloud but they all reference some common data.
Global cloud based replication: keeping databases synced across Azure regions for a subset of their data.

13. Not Recommended For
Disaster Recovery: since data sync works on an individual table level, DR is usually the whole database and there are more efficient ways to do that.
Read-Scale: same as the DR scenario, plus the latency on Data Sync might not make it acceptable for read scalability.
ETL: it is not an ETL solution since there are no transformation capabilities.
Migration from on-prem to Azure: there are other tools for this as well.

14. Master-Slaves Configuration

15. Example: Primary in Azure, Slaves in Azure and On-Premises
Azure Region
Sync Direction
Sync Database
Member: SQL Server On-Premises
Hub Database (Primary)
Sync Group
Slave Azure Database

16. Example: Primary On-Premises, Slaves in Azure
Azure Region
Member: SQL Server On-Premises (Primary)
Sync Direction
Sync Database
Hub Database
Sync Group
Azure Region
Slave Azure Database

17. Multi-Master Configuration

18. Example: Read-Write in Azure and On-Premises
Azure Region
Sync Direction
Sync Database
Hub Database
Sync Group
Member: SQL Server On-Premises

19. Example: Read-Write on both SQL Servers, Azure used for Hub to coordinate
Azure Region
Sync Direction
Sync Database
Hub Database
Sync Group
Member: SQL Server On-Premises
Member: SQL Server On-Premises

20. Careful: watch out for circular references!
Sync Group
Sync Group
Azure Region
Azure Region
Member: SQL Server On-Premises
Sync Database
Hub Database
Sync Database
Hub Database
Member: SQL Server On-Premises

21. Multi-Master
There is no complex conflict resolution, either Hub Wins or Member Wins.
For bigger safety, recommended to add a column to differentiate origin to the schema.
There is no circular reference detection, it’s up to you to setup a valid topology.
Don’t Go Crazy!

22. SQL Replication vs SQL Data Sync
SQL Server Replication
SQL Data Sync
Reads from transaction log
Uses scheduled agents
Complex conflict detection and handling
Can be continuous
Supports DDL
SQL Server only for publishing
Uses table triggers
Runs as a cloud service
Simple conflict detection and handling
5 minute minimum
No DDL support
Can publish from SQL Server or Azure SQL Db.

23. Service Cost The sync service is free right now until it goes GA
You pay for the Sync database depending on tier.
For Production use, start with at least S0.
If you add Hubs in different regions, your cost will go up since you will need another Sync database for that Hub.

24. Monitoring
You can use the PowerShell cmdlets to roll your own monitoring.
You can integrate the Data Sync log with Azure Monitor Log Analytics through a scheduled runbook.

25. DEMO: Data Sync in Action

26. Recap
Data replication scenarios are requiring a cloud component more often.
SQL Data Sync comes as a “SQL replication light” alternative using Azure SQL Db.
It’s not fit for all use cases, definitely analyze before moving forward.
It enables some interesting scenarios in terms of geographic sync and distributed applications.

27. Questions?

28. Thanks!!