1. Scaling out and in with Azure SQL DB Elastic Scale DBA-203 Warner Chaves, MCM/MVP, SQLTurbo.com, Pythian.com

2. SQL Server DBA for 10 years. Previously an L3 DBA at HP in Costa Rica, now a Principal Consultant at Pythian in Ottawa Ontario. SQL Server MCM and MVP. Twitter: @warchav Email: warner@sqlturbo.comwarner@sqlturbo.com Blog: SQLTurbo.com Company: Pythian.com About me 2

3. Agenda Goal of today: provide an overview of the elastic capabilities of the Azure SQL Database platform. Database elasticity intro Database Elastic Tools Database Elastic Query Database Elastic Pools Database Elastic Jobs 3

4. We’re going to focus on DbaaS, Azure SQL Db. Building and shrinking database capacity on demand. Elastic scale focuses on scale OUT (“Sharding”), as opposed to just scaling UP. Assume one application, common schema on all dbs. Database layer elasticity 4

5. 1.Hitting resource limits on a scale up solution (CPU-RAM-size). 2.Geographic considerations. 3.Security concerns require separate data. 4.Easier chargeback mechanism for client. 5.Easier to manage individually for a particular client. Why scale out? 5

6. Provisioning speed. Controlling cost. Shard data movement. Routing the application to the proper database. Querying the databases individually as well as easily querying the global set. Common challenges to scale out 6

7. A Db elastic solution architecture 7

8. Creating and working with an elastic data tier

9. The purpose of the library is to facilitate building applications with an elastic db layer on Azure SQL db. Main concepts: Shard ~ Database Shard Set Shardlet Shard Map Manager Reference tables Elastic db tools:.NET client library 9

10. Db Elastic Client Library 10

11. 1.Create a database called a “Shard Map Manager”. 2.Register databases as shards in the SMM. 3.Setup the shard distribution method (lists of keys or ranges). 4.Request a connection from the library based on the sharding key (data dependent routing). 5.Use that connection for the db interaction of the shard. 6.Multi-shard queries are possible too. Workflow for the client library 11

12. The Split/Merge tool helps with rebalancing data in the shard set. You can SPLIT a db into multiple. You can MERGE multiple dbs into one. These operations allow to respond to changes in capacity needs. It’s delivered as a customer-hosted Azure service. Elastic db tools: Split/Merge tool 12

13. Elastic db tools: Split/Merge tool 13 Db1 DbN MERGESPLIT Split/Merge Service ShardMap SplitMergeDb

14. Elastic db tools: Split/Merge interface 14

15. Elastic db tools: Split/Merge interface 15

16. Allows querying a set of databases as a single entity through one connection string. This is useful for 3 rd party apps where we can’t insert the.NET elastic scale library (Excel, SSRS, etc). Elastic Database Query 16

17. 1.A set of Azure SQL Dbs with a common schema (any tier). 2.A shard map and Standard or Premium tier db to be the query connection endpoint. 3.Creating a credential for the elastic query: Configuring Elastic Database Queries 17 CREATE MASTER KEY ENCRYPTION BY PASSWORD = 'password'; CREATE DATABASE SCOPED CREDENTIAL WITH IDENTITY = ' ', SECRET = ' ';

18. 4.Create an external data source on the endpoint db: Configuring Elastic Database Queries 18 CREATE EXTERNAL DATA SOURCE WITH (TYPE = SHARD_MAP_MANAGER, LOCATION = ' ', DATABASE_NAME = ' ', CREDENTIAL =, SHARD_MAP_NAME = ' ');

19. 5.Create an external table: Configuring Elastic Database Queries 19 CREATE EXTERNAL TABLE [ database_name. [ dbo ]. | dbo. ] table_name ( { } [,...n ]) { WITH ( ) } )[;] ::= DATA_SOURCE =, DISTRIBUTION = SHARDED( ) | REPLICATED | ROUND_ROBIN

20. SP_EXECUTE_FANOUT: can consume a shard map to run a fanout query over the set of sharded databases and concatenate the results. TARGET=RDBMS: cross database queries are supported now by creating a external source that targets a single database. Other flavors of Elastic Db Queries 20

21. Sample App Demo

22. Managing elastic resources

23. Allows pooling Azure SQL resources for a set of dbs for a more efficient performance/cost ratio. The resources are quantified as eDTUs (elastic DTUs). Storage size is also tied to the eDTUs and depend on tier. Different pools per Service Tier. Elastic Database Pools 23

24. TierMax Pool Size Max DbseDTU pool range Max eDTU/db Max Size / db Basic120GB400100-1200 edtu 52GB Standard1200GB400100-1200 edtu 100250GB Premium750GB50125-1500 edtu 1000500GB Database pool specifications 24

25. One db patternThe pooled pattern Elastic Database Pools 25

26. Fixed cost per eDTU (min 100, max 1500). Example: 16 S3 databases: 16x$150 per month = $2400, each db has static 100 DTU. 250GBx16 storage. One Standard pool: (800 eDTUx$2.23) per month=$1784, each db can go 10-100 DTU. 800GB storage. Elastic Database Pools: pricing 26

27. Create a db pool Demo

28. Running jobs on an elastic data tier

29. Allows running jobs against a set of databases. Either a T-SQL script or a DACPAC. Can be a custom set, a shard set or a db pool. Scheduling and retry similar to the SQL Agent. Elastic Database Jobs 29

30. It’s a customer-hosted Azure solution. Cost is per component. Requires the following components setup during install: Cloud service (worker role to run tasks). SQL Db (control db to store metadata, S0 default). Service bus (used for work coordination). Azure storage (for diagnostic output). Setting up Elastic Database Jobs 30

31. How do Elastic Database Jobs work? 31 Control DB Controller Job Task Exec Portal/P$ A database set Job info Job tasks Execution and results

32. Using Elastic db jobs Demo

33. If you want easy scale out of a.NET app: elastic db tools. If you want to be flexible on resource consumption: elastic database pools. Recap 33

34. If you want queries that span multiple dbs: elastic db query. If you want to run jobs on 1 to N dbs: elastic database jobs. Recap 34

35. Q/A

36. Thank You!