1. Redgate's New Tool for SQL Server Database Provisioning
SQL Clone
Redgate's New Tool for
SQL Server Database Provisioning

2. Who am I?
Kevin G. Boles SQL Server Consultant “The Wolfe” for the Relational Engine Indicium Resources, MCITP, MCT, MVP , yada-yada

3. What do I do?
World-class Better than average relational engine expert 
Over 45,000 man-hours invested in SQL Server
Absolutely LOVE doing targeted performance analysis and tuning work
VERY good at making others better at interacting with SQL Server
Couldn’t SSAS my way out of a paper bag
… with both ends open! 

4. Agenda Why database provisioning? Who does SQL Clone benefit?
How does SQL Clone work?
Demos
Usage Scenarios
Pricing and Licensing
Q+A

5. SQL Server Provisioning Study - 2017
Had over 2000 involved in beta testing

6. Database Sizes NOTE: This is just dev/test.
There are undoubtedly many large databases not being dropped down due to time/space constraints
This is just in test/dev. There are undoubtedly large databases that are not moved down-stream due to size related issues (space or restore times)

7. Why Database Provisioning?
Quality (at speed). Realistic testing with repeatable processes. Move problems ‘left’.
Replace current (painful) processes.
It’s a job that’s ‘delegated up’.
SIGNPOST – FIRSTLY!! THREE reason
1. LMAX, birthplace of CD. State of DevOps report. Gene Kim quote. Deterministically creating and configuring your resource as early as possible is also great practice for moving to nimbler developments workflows as demanded by DevOps teams.
2. Why is it painful? Databases are unlike other resources; they are by definition mutable, so only really exist as of a given point in time.
And they are large, averaging around 100GB according to our latest survey. Not only that, but they have to be mounted or attached to a database management system; the commands required to make them available in the application context are complex and specific, variable by environment.
3. Purchasers feel the pain. People with the keys to Production have to do this, and are also responsible for the dead time, they feel the pain of blocking the software team that’s waiting to work on a hotfix or feature branch.

8. Problems: Quality at Speed
Matching production volumes
Mirroring data distribution of real data
Repeatable processes
Time taken

9. Problems: Drive Space Multiple teams and multiple branches
Realistic data
Isolated environments
N times the amount of production data

10. Problems: Pressure on Environments
Multiple development teams
Multiple developers per team
Multiple branches of code
Main branch
Hot fixes
New Development
Automated testing and QA

11. Problems: Regulatory Compliance
Business regulations are increasing
Health Care
Banking
Insurance
International compliance (GDPR)
Cost of data breaches!

12. Challenges of Using Realistic Data
Shouldn’t be on production (it is a pet peeve of mine to see test databases on production box!)
Moving/restoring large amounts of data takes time
Storing large amounts of data costs money
Data may be sensitive

13. Primary Target Audience
Who is responsible for data provisioning?
DBA
IT Manager
Head of Data Services
Chief Data Officer

14. Who Benefits from Production-realistic Data?
Developers
Testers/QA
BI/Reporting teams
Performance Troubleshooting staff

15. What Is SQL Clone?
“SQL Clone is a SQL Server database cloning tool that allows you to create editable copies of databases within seconds while using very little additional disk space.”

16. Technical details and architecture

21. Volume Shadow Copy (VSS)
64-bit Windows technology
Back up/Copy a file/disk at a point in time while in use
Go to that point in time quickly
Other uses:
SQL Server snapshots
VMs

22. Virtual Disk Service? 64-bit Windows technology
Allocate logical storage without taking disk space
Mount remote storage as local
Mount read-only with differencing disks
Virtual hard drive ‘chaining’

23. Image Creation In Action
Create an initial, small VHD, at the data image location (shown as 10MB in size in Figure 8)
Create a mount point to this VHD on the local operating system of the machine hosting the source database
Use the VSC service to “remote copy” the MDF and LDF files to the data image location
Un-mount the VHD

24. DEMOS

25. Solutions: Realistic Data
Clean the data as necessary
Create a SQL Clone database copy
Share that copy, including realistic data, with all teams
Troubleshoot not on production

26. Solutions: Drive Space
Clones are small (save up to 99% of disk space)
Differential storage only
Refresh of an environment resets space used

27. Solutions: Shared Environments
Isolated Environments!
Database is a mutable object. Multiple people can cause conflicts, breakages, etc
Multiple clones per data image
Multiple environments
Clones are very small
Creating clones takes seconds

28. Solutions: Regulatory Compliance
Clean data in a clone or a copy of database
Create new data image
Distribute the cleaned data image as clones
Works with database internal processes such as Dynamic Data Masking (if your security is the same)

29. Solutions: Data Obfuscation
Clean data prior to clone process OR
Create a clone from a data image
Clean the clone
Create a data image of the cleaned clone
Clone the new data image
(Pursuing clone chaining)

30. Solutions: A/B Testing
Clone A
Series A Mods
Clone B
Series B Mods
Comparative Analytics or Testing
\\FileShare\Images\Financial

31. Solutions: Do More Faster than normal restores
Smaller than normal restores
Fully automatable
Self-service

32. Features & limitations

33. V1 features Central Management Server Image from live DB
Image from backup file
Image from Clone – full sized
Automation – PowerShell cmdlets
Permissions – Basic

34. V1 limitations 2TB image limit Image from backup with passwords
64 bit Windows 2008R2/7 upwards only
SQL Server 2008 upwards only*
Beware default image location in C drive
* Clones do not circumvent Express 10GB database size limit since SQL Server still perceives the clone as its original size.

35. Data Masking Workflows
Image > Clone > Mask > Image
In V1 and Image of Clone is full size
Data Generator
Custom script
Active Research for Masking and Chaining

36. Roadmap Advanced access and permissions
Create new image from a Clone using chaining
Potential:
Incrementing images from changes
Extending the 2TB limit
Data masking

37. SQL Clone Goals
Encourage you to give your developers, analysts, troubleshooters, etc. the necessary data for more accurate and timely work
Provide a mechanism for automating and managing provisioning of development/test/reporting databases
Enable you to better support a DevOps approach within your environment

38. Bonus Round
SQL Clone Competition (25 Amazon Echo Dots, 5 Clone Licenses!!). Open until April 30!

39. What About TDE?? Extracting image from live database works fine
Creating a clone database requires a Database Master Key (can be different from source server) and restores of the original Certificate and Database Encryption Key
Extracting from backup also requires DMK and Cert

40. Packaging and Pricing
The product is clearly team focused, so it is sold in user packs
Pricing scales as number of seats goes up
UNLIMITED usage regarding Images and Clones
Primary other entry in this market is Delphix. It is upscale and does already have data masking, but it also runs six-figures

41. SQL Clone Licensing

42. Questions? Links SQLClone@red-gate.com
Tell them I sent you!!! I may get a small spiff - and I may split it with you! 
@TheSQLGuru