Download presentation
Published byLenard Cox Modified over 7 years ago
1
Redgate's New Tool for SQL Server Database Provisioning
SQL Clone Redgate's New Tool for SQL Server Database Provisioning
2
Who am I? Kevin G. Boles SQL Server Consultant “The Wolfe” for the Relational Engine Indicium Resources, MCITP, MCT, MVP , yada-yada
3
What do I do? World-class Better than average relational engine expert Over 45,000 man-hours invested in SQL Server Absolutely LOVE doing targeted performance analysis and tuning work VERY good at making others better at interacting with SQL Server Couldn’t SSAS my way out of a paper bag … with both ends open!
4
Agenda Why database provisioning? Who does SQL Clone benefit?
How does SQL Clone work? Demos Usage Scenarios Pricing and Licensing Q+A
5
SQL Server Provisioning Study - 2017
Had over 2000 involved in beta testing
6
Database Sizes NOTE: This is just dev/test.
There are undoubtedly many large databases not being dropped down due to time/space constraints This is just in test/dev. There are undoubtedly large databases that are not moved down-stream due to size related issues (space or restore times)
7
Why Database Provisioning?
Quality (at speed). Realistic testing with repeatable processes. Move problems ‘left’. Replace current (painful) processes. It’s a job that’s ‘delegated up’. SIGNPOST – FIRSTLY!! THREE reason 1. LMAX, birthplace of CD. State of DevOps report. Gene Kim quote. Deterministically creating and configuring your resource as early as possible is also great practice for moving to nimbler developments workflows as demanded by DevOps teams. 2. Why is it painful? Databases are unlike other resources; they are by definition mutable, so only really exist as of a given point in time. And they are large, averaging around 100GB according to our latest survey. Not only that, but they have to be mounted or attached to a database management system; the commands required to make them available in the application context are complex and specific, variable by environment. 3. Purchasers feel the pain. People with the keys to Production have to do this, and are also responsible for the dead time, they feel the pain of blocking the software team that’s waiting to work on a hotfix or feature branch.
8
Problems: Quality at Speed
Matching production volumes Mirroring data distribution of real data Repeatable processes Time taken
9
Problems: Drive Space Multiple teams and multiple branches
Realistic data Isolated environments N times the amount of production data
10
Problems: Pressure on Environments
Multiple development teams Multiple developers per team Multiple branches of code Main branch Hot fixes New Development Automated testing and QA
11
Problems: Regulatory Compliance
Business regulations are increasing Health Care Banking Insurance International compliance (GDPR) Cost of data breaches!
12
Challenges of Using Realistic Data
Shouldn’t be on production (it is a pet peeve of mine to see test databases on production box!) Moving/restoring large amounts of data takes time Storing large amounts of data costs money Data may be sensitive
13
Primary Target Audience
Who is responsible for data provisioning? DBA IT Manager Head of Data Services Chief Data Officer
14
Who Benefits from Production-realistic Data?
Developers Testers/QA BI/Reporting teams Performance Troubleshooting staff
15
What Is SQL Clone? “SQL Clone is a SQL Server database cloning tool that allows you to create editable copies of databases within seconds while using very little additional disk space.”
16
Technical details and architecture
21
Volume Shadow Copy (VSS)
64-bit Windows technology Back up/Copy a file/disk at a point in time while in use Go to that point in time quickly Other uses: SQL Server snapshots VMs
22
Virtual Disk Service? 64-bit Windows technology
Allocate logical storage without taking disk space Mount remote storage as local Mount read-only with differencing disks Virtual hard drive ‘chaining’
23
Image Creation In Action
Create an initial, small VHD, at the data image location (shown as 10MB in size in Figure 8) Create a mount point to this VHD on the local operating system of the machine hosting the source database Use the VSC service to “remote copy” the MDF and LDF files to the data image location Un-mount the VHD
24
DEMOS
25
Solutions: Realistic Data
Clean the data as necessary Create a SQL Clone database copy Share that copy, including realistic data, with all teams Troubleshoot not on production
26
Solutions: Drive Space
Clones are small (save up to 99% of disk space) Differential storage only Refresh of an environment resets space used
27
Solutions: Shared Environments
Isolated Environments! Database is a mutable object. Multiple people can cause conflicts, breakages, etc Multiple clones per data image Multiple environments Clones are very small Creating clones takes seconds
28
Solutions: Regulatory Compliance
Clean data in a clone or a copy of database Create new data image Distribute the cleaned data image as clones Works with database internal processes such as Dynamic Data Masking (if your security is the same)
29
Solutions: Data Obfuscation
Clean data prior to clone process OR Create a clone from a data image Clean the clone Create a data image of the cleaned clone Clone the new data image (Pursuing clone chaining)
30
Solutions: A/B Testing
Clone A Series A Mods Clone B Series B Mods Comparative Analytics or Testing \\FileShare\Images\Financial
31
Solutions: Do More Faster than normal restores
Smaller than normal restores Fully automatable Self-service
32
Features & limitations
33
V1 features Central Management Server Image from live DB
Image from backup file Image from Clone – full sized Automation – PowerShell cmdlets Permissions – Basic
34
V1 limitations 2TB image limit Image from backup with passwords
64 bit Windows 2008R2/7 upwards only SQL Server 2008 upwards only* Beware default image location in C drive * Clones do not circumvent Express 10GB database size limit since SQL Server still perceives the clone as its original size.
35
Data Masking Workflows
Image > Clone > Mask > Image In V1 and Image of Clone is full size Data Generator Custom script Active Research for Masking and Chaining
36
Roadmap Advanced access and permissions
Create new image from a Clone using chaining Potential: Incrementing images from changes Extending the 2TB limit Data masking
37
SQL Clone Goals Encourage you to give your developers, analysts, troubleshooters, etc. the necessary data for more accurate and timely work Provide a mechanism for automating and managing provisioning of development/test/reporting databases Enable you to better support a DevOps approach within your environment
38
Bonus Round SQL Clone Competition (25 Amazon Echo Dots, 5 Clone Licenses!!). Open until April 30!
39
What About TDE?? Extracting image from live database works fine
Creating a clone database requires a Database Master Key (can be different from source server) and restores of the original Certificate and Database Encryption Key Extracting from backup also requires DMK and Cert
40
Packaging and Pricing The product is clearly team focused, so it is sold in user packs Pricing scales as number of seats goes up UNLIMITED usage regarding Images and Clones Primary other entry in this market is Delphix. It is upscale and does already have data masking, but it also runs six-figures
41
SQL Clone Licensing
42
Questions? Links SQLClone@red-gate.com
Tell them I sent you!!! I may get a small spiff - and I may split it with you! @TheSQLGuru
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.