Mission-critical performance with Microsoft SQL Server 2016

Mission-critical performance with Microsoft SQL Server 2016
12/2/2017 1:49 AM Mission-critical performance with Microsoft SQL Server 2016 Speaker Name © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Summary of Day 1 Real-Time Operational Analytics In-Memory Optimization (OLTP) enhancements Stretch Database Always Encrypted

Summary of Day 2 Query Data Store and Live Query Statistics PolyBase Row-Level Security and Dynamic Data Masking High availability and disaster recovery using Microsoft Azure Virtual Machines

Capabilities and benefits
BI and analytics Dashboards Reporting Capabilities Runs analytics queries concurrently with operational workload using same schema Requires no application changes to run data warehouse queries on In-Memory OLTP workload Benefits Minimal impact on OLTP workload Offers best available performance and scalability Offloads analytics workload to readable secondary BI analysts Presentation layer Dashboards Reporting IIS Server SQL Server Analysis Services Application tier Objective: This slide introduces the concept of Real-Time Operational Analytics, and how you can run both analytics and OLTP workloads on the same database tables at the same time. Talking points: SQL Server 2016 can help everyone struggling with the problem stated in the last slide; the SQL Server contains a solution for allowing the operational database to run analytical queries in real-time. This is possible by allowing you to add analytics-specific updatable columnstore indexes on rowstore tables. The OLTP workload runs against the rowstore table and the analytics workload runs against the columnstore. SQL Server automatically maintains all changes to the indexes, ensuring that OLTP changes are up-to-date for analytics. Combining these factors, it’s possible and practical to run analytics in real-time on up-to-date data. This works for both disk-based and memory-optimized tables. With Real-Time Operational Analytics, you can eliminate both the ETL and the need to maintain a separate data warehouse. This can improve the overall performance of your OLTP workload since there will be fewer indexes to maintain. This is all you need to do. You are now ready to perform Real-Time Operational Analytics without requiring any changes to your application. Analytics queries will run against the columnstore index and OLTP operations will keep running against the b-tree index(s). The OLTP workloads will continue to perform, but will incur some additional overhead to maintain the columnstore index. You can minimize the columnstore index maintenance using a filtered columnstore index; however, analytics queries can still take significant computing resources (CPU, IO, Memory) and thereby impact the performance of operational workload. For most mission critical workloads, the SQL Server recommendation is to use AlwaysOn configuration. In this configuration, you can eliminate the impact of running analytics by offloading it to a readable secondary. SQL Server Database Add analytics-specific indexes

Enabling real-time analytics
12/2/2017 1:49 AM Enabling real-time analytics Dashboards Reporting Add columnstore index BI and analytics SQL Server Database Application tier Presentation layer IIS Server Analysis Services OLTP workload Real-time Operational Analytics targets the scenario that has a single data source, such as an enterprise resource planning (ERP) application, and makes it possible to run both operational and analytics workloads. This doesn’t replace the need for a separate data warehouse when: Data from multiple sources must be integrated before running the analytics workload Extreme analytics performance using pre-aggregated data, such as cubes, is required Dashboards Reporting Objective: This slide introduces the intent of Real-Time Operational Analytics and where it cannot be used or replaced. Talking points: Real-Time Operational Analytics targets the scenario where you have single data source such as an ERP application where you can run both the operational workload and analytics workload concurrently. This does not replace the need for a separate data warehouse where you need to integrate data from multiple sources before running an analytics workload or when you require extreme analytics performance-using schemas to be customized (for example, Star/Snowflake) or pre-aggregated data, such as cubes. Also this is not replacement where data is coming from non-relational sources. © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Non-clustered columnstore index
Create updatable non-clustered columnstore index (NCCI) for analytics queries Drop all other indexes that were created for analytics Maintain columnstore like any other index Use Query Optimizer to choose columnstore index where needed Requires no application changes Relational table (clustered index/heap) B-tree index Relational table (clustered index/heap) B-tree index Delete bitmap Objective: This slide introduces how you can get Real-Time Operational Analytics on a disk-based table. Talking points: Non-clustered columnstore indexes (NCCIs) are also getting some enhancements in SQL Server The most notable of these enhancements is the ability to be updated. First offered in 2012, NCCIs were designed to be read-only snapshots of a normal heap or b-tree table. The idea is that you would drop and rebuild them on a regular basis, perhaps during a nightly or weekly maintenance cycle. Alternately, you could use partition swapping to load more data. SQL Server automatically maintains all changes to the indexes, ensuring OLTP changes are up-to-date for analytics. This works for both disk-based and memory-optimized tables. To get started with real-time analytics for a disk-based table, simply create an NCCI for analytics queries and drop all other indexes that were created for analytics. You don’t need to change your application since the query optimizer estimates the performance cost and automatically chooses the best index for each query. Analytics queries will run against the columnstore index and OLTP operations will keep running against the b-tree index. The OLTP workloads will continue to perform, but will incur some additional overhead to maintain the columnstore. Animation <<first click>> Top of house (B-tree index and Relational table) appears Animation <<second click>> Bottom of house (Non-clustered columnstore index) appears Delta rowgroups Non-clustered columnstore index

Using columnstore on in-memory tables
12/2/2017 Using columnstore on in-memory tables Requires no explicit delta rowgroup Keeps rows (tail) not in columnstore In-Memory OLTP table Requires no columnstore index overhead when operating on tail Uses background task to migrate rows from tail to columnstore in chunks of 1 million rows Uses Deleted Rows Table (DRT) to track deleted rows Keeps columnstore data fully resident in memory and persisted together with operational data You can use Compression Delay to keep data longer in the “tail” to eliminate impact on OLTP workload operating on the “hot” data Requires no application changes Hash index nonclustered index tail Deleted rows table Objective: This slide explains how you can add a columnstore index to an in-memory table. Talking points: SQL Server 2016 has the ability to place a columnstore index on a memory-optimized table. The columnstore index on an in-memory table allows operational analytics by integrating In-Memory OLTP and in-memory columnstore technologies to deliver high performance for OLTP and analytics workloads. You can create a columnstore index on an empty table but all columns must be included in the columnstore. A columnstore index on in-memory table can only be added as part of table creation. You cannot add a columnstore index on an existing in-memory table. This limitation will be removed before SQL Server Community Technology Preview 3.2 (CTP 3.2) is released for RTM. Create a Clustered Columnstore Index To create a clustered columnstore index, first create a rowstore table as a heap or clustered index, and then use the CREATE CLUSTERED COLUMNSTORE INDEX (Transact-SQL) statement to convert the table to a clustered columnstore index. If you want the clustered columnstore index to have the same name as the clustered index, use the DROP_EXISTING option. Load Data into a Clustered Columnstore Index You can add data to an existing clustered columnstore index by using any of the standard loading methods. For example, the bcp bulk loading tool, Integration Services, and INSERT … SELECT can all load data into a clustered columnstore index. In-Memory OLTP table Hot © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Why In-Memory OLTP? Customers are demanding ever-higher concurrency and lower latency, at a lower cost Hardware trends demand architectural changes to the relational database management system (RDBMS) to meet those demands In-Memory Online Transaction Processing (OLTP), a key new feature of SQL Server 2016, offers: High performance Memory-optimized OLTP engine SQL Server integration Modern hardware architecture Objective: This slide describes the design considerations and purpose for developing the In-Memory OLTP by SQL Server. Talking points: The move to produce a true main-memory database has been driven by two basic motivating trends and needs, including: Market demands for lower latency time and high concurrency for data operations, and specialized workload like In-Memory OLTP. Demands from recent hardware changes that allow fitting most or all of the data required by a workload into main memory, requiring specialized database engines that target specific types of workloads need to be tuned just for those workloads. SQL Server was originally designed at a time when it could be assumed that main memory was very expensive, so data needed to reside on disk except when it was needed for processing. This assumption is no longer valid, as memory prices have dropped enormously over the last 30 years. At the same time, multi-core servers have become affordable, so that today one can buy a server with 32 cores and 1 TB of memory for under $50,000. Since many, if not most, of the OLTP databases in production can fit entirely in 1 TB, we need to re-evaluate the benefit of storing data on disk and incurring the I/O expense when the data needs to be read into memory to be processed. In addition, OLTP databases also incur expenses when this data is updated and needs to be written back out to disk. If reading from disk isn’t required, the optimizer can use a different costing algorithm. In addition, if there is no wait time required for disk reads, other wait statistics―such as waiting for locks to be released, waiting for latches to be available, or waiting for log writes to complete―can become disproportionately large. In- Memory OLTP addresses all of these issues. In-Memory OLTP removes the issues of waiting for locks to be released, using a new type of multi-version optimistic concurrency control. Hence latency and concurrency improves a lot with these In-Memory OLTP architectural changes.

Why In-Memory OLTP? (continued)
12/2/2017 1:49 AM Why In-Memory OLTP? (continued) In-Memory OLTP is a memory-optimized database engine integrated into the SQL Server engine, optimized for OLTP, that can significantly improve database application performance Features of In-Memory OLTP include: Memory-optimized tables Natively compiled stored procedures Objective: This slide introduces SQL Server In-Memory OLTP and its feature sets. Talking points: In-Memory OLTP is the in-memory processing technology from Microsoft. It can significantly improve OLTP database application performance. In-Memory OLTP is a memory-optimized database engine integrated into the SQL Server engine, and can be used in the exact same manner as any other database engine component. In-Memory OLTP originally shipped with SQL Server 2014 and features two new data structures, including memory-optimized tables and natively compiled stored procedures. Memory-optimized tables Memory-optimized tables store their data into memory using multiple versions of the data from each row. Memory-optimized tables are fully transactional, durable, and are accessed using Transact-SQL in the same manner as disk-based tables. A query can reference both memory-optimized and disk-based tables. A transaction can update data in memory-optimized and disk-based tables. (This will be discussed more on the next slide.) Natively compiled stored procedures A natively compiled stored procedure is a SQL Server object that can access only memory-optimized data structures such as memory-optimized tables, table variables, etc. SQL Server can natively compile stored procedures that access memory-optimized tables. Native compilation allows faster data access and more efficient query execution than interpreted (traditional) Transact-SQL. Natively compiled stored procedures are parsed and compiled when they are loaded to native DLLs. This is in contrast to other stored procedures which are compiled on first run, have an execution plan created and reused, and use an interpreter for execution. (This will be discussed more on the next slide.) © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Why In-Memory OLTP? (continued)
12/2/2017 1:49 AM Why In-Memory OLTP? (continued) Memory-optimized table A memory-optimized table has one representation of itself in active memory, in addition to standard representation on hard drive Business transactions against a table run faster because they directly interact with only representation in active memory Natively compiled stored procedures A natively compiled stored procedure is a SQL Server object that can access only memory-optimized data structures (memory-optimized tables, table variables, etc.) Natively compiled stored procedures require fewer machine instructions during runtime than they would if they were created as traditional interpreted stored procedures Objective: This slide gives more insight on memory-optimized tables and natively compiled stored procedures. Talking point: The features of In-Memory OLTP include memory-optimized tables and natively compiled stored procedures. A memory-optimized table has one representation of itself in active memory, in addition to the standard representation on a hard drive. Business transactions against the table run faster because they directly interact with only the representation that is in active memory. The main features of a memory-optimized table are: Rows in tables are read from, and written to, memory. The entire table resides in memory. It uses latch-free data structures and optimistic, multi-version concurrency control. It maintains a second copy on disk for durability (if enabled). Data in memory-optimized tables is only read from disk during database recovery. It is interoperable with disk-based tables. A natively compiled stored procedure is a SQL Server object that can access only memory-optimized data structures such as memory-optimized tables, table variables, etc. Natively compiled stored procedures require fewer machine instructions during runtime than they would if they were created as traditional interpreted stored procedures. The main features of a natively compiled stored procedure are: It is compiled to native code (DLL) upon its creation (the interpreted stored procedures are compiled at first execution). Its aggressive optimization takes time during compilation. It can only interact with memory-optimized tables. The call to a natively compiled stored procedure is actually a call to its DLL entry point. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

In-Memory OLTP value proposition
Target scenarios: OLTP apps with high throughput and low-latency requirements Extreme OLTP performance (up to 30x) due to two factors: High scalability through lock- and latch-free architecture enables linear scaling Low latency through efficient data access and query processing: memory-optimized data structures and native compilation Integrated into SQL Server for low total cost of ownership (TCO) Part of Enterprise Edition Same databases can have in-memory and on-disk Integrated developer and DBA experience: same T-SQL, client stack, tooling, backup/restore, AlwaysOn Objective: This slide highlights the value proposition of In-Memory OLTP, such as high performance and low total cost of ownership (TCO). Talking point: The target scenario In-Memory OLTP includes OLTP apps with high throughput and low-latency requirements. Extreme OLTP performance SQL Server In-Memory OLTP is a new data-processing technology that allows a many-fold improvement in performance for OLTP applications that demands high throughput and low-latency. The In-Memory OLTP engine is designed for extremely high session concurrency for OLTP-type transactions driven from a highly scaled-out middle-tier. You can gain performance improvement up to 30x due to two factors: It uses latch-free data structures and optimistic, multi-version concurrency control. The result is predictable, sub-millisecond low latency and high throughput with linear scaling for database transactions. A transaction can update data in memory-optimized tables and disk-based tables. Stored procedures that only reference memory-optimized tables can be natively compiled into machine code for further performance improvements. Integrated into SQL Server for low TCO Another impressive thing about In-Memory OLTP is that it achieves breakthrough improvement in transactional processing capabilities without requiring a separate data management product or new programming model. It is still SQL Server! This allows for an integrated developer and database administrator (DBA) experience with the same T-SQL, client stack, tooling, backup and restore, and AlwaysOn capabilities. By offering in-memory functionality within SQL Server, your total cost of ownership is lower than if you had to purchase, manage, and maintain a separate system for handling in-memory processing.

In-Memory OLTP enhancements
12/2/2017 1:49 AM In-Memory OLTP enhancements Better T-SQL coverage, including: Full collations support in native modules Query surface-area improvements Nested stored procedures (EXECUTE) Natively compiled scalar user-defined functions Query Store support Other improvements: Full schema change support: add/alter/drop column/constraint Increased size allowed for durable tables ALTER TABLE support Multiple Active Results Sets (MARS) support ALTER TABLE Sales.SalesOrderDetail ALTER INDEX PK_SalesOrderID REBUILD WITH (BUCKET_COUNT= ) T-SQL surface area: New {LEFT|RIGHT} OUTER JOIN Disjunction (OR, NOT) UNION [ALL] SELECT DISTINCT Subqueries (EXISTS, IN, scalar) Objective: This slide highlights the In-Memory OLTP enhancements in SQL Server 2016. Talking points In-Memory OLTP was introduced in SQL Server 2014 and has been enhanced for SQL Server You will find numerous enhancements to the technology, with some limitations removed. Many of these changes are described below. We will talk more on this in the coming slides: SQL Server 2014 does not allow any changes to memory-optimized tables after creation. In SQL Server 2016, ALTER TABLE can be used on memory-optimized tables to add, drop, or alter columns, or to add, drop, or rebuild indexes. SQL Server 2016 has better coverage of T-SQL query surface area, adding support for left outer joins, union all, and distinct syntax. It also includes support for the nesting of natively compiled stored procedures and other increases in the T-SQL surface area. Other T-SQL native compilation improvements include: Full collations support in native modules Natively compiled scalar user-defined functions (UDFs) Access from both native and interop Improved performance of scalar UDFs in traditional workloads, if no table access is required Query Store support Scaling has also been improved by including a large increase to the total size of durable tables and improved throughput, with increased socket availability and larger log generation capacity for AlwaysOn configuration. Other improvements include: T-SQL table improvements through: Collations Full collations support in index key columns All code pages supported for (var)char columns Constraints FOREIGN KEY CHECK UNIQUE constraints DML triggers on memory-optimized tables (AFTER; natively compiled) Row-Level Security Temporal tables (history is disk-based) In-Memory OLTP now supports Multiple Active Result Sets (MARS) using queries, as well as parallel plans in interop for reporting queries and Transparent Data Encryption (TDE). Enhancements also include support for the Transaction Performance Analysis report in SQL Server Management Studio to evaluate In-Memory OLTP and improve database application performance. You can access lightweight migration reports and checklists that show unsupported features used in current disk-based tables and interpreted T-SQL stored procedures. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

What is Microsoft Stretch Database?
12/2/2017 1:49 AM What is Microsoft Stretch Database? Solution for securely stretching cold tables to Microsoft Azure with remote query processing Capability Stretches large operational tables from on-premises to Azure with ability to query Benefits Offers cost-effective online cold data Archives historical data transparently and securely Keeps entire table online and queryable via on-premises apps Requires no application changes Supports Always Encrypted and Row-Level Security Order History Order History SQL Azure Objective: This slide introduce a new feature of SQL Server 2016: Stretch Database. This is a solution for securely stretching cold tables to Microsoft Azure with remote query processing. Talking points: The Stretch Database feature securely and transparently archives your cold or historical data from a local SQL Server database to Azure SQL Database with remote query processing capability. (The SQL Database service in the Azure cloud is provided as a platform as a service [PaaS]). Animation <<first click>> Once you enable this feature for a table, SQL Server silently and transparently moves/migrates table data to Azure SQL Database… Animation <<second click>> …and no application change is required to access this data. You can still have a single query accessing these two types of data or tables. These are some of the benefits of using this feature: Storage of cold data in Azure SQL Database is cost effective; that is, there is reduced cost and complexity in keeping cold data online in Azure SQL Database. Secure and transparent movement of cold or historical data without writing a data movement module; makes local queries and other database operations run faster as they have to work on hot data or local data most of the time. Archived data remains online and they are queryable like any other table in a local SQL Server database. No application change is required to access these archived tables or data; a single query accessing these two types of data or tables at a given time or in the same query. A top benefit of this feature is that (even though cold data is stored externally in Azure SQL Database but is online) it is transitionally consistent and works with other SQL features like Always Encrypted, Row Level Security, etc. Customers Products Order History Stretch to cloud Order History Server 2016 SQL App © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Capabilities and functions
12/2/2017 1:49 AM Capabilities and functions Migrates your historical data to Microsoft Azure SQL Database Offers option to pause data migration Troubleshoots problems on local server Maximizes available network bandwidth Ensures no data is lost Retries logic to handle connection issues Uses dynamic management view to check migration status Identifies databases and tables using Stretch Database Advisor, a feature of Microsoft SQL Server 2016 Upgrade Advisor Objective: This slide shows a brief overview of the capabilities and functions of Stretch Database and provides an understanding of what Stretch Database does. Talking points: After you enable Stretch Database for a local server instance, a database, and at least one table, it silently begins to migrate your historical data to an Azure SQL Database. You can pause data migration to troubleshoot problems on the local server or to maximize the available network bandwidth. Stretch Database ensures that no data is lost if a failure occurs during migration. It also has retry logic to handle connection issues that may occur during migration. A dynamic management view provides the status of migration. You don't have to change existing queries and client apps. You continue to have seamless access to both local and remote data, even during data migration. There is a small amount of latency for remote queries, but you only encounter this latency when you query the historical data that's archived remotely. You can use Stretch Database Advisor, a feature of SQL Server 2016 Upgrade Advisor, to identify databases and tables for Stretch Database. Stretch Database Advisor helps you to adopt Stretch Database by analyzing existing database tables based on adjustable table size thresholds to identify likely candidates. Stretch Database Advisor also identifies blocking issues. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

How Stretch Database works
Creates a secure linked server definition in the on-premises SQL Server Targets linked server definition as the remote endpoint Provisions remote resources and begins to migrate eligible data, if migration is enabled Queries against tables run for both local database and remote endpoint On-premises instance Azure Internet boundary Linked servers Remote Endpoint Remote Data Local Database Eligible Data Local Data Objective: This slide shows the architecture of Stretch Database and how its components work. This slide also introduces some new terms that must be understood before diving into the architecture of Stretch Database. Talking points: Here are the concepts and architecture for Stretch Database. Terms Local database: The on-premises SQL Server 2016 database. Remote endpoint: The location in Microsoft Azure that contains the database’s remote data. In SQL Server 2016, this is an Azure SQL Database. This is subject to change in the future. Local data: Data in a database with Stretch Database enabled that will not be moved to Azure based on the Stretch Database configuration of the tables in the database. Eligible data: Data in a database with Stretch Database enabled that has not yet been moved, but will be moved to Azure based on the Stretch Database configuration of the tables in the database. Remote data: Data in a database with Stretch Database enabled that has already been moved to Azure. Architecture Stretch Database leverages the resources in Microsoft Azure to offload archival data storage and query processing. When you enable Stretch Database on a database, it creates a secure linked server definition in the on- premises SQL Server. This linked server definition has the remote endpoint as the target. When you enable Stretch Database on a table in the database, it provisions remote resources and begins to migrate eligible data, if migration is enabled. Queries against tables with Stretch Database enabled automatically run against both the local database and the remote endpoint. Stretch Database leverages processing power in Azure to run queries against remote data by rewriting the query. You can see this rewriting as a "remote query" operator in the new query plan. Source:

Work without disruption
12/2/2017 1:49 AM Work without disruption Business applications continue working without disruption Database administrator (DBA) scripts and tools work as before; all controls still held in local SQL Server Developers continue building or enhancing applications with existing tools and methods Trickle migration Orders Orders History Objective: With SQL Server 2016 Stretch Tables, you can stretch large operational tables from on-premises to Azure with the ability to query with near-infinite capacity. This slide depicts that you don't have to change existing queries and client apps to work with Stretch Database. Talking points: With Stretch Database, you continue to have seamless access to both local and remote data, even during data migration. Once the selection is made, trickle data migration is used to move that data to Azure. The data can then be returned to on-premises storage. The data exchange is both transparent and bi-directional. During the stretching process, all the characteristics of the database stay intact. The code or stored procedures do not change. The user access control does not change either. This maintains the integrity of the data, while still enabling staff to work with it. Applications continue to work without code changes or any disruption. Existing database administrator (DBA) skills and processes remain relevant; the scripts and tools work as before; all controls are still held in local SQL Server. Developers can continue using current tools and APIs. They can continue building or enhancing applications with existing tools and methods. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

What is Always Encrypted?
12/2/2017 What is Always Encrypted? Prevention of data disclosure Client-side encryption of sensitive data using keys that are never given to database system Queries on encrypted data Support for equality comparison, including join, group by, and distinct operators Application transparency Minimal application changes through server and client library enhancements Allows customers to securely store sensitive data outside of their trust boundary while protecting data from highly privileged (yet unauthorized) users Objective: This slide depicts the need for Always Encrypted and the purpose of the feature in SQL Server 2016. Talking points: Always Encrypted is a feature designed to protect sensitive data, such as credit card numbers or national identification numbers (for example, U.S. Social Security numbers) stored in Azure SQL Database or SQL Server databases. Always Encrypted allows clients to encrypt sensitive data inside client applications and never reveal the encryption keys to the database engine (SQL Database or SQL Server). As a result, Always Encrypted provides a separation between those who own the data (and can view it) and those who manage the data (but should have no access). In addition, you can query encrypted data and perform various query operations like equality comparison, including join, group by, and distinct operators. Always Encrypted allows clients to encrypt sensitive data inside client applications and never reveal the encryption keys to the SQL Server. An Always Encrypted-enabled driver installed on the client computer achieves this by automatically encrypting and decrypting sensitive data in the SQL Server client application. The driver encrypts the data in sensitive columns before passing the data to SQL Server, and automatically rewrites queries to preserve the semantics to the application. Similarly, the driver transparently decrypts data stored in encrypted database columns that are contained in query results. This technology ensures that your data is constantly encrypted. Best of all, no application changes are required. © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Server & Tools Business
12/2/2017 What is Always Encrypted? Capability ADO.NET client library provides transparent client-side encryption, while SQL Server executes T-SQL queries on encrypted data Benefits Sensitive data remains encrypted and queryable at all times, on-premises and in the cloud Unauthorized users never have access to data or keys No application changes Data remains encrypted during query Apps TCE-enabled ADO.NET SQL Server Encrypted query No app changes Master key Columnar key Objective: This slide summarizes Always Encrypted and its benefits. Talking points: Always Encrypted makes encryption transparent to applications. An Always Encrypted-enabled driver installed on the client’s computer achieves this by automatically encrypting and decrypting sensitive data in the client application. The architecture for Always Encrypted has the application performing the column- level encrypting prior to the confidential columns being sent to SQL Server. The actual encryption is done by the ADO.NET drivers on an application, or client machine. When a .NET application sends plain text data to ADO.NET, it’s encrypted prior to sending it to SQL Server. The only change to storing encrypted data that the application needs to make is to change the connection string to indicate column encryption is enabled. When column encryption is enabled, ADO.NET will encrypt Always Encrypted columns prior to sending the data to SQL Server, and will decrypt Always Encrypted columns when they are read from SQL Server. The diagram on the slide shows this architecture. Benefits: When it comes to mission-critical security, we have a unique encryption technology that protects data at rest and in motion, allowing data to be fully queried while encrypted. The new ADO.NET library provides transparent, client-side encryption, while SQL Server executes T-SQL queries on encrypted data. The master keys stay with the application and not with the SQL Server. This can work on-premises or via SQL Server in Azure Virtual Machines. So think about the hybrid scenarios in which you want to take advantage of Azure cloud computing, keeping in mind that certain data cannot take advantage of cloud scale due to data security requirements. Always Encrypted allows organizations to encrypt data at rest and in use for storage in Azure, in order to enable delegation of on-premises database administration to third parties or reduce security clearance requirements for their own DBA staff. As a result, Always Encrypted provides a separation between those who own the data (and can view it) and those who manage the data (but should have no access). © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

How does Always Encrypted work?
12/2/2017 1:49 AM How does Always Encrypted work? Encrypted sensitive data and corresponding keys are never seen in plain text in SQL Server SQL Server or SQL Database "SELECT Name FROM Customers WHERE SSN " " ADO.NET "SELECT Name FROM Customers WHERE SSN 0x7ff654ae6d Ciphertext Result set Result set Objective: This slide shows how Always Encrypted works. Talking points: The Always Encrypted-enabled driver installed on the client computer automatically encrypts and decrypts sensitive data in the SQL Server client application. The driver encrypts the data in sensitive columns before passing the data to SQL Server, and automatically rewrites queries to preserve the semantics to the application. Similarly, the driver transparently decrypts data, which is stored in encrypted database columns and contained in query results. Here is how these features work: Users specify individual columns of particular tables to be encrypted. Once encrypted, data appears as an encrypted binary blob at all stages within the SQL Server database— on disk, in memory, during computations, and over the network. Users employ a certificate store to save the encryption key. Both encryption and decryption are done by the ADO.NET SqlClient driver for .Net 4.6. This driver will require access to the encryption key (via the certificate store, generally) and thereafter will communicate with the SQL Server directly to effect transparent encryption. Specifically, when queries are parameterized, SqlClient will handshake with the SQL Server and identify which parameters are encrypted and will manage this process in both directions. Example: the client code specifies a select statement with the parameter “where SSN and provides the parameter value and the driver itself intercepts the parameter value and properly encrypts it. Animation <<first click>> “SELECT Name FROM Customers WHERE SSN “ ” with arrow Animation <<second click>> “SELECT Name FROM Customers WHERE SSN encrypted with arrow Animation <<third click>> Result Set encrypted with arrow Animation <<fourth click>> Result Set with arrow Name Wayne Jefferson Name 0x19ca706fbd9a Name SSN Country 0x19ca706fbd9a 0x7ff654ae6d USA Ciphertext © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Key provisioning 1. Generate CEKs and master key 2. Encrypt CEK
Column encryption key (CEK) Column master key (CMK) 2. Encrypt CEK Encrypted CEK 3. Store master key securely CMK store: Certificate store HSM Azure Key Vault Objective: This slide highlights the key provisioning steps in Always Encrypted. Talking points: Several core concepts are used in Always Encrypted: Column master key (CMK): Encryption key that protects the column encryption key(s). You must have at least one master key before encrypting any columns. Column encryption key (CEK): Encryption key that actually protects encrypted columns. CMK store provider: Client-side software component that encapsulates a key store containing the column master key. Providers for common types of key stores are available in client-side driver libraries from Microsoft or as standalone downloads. You can also implement your own provider. Note that the .NET Framework 4.6 must be installed in the machine hosting your client application. The .NET Framework 4.6 is available with SQL Server 2016 Community Technology Preview 3 (CTP 3.0) and is installed with SQL Server Management Studio. Included below is information the driver receives from SQL Server for query parameters that need to be encrypted and for query results that need to be decrypted: An encrypted value of a CEK, used to encrypt or decrypt a parameter or result. The name of a key store provider that encapsulates a key store containing the column master key that was used to encrypt the CEK. A key path that specifies the location of the CMK in the key store. The name of the algorithm that was used to encrypt the CMK. Here are the steps for key provisioning: Create a local, self-signed certificate on the development machine, which will act as a CMK. Animation <<first click>> The CMK will be used to protect CEKs, which encrypts the sensitive data. Animation <<second click>> Create a CMK store definition object in the database, which will store the information about the location of the CMK. Note that the certificate will never be copied to the database or SQL Server machine. Animation <<third click>> Finally, upload the encrypted CEK to the required database. Animation <<fourth click>> CMK Security officer 4. Upload encrypted CEK to DB Encrypted CEK database

1 2 Query performance tools Live Query Statistics (LQS) Query Store
12/2/2017 Query performance tools 1 2 Live Query Statistics (LQS) Query Store © 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

How Query Store works SQL Query Store
Durability latency controlled by DB option DATA_FLUSH_INTERNAL_SECONDS Query Store Async write-back Compile Execute SQL Plan store Runtime stats Query Store schema Compile MSG Execute MSG Collects query texts (plus all relevant properties) Stores all plan choices and performance metrics Works across restarts/upgrades/recompiles Lowers bar for performance troubleshooting Allows new views Offers intuitive and easy plan forcing Objective: This slide discusses how Query Store works and also explains the architecture of this feature. Talking points: Here we’ll take a look at how Query Store works. How it works Query Store collects query texts and all relevant properties, as well as query plan choices and performance metrics. This collection process works across restarts or upgrades of the server and across recompilation of indexes. It provides many configurable options for customization. Query Store integrates with existing query execution statistics, plan forcing, and manageability tools. It is a dedicated store for query workload performance data and captures plan history for each query. Query Store can be viewed and managed through Management Studio or by using views and procedures. Seven Query Store catalog views can present information on Query Store. Query Store also has a rich user- interface component that uses several of the DMVs. Architecture diagram of Query Store Every time SQL Server compiles or executes a query, a message is sent to the Query Store. This compile and execution information is kept in memory first and then saved to disk, depending on the Query Store configuration. (Data is aggregated according to the INTERVAL_LENGTH_MINUTES parameter, which defaults to one hour, and flushed to disk according to the DATA_FLUSH_INTERVAL_SECONDS parameter.) The data can also be flushed to disk if there is memory pressure on the system. In any case, you can access all data, both in memory and disk, when you run the sys.query_store_runtime_stats catalog.

Live Query Statistics Collect actual metrics about query while running
12/2/2017 1:49 AM Live Query Statistics Collect actual metrics about query while running To view live query execution plan, from Tools menu, click Live Query Statistics Using LQS, you can view CPU/memory usage, execution time, query progress, and so on Enables rapid identification of potential bottlenecks for troubleshooting query performance issues Allows drill-down to live operator level statistics: Number of generated rows Elapsed time Operator progress Live warnings Objective: This slide presents the overview of Live Query Statistics (LQS), a new feature introduced in SQL Server 2016, and how to view the live execution plan of an active query using LQS. Talking points: SQL Server 2016 has a new feature called Live Query Statistics (LQS) that allows you to view what is happening during query execution. To view the live query execution plan of any running query, simply click the Live Query Statistics icon under the tools menu. LQS lets you view a list of active queries and associated statistics such as current CPU/memory usage, execution time, query progress, and so on. LQS enables rapid identification of potential bottlenecks for troubleshooting query performance issues. LQS also allows you to drill down into a query plan of an active query, and view live operator-level statistics such as the number of generated rows, elapsed time, operator progress, and live warnings. This facilitates in-depth troubleshooting of query performance issues, without having to wait for query completion. You can watch statistics change in real time during query execution. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Monitoring query performance
Query Store feature provides DBAs with insight on query plan choice and performance Regressed Queries pane shows queries and plans in Query Store Drop-down boxes at top allow you to select queries based on various criteria Objective: This slide emphasizes that Query Store comes with an extraordinary UI that will help a broad set of users immediately benefit from collected performance data. Talking points: Query Store also has a rich user-interface component that uses several DMVs (see screenshot). Familiarizing yourself with the UI can be an excellent introduction to the kind of information you can obtain by querying DMVs. SSMS is focused around a handful of the most important scenarios, making this feature instantly useful in the typical DBA’s everyday activities. Microsoft encourages people to try out new UI and learn from it: SSMS is a great knowledge source because people can easily learn the first steps of using Query Store DMVs by analyzing queries generated by SSMS. Example: After enabling Query Store, refresh the database portion of the Object Explorer pane to add the Query Store section. Selecting Regressed Queries opens the Regressed Queries pane in Management Studio. This pane shows you queries and plans in Query Store. Drop-down boxes at the top allow you to select queries based on various criteria. Select a plan to see the graphical query plan. Buttons are available to view the source query, force and unforce a query plan, and refresh the display. To force a plan, select a query and plan, and then click Force Plan. You can only force plans that were saved by the query plan feature and are still retained in the query plan cache.

Unique, innovative technology
12/2/2017 1:49 AM Why PolyBase? Component of PDW region in APS Transparent Database Encryption (TDE) and Always Encrypted Highly parallelized, distributed query engine accessing heterogeneous data via SQL Objective: This slide shows the competitive advantages of PolyBase. Talking points: The main competitive advantages of PolyBase include the following: Animation <<first click>> PolyBase, the component of the PDW region in the Analytic Platform System, is now built into SQL Server, expanding the power to extract value from unstructured and structured data using your existing T-SQL skills. PolyBase also supports Transparent Database Encryption (TDE) and Always Encrypted. PolyBase addresses one of the main customer pain points in data warehousing: accessing distributed data sets. It provides a highly parallelized, distributed query engine accessing heterogeneous data via SQL. PolyBase integrates seamlessly with all BI tools. Use it with the Microsoft BI products, such as SQL Server Reporting Services, SQL Server Analysis Services, PowerPivot, PowerQuery, and PowerView. Or use third- party BI tools, such as Tableau, Microstrategy, or Cognos. PolyBase is a unique, innovative technology that does not require additional software installed on the user’s Hadoop or Azure environment. It is fully transparent for the end user—no knowledge about Hadoop or Azure is required to use PolyBase successfully. Unique, innovative technology Seamless integration © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

How it works External table User perspective Systems perspective
External data source External file format Systems perspective Objective: This slide describes how PolyBase works. Talking points: Setting up and configuring PolyBase, as well as connecting to your Hadoop environment, is a multi-step process. Animation <<first click>> From a user perspective: First, establish the SQL Server-to-Hadoop connection. Next, configure your external data source, external file formats, and finally, the external table. Working with external data is generally a three-step process: Defining the external data sources, such as those on Windows Azure Blob Storage, or in Hadoop (either in Microsoft HDInsight or external Hadoop); passing in security credentials. Defining the external file/data format (delimited text files, Hive RC, Hive ORC, or Parquet). Defining an external table. Supported data definition language (DDL) statements include CREATE/DROP, ALTER, and CREATE STATISTICS. Animation <<second click>> From a systems perspective: PolyBase acts as a PDW engine that bridges the traditional relational database structures held in SQL Server with the massive amounts of data held in Hadoop. It is real time in that once you set up your Hadoop data set within SQL Server, you can query it using T-SQL (even to the point where you are joining relational tables with Hadoop tables). PDW engine service Bridge Microsoft Azure

Server & Tools Business
12/2/2017 PolyBase benefits Query relational and non-relational data with T-SQL Capability T-SQL for querying relational and non-relational data across Microsoft SQL Server and Hadoop Benefits New business insights across your Azure Data Lake Leveraging of existing skill sets and BI tools Faster time to insights and simplified ETL process Query relational and non-relational data, on-premises and in Azure T-SQL query SQL Server Hadoop Objective: This slide provides a summary of PolyBase’s capability and benefits. Talking points: When it comes to key business intelligence (BI) investments, PolyBase makes it much easier to manage relational and non-relational data, which allows you to query Hadoop data and SQL Server relational data through a single T-SQL query. Benefits: PolyBase does allow you to move data in a hybrid scenario. This ties into the concept of a data lake. A data lake can be thought of as providing full access to raw big data without moving it. This may be viewed as an alternate approach to processing big data to make its analysis easier, and then moving/synchronizing it in a data warehouse. SQL Server 2016 and PolyBase can be important components in setting up a data lake, combining it with your relational data, and doing analysis and BI on it. One of the challenges with Hadoop is that few people have Hadoop and MapReduce skill sets. This technology simplifies the skill set needed to manage Hadoop data. This can also work across your on- premises environment or SQL Server running in Azure. PolyBase also provides the ability to access and query data that is either on-premises or in the cloud and run analytics and BI on that data. PolyBase therefore can help you build out a solution that delivers deeper insights into your data, wherever it may be located. There is no need for a separate ETL or import tool. Apps © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

PolyBase view in SQL Server 2016
12/2/2017 1:49 AM PolyBase view in SQL Server 2016 Allow SQL Server 2016 customers to execute T-SQL queries against relational data in SQL Server and “semi-structured” data in HDFS and Azure Leverage existing T-SQL skills and BI tools to gain insights from different data stores Expand reach of SQL Server to Hadoop (HDFS) Query Results SQL Server Objective: This slide shows how PolyBase can be viewed in SQL Server Talking points: Customers will continue to invest in RDBMS systems. Customers would like to leverage existing T-SQL skills and BI tools to gain insights from different data stores. PolyBase allows users to query non-relational data in Hadoop, blobs, files, and combine it anytime, anywhere with their existing relational data in SQL Server. Hadoop Azure Blob Storage © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Capabilities and functions
12/2/2017 1:49 AM Capabilities and functions Users Queries Results PolyBase Hadoop Azure Blob Storage Use T-SQL statements to access and query data in Hadoop or Azure Blob Storage Use T-SQL to store data from Hadoop or Azure Blob Storage Export data to Hadoop or Azure Blob Storage Objective: This slide discusses the high-level capabilities and functions of PolyBase. Talking points: PolyBase allows you to use T-SQL statements to access data stored in Hadoop or Azure Blob Storage and query it in an ad hoc fashion. For this purpose, it introduces three main T-SQL constructs fully integrated in SQL Server as first-class SQL objects: external file formats, external data sources, and external tables. PolyBase allows you to use T-SQL to store data originating in Hadoop or Azure Blob Storage as regular tables. Users and applications can leverage SQL Server’s mature features, such as columnstore technology, for example, for frequent BI reports. There is no need for a separate ETL or import tool. PolyBase allows you to export data to Hadoop or Azure Blob Storage. You can leverage Hadoop or Azure Blob Storage for cold but queryable data stores. Other important capabilities of PolyBase are: PolyBase integrates seamlessly with Microsoft BI tools like SQL Server Reporting Services, SQL Server Analysis Services, PowerPivot, PowerQuery, and PowerView. Or, organizations can use third-party BI tools such as Tableau, Microstrategy, and Cognos. PolyBase does not require any additional software to be installed on the user’s Hadoop or Azure environment. It is fully transparent to the end user—no knowledge about Hadoop or Azure is required to use PolyBase successfully. © 2015 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Need for Row-Level Security (RLS) Collect actual metrics about query while running
Enable fine-grained access control over specific rows in database table Help prevent unauthorized access when multiple users share same tables, or implement connection filtering in multi-tenant applications Administer via SQL Server Management Studio or SQL Server Data Tools Use enforcement logic inside database and schema bound to table SQL Database Customer 1 Customer 2 Customer 3 Objective: This slide talks about the need for high-level security and access control features in today’s multi- tenant environment and how SQL Server 2016 Row-Level Security can help provide this. Talking points: Today, it’s common to have multi-tenant software-as-a-service architecture. This means we have to store data for multiple customers and protect their data from each other, even if it is in the same table. For this, we need to build our own access control system within the database and application, and spend time implementing, maintaining, and troubleshooting the systems and logics. In this situation, it would be great if there were some sort of high-level security feature in SQL Server that could help implement row granularity access control without having to write and debug all of that code. SQL Server is in the clear leadership position when it comes to security for mission-critical applications. In this regard, SQL Server 2016 Row-Level Security (RLS) enables developers and DBAs to implement fine- grained access control over rows in a table. Using RLS, you can store data for different customers, departments, or tenants in the same table, while restricting access to rows based on a query’s execution context. For example, you could filter rows returned by “SELECT * FROM myTable” according to the identity of the logged-on user or the value of a session-scoped variable like CONTEXT_INFO. Administrators or DBAs can easily access and manage RLS through SQL Server Management Studio or SQL Server Data Tools. RLS works transparently at query time, with no application changes required. It uses a centralized security logic that resides inside the database and is schema-bound to the table it protects, providing greater security. The database system applies the access restrictions every time data access is attempted from any tier. This makes the security system more reliable and robust by reducing its surface area.

Data dependent routing APIs connect to database
12/2/2017 1:49 AM RLS introduction Tenant 1 Tenant 2 Tenant 3 Tenant 4 Shard 1 Shard 2 Security Policy BlogID Name URL Tenant … 1 3 2 4 Row-Level Security filters based on CONTEXT_INFO Client App Objective: This slide introduces the Row-Level Security feature and how it is implemented to ensure fine- grained control over access to rows in a table. Talking points: SQL Server 2016 has introduced Row-Level Security (RLS), which is a feature that enables fine-grained control over access to rows in a table. RLS allows you to easily control which users can access which data with complete transparency to the application. This enables you to easily restrict the data based on user identity or security context in multi-tenant environments, where you may want to limit data access based on customer ID. RLS simplifies the design and coding of security in your application. It enables you to implement restrictions on data row access. For example, you can ensure that workers can access only those data rows that are pertinent to their department, or you can restrict a customer’s data access to only the data relevant to his or her company. With this feature, rows are filtered based on the execution context of the query from the client application rather than the current user access rights. You can set the CONTEXT_INFO value to the user’s application- specific user ID whenever a connection is opened. The CONTEXT_INFO value can then be referenced in the security function. A secure logic can be created to determine which user can see which rows and restrict any kind of data (rows) by designing a flexible and robust security policy for a table. (For more on the CONTEXT_INFO value, visit For example, imagine a function allows hospital staff to access rows in a patient table only where there is a match between the staff member’s assigned hospital wings and the dates she was assigned to each wing. RLS will allow the hospital to create a security policy that binds the search function to one or more tables. Once bound to the table, all access is routed through the security policy. So a staff member who queries patients would only see those patients who were in her wing during the time she was assigned to that location. Data dependent routing APIs connect to database RLS restricts which users can view which data in a table, based on a function. SQL Server 2016 introduces this feature, which is useful in multi-tenant environments where you may want to limit data access based on customer ID. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

RLS in three steps Database Nurse Policy manager Security Policy
Microsoft Research 2013 12/2/2017 1:49 AM RLS in three steps Nurse Policy manager Database Filter Predicate: INNER JOIN… Security Policy Application CREATE FUNCTION int) RETURNS TABLE WITH SCHEMABINDING AS return SELECT 1 as [fn_securitypredicate_result] FROM StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId) WHERE e.UserSID = SUSER_SID() = d.Wing; CREATE SECURITY POLICY dbo.SecPol ADD FILTER PREDICATE dbo.fn_securitypredicate(Wing) ON Patients WITH (STATE = ON) Patients SELECT * FROM Patients SEMIJOIN APPLY dbo.fn_securitypredicate(patients.Wing); SELECT * FROM Patients Objective: This slide walks through the three simple steps of how RLS works. Talking points: This example scenario illustrates the three steps to understanding how RLS works. Policy manager creates a filter predicate and security policy. App user (for example, a nurse) selects from the Patients table. The query is transparently rewritten to apply the filter predicate. Note: No app changes are required for RLS to work. Now let’s see each step in detail: Animation <<first click>> Policy manager creates a filter predicate and security policy. In RLS, the policy manager creates a filter predicate function that encapsulates the access logic and security policy. In this example, he creates a function that allows staff to access rows in a patient table only where there is a match between the staff member’s assigned hospital wings and the duties IDs that were assigned to each wing with the patient’s wing and user ID: CREATE FUNCTION int) RETURNS TABLE WITH SCHEMABINDING AS return SELECT 1 as [fn_securitypredicate_result] FROM StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId) WHERE e.UserSID = SUSER_SID() = d.Wing; The policy manager then creates a security policy that binds the predicate function to one or more tables. Once bound to the table, all access is routed through the security policy: CREATE SECURITY POLICY dbo.SecPol ADD FILTER PREDICATE dbo.fn_securitypredicate(Wing) ON Patients WITH (STATE = ON) Animation <<second click>> App user (for example, a nurse) selects from the Patients table. In the second step, a staff member (such as a nurse) who runs SELECT * FROM patients would only see those patients who were in her wing during the time she was assigned to that location. In this example, the predicate function might use the logged-on user’s SUSER_ID to identify the rights of that user. In the context of middle- tier apps, where application users share the same SQL logon (for instance, via connection pooling), the application could specify the currently logged-on user using CONTEXT_INFO upon opening the database connection. Animation <<third click>> The query is transparently rewritten to apply the filter predicate. Finally, in the third step, RLS is implemented through the query optimizer. Once the security policy has been created, all queries on the table are automatically rewritten by the optimizer to apply the predicate function. For example, “SELECT * FROM myTable” might be rewritten as “SELECT * FROM myTable WHERE StaffId = SUSER_SID”. SELECT Patients.* FROM Patients, StaffDuties d INNER JOIN Employees e ON (d.EmpId = e.EmpId) WHERE e.UserSID = SUSER_SID() AND Patients.wing = d.Wing; Two App user (nurse) selects from patient’s table Three Security policy transparently rewrites query to apply filter predicate One Policy manager creates filter predicate and security policy in T-SQL, binding predicate to patient’s table © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Dynamic Data Masking Dynamic Data Masking Prevent abuse of sensitive data by hiding it from users Configuration made easy in new Azure portal Policy-driven at table and column level for defined set of users Dynamic Data Masking applied in real time to query results based on policy Multiple masking functions available (full, partial) for various sensitive data categories (like credit card numbers, SSN) Table.CreditCardNo SQL Database SQL Server 2016 Real-time data masking; partial masking Objective: This slide shows what Dynamic Data Masking is and what its high-level capabilities are. Talking points: SQL Database Dynamic Data Masking limits sensitive data exposure by masking it from non-privileged users, thereby preventing abuse of data by hiding it. Dynamic Data Masking is also supported for the V12 version of Azure SQL Database. You can easily configure a Dynamic Data Masking policy in the Azure portal by selecting the Dynamic Data Masking operation in your SQL Database configuration blade or settings blade. Dynamic Data Masking helps prevent unauthorized access to sensitive data by enabling you to designate how much of the data to reveal, with minimal impact on the application layer. It’s a policy-based security feature applied in real time to hide sensitive data in the result set of a query over designated database fields, while the data in the database is not changed. There are multiple masking functions and methods that control the exposure of data for different scenarios. For example, a service representative at a call center may identify callers by several digits of their Social Security number or credit card number, but those data items should not be fully exposed to the representative. A masking rule can be defined that masks all but the last four digits of any Social Security number or credit card number in the result set of any query. As another example, an appropriate data mask can be defined to protect personally identifiable information (PII) data, so that a developer can query production environments for troubleshooting purposes without violating compliance regulations. Dynamic Data Masking does not aim to prevent database users from connecting directly to the database and running exhaustive queries that expose pieces of sensitive data. Dynamic Data Masking is complementary to other SQL Server security features (including auditing, encryption, and Row-Level Security), and it is highly recommended that you use this feature in conjunction with them in order to better protect sensitive data in the database.

Defining Dynamic Data Masking
A masking rule may be defined on a column in a table in order to protect data in that column. Four types of masks are available. A masking rule may be defined on a column in a table, in order to protect data in that column. Four types of masks are available. Objective: This slide shows the various masking rules that you can define on a column in a table. Talking points: The purpose of Dynamic Data Masking is to limit the exposure of sensitive data, preventing users who should not have access to this data from viewing it. A masking rule may be defined on a column in a table in order to protect the data in that column. Four types of masks are available: Default: Full masking according to the data types of the designated field (for example, string will result in “XXXX”). Masking will expose the first letter of an address and will end in “.com” (for example, Custom String: Masking will expose the first and last letters and add a custom padding string in the middle (for example, KXXXa). Random: For use only with numeric. Masking will replace the original value within a specified range. © 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

SQL Server on-premises with cloud resources
Backup to Azure Storage Optionally managed Data files in Azure Storage using file snapshots On-premises Microsoft Azure Objective: This slide focuses on how Microsoft SQL Server on-premises can use cloud resources to provide the hybrid cloud scenario for backup and high availability. Talking points: SQL Server 2016 on-premises and Microsoft Azure create hybrid cloud scenarios that span from on-premises SQL Server to Microsoft Azure public cloud offerings: SQL Server in Microsoft Azure Virtual Machines and Azure Storage. Specifically, we’ll discuss the following scenarios: Animation <<first click>> Backup to Microsoft Azure Storage Backup to Microsoft Azure Storage blob One of the most fundamental administrator tasks is backing up and restoring databases. With SQL Server and Microsoft Azure, you can safely back up your databases in the cloud. You can optionally manage your backup scheduling by choosing how long you want the data retained. Also, SQL Server managed backup to Azure now supports custom schedules. Animation <<second click>> Data files in Azure Storage using file snapshots Store SQL Server data files in Microsoft Azure Storage Storing on-premises SQL Server data files in Microsoft Azure Storage provides flexible, reliable, and limitless off-site storage for your databases. SQL Server file snapshot backup uses Azure snapshots to provide nearly instantaneous backups and quicker restores for data files stored using the Azure Blob Storage service. This capability enables you to simplify your backup and restore policies. Animation <<third click>> AlwaysOn replica in Azure Virtual Machines Maintain database replicas on Microsoft Azure Virtual Machines Most customers need to configure a disaster recovery site and purchase additional hardware for database replicas. With SQL Server and Microsoft Azure, you can maintain one or more replicas of your databases in the cloud. You can maintain secondary replicas in Microsoft Azure by using the Add Azure Replica Wizard, which allows you to deploy one or more replicas of your databases to a virtual machine in Microsoft Azure for disaster recovery. Azure Storage AlwaysOn replica in Azure Virtual Machines Primary Secondary

Benefits of Azure Backup
Block blobs Reduced storage costs Significantly improved restore performance More granular control over Azure Storage Azure Storage snapshot backup Fastest method for creating backups and running restores Support of SQL Server database files on Azure Blob Storage Managed backup Granular control of the backup schedule Local staging for faster recovery and greater network resiliency System database support Simple recovery mode support Objective: This slide introduces SQL Server back up to and restore from the Microsoft Azure Blob Storage service. It also provides a summary of the benefits of using the Azure Blob Storage service to store SQL Server backups. Talking points: SQL Server supports storing backups to the Microsoft Azure Blob Storage service in the following ways: Backup to Azure block blobs Azure block blobs allow you to manage backups to Azure Blob Storage with fine-grained control over the process. SQL Server 2016 supports multiple blobs to enable backing up large databases—up to a maximum of TB. This provides the following benefits: Cost savings on storage. Almost all backup and restore features now available, including mirrors and stripes. Significantly improved restore performance. More granular control over Azure Storage. Backup to Azure with file snapshots SQL Server 2014 introduced data files in Microsoft Azure. This enables native support for SQL Server database files stored as Azure blobs. SQL Server 2016 builds on this capability with backup with file snapshots. Benefits of this method are as follows: Backing up with file snapshots provides the fastest and cheapest method for creating backups and running restores. Unlike backup to Azure with block blobs, data is not actually moved. Instead, when SQL Server database files are directly stored in Azure Storage, a snapshot of those files is created. You only need to run a full backup once to establish the backup chain. Managed backup With the same methods used to back up to DISK and TAPE, you can now back up to Microsoft Azure Storage by specifying a URL as the backup destination. You can use this feature to manually back up or configure your own backup strategy like you would for local storage or other off-site options. This feature is also referred to as SQL Server Backup to URL. The feature has been enhanced in the SQL Server 2016 Community Technology Preview 3 (CTP 3.0) to increase performance and functionality through the use of block blobs, Shared Access Signatures, and striping. Benefits of this method are as follows: Managed backup provides granular control of the backup schedule using customization. You choose only how long you want the data retained and where to store it, and SQL Server schedules, performs, and maintains the backups. Backups are made locally and then uploaded to the cloud, allowing for faster recovery and resiliency to transient network issues. Databases in the full, bulk logged, and simple recovery models are supported. System databases can be backed up with managed backups. SQL Server 2016 now supports the simple recovery model.

Mission-critical performance with Microsoft SQL Server 2016

Similar presentations

Presentation on theme: "Mission-critical performance with Microsoft SQL Server 2016"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Mission-critical performance with Microsoft SQL Server 2016

Similar presentations

Presentation on theme: "Mission-critical performance with Microsoft SQL Server 2016"— Presentation transcript:

Similar presentations

About project

Feedback