Everything You Need to Know About SQL Server Indexes

Everything You Need to Know About SQL Server Indexes
11/14/2018 Everything You Need to Know About SQL Server Indexes Janis Griffin Senior DBA / Performance Evangelist

© 2018 SolarWinds Worldwide, LLC. All rights reserved.
11/14/2018 Who Am I Senior DBA / Performance Evangelist for SolarWinds Twitter® Current – 25+ Years in Oracle®, DB2®, ASE, SQL Server®, MySQL® DBA and Developer Specialize in Performance Tuning Review Database Performance for Customers Common Question – How do I tune it? © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Agenda SQL Server Index Structures and Options Rowstore Indexes Understanding B-Tree Indexes Clustered Index vs. Heap NonClustered Indexes Covering Indexes Filtered Indexes Columnstore Indexes Clustered vs. NonClustered Indexes Indexes on Memory-Optimized Tables Hash Indexes Index Maintenance & Statistics © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 What is an Index Can be an optional object associated with a table or view Can be on one or more columns of a table Can be unique or non-unique values Unique keys help enforce data integrity Can speed up data retrieval Reduces disk I/O SQL Server on-disk or in-memory index types Clustered or nonclustered Filtered or covering Rowstore or columnstore Hash Memory-optimized nonclustered Beyond this presentations scope XML, spatial, and full-text indexes © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Rowstore Indexes B-tree structure (on-disk) Can have 0-n levels Provides efficient navigation Root level points to intermediate level There may be several intermediate levels Leaf contains pointer to data location Can contain a RID if it’s a HEAP 8 byte row-ID (fileID, pageID, slot number) Or if it has a rowstore clustered index Nonclusered index contains the cluster key 0..19 20..39 40..59 20..28 29..34 35..39 0..8 9..14 15..19 40..48 49..54 55..59 35,rid 36,rid … 39,rid 15,rid 16,rid 19,rid 55,rid 56,rid 59,rid 20,rid 21,rid 28,rid 0,rid 1,rid 8,rid 40,rid 41,rid 48,rid 29,rid 30,rid 34,rid 9,rid 10,rid 14,rid 49,rid 50,rid 54,rid Root Intermediate Leaf © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Rowstore Indexes Clustered Index Table data is ordered and stored by the cluster key Can only have one clustered key As data can only be stored in one order Can be used to quickly retrieve data If query uses the indexed columns Is used by nonclustered indexes to look up table data Root Intermediate Leaf Clustered Index Data © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Rowstore Indexes Clustered Index Best Practices Preferably unique or primary key That rarely changes To reduce fragmentation Is small or narrow in size Nonclustered indexes will also store the cluster key Choose columns that are frequently sorted Examples of good clustering keys Identity column or a composite key e.g. some_date, integer NEWSEQUENTIALID() function Generates a sequential GUID Fills data and index pages completely Root Intermediate 2 Leaf Clustered Index Data Intermediate 1 © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Rowstore Indexes Heap Is a table that doesn’t have a clustered index Rows are not sorted in any order SQL Server maintains an internal unique key Data access will be full table scans Unless you add nonclustered indexes Nonclustered Index Separate object in database Holds only the indexed column(s) or key If table has a clustered index The nonclustered index will contain the cluster key Don’t need to explicitly add the cluster key Otherwise, it’s a heap table so contains a RID HEAP Data © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Rowstore Indexes Root Intermediate Clustered Index Leaf Nonclustered index – cont. Usually much smaller than table Gives faster retrieval of a few rows Uses less memory / fewer disk I0s Data Root Intermediate Leaf Nonclustered Index HEAP Data © 2018 SolarWinds Worldwide, LLC. All rights reserved.

NonClustered Rowstore Index
11/14/2018 NonClustered Rowstore Index SQL Server implicitly adds the cluster key to nonclustered indexes Beware! Showplan amd sp_helpindex Won’t show that the cluster key is part of a non-clustered index You must use other third-party tools or sp_spaceused © 2018 SolarWinds Worldwide, LLC. All rights reserved.

NonClustered Rowstore Index – Cont.
11/14/2018 NonClustered Rowstore Index – Cont. NonClustered Index Only put the columns required by the query in the index SQL Server implicitly adds the cluster key to nonclustered indexes You must use other third-party tools or sp_spaceused © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Heap Example With nonclustered primary key Index Seek 192 rows affected CPU Time:0 Elapsed: 128ms Scan count 1 Logical reads 195 Physical reads 0 When to use heaps If no nonclustered indexes on heap A full table scan will occur Best used for tiny / temp tables Don’t Use If rows need to be sorted If ranges of data are needed © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Clustered Index Example
11/14/2018 Clustered Index Example AdventureWorks.Sales.SalesOrderDetail Index Seek 192 rows affected CPU Time:0 Elapsed: 128ms Scan count 1 Logical reads 9 Much better than 195 for heap Physical reads 0 © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Clustered Index Example – Cont.
11/14/2018 Clustered Index Example – Cont. Reference second column only Changes to Index Scan 1 row affected CPU time: 453ms, Elapsed: 1568ms Scan count 1 Logical reads 64,145 Physical reads 2 Create missing index On SalesOrderDetailID CREATE NONCLUSTERED INDEX <Name of Missing Index,sysname,> ON Sales.SalesOrderDetail (SalesOrderDetailId) © 2018 SolarWinds Worldwide, LLC. All rights reserved.

NonClustered Index Example
11/14/2018 NonClustered Index Example Add nonclustered index On SalesOrderDetailID 1 row affected CPU time: 0ms CI time: 453ms Elapsed: 273ms CI time: 1568ms Scan count 1 Logical 6 CI: 64,145 Physical 5 CI: 2 © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Other Index Types to Consider
11/14/2018 Other Index Types to Consider Adding indexes is not always the right thing to do! Consider Insert, Update & Delete (DML) activity Takes up storage Can hurt performance Missing or stale statistics Index too wide or not selective enough Covering index CREATE NONCLUSTERED INDEX cIX_OrderHeader_OnlineOrderFlag ON Sales.OrderHeader(OnlineOrderFlag) INCLUDE (OrderID,OrderDate,CustomerID,SubTotal) © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Other Index Types to Consider – Cont.
11/14/2018 Other Index Types to Consider – Cont. Filtered index CREATE NONCLUSTERED INDEX FIX_OrderHeader_OnlineOrderFlag ON Sales.OrderHeader(OnlineOrderFlag) WHERE OnlineOrderFlag = 1 Combination CREATE NONCLUSTERED INDEX FcIX_OrderHeader_OnlineOrderFlag INCLUDE (OrderID,OrderDate,CustomerID,SubTotal) © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Covering Index With INCLUDE
11/14/2018 Covering Index With INCLUDE Nonclustered index Includes all columns from table in the query Reduces IO Doesn’t have to do a table lookup Can help with update performance and deadlocks Include columns are stored at the leaf level Keeps the index narrow so navigation is still efficient DML / maintenance is quicker than adding to the actual index Can hurt performance if too wide Bulks inserts on very large tables Lots of inserts, updates, merges, and deletes 1 row affected CPU time: 0ms Elapsed: 49ms (NCI: 273ms) Scan count 1 Logical (NCI: 6) Physical (NCI: 5) © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Better Index Example – Customer Lookup Query
11/14/2018 Better Index Example – Customer Lookup Query © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Covering Index Example
11/14/2018 Covering Index Example Add Covering Index on SalesOrderDetail Vs. 2287ms For Clustered Scan © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Covering Index Example
11/14/2018 Covering Index Example Add Covering Index on SalesOrderHeader? CREATE NONCLUSTERED INDEX CIX_SalesOrderHeader_OnlineOrderFlag ON Sales.SalesOrderHeader(OnlineOrderFlag) INCLUDE (SalesOrderID,OrderDate,CustomerID,SubTotal); Vs. 423ms, LIOs With SOD CIX © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Customer Lookup Results
11/14/2018 Customer Lookup Results Table Size Timings Metric Clustered Index Only Covering Index – SOD Covering Index – SOH SOH LIO 19513 19386 SOH PIO 44 11 SOD LIO 64145 19 SOD PIO 2 CPU Time 594ms 47ms 31ms Elapsed Time 2287ms 423ms 305ms © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Filtered Index Works well on selective filters predicates Small subset of full table index Can improve performance and plan quality Filtered statistics are more accurate than full table statistics Reduce index maintenance and storage cost Filtered index is maintained only when data in the index change Small index = less storage Ideal columns for filtered indexes Categories containing small values (Status, Flag or Bit, etc…) Columns with distinct ranges of values (Dept_id, Warehouse_id, etc…) Columns that have mostly NULL values E.G. Ship_date, Saleperson_id, End_date, etc… © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Filtered Index – Cont. Can’t use filter indexes When queries return computed columns When query contains complex predicate expressions (e.g. LIKE operator) Review required SET options for filtered indexes Filtered Set Options - sql-server-2017 Tip: Review current settings in dm_exec_sessions Let’s add a filtered index to SalesOrderHeader set ansi_nulls on set ansi_padding on set ansi_warnings on set arithabort on set concat_null_yields_null on set numeric_roundabort off set quoted_identifier on CREATE NONCLUSTERED INDEX FIX_OrderHeader_OnlineOrderFlag ON Sales.OrderHeader(OnlineOrderFlag) WHERE OnlineOrderFlag = 1; © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Filtered Index Example
11/14/2018 Filtered Index Example Why didn’t the optimizer use it? Data Skew © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Filtered & Covering Index Example
11/14/2018 Filtered & Covering Index Example CREATE NONCLUSTERED INDEX FCIX_OrderHeader_OnlineOrderFlag ON Sales.OrderHeader(OnlineOrderFlag) INCLUDE (SalesOrderID,OrderDate, CustomerID,SubTotal) WHERE OnlineOrderFlag = 1 ; Timing almost exactly like the covered index © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Filtered Index Need a well-defined subset of data CREATE NONCLUSTERED INDEX FCIX_OrderHeader_OnlineOrderFlag ON Sales.OrderHeader(OnlineOrderFlag) INCLUDE (SalesOrderID,OrderDate, CustomerID,SubTotal) WHERE OnlineOrderFlag = 0; © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Data Compression for Rowstore Tables & Indexes
11/14/2018 Data Compression for Rowstore Tables & Indexes Compression helps reduce size of the database Can improve performance of I/O intensive workloads Data is stored in fewer pages Compression is not available in all editions Applied to SQL Server 2016 (13.x) SP1 and up Trade offs or limitations: Extra CPU resources are needed to compress and decompress the data Data-compression/data-compression?view=sql-server-2017 Example of filtered, covering index with compression = row CREATE NONCLUSTERED INDEX FCIX_SalesOrderHeader_OnlineOrderFlag ON Sales.SalesOrderHeader(OnlineOrderFlag) INCLUDE (SalesOrderID,OrderDate,CustomerID,SubTotal) WHERE OnlineOrderFlag = 1 WITH (DATA_COMPRESSION = ROW); Row compression is defined here. The simple explanation is that it take fixed-length columns and makes them variable length, adding additional bytes for the overhead of tracking the changes being made. The link provided has a table that references the savings for the datatypes used. It's interesting reading. And by "interesting," I mean "it won't matter for my summary later." Prefix and dictionary compression come next, as part of page compression. You can read all the details here. The simple explanation is that the process looks for repeated patterns. First it looks for repeated values by column (that's the prefix part), then it looks for patterns on the entire page (that's the dictionary part). The link provided has the details and is more interesting than the row compression article for reasons that will be apparent soon-ish, I promise. When to use page compression:A more detailed approach to deciding what to compress involves analyzing the workload characteristics for each table and index. It is based on the following two metrics:U: The percentage of update operations on a specific table, index, or partition, relative to total operations on that object. The lower the value of U (that is, the table, index, or partition is infrequently updated), the better candidate it is for page compression. S: The percentage of scan operations on a table, index, or partition, relative to total operations on that object. The higher the value of S (that is, the table, index, or partition is mostly scanned), the better candidate it is for page compression. Row compression Prefix compression Dictionary compression Page Compression Update % Low / Scan % High © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Index Size Index size and number of rows SELECT si.index_id, si.name as index_name, si.fill_factor,si.is_primary_key, ps.reserved_page_count * 8./1024. as reserved_MB, ps.lob_reserved_page_count * 8./1024. lob_reserved_MB, ps.row_count FROM sys.dm_db_partition_stats ps JOIN sys.indexes si on ps.object_id=si.object_id and ps.index_id=si.index_id JOIN sys.objects so on si.object_id=so.object_id JOIN sys.schemas sc on so.schema_id=sc.schema_id WHERE sc.name='SALES' and so.name='SalesOrderHeader'; © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Data Compression Queries
11/14/2018 Data Compression Queries SP for compression savings For SalesOrderHeader table sp_estimate_data_compression_savings @schema_name = 'schema_name' , @object_name = 'object_name' , @index_id = index_id , @partition_number = partition_number , @data_compression = 'data_compression' ; 130m savings © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Columnstore Indexes Good for analytical queries (DW) Columnar structure Can be clustered or nonclustered Clustered columnstore index is the primary storage for table Nonclustered columnstore index is on rowstore table Good for analytical queries on OLTP data Rowgroups Group of rows Compressed into columnstore format Maximum number of rows per rowgroup 1,048,576 rows © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Columnstore Indexes – Cont.
11/14/2018 Columnstore Indexes – Cont. Column Segment Column of data within the rowgroup Each segment is compressed together Stored on physical media Up to 10x query performance Batch mode execution Or vector-based processing Processes multiple rows together Can improve performance By 2 to 4 times © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Why Use Columnstore Indexes
11/14/2018 Why Use Columnstore Indexes High level of data compression Up to 7-10 times Smaller in-memory footprint Great for queries That only get few columns Reduces physical I/O That have large fact tables Use clustered columnstore index Real-time analytics Use nonclustered columnstore indexes CREATE NONCLUSTERED COLUMNSTORE INDEX csi_SalesOrderDetail ON Sales.SalesOrderDetail (SalesOrderID, SalesOrderDetailID, ProductID,OrderQty); Almost 5x Space Savings © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Nonclustered Columnstore Index Example
11/14/2018 Nonclustered Columnstore Index Example Rowstore Covering index On SaleOrderDetail CREATE NONCLUSTERED INDEX CIX_SalesOrderDetail_ProductID ON Sales.SalesOrderDetail (ProductID) INCLUDE (SalesOrderId, OrderQty); © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Nonclustered Columnstore Index Example – Cont.
11/14/2018 Nonclustered Columnstore Index Example – Cont. Vs. 355ms For covering Columnstore Indexes are stored in LOBs After compression Create statement CREATE NONCLUSTERED COLUMNSTORE INDEX csi_SalesOrderDetail ON Sales.SalesOrderDetail (SalesOrderID, SalesOrderDetailID, ProductID, OrderQty); © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Nonclustered Columnstore Index Example – Cont.
11/14/2018 Nonclustered Columnstore Index Example – Cont. Put a columnstore index on SalesOrderHeader? CREATE NONCLUSTERED COLUMNSTORE INDEX csi_SalesOrderHeader ON Sales.SalesOrderHeader (SalesOrderID,CustomerID,TotalDue) Computed Column TotalDue = (isnull((SubTotal+TaxAmt)+Freight,(0))) Msg 35307, Level 16, State 1, Line 10 The statement failed because column 'TotalDue' on table 'SalesOrderHeader' is a computed column. Columnstore index cannot include a computed column implicitly or explicitly. © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Another Example of a Nonclustered Columnstore Index
11/14/2018 Another Example of a Nonclustered Columnstore Index Similar Query SELECT od.SalesOrderID, sum(od.OrderQty) total_qty FROM Sales.SalesOrderHeader AS oh INNER JOIN Sales.SalesOrderDetail AS od ON od.SalesOrderID = oh.SalesOrderID WHERE oh.OnlineOrderFlag = 1 GROUP BY od.SalesOrderID Covering Index CREATE INDEX cix_SalesOrderDetail_qty ON Sales.SalesOrderDetail(OrderQty) INCLUDE (SalesOrderID); CPU time = 2422 ms, elapsed time = 2978 ms Nonclustered Columnstore Index CREATE NONCLUSTERED COLUMNSTORE INDEX csi_SalesOrderDetail_qty ON Sales.SalesOrderDetail (SalesOrderID, SalesOrderDetailID, OrderQty); CPU time = 203 ms, elapsed time = 446 ms Performance 6x better! © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Clustered Columnstore Index Example
US DOT - On-time Performance Good for star schema (fact tables) CREATE TABLE ONTIME_clu (id int identity(1,1), YEAR varchar(50) NULL, QUARTER varchar(50) NULL, MONTH varchar(50) NULL, DAY_OF_MONTH varchar(50) NULL, DAY_OF_WEEK varchar(50) NULL, FL_DATE varchar(50) not NULL, UNIQUE_CARRIER varchar(50) NULL, AIRLINE_ID varchar(50) NULL, CARRIER varchar(50) not NULL, TAIL_NUM varchar(50) NULL, FL_NUM varchar(50) not NULL, ORIGIN_AIRPORT_ID varchar(50) NULL, ORIGIN_AIRPORT_SEQ_ID varchar(50) not NULL, ORIGIN_CITY_MARKET_ID varchar(50) not NULL, DEST_AIRPORT_ID varchar(50) NULL, DEST_AIRPORT_SEQ_ID varchar(50) not NULL, DEST_CITY_MARKET_ID varchar(50) NULL, ACTUAL_ELAPSED_TIME decimal(6,2) NULL, AIR_TIME varchar(50) NULL, FLIGHTS varchar(50) NULL, DISTANCE varchar(50) NULL, DISTANCE_GROUP varchar(50) NULL, TOTAL_ADD_GTIME varchar(50) NULL, CONSTRAINT PK_ONTIME PRIMARY KEY CLUSTERED (origin_city_market_id,carrier,fl_num,fl_date ,origin_airport_seq_id,dest_airport_seq_id )) ON PRIMARY © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Clustered Columnstore Index Example Easy way to create a clustered columnstore index First create a heap select * into ontime_heap from ontime_clu; Then create the clustered columnstore index Index has no key fields – all data is stored at the leaf block level Table data now resides in columnstore index CREATE CLUSTERED COLUMNSTORE INDEX ontime_cci ON ontime_heap; Add unique nonclustered index CREATE UNIQUE INDEX ontime_unique_nc1 ON ontime_heap (origin_city_market_id,carrier,fl_num,fl_date,origin_airport_seq_id,dest_airport_seq_id) © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Clustered Rowstore Index Example
11/14/2018 Clustered Rowstore Index Example Most popular flights by carriers and cities ALTER TABLE ONTIME_clu ADD CONSTRAINT PK_ONTIME PRIMARY KEY CLUSTERED (ORIGIN_CITY_MARKET_ID ,CARRIER ,FL_NUM , FL_DATE ,ORIGIN_AIRPORT_SEQ_ID ,DEST_AIRPORT_SEQ_ID ); © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Clustered Columnstore Index Size & Restrictions
11/14/2018 Clustered Columnstore Index Size & Restrictions SELECT object_name(si.object_id) as obj_name, si.index_id,si.name as index_name, ps.reserved_page_count * 8./1024. as reserved_MB, ps.lob_reserved_page_count * 8./1024. lob_reserved_MB, ps.row_count FROM sys.dm_db_partition_stats ps JOIN sys.indexes si on ps.object_id=si.object_id and ps.index_id=si.index_id JOIN sys.objects so on si.object_id=so.object_id JOIN sys.schemas sc on so.schema_id=sc.schema_id WHERE sc.name='DBO' and so.name like 'ONTIME%‘ Columnstore index limitations and restrictions Can’t be on a view Can’t use INCLUDE, ASC, DESC Can’t use ALTER INDEX Columstore Limits © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Indexes on Memory-Optimized Tables
11/14/2018 Indexes on Memory-Optimized Tables Nonclustered indexes Best for range or ordered scans When in doubt, use this type of index Columnstore indexes Best for analytical queries That scan or aggregate over large amounts of data Hash indexes Best for equality seeks Doesn’t support range or ordered scans Query must include all columns in the index Data returned is not sorted More info © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Nonclustered Index on a Memory-Optimized Table Example
11/14/2018 Nonclustered Index on a Memory-Optimized Table Example Create ONTIME table in memory CREATE TABLE ontime_inmem ( id int IDENTITY(1,1) NOT NULL,YEAR varchar(50) NULL,QUARTER varchar(50) NULL,MONTH varchar(50) NULL,DAY_OF_MONTH varchar(50) NULL, DAY_OF_WEEK varchar(50) NULL,FL_DATE varchar(50) NOT NULL,UNIQUE_CARRIER varchar(50) NULL, AIRLINE_ID varchar(50) NULL, CARRIER varchar(50) NOT NULL,TAIL_NUM varchar(50) NULL,FL_NUM varchar(50) NOT NULL, ORIGIN_AIRPORT_ID varchar(50) NULL, ORIGIN_AIRPORT_SEQ_ID varchar(50) NOT NULL,ORIGIN_CITY_MARKET_ID varchar(50) NOT NULL, DEST_AIRPORT_ID varchar(50) NULL, DEST_AIRPORT_SEQ_ID varchar(50) NOT NULL,DEST_CITY_MARKET_ID varchar(50) NULL, ACTUAL_ELAPSED_TIME decimal(6, 2) NULL, AIR_TIME varchar(50) NULL,FLIGHTS varchar(50) NULL,DISTANCE varchar(50) NULL, DISTANCE_GROUP varchar(50) NULL,TOTAL_ADD_GTIME varchar(50) NULL, CONSTRAINT pk_ontime_inmem PRIMARY KEY nonCLUSTERED ( origin_city_market_id,carrier,fl_num,fl_date, origin_airport_seq_id,dest_airport_seq_id)) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA); Must first create memory optimized filegroup ALTER DATABASE test ADD FILEGROUP test_mod CONTAINS MEMORY_OPTIMIZED_DATA; ALTER DATABASE test ADD FILE (name='test_mod1', filename= 'C:\Program Files\Microsoft SQL Server\MSSQL13.SS2016\MSSQL\DATA\test_mod1') TO FILEGROUP test_mod; © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Clustered Columnstore Index on a Memory-Optimized Table
11/14/2018 Clustered Columnstore Index on a Memory-Optimized Table CREATE ONTIME table in memory CREATE TABLE ontime_inmem ( id int IDENTITY(1,1) NOT NULL,YEAR varchar(50) NULL,QUARTER varchar(50) NULL,MONTH varchar(50) NULL, DAY_OF_MONTH varchar(50) NULL,DAY_OF_WEEK varchar(50) NULL,FL_DATE varchar(50) NOT NULL,UNIQUE_CARRIER varchar(50) NULL, AIRLINE_ID varchar(50) NULL, CARRIER varchar(50) NOT NULL,TAIL_NUM varchar(50) NULL,FL_NUM varchar(50) NOT NULL, ORIGIN_AIRPORT_ID varchar(50) NULL, ORIGIN_AIRPORT_SEQ_ID varchar(50) NOT NULL,ORIGIN_CITY_MARKET_ID varchar(50) NOT NULL, DEST_AIRPORT_ID varchar(50) NULL, DEST_AIRPORT_SEQ_ID varchar(50) NOT NULL,DEST_CITY_MARKET_ID varchar(50) NULL, ACTUAL_ELAPSED_TIME decimal(6, 2) NULL,AIR_TIME varchar(50) NULL, FLIGHTS varchar(50) NULL,DISTANCE varchar(50) NULL, DISTANCE_GROUP varchar(50) NULL,TOTAL_ADD_GTIME varchar(50) NULL, CONSTRAINT pk_ontime_inmem PRIMARY KEY nonCLUSTERED ( origin_city_market_id,carrier,fl_num,fl_date, origin_airport_seq_id,dest_airport_seq_id), INDEX ontime_inmem_IMCCI CLUSTERED COLUMNSTORE) WITH (MEMORY_OPTIMIZED = ON, DURABILITY = SCHEMA_AND_DATA); Can alter ONTIME_INMEM in SQL Server 2016 (13.x) ALTER TABLE ontime_inmem ADD INDEX ontime_inmem_IMCCI CLUSTERED COLUMNSTORE; © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Clustered Columnstore Index on a Memory-Optimized Table
11/14/2018 Clustered Columnstore Index on a Memory-Optimized Table 9x better performance CPU: 7687 Elasped: 8037 © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Hash Index on a Memory-Optimized Table Example
11/14/2018 Hash Index on a Memory-Optimized Table Example ALTER TABLE ontime_inmem ADD INDEX Hash_ontime_inmem HASH (DEST_AIRPORT_ID,ID) WITH (BUCKET_COUNT = ); 16x better performance CPU: 579 Elasped: 704 © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Index Maintenance Indexes can become fragmented overtime If too many indexes on a table If there is lots of DML activity Slows down performance Optimizer may stop using if too fragmented Use sys.dm_db_index_physical_stats To find index fragmentation View avg_fragmentation_in_percent Microsoft recommendation If >5% and <= 30% Use alter index reorganize If > 30% Use alter index rebuild with (ONLINE=ON) SELECT OBJECT_NAME(IX.OBJECT_ID) AS Table_Name, IX.name AS Index_Name, FPCT.index_type_desc AS Index_Type, FPCT.avg_fragmentation_in_percent Fragmentation_Pct FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, NULL) FPCT INNER JOIN sys.indexes IX ON IX.object_id = FPCT.object_id AND IX.index_id = FPCT.index_id ORDER BY Fragmentation_Pct DESC; © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Index Maintenance Unused indexes or little used indexes - sys.dm_db_index_usage_stats Slows down DML performance Takes up storage space Need to find and delete SELECT OBJECT_NAME(IX.OBJECT_ID) Table_Name ,IX.name AS Index_Name ,IX.type_desc Index_Type ,SUM(PS.used_page_count) * 8 IndexSizeKB,IUS.user_seeks AS NumOfSeeks ,IUS.user_scans AS NumOfScans ,IUS.user_lookups AS NumOfLookups ,IUS.user_updates AS NumOfUpdates,IUS.last_user_seek AS LastSeek ,IUS.last_user_scan AS LastScan ,IUS.last_user_lookup AS LastLookup ,IUS.last_user_update AS LastUpdate FROM sys.indexes IX INNER JOIN sys.dm_db_index_usage_stats IUS ON IUS.index_id = IX.index_id AND IUS.OBJECT_ID = IX.OBJECT_ID INNER JOIN sys.dm_db_partition_stats PS ON PS.object_id=IX.object_id WHERE OBJECTPROPERTY(IX.OBJECT_ID,'IsUserTable') = 1 GROUP BY OBJECT_NAME(IX.OBJECT_ID) ,IX.name ,IX.type_desc ,IUS.user_seeks ,IUS.user_scans ,IUS.user_lookups,IUS.user_updates,IUS.last_user_seek ,IUS.last_user_scan,IUS.last_user_lookup ,IUS.last_user_update © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Index Maintenance Missing indexes sys.dm_db_missing_index_columns(index_handle) DMF to get information about columns missing an index Get index_handle from sys.dm_db_missing_index_details Excludes spatial indexes © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Index Maintenance SELECT OBJECT_NAME(A.OBJECT_ID) AS OBJECT NAME, I.NAME AS INDEX NAME, A.LEAF_INSERT_COUNT, A.LEAF_UPDATE_COUNT, A.LEAF_DELETE_COUNT, A.RANGE_SCAN_COUNT, A.SINGLETON_LOOKUP_COUNT FROM sys.dm_db_index_operational_stats (DB_ID(), NULL, NULL, NULL) A INNER JOIN SYS.INDEXES AS I ON I.OBJECT_ID = A.OBJECT_ID AND I.INDEX_ID = A.INDEX_ID WHERE OBJECTPROPERTY(A.OBJECT_ID,'IsUserTable') = 1 ORDER BY A.LEAF_INSERT_COUNT DESC; Index inserts, update and deletes Slows down performance sys.dm_db_index_operational_stats © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Index Maintenance Stale index statistics Quality of execution plans = quality of statistics SELECT DB_NAME() AS DatabaseName, t.name AS TableName, ix.name AS IndexName ,STATS_DATE(ix.id,ix.indid) AS 'StatsLastUpdate' ,ix.rowmodctr AS '#RowsChanged' ,CAST((CAST(ix.rowmodctr AS DECIMAL(20,8)) / CAST(ix.rowcnt AS DECIMAL(20,2)) * 100.0) AS DECIMAL(20,2)) AS '%RowsChanged' FROM sys.sysindexes ix INNER JOIN sys.tables t ON t.[object_id] = ix.[id] WHERE ix.id > 100 AND ix.indid > 0 AND ix.rowcnt >= 500 ORDER BY [%RowsChanged] DESC; © 2018 SolarWinds Worldwide, LLC. All rights reserved.

11/14/2018 Summary Indexes can speed up query performance By reducing disk I/O and memory footprint It’s important to know when to use each index type Rowstore indexes are traditionally used for highly transactional (OLTP) systems Clustered index leaf level holds data Non-clustered index contains clustered key and can be a covering, filtered index or a combination Columnstore indexes are good for analytical (OLAP) queries Good for aggregating a few columns in very large tables. Can be clustered or non-clustered on a regular table or memory-optimized table Hash indexes are used with memory-optimized tables only Good for single row lookups Indexes require maintenance and up-to-date statistics Quality of execution plan = quality of statistics © 2018 SolarWinds Worldwide, LLC. All rights reserved.

Everything You Need to Know About SQL Server Indexes

Similar presentations

Presentation on theme: "Everything You Need to Know About SQL Server Indexes"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Everything You Need to Know About SQL Server Indexes

Similar presentations

Presentation on theme: "Everything You Need to Know About SQL Server Indexes"— Presentation transcript:

Similar presentations

About project

Feedback