Best Practices for Columnstore Indexes Warner Chaves SQL MCM / MVP SQLTurbo.com Pythian.com

Bio  SQL Server DBA for 10 years.  Previously an L3 DBA at HP, now a Principal Consultant at Pythian in Ottawa Ontario.  SQL Server MCM and MVP.    Blog: sqlturbo.com  Company site: pythian.com

Agenda  GOAL: simple and straightforward intro to best practices when working with Columnstore Indexes.  We’ll cover:  Brief intro to Columnstores and their use cases.  Demo: fast loading of a Non-clustered columnstore index.  Demo: loading a clustered Columnstore without hitting the Delta store.  Demo: deleting data off a Columnstore.  Demo: Batch mode and segment elimination.

What is it?  Optimized columnar storage for analytic style queries.  Used for large amounts of data ingestion and reading.  It’s a static index on SQL 2012, read-write on  Enterprise license required or Azure SQL Premium tier.

What’s the big deal?  Big benefits for data warehouse tables.  Really high compression on the columnar format.  Batch mode can process rows with higher throughput.  Simplifies the management of large warehousing tables.  The queries that it’s optimized for will perform better.

Columnstore structure Rowgroups: grouping of records, records max. Column segment: storage unit, a rowgroup is split into segments, one per column. Deltastore: used by clustered Columnstores as temp storage for new records. Inserts with records avoid the Deltastore. Deleted records b-tree: clustered Columnstores keep an internal table identifying deleted records. This is also kept on an in-memory structure for fast access. Tuple-mover: this process checks the deltastore for full rowgroups (closed) to compress and move to the columnstore.

Let’s go over it

Demos!  Demo: fast loading of a Non-clustered columnstore index.  Demo: loading a clustered Columnstore without hitting the Delta store.  Demo: deleting data off a Columnstore.  Demo: taking full advantage of Batch mode.

Summary  If possible combine with partitioning (a must for the non- clustered variation).  Bulk load and avoid small trickle inserts. Delayed Durability is a plus.  Verify queries are using batch mode and work around any batch mode inhibitors. Avoid MAXDOP 1.

Summary (2)  Tuple mover is single thread, REORG is multi-thread.  Maintain stats but watch out for auto update triggering.  Monitor segment sizes and deleted row amounts, REBUILD if necessary.  Exploit sort order if possible for SEGMENT elimination.

Future ahead  New Features for SQL 2016: – Parallel insert. – Nonclustered B-tree indexes on top of the columnstore. – Read/write nonclustered columnstore on top of a regular clustered index. – In-memory columnstore on in-memory oltp table. – More T-SQL constructs run in batch mode (including sorting). – More predicates and aggregates pushed down to the storage engine. – REORGANIZE will remove deleted rows and merge small rowgroups, online. – New DMVs, XEvents and Perfmon counters. – Support for RCSI or Snapshot isolation. – Fully readable on an Availability Groups secondary.  Check videos of the sessions from Ignite 2015 on channel9.microsoft.com

QA?