Statistics What are the chances Lori Edwards, SQL Sentry

Introduction Sales Engineer for SQL Sentry since 2/2013 Previously – DBA since 2003 PASS volunteer PASSion Award winner 2011 Email: ledwards@sqlsentry.net Twitter: @loriedwards LinkedIn: www.linkedin.com/in/loriedwards/ Blog: http://blogs.sqlsentry.com/author/LoriEdwards/ 2 | Statistics - What are the chances

Why statistics? This was me circa 2006… 3 | 3 | Statistics - What are the chances

Agenda What are statistics? How and when are they created? Statistics and the query optimizer Demo Statistics maintenance Changes in SQL Server 2014 Questions 4 | Statistics - What are the chances

What are statistics? Statistics contain information about the distribution of data within a columns Statistics can also be created on unindexed columns Statistics on multi-column indexes are on the first column of the index 5 | Statistics - What are the chances

Important Concepts Predicate – The condition in the WHERE or JOIN element of a query Density – relates to the number of unique values within a column Selectivity – percentage of rows that match predicate Cardinality – The estimated number of rows returned by a query operator Filter and join predicates Higher density – fewer unique values Density values can be >0 – 1. 1 means a single distinct value Selectivity – higher selectivity indicates a lower percentage of matching values Cardinality estimates come from density vector and histogram information when available. Otherwise heuristics (WAG) are used 6 | Statistics - What are the chances

How are statistics created? Creating indexes AUTO CREATE STATISTICS CREATE STATISTICS fullscan, sample, norecompute, incremental [2014 + only] sp_createstats indexonly, fullscan, norecompute, incremental [2014 + only] Filtered statistics Statistics on ready-only dbs or read-only snapshots Auto Create Statistics is on by default This works the same way for temporary tables. For very large tables, you may want to create statistics manually rather than have them created at runtime. Sp_createstats can be useful for data warehouses. When creating filtered statistics, include a where clause Statistics on read only dbs or snapshots are temporary and created within tempdb. Viewing the is_temporary column in sys.stats or sys_stats_columns view will indicate temporary statistics 7 | Statistics - What are the chances

Viewing statistics sp_helpstats(‘tablename’, statistic name or ALL) sys.stats catalog view Stats_date sys.dm_db_stats_properties(object_id, stats_id) 8 | Statistics - What are the chances

Viewing statistics DBCC SHOW_STATISTICS(‘tablename’, statistic name) Output is the statistics object (or stat blob) The stat blob is made up of three distinct parts Header Density Vector Histogram 9 | Statistics - What are the chances

The Stat Blob Header – meta data about the statistic Update date and rows v. rows sampled are most important Density value here can be ignored – displayed for backward compatibility for versions before 2008 10 | Statistics - What are the chances

The Stat Blob Density Vector – Density values for the column(s) in the statistic For index based statistics, all column combinations will appear including the clustered key value Density value over multiple columns not as accurate – assuming independence, the density is found by multiplying the individual selectivities 11 | Statistics - What are the chances

The Stat Blob Histogram – Shows the data distribution for a column in a tabular format 12 | Statistics - What are the chances

The Stat Blob Histogram 13 | Statistics - What are the chances

The Stat Blob Histogram 14 | Statistics - What are the chances

Statistics, Cardinality and the Query Optimizer Statistics are necessary to estimate row count, or cardinality Cardinality along with the operator cost model = total operator cost Costing is taken into account when choosing a plan Query optimizer’s job is not to find the best plan – it’s to find a good plan quickly. 15 | Statistics - What are the chances

Statistics, Cardinality and the Query Optimizer Getting the most from cardinality estimates Keep it clean Options Parameters Temporary tables Column order Using multiple columns from the same table Table variables and Table Valued Parameters Option (optimize for unknown) No statistics on multi-statement table-valued functions No statistics created for table variables Especially with columns within the same table. Trace flag 2453 for table variables – available 2012 SP2 and 2014 CU#3 or with recompile 11/20/201811/20/2018 | Statistics - What are the chances

Decisions affected by cardinality Parallel or serial plans Index seek or scan Join algorithms Including inner/outer table selection Spool generation Key lookup vs. table scan Stream or hash aggregates Implicit conversion can impact cardinality estimates Comparing columns within the same table can affect cardinality estimates – computed columns can help Keep predicates as clean as possible 11/20/201811/20/2018 | Statistics - What are the chances

Cardinality and memory grants Many execution plans require memory grants for specific operators (hash, sort, spool) Memory grants = cost of operator * estimated row count Both overestimates and underestimates can have detrimental effects Memory grants can be viewed via sys.dm_exec_query_memory_grants 11/20/201811/20/2018 | Statistics - What are the chances

Demo 19 | Statistics - What are the chances

Updating statistics In the majority of environments, AUTO UPDATE STATISTICS should be on When does that kick in? It depends on the number of rows in the table As a note, the AUTO_UPDATE_STATISTICS_ASYNC option will determine whether the update occurs when an execution plan with out of date statistics is called or after the query executes Sp_autostats on table/index/statistics object Updates when -> Rowcount goes from 0 to >0 Rowcount <= 500 and more than 500 changes have occurred Rowcount > 500 and 500 + 20% of rowcount changes have been made Asynchronously will use old statistics (and execution plans based on them) until the statistics are built in the background – synchronously – the query will halt while the statistics are updated – depending on the size of your table this could be a few seconds or many minutes 20 | Statistics - What are the chances

Updating statistics For very large tables Trace flag 2371 Available SS 2008 R2 SP1 21 | Statistics - What are the chances

Updating statistics manually sp_updatestats vs UPDATE STATISTICS sp_updatestats updates all of the statistics within a database that have had at least one modification Note: will update with the default (or last) sample Filtered statistics will not be optimized For memory optimized tables, ALL statistics are updated UPDATE STATISTICS requires a table name with optional index or statistics name Updating statistics as part of index maintenance 22 | Statistics - What are the chances

Updating statistics How important is it to keep statistics updated? It’s important, but items to consider Does the table data change frequently (either in size or data distribution)? How will plan recompilation impact your environment? And is plan recompilation a factor on your hardware? 23 | Statistics - What are the chances

Cardinality Estimator changes in 2014 Independancy assumption Ascending key problem Filtered predicates on different tables Join estimate algorithm Trace flags 2312 and 9481 Independency assumption Legacy – (count from predicate 1 * count from predicate 2)/ total row count – assuming total independence New – selectivity of the most selective filter * sqrt(next most selective filter) – doesn’t assume total independence or total correlation Ascending key – legacy if value fell outside of histogram, cardinality was set at 1 New – rows sampled * all density (average frequency) for samples. For fullscan, it uses the number of rows inserted since last stats rebuild Filtered predicatesLegacy – assumed correlation New – assumes no correlation Join estimation – Legacy step by step correlation in histograms New – uses a coarse alighnment – this is where you might see more inaccurate estimations using the new CE TF 2312 – forces use of new CE on 2012 dbs TF 9481 – forces use of legacy CE on 2014 dbs Can be enabled at server, session or query level – to enable on a query level use QUERYTRACEON hint TF 2453 can not be set on a query level using QUERYTRACEON 11/20/201811/20/2018 | Statistics - What are the chances

Summary Statistics provide valuable information on the data within columns Statistics are created on indexes automatically Can also be created manually and autocreated Accurate statistics help the query optimizer to find the best plan View stats with sp_helpstats and DBCC SHOW_STATISTICS Maintain stats by setting AUTO_UPDATE_STATISTICS on and using (sometimes) sp_updatestats 25 | Statistics - What are the chances

Resources Statistics in SQL Server 2014: https://msdn.microsoft.com/en-us/library/ms190397.aspx AUTO_UPDATE_STATS_ASYNC option: http://www.brentozar.com/blitz/auto-update-stats-async-enabled/ Trace flag 2371: https://support.microsoft.com/en-us/kb/2754171 Understanding Statistics Updates: http://www.sqlskills.com/blogs/erin/understanding-when-statistics-will-automatically-update/ And some additional statistics questions: https://www.simple-talk.com/content/article.aspx?article=1744 26 | Statistics - What are the chances

Questions? Email me at ledwards@sqlsentry.net Or tweet me @loriedwards Thank you! 27 | Statistics - What are the chances

Thank You Sponsors! Visit the Sponsor tables to enter their end of day raffles. Turn in your completed Event Evaluation form at the end of the day in the Registration area to be entered in additional drawings. Want more free training? Check out the Houston Area SQL Server User Group which meets on the 2nd Tuesday of each month. Details at http://houston.sqlpass.org 6/13/2015