Optimizer Statistics Julian Dyke Independent Consultant Web Version

Slides:



Advertisements
Similar presentations
Tuning: overview Rewrite SQL (Leccotech)Leccotech Create Index Redefine Main memory structures (SGA in Oracle) Change the Block Size Materialized Views,
Advertisements

Copyright © 2011 Pearson Education, Inc. Publishing as Pearson Addison-Wesley Chapter 5 More SQL: Complex Queries, Triggers, Views, and Schema Modification.
California State University Common Management Systems TUG Session: April 21, Overview Statistics in CMS PRD / Non-PRD Environments.
Introduction to Structured Query Language (SQL)
1 - Oracle Server Architecture Overview
Harvard University Oracle Database Administration Session 2 System Level.
Harvard University Oracle Database Administration Session 5 Data Storage.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 11 Database Performance Tuning and Query Optimization.
1 How to improve SQL Performance with new Health Check Tool? Carlos Sierra Consulting Technical Advisor © 2012 Oracle Corporation – Proprietary and Confidential.
1 Copyright © 2011, Oracle and/or its affiliates. All rights reserved. Understanding Optimizer Statistics.
Oracle 10g Database Administrator: Implementation and Administration
1 juliandyke.com © 2005 Julian Dyke Data Segment Compression Julian Dyke Independent Consultant Web Version.
Oracle Database Administration Database files Logical database structures.
ORAMAIN Consulting Services, LLC1 The Do’s and Don’ts of CBO by Jim Gillespie ORAMAIN Consulting Services, LLC id: website:
12 Copyright © 2007, Oracle. All rights reserved. Database Maintenance.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 10 Database Performance Tuning and Query Optimization.
Chapter Oracle Server An Oracle Server consists of an Oracle database (stored data, control and log files.) The Server will support SQL to define.
Chapter 6 Additional Database Objects
Oracle9i Database Administrator: Implementation and Administration 1 Chapter 9 Index Management.
Oracle Database Administration Lecture 6 Indexes, Optimizer, Hints.
By Lecturer / Aisha Dawood 1.  You can control the number of dispatcher processes in the instance. Unlike the number of shared servers, the number of.
1 Robert Wijnbelt Health Check your Database A Performance Tuning Methodology.
1 RAC Internals Julian Dyke Independent Consultant Web Version juliandyke.com © 2007 Julian Dyke.
RBO RIP George Lumpkin Director Product Management Oracle Corporation Session id:
1 Chapter 7 Optimizing the Optimizer. 2 The Oracle Optimizer is… About query optimization Is a sophisticated set of algorithms Choosing the fastest approach.
Other database objects (Sequence). What Is a Sequence? A sequence: Automatically generates sequential numbers Is a sharable object Is typically used to.
Chapter 6 Additional Database Objects Oracle 10g: SQL.
1 What’s Up with dbms_stats? Terry Sutton Database Specialists, Inc.
PL/SQLPL/SQL Oracle11g : PL/SQL Programming Chapter 6 Functions.
Triggers A Quick Reference and Summary BIT 275. Triggers SQL code permits you to access only one table for an INSERT, UPDATE, or DELETE statement. The.
Using Procedures & Functions Oracle Database PL/SQL 10g Programming Chapter 9.
Views Lesson 7.
Database structure and space Management. Database Structure An ORACLE database has both a physical and logical structure. By separating physical and logical.
Database Systems Design, Implementation, and Management Coronel | Morris 11e ©2015 Cengage Learning. All Rights Reserved. May not be scanned, copied or.
Indexes / Session 2/ 1 of 36 Session 2 Module 3: Types of Indexes Module 4: Maintaining Indexes.
1 © 2010 Julian Dyke Edition-Based Redefinition Julian Dyke Independent Consultant juliandyke.com Web Version.
Objectives Database triggers and syntax
PL/SQLPL/SQL Oracle10g Developer: PL/SQL Programming Chapter 9 Database Triggers.
14 Copyright © 2006, Oracle. All rights reserved. Tuning Block Space Usage.
Database Programming Sections 11 & 12 –Sequences, Indexes, and Synonymns.
1 Logical I/O Julian Dyke Independent Consultant Web Version juliandyke.com © 2005 Julian Dyke.
1 Chapter 13 Parallel SQL. 2 Understanding Parallel SQL Enables a SQL statement to be: – Split into multiple threads – Each thread processed simultaneously.
Dynamic SQL. 2 home back first prev next last What Will I Learn? Recall the stages through which all SQL statements pass Describe the reasons for using.
PL/SQLPL/SQL Oracle11g: PL/SQL Programming Chapter 9 Database Triggers.
PL/SQLPL/SQL Oracle10g Developer: PL/SQL Programming Chapter 9 Database Triggers.
1 © 2005 Julian Dyke Reducing Redo Julian Dyke Independent Consultant Web Version juliandyke.com.
Chapter 5 : Integrity And Security  Domain Constraints  Referential Integrity  Security  Triggers  Authorization  Authorization in SQL  Views 
Chapter 5 Index and Clustering
Transactions, Roles & Privileges Oracle and ANSI Standard SQL Lecture 11.
Creating Indexes on Tables An index provides quick access to data in a table, based on the values in specified columns. A table can have more than one.
1 Chapter 9 Tuning Table Access. 2 Overview Improve performance of access to single table Explain access methods – Full Table Scan – Index – Partition-level.
Stored Procedures / Session 4/ 1 of 41 Session 4 Module 7: Introducing stored procedures Module 8: More about stored procedures.
Oracle9i Developer: PL/SQL Programming Chapter 5 Functions.
1 CS 430 Database Theory Winter 2005 Lecture 13: SQL DML - Modifying Data.
Last Updated : 27 th April 2004 Center of Excellence Data Warehousing Group Teradata RDBMS Concepts.
8 Copyright © 2005, Oracle. All rights reserved. Gathering Statistics.
1 11g NEW FEATURES ByVIJAY. 2 AGENDA  RESULT CACHE  INVISIBLE INDEXES  READ ONLY TABLES  DDL WAIT OPTION  ADDING COLUMN TO A TABLE WITH DEFAULT VALUE.
SQL Server Statistics DEMO SQL Server Statistics SREENI JULAKANTI,MCTS.MCITP,MCP. SQL SERVER Database Administration.
SQL Server Statistics DEMO SQL Server Statistics SREENI JULAKANTI,MCTS.MCITP SQL SERVER Database Administration.
Views / Session 3/ 1 of 40 Session 3 Module 5: Implementing Views Module 6: Managing Views.
2 Copyright © 2009, Oracle. All rights reserved. Managing Schema Objects.
SQL Server Statistics and its relationship with Query Optimizer
SQL Trace and TKPROF.
Database Performance Tuning and Query Optimization
Oracle Statistics by Example
Statistics: What are they and How do I use them
What’s Up with dbms_stats?
Oracle9i Developer: PL/SQL Programming Chapter 8 Database Triggers.
Chapter 11 Database Performance Tuning and Query Optimization
Prof. Arfaoui. COM390 Chapter 9
Presentation transcript:

Optimizer Statistics Julian Dyke Independent Consultant Web Version juliandyke.com

Agenda Statistics Strategies ANALYZE DBMS_STATS ANALYZE versus DBMS_STATS System Statistics Automatic Statistics Collection Statistics History Manual Statistics Partition Statistics Oracle 11g Enhancements

Statistics Strategies

Statistics Strategies Introduction Only gather statistics when object has changed significantly Consider generating manual statistics for: Very large objects Temporary tables Highly volatile tables Try to collect statistics during a specified window Do not execute SQL statements during statistics collection Use DBMS_STATS parallelism Review sample sizes for very large objects In complex databases do not rely on Auto job Unpredictable collection behaviour / duration Unpredictable execution plan changes See Metalink Note 44961.1 - Statistics Gathering; Frequency and Strategy

ANALYZE

ANALYZE Introduction SQL statement Introduced in Oracle 7.0 Collects object statistics for Cost Based Optimizer Initially collected statistics on tables and indexes Subsequently on histograms Advantages Online analyze - Oracle 9.0 and above Optionally validates structure of tables, indexes and clusters Optionally detects chained / migrated rows Disadvantages Serial statistics collection only Limited partition-awareness

ANALYZE VALIDATE STRUCTURE Syntax is ANALYZE INDEX <index_name> VALIDATE STRUCTURE; Populates session-level dynamic performance views INDEX_STATS INDEX_HISTOGRAMS Reports index statistics including: height number of leaf blocks number of branch blocks number of distinct keys Reports space released by deletions Use to determine when to coalesce or rebuild index Recommends index compression columns Use to determine which indexes should be compressed

ANALYZE VALIDATE STRUCTURE ONLINE Introduced in Oracle 9.0.1 Syntax is ANALYZE INDEX <index_name> VALIDATE STRUCTURE ONLINE; ONLINE option only checks structure does not require TM (DML) lock does not populate INDEX_STATS or INDEX_HISTOGRAM

DBMS_STATS

DBMS_STATS Introduction PL/SQL supported package Introduced in Oracle 8.1.5 In Oracle 11.1 there are 107 subroutines Defined in $ORACLE_HOME/rdbms/admin/dbmsstat.sql Advantages Parallel statistics collection Partition-awareness Disadvantages Cannot perform structure validation Cannot detect chaining / migration

DBMS_STATS Gathering Statistics Statistics can be gathered using the following subroutines: GATHER_DATABASE_STATS GATHER_SCHEMA_STATS GATHER_TABLE_STATS GATHER_INDEX_STATS These routines allow you to specify: Sample size Granularity (partitioned tables only) Histogram policy Cursor invalidation policy

DBMS_STATS Deleting Statistics Statistics can be deleted using the following subroutines: DELETE_DATABASE_STATS DELETE_SCHEMA_STATS DELETE_TABLE_STATS DELETE_INDEX_STATS DELETE_COLUMN_STATS DELETE_TABLE_STATS includes CASCADE_PART - optionally delete partition statistics CASCADE_COLUMNS - optionally delete column statistics CASCADE_INDEXES - optionally delete index statistics DELETE_INDEX_STATS, DELETE_COLUMN_STATS include:

DBMS_STATS Statistics Tables Used to export / import statistics store object statistics values PRIOR to GATHER operations To create a statistics table use CREATE_STATS_TABLE. For example: dbms_stats.create_stats_table ( ownname => 'SYSTEM', tabname => 'STATS1', tblspace=> 'SYSAUX' ); To drop a statistics table use DROP_STATS_TABLE To drop a statistics table use DROP_STATS_TABLE. For example: dbms_stats.drop_stats_table ( ownname => 'SYSTEM', tabname => 'STATS1' );

DBMS_STATS Exporting and Importing Statistics Statistics can be exported to a statistics table using: EXPORT_COLUMN_STATS EXPORT_DATABASE_STATS EXPORT_INDEX_STATS EXPORT_SCHEMA_STATS EXPORT_TABLE_STATS Statistics can be imported from a statistics table using: IMPORT_COLUMN_STATS IMPORT_DATABASE_STATS IMPORT_INDEX_STATS IMPORT_SCHEMA_STATS IMPORT_TABLE_STATS To transfer the statistics to another database export and import the statistics table

DBMS_STATS OPTIONS parameter Specifies how to determine which objects to gather statistics for: GATHER - gather statistics for all objects GATHER AUTO - gather statistics for all objects needing new statistics GATHER STALE - gather statistics for all objects having stale statistics GATHER EMPTY- gather statistics for all objects having no statistics LIST AUTO - list objects needing new statistics LIST STALE - list objects having stale statistics LIST EMPTY - list objects having stale statistics The OBJLIST parameter must be specified as an OUT parameter for the LIST options GATHER STALE and GATHER EMPTY can also return a list of objects if this parameter is specified

DBMS_STATS OPTIONS parameter If GATHER STALE is specified statistics will be gathered for objects where at least 10% of rows have changed Total of INSERTS, UPDATES and DELETES since last statistics collection In Oracle 10g percentage is fixed at 10% In Oracle 11g percentage is user-configurable

DBMS_STATS Stale Statistics In Oracle 10.1 and above table modification statistics: are enabled if STATISTICS_LEVEL is TYPICAL or ALL are disabled if STATISTICS_LEVEL is BASIC are collected in the SGA are automatically flushed to SYS.MON_MODS$ by SMON background process every 15 minutes to SYS.MON_MODS$ if a significant number of rows has changed (possibly 10%) can be manually flushed to SYS.MON_MOD$ using: DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO are reported in DBA_TAB_MODIFICATIONS

DBMS_STATS Stale Statistics SYS.MON_MODS$ has the following columns: Column Name Data Type OBJ# NUMBER INSERTS UPDATES DELETES TIMESTAMP DATE FLAGS DROP_SEGMENTS

DBMS_STATS Stale Statistics In Oracle 8i, table modification monitoring can only be enabled manually using: ALTER TABLE <table_name> MONITORING; ALTER TABLE <table_name> NOMONITORING; In Oracle 9i, table modification monitoring can also be enabled using DBMS_STATS.ALTER_SCHEMA_TAB_MONITORING ('<owner>',TRUE); DBMS_STATS.ALTER_DATABASE_TAB_MONITORING (TRUE); Reported in DBA_TABLES.MONITORING In Oracle 10.1 and above MONITORING clause is deprecated and ignored

DBMS_STATS ESTIMATE_PERCENT Parameter Specified using ESTIMATE_PERCENT parameter in GATHER_DATABASE_STATS GATHER_DICTIONARY_STATS GATHER_INDEX_STATS GATHER_SCHEMA_STATS GATHER_TABLE_STATS Valid range is 0.000001 to 100 NULL means COMPUTE Default (10.2) is DBMS_STATS.AUTO_SAMPLE_SIZE Oracle determines sample size automatically Default can be changed using SET_PARAM procedure

DBMS_STATS GRANULARITY Parameter Only relevant for partitioned objects ALL - gather global, partition and subpartition statistics AUTO - determine granularity based on partitioning type. DEFAULT - gather global and partition-level statistics. Now obsolete. Use GLOBAL AND PARTITION instead GLOBAL - gather global statistics GLOBAL AND PARTITION - gather global and partition-level statistics PARTITION - gather partition-level statistics SUBPARTITION - gather subpartition-level statistics In Oracle 10.2 default is AUTO (not DEFAULT)

DBMS_STATS NO_INVALIDATE parameter In earlier versions ANALYZE and DBMS_STATS gather procedures always invalidated cursors referencing affected object In Oracle 10.1 and above, invalidation can be controlled using NO_INVALIDATE parameter TRUE - do not invalidate dependent cursors FALSE - invalidate dependent cursors DBMS_STATS.AUTO_INVALIDATE (default) - Oracle determines whether to invalidate dependent cursors or not. In Oracle 10.2 the default value can be changed using SET_PARAM procedure Only relevant for DBMS_STATS gather procedures ANALYZE always invalidates analyzed objects

DBMS_STATS Data Dictionary Statistics Oracle recommends that data dictionary statistics are collected in Oracle 9.0.1 and above Subroutines are: GATHER_DICTIONARY_STATS DELETE_DICTIONARY_STATS EXPORT_DICTONARY_STATS RESTORE_DICTIONARY_STATS In Oracle 9i data dictionary statistics should only be collected using: dbms_stats.gather_schema_stats (ownname => 'SYS',cascade=> TRUE); In Oracle 9i data dictionary statistics can be deleted using: dbms_stats.delete_schema_stats (ownname => 'SYS'); See Metalink Note 245051.1 Note ANALYZE should not be used with tables owned by SYS. See Metalink Note 35272.1

DBMS_STATS Fixed Object Statistics In Oracle 10.1 and above Oracle recommends that statistics are collected on fixed objects These include X$ and V$ views Subroutines include: GATHER_FIXED_OBJECTS_STATS DELETE_FIXED_OBJECTS_STATS EXPORT_FIXED_OBJECT_STATS RESTORE_FIXED_OBJECTS_STATS For example: dbms_stats.gather_fixed_objects_stats; According to Metalink Note 272479.1 fixed object statistics can be collected and stored in a temporary table and subsequently written to the data dictionary. To create the temporary statistics table use: EXECUTE dbms_stats.create_stat_table ('SYS','FIXED_STATS'); To collect statistics and to store them in the temporary statistics table use: EXECUTE dbms_stats.gather_fixed_objects ('FIXED_STATS'); To copy the statistics from the temporary statistics table to the data dictionary use: EXECUTE dbms_stats.gather_fixed_objects (NULL); JMD Not tested by me. Only gather fixed object statistics after the database has been running a representative workload See Metalink Note 272479.1

DBMS_STATS METHOD_OPT Parameter Specifies histogram collection policy Can be specified for: GATHER_DATABASE_STATS, GATHER_DICTIONARY_STATS, GATHER_SCHEMA_STATS GATHER_TABLE_STATS Default is FOR ALL COLUMNS SIZE AUTO Uses data from COL_USAGE$ table

DBMS_STATS METHOD_OPT Parameter Syntax is: FOR ALL [ INDEXED | HIDDEN ] COLUMNS [ size_clause ] FOR COLUMNS [ [ size_clause ] column|attribute [size_clause] [, column|attribute [size_clause]..] size_clause is: SIZE { integer | REPEAT | AUTO | SKEWONLY } where: integer - number of histogram buckets (1..254) REPEAT - only collect histograms for columns that already have histograms AUTO - determine which columns need histograms automatically SKEWONLY - determine which columns need histograms based on data distribution

DBMS_STATS Column Usage Monitoring Column usage is recorded in SYS.COL_USAGE$ Column Name Data Type OBJ# NUMBER INTCOL# EQUALITY_PREDS EQUIJOIN_PREDS NONEQUIJOIN_PREDS RANGE_PREDS LIKE_PREDS NULL_PREDS TIMESTAMP DATE COL_USAGE$ is Updated when statements are parsed Reset when statistics are gathered for table

DBMS_STATS Histograms Summary Do not collect histograms if not necessary Expensive to collect Check accuracy of histograms Watch for columns with high proportion of NULL values Watch for highly skewed columns Height-balanced histograms Limited use if column is not highly skewed Object oriented designs rarely work with histograms Using histograms introduces bind variable peeking issues

DBMS_STATS Locking Statistics Statistics can be locked for schemas, tables, and partitions LOCK_SCHEMA_STATS / UNLOCK_SCHEMA_STATS LOCK_TABLE_STATS / UNLOCK_TABLE_STATS LOCK_PARTITION_STATS / UNLOCK_PARTITION_STATS dbms_stats.lock_table_stats ( ownname => 'USER1', tabname => 'T1', stattype => 'ALL' ); Possible values for STATTYPE are NULL, DATA, CACHE and ALL By default GATHER procedures will not overwrite locked statistics Specify FORCE => TRUE to overwrite Statistics locking reported in STATTYPE_LOCKED column in DBA_TAB_STATISTICS DBA_IND_STATISTICS

DBMS_STATS Default Parameters In Oracle 10.1 and above default parameters can be specified System-wide defaults can be set for Column Name Factory Default CASCADE DBMS_STATS.AUTO_CASCADE DEGREE NULL ESTIMATE_PERCENT DBMS_STATS.AUTO_SAMPLE_SIZE METHOD_OPT FOR ALL COLUMNS SIZE AUTO NO_INVALIDATE DBMS_STATS.AUTO_INVALIDATE GRANULARITY AUTO AUTOSTATS_TARGET AUTOSTATS_TARGET can be ALL collect statistics for all objects in the database ORACLE collect statistics for all Oracle-owned objects AUTO determine which objects need new statistics Statistics default values will be used in GATHER, EXPORT and IMPORT functions. Note that CASCADE is only applicable for GATHER Factory defaults in Oracle 10.2 are: CASCADE - DBMS_STATS.AUTO_CASCADE DEGREE - NULL ESTIMATE_PERCENT - DBMS_STATS.AUTO_SAMPLE_SIZE METHOD_OPT- FOR ALL COLUMNS SIZE AUTO NO_INVALIDATE - DBMS_STATS.NO_INVALIDATE GRANULARITY - AUTO AUTOSTATS_TARGET - AUTO When setting parameters NULL resets parameter to factory default 'NULL' specifies a value of NULL Parameters can be reset to factory settings using the RESET_PARAM_DEFAULTS procedure

DBMS_STATS Default Parameters To set default parameters use SET_PARAM: dbms_stats.set_param (pname => 'CASCADE',pval => TRUE); Take care with NULL values For example: dbms_stats.set_param ('ESTIMATE_PERCENT','NULL'); Sets the default to COMPUTE statistics dbms_stats.set_param ('ESTIMATE_PERCENT',NULL); Restores factory default (DBMS_STATS.AUTO_SAMPLE_SIZE) To check default parameters use GET_PARAM e.g.: GET_PARAM is obsolete in Oracle 11.1 - use GET_PREFS instead SET_PARAM is obsolete in Oracle 11.1 - use SET_GLOBAL_PREFS instead RESET_PARAM_DEFAULTS is obsolete in Oracle 11. Use RESET_GLOBAL_PREF_DEFAULTS instead SELECT dbms_stats.get_param (pname => 'CASCADE') FROM dual; To restore all factory default parameters use: dbms_stats.reset_param_defaults;

ANALYZE versus DBMS_STATS

ANALYZE versus DBMS_STATS Differences ANALYZE updates the following columns in DBA_TAB_STATISTICS EMPTY_BLOCKS AVG_SPACE DBMS_STATS sets these columns to 0 Tables AVG_ROW_LEN differs - e.g.: Indexes No obvious differences between ANALYZE and DBMS_STATS Columns AVG_COL_LEN differs - e.g.: ANALYZE 43 DBMS_STATS 39 ANALYZE 4 DBMS_STATS 5

ANALYZE versus DBMS_STATS Differences ANALYZE <table> gathers statistics for related indexes Prior to Oracle 10.1 DBMS_STATS requires CASCADE => TRUE For empty indexes ANALYZE sets BLEVEL to 0 DBMS_STATS sets BLEVEL > 0

ANALYZE versus DBMS_STATS Summary Use ANALYZE to Collect index statistics without updating data dictionary Identify candidates for index coalesce/rebuild Identify candidates for index compression Identify tables containing excessive chained rows Use DBMS_STATS Gather statistics on tables / indexes Do not mix ANALYZE / DBMS_STATS

System Statistics

System Statistics Introduction System statistics were introduced in Oracle 9.0.1 Subroutines are: GATHER_SYSTEM_STATS DELETE_SYSTEM_STATS GET_SYSTEM_STATS SET_SYSTEM_STATS EXPORT_SYSTEM_STATS IMPORT_SYSTEM_STATS RESTORE_SYSTEM_STATS System statistics are stored in SYS.AUX_STATS$

System Statistics Workload versus No Workload Statistics Oracle 10.1 and above supports: noworkload statistics - default values supplied during database creation workload statistics - collected from representative workload Workload statistics include: CPUSPEED - CPU speed SREADTIM - Single block read time in milliseconds MREADTIM - Multi block read time in milliseconds MBRC - Multi block read count MAXTHR - Maximum I/O system throughput (parallel execution only) SLAVETHR - Average slave I/O throughput (parallel execution only) No workload statistics include: CPUSPEEDNW - CPU speed IOSEEKTIM - IO Seek Time IOTFRSPEED - IO Transfer Speed In Oracle 10.2 the optimizer uses the value of MBRC instead of db_file_multiblock_read_count to compute costs for full table scans and index fast full scans. Oracle 9.0 includes CPUSPEED, SREADTIM, MREADTIM and MBRC. MAXTHR and SLAVETHR were added in Oracle 9.2 to set a lower limit for full table scans CPUSPEEDNW, IOSEEKTIM and IOTFRSPEED were added in Oracle 10.1 Values for CPUSPEEDNW, IOSEEKTIM and IOTFRSPEED can be collected using: DBMS_STAT.GATHER_SYSTEM_STATS (gathering_mode =>NOWORKLOAD);

System Statistics Example Create a statistics table using: dbms_stats.create_stat_table ('SYS','OLTP_STATS'); Gather system statistics for a typical period using: dbms_stats.gather_system_stats ( gathering_mode => 'INTERVAL', interval => 60, -- 60 seconds stattab => OLTP_STATS', statid => 'OLTP' ); Import system statistics into AUX_STATS$ using: In order to collect system statistics JOB_QUEUE_PROCESSES must be set to 1 or more: ALTER SYSTEM SET job_queue_processes = 1; dbms_stats.import_system_stats ( stattab => OLTP_STATS', statid => 'OLTP', statown => 'SYS' );

System Statistics Summary Enable system statistics for single instance databases Usually improve execution plans Consider carefully before enabling system statistics in RAC databases System statistics are database-specific Watch for asymmetric nodes Hardware differences Service configuration System statistics are: Difficult to monitor Very difficult to update / remove If exporting object statistics to another system for testing: Remember to export system statistics

Automatic Statistics Collection

Automatic Statistics Collection Introduction Oracle 10.1 and above Statistics collected during Maintenance Window Monday - Friday 22:00 to 06:00 Saturday / Sunday All day Note that weekend window effectively starts at 06:00 on Saturday morning Scheduler job GATHER_STATS_JOB Scheduler job class AUTO_TASKS_JOB_CLASS Scheduler windows WEEKNIGHT_WINDOW WEEKEND_WINDOW Scheduler window group MAINTENANCE_WINDOW_GROUP

Statistics History

Statistics History Introduction In Oracle 10.1 and above, existing statistics are stored in the data dictionary when new statistics are collected Statistics can be restored using: RESTORE_DATABASE_STATS RESTORE_DICTIONARY_STATS RESTORE_FIXED_OBJECTS_STATS RESTORE_SCHEMA_STATS RESTORE_SYSTEM_STATS RESTORE_TABLE_STATS Statistics history for tables only is reported in DBA_TAB_STATS_HISTORY SELECT stats_update_time FROM dba_tab_stats_history WHERE owner = 'USER1' AND table_name = 'T1'; 12-FEB-09 04.36.32.997000 PM +00:00

Statistics History Optimizer Statistics Operations Statistics gathering operations are reported in DBA_OPTSTAT_OPERATIONS SELECT operation, target, start_time, end_time FROM dba_optstat_operations ORDER BY start_time; gather_database_stats(auto) 07-FEB09 06.00.03 07-FEB-09 06.01.52 gather_database_stats(auto) 09-FEB09 10.00.03 09-FEB-09 10.03.37 gather_database_stats(auto) 10-FEB09 10.00.03 10-FEB-09 10.02.01 DBA_OPTSTAT_OPERATIONS includes: gather_database_stats(auto) gather_schema_stats DBA_OPTSTAT_OPERATIONS does not include: gather_table_stats gather_index_stats

Statistics History Data Dictionary Tables Historic statistics are stored in tables created by $ORACLE_HOME/rdbms/admin/catost.sql Tables created are: WRI$_OPTSTAT_TAB_HISTORY WRI$_OPTSTAT_IND_HISTORY WRI$_OPTSTAT_HISTHEAD_HISTORY WRI$_OPTSTAT_HISTGRM_HISTORY WRI$_OPTSTAT_AUX_HISTORY WRI$_OPTSTAT_OPR OPTSTAT_HIST_CONTROL$

Statistics History Retention Period To check statistics history retention period use: SELECT dbms_stats.get_stats_history_retention FROM dual; Default is 31 days To check earliest historic statistics use: SELECT dbms_stats.get_stats_history_availability FROM dual; 12-JAN-09 11.17.50.176000000 PM +00:00 To set statistics history retention period to 90 days: dbms_stats.alter_stats_history_retention ( retention => 90 );

Statistics History Summary Statistics history rows contain previous values for optimizer statistics Statistics history does not include current value SAVTIME column is time row was written ANALYZETIME column is time statistics were analyzed SAVTIME != ANALYZETIME ANALYZE does not update statistics history [UN]LOCK_TABLE_STATS do update statistics history

Manual Statistics

Setting Table Statistics Introduction Table statistics can be set manually using SET_TABLE_STATS Values can be specified for: NUMROWS - number of rows NUMBLKS - number of blocks AVGRLEN - average row length

Setting Table Statistics Example (1 of 4) DECLARE l_numrows NUMBER; l_numblks NUMBER; l_avgrlen NUMBER; l_flags NUMBER; l_total_blocks NUMBER; l_total_bytes NUMBER; l_unused_blocks NUMBER; l_unused_bytes NUMBER; l_last_used_extent_file_id NUMBER; l_last_used_extent_block_id NUMBER; l_last_used_block NUMBER;

Setting Table Statistics Example (2 of 4) BEGIN dbms_stats.get_table_stats ( ownname => 'GP', tabname => 'CAR', numrows => l_numrows, numblks => l_numblks, avgrlen => l_avgrlen, ); l_numrows := 0; l_numblks := 0; SELECT COUNT(*) INTO l_numrows FROM gp.car;

Setting Table Statistics Example (3 of 4) dbms_space.unused_space ( segment_owner => 'GP', segment_name => 'CAR', segment_type => 'TABLE', total_blocks => l_total_blocks, total_bytes => l_total_bytes, unused_blocks => l_unused_blocks, unused_bytes => l_unused_bytes, last_used_extent_file_id => l_last_used_extent_file_id, last_used_extent_block_id => l_last_used_extent_block_id, last_used_block => l_last_used_block ); l_numblks := l_total_blocks;

Setting Table Statistics Example (4 of 4) dbms_stats.set_table_stats ( ownname => 'GP', tabname => 'CAR', numrows => l_numrows, numblks => l_numblks, avgrlen => l_avgrlen ); END;

Setting Index Statistics Introduction Index statistics can be set manually using SET_INDEX_STATS Values can be specified for: NUMROWS - number of rows NUMBLKS - number of blocks NUMDIST - number of distinct values AVGLBLK - average leaf blocks per key AVGDBLK - average data blocks per key CLSTFCT - clustering factor INDLEVEL - index level (height) Use ANALYZE INDEX VALIDATE STRUCTURE to determine values

Setting Column Statistics Introduction Column statistics can be set manually using SET_COLUMN_STATS Values can be specified for: DISTCNT - number of distinct values DENSITY - 1 / (number of distinct values) NULLCNT - number of NULL values Low and high values Histograms Note that DISTCNT usage differs for columns with histograms

Setting Column Statistics Data Types Requires additional data types declared in DBMS_STATS package See $ORACLE_HOME/rdbms/admin/dbmsstat.sql TYPE numarray IS VARRAY(256) OF NUMBER; TYPE chararray IS VARRAY(256) OF VARCHAR2(4000); TYPE statrec IS RECORD ( EPC NUMBER, MINVAL RAW(2000), MAXVAL RAW(2000), BKVALS NUMARRAY, NOVALS NUMARRAY, CHVALS CHARARRAY, EAVS NUMBER )

Setting Column Statistics Example (1 of 3) PROCEDURE copy_col_stats (p_owner VARCHAR2,p_table_name VARCHAR2) IS l_row DBA_TAB_COLUMNS%ROWTYPE; l_statrec dbms_stats.statrec; l_numvals dbms_stats.numarray := dbms_stats.numarray(); l_charvals dbms_stats.chararray := dbms_stats.chararray(); l_datevals dbms_stats.datearray := dbms_stats.datearray(); CURSOR c1 RETURN DBA_TAB_COLUMNS%ROWTYPE IS SELECT * FROM dba_tab_columns WHERE owner = p_owner AND table_name = p_table_name; BEGIN l_numvals.extend(2); l_charvals.extend(2); l_datevals.extend(2); OPEN c1; LOOP FETCH c1 INTO l_row; EXIT WHEN c1%NOTFOUND;

Setting Column Statistics Example (2 of 3) IF l_row.num_buckets IS NOT NULL THEN l_statrec.epc := 2; l_statrec.bkvals := NULL; l_numvals(1) := NULL; l_numvals(2) := NULL; l_charvals(1) := NULL; l_charvals(2) := NULL; l_datevals(1) := NULL; l_datevals(2) := NULL; IF l_row.column_name = 'COL1' THEN l_numvals(1) := get_min_col1 (p_owner,p_table_name); l_numvals(2) := get_max_col2 (p_owner,p_table_name); DBMS_STATS.PREPARE_COLUMN_VALUES (l_statrec,l_numvals); ELSIF l_row.column_name = 'COL2' THEN l_datevals(1) := TO_DATE (v_target_date,'YYYYMMDD'); l_datevals(2) := TO_DATE (v_target_date,'YYYYMMDD'); DBMS_STATS.PREPARE_COLUMN_VALUES (l_statrec,l_datevals); ELSIF l_row.column_name = 'COL3' THEN l_charvals(1) := get_min_col3 (p_owner,p_table_name); l_charvals(2) := get_max_col3 (p_owner,p_table_name); DBMS_STATS.PREPARE_COLUMN_VALUES (l_statrec,l_charvals); END IF;

Setting Column Statistics Example (3 of 3) Setting minimum and maximum values (without histograms) DBMS_STATS.SET_COLUMN_STATS ( ownname => p_owner, tabname => p_table_name, colname => l_row.column_name, distcnt => l_row.num_distinct, density => l_row.density, nullcnt => l_row.num_nulls, srec => l_statrec, avgclen => l_row.avg_col_len ); END IF; END LOOP; CLOSE c1; END;

Setting Column Statistics - Histograms Example (1 of 2) DECLARE l_statrec dbms_stats.statrec; l_charvals dbms_stats.chararray := dbms_stats.chararray (); l_bkvals dbms_stats.numarray := dbms_stats.numarray (); BEGIN l_charvals.extend (11); l_bkvals.extend (11); l_charvals(1) := 'MSCH'; l_bkvals(1) := 91; l_charvals(2) := 'APRO'; l_bkvals(2) := 51; l_charvals(3) := 'ASEN'; l_bkvals(3) := 41; l_charvals(4) := 'NMAN'; l_bkvals(4) := 31; l_charvals(5) := 'JSTE'; l_bkvals(5) := 27; l_charvals(6) := 'NLAU'; l_bkvals(6) := 25; l_charvals(7) := 'JCLA'; l_bkvals(7) := 25; l_charvals(8) := 'NPIQ'; l_bkvals(8) := 23; l_charvals(9) := 'FALO'; l_bkvals(9) := 22; l_charvals(10) := 'DHIL'; l_bkvals(10) := 22; l_charvals(11) := 'MHAK'; l_bkvals(11) := 20;

Setting Column Statistics - Histograms Example (2 of 2) l_statrec.epc := 11; l_statrec.bkvals := l_bkvals; l_statrec.eavs := 0; DBMS_STATS.PREPARE_COLUMN_VALUES (l_statrec,l_charvals); DBMS_STATS.SET_COLUMN_STATS ( ownname => 'GP', tabname => 'CAR', colname => 'DRIVER_KEY', distcnt => 11, density => 0.00210084, -- 1 / 476 rows nullcnt => 0, srec => l_statrec, avgclen => 4 ); END; /

Partition Statistics

Partition Statistics Introduction Partition statistics are reported by: DBA_TABLES DBA_TAB_PARTITIONS DBA_TAB_STATISTICS DBA_TAB_SUBPARTITIONS (Oracle 10.1 and above) Indexes DBA_INDEXES DBA_IND_PARTITIONS DBA_IND_STATISTICS DBA_IND_SUBPARTITIONS (Oracle 10.1 and above) Columns DBA_TAB_COL_STATISTICS DBA_PART_COL_STATISTICS DBA_SUBPART_COL_STATISTICS Histograms DBA_TAB_HISTOGRAMS DBA_PART_HISTOGRAMS DBA_SUBPART_HISTOGRAMS

Partition Statistics Example (1 of 12) Create a range partitioned table containing data for last four seasons CREATE TABLE car3 PARTITION BY RANGE (season_key) ( PARTITION p2005 VALUES LESS THAN ('2006'), PARTITION p2006 VALUES LESS THAN ('2007'), PARTITION p2007 VALUES LESS THAN ('2008'), PARTITION p2008 VALUES LESS THAN ('2009') ) AS SELECT * FROM car WHERE season_key >= '2005'; CREATE INDEX car3_i1 ON car3(season_key,race_key,position) LOCAL; dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR3', estimate_percent => NULL, cascade => TRUE );

Partition Statistics Example (2 of 12) Table Statistics SELECT num_rows,blocks,avg_row_len FROM dba_tables WHERE owner = 'GP' AND table_name = 'CAR3'; NUM_ROWS BLOCKS AVG_ROW_LEN 1518 24 37 SELECT partition_name,num_rows,blocks,avg_row_len FROM dba_tab_partitions WHERE table_owner = 'GP' AND table_name = 'CAR3'; PARTITION_NAME NUM_ROWS BLOCKS AVG_ROW_LEN P2005 380 6 36 P2006 396 38 P2007 374 37 P2008 368

Partition Statistics Example (3 of 12) Index Statistics SELECT blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_indexes WHERE owner = 'GP' AND index_name = 'CAR3_I1'; BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS 1 8 1518 15 SELECT partition_name, blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_ind_partitions WHERE index_owner = 'GP' AND index_name = 'CAR3_I1'; PARTITION_NAME BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS P2005 1 2 380 3 P2006 396 P2007 374 5 P2008 368 4 Note in the above examples, AVG_LEAF_BLOCKS_PER_KEY and AVG_DATA_BLOCKS_PER_KEY were always 1so have been omitted to improve readability.

Partition Statistics Example (4 of 12) Add a new partition for 2009 season ALTER TABLE car3 ADD PARTITION p2009 VALUES LESS THAN ('2010'); Gather statistics again dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR3', estimate_percent => NULL, cascade => TRUE );

Partition Statistics Example (5 of 12) Table Statistics SELECT num_rows,blocks,avg_row_len FROM dba_tables WHERE owner = 'GP' AND table_name = 'CAR3'; NUM_ROWS BLOCKS AVG_ROW_LEN 1518 24 37 SELECT partition_name,num_rows,blocks,avg_row_len FROM dba_tab_partitions WHERE table_owner = 'GP' AND table_name = 'CAR3'; PARTITION_NAME NUM_ROWS BLOCKS AVG_ROW_LEN P2005 380 6 36 P2006 396 38 P2007 374 37 P2008 368 P2009

Partition Statistics Example (6 of 12) Index Statistics SELECT blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_indexes WHERE owner = 'GP' AND index_name = 'CAR3_I1'; BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS 1 8 1518 15 SELECT partition_name, blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_ind_partitions WHERE index_owner = 'GP' AND index_name = 'CAR3_I1'; PARTITION_NAME BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS P2005 1 2 380 3 P2006 396 P2007 374 5 P2008 368 4 P2009 Note in the above examples, AVG_LEAF_BLOCKS_PER_KEY and AVG_DATA_BLOCKS_PER_KEY were always 1so have been omitted to improve readability.

Partition Statistics Example (7 of 12) Copy table statistics from 2008 to 2009 partitions dbms_stats.copy_table_stats ( ownname => 'GP', tabname => 'CAR3', srcpartname => 'P2008', dstpartname => 'P2009' );

Partition Statistics Example (8 of 12) Table Statistics SELECT num_rows,blocks,avg_row_len FROM dba_tables WHERE owner = 'GP' AND table_name = 'CAR3'; NUM_ROWS BLOCKS AVG_ROW_LEN 1518 24 37 SELECT partition_name,num_rows,blocks,avg_row_len FROM dba_tab_partitions WHERE table_owner = 'GP' AND table_name = 'CAR3'; PARTITION_NAME NUM_ROWS BLOCKS AVG_ROW_LEN P2005 380 6 36 P2006 396 38 P2007 374 37 P2008 368 P2009

Partition Statistics Example (9 of 12) Index Statistics SELECT blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_indexes WHERE owner = 'GP' AND index_name = 'CAR3_I1'; BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS 1 8 1518 15 SELECT partition_name, blevel, leaf_blocks, distinct_keys, clustering_factor, num_rows FROM dba_ind_partitions WHERE index_owner = 'GP' AND index_name = 'CAR3_I1'; PARTITION_NAME BLEVEL LEAF_BLOCKS DIST KEYS CLUFAC NUM_ROWS P2005 1 2 380 3 P2006 396 P2007 374 5 P2008 368 4 P2009 Note in the above examples, AVG_LEAF_BLOCKS_PER_KEY and AVG_DATA_BLOCKS_PER_KEY were always 1so have been omitted to improve readability.

Partition Statistics Example (10 of 12) Copy rows from 2008 to 2009. For example: INSERT INTO car3 SELECT '2009', race_key, driver_key, team_key, engine_key, position, laps_completed, classification_key, notes, driver_points, team_points FROM gp.car3 WHERE season_key = '2008'; Statistics are unchanged

Partition Statistics Example (11 of 12) Execution plans for statement against 2008 and 2009 partitions: SELECT SUM(team_points) FROM gp.car3 WHERE season_key = '2008'; 0 SELECT STATEMENT 1 0 SORT AGGREGATE 2 1 PARTITION RANGE (SINGLE) 3 2 TABLE ACCESS (FULL) OF 'CAR3' SELECT SUM(team_points) FROM gp.car3 WHERE season_key = '2009'; 0 SELECT STATEMENT 1 0 SORT AGGREGATE 2 1 PARTITION RANGE (SINGLE) 3 2 TABLE ACCESS (BY INDEX ROWID) OF 'CAR3' 4 3 INDEX (RANGE SCAN) OF 'CAR3_I1' Plans are different even though statistics and data are theoretically identical

Partition Statistics Example (12 of 12) In Oracle 10.2.0.4 column values are not updated by COPY_TABLE_STATS SELECT partition_name,low_value,high_value FROM dba_part_col_statistics WHERE owner = 'GP' AND table_name = 'CAR3' AND column_name = 'SEASON_KEY'; PARTITION_NAME LOW_VALUE HIGH_VALUE P2005 0x32303035 (2005) P2006 0x32303036 (2006) P2007 0x32303037 (2007) P2008 0x32303038 (2008) P2009 Caused by bug 5643297 - only affects 10.2.0.4 Fixed in patch 7381308 includes fixes for COPY_TABLE_STATS and APPROX_GLOBAL

Partition Statistics Summary Range-partitioning Only collect statistics for partitions undergoing change Specify partition names and GRANULARITY parameter to restrict partitions that are analyzed Consider setting statistics manually for new partitions COPY_STATS has limited functionality Watch for changing high values Use DBMS_STATS NO_INVALIDATE option ANALYZE invalidates all cursors referencing table being analyzed Can result in hard-parse rates approaching 100%

Oracle 11g Enhancements

Statistics Preferences

Oracle 11g Enhancements Statistics Preferences In Oracle 10g preferences are: set using SET_PARAM reported by GET_PARAM stored in SYS.OPTSTAT_HIST_CONTROL$ In Oracle 11.1 and above preferences are: set using: SET_TABLE_PREFS SET_SCHEMA_PREFS SET_DATABASE_PREFS SET_GLOBAL_PREFS reported by GET_PREFS DBA_TAB_STAT_PREFS stored in SYS.OPTSTAT_USER_PREFS$ In Oracle 11.1 and above global preferences can be reset using RESET_GLOBAL_PREF_DEFAULTS By default SET_DATABASE_PREFS set statistics preferences for all tables in the database EXCEPT for tables owned by Oracle. Tables owned by Oracle can be included by specifying ADD_SYS => TRUE SET_SCHEMA_PREFS sets statistics preferences for all tables in the specified schema. Specifying a NULL value in SET_SCHEMA_PREFS restores all tables in the schema to the default value. Rows in OPTSTAT_USER_PREFS$ are updated, but not deleted.

Oracle 11g Enhancements Statistics Preferences Allow automatic statistics collection to be customized for individual objects For example heavily skewed data distributions Can be set at table, schema, database and global level In Oracle 10g preferences can be set for: CASCADE DEGREE ESTIMATE_PERCENT METHOD_OPT NO_INVALIDATE GRANULARITY In Oracle 11g preferences can also be set for: PUBLISH - if TRUE publish statistics; if FALSE store as pending STALE_PERCENT - threshold level at which statistics considered stale INCREMENTAL - if TRUE collect incremental global partition statistics

Oracle 11g Enhancements Statistics Preferences Delete preferences using: DELETE_TABLE_PREFS DELETE_SCHEMA_PREFS DELETE_DATABASE_PREFS Export and import preferences using: EXPORT_TABLE_PREFS / IMPORT_TABLE_PREFS EXPORT_SCHEMA_PREFS / IMPORT_SCHEMA_PREFS EXPORT_DATABASE_PREFS / IMPORT_DATABASE_PREFS

Pending Statistics

Oracle 11g Enhancements Pending Statistics In Oracle 11.1 and above statistics can be Published - current statistics used by optimizer Pending - private statistics not yet published In Oracle 11.1 statistics for a set of objects can be collected as an atomic transaction Published at same time Avoids inconsistencies PUBLISH mode can be set at the following levels Global Database Schema Table

Oracle 11g Enhancements Pending Statistics To defer publication of all statistics collected by USER1 use: dbms_stats.set_schema_prefs ('USER1','PUBLISH','FALSE'); To make pending statistics temporarily available to the optimizer use: ALTER SESSION optimizer_use_pending_statistics = TRUE; To make pending statistics permanently available to the optimizer use: dbms_stats.publish_pending_statistics ( ownname => 'USER1', tabname => NULL ); The default value for OPTIMIZER_USE_PENDING_STATISTICS is FALSE. To make all pending statistics permanently available to the optimizer use: dbms_stats.publish_pending_statistics ( ownname => NULL, tabname => NULL ); Statistics can also be published for individual tables. For example: dbms_stats.publish_pending_statistics ( ownname => 'USER1', tabname => 'T1' ); Both the NO_INVALIDATE and FORCE options can be specified for PUBLISH_PENDING_STATISTICS. Pending statistics can be deleted using DELETE_PENDING_STATS Pending statistics can be exported using EXPORT_PENDING_STATS. There is no corresponding import procedure Statistics can be published at database, schema or table level To delete pending statistics use DELETE_PENDING_STATS To export pending statistics use EXPORT_PENDING_STATS There is no corresponding import procedure

Extended Statistics

Oracle 11g Enhancements Extended Statistics Extended statistics attempt to reflect true selectivity of data There are two types: multi-column statistics expression statistics Extended statistics use the following procedures: CREATE_EXTENDED_STATS DROP_EXTENDED_STATS SHOW_EXTENDED_STATS_NAME

Oracle 11g Enhancements Multi-Column Statistics In Oracle 11g the following statistics can be gathered on multiple columns in a table (column groups): Number of distinct values Density Number of nulls Frequency histograms Multicolumn statistics Appear to work for frequency histograms <= 254 combinations of values Do not appear to work for height-based histograms > 254 combinations of values

Oracle 11g Enhancements Multi-Column Statistics SELECT COUNT(*) FROM gp.car WHERE team_key = 'FER' AND engine_key = 'FER'; COUNT(*) 1410 Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 35 (100) 1 SORT AGGREGATE 13 2 TABLE ACCESS FULL CAR 137 1781 35 (0) 00:00:01 Incorrect Cardinality

Oracle 11g Enhancements Multi-Column Statistics DECLARE l_extension_name VARCHAR2(30); BEGIN l_extension_name := dbms_stats.create_extended_stats ( ownname => 'GP', tabname => 'CAR', extension => '(team_key,engine_key)' ); END; BEGIN dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL, method_opt => 'FOR ALL COLUMNS SIZE 254 '|| 'FOR COLUMNS (TEAM_KEY,ENGINE_KEY) SIZE 254' ); END;

Oracle 11g Enhancements Multi-Column Statistics SELECT COUNT(*) FROM gp.car WHERE team_key = 'FER' AND engine_key = 'FER'; COUNT(*) 1410 Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 35 (100) 1 SORT AGGREGATE 13 2 TABLE ACCESS FULL CAR 1410 18330 35 (0) 00:00:01 Correct Cardinality

Oracle 11g Enhancements Multi-Column Statistics SELECT extension_name,extension FROM dba_stat_extensions WHERE owner = 'GP' AND table_name = 'CAR'; Extension Name Extension SYS_STUWIHETSRHT#5P210Z$AO1ON9 ("TEAM_KEY","ENGINE_KEY")

Oracle 11g Enhancements Multi-Column Statistics SELECT c.intcol#,c.col#,c.name FROM sys.col$ c, sys.obj$ o, sys.user$ u WHERE c.obj# = o.obj# AND o.name = 'CAR' AND o.owner# = u.user# AND u.name = 'GP'; INTCOL# COL# NAME 1 SEASON_KEY 2 RACE_KEY 3 DRIVER_KEY 4 TEAM_KEY 5 ENGINE_KEY 6 POSITION 7 LAPS_COMPLETED 8 CLASSIFICATION_KEY 9 NOTES 10 SYS_STUWIHETSRHT#5P210Z$AO1ON9 New Virtual Column in Table for Multi-Column Statistics

Oracle 11g Enhancements Expression Statistics CREATE OR REPLACE FUNCTION points (season_key VARCHAR2,race_key NUMBER,position NUMBER) RETURN NUMBER DETERMINISTIC IS l_result NUMBER; BEGIN l_result := CASE position WHEN 1 THEN 10 WHEN 2 THEN 8 WHEN 3 THEN 6 WHEN 4 THEN 5 WHEN 5 THEN 4 WHEN 6 THEN 3 WHEN 7 THEN 2 WHEN 8 THEN 1 ELSE 0 END; RETURN l_result; END; Note: The real algorithm is MUCH more complicated for various reasons including: changes to number of points awarded disqualifications, penalties etc differences between drivers and team championship points systems

Oracle 11g Enhancements Expression Statistics SELECT COUNT(*) FROM gp.car WHERE POINTS (season_key,race_key,position) = 10; COUNT(*) 709 Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 38 (100) 1 SORT AGGREGATE 13 2 TABLE ACCESS FULL CAR 177 4248 38 (0) 00:00:01 Incorrect Cardinality

Oracle 11g Enhancements Expression Statistics DECLARE l_extension_name VARCHAR2(30); BEGIN l_extension_name := dbms_stats.create_extended_stats ( ownname => 'GP', tabname => 'CAR', extension => '(points(season_key,race_key,position))' ); END; BEGIN dbms_stats.gather_table_stats ( ownname => 'GP', tabname => 'CAR', estimate_percent => NULL, method_opt => 'FOR ALL COLUMNS SIZE 254 '|| 'FOR COLUMNS (points(season_key,race_key,position)) SIZE 254' ); END;

Oracle 11g Enhancements Expression Statistics SELECT COUNT(*) FROM gp.car WHERE POINTS (season_key,race_key,position) = 10; COUNT(*) 709 Id Operation Name Rows Bytes Cost (%CPU) Time SELECT STATEMENT 38 (100) 1 SORT AGGREGATE 13 2 TABLE ACCESS FULL CAR 709 19143 38 (0) 00:00:01 Correct Cardinality

Oracle 11g Enhancements Incremental Statistics For partitioned tables statistics can be collected: For sub-partitions For partitions Globally Global statistics should reflect underlying partitioned statistics Prior to Oracle 11.1 calculation of global statistics required full table scan of each partition in table For many applications using range partitioning only the most recent partition is subject to change Older partitions contain historical data

Oracle 11g Enhancements Incremental Statistics For example: CREATE TABLE car4 PARTITION BY RANGE (season_key) ( PARTITION p2006 VALUES LESS THAN ('2005'), PARTITION p2007 VALUES LESS THAN ('2006'), PARTITION p2008 VALUES LESS THAN ('2007'), PARTITION p2009 VALUES LESS THAN ('2008') ) AS SELECT * FROM car WHERE season_key >= '2005'; CREATE INDEX i_car4_1 ON carp(season_key,race_key,position) LOCAL; INSERT INTO car4 SELECT * FROM car WHERE season_key BETWEEN 2006 AND 2009; Create table with partitions for four years data Insert data

Oracle 11g Enhancements Incremental Statistics Gather statistics DBMS_STATS.GATHER_TABLE_STATS ( ownname => 'GP', tabname => 'CAR4' ); Requires full table scan for each partition 2006 Full Table Scan 2007 Full Table Scan 2008 Full Table Scan 2009 Full Table Scan

Oracle 11g Enhancements Incremental Statistics Update data in one partition: UPDATE car4 SET team_points = driver_points WHERE season_key = '2009'; Gather table statistics again: DBMS_STATS.GATHER_TABLE_STATS ( ownname => 'GP', tabname => 'CAR4' ); Still requires full table scan for each partition 2006 Full Table Scan 2007 Full Table Scan 2008 Full Table Scan 2009 Full Table Scan

Oracle 11g Enhancements Incremental Statistics In Oracle 11.1 and above statistics can be gathered incrementally for partitioned tables To gather incremental statistics on a specific table INCREMENTAL and PUBLISH preferences for table must be TRUE ESTIMATE_PERCENT must be AUTO_SAMPLE_SIZE GRANULARITY must be AUTO For each partition a synopsis is created Contains data about distinct values for each column in partition Stored in SYS.WRI$_OPTSTAT_SYNOPSIS_HEAD$ SYS.WRI$_OPTSTAT_SYNOPSIS$ If a partition has not been modified synopsis can be used to calculate global statistics Synopsis must be generated for all partitions first time statistics are gathered after incremental statistics are enabled

Oracle 11g Enhancements Incremental Statistics Set INCREMENTAL preference to TRUE for table: BEGIN DBMS_STATS.SET_TABLE_PREFS ( ownname => 'GP', tabname => 'CAR4', pname => 'INCREMENTAL', pvalue => 'TRUE' ); END; / Gather table statistics again to generate synopsis for each partition DBMS_STATS.GATHER_TABLE_STATS ( ownname => 'GP', tabname => 'CAR4' ); All partitions will be scanned first time to create synopsis

Oracle 11g Enhancements Incremental Statistics Update data in one partition: UPDATE car4 SET team_points = driver_points WHERE season_key = '2009'; Gather table statistics again: DBMS_STATS.GATHER_TABLE_STATS ( ownname => 'GP', tabname => 'CARP' ); Uses synopsis for unmodified partitions Requires full table scan for modified partition 2006 Synopsis 2007 Synopsis 2008 Synopsis 2009 Full Table Scan

Thanks to the following for corrections: Greg Rahn Tony Hasler info@juliandyke.com