Download presentation
Presentation is loading. Please wait.
2
An Insiders View of How the Optimizer Works
Tang Tao Oracle University Principal Instructor
3
The following is intended to outline our general product direction
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
4
Optimizer evolution Databases became more feature rich
Optimizer evolved to be cost based CBO Rule are not enough Databases became more feature rich As environment changes Potential for plan changes RULES Reactive tuning with the use of advisors and auto jobs In the beginning there were rules CBO Optimizer proactively adapts to become self-learning Databases become more real-time, ad-hoc environments Reactive tuning not enough
5
Adaptive Query Optimization
Overview Adaptive Query Optimization Adaptive Plans Join methods Parallel distribution methods Adaptive Statistics Discovered at compile time Discovered at run time Adaptive Plans Adaptive Statistics Join Methods Parallel distribution Methods At compile time At run time
6
Adaptive Execution Plans
Adapt join methods Query: Find all of the products with a unit price of 15 that we have sold more that 1 of Two possible join methods for this query Nested Loops Hash Join
7
Adaptive Execution Plans
Adapt join methods Rows coming out of order_items table are buffered up to a point If row count is less than the threshold use nested Loops otherwise switch to hash join Alternative sub-plans are pre-computed Sub-plans stored in the cursor Stats collect inserted before join Rows buffered until final decision is made Statistics Collector Table scan Order _items NESTED LOOPS Index Scan Prod_info_ind HASH JOIN Table scan Prod_info Default Plan is a nested loops join
8
Adaptive Execution Plans
Adapt join methods Statistics collector disabled after decision is made and becomes a pass through operation Table scan Order _items NESTED LOOPS Index Scan Prod_info_ind HASH JOIN Prod_info Statistics Collector Number of rows seen in statistics collector exceeds threshold Plan switches to hash join Statistics collect disabled Plan resolved on first execution & remains the same for subsequent executions The plan is resolved after one execution and after that, it is always the same, and the statistics collector and buffering are disabled. Final Plan is a hash join
9
Adaptive Execution Plans
Displaying the default plan Explain plan command always shows default plan Example shows a nested loops join as default plan No statistics collector shown in plan
10
Adaptive Execution Plans
Displaying the final plan After the statement has completed use DBMS_XPLAN.DISPLAY_CURSOR to see the final plan selected Example shows that hash join picked at execution time Again the statistics collector is not visible in the plan
11
Adaptive Execution Plans
Displaying the full adaptive plan Full adaptive plan displayed when format parameter ‘+adaptive’ is set Example shows both the nested loops and hash join in the plan
12
Adaptive Execution Plans
Displaying plan with +adaptive & +report formats Additional information displayed on why operations are inactive can be seen with format parameter ‘+report’
13
Adaptive Execution Plans
Indicator in V$SQL New column in V$SQL IS_RESOLVED_ADAPTIVE_PLAN Indicates statement had an adaptive plan which was resolved on first execution Resolved plan is used for subsequent executions Statistics collectors and buffering is disabled
14
Adaptive Execution Plans
Reporting mode Adaptive plans are enabled by default Can be put in reporting mode OPTIMIZER_ADAPTIVE_REPORTING_ONLY Reporting mode shows what would have happened during execution in the plan Note: Reporting mode incurs the same overhead of buffering, plan size, etc. that regular mode does. We just don’t change the plan on the fly.
15
Adaptive Execution Plans
Adaptive join methods Join method decision deferred until runtime Default plan is computed using available statistics Alternate sub-plans are pre-computed and stored in the cursor Statistic collectors are inserted at key points in the plan Final decision is based on statistics collected during execution Default plan and sub-plans have a valid range for statistics collected If statistics prove to be out of range, the sub-plans will be swapped Requires buffering near the swap point to avoid returning rows to user Possible sub-plans are nested loop joins or hash joins and vice versa
16
Adaptive Execution Plans
Adaptive parallel data distribution User User connects to the database Shadow process is spawned When user issues a parallel SQL statement the shadow process becomes the Query Coordinator Parallel servers communicate among themselves & the QC using messages that are passed via memory buffers in the shared pool How the data is distributed among the processes is based on the operation being done & number of rows expected QC gets parallel servers from global pool and distributes the work to them Parallel servers - individual sessions that perform work in parallel Data distribution is necessary when parallel execution is used Distribution method decision based on operation and expected number of rows New adaptive distribution method HYBRID-HASH Statistic collectors are inserted in front of the parallel server process on the left hand side of the join If the actual number of rows is less than threshold, switch from HASH distribution to Broadcast
17
Adaptive Distribution Methods
Hybrid-HASH Join Cardinality based distribution skew common scenario Crucial for parallel join of very small data sets with very large data sets Distribution method decision based on expected number of rows New adaptive distribution method HYBRID-HASH Statistic collectors inserted in front of PX process on the left hand side of the join If actual number of rows less than threshold, switch from HASH to Broadcast Threshold number of total rows < 2x DOP Enabled by default
18
Adaptive Execution Plans
Adaptive parallel data distribution # rows returned less than threshold so rows distributed via Broadcast Query coordinator Hybrid hash join between EMP and DEPT Distribution method based on runtime stats Statistics collector inserted in front of PX processes scanning DEPT P1 P2 P3 P4 Statistics Collector threshold 2X DOP DEPT P5 P6 P7 P8 EMP Threshold is set at 2X DOP of the query
19
Adaptive Distribution Methods
Broadcast/Round Robin Hybrid hash join between EMP and DEPT Distribution method based on runtime stats If DEPT uses Broadcast - EMP uses Round-Robin DOP used is 6 Number of rows returned is 4 Broadcast used because 24 rows distributed (6 X 4)
20
Adaptive Query Optimization
Overview Adaptive Query Optimization Adaptive Plans Join methods Parallel distribution methods Adaptive Statistics Discovered at compile time Discovered at run time Adaptive Plans Adaptive Statistics Join Methods Parallel distribution Methods At compile time At run time
21
Dynamic Statistics During compilation optimizer decides if statistics are sufficient to generate a good plan or not Dynamic statistics are used to compensate for missing, stale, or incomplete statistics They can be used for table scans, index access, joins and group bys One type of dynamic statistics is dynamic sampling Dynamic statistics is a new and improved flavor of dynamic sampling. Dynamic statistics allow the optimizer to augment existing statistics to Get more accurate cardinality estimates for single table accesses joins And group by predicates.
22
Dynamic Statistics Dynamic Sampling
Dynamic sampling has a new level 11(AUTO) Decision to use dynamic sampling depends on the complexity of predicate, existing statistics and total execution time Dynamic statistics shared among queries Dynamic sampling has a new level, 11 in 12c In level 11 the optimizer automatically decides To use DS base on the complexity of the predicates The existing base statistics and the total execution Time expected for the SQL statement.
23
Dynamic Statistics SGA 5 4 2 3 1 DATA DICTIONARY
SHARED_POOL SHARED DYNAMIC STATISTICS Select * From product_information Where list_price-min_price=29 And category_id not in (11,22) And prod_name like ‘Smart%’; Table cardinality estimate for prod_info Optimizer determines plan using dynamic statistics 5 Cursor 0: Select * from product_information … SQL statement is submitted 1 Dynamic sampling occurs on small number of blocks from table Resulting dynamic statistics are stored in cache 4 Optimizer checks for existing statistics in data dictionary 2 DATA DICTIONARY Statistics found but need to be augmented due to complex predicates 3
24
Dynamic Statistics SGA 10 9 7 8 6 DATA DICTIONARY
SHARED_POOL SHARED DYNAMIC STATISTICS Select supplier_id, prod_name From product_information Where list_price-min_price=29 And category_id not in (11,22) And prod_name like ‘Smart%’; Table cardinality estimate for prod_info Optimizer determines plan using dynamic statistics 10 Cursor 0: Select supplier_id, prod_name … Different SQL statement is submitted with same predicates 6 Necessary Dynamic statistics found in shared cache 9 Optimizer checks for existing statistics in data dictionary 7 DATA DICTIONARY Statistics found but need to be augmented due to complex predicates 8
25
Adaptive Statistics Re-optimization
During execution optimizer estimates are compared to execution statistics If statistics vary significantly then a new plan will be chosen for subsequent executions based on execution statistics Re-optimization uses statistics gathered from previous executions First introduced as Cardinality Feedback in 11.2
26
Adaptive Statistics Cardinality Feedback pre 12c
Initial execution of a query is monitored Example shows initial plan is hash join between sales and customers Cardinality estimates 8X off Initial Cardinality estimates are more than 8X off
27
Adaptive Statistics Cardinality Feedback pre 12c
Execution Plan after the second execution Execution statistics used to reparse the statement on the second execution New plan shows correct cardinality estimates and a new join order Information learnt is stored in the cursor only and is lost if cursor ages out Estimates are now correct
28
Adaptive Statistics Re-optimization in 12c
Join statistics are also monitored Works with adaptive cursor sharing for statement with binds New Column in V$SQL IS_REOPTIMIZABLE Information found at execution time is persisted as SQL Plan Directives
29
Adaptive Statistics Re-optimization – indicator in V$SQL
New column in V$SQL IS_REOPTIMIZABLE Indicates that the statement will be re-parsed on the next execution
30
Adaptive Statistics Adaptive Plans & Re-optimization working together
Adapt join method example showed the join change from nested loops to hash join But cursor is also marked IS_REOPTIMIZABLE Why? Re-optimization will occur on second execution because cardinality estimates off Initial Cardinality estimates are off
31
Adaptive Statistics Adaptive Plans & Re-optimization working together
On second execution statement is re-parsed using execution statistics New child cursor created with new plan New plan changes the join order
32
Adaptive Statistics SQL Plan Directives
Directives are additional information used during optimization to generate a better plan For example, when table T1 is joined to T2 use dynamic statistics to get accurate cardinality estimate Directives are collected on query expressions not at a statement level Allows for directives to be used for multiple statements Persisted on disk in the SYSAUX tablespace Directives will be automatically maintained Managed using the new package DBMS_SPD
33
SQL Plan Directives SGA 2 3 1 DIRECTIVE CACHE SHARED_POOL
During execution cardinality estimate discovered to be wrong and directive created 3 IS_REOPTIMIZABLE = Y Directive: Use DS for customers table when column city, country are used Select * FROM customers where state =‘CA’ AND country=‘USA’; Cursor 0: Select * from customers …….. Optimizer determines plan 2 1 SQL statement is submitted Sysaux Tablespace
34
SQL Plan Directives SGA 5 6 4 SQL Directives SQL Directives
Directives periodically persisted to disk in the sysaux tablespace Extension Binds Stats Sqlid SQL Directives DIRECTIVE CACHE DIRECTIVE CACHE Directive: Use DS for customer table when column city, country are used Select * FROM customers where state=‘CA’ AND country=‘USA’; Optimizer determines new plan using execution statistics 5 Cursor 1: Select * from customers ….. IS_REOPTIMIZABLE = N SHARED_POOL Cursor 0: Select * from customers ….. Same SQL statement is submitted again 4 IS_REOPTIMIZABLE = Y This execution and subsequent execution use child cursor 1 6 Sysaux Tablespace SQL Directives
35
SQL Plan Directives SGA 9 10 7 8 SQL Directives DIRECTIVE CACHE
Extension Binds Stats Sqlid SQL Directives DIRECTIVE CACHE DIRECTIVE CACHE Select FROM customers where state=‘MA’ AND country=‘USA’; Cursor 0: Select from customers Optimizer determines plan with help of directive 9 SHARED_POOL Optimizer adds column group creation for state & country columns to next statistics gather on customer table 10 New SQL statement is submitted on customers table 7 Optimizer checks for directives on customers table and finds one on the columns state and country 8 Sysaux Tablespace
36
Summary Optimizer begins to learn from its experiences Adaptive Plans
Adapt join methods Adapt parallel distribution methods Adaptive Statistics Dynamic statistics Re-optimization SQL Plan Directives
37
2013 leader in IT Education According to Industry Analyst Firm IDC
“With an ever-expanding portfolio of tools and applications, Oracle University is strongly oriented toward the individual learner.” 2013 leader in IT Education According to Industry Analyst Firm IDC “Of the vendors evaluated, Oracle is most adept at describing the value of training at multiple parts of the technology deployment life cycle.” According to the latest IDC Marketscape report for IT Education and Training Market, Oracle University is a leading education provider. Source: “This decentralized control [of the curricula development] helps maximize the relevance of the training offering to the elements that are most useful to end users.”
38
One of the World’s Largest Training Organizations
Worldwide classrooms: 200+ in 89 countries Languages supported: 28 Courses: 2000+ Classes per week: 600+, 90+ Virtual Classes On Demand Courses: 100 and growing Students trained per year: 400,000+ We continue to have one of the largest footprints in the IT training industry and are continuing to expand our portfolio to include more curriculum areas to align with Oracle’s expanding product stack and offer more on demand training courses that will help organizations train their teams more quickly and cost-effectively. **** Countries supported: 89 Languages supported: 28 Instructor-led course titles: 2,723 Students trained per year: 300,000+ Worldwide locations: 297 In-Class training sessions per week: 600 Live Virtual Classes per week: 90 Oracle education partners: 2,479 Oracle certified professionals: 1.3 million Global education partnerships: 500+ Oracle certified professionals: 1.6+ million
39
Training & Certification for the Entire Stack
ROLE-BASED TRAINING Architects Implementers Developers Administrators Business Users WHEN AND HOW YOU NEED IT In-Class Live Virtual Class Training On Demand Private VALIDATED BY INDUSTRY-RECOGNIZED CERTIFICATIONS Training Solutions are available for Oracle’s complete hardware and software stack We coordinate development of our courses across all of our engineering teams responsible for different layers of our stack. Our training is based on real-world Oracle implementations offering you the most relevant training available for Oracle technology Oracle Certified Associate Oracle Certified Professional Oracle Certified Expert Oracle Certified Master
40
More than 2000 Courses Across 50+ Products
Database Fusion Middleware Applications Server & Storage Systems Industries Oracle Database 11g, Database 10g, 12c Real Application Clusters Grid Infrastructure Enterprise Manager for Database SQL and PL/SQL MySQL Data Warehousing Exadata and Engineered Systems Database Security Data Guard Application Express Embedded Database In-Memory Database Cache Spatial Secure Enterprise Search Java Application Server and Infrastructure Exalogic Exalytics Service-Oriented Architecture and Process Management WebCenter Business Intelligence Identity Management Developer Tools Data Integration Transaction Processing Enterprise Management Oracle Cloud Services Fusion Applications Oracle E-Business Suite PeopleSoft Enterprise Siebel JD Edwards EnterpriseOne Hyperion Primavera Application Integration Architecture User Productivity Kit Oracle CRM On Demand JD Edwards World Governance, Risk, and Compliance Management Agile Product Lifecycle Management AutoVue Enterprise Visualization Oracle Policy Automation Demantra ATG Commerce Suite Transportation Management Tutor Oracle Solaris Oracle Solaris Cluster Virtualization Linux Security Enterprise Manager Ops Center Sun Oracle Database Machine Sun Servers Sun Storage and Tape Communications Life Sciences Insurance Retail Utilities Higher Education Oracle University’s training portfolio covers every Oracle product and technology and we offer learning paths for all key roles.
41
160+ Certification Credentials Available
Products Administrators, Developers & Architects Implementers & Consultants Database PL/SQL SQL Oracle Database MySQL RAC & Grid Performance Tuning Oracle Database Enterprise Manager RAC Security Data Warehousing Middleware Java Development Java Architecture WebLogic WebCenter SOA Identity Management Cloud Application Foundation Server & Storage Systems Oracle Solaris Oracle Solaris Cluster Linux Oracle VM Exadata Exalogic Storage SPARC Pillar Axiom Applications Siebel Oracle Hyperion Fusion Applications E-Business Suite User Productivity Kit Oracle Hyperion JD Edwards Siebel CRM PeopleSoft Enterprise PeopleTools Primavera Oracle Cloud In addition to our training, industry-recognized certifications are available for administrators, developers, implementers and consultants
42
Contact Oracle University at:
Telephone: or Website: 2013 leader in IT Education education.oracle.com Training is critical component of any IT strategy – be it implementing and integrating new technology, upgrading to newer versions of your technology or migrating to your workforce now will prepare you for success in the future Oracle University Training can prepare your teams to implement and use Oracle technology resulting in: Achievement of project objectives Timely delivery of projects Reduced costs Lower risk Fewer service calls Not to mention, it will help improve retention of your highly skilled staff members.
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.