Presentation is loading. Please wait.

Presentation is loading. Please wait.

Extreme Performance with Oracle Database 11g and In-Memory Parallel Execution Maria Colgan & Thierry Cruanes.

Similar presentations


Presentation on theme: "Extreme Performance with Oracle Database 11g and In-Memory Parallel Execution Maria Colgan & Thierry Cruanes."— Presentation transcript:

1

2 Extreme Performance with Oracle Database 11g and In-Memory Parallel Execution Maria Colgan & Thierry Cruanes

3 Agenda Introduction to Parallel Execution Automatic Degree Of Parallelism Parallel Statement Queuing In Memory Parallel Execution Summary

4 Introduction to parallel execution

5 How Parallel Execution works User connects to the database User Background process is spawned When user issues a parallel SQL statement the background process becomes the Query Coordinator QC gets parallel servers from global pool and distributes the work to them Parallel servers - individual sessions that perform work in parallel Allocated from a pool of globally available parallel server processes & assigned to a given operation Parallel servers communicate among themselves & the QC using messages that are passed via memory buffers in the shared pool

6 Parallel Servers do majority of the work Query Coordinator Parallel Execution PlanIDOperationNameTQIN-OUT PQ Distribution 0 SELECT STATEMENT 1 PX COORDINATOR 2 PX SEND QC {RANDOM} Q1,01P->S 3 HASH JOIN Q1,01PCWP 4 PX RECEIVE Q1,01PCWP 5 PX SEND BROADCAST Q1,01P->PBROADCAST 6 PX BLOCK ITERATOR Q1,01PCWP 7 TABLE ACCESS FULL CUSTOMERSQ1,01PCWP 8 PX BLOCK ITERATOR Q1,01PCWP 9 TABLE ACCESS FULL SALESQ1,01PCWP SELECT c.cust_name, s.purchase_date, s.amount FROM sales s, customers c WHERE s.cust_id = c.cust_id; SELECT c.cust_name, s.purchase_date, s.amount FROM sales s, customers c WHERE s.cust_id = c.cust_id;

7 Parallel Execution of a Query SELECT c.cust_name, s.date, s.amount FROM sales s, customers c WHERE s.cust_id = c.cust_id; SELECT c.cust_name, s.date, s.amount FROM sales s, customers c WHERE s.cust_id = c.cust_id; Producers Consumers

8 Producers Producers Consumers Consumers Query Coordinator Producers and Consumer in the execution planIDOperationNameTQIN-OUT PQ Distribution 0 SELECT STATEMENT 1 PX COORDINATOR 2 PX SEND QC {RANDOM} Q1,02P->S 3 HASH JOIN Q1,02PCWP 4 PX RECEIVE Q1,02PCWP 5 PX SEND HASH Q1,00P->P 6 PX BLOCK ITERATOR Q1,00PCWP 7 TABLE ACCESS FULL CUSTOMERSQ1,00PCWP 8 PX RECEIVE Q1,02PCWP 9 PX SEND HASH Q1,01P->P 10 PX BLOCK ITERATOR Q1,01PCWP 11 TABLE ACCESS FULL SALESQ1,01PCWP

9 Parallel Execution of a Scan Data is divided into Granules – block range or partition Each Parallel Server is assigned one or more Granules No two Parallel Servers ever contend for the same Granule Granules are assigned so that the load is balanced across all Parallel Servers Dynamic Granules chosen by the optimizer Granule decision is visible in execution plan

10 Identifying Granules of Parallelism during scans in the plan

11 Enabling Parallel Execution There are three ways to enable parallel Execution 1.Enable the table(s) for parallel execution: alter table sales parallel ; alter table customers parallel ; 2.Use a parallel hint select /*+ parallel(c) parallel(s) */ c.state_province, sum(s.amount) revenue from customers c, sales s where s.customer_id = c.id and s.purchase_date=to_date('01-JAN-2007','DD-MON-YYYY') and c.country = 'United States' group by c.state_province; 3.Use alter session force parallel query ;

12 Controlling Parallel Execution on RAC Use RAC Services Create two services Srvctl add service –d database_name -s ETL -r sid1, sid2 Srvctl add service –d database_name -s AHOC -r sid3, sid4 ETL Ad-Hoc queries Note:New Parameter to force a parallel statement to run on just node the query was issued on called PARALLEL_FORCE_LOCAL default FALSE

13 How could we enhance Parallel Execution? Current Issues Difficult to determine ideal DOP for each table without manual tuning One DOP does not fit all queries touching an object Not enough PX server processes can result in statement running serial Too many PX server processes can thrash the system Only uses IO resources Solution Oracle automatically decides if a statement –Executes in parallel or not and what DOP it will use –Can execute immediately or will be queued –Will take advantage of aggregated cluster memory or not

14 Automatic Degree Of Parallelism

15 Automatic Degree of Parallelism Business Requirement Parallelism is completely manual – Tuning typically required to determine ideal DOP – Generally reserved for well-defined workload (large SQL) – One DOP does not fit all queries touching an object Solution Oracle automatically decides if a statement –Executes in parallel or not –What DOP the statement will use

16 Automatic Degree of Parallelism Auto DOP Statement with elapse times less than the threshold go serial Statement with elapse times greater than threshold are candidates for Parallel Optimizer derives the DOP for the statement based on resource requirements for all scans operations Applies to all types of statements Query, DML, or DDL Explain plan has been enhanced to show DOP selected

17 How Auto Degree of Parallelism works SQL statement Statement is hard parsed And optimizer determines the execution plan Statement executes in parallel Actual DOP = MIN( PARALLEL_DEGREE_LIMIT, ideal DOP) Statement executes serially If estimated time less than threshold Optimizer determines ideal DOP If estimated time greater than threshold

18 Controlling Auto DOP Controlled by two init.ora parameters: – PARALLEL_DEGREE_POLICY Controls whether or not auto DOP will be used Default is MANUAL which means no Auto DOP Set to AUTO to enable auto DOP – PARALLEL_MIN_TIME_THRESHOLD controls which statements are candidate for parallelism Default is 10 seconds

19 Controlling Auto DOP Statement with an elapse time less than PARALLEL_MIN_TIME_THRESHOLD run serial Statement above threshold are candidate for parallelism Maximum DOP controlled by PARALLEL_DEGREE_LIMIT – Default value is CPU PARALLEL_THREADS_PER_CPU X CPU_COUNT Actual DOP = MIN( PARALLEL_DEGREE_LIMIT, ideal DOP) Statements excluded from auto-dop – SQL issued from PL/SQL system packages Index rebuild, statistics collection etc – Typically PL/SQL packages control their own parallelism

20 Explain plan enhancement for Auto DOP Plan hash value: IDOperationNameRowsBytesCostTimePstartPstop 0 Select Statement :00:01 1 PX COORDINATOR 2 PX SEND QC (RANDOM) :TQ :00:01 3 PX BLOCK ITERATOR :00:01 4 Table Access Full Sales :00: PLAN_TABLE_OUTPUT Note - Computed Degree of Parallelism is 16 because of parallel threshold

21 Parallel Statement Queuing

22 Business Requirement With the introduction of Auto DOP – More statements will run in parallel – Possible to exhaust all parallel execution server processes – Potential system thrashing due to too many processes Solution Parallel Statement Queuing –Oracle automatically decides if a statement can execute immediately or not –Prevents serializing parallel queries when parallel servers are not available –Prevents system thrashing

23 Parallel Statement Queuing When a parallel statement starts checks if PX servers are available – Let it run if there are enough PX servers available – Queue the statement if there are not enough PX servers available Monitors RAC-wide availability of PX servers – Adaptive to dynamic environments Services – The service your session belongs to determines the limits on queuing Cluster reconfiguration – Queue is aware of nodes leaving and joining the cluster and adjusts the limits accordingly

24 How Parallel Statement Queuing Works SQL statements Statement is parsed and oracle automatically determines DOP If enough parallel servers available execute immediately If not enough parallel servers available queue the statement FIFO Queue When the required number of parallel servers become available the first stmt on the queue is dequeued and executed

25 Controlling Parallel Statement Queuing Enabled when PARALLEL_DEGREE_POLICY is set to AUTO The Statement queue is enforced with a strict FIFO policy PARALLEL_SERVER_TARGET indicates how many PX servers are available to run queries before queuing kicks-in – Default values 4 X PARALLEL_THREADS_PER_CPU X CPU_COUNT – This a soft limit and does not replace PARALEL_MAX_SERVERS PX server available to run queries before queuing kicks in Total PX servers available On an 8 CPU system CPU Count Parallel Server Target Parallel Max Server

26 Controlling Parallel Statement Queuing Two new hints – To by-passes parallel statement queuing SELECT /*+ NO_STMT_QUEUING */ col1 FROM foo; – To delay statement execution until resources are available without having PARALLEL_DEGREE_POLICY is set to AUTO SELECT /*+ STMT_QUEUING */ col1 FROM foo; V$SQL_PLAN_MONITOR has a new status value for SQL that is queued SELECT s.sql_id, s.sql_text FROM v$SQL_MONITOR m, v$SQL s WHERE m.status='QUEUED AND m.sql_id = s.sql_id; Two new wait events – PX Queuing: Statement queue Indicates the first query in the queue – ENQ JX SQL statement queue All other queries in the queue wait on this enqueue

27 Monitoring Statement Queuing in EM Awaiting screen shot from EM Clock symbol indicated a queued statement Click on the SQL ID for more info

28 Monitoring Statement Queuing in EM Wait event indicates stmt is at the head of the queue

29 Monitoring Statement Queuing in EM Wait event indicates stmt is queued

30 In-Memory Parallel Execution

31 Business Requirement Traditionally Parallel Execution takes advantage of the IO capacity of a system Disk speeds are not keeping up with Moores law while CPU and Memory are Solution In-Memory Parallel Execution harness the memory capacity of the entire system Scan data nearly 10 X faster than scanning from disk

32 Prior to Oracle Database 11gR2 Parallel Execution and the buffer cache

33

34 How In-Memory Parallel Execution Works Detect if the object fits in the aggregated buffer cache of the cluster – If so, distribute & affinitizes the blocks among the nodes and make PQ aware of the affinity – If not, continue to by-pass the buffer cache and read directly from disk Subsequent access to the object will be conducted only by PX servers on the node to each the data was affinitized

35 In-Memory Parallel Execution

36 How In-Memory Parallel Execution Works in detail Decision to use the buffer cache is based on set of heuristics including – Ratio between buffer cache size and object size – Frequency at which the object is accessed – How much the object changes between accesses In RAC fragments of the object are affinitized in the buffer cache on each of the active instances – Affinity is based on FileNumber and ExtentNumber unless hash partitioned – Automatically prevents multiple instances reading the same data from disk – Only PX process on the same RAC node can access each of the fragments of the object

37 How In-Memory Parallel Execution Works SQL statement Determine the size of the table being looked at Table is extremely Large Always use direct read from disk Table is a good candidate for In-Memory Parallel Execution Fragments of Table are read into each nodes buffer cache Only parallel server on the same RAC node will access each fragment Read into the buffer cache on any node Table is extremely small

38 Controlling In-Memory Parallel Execution Controlled by PARALLEL_DEGREE_POLICY – Active only when set to AUTO – No way to turn it off

39 Summary

40 New Parallel Execution Init.ora Parameters ParameterValueDescription PARALLEL_DEGREE_LIMITCPU Max DOP that can be selected with AUTO DOP PARALLEL_DEGREE_POLICYMANUAL Specifies if AUTO DOP, Queuing, & In-memory PE will be enabled PARALLEL_FORCE_LOCALFALSE Restricts parallel server processes to the node where query is issued PARALLEL_MIN_TIME_THRESHOLDAUTO Specifies min execution time a statement should have before AUTO DOP will kick in PARALLEL_SERVERS_TARGET 4*CPU_COUNT* PARALLEL_THREAD S_PER_CPU * ACTIVE_INSTANCES Specifies # of parallel processes allowed to run parallel stmts before queuing will be use PARALLEL_EXECUTION_MESSAGE_SIZE16KB Specifies size of the message buffers used for communication

41 Gradually introduce Auto Parallel Execution PARALLEL_DEGREE_POLICY has three possible modes Manual –As before, DBA must manually specify all aspects of parallelism –No AUTO DOP, Stmt Queuing, In-Memory Parallel Execution – Useful for well-understood existing applications Limited –Restricted AUTO DOP for queries with tables decorated with default PARALLEL –No Stmt Queuing, In-Memory Parallel Execution –Useful in a mixed-world environment when a limited number of statements would benefit from parallel execution Auto –All qualified statements subject to executing in parallel –Statements can be queued –IN-memory PQ available –Useful when deploying new applications in 11g that would benefit from parallel execution

42 For More Information search.oracle.com Parallel Execution Or

43 The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracles products remains at the sole discretion of Oracle.

44


Download ppt "Extreme Performance with Oracle Database 11g and In-Memory Parallel Execution Maria Colgan & Thierry Cruanes."

Similar presentations


Ads by Google