1. Oracle 11g Real Application Testing: Avoiding Performance Regressions with SQL Performance Analyzer Khaled Yagoub, Pete Belknap, Benoit Dageville, Karl Dias Database Manageability Group

2. Presentation Outline SQL Performance Analyzer Overview Common Usage Scenarios Workflow Phases SQL Performance Analyzer Interface Enterprise Manager Command Line Q/A

3. SQL Performance Analyzer: Overview Helps users predict the impact of system changes on SQL workload response time Builds different versions of SQL workload performance (i.e., SQL execution plans and statistics) Analyzes performance differences Offers fine-grained performance analysis on individual SQL Integrated with SQL tuning set, SQL plan baselines, and SQL tuning advisor to form an End-to- end solution Analysis Report Compare SQL Performance SQL plans + stats Pre-change Post-change SQL Workload (SQL text + binds)

4. Common Usage Scenarios Database upgrades and patches Database parameter changes Schema changes Statistics gathering Implementation of tuning recommendations OS/hardware changes

5. SQL Performance Analyzer: Workflow ProductionTest Capture SQL (STS) Transport STS Execute SQL Pre-change Execute SQL Post-change Compare Perf. Steps (1) (2) (3) (4) (5) (6) Reiterate (7) No Yes (7) Done? Make Change Production Change / Tuning Deployment Tuned System

6. Step 1: Capture SQL Workload SQL Tuning Set (STS) used to store & manage SQL workload STS includes complete execution context SQL Text, bind variables, parsing schema Execution plans and run-time statistics Number of executions STS can be built from AWR, Cursor Cache, user workload and existing STS Incremental capture used to populate STS from cursor cache over a time period Negligible performance overhead (<2 %) Filtering and ranking capabilities filters out undesirable SQL Production Database Cursor Cache SQL Tuning Set Incremental Capture

7. Step 2: Move SQL Workload to Test System Production Cursor Cache SQL Tuning Set Test SQL Tuning Set Export/Import Test system should resemble production (stats… ) Copy SQL tuning set to staging table (“pack”) Transport staging table to test system (datapump, db link, etc.) Copy SQL tuning set from staging table (“unpack”)

8. Step 3: Execute SQL Before Making Change Establishes SQL workload performance baseline before change Each SQL “test-executed” once in isolation with production context SQL Trial: SQL execution plans and run-time statistics captured DDL skipped Only query part of DML is executed Customizable global and per-SQL time limit to handle long running/run-away SQL Option to do Explain Plan only analysis Remote test-execute using database link (on 10.2 and up) SQL Tuning Set Fetch Next SQL SQL Performance Analyzer Execution Plan & Statistics Test Execute Save Result Trial: Before Change

9. Step 4: Execute SQL After Making Change Re-execute SQL after change Gathers new SQL execution plans and run-time statistics (SQL Trial) SQL Tuning Set SQL Performance Analyzer Trial: Before Change Fetch Next SQL Execution Plan & Statistics Test Execute Save Result Trial: After Change Completed Manually implement the planned change Database upgrade, patches Optimizer statistics refresh Schema changes Database parameter changes Validate tuning actions, e.g., SQL Profile creation, Partitioning, etc. Hardware changes, e.g., Enabling direct IO, File System to ASM conversion, etc.

10. Step 5: Compare & Analyze Performance Trail: Before Change Completed Trial: After Change Completed Compare SQL Performance SQL Performance Analyzer Analysis Report Compare performance using different metrics, e.g., Buffer Gets CPU Time Elapsed Time Optimizer Cost Weigh SQL using original number of executions SPA Report shows change impact on both workload, individual SQL Improved SQL Regressed SQL Unchanged SQL SQL with plan changes Fix regressed SQL using SQL Tuning Advisor or SQL Plan Baselines

11. SQL Performance Analyzer Report 4 1 3 2

12. SQL Performance Analyzer Report Regressed SQL Statements

13. Two recommended solutions to fix regressed SQL: Create SQL Plan Baselines: Helps revert to previous stable plans SQL Tuning Advisor: Helps explore better execution plans Step 6: Reiterate Regression Remediation

14. 1 2 Step 6: Reiterate Report before using Plan Baselines

15. 1 2 Change Accepted No Regressions after SPM fix Step 6: Reiterate Report after using Plan Baselines

16. Step 7: Production Change & Tuning Deployment Implement validated changes permanently on production Transport any tuning solution to production Export/Import SQL Plan Baselines Export/Import SQL Profiles Make schema changes Gather new optimizer statistics Change parameter values

17. Enterprise Manager Interface SQL Performance Analyzer Home Page

18. Enterprise Manager Interface Guided Workflow

19. Enterprise Manager Interface Parameter Change Workflow

20. Parameter Changes SPA Report 1 2

21. Parameter Changes SQL Tuning Advisor Recommendations

22. Parameter Changes SPA Report After Profiles Implemented Change Accepted 1

23. Command Line Interface dbms_sqlpa PL/SQL Package -- create an empty sql tuning set exec dbms_sqltune.create_sqlset(‘my_sts’); -- incrementally capture sql from the sql cache exec dbms_sqltune.capture_cursor_cache_sqlset( - sqlset_name => 'my_sts', - time_limit => 24*60*60, - repeat_interval => 5*60); -- create an analysis task Var tname varchar2(30); exec :tname := dbms_sqlpa.create_analysis_task(- sqlset_name => ‘my_sts’ – task_name => ‘my_task’);

24. Command Line Interface dbms_sqlpa PL/SQL Package -- before change execution exec dbms_sqlpa.execute_analysis_task(:tname, - execution_type=> 'test execute' ); -- make your change here -- alter session set … -- after change execution exec dbms_sqlpa.execute_analysis_task(:tname, - execution_type=> 'test execute' ); -- compare the before and after execution using buffer_gets exec dbms_sqlpa.set_analysis_task_parameter(:tname, - 'comparison_metric', 'buffer_gets' ); exec dbms_sqlpa.execute_analysis_task(task_name => :tname, - execution_type=> 'compare performance' );

25. Command Line Interface dbms_sqlpa PL/SQL Package Projected Workload Change Impact: ------------------------------------------- Overall Impact : 49.18% Improvement Impact : 61.48% Regression Impact : -12.3% SQL Statement Count ------------------------------------------- SQL Category SQL Count Plan Change Count Overall 12 9 Improved 1 1 Regressed 3 3 Unchanged 8 5 SQL Statements Sorted by their Absolute Value of Change Impact on the Workload -------------------------------------------------------------------------------------- --------------------------------------------------------------------------------------------------------- | | | Impact on | Metric | Metric | Impact | % Workload | % Workload | Plan | | object_id | sql_id | Workload | Before | After | on SQL | Before | After | Change | --------------------------------------------------------------------------------------------------------- | 26 | 8answpy3djcj5 | 61.48% | 67 | 7 | 89.55% | 68.65% | 14.11% | y | | 29 | gcfysssf6hykh | -4.1% | 3 | 7 | -133.33% | 3.07% | 14.11% | y | | 30 | 8yyjua0yhcc92 | -4.1% | 3 | 7 | -133.33% | 3.07% | 14.11% | y | | 32 | 0782mpfq27aq5 | -4.1% | 3 | 7 | -133.33% | 3.07% | 14.11% | y | ---------------------------------------------------------------------------------------------------------

26. SQL Performance Analyzer Data Dictionary Views Modified views in Oracle Database 11g: DBA{USER}_ADVISOR_TASKS: details about the advisor task DBA{USER}_ADVISOR_FINDINGS: analysis findings New views in Oracle Database 11g: DBA{USER}_ADVISOR_EXECUTIONS: metadata information for a task execution DBA{USER}_ADVISOR_SQLPLANS : SQL execution plans DBA{USER}_ADVISOR_SQLSTATS : SQL compilation and execution statistics

27. Enables identification of SQL performance changes (e.g., regressions) before end-users can be impacted Helps with any change that impacts SQL execution plans (e.g., DB upgrades, Optimizer statistics refresh, schema changes, etc.) Automates SQL performance tracking of hundreds of thousands of SQL statements – impossible to do manually Prevents side effects to database content when executing SQL. Keeps history of all experiments performed on SQL workloads. Integrated with SQL Tuning Advisor and SQL Plan Baselines for regression remediation GUI provides predefined workflows for Database upgrades and parameter changes Summary

28. A Q &

29. SQL Performance Analyzer In-house Testing SQL Performance Regression Bugs Errors + performance degradations caused by code changes SQL tuning set with 500K SQL statements Nightly runs SQL Plan Management Failures in reproducing execution plans Partitioning Advisor Verify partition recommendations SQL Tuning Advisor SQL Profiles, index recommendations SQL Repair Advisor

30. Real Application Testing: Tools of the Trade SQL Performance AnalyzerDatabase Replay What is it?Predicts SQL performance deviations before end-users can be impacted Replays real database workload on test system What Purpose?Assess impact of change on SQL response time Assess impact of change on workload throughput How it works?Executes each SQL, stored in SQL Tuning Set, in isolation using production context and then compares before and after execution plans and run-time statistics Captures workloads and replays it with production context including concurrency, synchronization & dependencies When to use?Unit testing of SQL with the goal to identify the set of SQL statements with improved/regressed performance Comprehensive testing of all sub-systems of the database server using real production workload SQL Performance Analyzer and Database Replay are complimentary