The Self-managing Database: Automatic Performance Diagnosis Graham Wood Kyle Hailey Oracle Corporation Session id: 40092

Problem Definition  Performance Diagnosis & Tuning is complex  Diagnosis often requires additional data capture  Database wide view of operations is lacking  Data overload rather than information  Misguided tuning efforts waste time & money

Problem Solution: Oracle10g  Performance Diagnosis & Tuning are complex automated problem diagnosis  Diagnosis often requires additional data capture complete, lightweight capture of workload data  Database wide view of operations is lacking holistic time based analysis  Data overload rather than information reports top problems and solutions  Misguided tuning efforts reports non-problem areas

Intelligent Infrastructure Application & SQL Management System Resource Management Space Management Backup & Recovery Management Storage Management Database Control Database Management Oracle Database 10g – Self-Managing Database

Intelligent Infrastructure  Automatic Workload Repository – “Data Warehouse” of the Database – Code instrumentation  Automatic Maintenance Tasks – Pre-packaged, resource controlled  Server-generated Alerts – Push vs. Pull, Just-in-time, Out-of-the-box  Advisory Infrastructure – Integrated, uniform Intelligent Infrastructure Application & SQL Management System Resource Management Space Management Backup & Recovery Management Storage Management Database Management Automatic Workload Repository Automatic Maintenance Task Infrastructure Server-generated Alert Infrastructure Advisory Infrastructure

Automatic Database Diagnostic Monitor (ADDM)  Performance Diagnostic engine in the database  Automatically diagnoses performance problems  Provides Root Cause Analysis with recommended solutions  Identifies non-problems areas  Integrates all components Intelligent Infrastructure Application & SQL Management System Resource Management Space Management Backup & Recovery Management Storage Management Database Management Proactive and effective tuning

Performance Monitoring Solutions Snapshots ADDM ADDM Results Alerts In memory statistics Workload Repository SGA Reactive Monitoring Proactive Monitoring

Automatic Workload Repository (AWR) a.k.a. Statspack++ Server captures workload data Every 30 minutes, or manually Efficient capture Self manages space requirements Saves data for 7 days by default

Automatic Workload Repository (AWR) Classes of Data  BASE STATISTICS e.g. physical reads  SQL STATISTICS e.g. disk reads (per sql stmt)  METRICS e.g. physical reads / sec  ACTIVE SESSION HISTORY e.g. sid : 10 event : db file sequential read file# : 33, block# : 209, obj# : 19 time : μs

New Base Statistics Extensive code instrumentation  Time Model (v$sys_time_model) –Db time –Connection Management (logon, logoff) –Parse (hard, soft, failed,..) –SQL, PLSQL and Java execution times  Wait Model (v$system_event) – 700 different wait events – 12 wait classes  OS Stats (v$osstat) –CPU + Memory SQL Exec PLSQL Exec Conn Mgmt Parse Java Exec

New SQL Statistics  SQL_id – more unique hash value  SQL statement statistics – Wait class time – PLSQL time – Java time  Sampled bind values (v$sql_bind_capture)  Efficient top SQL identification using Δs in the kernel, by 6 dimensions: –CPU –Elapsed –Parse –...

Active Session History (ASH) Sampled history of v$session_wait Samples active sessions every second into memory (v$active_session_history) Direct access to kernel structures Selected samples flushed to AWR Data captured includes: – SID – SQL ID – Program, Module, Action – Wait event# – Object, File, Block – actual wait time (if captured while waiting)

Performance Monitoring Solutions Snapshots ADDM ADDM Results Alerts In memory statistics Workload Repository SGA Reactive Monitoring Proactive Monitoring

Automatic Database Diagnostic Monitor (ADDM)  Performance expert in a box  Integrate all components together  Automatically provides database-wide performance diagnostic, including RAC  Real-time results using the Time Model  Provides impact and benefit analysis  Provides Information vs. raw data  Runs proactively Intelligent Infrastructure Application & SQL Management System Resource Management Space Management Backup & Recovery Management Storage Management Database Management

ADDM’s Architecture SQL Advisor High-load SQL IO / CPU issues RAC issues Automatic Diagnostic Engine Snapshots in Automatic Workload Repository Automatic Diagnostic Engine System Sizing Advice Network + DB config Advice  Uses Time & Wait Model data from Workload Repository  Classification Tree is based on decades of Oracle performance tuning expertise  Time based analysis  Recommends solutions or next steps  Runs proactively & manually

ADDM Methodology Top down analysis of where time is spent  Period Analysis using AWR snapshots  Throughput centric  Focus on reducing time ‘DB time’  Time based quantification  Problems with impact  Recommendations with benefit

ADDM Methodology Problem classification system Decision tree based on the Wait Model and Time Model Stats …… System Wait RAC Waits IO Waits Concurrency …… Buffer Busy Parse Latches Buf Cache latches …… Root CausesSymptoms

ADDM Methodology Problem classification system Decision tree based on the Wait Model and Time Model Stats …… System Wait RAC Waits IO Waits Concurrency …… Buffer Busy Parse Latches Buf Cache latches …… Non - Problems areas.

Top Performance Issues - Top SQL - IO Issues -Bandwidth, Hot Files - Parsing - hard, soft, failed - Configuration issues - Log file sizing - Log buffer sizing - Archiving - MTTR settings. - Application usage Not rocket science anymore

Top Performance Issues - Excessive Logon/Logoff - Undersized memory -SGA, PGA - Hot Blocks & Objects with SQL -buffer busy waits -cache buffer chain latches - RAC service issues - network, LMS, remote instance - Locks & ITL contention with object & SQL - Checkpoint causes - PL/SQL, Java time Not diagnosable using Statspack data

ADDM Output  Set of Findings with impact – Root cause – Symptoms – Non-problem areas  Recommendations with benefit and rationale  Inference Path of the analysis  Output in Advisor Framework  Externalized through EM screens or ADDM report

Database Home Page

ADDM Findings

ADDM Recommendations

Performance Diagnostic: Before and Now Before  Examine system utilization  Look at wait events  Observe latch contention  See wait on shared pool and library cache latch  Review v$sysstat  See “parse time elapsed” > “parse time cpu” and #hard parses greater than normal  Identify SQL by..  Identifying sessions with many hard parses and trace them, or  Reviewing v$sql for many statements with same hash plan  Examine objects accessed and review SQL  Identify “hard parse” issue by observing the SQL contains literals  Enable cursor sharing Oracle10 G  Review ADDM recommendations  ADDM recommends use of cursor_sharing Scenario: Hard parse problems

Manually running ADDM  Manual snapshot automatically invokes ADDM  ADDM Analysis across any 2 snapshots

Performance Monitoring Solutions Snapshots ADDM ADDM Results Alerts In memory statistics Workload Repository SGA Reactive Monitoring Proactive Monitoring

Reactive Monitoring Overview  Reactive monitoring may still be necessary –User calls up –Real time problem diagnosis –Validate ADDM diagnosis –When an alert is raised  Uses new AWR data sources  Integrates graphical displays with ADDM  Oracle provides an integrated performance management console using all relevant data sources

EM Product Layout for Performance Database Home Page Database Performance Page Drilldowns SQL Session

EM Pages Layout Home Page Perf Page Top Session Wait Detail Top SQL SQL Detail Session Detail ADDM ADDM Details

Buffer Busy Waits Case Study

Home Page Perf Page Top Session Wait Detail Top SQL SQL Detail Session Detail ADDM ADDM Details Two Paths

ADDM Path Home Page Perf Page Top Session Wait Detail Top SQL SQL Detail Session Detail ADDM ADDM Details

Database Home Page

ADDM Home Home Page Perf Page Top Session Wait Detail Top SQL SQL Detail Session Detail ADDM ADDM Details

ADDM Home

ADDM Details Home Page Perf Page Top Session Wait Detail Top SQL SQL Detail Session Detail ADDM ADDM Details

Home Page Perf Page Top Session Wait Detail Top SQL SQL Detail Session Detail ADDM ADDM Details Manual Path

Database Home Page

Performance Page Home Page Perf Page Top Session Top SQL Wait Detail SQL Detail Session Detail ADDM ADDM Details

Performance Page

Performance Page highlight

Wait Drill Down Home Page Perf Page Top Session Wait Detail Top SQL SQL Detail Session Detail ADDM ADDM Details

Wait Drill Down

Wait Drill Down highlight

Wait Drill Down

Wait Drill Down highlight

Wait Drill Down – Top SQL

SQL Details Home Page Perf Page Top Session Wait Detail Top SQL SQL Detail Session Detail ADDM ADDM Details

SQL Details

Problem Solution: Oracle10g  Performance Diagnosis & Tuning are complex ADDM performs automated problem diagnosis  Diagnosis often requires additional data capture AWR performs capture of workload data  Database wide view of operations is lacking ADDM performs holistic time based analysis  Data overload rather than information EM reports ADDM findings and solutions  Misguided tuning efforts ADDM reports non-problem areas

Conclusion  Oracle 10g revolutionizes performance management – Built in automatic diagnostic engine – Extensive code instrumentation – Automatic collection of workload information – Proactive performance diagnostics and recommendations  The new Enterprise Manager provides an integrated performance management console using all relevant data sources

Next Steps….  Recommended hands-on labs – Oracle Database 10g : Manage the Oracle Environment Hands-On Lab  Campground Demos – Self-Managing Database : Easy Upgrade – Self-Managing Database:Invisible Installation & Deployment – Self-Managing Database: Proactive Performance Management – Self-Managing Database: Automatic Memory Management – Self-Managing Database: Proactive Space Management  Relevant web sites to visit for more information –

Next Steps….  Recommended sessions – The Self-Managing Database: Guided Application & SQL Tuning (Tuesday, 3:30 PM) – The Self-Managing Database: Automatic SGA Memory Management (Tuesday, 5:00 PM) – The Invisible Oracle: Deploying Oracle Database in Embedded Environment (Wednesday, 4:30 PM) – The Self-Managing Database: Proactive Space and Schema Object Management (Thursday, 8:30 AM) – The Self-Managing Database: Automatic Health Monitoring (Thursday, 11 AM)

A Q & Q U E S T I O N S A N S W E R S

Reminder – please complete the OracleWorld online session survey Session id: Thank you.