1. SQLd360 SQL diagnostic made easy a.k.a. Tune SQLs 21st century style
Mauro Pagano

2. Mauro Pagano Consultant & Developer Oracle  Enkitec  Accenture
DBPerf and SQL Tuning
Training, Conferences and Workshops
Tools (SQLT, SQLd360, TUNAs360, Pathfinder)
Blogger & Oracle ACE

3. Trivia! History quiz about Oracle RDBMS!
Was any feature introduced since 1998(*)?
Did any of them has to do the optimizer?
How much has the optimizer changed since?
How much more complex has the CBO become?
One answer to all questions => YES! A LOT
(*) Oracle 8i release date

4. The last 20 years of Optimizer
Shift to cost-based optimization
Instead of heuristic-based
Introduced statistics necessity
Query Transformations
Made optimizer way more sophisticated (complex)
Stability vs flexibility challenge
Conservative way, ensure plans NOT change
Progressive way, plans DO change

5. My SQL runs slow… This is 1998 SQL Tuning!!!!
How to approach a SQL tuning challenge?
“Classic” 3 steps approach:
Enable tracing events (10053 & 10046)
Execute the SQL again
Review trace files
This is 1998 SQL Tuning!!!!

6. Seriously? That’s your approach?
By “classic” I meant old! 
10053&10046 has many limitations
Need to be put in place BEFORE exec starts
No after-the-fact investigation
Hard to review and digest
Have you ever read a 2M lines long 10053?
Provide limited info about surrounding env
You need to reconnect & gather missing pieces!
5/23/2019
Enkitec ©

7. What can be improved? A lot more info needed but: Need a new way to
Not trivial to collect
Time sensitive
Some are pieces you just don’t know about
Need a new way to
Collect everything in one shot
Be able to execute it anytime
5/23/2019
Enkitec ©

8. Flash forward to 2007 Meet SQLTXPLAIN a.k.a. SQLT
Free tool from Oracle
Authored by the one and only Carlos Sierra
Designed by Support, for Support
Can be executed at any time
Collects everything around a SQL statement
5/23/2019
Enkitec ©

9. So SQLT is the perfect tool?
SQLT still has some limitations
Requires installation, two schemas
Presents a lot of RAW info, hard to digest
Main focus is plan generation, not much on exec
Heavily depends on rowsource info
This is 2007 SQL Tuning!!!!
5/23/2019
Enkitec ©

10. What’s wrong with 2007? Oracle RDBMS 11g was released
Some decisions deferred to exec time
Plan started to diverge from the one “on paper”
New features changed SQL Tuning forever
SQL Monitoring (got better in 11.2)
ASH data collected by Execution Plan Line
5/23/2019
Enkitec ©

11. Can we do better than SQLT?
Not on plan generation, SQLT still rocks
But most of the time that’s Support role, not ours
Yes on data presentation, use charts to
Consume large amount of info quickly
Hide the complexity of some information
Allow for trend and pattern recognition
5/23/2019
Enkitec ©

12. Really? Better than SQLT?
5/23/2019
Enkitec ©

13. What else can we do? Leverage new features from Oracle
SQL Monitor removes dependency on 10046
Allow to better focus on SQL execution
Also historical from 12c onwards
ASH by Plan Line allows to:
Group info by plan line, not possible with 10046
Identify bottlenecks with no SQL Mon nor 10046
5/23/2019
Enkitec ©

14. Meet the 2016 runaway-approved SQL Tuning tool
5/23/2019
Enkitec ©

15. Meet SQLd360 FREE! No installation Can be executed any time
Single step execution
Extensive use of visualization
Leverage existing Oracle features
5/23/2019
Enkitec ©

16. SQLd360 use Google Charts Present large amount of info as charts
Easy to consume and spot trends / patterns
FREE, no license required
Libraries are downloaded from Google
Similar to SQL Monitor or PerfHub
Data is kept local, not sent to Google

17. How to run it Download SQLd360 (my blog or github)
Refer to readme.txt 
Connect to SQL*Plus as any DBA or user with access to Data Dictionary
Parameters
SQL ID
Oracle Tuning or Diagnostics Pack? [T | D | N]

18. What’s the output? Single ZIP file Thousands of files
Self-consistent, allows offline analysis
Only Metadata extracted, no application data
Thousands of files
Each small and easy to consume
Index.html drives navigation
Reports with different granularity
Specific “drill-down” pages
5/23/2019
Enkitec ©

19. 00001_sqld360_dbnamehash_sqlid_index.html

20. Source
Report Data
DESC of source table
Query
Copy&Paste

21. You are given the execution plan for the SQL.
5/23/2019
You are given the execution plan for the SQL.
Are you comfortable in reading this plan?
Can you tell me why it was slow yesterday from this info?
5/23/2019
Enkitec ©
Enkitec (c)

22. Bind peeking, data distrib on histogram
5/23/2019
Bind peeking, data distrib on histogram
1998 SQL Tuning (aka traces)
Trace each exec and collect histgram info manually
2007 SQL tuning (aka SQLT)
Provides every info necessary
Straightforward to spot distribution
But still requires to read a table of up to 2048 rows
5/23/2019
Enkitec ©
Enkitec (c)

23. One chart per histogram created automatically
Easier this way? 
One chart per histogram created automatically
Cardinality, selectivity and endpoint conversion available with mouse-over
5/23/2019
Enkitec ©

24. SQL Y breaks SLA since Monday
5/23/2019
SQL Y breaks SLA since Monday
1998 SQL Tuning (aka traces)
Sorry, can’t help you
2007 SQL tuning (aka SQLT)
Plan stability issue, second plan is takes 909 secs
PLAN_HASH_VALUE EXECS AVG_BUFFER_GETS AVG_ELAPSED_TIME_SECS
“But my SLA is 1000secs!!!!”
and PHV 3716… was used before Monday
5/23/2019
Enkitec ©
Enkitec (c)

25. Old approaches make it hard
5/23/2019
Old approaches make it hard
2007 SQL tuning (aka SQLT)
Ops, here is the data, figure it out
5/23/2019
Enkitec ©
Enkitec (c)

26. Easier this way?  Plan performance evaluation needs time dimension!
Average elapsed time/exec over time per PHV
Plan performance evaluation needs time dimension!
Executions started to take longer and longer over time, even original plan degraded.
Likely not a plan issue
5/23/2019
Enkitec ©

27. CBO used this plan, it seemed slow
5/23/2019
CBO used this plan, it seemed slow
| Id | Operation | Name | E-Rows |E-Bytes|E-Temp | Cost (%CPU)| E-Time | | 0 | SELECT STATEMENT | | | | | (100)| | |* 1 | FILTER | | | | | | | |* 2 | HASH JOIN | | | 8738K| 2792K| (1)| 00:02:22 | | 3 | TABLE ACCESS BY INDEX ROWID | MTL_SYSTEM_ITEMS_B | | 1831K| | 9762 (1)| 00:01:58 | |* 4 | INDEX RANGE SCAN | MTL_SYSTEM_ITEMS_B_N10 | | | | 277 (1)| 00:00:04 | |* 5 | COUNT STOPKEY | | | | | | | | 6 | INDEX FULL SCAN | MTL_PARAMETERS_N1 | 1 | 4 | | 1 (0)| 00:00:01 | |* 7 | TABLE ACCESS FULL | SPRN_INV_ITEM_IN_INT | | 4423K| | 1608 (2)| 00:00:20 | | 8 | NESTED LOOPS | | 1 | 37 | | 643 (1)| 00:00:08 | | 9 | NESTED LOOPS | | 1 | 37 | | 643 (1)| 00:00:08 | | 10 | NESTED LOOPS | | 1 | 20 | | 642 (1)| 00:00:08 | |* 11 | TABLE ACCESS FULL | QP_LIST_LINES | 1 | 9 | | 641 (1)| 00:00:08 | |* 12 | TABLE ACCESS BY INDEX ROWID| QP_LIST_HEADERS_B | 1 | 11 | | 1 (0)| 00:00:01 | |* 13 | INDEX UNIQUE SCAN | QP_LIST_HEADERS_B_PK | 1 | | | 0 (0)| | |* 14 | INDEX UNIQUE SCAN | QP_LIST_HEADERS_TL_PK | 1 | | | 0 (0)| | |* 15 | TABLE ACCESS BY INDEX ROWID | QP_LIST_HEADERS_TL | 1 | 17 | | 1 (0)| 00:00:01 | | 16 | SORT AGGREGATE | | 1 | 20 | | | | |* 17 | FILTER | | | | | | | |* 18 | TABLE ACCESS FULL | SPRN_INV_ITEM_IN_INT | 3 | 60 | | 1588 (1)| 00:00:20 | |* 19 | TABLE ACCESS FULL | QP_LIST_LINES | 1 | 10 | | 641 (1)| 00:00:08 |
You are given the execution plan for the SQL.
Are you comfortable in reading this plan?
Can you tell me why it was slow yesterday from this info?
5/23/2019
Enkitec ©
Enkitec (c)

28. Old approaches make it hard
1998 SQL Tuning (aka traces)
Maybe can’t go back and run the SQL again
Maybe run it again and it runs fine
2007 SQL tuning (aka SQLT)
Plan looks questionable but don’t have A-Rows
Need to restore SQLT repo and write SQLs
5/23/2019
Enkitec ©

29. Old approaches make it hard
2007 SQL tuning (aka SQLT)
You have the data, figure it out …
SELECT sql_plan_hash_value, count(*), count(distinct sql_exec_id)
FROM dba_hist_active_sess_history
WHERE sql_id = ‘…’
GROUP BY sql_plan_hash_value
SELECT sql_exec_id, sql_exec_start, count(*)
AND sql_plan_hash_value = …
GROUP BY sql_exec_id, sql_exec_start
<<another 10 SQL statements>>
5/23/2019
Enkitec ©

30. Easier this way? 
Exec plan reported as a tree
Just read it left to right
Tree available also for individual executions, easy to diagnose random slowness
Nodes are colored depending on time sampled on them. Easy to spot bottlenecks
5/23/2019
Enkitec ©

31. What’s the impact of SQL X on DB A?
1998 SQL Tuning (aka traces)
Can’t quite do that
Would require to trace every single SQL
2007 SQL tuning (aka SQLT)
Focused on individual SQL, not much on overall
Nothing wrong but makes it hard to scope impact
5/23/2019
Enkitec ©

32. Significant part of the workload, peaks up to 30%
Easier this way? 
Significant part of the workload, peaks up to 30%
5/23/2019
Enkitec ©

33. System has 10 CPUs, up to 18 sessions demanding CPU
5/23/2019
Easier this way? 
System has 10 CPUs, up to 18 sessions demanding CPU
5/23/2019
Enkitec ©
Enkitec (c)

34. SQL runs slow, must be plan issue
1998 SQL Tuning (aka traces)
Need tracing every execution
Might translate in 100s of files to review
2007 SQL tuning (aka SQLT)
Provides every info about the plan and exec
Main focus on the plan, up to you to figure out
5/23/2019
Enkitec ©

35. Easier this way? 
Up to 70% on Cluster, plan might be secondary issue, investigate app first
5/23/2019
Enkitec ©

36. Performance broke, not sure why
1998 SQL Tuning (aka traces)
Can trace next exec, no idea of previous one
2007 SQL tuning (aka SQLT)
Provides info about all plans, some slower
No easy way to spot if plans are cause here
Require you to parse raw AWR / ASH info
5/23/2019
Enkitec ©

37. Plan switched from 2210 (good) to 2898 (bad), also plan 711 exists
Easier this way? 
Plan switched from 2210 (good) to 2898 (bad), also plan 711 exists
5/23/2019
Enkitec ©

38. Same plan runs randomly slow
1998 SQL Tuning (aka traces)
Need to trace every exec until it reproduces
2007 SQL tuning (aka SQLT)
Nothing specific about executions
Data is available but partially presented raw
Need to restore SQLT repo and query manually
5/23/2019
Enkitec ©

39. Easier this way? 
List of all sampled executions by instance, session, serial with elapsed time and many more info
Instance 4 session 1474 was the one with slow execution on August 24
Several reports specific to that execution
5/23/2019
Enkitec ©

40. Need to investigate my PX execs
1998 SQL Tuning (aka traces)
Every slave generates one trace
Need to manually glue them together
2007 SQL tuning (aka SQLT)
Nothing specific about PX details
No way to spot downgrades, skewness, etc
Need to restore SQLT repo and query manually
5/23/2019
Enkitec ©

41. PX is never easy, but faster this way?
Number of sessions executing the SQL over time by wait event
Little skewness in distribution
5/23/2019
Enkitec ©

42. PX is never easy, but faster this way?
Execution DoP
PX servers requested
PX servers allocated
Number of processes sampled by ASH
Bottom two executions are serial
5/23/2019
Enkitec ©

43. Closing Remarks SQLd360 takes few minutes to run
Can be executed any time even during the issue
SQLd360 speeds up SQL Tuning process
Automating collection and improving presentation
A new Release about once a month
Installs nothing and it is free!

45. References SQLd360 introduction SQLd360 examples and download
SQLd360 examples and download

46. Contact Information
Download
SQLd360 vYYMM (date)
Pages
SQLd360