Average Session Load (ASL) The Golden Metric ? Kyle Hailey

#.2 Copyright 2006 Kyle Hailey In this Session The Holy Grail of Performance : ASL 1.ASL : stethoscope for Database Health  Tap into the heart beat of the database  Heart stopped - Hung?  Beating very slow - Idle ?  Beating fast - Heavily Loaded ? 2.Subcomponent Trinity  CPU  Waits  Time series 3.Extra: use Maximum CPU as Yardstick

#.3 Copyright 2006 Kyle Hailey Word of Wisdom My Goal is to cut out a lot of the noise and extraneous data and hone in on the essential Half of the game is knowing when to act and how much effort to put in

#.4 Copyright 2006 Kyle Hailey Idle Database  Value of proving the database is Idle  It’s the Databases Fault  How many times do you hear that?  Database Idle  No load on database  Database “performance” is fine  Under utilized  Problem lies elsewhere  Saved me time and stress many times

#.5 Copyright 2006 Kyle Hailey What’s the Database Doing  Often I want a quick and easy way to see what the database is doing  Is it working?  Is it blocked  How much is going on?  Is the database “healthy”?

#.6 Copyright 2006 Kyle Hailey Checking the Database  How do *you* check the Database Health?  Routine Exams?  Statspack?  1300 lines of data  Which lines  How many stats  Automated Alerts?  What do you set alerts on?  What if no alerts fire ? Are you ok?  Do alerts really tell you what’s happening?

#.7 Copyright 2006 Kyle Hailey What’s the Database Doing Whip out the stethoscope ASL

#.8 Copyright 2006 Kyle Hailey The Cult of ASL Once you’ve been initiated … there is no going back

#.9 Copyright 2006 Kyle Hailey Welcome to … The Cult of ASL  Magic Metric for Database Health Average Session Load (ASL) For those of you who are already members, welcome back

#.10 Copyright 2006 Kyle Hailey ASL  The Golden Metric  Powerful  Multidimensional  Indispensable

#.11 Copyright 2006 Kyle Hailey Average Session Load (ASL) goes under the guise of  Session Load  I often refer to it this way  Average Active Sessions (aas?)  The way I named it in OEM 10g Graphs  Centi-seconds per second (or secs/sec)  In the dark ages, before OEM 10g, waits were often measured as Centi-secs per sec

#.12 Copyright 2006 Kyle Hailey ASL – Average Session Load  Average Value  Averaged over 15 secs in OEM 10g  Time period could 5 minutes, an hour  Active Session Only  Active sessions put load the database  Inactive Sessions don’t put load  Except for memory usage  Active sessions are  Sessions in a Call  Started a SQL Statement but hasn’t returned yet  DBWR writing blocks out

#.13 Copyright 2006 Kyle Hailey ASL Calculations ASL = DB TIME / Time Period DB Time (DBT) = sum over all sessions of time in the states   CPU   Wait DB TIME (10g) = DB TIME = select value from v$sysstat where name = ‘DB time’; select value from v$sysstat where name = ‘DB time’; Select sum(time_waited) from v$system_event where event not in (... idle events …); + Select value from v$sysstat where name = ‘CPU used by this session’; Select sum(time_waited) from v$system_event where event not in (... idle events …); + Select value from v$sysstat where name = ‘CPU used by this session’; ‘DB time’

#.14 Copyright 2006 Kyle Hailey ASL – sources ASL can be found from or derived from  10g  OEM  V$systat ‘DB time’ / elapsed time  9i  v$system_event  $sysstat  This works Oracle 7 to 10g (probably 11)

#.15 Copyright 2006 Kyle Hailey ASL – 9i  Data  V$system_event  Sum wait times  Non-idle waits  V$sysstat  CPU used by this session  ASL= (sums(wait)-cpu)/elapsed time  ASL(CPU)=cpu/elasped  ASL(wait)= sum(waits)/elapsed  Produces  Session time / elapsed time  Session cent-secs/sec  Session secs/ sec  Avg Session Load (ASL)

#.16 Copyright 2006 Kyle Hailey ASL in OEM DB Home Page

#.17 Copyright 2006 Kyle Hailey Calculating ASL : Statspack  Uses  v$session_wait  v$sysstat  Look for  Top 5 Timed Events  Elapsed Time  cpu_count helpful Seconds in Wait / Elapsed time = ASL

#.18 Copyright 2006 Kyle Hailey Use Statspack to Find Waits Statspack Cheat Sheet:  Install  Connect as SYSDBA  Run  Exec statspack.snap;  Generate Reports

#.19 Copyright 2006 Kyle Hailey Statspack Trusty statspack report  Elapased Time  Check “Top 5 Timed Events” Start at line 52 of about 1300!

#.20 Copyright 2006 Kyle Hailey Elapsed Time STATSPACK report for DB Name DB Id Instance Inst Num Release RAC Host LABSF labsf NO labsfr Snap Id Snap Time Sessions Curs/Sess Begin Snap: 1 03-Apr-06 12:34: End Snap: 2 03-Apr-06 12:34: Elapsed: 1.00 (mins) STATSPACK report for DB Name DB Id Instance Inst Num Release RAC Host LABSF labsf NO labsfr Snap Id Snap Time Sessions Curs/Sess Begin Snap: 1 03-Apr-06 12:34: End Snap: 2 03-Apr-06 12:34: Elapsed: 1.00 (mins)

#.21 Copyright 2006 Kyle Hailey Used CPU Time and Wait Time  Look at Top 5 Timed Events Top 5 Timed Events ~~~~~~~~~~~~~~~~~~ % Total Event Waits Time (s) Call Time buffer busy waits 2, CPU time free buffer waits 1, write complete waits log buffer space Top 5 Timed Events ~~~~~~~~~~~~~~~~~~ % Total Event Waits Time (s) Call Time buffer busy waits 2, CPU time free buffer waits 1, write complete waits log buffer space

#.22 Copyright 2006 Kyle Hailey Example  CPU + WAITS CPU = 32 WAITS = = 278 secs  Elapsed Time = 60 secs  ( ) user secs / 60 secs = 5.1 average session load ( 4.6 waiting 0.5 on CPU ) Top 5 Timed Events Event Time (s) buffer busy waits 250 CPU time 32 free buffer waits 15 write complete waits 8 log buffer space 5 Top 5 Timed Events Event Time (s) buffer busy waits 250 CPU time 32 free buffer waits 15 write complete waits 8 log buffer space 5

#.23 Copyright 2006 Kyle Hailey Available CPU – init.ora Statspack 10g shows # of CPUS Statspack 9i # of CPUs SQLPLUS> show parameters cpu_count NAME TYPE VALUE cpu_count integer 2 SQLPLUS> show parameters cpu_count NAME TYPE VALUE cpu_count integer 2

#.24 Copyright 2006 Kyle Hailey Available CPU vs ASL Far above available CPU => problem Plenty of Free CPU => wait bottleneck 4.6 waiting 0.5 on CPU 4.6 waiting 0.5 on CPU ASL = 5.1 # of CPU = 2 ASL = 5.1 # of CPU = 2

#.25 Copyright 2006 Kyle Hailey ASL – Primary Purpose  Answers the Question Is the database idle ? Active? How Active?

#.26 Copyright 2006 Kyle Hailey ASL < 1 Database is not blocked ASL ~= 0 Database basically idle ASL < # of CPUs Extra CPU to be had Database is probably not blocked ASL > # of CPUs Could have performance problems  ASL >> # of CPUS There is a bottleneck

#.27 Copyright 2006 Kyle Hailey Going Farther with ASL  ASL can tell you a lot  But it’s components tell you much more  To go farther need the components of ASL  CPU  How many CPUs (max CPU available)  Wait  Which waits  Value over time

#.28 Copyright 2006 Kyle Hailey Components of ASL DB Home Page Performance Page ASL Point in Time ASL over Time

#.29 Copyright 2006 Kyle Hailey EM DB Home Page

#.30 Copyright 2006 Kyle Hailey Performance Page

#.31 Copyright 2006 Kyle Hailey Performance Page

#.32 Copyright 2006 Kyle Hailey ASL – Performance Page

#.33 Copyright 2006 Kyle Hailey OEM 10g - ASL ASL is the top of the curve

#.34 Copyright 2006 Kyle Hailey OEM 10g - CPU CPU

#.35 Copyright 2006 Kyle Hailey OEM 10g - Waits Waits

#.36 Copyright 2006 Kyle Hailey OEM 10g – CPU vs Wait Waits CPU

#.37 Copyright 2006 Kyle Hailey OEM 10g – Max CPU Maximum Possible CPU

#.38 Copyright 2006 Kyle Hailey OEM 10g Zoom-In Available CPU CPU + WAIT

#.39 Copyright 2006 Kyle Hailey OEM 10g – get to work ! Relax Get to Work! Looks OK But …

#.40 Copyright 2006 Kyle Hailey Calculating ASL

#.41 Copyright 2006 Kyle Hailey ASL Calculations ASL = DB TIME / Time Period DB TIME (10g) = select value from v$sysstat where name = ‘DB time’; select value from v$sysstat where name = ‘DB time’; But there is another way …

#.42 Copyright 2006 Kyle Hailey ASL alternative Calculation  active sessions  Count Active sessions over interval  average by interval  Less accurate, but surprisingly close  v$session_wait (or v$active_session_history)  Wait_time > 0 = ON CPU  Filter out idle events  9i or lower, join to v$session  status='ACTIVE'  type='USER‘  10g v$sessions has all the columns

#.43 Copyright 2006 Kyle Hailey Two Sources comparison  V$system_event & v$sysstat  Indirect measure via time spend in DB  Accurate Measure of time counters  Values can lag (especially CPU)  v$session_wait  Direct measure of # of sessions  Closer to “real time”  statistical approximation via samples

#.44 Copyright 2006 Kyle Hailey ASL in OEM DB Home Performance Page Top Activity Page Same Chart but calculated differently 1 2

#.45 Copyright 2006 Kyle Hailey ASL – Performance Page v$sysstat v$system_event ASL = ΔDBT / ΔTime ASL(tn)= ( DBT(tn) - DBT(t0) ) / (tn-t0) ASL 1

#.46 Copyright 2006 Kyle Hailey ASL – Top Activity Page active sessions(ti) n Σ 0 ASL(tn) = / n t0 t1 2 n = # of samples

#.47 Copyright 2006 Kyle Hailey Active Average Sessions = Top Activty ? active sessions(ti) n Σ 0 ASL(tn) = / n n = # of samples ASL = ΔDBT / ΔTime ASL(tn)= ( DBT(tn) - DBT(t0) ) / (tn-t0) Performance Page : Average Active Sessions Top Activity = ?

#.48 Copyright 2006 Kyle Hailey DB TIME = area under the curve ΔDBTΔTime ASL = ΔDBT/ ΔTime ΔDBT = DBT(tn) – DBT(t0) = Δt ΔDBT = DBT(tn) – DBT(t0) = active sessions(ti) * Δt n Σ 0 ΔDBT ΔDBT = Area under curve t0 tn t0 tn

#.49 Copyright 2006 Kyle Hailey ASL – Top Activity Page ΔTime = n Σ 0 Δt / Δt active sessions(ti) * Δt n Σ 0 ASL(t1) = n Σ 0 Δt active sessions(ti) n Σ 0 ASL(t1) = / n Σ 0 1 active sessions(ti) n Σ 0 ASL(t1) = / n n = # of samples Δt active sessions(ti) * Δt n Σ n ASL(t1) = / ΔTime ΔDBTΔTime ASL = ΔDBT/ ΔTime ΔDBT = Δt ΔDBT = active sessions(ti) * Δt n Σ 0

#.50 Copyright 2006 Kyle Hailey ASL – Top Activity Page t0 tn / active sessions(ti) n Σ 0 ASL(t1) = n

#.51 Copyright 2006 Kyle Hailey Samples VS Counters Counters Samples Slight Lags v$system_event v$active_session_history

#.52 Copyright 2006 Kyle Hailey The Power ASH gives ASL DB Home Performance Page Top Activity Page

#.53 Copyright 2006 Kyle Hailey ASH in OEM Top Activity gives more information

#.54 Copyright 2006 Kyle Hailey Top Activity : Based on ASH missing ThanksToASH

#.55 Copyright 2006 Kyle Hailey Top Activity : ASH Dimensions

#.56 Copyright 2006 Kyle Hailey ASL – %Session Time Issue Shown in % DB Time Missing % Session Time

#.57 Copyright 2006 Kyle Hailey Top Activity: ASH Sessions Many Users Active On Performance Page, no way to tell how many users But Top Activity Page fixes that

#.58 Copyright 2006 Kyle Hailey Top Activity: ASH Sessions Two Users Active

#.59 Copyright 2006 Kyle Hailey SQL and Session DB Home Performance Top Activity SQL Session

#.60 Copyright 2006 Kyle Hailey Session : ASH Activity

#.61 Copyright 2006 Kyle Hailey SQL : ASH Activity

#.62 Copyright 2006 Kyle Hailey Getting the Most  Need to know the System’s Profile  What your application is like  Data Warehouse  OLTP  Typical load  Once you get to know it you can see anomalies  Is ASL near 0 when it should be higher  Is that Data Warehouse query running normal  Do you know what it looks like?  Is there an unusual bottleneck

#.63 Copyright 2006 Kyle Hailey Knowing your DB Profile

#.64 Copyright 2006 Kyle Hailey When to tune General rules of Thumb  Waits >> CPU  CPU > Max CPU

#.65 Copyright 2006 Kyle Hailey Waits > CPU

#.66 Copyright 2006 Kyle Hailey CPU > Max CPU

#.67 Copyright 2006 Kyle Hailey CPU > Max CPU

#.68 Copyright 2006 Kyle Hailey Getting More out of ASL DB Home Performance Top Activity SQL Session Session

#.69 Copyright 2006 Kyle Hailey In resume  ASL is simple and Powerful  ASL’s components are even more powerful  CPU  WAIT  Value over Time  Use # of CPUs as a yardstick  Know your application load profile to see anomalies  Data warehouse  OLTP  Heavy Load  Light Load