The MDA tables - new monitoring capabilities in ASE EMEA Technical QuickStart 2003 Rob Verschoor Sypron B.V. Consultant

MDA tables: new monitoring feature in  Consultant for ASE & Replication Server  Worked with ASE since 1989  based in The Netherlands  occasionally doing SPS assignments Contact:   About myself:

Published two books about ASE: "Tips, Tricks & Recipes for Sybase ASE" "The Complete Sybase ASE Quick Reference Guide" (see for details) About myself:

But first (not MDA-related at all):  Help promote ASE as the best RDBMS on Linux  Go to and vote for ASE as "best RDBMS on Linux" before 31-Dec-2003.

MDA tables: new monitoring feature in  MDA tables: new feature in  How it works  Overview of all MDA tables  Installation & configuration  Examples & applications  Comparison with other ASE monitoring tools Overview

MDA tables: new monitoring feature in  MDA = 'Monitoring Data Access” (or “Monitoring and Diagnostics for ASE” or “Monitoring and Diagnostic Access or “Monitoring and Diagnostic API ”)  35 proxy tables in the master database  all named 'mon%' (i.e. 'monObjectActivity')  can be queried with regular SQL  proxy tables mapped to native RPCs What is it?

MDA tables: new monitoring feature in Example: monProcessSQLText -- returns currently executing SQL statements create existing table monProcessSQLText ( SPIDsmallint, KPIDint, BatchIDint, LineNumberint, SequenceInLineint, SQLTextvarchar(255) NULL) external procedure at 'loopback...$monProcessSQLText' go

MDA tables: new monitoring feature in MDA provides low-level details not previously available  #logical I/Os for each process  SQL 'stacktrace' (stored proc/trigger call stack)  cache usage details per table  and much, much more...

MDA tables: new monitoring feature in MDA licensing:  no license needed  included in ASE base product  DBXRay (by BMC; built on the MDA tables) does require an additional license (ASE_XRAY)

MDA tables: new monitoring feature in monState  some global server-level info  # days since startup  #deadlocks since startup  current # active connections

MDA tables: new monitoring feature in monEngine  details about engines  current status  online/offline time  affinity information

MDA tables: new monitoring feature in monOpenDatabases  details about databases  start time of most recent dump  whether dump currently running  whether dump failed

MDA tables: new monitoring feature in monDataCache monProcedureCache  overall data about data caches / proc cache  not specific per object in the cache

MDA tables: new monitoring feature in monCachedObject  all objects (tables, indexes) currently in the data caches  #Kbytes  #processes currently accessing the object  does not indicate which I/O buffer pools are involved

MDA tables: new monitoring feature in monCachePool  I/O buffer pool usage details  no object-specific details

MDA tables: new monitoring feature in monCachedProcedures  all plans for compiled objects currently in the proc cache  compilation date  size  no more dbcc procbuf !

MDA tables: new monitoring feature in monOpenObjectActivity  all plans for compiled objects currently in the proc cache  #logical, physical I/Os (use for cache effectiveness!)  #rows inserted/deleted/updated  #times a lock on this object was not immediately granted  note: these counters are reset when the object descriptor is removed from the cache.

MDA tables: new monitoring feature in monLocks  similar to syslocks

MDA tables: new monitoring feature in monProcess monProcessLookup  similar to sysprocesses

MDA tables: new monitoring feature in monProcessActivity  similar to sysprocesses, but more information about:  #logical I/Os  #physical I/Os  ULC writes/flushes

MDA tables: new monitoring feature in monProcessObject  objects (tables, indexes) currently being accessed by each process

MDA tables: new monitoring feature in monProcessProcedures  objects (procedures, triggers) currently being executed by each process

MDA tables: new monitoring feature in monWaitEventInfo  things processes may wait for  very detailed: no less than 292 different reasons EventID ClassID Description waiting for CTLIB event to complete waiting for disk read in parallel dbcc waiting in sort manager due to traeflag wait for killed tasks to die waiting for a SSL read to complete

MDA tables: new monitoring feature in monWaitClassInfo  main categories of things processes may wait for  grouped into 10 categories 0 Process is running 1 waiting to be scheduled 2 waiting for a disk read to complete 3 waiting for a disk write to complete 4 waiting to acquire the log semaphore 5 waiting to take a lock 6 waiting for memory or a buffer 7 waiting for input from the network 8 waiting to output to the network 9 waiting for internal system event 10 waiting on another thread

MDA tables: new monitoring feature in monProcessWaits  indicates what each process has been waiting for up till now  cumulative wait times (milliseconds) for all event types monSysWaits  statistics about why processes waited for something  cumulative wait times (seconds) for all event types

MDA tables: new monitoring feature in monIOQueue monDeviceIO  disk I/O statistics  indicates data/log/tempdb I/O  average I/O duration

MDA tables: new monitoring feature in monProcessNetIO monNetworkIO  network I/O statistics

MDA tables: new monitoring feature in monProcessStatement  Currently executing statements  indicates procedure ID, statement starttime, #I/Os  does not show the T-SQL text submitted by the client  wait time during execution (for locks, for example --> indicates effect of concurrency improvements)  does not indicate what type of statement is being executed (this information is available in monProcess.Command ) monSysStatement  Completed, recently executed statements  pipe

MDA tables: new monitoring feature in monProcessSQLText  T-SQL text (batch) currently executing monSysSQLText  T-SQL text (batch) that was recently executed but now completed  pipe

MDA tables: new monitoring feature in monSysPlanText  recently generated query plan  pipe

MDA tables: new monitoring feature in monDeadlock  recent deadlocks  find SQL text by joining with monSysSQLText  pipe

MDA tables: new monitoring feature in monErrorLog  most recently errorlog messages  pipe

MDA tables: new monitoring feature in monSysWorkerThread  currently active worker processes monProcessWorkerThread  parent process of worker processes

MDA tables: new monitoring feature in Out-of-the-box: no MDA tables present. Installation steps:  Add a 'loopback' server alias name sp_addserver loopback, null, (assuming is also in the interfaces file) Test this configuration: exec loopback…sp_who (note: 3 dots!)  Run $SYBASE/ASE-12_5/scripts/installmontables (when successful, prints no output whatsoever!)  Assign mon_role to logins allowed MDA access (incl. sa ) grant role mon_role to sa Test this configuration: select * from master..monState Installation

MDA tables: new monitoring feature in Set configuration parameters: sp_configure 'enable monitoring', 1 sp_configure 'sql text pipe active', 1 'sql text pipe max messages', 100 'plan text pipe active', 1 'plan text pipe max messages', 100 'statement pipe active', 1 'statement pipe max messages', 100 'errorlog pipe active', 1 'errorlog pipe max messages', 100 'deadlock pipe active', 1 'deadlock pipe max messages', 100 Installation

MDA tables: new monitoring feature in Set configuration parameters (cont’d): sp_configure 'wait event timing', 1 'process wait events', 1 'object lockwait timing', 1 'SQL batch capture', 1 'statement statistics active', 1 'per object statistics active', 1 'max SQL text monitored', 255 (static!) (pick up this list from Installation

MDA tables: new monitoring feature in Installation Gotchas  running installmon instead of installmontables  forgetting to create a 'loopback' server alias  running installmontables with sqsh ('$' characters are interpreted by sqsh) --> use isql instead  forgetting to assign mon_role before querying MDA tables (& disconnect/reconnect) Installation

MDA tables: new monitoring feature in Installation Gotchas  not running the / installmaster first (creates mon_role )  forgetting to set the 'enable monitoring' parameter (and lots of other configuration parameters) Installation

MDA tables: new monitoring feature in Practical problem: which table/column to query?  35 tables  365 columns  MDA tables are self-documenting MDA table schema information

MDA tables: new monitoring feature in monTables monTableColumns monTableParameters  contain documentation about MDA tables and their columns  name  datatype  short description of function  identify parameters  parameters : most efficient filtering columns (‘=‘ condition)

MDA tables: new monitoring feature in  sp_mda_help is an easy interface to these tables (download from 1> sp_mda_help cach 2> go TableName monCachePool Provides statistics for all pools allocated for all caches monCachedObject Provides statistics for all objects and indexes that current monCachedProcedures Provides statistics about all procedures currently stored in monDataCache Provides statistics relating to data cache usage [...] MDA table schema information

MDA tables: new monitoring feature in > sp_mda_help monDataCache 2> go TableName Description monDataCache Provides statistics relating to data cache usage CacheID int Unique identifier for the cache RelaxedReplacement int Whether the cache is using Relaxed cached replacement strategy BufferPools int The number of buffer pools within the cache CacheSearches int Cache searches directed to the cache PhysicalReads int Number of buffers read into the cache from disk LogicalReads int Number of buffers retrieved from the cache PhysicalWrites int Number of buffers written from the cache to disk Stalls int Number of 'dirty' buffer retrievals CachePartitions smallint Number of partitions currently configured for the cache CacheName varchar Name of the cache CacheID int Unique identifier for the cache CacheName varchar Name of the cache MDA table schema information

MDA tables: new monitoring feature in > sp_mda_help null, sql 2> go TableName ColumnName TypeName monProcessSQLText SQLText varchar monSysSQLText SQLText varchar MDA table schema information

MDA tables: new monitoring feature in Currently executing SQL statements  Show SQL text: select SPID, SQLText from monProcessSQLText  Show I/O info, start time, procedure name etc: select * from monProcessStatement Rows in these tables disappear immediately when execution is completed Examples

MDA tables: new monitoring feature in select ps.SPID, ps.CpuTime, pst.LineNumber, pst.SQLText from master..monProcessSQLText pst, master..monProcessStatement ps where ps.SPID = pst.SPID and ps.CpuTime = ( ) order by SPID, LineNumber Examples select max(CpuTime) from master..monProcessStatement Which process is currently using the most CPU (and what's it doing?) Change CpuTime to LogicalReads or PhysicalReads as required

MDA tables: new monitoring feature in Measuring # logical I/Os for a process select SPID, LogicalReads, PhysicalReads, PhysicalWrites from master..monProcessActivity where SPID=18  This table shows cumulative I/O counts (zeroed at ASE start)  Always increases; determine the difference between two calls  Can be used to identify session doing heavy queries (= many logical I/Os) Examples

MDA tables: new monitoring feature in Measuring # logical I/Os for a process  Easier interface to monProcessActivity : sp_mda_io (download from Usage(1): determine logical I/O since previous call exec sp_mda_io --SQL statement-- exec sp_mda_io --SQL statement-- exec sp_mda_io output: [spid=24]#secs=4 #Log.Reads=1696 #Phys.Reads=49 #Phys.Writes=0 sp_mda_io

MDA tables: new monitoring feature in Measuring # logical I/Os for a process Usage(2): determine logical I/O for a statement batch exec sp_mda_io 'select count(*) from MyTable' exec sp_mda_io 'create index ix1 on MyTable(a)' sp_mda_io

MDA tables: new monitoring feature in Notes:  MDA tables also measure logical I/O on system tables ( set statistics io doesn't). This is quickly a few hundred logical I/Os.  When showplan (or other options) are enabled, the additional logical I/Os are even more. sp_mda_io

MDA tables: new monitoring feature in Number of pages in cache for each table/index select ObjectID, ObjectName, IndexID, CachedKB, ProcessesAccessing from master..monCachedObject where ObjectID > 99 ObjectID ObjectName IndexID CachedKB ProcessesAccessing MyTable spt_values spt_values Customers_tb Customers_tb YourTable syblicenseslog Examples

MDA tables: new monitoring feature in Number of pages in cache for each table/index  How much cache does each object use at this moment  This information can be useful when considering/evaluating cache configuration Examples

MDA tables: new monitoring feature in Find most frequently accessed/executed tables/procedures select object_name(ObjectID, DBID), Operations from master..monOpenObjectActivity where IndexID = 0 order by 2 desc Operations MyTable 475 my_proc 381 YourTable 129 sp_mda_help 19 […] Examples

MDA tables: new monitoring feature in Find most frequently accessed/executed tables/procedures  Operations column doesn't increase by 1 (but more…) Examples

MDA tables: new monitoring feature in Find unused indexes select object_name(ObjectID, DBID), IndexID, OptSelectCount, UsedCount, Operations from master..monOpenObjectActivity where ObjectID > 99 and DBID > 1 order by 1,2,3 IndexID OptSelectCount UsedCount Operations MyTab MyTab MyTab MyTab MyTab YourTab YourTab […] Examples

MDA tables: new monitoring feature in Find unused indexes  OptSelectedCount: increased each time the optimizer selects this index  UsedCount: increased each time the table is actually accessed by this index  Unused indexes are indicated by OptSelectCount = 0 / UsedCount = 0  This information applies only to objects currently in the metadata cache, and only since the last time the descriptor was initialised Examples

MDA tables: new monitoring feature in SQL 'stacktrace' select ContextID, DBName, ObjectName, ObjectType from master..monProcessProcedures where SPID = your_spid order by ContextID ContextID DBName ObjectName ObjectType PROD_DB Trig1 trigger procedure 2 PROD_DB Proc1 stored procedure 3 OTHER_DB Proc2 stored procedure (3 rows affected) Examples

MDA tables: new monitoring feature in SQL 'stacktrace' Possible applications:  debug code (where is this procedure executed from?)  allow updates to a table by one specific procedure only Examples

MDA tables: new monitoring feature in Kill idle processes  much-requested feature  can now be implemented by using monProcessWaits  implemented by David Wein (‘ sp_idleReaper ’) Examples

MDA tables: new monitoring feature in  Some tables contain only current data, others keep some history monProcessSQLText --> currently executing SQL monSysSQLText --> recently executed SQL Amount of history is configurable: sp_configure 'sql text pipe max messages' Programming issues

MDA tables: new monitoring feature in Find out what the highest-I/O-consuming process is doing select ps.SPID, ps.LogicalReads, pst.LineNumber, pst.SQLText from master..monProcessSQLText pst, master..monProcessStatement ps where ps.SPID = pst.SPID and ps.CpuTime = ( select max(LogicalReads) from master..monProcessStatement ) order by SPID, LineNumber This query may not work for various reasons... Programming issues

MDA tables: new monitoring feature in Find out what highest-I/O-consuming process is doing Better way of coding: select * into #ps from monProcessStatement select * into #pst from master..monProcessSQLText … query as on previous slide, but using these #tables… Programming issues

MDA tables: new monitoring feature in History tables will never return the same data twice for a session The following queries will never return the same result set: select * from master..monSysSQLText Solution: regularly copy data into a permanent table Programming issues

MDA tables: new monitoring feature in  MDA's strength: huge amount of low-level details  Also its weakness: how to distill usable information from this data?  Need an application; DBXRay does this for you  Great opportunities for open-source movement  Possibilities for combining MDA tables with the ASE job scheduler (12.5.1) ?

MDA tables: new monitoring feature in Notes  When ASE is very busy (99% CPU), access to MDA tables may 'hang' until busy period is over (this is a CIS issue)  Performance impact: to be determined in more detail, but seems minimal (3-5%)  Some MDA tables are similar to the 'fake' tables in 'master', but don't have this problem when tempdb is full  Counters are set 0 upon ASE reboot; currently no counter reset is possible.  Counter columns (4-byte int) may wrap around  Some features aren’t fully clear (like monOpenObjectActivity.Operations )  Security concerns: be careful with giving out mon_role

MDA tables: new monitoring feature in  MDA tables vs. sp_sysmon  MDA provide per-object, per-session data, since ASE start  sp_sysmon provides cumulative data over a limited interval  sp_sysmon does interferes with some MDA counters (see next slide)  MDA tables vs. sysprocesses/syslocks  MDA provide *much* more detail  MDA tables vs. MonServer/HistServer  MonServer/HistServer have great capabilities, but aren't easy to use (learning curve)  MDA tables can be queried with straightforward SQL statements  MDA tables vs. DBXRay  MDA is free; DBXRay is a license option  DBXRay pulls MDA details together for you MDA compared with other features

MDA tables: new monitoring feature in Interference between MDA tables and sp_sysmon  Most MDA counters are not affected by sp_sysmon  … but some are: they'll be reset when sp_sysmon is run. This query shows which ones: select TableName, ColumnName from monTableColumns where Indicators & 2=2  Also (not displayed by the above query): monIOQueue.IOs monIOQueue.IOTime MDA compared with other features

MDA tables: new monitoring feature in  : Performance & Tuning Guide; ‘Monitoring & Analyzing’ volume  : 'newfunc.pdf' for   examples, applications, tricks… Documentation

MDA tables: new monitoring feature in That’s it ! Have fun with the MDA tables!