1. 5
IQ Monitor

2. IQ Monitor
IQ Monitor is an internal task that collects and reports on internal counters during IQ 12 execution
In IQ 11 IQ Monitor ran as an external “shared memory” task
There are a series of “views” of the counters to show differing aspects of the server workload

3. Starting the IQ Monitor
Started using IQ UTILITIES command
iq utilities [main|private] into <dummy_tblname> start monitor “options”
The table name is never used, output is to an ASCII file.
The table name is only present for syntactical reasons

4. Counter Output
The counters are output to an ASCII file, in the directory in which IQ is running
<dbname>.<conn#>-[main|temp]-iqmon
In ASIQ the suffix “-iqmon” can be changed to reflect user requirements
Example : asiqdemo.2-main-iqmon

5. Stopping the IQ Monitor
Stopped using the same basic command
iq utilities [main|private] in <dummy_tblname> stop monitor
Again as with the start command the “dummy_tblname“ is only present for the syntactical analyser in ASA

6. Issues in Monitor Operation
You can have 2 monitors running
One monitoring IQ_MAIN
One monitoring IQ_TEMP
They will write to different files
asiqdemo.2-main-iqmon
asiqdemo.2-temp-iqmon
Also they must be explicitly started and stopped with the correct syntax

7. File options
The –file_suffix option allows you to change the –iqmon suffix to the output file name
This is so you can keep multiple copies of the output showing what, when and why you ran it
The –truncate and –append options allow you to specify whether the output file is truncated (default) or appended to when the monitor starts up

8. Graphic of Operation Monitor Temp Monitor Main Language Processor
Catalog
Catalog Store IQ
Store
IQ Temp
Store
Catalog
Store
IQ Store
Temp Counters O/P
Main Counters O/P
IQ Temp Store

9. Primary Options There are two “sets” of options
interval
Collected counters to display
Interval is the time interval that IQ Monitor collects the information
The first iteration of Monitor displays counters from the start of the server
The subsequent displays are deltas of the information from the last display

10. Interval
Interval can be from 2 seconds to 4,294,967,295 seconds (2^32-1) – as this is around 136 years I do not suggest we use this
The default interval is 60 seconds - 1 minute
Beware of setting the interval too small the data may not be too meaningful

11. Basic Options
-summary Reports the key statistics for both the main and temporary cache
-cache More detailed reports on specified cache
-cacheByType Reports the cache cache_by_type statistics by type of buffer
-io Reports IO subsystem counters

12. Options
In AS IQ-M there are a series of new monitor options
-contention
-bufalloc
-threads
-debug
These are described after the basic 12.4 (and still in ) options

13. -summary Reports - 1 Find Rate No. of times a buffer was requested
Hit Rate Percentage of cache hits
Read Rate No. of read operations
Write Rate No. of write operations
Pin Percentage No. of pinned buffers – generally buffers that need to be modified
Note – these are for both caches – for the specified interval

14. -summary Reports - 2 Sybase Adaptive Server IQ Performance Monitor
Version 3.1
Options string for Main cache: "-summary -file_suffix summary-iqmon -append -interval 10"
Summary
:51:17
Active| Main Cache | Temp Cache
Users| Finds HR% Reads/Writes GDirty Pin% Dirty% InUse%| Finds HR% Reads/Writes GDirty Pin% Dirty% InUse%
/ /
/ /
/ /
/ /
/ /

15. Time Out! – LRU/MRU Chain
The layout of the caches is similar to ASE in that they are one long MRU/LRU chain
There is a wash marker
One major difference is there are multiple sweeper threads to write dirty pages to disk
Buffer Movement
MRU
Wash Marker
LRU
Sweeper Write Activity

16. -cache Reports - 1 Find Rate as summary
Creates This is the number of times the create buffer operation was called
Destroy This is the number of times the destroy operation was called
Dirty This is the number of times the dirty operation was called
Hit Rate as summary

17. -cache Reports BWAIT
This is the count of the number of times a thread tries to get a buffer
but the thread had to hold before the buffer could be made available
This may be because the buffer was locked or whatever
Note- Bwait should be small

18. -cache Reports – REREADS
this is a count of number of times a buffer was requested for read, after the same thread had been written
If this is the case it means that
1) the cache activity is so high that the write transaction cannot hold a written buffer in memory
2) Or, the transaction is very long and hits the same page multiple times, but with a large time interval between hits
This should be very low – except, possibly in case 2 above.

19. -cache Reports – F.MISS
This is an internal counter that counts the number of times that a hash operation had to probe a hash table twice
Generally this indicates that there has been a rollback shortly before
If there is no rollback activity – watch this counter – it should be zero

20. -cache Reports - CLONED
This is a count of the number of times a buffer was cloned
This is an example of versioning in operation
For each page that must be modified (not new pages) a cloned operation takes place
A page will only CLONE if there are other users looking at that page – if not then an Opportunistic clone takes place – we update the page in place

21. Time Out! - Cloning Page 001 If page 004 need to be “modified”
Then the system will check to see
If other users are using the page
Page 002
Page 003
If no user is using the page – it is modified, marked as dirty – and written to a new location (maybe as page 005)
Page 004
If a user is using the page – then the
Page has to be cloned – and the CLONE
Count is incremented
If a user now needs the (old) page 004 then
this is recorded as a cache miss the the page
is read from disk

22. -cache Report - Prefetch
prefetch Count of how many prefetch operations are requested
Prefetch Read count of how may prefetch operations result in disk reads
Generally we should hope that a good part of the prefetch operations can be satisfied in cache – too many reads indicate that we might be able to use more memory

23. -cache Report - GDIRTY
We really don’t want to see this count above zero
This counts the number of times that a buffer get operation has had to stall whilst a dirty page is flushed to disk, effectively this means cache is full of dirty pages
If this is the case then we need to move the wash marker - or increase the number of sweeper threads
Note this is Grabbed Dirty not Got Dirty

24. -cache Report - 8
Pin Percent Percentage of buffers that are pinned (locked into memory)
Dirty Percent Percentage of pages that are dirty
Try not to let the Dirty Percent get above 85-90% otherwise the GDirty will start to go above zero – not good.

25. -cache Report - 9 Sybase Adaptive Server IQ Performance Monitor
Version 3.1
Options string for Main cache: "-cache -file_suffix cache-iqmon -append -interval 10"
Main Shared Buffer Cache
:52:26
Finds Creats Dests Dirty HR% BWaits ReReads FMiss Cloned Reads/Writes PF/PFRead GDirty Pin% Dirty%
Mn: / /
Mn: / /
Mn: / /

26. Cache Reports By Type
This reports the same basic counters as the –cache option, but in this case the counts are broken out by block type in memory
These may be of slightly less interest than the –cache reports – but over a longer interval they can provide some interesting statistics on the performance of the buffer manager

27. Object Types in Cache 4 Variable Length B- Tree 5 Fixed Length B- Tree
7 VDO – used by the catalogue to find indices
8 Dbspace header
9 Database header
10 Sort buffer (temp)
12 G-Array (HG)
13 B-Array (FP)
14 Block Map
15 Hash Bucket
16 Checkpoint Block
17 Bitmap (of any sort!)
18 Reserved

28. -cache_by_type Sybase Adaptive Server IQ Performance Monitor
Version 3.1
Options string for Main cache: "-cachebytype -file_suffix cachebytype-iqmon -append -interval 10"
Main Shared Buffer Cache
:53:26
Btype Finds Creats Dests Dirty HR% BWaits ReReads FMiss Cloned ReadKB/WriteKB Reads/Writes PF/PFRead GDirty Pin% Dirtyy%
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: / / /
Mn: other / / /
Mn: total / / /

29. -IO Reports
IO simply reports the input and output operations the server conducted during the specified interval
This is not broken out by device, the counters relate to entire server activity

30. -IO Report - Input Reads The number of read requests
LRd(KB) The page size multiplied by the number of requests
PRd(KB) The actual number of bytes read
Rratio The ratio of logical to physical
The last figure relates the efficiency of the compression to disk - for reads

31. -IO Report - Output Writes The number of write requests
LWrt(KB) The page size multiplied by the number of requests
PWrt(KB) The actual number of bytes written
Wratio The ratio of logical to physical
The last figure relates the efficiency of the compression to disk - for writes

32. -IO Report Sybase Adaptive Server IQ Performance Monitor
Version 3.1
Options string for Main cache: "-io -file_suffix io-iqmon -append -interval 10"
Main Shared Buffer Cache
:54:21
Input Output
Reads LRd(KB) PRd(KB) Rratio Writes LWrt(KB) PWrt(KB) Wratio
Mn:
Mn:
Mn:
Mn:
Mn:
Mn:
Mn:

33. Options
The following slides detail the output from the following new (in ) options
-contention
-threads
-bufalloc
-debug

34. -contention
The –contention option basically displays all of the lock and mutex counters
These are counters that show the activity within the buffer cache and how quickly these locks were resolved
Contention shows statistics for both the main and temp buffer caches and the “heap” memory
Heap is the load and user memory

35. -contention described – 1
AU Active Users
LRULks # of times a lock was requested
woTO # of times a lock did not have to time out (withoutTimeOut)
Loops How many lock request had to spin waiting for the lock to be granted
TO # of Timeouts (opposite of woTO)

36. -contention described – 2
BWaits (as described above)
IOLock # of locks taken while looking for place to write the data
IOWait how many had to wait
HTLock # of locks on the internal Hash table
HTWait how many had to wait
FLLock # of locks on the free list
FLWait how many had to wait

37. -contention – Heap Memory
The heap is any memory that IQ-M has to use that is not within either of the 2 caches
This memory is composed of Load Memory, User Memory, thread stacks etc.
MemLks # of locks taken on the heap
MemWts how many had to wait

38. -contention Sybase Adaptive Server IQ Performance Monitor
Version 3.1
Options string for Main cache: "-contention -file_suffix contention-iqmon -append -interval 10"
Contention
:57:03
AU| Main Cache |
LRULks woTO Loops TOs BWaits IOLock IOWait HTLock HTWait FLLock FLWait
| Temp Cache | Memory Mgr
|LRULks woTO Loops TOs BWaits IOLock IOWait HTLock HTWait FLLock FLWait |MemLks MemWts

39. -threads Option
The –threads option details the counters held by the thread manager
This tells you actually what is going executing (sort of) inside the server
This option can be selected for main or private, however it is irrelevant what you select the thread manager is server wide!

40. -threads described
CPU # of CPUs IQ-M is using (maybe not the total number in the box)
Limit max # of threads IQ-M can run
NTeams # of teams currently running
MaxTeams High Water Mark for teams
NThreads Current # of threads in existence
Resrvd # of threads reserved for IQ-M
Free # of free threads inc. Resrvd
Locks # of locks on the thread manager
Waits how many had to wait

41. -threads Sybase Adaptive Server IQ Performance Monitor
Version 3.1
Options string for Main cache: "-threads -file_suffix threads-iqmon -append -interval 10"
Threads
:59:24
CPU Limit Nteams MaxTeams Nthreads Resrvd Free Locks Wait

42. -bufalloc
-bufalloc prints the statistics from the buffer allocation routine
This is the control of how many new pages / buffers were requested and by what
This can be executed for both main and temp cache – and is useful for both

43. -bufalloc described - 1 OU User Resource Allocation number
AU # of Active Users
MaxBuf # of buffers on the cache
Avail # of buffers available in the cache
AvPF # of buffers available to the pre-fetch mechanism
Slots # of buffers active for pre-fetch
PinUsr # of users using the buffer manager

44. -bufalloc described - 2 PFUsr # of pre-fetch users
Posted # of posted users. Users who think they know how many buffers they will use
UnPost # of unposted users. See Above
Quota # buffers reserved for use
Locks # of locks in the buffer manager
Waits How many had to wait

45. -bufalloc Sybase Adaptive Server IQ Performance Monitor
Version 3.1
Options string for Main cache: "-bufalloc -file_suffix bufalloc-iqmon -append -interval 10"
Buffer Allocation
:58:39
OU/AU MaxBuf Avail AvPF Slots PinUsr PFUsr Posted UnPost Quota Locks Waits 1/ 1/

46. -debug - 1
This is an undocumented (sort of) option for the IQ Monitor.
Basically it displays everything we could think of
Beware – this option generates a serious amount of paper, but can be very useful
It can be run for either the main or temp caches – although some of the information is the same for both
Some of the same counters are displayed as described above, but they are called by their internal counter names – do not be confused

47. -debug – 2 There are 10 subsections to the -debug report
Buffer Manager (main or temp)
Contention Counters
Dirty Page and Sweeper Thread Counters
Heap Memory Manager
Thread Manager
Free List
Buffer Allocation Manager
Buffer Allocation Histogram
Prefetch Information

48. -debug – Buffer Manager - 1
This section of the –debug report is a sort of re-hash of the –cache-by-type report. The fields are the same, but named subtly differently, and the buffer types are named rather than shown by number
The buffer types correspond exactly to the numbers in the cache_by_type report
I have put a “decode” matrix together on the next few slides

49. -debug – Buffer Manager – 2
Finds Find Rate (as cache_by_type)
Hits # of cache hits
Hit% Hit Rate (as cache_by_type)
False Miss After rechecking to confirm that the buffer we want is not in memory - we find it is!
Cloned Cloned (as cache_by_type)
Creates Creates (as cache_by_type)

50. -debug – Buffer Manager – 3
Destroys Destroys (as cache_by_type)
Dirties Dirty (as cache_by_type)
RealDirties This is where the dirty flag actually was set
Prefetches prefetch (as cache_by_type)
Prefetchmiss When we did the prefetch we found that the buffer was actually in memory

51. -debug – Buffer Manager – 4
Reads Reads (as io)
PReadBLks # of physical blocks read
PReadKB PRd(KB) (as io)
ReReads ReReads (as cache_by_type)
Writes Writes (as io)
PWriteBlks # of physical blocks written
PWriteKB PWrt(KB) (as io)

52. -debug – Buffer Manager – 5
GrabbedDirty GDirty (as cache_by_type)
MovedtoMRU # of buffers that after use were put back at the MRU
MovedtoWash # of buffers that after use were put straight into the wash
RemovedfromLRU # of buffers that were deleted in the MRU-LRU chain
Removedfromwash # of buffers that were deleted from the wash area
RemovedinScanMode # of buffers that were “used” when we thought that they were just being scanned

53. -debug – Buffer Manager – 6
BType Stats Name Total none rfu rfu rfu btreev btreef rfu vdo dbext dbid sort store garray barray blkmap hash ckpt bm test cmid
Finds
Hits
Hit% e+01 7e e e+01 7e+01 9e e
FalseMiss
Cloned
Creates
Destroys
Dirties
RealDirties
Prefetchs
PrefetchMiss
Reads
PReadBlks
PReadKB
ReReads
Writes
PWriteBlks
PWriteKB
GrabbedDirty
MovedToMRU
MovedToWash
RemovedFromLRU
RemovedFromWash
RemovedInScanMode

54. -debug – Contention – 1
This section of the report is a rehash of the contention report
This section is cache dependant, so if you need information from both main and temp you will have to run 2 monitors

55. -debug – contention – 2
Busywaits BWaits (as cache_by_type) and (as contention)
LRUNumLocks LRULks (as contention)
LRUNumSpinsWoTO WoTO (as contention)
LRUNumSpinLoops Loops (as contention)
LRUNumTimeOuts TOs (as contention)
HTNumLocks HTLock (as contention)
HTNumWaits HTWait (as contention)
IONumLocks IOLock (as contention)
IONumWaits IOWait (as contention)

56. -debug – contention – 3 BusyWaits 0 LRUNumLocks 7650
LRUNumSpinsWoTO ( %)
LRUNumSpinLoops
LRUNumTimeOuts ( %)
HTNumLocks
HTNumWaits ( %)
IONumLocks
IONumWaits ( %)

57. -debug – flush operators – 1
This section of the report concentrates of the handling of dirty pages and how the wash area and sweeper threads are working
If you have a write intensive server or a Multiplex Writer Node then this section is useful
As with contention this part of the report is cache dependant

58. -debug – flush operators – 2
Pages # of buffers in the cache
InUse # of buffers in the cache marked as being used
Dirty Dirty (as cache_by_type)
Pinned # of buffers in the cache pinned
Flushes # of times the flush operator was called
FlushedBufferCount # of buffers flushed to disk

59. -debug – flush operators – 3
GetPageFrame # of buffers requested from the buffer manager
GetPageFrameFailure # of buffers requested but request failed (maybe due to Grabbed Dirty)
GotEmptyFrame # of buffers that we got that were empty, that we did not have to clean or “make”

60. -debug – flush operators – 4
Washed # of buffers passing the wash mark
TimesSweepersWoken # of times that the sweeper threads went to sleep (with nothing to do) and were woken up
WashTeamSize # of threads in the sweeper team (option set)
WashMaxSize # of buffers in the wash area (option set)

61. -debug – flush operators – 5
washNBuffers # of clean buffers passing the wash marker
washNDirtyBuffers # of dirty buffers passing the wash marker
washSignalThreshold # of dirty buffers that have to pass the wash marker before the sweepers are woken up
washNActiveSweepers # of sweepers actually working
washIntensity Internal flag

62. -debug – flush operators – 6
Pages= ( %)
InUse= ( %)
Dirty= ( %)
Pinned= ( %)
Flushes
FlushedBufferCount
GetPageFrame
GetPageFrameFailure
GotEmptyFrame
Washed
TimesSweepersWoken
washTeamSize=
washMaxSize= ( %)
washNBuffers= ( %)
washNDirtyBuffers= ( %)
washSignalThreshold= ( %)
washNActiveSweepers=
washIntensity=

63. -debug – Heap Manager – 1
This section of the report concerns the ASIQ-M “overhead memory”
This is the Load Memory, the User Memory and any other bits of RAM that IQ-M wants during execution
For obvious reasons this section is not cache dependant

64. -debug – Heap Manager – 2
Memallocated Amount of memory currently in the heap
MemAllocatedMax High Water Mark for this run
MemAllocatedEver High Water Mark since the server started
MemNAllocated # of calls to allocate memory

65. -debug – Heap Manager – 3
MemNAllocatedEver # of calls to allocate memory since the server started
MemNTimesLocked # of calls to lock (part of) the heap
MemNTimesWaited # of times we had to wait for a lock

66. -debug – Heap Manager – 4 MemAllocated= 215413736 ( 210364 KB)
MemAllocatedMax= ( KB)
MemAllocatedEver ( KB)
MemNAllocated=
MemNAllocatedEver
MemNTimesLocked
MemNTimesWaited ( %)

67. -debug – Thread Manager – 1
This section (almost) duplicates the –thread report
Again this report tells you how active the server processing is, rather than how active memory is
This is not cache dependent

68. -debug – Thread Manager – 2
ThrNumOfCpus CPU (as threads)
ThreadLimit Limit (as threads)
ThrNumThreads # of threads actually useable
ThrReserved Resrvd (as threads)
ThrNumFree Free (as threads)
ThrNumUsed NThreads (as threads)
UsedPerActiveCmd # of threads per command

69. -debug – Thread Manager – 3
ThrNTeamsInUse NTeams (as threads)
ThrMaxTeams MaxTeams (as threads)
CumNumTeams Total # of Teams during the run duration
CumTeamThreads Total # of threads used in total # of teams
CumSingleThr # of teams that had only thread
ThrMutexLocks Locks (as threads)
ThrMutexWaits Waits (as threads)

70. -debug – Thread Manager – 4
ThrNumOfCpus=
ThreadLimit= ( %)
ThrNumThreads= ( %)
ThrReserved= ( %)
ThrNumFree= ( %)
NumThrUsed ( %)
UsedPerActiveCmd
ThrNTeamsInUse=
ThrMaxTeams=
CumNumTeams
CumTeamThr
CumSingleThr
ThrMutexLocks
ThrMutexWaits ( %)

71. -debug – Free List – 1
This section describes the activity in the free list
This gives an indication on the allocation and de-allocation activity in the server
This is cache dependent, as there is one free list per cache
An important point, the free list is a misnomer, it should be called the allocated page list

72. -debug – Free List – 2
FreeBitCount # of allocated blocks in the store (IQ Main or IQ Temp)
FLIsOutOfSpace Flag to indicate when the store is fully allocated (full)
FLMutexLocks # of times the free list lock was taken out
FLMutexWaits # of times an operator had to wait on the free list lock

73. -debug – Free List – 3 FLBitCount= 320116 FLIsOutOfSpace= 0
FLMutexLocks
FLMutexWaits ( %)

74. -debug – Buffer Allocation – 1
This section of the report details the counters showing the performance of the buffer allocation manager
This is almost the same as –bufalloc
This is a cache dependent report

75. -debug – Buffer Allocation – 2
Elapsed Seconds # of CPU seconds used during the run interval
CPU User Seconds # of USER Mode CPU Seconds
CPU Sys Seconds # of System Mode CPU Seconds
CPU Total Seconds Total of System and User Seconds

76. -debug – Buffer Allocation – 3
NactiveCommands AU (as bufalloc)
BufAllocMaxBufs MaxBuf (as bufalloc)
BufAllocAvailBufs Avail (as bufalloc)
BufAllocReserved Quota (as bufalloc)
BufAllocAvailPF AvPF (as bufalloc)
BufAllocSlots Slots (as bufalloc)
BufAllocNPinUsers PinUsr (as bufalloc)

77. -debug – Buffer Allocation – 4
BufAllocNPFUsers AU (as bufalloc)
BufAllocNPostedUsers Posted (as bufalloc)
BufAllocNUnpostUsrs Unpost (as bufalloc)
BufAllocPinQuota Quota (as bufalloc)
BufAllocMutexLocks Locks (as bufalloc)
BufAllocMutexWaits Waits (as bufalloc)

78. -debug – Buffer Allocation – 5
Elapsed Seconds ( %)
CPU User Seconds ( %)
CPU Sys Seconds ( %)
CPU Total Seconds ( %)
NActiveCommands=
BufAllocMaxBufs= ( %)
BufAllocAvailBufs= ( %)
BufAllocReserved= ( %)
BufAllocAvailPF= ( %)
BufAllocSlots=
BufAllocNPinUsers=
BufAllocNPFUsers=
BufAllocNPostedUsrs=
BufAllocNUnpostUsrs=
BufAllocPinQuota=
BufAllocMutexLocks
BufAllocMutexWaits ( %)

79. -debug – Allocation Hist. – 1
This report splits out the Buffer Allocation Statistics by buffer type
Then it takes a count of the users pinning buffers and produces a size histogram showing buffer type against a histogram count
This is cache dependent

80. -debug – Allocation Hist. – 2
BufferAlloc StatName Total unknwn hash csort fp test garray btree bm
NumClients=
PinUserQuota=
PrefetchUserQuota=
ClientCountOfPinners
Unknown=
Hash=
Sort=
FP=
Test=
Garray=
BTree=
BM=
BV=

81. -debug – Prefetch – 1
This section of the report details what has happened in the prefetch mechanism
This is a sort of amalgamation of a series of reports, but I will describe each counter
This is cache dependent

82. -debug – Prefetch – 2 PFMgrNThreads # of Prefetch Threads
PFMgrNSubmitted # of Prefetch submissions
PFMgrNDropped # of dropped requests (buffer already in memory)
PFMgrNValid # of valid Requests
PFMgrNRead # of PF Reads
PFMgrReading # of PF Reads completed

83. -debug – Prefetch – 3 PFMgrNThreads= 5 PFMggrNSubmitted= 0
PFMgrNDropped= 0
PFMgrNValid= 0
PFMgrNRead= 0
PFMgrNReading= 0

84. IQ Monitor - End