1. Progress System Tables
Dan Foreman
BravePoint, Inc.
BravePoint

2. Introduction - Dan Foreman
Progress User since 1984 (V2.1)
Presenter at lots of Progress conferences dating back to 1990
BravePoint

3. Introduction - Dan Foreman
Author of:
Progress Performance Tuning Guide
Progress Database Admin Guide
Progress Virtual System Tables
Pro D&L - Dump/Load with short down time no matter how large the Database is
ProMonitor – DB monitoring tool (uses VSTs)
ProCheck – Monitor AppServers & WebSpeed
BravePoint

4. What are Virtual System Tables
Virtual System tables (VSTs) provide access to Database Shared Memory data that was previously only accessible through promon
It’s possible to customize the user interface
Accessible Locally or Remotely via the 4GL or SQL
BravePoint

5. What are Virtual System Tables
VSTs are Schema Tables
They do not interfere with Application tables and Indexes
VSTs are not part of the empty Database in V8
Enable with proutil <db> -C enablevst
Are part of the V9/V10/V11 empty Database
Are visible in the Dictionary (must view ‘Hidden’ tables)
BravePoint

6. What are Virtual System Tables
Some new tables and fields are not in the “native” empty DB unless:
proutil –C updatevst is run
Example: New VST fields added in V10.2B SP6
The feature must be enabled:
Auditing: proutil enableauditing
TDE
Multi-Tenancy
BravePoint

7. What are Virtual System Tables
VSTs in the Standard empty DB
V tables
V tables
V9.0-V9.1C 39 tables
V9.1D 42 tables
V tables
V10.1B 47 tables
BravePoint

8. What are Virtual System Tables
promon can lag behind VSTs; Examples:
_TableStat & _IndexStat (V8.3)
_AreaStatus (V9 and later)
Conversely VSTs can lag behind also:
_Startup
BravePoint

9. Resource Impact of VSTs
There is no performance penalty for accessing VSTs with 2 exceptions that will be covered later
VSTs take up no additional disk space except for the VST schema
VSTs take up a small amount of memory to cache the additional schema
Except –tablerangesize & -indexrangesize can increase DB Shared Memory by a surprising amount
BravePoint

10. VST Implementation
Whenever a record is requested, the table is checked to see if it’s a VST table.
VST tables have table numbers between to
The check is simple arithmetic which takes place regardless of whether VSTs have been enabled or not
BravePoint

11. VST Implementation
If a requested record is a VST record, the fields are populated from shared memory structures instead of from a database record
BravePoint

12. VST Basics
Each VST has one, unique index on the ‘Id’ field which is a ‘virtual’ RECID
Each VST record is numbered starting at 1 and continues sequentially (some exceptions)
It is not possible to add new indexes or triggers to VSTs
BravePoint

13. VST Basics Some VST Record Counts are controlled by startup parameters
_connect -n & -Mn
_LockReq -n & -Mn
_UserIO -n & -Mn
_UserLock -n & -Mn
_UserStatus -n & -Mn
_Trans -n
_Servers -Mn
_Lock -L
BravePoint

14. VST Basics
The tables on the last slide are pre-populated at Database Broker startup time
The empty slots have “?” values
To avoid displaying the empty slots:
FOR EACH _connect
WHERE _connect-usr <> ?
The <> operator is not a performance problem since VSTs are not really ‘indexed’
BravePoint

15. Which Database
If a Client is connected to multiple Databases, the VST Statistics will be based upon the Current Working Database
BravePoint

16. Updatable VSTs _Startup-spin -spin _Startup-APWQTime -pwqdelay
_Startup-APWSTime -pwsdelay
_Startup-APWBuffs -pwscan
_Startup-APWMaxWrites -pwwmax
_Startup-
BravePoint

17. Updateable in V10.2B SP6 _Startup._Startup-LRU-Skips
_Startup._Startup-NmsgWait
_Startup._Startup-Pefetch-Delay
_Startup._Startup-Prefetch-Factor
_Startup._Startup-Prefetch-Num-Recs
_Startup._Startup-Prefetch-Priority
BravePoint

18. Updatable VSTs _MyConnect-NumSeqBuffers Added in V9
Same as –Bp parameter
Number of Sequential Read Buffers
Makes it possible to run a large report with a less severe impact on the -B Buffer Cache
Example on the next slide
BravePoint

19. Updatable VSTs DO FOR _MyConnect TRANSACTION : FIND FIRST _MyConnect.
_myconnect-numseqbuffers = 32.
END.
RUN killer_report.p.
_myconnect-numseqbuffers = 0.
BravePoint

20. Updatable VSTs _StatBase._TableBase _StatBase._IndexBase V9 and later
Used to set the window size of Tables and Indexes monitored using the _TableStat and _IndexStat VSTs or to reset (zero out) the statistics by moving the “window” up and then down
BravePoint

21. Table & Index I/O VSTs
Note that _TableStat and _IndexStat are available in V8.3 and very helpful for Storage Area Planning
Use _IndexStat to identify unused indexes; COMPILE/XREF is only good for finding indexes that are defined but not used in the code; COMPILE/XREF is useless for Dynamic Queries
BravePoint

22. VST Problems Potential Performance Killers
_Block ( V8, useless in V9/V10/V11 )
_Lock ( FOR EACH _lock WHILE … )
_UserLock ( DO loops of 512 iterations )
Too Much Data for One Screen
_Block-block x(1024)
_UserLock-* Several arrays of 512
Missing Data
_DBStatus
BravePoint

23. VST Problems
Storage
VSTs are defined as INTEGERs which limits the maximum value to about 2 billion – prior to V10.1B
Poorly documented (see my System Table Guide for better documentation)
Stale Data
_Lock (total # of records) compared to _DbStatus-NumLocks
BravePoint

24. System Tables - MetaSchema
_File
_Field
_Index
_Index-Field
_File-Trig
_Field-Trig
_Db
_Sequence
BravePoint

25. System Tables - Other SQL Auditing Encryption Multi-Tenancy
Authentication (GRANTs & REVOKEs)
Views
Constraints
Query Optimizer statistics
Auditing
Encryption
Multi-Tenancy
BravePoint

26. Usefulness VSTs are useful for two types of Progress users
Application Developers
Database Administrators
Who else is there?
BravePoint

27. For Developers Can view the lock status for an individual record
_Lock VST shows all active locks and who has the lock
We can easily find the Table name of the record being locked by doing a join to the _File table
_Trans VST shows all active transactions
BravePoint

28. For Developers Deadly Embrace (Deadlock) Detection
_Connect
“Delinquent” Active Transactions that grow the BI File to an abnormally large size
_Trans
BravePoint

29. For Developers Improve code efficiency by measuring:
User DB Accesses, Reads, etc.
Reads, Writes, Creates, Deletes by Table
_TableStat
Unused indexes; Logical Scatter Factor
_IndexStat
BravePoint

30. For Developers Name & attributes of a Field
find _file “customer” no-lock.
for each _field of _file no-lock :
display
_field-name
_data-type
_format
_extent.
end.
BravePoint

31. DBAs - Tables of interest
Monitoring the BI High Water Mark
_DbStatus._dbstatus-bisize
Available in V8.3 and later
V8: BI Size in Bytes
V9 & later: BI Size divided by BI Block Size
BravePoint

32. DBAs - Tables of interest
Monitoring Database Size
promon doesn’t have the correct data
Each Storage Area has a separate High Water Mark
_AreaStatus
_AreaStatus-AreaName
_AreaStatus-HiWater
_AreaStatus-TotBlocks
_AreaStatus-LastExtent
_AreaStatus-Extents
BravePoint

33. DBAs - Tables of Interest
Record Fragmentation
_Record._Record-RecRead
_Record._Record-FragRead
dbanalys/tabanalys only shows how many Fragments exist, not how often the Fragments are accessed
BravePoint

34. DBAs - Tables of Interest
What Client is running what Program
_Connect-Cache*
Also Line# for 4GL Code
Or SQL Statement
Since V10.1C
What Tables is a Client Hitting
_UserTableStat
Since V10.1B
BravePoint

35. For More Information Progress System Tables Guide 200+ pages
Covers V8 through V11
Covers VSTs and System Tables
Has lots of undocumented information
BravePoint

36. ProMonitor Written using (mostly) VSTs
Source Code provided for most Programs
Threshold Alerts warn of problems
Monitor an unlimited number of DBs
Monitors more than just VST Statistics
Session Temp File I/O
Client Memory Utilization
DB Log File Monitoring
DB Fragmentation/Scatter Monitoring
AppServers, WebSpeed, AdminServer, etc
BravePoint

37. Conclusion Questions? Dan Foreman danf@prodb.com www.BravePoint.com