1. Monitoring the Essentials in DB2 10.5
Philip K. Gunning
Gunning Technology Solutions, LLC
Session Code: C03
May 13, :15 PM – 4:15 PM| Platform: DB2 LUW

2. DISCLAIMER DB2™ is a registered trademark of IBM Corporation.
DB2 BLU, BLU Acceleration are registered trademarks or copyrights of IBM Corporation
Other logos and product/trade names are registered trademarks or copyrights of their respective companies
NOTES:

3. The Warp Speed Achieved with BLU Acceleration
“For data analytics the volumes involved are usually dramatically larger than cache sizes, which highly limits exploitation of CPU caches. The secret to the warp speed-up achieved in BLU Acceleration is that IBM has designed every algorithm to minimize access to RAM, and maximize processing time in L3 and L2 caches. For the
processor shown that provides
a whopping 15x-41x speed-up for memory
access” (Sam Lightstone, IBM,
NOTES:

4. Overview Overview of BLU Acceleration DB2 in the New Galaxy
Concepts
Terminology
DB2 in the New Galaxy
When RAM is too slow
Learning to speak and think “Column organized” in BLU Acceleration
Rules of Thumb for column organized data in terms workload, concurrency, bufferpools, sort heap threshold, sortheap
NOTES:

5. Overview Important role of UTIL_HEAP_SZ
New “Column Organized” monitoring metrics
How efficient are my column-organized tables
Are you getting what you think?
Using MONREPORT reporting module
Tuning Recommendations
Best Practices
NOTES:

6. Quick Look at Prerequisites
DB2 Workgroup Advanced Enterprise Server Edition
DB2 Advanced Enterprise Server Edtion
OS and Hardware prerequisites
NOTES:

7. New Registry Variable Setting for Analytical Workload
Before creating a new database, set the DB2_WORKLOAD registry variable to ANAYLTICS
This will result in the following:
Makes column organized tables the default table type
DFT_TABLE_ORG=COLUMN
Page and extent size configured for analytical processing
PAGESIZE=32
DFT_EXTENT_SZ=4
DFT_DEGREE=ANY
Memory for sorting, caching and hashing will be set based on the amount of RAM and server size
Enables automatic space reclamation
Compression is always on
NOTES: Set the DB2_WORKLOAD registry variable to ANALYTICS before creating any database.

8. Setting the DB2_WORKLOAD Registry Variable Continued
Memory for sorting, caching and hashing will be set based on the amount of RAM and server size
CATALOGCACHE_SZ, SHEAPTHRES_SHR and SORTHEAP set to a value that is optimized for the hardware you are using
Intraquery Parallelism enabled for any workload even if DBM CFG parameter INTRA_PARALLEL=OFF
The default concurrency threshold on the SYSDEFAULTMANAGEDSUBCLASS service subclass is enabled to ensure maximum efficiency and utilization of the server
Automatic Space Reclamation is enabled
AUTO_MAINT=ON
AUTON_REORG=ON
Page and extent size configured for analytical processing
Enables automatic space reclamation
Compression is always on
NOTES:

9. Setting the DB2_WORKLOAD Registry Variable Continued
Automatic Space Reclamation is enabled
AUTO_MAINT=ON
AUTO_REORG=ON
Space reclamation performed for column-organized tables by default
NOTES: Set the DB2_WORKLOAD registry variable to ANALYTICS before creating any database.

10. Use of AUTOCONFIGURE
For an existing database that you want to configure for BLU Acceleration
Set the DB2_WORKLOAD=ANALYTICS registry variable and restart the instance
Connect to the database you want to run autoconfigure on
Run “db2 autoconfigure apply dbname”
After running autoconfigure, review the bufferpool. SHEAPTHRES_SHR, SORTHEAP , UTIL_HEAP_SZ and adjust as necessary
NOTES:

11. Use of AUTOCONFIGURE
In an instance with multiple databases some of which are not using BLU, set the registry variable before you create the databases that will be using BLU, then unset the registry variable after creating the databases that will use BLU
NOTES:

12. Setting SHEAPTHRES_SHR and SORTHEAP Parameters
When BLU Acceleration is enabled, the self-tuning memory manager (STMM) does not adjust the values of the SHEAPTHRES_SHR and SORTHEAP database configuration parameters automatically.
Set with static values by the AUTOCONFIGURE command when DB2_WORKLOAD=ANALYTICS registry setting is enabled
NOTES:

13. Setting SHEAPTHRES_SHR and SORTHEAP
Analytical workloads require larger sort memory areas due to the nature of the workload
Optimal setting provides degree of concurrency required and associated SORTHEAP sizing
Setting SHEAPTHRES_SHR to AUTOMATIC not supported for databases using column organized tables
Tuning SHEAPTHRES_SHR and SORTHEAP as important as tuning BUFFERPOOLS
Aim to prevent sort overflows to disk which can significantly slow down queries
SET SEHAPTHRES_SHR to 40-50% of the memory available to the database
DATABASE_MEMORY DB CFG Parameter
NOTES: The SHEAPTHRES_SHR database configuration parameter represents a soft limit on the total amount of database shared memory that can be used by sort memory consumers at any one time. Set to handle degree of concurrent sort consumers (nbr of sort consumers x sortheap setting).

14. Setting SHEAPTHRES_SHR and SORTHEAP
The SORTHEAP DB CFG parameter controls the amount of memory an application can use for SORTS
SORTHEAP memory comes out of total SHEAPTHRES_SHR memory
Size the SHEAPTHRES_SHR DB CFG parameter and SORTHEAP DB CFG parameter to provide the amount of concurrency desired and to prevent sort overflows
Rule of thumb for sizing SORTHEAP is set it to 5-20% of SHEAPTHRES_SHR
Monitor for sort overflows and post threshold sorts via MON_GET_DATABASE and MON_GET_UNIT_OF_WORK table functions
NOTES: The higher the degree of sort concurrency requires, increase SHEAPTHRES_SHR and reduce the SORTHEAP DB CFG parameter. This should be done in small increments and through dedicated monitoring sessions.

15. Monitoring SHEAPTHRES_SHR and SORTHEAP
Monitor for sort overflows and post threshold sorts via MON_GET_DATABASE and MON_GET_UNIT_OF_WORK table functions
Adjust SHEAPTHRES_SHR upward and SORTHEAP upward gradually to eliminate sort overflows
Don’t increase SORTHEAP without concomitant adjustment to SORTHEAP
Can decrease SORTHEAP independently
Mine the DYNAMIC SQL and target queries with high number of overflows
Good way to train a DBA in tuning SQL or identifying proper indexes to eliminate SORTS
NOTES:

16. Setting the STMTHEAP DB CFG Parameter
Since analytic queries are more complex than OLTP type queries, they can benefit from an increased STMTHEAP DB CFG size
The STMTHEAP DB CFG parameter specifies the size of the workspace used by the SQL compiler during compilation of an SQL statement
NOTES: STMTHEAP is essentially a work space used by the SQL Compiler during compilation of an SQL statement. If you have queries that receive an SQL0437W with reason code 1, adjust the STMTHEAP size by 50% and continue to increment by same percentage as needed.

17. Importance of Setting UTIL_HEAP_SZ Correctly
For LOAD operations and for building compression dictionaries UTIL_HEAP_SZ must be sufficiently sized
Set to AUTOMATIC if DB2_WORKLOAD=ANALYTICS
If DATABASE_MEMORY set to AUTOMATIC should be sufficient as it will grow as necessary
If not sized sufficiently, can result in less than optimal compression
NOTES:

18. Monitoring Adequacy of UTIL_HEAP_SZ setting
Check the new PCTENCODED column in the SYSCAT.COLUMNS catalog view which is the the percentage of values that are encoded as a result of compression for a column in a column-organized table.
If the overall compression ratio for you’re the table in is too low, check this statistic to see if values in specific columns were left uncompressed.
Many columns with a very low value (or even 0) can be an indicator that utility heap might have been too small when the column compression dictionaries were created
You might also see very low values for columns that were incrementally loaded with data that was outside of the scope of the column compression dictionary
NOTES:

19. So, How are your Column-organized Tables Performing?
Use monitor elements returned by the MON_GET_TABLE table function to determine the efficiency of column-organized tables and to identify candidate tables suitable to be organized by column
NOTES:

20. New Column-organized Metrics from MON_GET_TABLE table function
Use monitor elements returned by the MON_GET_TABLE table function to determine the efficiency of column-organized tables and to identify candidate tables suitable to be organized by column
By dividing the NUM_COLUMNS_REFERENCED monitor element by the SECTION_EXEC_WITH_COL_REFERENCES monitor element, you can determine the average number of columns that a particular query accesses. If this average is much less than the number of columns in the table, the workload favors column organization.
Controlled by the mon_obj_metrics DB CFG parameter setting
Extended is the default and is required to return column-organized metrics
NOTES:

21. Monitor Collection DB CFG Parameters (Review)
Mon_act_metrics – controls collection of activity level monitor elements on the entire database (DEFAULT – BASE)
MON_GET_ACTIVITY_DETAILS
MON_GET_PKG_CACHE_STMT
Activity event monitor (DETAILS_XML monitor element in the event_activity logical data groups)
Mon_deadlock – controls generation of deadlock events on the entire database (DEFAULT- WITHOUT_HIST)
Mon_locktimeout – controls generation of lock timeout events on the entire database (DEFAULT – NONE)
Mon_lockwait – controls generation of lock wait events for the lock event monitor (DEFAULT – NONE)
Mon_lw_thresh – the amount of time spent in lock wait before an event for mon_lockwait is generated (DEFAULT )
Mon_obj_metrics – controls collection of data object monitor elements on the entire database (DEFAULT- EXTENDED *formerly BASE)
MON_GET_TABLE
NOTES:

22. Monitor Collection DB CFG Parameters (Review)
Mon_req_metrics – controls the collection of request monitor elements on the entire database (DEFAULT – BASE)
MON_GET_UNIT_OF_WORK
MON_GET_UNIT_OF_WORK_DETAILS
MON_GET_CONNECTION
MON_GET_CONNECTION_DETAILS
MON_GET_SERVICE_SUBCLASS
MON_GET_SERVICE_SUBCLASS_DETAILS
MON_GET_WORKLOAD
MON_GET_WORKLOAD_DETAILS
Statistics event monitor (DETAILS_XML monitor element in the event_wlstats and event_scstats logical data groups)
Unit of work event monitor
Mon_uow_data – controls the generation of UOW events at the database level for the UOW event monitor (DEFAULT – NONE)
NOTES:

23. Monitoring and MONREPORT Reporting Enhancements
MONREPORT.DBSUMMARY and MONREPORT.CONNECTION reporting modules enhanced to support on column processing
New monitor elements to assess bufferpool efficiency
Monitor elements contain COL in the monitoring element name
New monitor elements to monitor prefetch performance for data in column organized tables
New monitor elements to measure column data size
ADMIN_GET_TAB_INFO
New monitor elements to measure time spent
New monitor elements for the HASHED GROUP BY operator
New monitor interfaces to get request metrics that are aggregated along different dimensions
New monitor interfaces to get activity metrics
Monitor interfaces to get database object metrics
Refer to Database Monitoring Guide and Reference for complete details
NOTES:

24. New CTQ Optimization Operator
Use to denote operations on column organized tables
Most processing should be below the CTQ operator in the access path (db2exfmt, db2explain)
Optim Performance Manager
Optim Query Tuner
If many operations above this operator indicator of inefficient use of column organized table
Re-look how table is being used
NOTES:

25. Reviewing time spent in column processing versus row processing with new monitor elements
New monitoring elements returned from MON_GET_ACTIVITY_DETAILS tabe function to show where time is being spent
Total_app_section_excutions
Total_col_executions
Total_col_proc_time
Total_col_time
Total_cpu_time
NOTES:

26. Understanding Automatic Space Reclamation
Space reclamation for column-organized tables
Space reclamation after deletions
After a delete operation the row is logically deleted
Space that was used by the deleted row is not available to subsequent transactions and remains unusable until space reclamation occurs
REORG TABLE is required to reclaim the space
NOTES:

27. Understanding Automatic Space Reclamation
Space reclamation after updates
After a delete operation the row is first deleted and a new copy of the row inserted at the end of the table
Updated row uses space in proportion to the number of times that the row is updated until space reclamation occurs
All rows in the extent where the update took place must be deleted before any space reclamation can occur
REORG TABLE is required to reclaim the space
NOTES:

28. Understanding Automatic Space Reclamation
If DB2_WORKLOAD=ANALYTICS then AUTO_REORG enabled
Define an automatic reorg online and/or offline policy using the AUTOMAINT_SET_POLICY or AUTOMAINT_SET_POLICYFILE stored procedures with the appropriate parameters
Offline reorg considerations
Need appropriate window
Reclaims space
Faster than online REORG
Sample POLICYFILE in
SAMPLE DB2 directory
NOTES:

29. Automatic Space Reclamation
Online REORG considerations
Still need appropriate window
Schedule ONLINE REORG window during period of low activity
Reclaims space by specifying
Specify RECLAIM EXTENTS option of REORG command to reclaim space
To determine when a table REORG is necessary, use the ADMIN_GET_TAB_INFO table function to review an estimate of the amount of reclaimable space to reclaim
Monitor REORG status with db2pd or SNAPTAB_REORG administrative view
NOTES:

30. Summary DB2 BLU New Column-organized capabilities Use of AUTOCONFIGURE
Review of DB2_WORKLOAD=ANALYTICS registry setting
SHEAPTHRES_SHR and SORTHEAP setting discussion and recommendations
Table monitoring metrics
Monitoring Enhancements and some review
Activity detail monitoring elements
New Monitoring Elements
Importance of adequate UTIL_HEAP_SZ
REORG Considerations
Space Usage and Reclamation
NOTES:

31. References
Tuning and Monitoring System Performance, located at
DB Database Monitor Guide and Reference,
Best Practices, Optimizing Analytical Workloads using DB with BLU Acceleration,
NOTES:

32. References
DB2 BLU Acceleration, Rapid Adoption Guide,
When RAM is too Slow,
DB2 10.5, What’s New,
NOTES:

33. Monitoring the Essentials in DB2 10.5 DB2 LUW
Philip K. Gunning Gunning Technology Solutions, LLC
Monitoring the Essentials in DB2 10.5
DB2 LUW
Please fill out your session evaluation before leaving!