1. Profiling Suspicious Code
Tom Bascom
White Star Software

2. A Few Words about the Speaker
Tom Bascom; Progress 4gl coder & roaming DBA since 1987
President, DBAppraise, LLC
Remote database management service for OpenEdge.
Simplifying the job of managing and monitoring the world’s best business applications.
VP, White Star Software, LLC
Expert consulting services related to all aspects of Progress and OpenEdge.
I have been working with Progress since 1987
… and today I am both President of DBAppraise;
The remote database management service…
where we simplify the job of managing and monitoring the worlds best business applications;
and Vice President of White Star Software;
where we offer expert consulting services covering all aspects of Progress and OpenEdge.

3. Users want the right answer, with the best response time at the lowest cost.

4. The performance enhancement possible with a given improvement is limited by the fraction of the execution time that the improved feature is used.
-- Amdahl’s Law

5. Performance is not just about the database.
The most bang for the performance tuning buck is often in the application code.
But figuring out where to look is often hard.
It’s the code that drives the database…
But figuring out where to look is hard

6. Target the largest response time component of the most important Business process first.

7. Some Tools

8. Finding “Likely Suspects”
User Complaints
Compile with XREF
compile “program.p” xref “tmp/program.xrf” debug-list “tmp/program.dbg”.
“I/O By User” Data
CRUD Data
Testing – Add Performance Criteria to Test Plans for New Releases

9. XREF c:\examples\t5.p 1 COMPILE c:/profiler/examples/t5.p
c:\examples\t5.p 1 CPINTERNAL ISO8859-1
c:\examples\t5.p 1 CPSTREAM ISO8859-1
c:\examples\t5.p 7 STRING "i" 1 NONE UNTRANSLATABLE
c:\examples\t5.p 13 STRING "Customer" 8 NONE UNTRANSLATABLE
c:\examples\t5.p 13 ACCESS sports2000.Customer Phone
c:\examples\t5.p 13 STRING " " 12 NONE TRANSLATABLE
c:\examples\t5.p 13 SEARCH sports2000.Customer CustNum WHOLE-INDEX
c:\examples\t5.p 15 STRING "->,>>>,>>9" 10 NONE TRANSLATABLE FORMAT
c:\examples\t5.p 17 STRING "t5" 2 NONE TRANSLATABLE
c:\examples\t5.p 17 STRING "x(2)" 4 NONE TRANSLATABLE FORMAT
c:\examples\t5.p 22 STRING "i" 1 LEFT TRANSLATABLE
c:\examples\t5.p 22 STRING " " 10 NONE UNTRANSLATABLE
c:\examples\t5.p 22 STRING "CustNum" 7 NONE UNTRANSLATABLE

10. PROMON – IO By Process 04/08/04 I/O Operations by Process 11:00:00
Database BI AI -----
Usr Name Access Read Write Read Write Read Write
0 lakewood
10 lakewood
11 lakewood
12 lakewood
13 smcnulty
14 eratclif
16 sstout
17 lakewood
18 lakewood
19 aracey
20 lakewood

11. ProTop – IO By User
09:37: ProTop -- Progress Database Monitor (release xv) /19/04
Sample sports2000 [/data/s2k/sports2000] Rate
Hit Ratio: : : Commits: Local:
Miss% : % % Latch Waits: Remote:
Hit% : % % Tot/Mod Bufs: Batch:
Log Reads: Evict Bufs: Server:
OS Reads: Lock Table: Other:
Chkpts: Lock Tbl HWM: TRX:
Flushed: Old/Curr BI: Blocked:
Area Full: % After Image: DISABLED Total:
UIO
Usr Name Flags PID DB Access OS Reads OS Writes Hit%
31 julia SB %
30 jami SB %
34 tucker SB %
9 tucker SB* %
6 julia SB %
32 peter SB %
16 julia SB %

12. ProTop – CRUD Data
09:38: ProTop -- Progress Database Monitor (release xv) /19/04
Sample sports2000 [/data/s2k/sports2000] Rate
Hit Ratio: : : Commits: Local:
Miss% : % % Latch Waits: Remote:
Hit% : % % Tot/Mod Bufs: Batch:
Log Reads: Evict Bufs: Server:
OS Reads: Lock Table: Other:
Chkpts: Lock Tbl HWM: TRX:
Flushed: Old/Curr BI: Blocked:
Area Full: % After Image: DISABLED Total:
Table Statistics
Tbl# Table Name Create Read Update Delete
4 OrderLine
24 POLine
23 PurchaseOrder
18 Order
21 Bin
2 Customer
12 Vacation

13. Why “Logical I/O” ??? Consistent and Repeatable Measurement
The same query against any given dataset will always return the same result.
Not subject to external factors such as CPU speed, disk throughput, user activity or the buffer cache hit ratio.
Shows Hidden Problems even with small datasets.
Shows Impact on Other Users.
“Chokepoint” on rate of Logical IO ops.

14. Why NOT etime() ??? Non-Repeatable
Subject to a host of external factors
CPU speed, disk throughput, other user activity, buffer cache efficiency, phase of the moon
Granularity is too gross (millisecond)
Does measure non-db activity…

15. LRTEST.p define variable i as integer no-undo.
define variable lr as integer no-undo.
find _myconnection no-lock.
find _userio no-lock where
_userio-usr = _myconn-userid no-error.
lr = _userio._userio-dbaccess.
etime( yes ).
find <table> no-lock where <whatever> no-error.
display i ( _userio-dbaccess - lr ) etime().

16. Example Production Development QA
for each loanfile no-lock where loan-amount > :
Records
Log I/O
Ratio
etime()
Production
383
335,597
876:1
6,685ms 2,788ms
Development 41
28,517
695:1
647ms 394ms QA
365
327,284
896:1
4,880ms 2,965ms
for each loanfile no-lock where price-locked >= 9/01/2011 and price-locked <= 9/30/2011:
1,498
3,001
2:1
64ms 52ms 3
N/A
19ms 0ms
1,501
3,010

17. The performance enhancement possible with a given improvement is limited by the fraction of the execution time that the improved feature is used.
-- Amdahl’s Law

18. Target the largest response time component of the most important Business process first.

19. Wouldn’t it be nice if… Description: XYZZY Date: 10/04/11
Top Total Time Lines
Program Line Avg Time Time Calls
xtabsms2.p
getdocprep2.p
proc_create_sitm xtabsms2.p
xtcountry_x2_x3.p ,967
proc_read-database sysval.p ,879
xtcountry_x2_x3.p ,894
getdocprep2.p
proc_upd_nref xtmfintb2.p
proc_upd_nref xtmfintb2.p
proc_process_tasks xttskscn.r
findClient sysval.p ,763

20. The Profiler

21. Profiler First introduced with version 8.2 (-zprofile)
“Unsupported” (meaning the analysis tool)
Improved with version 9.0 (session:profiler handle, no more -zecret)
Microsecond timings
Does not include “think time”

22. Using the Profiler -profile profiler: handle Analysis tools
Non-intrusive
Non-selective
profiler: handle
Selective
But requires code insertion or “wrappering”
Analysis tools
$DLC/src/samples/profiler

23. PROFILER Attributes
DESCRIPTION – optional text describing this session
LISTINGS – whether or not to create debug listings
DIRECTORY – where to create debug listings (default to –T)
FILE-NAME – name of output file (default profile.out)
ENABLED – yes/no; initializes listings and so forth
PROFILING – turn profiling on or off

24. Other PROFILER Attributes
TRACE-FILTER – CSV list of “matches” criteria for procedure tracing
TRACING – line level tracing
COVERAGE - % coverage support

25. PROFILER Methods Write-Data() – flush accumulated data to output file.
User-Data(char) – write user defined data, such as VST statistics, to the output file.

26. Minimal Embedded Usage
assign profiler:enabled = yes profiler:profiling = yes . do i = 1 to : /* do something */ end. assign profiler:enabled = no profiler:profiling = no . profiler:write-data().

27. Targeted Profiling Embedded in Code Being Investigated
define variable i as integer no-undo.
run profiler/on.p ( “batch001” ).
do i = 1 to 10:
find customer no-lock where customer.cust-num = 1 no-error.
end.
find customer no-lock where customer.phone = "(702) " no-error.
run profiler/off.p ( “batch001” ).

28. Unsupported Profiling Utility

29. Profiling a Session
Create file called profiler.cfg with 3 lines: -OUTFILE /tmp/profiler.out -LISTINGS /tmp -DESCRIBE someDescription
Add –profile to session startup: mpro dbName –p start.p –profile profiler.cfg
Run normally.
Terminate cleanly & analyze the output.

30. Sample Profiling Output
Description: XYZZY Date: 10/04/11
Top Total Time Lines
Program Line Avg Time Time Calls
xtabsms2.p
getdocprep2.p
proc_create_sitm xtabsms2.p
xtcountry_x2_x3.p ,967
proc_read-database sysval.p ,879
xtcountry_x2_x3.p ,894
getdocprep2.p
proc_upd_nref xtmfintb2.p
proc_upd_nref xtmfintb2.p
proc_process_tasks xttskscn.r
findClient sysval.p ,763

31. Profiler Example A Calculation Bottleneck?
p = 4 * ( )

32. Profiler Example A Calculation Bottleneck?
0009 function piterm returns decimal ( input n as integer ).
return ( 1.0 / (( n * 2 ) + 1 )).
0011 end.
0012
0013 do while abs( newpi - oldpi ) > precision:
oldpi = newpi.
if i modulo 2 = 0 then
pi = pi + piterm( i ).
else
pi = pi - piterm( i ).
newpi = ( 4.0 * pi ).
display i newpi oldpi.
i = i + 1.
0022 end.

33. Sample Profiling Output
Description: pi Date: 10/07/11
Top Total Time Lines
Program Line Avg Time Time Calls
./pi.p ,001
piterm ./pi.p ,001
./pi.p ,002
./pi.p ,000
./pi.p ,001
./pi.p ,001
./pi.p ,001
./pi.p ,001
./pi.p ,001
/home/tom/p26226_Untitled1.ped
piterm ./pi.p ,001
./pi.p ,001

34. Profiler Example A Calculation Bottleneck?
0009 function piterm returns decimal ( input n as integer ).
return ( 1.0 / (( n * 2 ) + 1 )).
0011 end.
0012
0013 do while abs( newpi - oldpi ) > precision:
oldpi = newpi.
if i modulo 2 = 0 then
pi = pi + piterm( i ).
else
pi = pi - piterm( i ).
newpi = ( 4.0 * pi ).
if i modulo 100 = 0 then display i newpi oldpi.
i = i + 1.
0022 end.

35. Sample Profiling Output
Description: pi Date: 10/07/11
Top Total Time Lines
Program Line Avg Time Time Calls
piterm ./pi.p ,001
./pi.p ,002
./pi.p ,000
./pi.p ,001
./pi.p ,001
./pi.p ,001
./pi.p ,001
./pi.p ,001
./pi.p ,001
piterm ./pi.p ,001
./pi.p ,001

36. Caveat!
define variable i as integer no-undo. assign profiler:enabled = yes profiler:profiling = yes . do i = 1 to : end. i = 0. do while i < : i = i + 1. end. i = 0. do while i < : i = i + 1. end. assign profiler:enabled = no profiler:profiling = no. profiler:write-data().

37. Caveat!
1 02/28/2007 "Generic" 07:55:03 "tom" . 2 "profile.p" ""
Note the result for line 16 (the last line shown in prof.out). (The module 0, line 0 line is also curious but it isn't my focus right now.) I can assure you that it took much longer the seconds for that code to execute It would seem that source lines with multiple statements on them are not correctly accounted for.

38. Reminders & Hints Line numbers are DEBUG LISTING line numbers
You need to have .DBG files
They need to be created with the same source and PROPATH as your .r files
Session profiling must terminate cleanly – no “kill” & no crash.
Temp files can become very, very large.

39. Summary Things to be suspicious of. Tools to narrow your search.
A better way to gauge query effectiveness.
An introduction to Profiling.
Strategies for attacking code performance problems.
The Stuff That We’ve Learned Today!

40. Target the largest response time component of the most important Business process first.

41. ?
Questions

42. There is no silver bullet. -- Fred Brooks
You just have to be persistent.
-- Tom Bascom
Ok, he wasn’t talking about quite the same thing ;-)

43. Thank you for your time! tom@wss.com http://wss.com