Presentation is loading. Please wait.

Presentation is loading. Please wait.

PeopleSoft Performance Tuning at ab Technology Session Number 610 October 23, 13.15.

Similar presentations


Presentation on theme: "PeopleSoft Performance Tuning at ab Technology Session Number 610 October 23, 13.15."— Presentation transcript:

1 PeopleSoft Performance Tuning at ab Technology Session Number 610 October 23,

2 Page 2 abab © R.Dijkman, © Go-Faster Consultancy Ltd. About UBS Group  UBS is a global, integrated investment services firm and the leading bank in Switzerland:  UBS Wealth Management & Business Banking consists of the world's leading private banking and Swiss retail banking business  UBS Global Asset Management is a leading institutional asset manager and mutual fund provider.  UBS Warburg operates globally as a client-driven securities, investment banking & wealth management firm  UBS PaineWebber is one of the top US wealth managers

3 Page 3 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Project Overview  HRMS 8  Global Payroll  Swiss Extension  employees (34000 payroll)  > 200Gb Data  400 HR CRM users  All users via PIA Self-Service

4 Page 4 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Configuration  HP V Class, 2x20CPU, 24/10Gb  HP/UX11, MC ServiceGuard, EMC  Oracle , PT8.15, Apache Production

5 Page 5 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Challenges  Large HRMS implementation  Lots of customisations  Payroll is a ‘financial’ batch  Oracle bugs  Performance Problems

6 Page 6 abab © R.Dijkman, © Go-Faster Consultancy Ltd.

7 Page 7 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Agenda  What is Performance Tuning?  On-Line  Batch  What did we achieve at UBS?  Questions?

8 Rules of Engagement  If you can’t hear me - say so now!  Feel free to ask on-topic questions during the presentation.  This presentation (with notes) is available on the conference website (after the conference) and on 

9 Aphorism  Performance is exactly what the user perceives it to be. No more, no less.

10 Page 10 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Aphorism  Poor performance is when the user’s perception does not match their expectation.

11 Page 11 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Aphorism  Performance Tuning is a search for lost time.

12 Page 12 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Aphorism  When you have eliminated the impossible, whatever remains, however improbable, must be the truth.  A Study in Scarlet, Arthur Conan-Doyle  Detection is, or ought to be, an exact science. It should be be treated in the same cold and unemotional manner.  The Sign of Four, Arthur Conan-Doyle

13 Page 13 abab © R.Dijkman, © Go-Faster Consultancy Ltd. PIA (On-Line) Performance  Architecture  brief overview  Sources of Metrics  Graphical Techniques  Tracing  Techniques

14 Page 14 abab © R.Dijkman, © Go-Faster Consultancy Ltd. PIA Architecture  PIA is 4-tier architecture  Each component can contribute to response time  Need to measure each component Java Servlet (presentation logic) Java Servlet (presentation logic) Tuxedo Application Server (application logic) Tuxedo Application Server (application logic) DBMS (application data & meta-data DBMS (application data & meta-data SQL Tuxedo Message http / https

15 Page 15 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Sources of Metrics  Browser  3rd Party software  Local agent on PC  Proxy Server  Web Server  Access Log  BEA/Tuxedo  Service Trace  TMADMIN  Database (Oracle)  SQL_Trace  PSFT Application Server Trace

16 Page 16 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Web/Proxy Access Log  Standard format for access log   (Apache is different - see manual)  Fields  Date  Time  Time Taken  accurate to OS time units (1/100 th s or 1/1000 th s)  only 1s on Apache  Message bytes  IP addresses & DNS names  URI stem & Query

17 Page 17 abab © R.Dijkman, © Go-Faster Consultancy Ltd. How to Enable Weblogic Trace #weblogic.properties #required weblogic.httpd.enableLogFile=true weblogic.httpd.logFileName=access.log weblogic.httpd.logFileFormat=extended #recommended weblogic.httpd.logRotationType=date weblogic.httpd.logRotationPeriodMins=1440 weblogic.httpd.logRotationBeginTime= :00:00

18 Page 18 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Sample Weblogic Access Log #Version:1.0 #Fields:date time time-taken bytes c-ip c-dns cs- method sc-status cs-uri-stem cs-uri-query :51: go- faster-1GET200 /servlets/iclientservlet/peoplesoft8/signon.html :51: go- faster-1GET200/peoplesoft8/signin.css :51: go- faster-1GET200 /peoplesoft8/images/secondaryPSFTLogo.gif :51: go- faster-1POST200 /servlets/iclientservlet/peoplesoft8/ ICType=Panel&Menu=ADMINISTER_WORKFORCE_(GBL)&Market= GBL&PanelGroupName=PERSONAL_DATA&RL=&target=main2

19 Page 19 abab © R.Dijkman, © Go-Faster Consultancy Ltd. How to Enable Apache Access Log #httpd.conf LogFormat "%{%Y.%m.%d %H:%M:%S}t|%T|%B|%u|%h|%{User- Agent}i|%>s|%m|%U|%q" monitoring CustomLog logs/access.log monitoring

20 Page 20 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Sample Apache Access Log :57:06|0|275|-| |Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; Q312461)|200|GET|/peoplesoft8/cache/PT_NEXTTAB_E NG_1.gif| :57:17|0|31847|-| |Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; Q312461)|200|POST|/servlets/iclientservlet|?ICTy pe=Panel&Menu=ADMINISTER_WORKFORCE_(GBL)&Market= GBL&PanelGroupName=JOB_DATA :57:26|0|30959|-| |Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0; Q312461)|200|POST|/servlets/iclientservlet|?ICTy pe=Panel&Menu=ADMINISTER_WORKFORCE_(GBL)&Market= GBL&PanelGroupName=JOB_DATA

21 Page 21 abab © R.Dijkman, © Go-Faster Consultancy Ltd. What does the access log give us?  A date and time for every request  Who made the request?  How long did it take to serve?  What was requested?  Which component (panel group)?

22 Page 22 abab © R.Dijkman, © Go-Faster Consultancy Ltd. BEA/Tuxedo Service Trace  Edit psappsrv.ubx file  CLOPT=“-r -e APPQ.stderr…”  -r enables trace to stderr file  -e qualifies name of stderr file

23 Page 23 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Edit psappsrv.cfg  Add a new variable to the trace section  Thus service trace can be enabled/disabled from configuration file  New variable will appear in the configuration dialogue [Trace] TuxedoServiceTrace=-r

24 Page 24 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Edit psappsrv.ubx  Change every CLOPT entry in psappsrv.ubx.  File name is set manually.  Same as queue name RQADDR="APPQ" CLOPT="{$Trace\TuxedoServiceTrace} -e {LOGDIR}{FS}APPQ.stderr {$PSAPPSRV\Spawn Server} -- -C {CFGFILE} -D {$Domain Settings\Domain ID} -S PSAPPSRV"

25 Page 25 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Resultant psappsrv.ubb  When domain is configured.ubb file is generated. All variables are fully resolved. CLOPT=”-r -e D:\ps\hr8\appserv\hr8d\LOGS\APPQ.stderr -p 1,600:3,10 - -sICQuery - sSqlQuery:SqlRequest -- -C psappsrv.cfg -D HR8D -S PSAPPSRV"

26 Page 26 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Sample Tuxedo Service Trace  Each service called to domain is logged SERVICE PID SDATE STIME EDATE ETIME

27 Page 27 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Tuxedo Service Trace  Service  Name of Service  Pid  An OS PID of listener or handler  Sdate/Edate  Start/End Date in Seconds since 00:00hrs GMT 1st January  Stime/Etime  Start/End Time in OS time units (1/ 100 th s or 1/1000 th s)  Hence calculate accurate service duration  Time on Server - not queuing

28 Page 28 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Txrpt Utility SERVICE SUMMARY REPORT SVCNAME 9a-10a TOTALS Num/Avg Num/Avg ICScript 8/1.43 8/1.43 ICPanel 6/3.47 6/3.47 JavaMgrGetObj 1/0.04 1/0.04 GetCertificate 1/0.73 1/ TOTALS 16/ /2.06

29 Page 29 abab © R.Dijkman, © Go-Faster Consultancy Ltd. BEA/Tuxedo TMADMIN Utility  Interactive command line interface  BEA/Tuxedo utility not PeopleSoft  Wrapped inside psadmin  Can be included in scripts  -r for read only in monitoring scripts  pq (printqueue)  psr (printserver)  pclt (printclient)

30 Page 30 abab © R.Dijkman, © Go-Faster Consultancy Ltd. PQ  What queues in domain  How many servers on queue  How many requests queued  (How much work on queue) Prog Name Queue Name # Serve Wk Queued # Queued Ave. Len Machine JSL.exe GO-FASTER+ JREPSVR.exe GO-FASTER+ PSSAMSRV.exe SAMQ GO-FASTER+ BBL.exe GO-FASTER+ WSL.exe GO-FASTER+ PSAPPSRV.exe APPQ GO-FASTER+

31 Page 31 abab © R.Dijkman, © Go-Faster Consultancy Ltd. PSR  What servers exist?  How many requests have they handled?  What are they doing right now? Prog Name Queue Name Grp Name ID RqDone Load Done Current Service BBL.exe GO-FAST ( IDLE ) PSAPPSRV.exe APPQ APPSRV ICPanel PSSAMSRV.exe SAMQ APPSRV ( IDLE ) WSL.exe BASE ( IDLE ) JSL.exe JSLGRP ( IDLE ) JREPSVR.exe JREPGRP ( IDLE )

32 Page 32 abab © R.Dijkman, © Go-Faster Consultancy Ltd. PCLT  Who is connected?  Are they doing something?  When did they connect? LMID User Name Client Name Time Status Bgn/Cmmt/Abrt GO-FASTER-1 NT WSH 0:21:14 IDLE 0/0/0 GO-FASTER-1 NT JSH 0:21:13 IDLE 0/0/0 GO-FASTER-1 NT tmadmin 0:00:00 IDLE 0/0/0 GO-FASTER-1 PS JavaClient 0:02:30 BUSY/W 0/0/0

33 Page 33 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Oracle SQL Trace  Tuning tool  Session trace  Format it with TKPROF  Find long running SQL statements  Total SQL Time for session call count cpu elapsed disk query current rows Parse Execute Fetch total

34 Page 34 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Location of Metrics Java Servlet (presentation logic) Java Servlet (presentation logic) Tuxedo Application Server (application logic) Tuxedo Application Server (application logic) DBMS (application data & meta-data DBMS (application data & meta-data SQL Tuxedo Message http / https Webserver Access Log Proxy Server Access Log Tuxedo Service Trace Oracle SQL*Trace Tuxedo tmadmin script

35 Page 35 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Graphical Analysis  Don’t drown in numbers  Draw a picture  Load Data into Database  SQL_Loader  Oracle 9i External Table  Process Data  Matching  Aggregation  Oracle Analytic Function  Load data into Excel  Draw Graph

36 Page 36 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Tips about Graphs  Connect Excel directly to a database  Aggregate data carefully  Too little and excel draws it badly because it adds a black border around an area  Too much and you lose details  Pivot tables are useful  but Excel97 buggy  Scatters and Trendlines  joining dots can obscure data  Excel supports points per series

37 Page 37 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Example: CPU Utilisation

38 Page 38 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Example: Apache Servlet Times

39 Page 39 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Longest running http conversations

40 Page 40 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Example: Tuxedo Service Time

41 Page 41 abab © R.Dijkman, © Go-Faster Consultancy Ltd. tmadmin - PCLT - Connect Client Sessions

42 Page 42 abab © R.Dijkman, © Go-Faster Consultancy Ltd. tmadmin - PCLT - Busy/Wait Users

43 Page 43 abab © R.Dijkman, © Go-Faster Consultancy Ltd. tmadmin - PQ - Queue Length

44 Page 44 abab © R.Dijkman, © Go-Faster Consultancy Ltd. tmadmin - PSR - print server processes

45 Page 45 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Relating different metrics produces new information

46 Page 46 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Batch Performance  Sources of Metrics  Graph

47 Page 47 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Sources of Metrics  Process Scheduler Request Table  PSPRCSRQST  need to archive this table  maintain history in another table with on DELETE trigger  Application Engine Steps  PS_BAT_% tables  AE_TRACE=1024  In Process Scheduler configuration (psprcs.cfg)  Report to database and/or trace file  Cobol Statement  Report to trace file  SQL Trace

48 Page 48 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Graphical Analysis

49 Page 49 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Top AE Statements AE PC % Cum % % Cum % Stmt Execute SQL SQL PC SQL SQL Total Total Rank Statement ID Count Time Time Time Time Time Time Time GP_PMT_PREP.CA000.CF000.S PeopleCode.Rowset.Fill GP_RSLT_ACUM GP_PMT_PREP.FA000.FH000.S GP_PMT_PREP.FA000.FC000.S GP_PMT_PREP.FA000.FG000.S GP_PMT_PREP.FA000.FB000.S PeopleCode.Rowset.Fill GP_RSLT_PIN PeopleCode.CNuK_ PeopleCode.CDfF0_PPNw …

50 Page 50 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Oracle SQL*Trace & TKPROF  Enable SQL Trace with Trigger  on psprcsrqst.runstatus from !7 to 7  Process trace with TKPROF  report Top n statements  sort by  Parse/Fetch/Execute  CPU/Elapsed Time  statement timings/execution plan call count cpu elapsed disk query current rows Parse Execute Fetch total

51 Page 51 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Aphorism  The problems change from release to release, but the methods by which they are investigated remain the same.  Jonathan Lewis (author of Practical Oracle 8i)

52 Page 52 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Questions?

53 Page 53 abab © R.Dijkman, © Go-Faster Consultancy Ltd. Aphorism  You know my methods, apply them.  A Study in Scarlet, Arthur Conan-Doyle

54 PeopleSoft Performance Tuning at ab Technology Session Number 610 October 23,


Download ppt "PeopleSoft Performance Tuning at ab Technology Session Number 610 October 23, 13.15."

Similar presentations


Ads by Google