We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byViviana Manton
Modified over 2 years ago
Global Payroll Performance Optimisation - II David Kurtz Go-Faster Consultancy Ltd.
Global Payroll Performance Optimisation © Oracle Database Specialist –Independent consultant Performance tuning –PeopleSoft ERP –Oracle RDBMS Book –www.psftdba.comwww.psftdba.com UKOUG Director Server Tech & PeopleSoft Oak Table Who Am I?
Global Payroll Performance Optimisation © Agenda ‘Streaming’ –Parallel processing Data Volume Read Consistency Partitioning Reporting Archiving
Global Payroll Performance Optimisation © Warning This is an unashamedly technical session. I am going to talk about database internals.
Global Payroll Performance Optimisation © Size matters!
Global Payroll Performance Optimisation © Parallel processing All modern machines have multiple processors, –most of the processors have multiple cores. –Even the CPU in my 4 year old laptop has a 2 core CPU.
Global Payroll Performance Optimisation © Database Parallelism All objects in the PeopleSoft schema are explicitly set NOPARALLEL –Indexes are built parallel, but later reset. –Can invoke parallel query with PARALLEL hint –Parallel insert in direct path model –Parallel DML only works on partitioned objects 1 PQ slave per partition
Global Payroll Performance Optimisation © PeopleSoft Batch Programs Only run on one CPU at any one time. Client Server processes –Program (COBOL or Application Engine) –Database (eg. Oracle) Either busy executing COBOL or waiting for the database. –If your payroll calculation is a single process you are not getting value for money!
Global Payroll Performance Optimisation © Payroll ‘Streaming’ Several GP processes can be split up. –Each piece processes a distinct set of employees Range of EMPLID –The pieces can be run concurrently. –Maximum number of streams determined by hardware.
Global Payroll Performance Optimisation © Streamable Processes COBOL –Payroll Calculation Application Engine –Banking Preparation –GL Preparation –EDI Preparation –Payslip Preparation Database Intensive
Global Payroll Performance Optimisation © Payroll ‘Streaming’ Challenges Payroll isn’t over until the last stream completes. –Streams need to be evenly balanced. –Employee churn? One global definition of streams –Balance for largest payroll? Inter-stream contention –Shared working storage tables in COBOL
Global Payroll Performance Optimisation © Payroll Calculation Process Phases Identify –Populate working storage and some result tables Database Intensive Calculation –COBOL Intensive Cancellation –Delete results Database Intensive
Global Payroll Performance Optimisation © How Many Streams? In a well tuned systems, the payroll calculation phase spends about –2/3 of its time in COBOL –1/3 on the database. Number of streams should not exceed –3 * CPU on database server –1.5 * CPU on Process Scheduler server Payroll identification process is database intensive.
Global Payroll Performance Optimisation © How Many Streams? First GP I ever worked on –20 CPUs on Application/Batch server –20 CPUs on Database server Maximum number of streams? –20 / 1/3 = 60 on database server –20 / 2/3 = 30 on Application server So we used 30 streams –Application server fully utilised during payroll calc –Database about 50% during calc, –Probably overloaded during identification.
Global Payroll Performance Optimisation © How Many Streams? Optimise number of streams for calculation phase. Restrict concurrency of database intensive process on process scheduler. –To limit CPU consumption, and possibly also I/O contention. Consider use of Oracle Resource Manager –Mainly for Payroll identification –I’ve never had to do this myself. –Cancellation will be restricted by I/O
Global Payroll Performance Optimisation © Balancing Streams Balance employees across streams on basis of –80% number of payroll segments per stream –20% number of JOB history rows Longer serving employees in earlier streams likely to have more payroll segment and job history. –Make allowance for employee churn. You will need to periodically rebalance the streams. –Balance for the largest payroll.
Global Payroll Performance Optimisation © Employee Churn EMPLID is allocated as an accession number. Streams are a range of EMPLIDs –New employees are hired into the last stream –Employees are terminated across all streams Over time the streams will go out of balance –Last stream will take longest Periodically rebalance the streams
Global Payroll Performance Optimisation © Bulk Churn Effects Migration –If migrated to GP in tranches then order of migration could affect stream balance Company merger/divestment history can affect balance of payroll.
Global Payroll Performance Optimisation © Rebalancing the streams? Calculate new stream range values –Allow space for estimated future growth Rebuild all range partitioned tables –Half the I/O of partition merge/split –About 42 tables in UK tables. –Need working storage space to do this
Global Payroll Performance Optimisation © Reversing the EMPLID Reverse the EMPLID –Instead of EMPLID –Use EMPLID Streams stay balanced because new employees hired across range Improved search performance across HCM BUT you must do this before you go live!
Global Payroll Performance Optimisation © Reversing the EMPLID
Global Payroll Performance Optimisation © Inter-stream Contention Streams are just ranges of EMPLIDs. Oracle inserts data into the first available block (roughly speaking) Multiple streams insert data simultaneously into the same data blocks in result tables. Payroll cancel/recalculation deletes from result tables. Multiple transactions concurrently update different rows in the same block. –On Oracle/SQL Server >=2005: No locking, streams continue to run, but read consistency processing is expensive –Other database can experience page level locking
Global Payroll Performance Optimisation © Working Storage Tables COBOL –One shared instance of each working storage table Shared SQL –Candidate for Global – Temporary Table so one instance per session Application Engine –PeopleSoft Temporary Record –One instance of record per process Different SQL Still consider GTT to reduce redo
Global Payroll Performance Optimisation © Read Consistency The data set that you query remains the same throughout the life of your query. –If somebody else updates data that you are reading (and commits), after your query starts, then you see the original value. Thus, readers do not block writers or vice versa. Oracle has always done this, like this since SQL Server 2005 has ‘read committed snapshot’ option Other databases either block or can permit ‘dirty read’.
Global Payroll Performance Optimisation © Read Consistency Oracle achieves this by storing ‘undo’ information for every change –Recovers ‘read-consistent’ in-memory copy of data block to point in time when query started. –A good reason for buying Oracle –Resource intensive process –Performance problem if abused. Global Payroll is the perfect storm!
Global Payroll Performance Optimisation © Read Consistency
Global Payroll Performance Optimisation © Avoiding Inter-stream Contention Prevent different streams accessing the same data blocks –Range Partition result tables to match stream ranges –Use Global Temporary Tables (Oracle) for working storage tables –Partition these also on other platforms. Now different streams access different partitions. No code change, a job for the DBA –licensed option on most platforms
Global Payroll Performance Optimisation © Partitioning Partitioned Table –Different physical components Value of data determines physical location –Logically still one table –Transparent to application –Rather like a multi-part encyclopaedia. Partition Elimination
Global Payroll Performance Optimisation © What is Partitioning? Typically used in DSS But can also be effective in OLTP –(From Oracle documentation)
Global Payroll Performance Optimisation © Partitioning Keep similar things together –Employees for one stream in on partition Keep different things apart –Only one transaction in each block of each segment –No need for read consistency
Global Payroll Performance Optimisation © Partitioning GP Recommendation Range Partitioning –EMPLID – to match streams List Sub-partition –CAL_RUN_ID – calendar group ID.
Global Payroll Performance Optimisation © Secondary Benefits CAL_RUN_ID list sub-partition Easier to archive later –Historical partitions –Different Tablespaces –Different Data Files Old data on slower disk Read Only Less frequent back-up of read-only tables Faster Backup
Global Payroll Performance Optimisation © Global Temporary Tables Global because the data is private Temporary because the definition is permanent Global because everyone can see the definition Temporary because physical existance of the table is temporary so it does not need to be recovered.
Global Payroll Performance Optimisation © Global Temporary Tables A temporary object –No redo generation But there is undo, and there is redo on the undo! –Each session gets its own physical copy. Again no read consistency problems No high water mark issues Lower high water marks – less I/O
Global Payroll Performance Optimisation © Building the DDL Demonstrate GFCBUILD utility.
Global Payroll Performance Optimisation © Group Lists Specify a list of individual EMPLIDs for whom to run pay calc or another process. Some customers have experienced problems when run groups shortly before or during larger batch payroll calculations. Why?
Global Payroll Performance Optimisation © Cost Based Optimizer SQL Execution Plan Caching Bind Variable Peeking during Parse Different Plan for Group List –Because different bind variables But plan cached and gets used for main pay calculation which then runs longer than usual!
Global Payroll Performance Optimisation © Plan Stability Remember the good plan used by large payroll. Force it to be used for all payrolls including group list. –Data Volumes small so poor plan won’t really matter. Oracle Stored Outline –No code change, DBA can implement.
Global Payroll Performance Optimisation © Plan Stability Collect and applied stored outline with database trigger –http://www.go- faster.co.uk/gpdoc.htm#gp.stored_outlineshttp://www.go- faster.co.uk/gpdoc.htm#gp.stored_outlines Use Active Session History to demonstrate the problem and solution
Global Payroll Performance Optimisation © Capture Stored Outline CREATE OR REPLACE TRIGGER sysadm.gfc_create_stored_outlines BEFORE UPDATE OF runstatus ON sysadm.psprcsrqst FOR EACH ROW WHEN (new.prcsname = 'GPPDPRUN' AND (new.runstatus = 7 OR old.runstatus = 7)) DECLARE l_sql VARCHAR2(100); BEGIN l_sql := 'ALTER SESSION SET create_stored_outlines = '; IF :new.runstatus = 7 THEN EXECUTE IMMEDIATE l_sql||:new.prcsname; ELSIF :old.runstatus = 7 THEN EXECUTE IMMEDIATE l_sql||'FALSE'; END IF; --because I dont want to crash the process scheduler EXCEPTION WHEN OTHERS THEN NULL; END; /
Global Payroll Performance Optimisation © Apply Stored Outline CREATE OR REPLACE TRIGGER sysadm.gfc_use_stored_outlines BEFORE UPDATE OF runstatus ON sysadm.psprcsrqst FOR EACH ROW WHEN (new.prcsname = 'GPPDPRUN' AND (new.runstatus = 7 OR old.runstatus = 7)) DECLARE l_sql VARCHAR2(100); BEGIN l_sql := 'ALTER SESSION SET use_stored_outlines = '; IF :new.runstatus = 7 THEN EXECUTE IMMEDIATE l_sql||:new.prcsname; ELSIF :old.runstatus = 7 THEN EXECUTE IMMEDIATE l_sql||'FALSE'; END IF; --because I dont want to crash the process scheduler EXCEPTION WHEN OTHERS THEN NULL; END; /
Global Payroll Performance Optimisation © Three Scenarios Compared Large / Small / Plan Stable Small SQL_ID SCENARIO 1 ASH_SECS SCENARIO 2 ASH_SECS SCENARIO 3 ASH_SECS uzmzh74rdrnz **SAME** n482cm7r9qyn **SAME** 889 2f66y2u54ru1v **SAME** 531 1n2dfvb3jrn2m **SAME** y9682bqqvp **SAME** 110 d8gxmqp2zydta **SAME** 32 2np47twhd5nga **SAME** 27 4ru0618dswz3y ru0618dswz3y **SAME** 22 4ru0618dswz3y ru0618dswz3y gnnu2hfkjm2yd **SAME** 19 fxz4z38pybu3x xkjjwvmyf99c **SAME** 18 a05wrd51zy3kj **SAME** 15
Global Payroll Performance Optimisation © Data Volume Payroll generates a lot of data. Every pay period it generates more data. Partitioning can offer ways of accessing the data you want quickly –Without having to trawl through data you don’t want. Need to consider how long you need data –Do you still need data from last tax year?
Global Payroll Performance Optimisation © Archiving Put the data you do need to keep into a reporting table –Remove data from the live result tables –Partitioning can help you move/delete this data efficiently –May need to rebuild tables where you have to use DELETE Reduced data volumes should improve performance of reports.
Global Payroll Performance Optimisation © Reporting Payroll result tables delivered with single index –Not suitably indexed for all reporting requirements Particularly single PIN queries –Adding more indexes would degrade calculation performance –Consider generating reporting table Subset of data, and indexed as necessary.
Global Payroll Performance Optimisation © GFC_GPRPTGEN Reporting Table for Single Pin Queries –List Partitioned by Pin –One Partition for each Pin Incremental Maintenance by Application Engine –Uses Parallel DML to maintain reporting table. –Sub-Paritioned GP Result Tables may still be faster for single employee, single calendar group ID queries!
Global Payroll Performance Optimisation © Further Reading Configuring and Operating Streamed Processing in PeopleSoft Global PayrollConfiguring and Operating Streamed Processing in PeopleSoft Global Payroll –www.go- faster.co.uk/gpdocs.htm#Configuring_Operating_Streamed_Payrol l Managing Oracle Table Partitioning in PeopleSoft Applications with GFC_PSPART PackageManaging Oracle Table Partitioning in PeopleSoft Applications with GFC_PSPART Package –www.go- faster.co.uk/gpdocs.htm#Managing_Oracle_Table_Partitioning Use of Oracle Plan Stability (Stored Outlines) in PeopleSoft Global PayrollUse of Oracle Plan Stability (Stored Outlines) in PeopleSoft Global Payroll –www.go-faster.co.uk/gpdocs.htm#gp.stored.outlines
Global Payroll Performance Optimisation - I David Kurtz Go-Faster Consultancy Ltd.
Go-Faster Consultancy Ltd.1 Experiences of Global Temporary Tables in Oracle 8.1 David Kurtz Go-Faster Consultancy Ltd.
C Copyright © 2005, Oracle. All rights reserved. Practice Solutions.
11 1 Chapter 11 Database Performance Tuning and Query Optimization Database Systems: Design, Implementation, and Management, Seventh Edition, Rob and Coronel.
BMU - E I 1 Development of renewable energy sources in Germany in
Time for a BREAK! You have 45 Minutes. Time Left 44.
13 Copyright © 2005, Oracle. All rights reserved. Monitoring and Improving Performance.
Partition, Compress, Archive and Purge Keep Your System on the Road David Kurtz Go-Faster Consultancy Ltd.
BMU – KI III 1 Development of renewable energy sources in Germany in
PSSA Preparation. Question 1(no calculator) D Question 2 (no calculator)
11 Copyright © Oracle Corporation, All rights reserved. Managing Tables.
13 Copyright © 2005, Oracle. All rights reserved. Using the SQL Access Advisor.
Petersons Practice AP Exam Cengage Learning Infotrack Database.
1 Database Systems: Design, Implementation, and Management CHAPTER 10 Distributed Database Management System.
PP Test Review Sections 6-1 to 6-6 Mrs. Rivas 1. 2.
Copyright © 2012, Elsevier Inc. All rights Reserved. 1 Chapter 7 Modeling Structure with Blocks.
Graphing AWR Data in Excel David Kurtz Go-Faster Consultancy Ltd.
A Guide to MySQL 4. 2 Objectives Retrieve data from a database using SQL commands Use compound conditions in queries Use computed columns in queries Use.
Silberschatz, Galvin and Gagne Operating System Concepts Chapter 10: Virtual Memory Background Demand Paging Process Creation Page Replacement.
Modelling a Complex Batch Schedule in PeopleSoft David Kurtz Go-Faster Consultancy Ltd.
×1= 9 4 1×1= 1 5 8×1= 8 6 7×1= 7 7 8×3= 24.
1 BACKGROUND ALARM SYSTEM OF ZXJ10 Training Center Zhongxing Telecom Pakistan (Pvt.) Ltd.
Copyright © 2003 Pearson Education, Inc. Slide 1 Computer Systems Organization & Architecture Chapters 8-12 John D. Carpinelli.
9 Copyright © 2006, Oracle. All rights reserved. Automatic Performance Management.
Numbers Treasure Hunt
Chapter 12 Membrane Transport Essential Cell Biology Third Edition Copyright © Garland Science 2010.
Go-Faster Consultancy Ltd.1 Single Table Clusters, an alternative to partitioning? David Kurtz Go-Faster Consultancy Ltd.
User Friendly Price Book Maintenance A Family of Enhancements For iSeries 400 DMAS from Copyright I/O International, 2006, 2007, 2008, 2010 Skip Intro.
1 File Systems Chapter Files 6.2 Directories 6.3 File system implementation 6.4 Example file systems.
Threads, SMP, and Microkernels Chapter 4 1. Process Resource ownership - process includes a virtual address space to hold the process image Scheduling/execution-
Welcome. © 2008 ADP, Inc. 2 Overview A Look at the Web Site Question and Answer Session Agenda.
AP STUDY SESSION 2. Answers 1.A 2.E 3.A 4.D 5.B 6.E 7.B 8.E 9.A 10.D 11.C 12.B 13.D 14.B 15.E 16.A 17.E 18.C 19.C 20.D 21.B 22.C 23.A 24.D 25. B 26. E.
Microsoft Confidential. We look at the world... with our own eyes...
Copyright © Action Works 2008 All Rights Reserved - Photos by David D. Kempster 1.
Adding Up In Chunks. Category 1 Adding multiples of ten to any number.
Copyright © 2007 Ramez Elmasri and Shamkant B. Navathe Slide
1 Concurrency: Deadlock and Starvation Chapter 6.
1 Senn, Information Technology, 3 rd Edition © 2004 Pearson Prentice Hall James A. Senns Information Technology, 3 rd Edition Chapter 7 Enterprise Databases.
CS 6143 COMPUTER ARCHITECTURE II SPRING 2014 ACM Principles and Practice of Parallel Programming, PPoPP, 2006 Panel Presentations Parallel Processing is.
The DDS Benchmarking Environment James Edmondson Vanderbilt University Nashville, TN.
David Burdett May 11, 2004 Package Binding for WS CDL.
1 tRelational/DPS Overview. 2 ADABAS Data Transfer: business needs and issues tRelational & DPS Overview Summary Questions? Demo Agenda.
Sample Service Screenshots Enterprise Cloud Service 11.3.
Copyright © 2011, Elsevier Inc. All rights reserved. Chapter 6 Author: Julia Richards and R. Scott Hawley.
By CA. Pankaj Deshpande B.Com, FCA, D.I.S.A. (ICA) 1.
Break Time Remaining 10:00. Break Time Remaining 9:59.
© 2017 SlidePlayer.com Inc. All rights reserved.