1. Performance Load Testing Case Study – Agilent Technologies
September 20, 2018
Entegration Software

2. Agenda Introductions Background Testing Objectives Preparation Phase
Execution Phase
Analysis
Lessons Learnt
Contact Information
September 20, 2018
Entegration Software

3. Introduction Ramesh Padmanabhan Entegration Software
Consulting & product company based in San Jose
Proud to be service partners of
Oracle Corporation
Mercury Interactive
Yash Technologies
September 20, 2018
Entegration Software

4. Introduction Agilent Technologies $6 Billion Global Mfg Company
Over 30,000 employees in more than 50 countries
One of the largest global single instance installs of Oracle E-business suite
Consolidated over 150 legacy systems
Expect a maximum 5,000 concurrent users
September 20, 2018
Entegration Software

5. Background Largest single instance install
3 HP Superdomes –Production, Reporting, Planning
Single US based data center
Over 50 operating units
Significant business volume in Asia & Europe
Consolidating over 125 different legacy systems
Implemented all Financial & MFG Modules
September 20, 2018
Entegration Software

6. Testing Objectives
September 20, 2018
Entegration Software

7. Testing Objectives Validate single instance strategy
Validate network and hardware infrastructure
Scalability to 5000 concurrent users
Stress test for “high water mark”
Set user response time expectations
Identify and fix significant performance tuning issues within Oracle Applications
Identify and drive solutions for hardware issues
September 20, 2018
Entegration Software

8. Preparation Phase
September 20, 2018
Entegration Software

9. Data Gathering
Identified major transactions within each application module
Questionnaires sent for legacy data volumes by geography (US, Asia, Europe)
Short listed transactions with high volume or data intensive processing
Identified user distribution by region and by application areas
Determined estimation methodology for inquiry transactions
Numerous legacy systems and support groups. Data collection was complex.
September 20, 2018
Entegration Software

10. Hardware Preparation
Ensure that the production configuration of back-end server and middle tier machines were set-up and configured
Procure the Load generation agent boxes and have them installed and setup at the right locations
Ensure that the Cisco load balancing router was correctly set up
Set up network ‘sniffing’ devices to get detailed metrics of network traffic
September 20, 2018
Entegration Software

11. Software Preparation
Procure and install LoadRunner on the agent and controller boxes
Install LoadRunner and the Oracle Applications client on the machines of the scripters
Install/Setup other database monitoring software
Prepare scripts for detailed transaction analysis
September 20, 2018
Entegration Software

12. Data Preparation Validated various application setups
Initial cycles required all key master data to be fabricated
Developed numerous scripts to extract key data elements like items, customers, vendors etc. to be used in transactions
Ensured adequate breadth of data.
Identified key data and parameters for background load
September 20, 2018
Entegration Software

13. Develop LoadRunner Scripts
Recorded scripts for all the critical and high volume transactions
Adequate mix of inquiry and update txns.
Parameterized all the critical pieces of data like item, customer, orders etc.
Identified activities for which server response times were key and set up transaction timers around them e.g. commits, quick-picks etc.
September 20, 2018
Entegration Software

14. Execution Phase
September 20, 2018
Entegration Software

15. Build Test Scenarios
Develop matrix for users by geography by transaction
Manual scenarios
Goal oriented scenarios
Transactions split into three groups based on data dependency conditions
September 20, 2018
Entegration Software

16. Run Tests… 5 cycles of testing
1- validation cycle
2 – complete cycle with converted data
3- Stress test cycle
4- Complete integrated test with key interfaces and customizations
5- Production simulation run
Each cycle comprised of two major runs/day for two weeks. Each test run was about 4-7hrs long
September 20, 2018
Entegration Software

17. Run Tests…
5000 concurrent user load generated from 8 LoadRunner agents – 4 in US, 2 each in Europe & Asia
LoadRunner monitors set up for network, backend server & middle-tier boxes
Dedicated DBA and performance tuning experts monitored the HP Superdome server
September 20, 2018
Entegration Software

18. Analysis Used LoadRunner Analysis tool
Real time graphical interface to monitor the test progress
Post run analysis includes numerous graphs and transaction timers
More detailed analysis was done from the result data stored by LoadRunner in an Access database
September 20, 2018
Entegration Software

19. Analysis Data from the analysis used to
Set up realistic response time expectations from the end users
Modify various database parameters in the init.ora to better performance
Tweak settings of the Cisco load balancer for middle tier machines
Identify and tune some of the application code that had bad performance
September 20, 2018
Entegration Software

20. Limitations
Some performance intensive processes could not be tested due to data dependency issues e.g. lock-box receipts
Some dynamic and interactive processes could not be tested very well e.g. configured orders
Some custom code not stable till the last cycle
Some of the newer application modules not stable for a reasonable test
Application version and patch set lags
September 20, 2018
Entegration Software

21. Lessons Learnt
Performance test will only be as good as the data collected in the analysis phase
While performance test can significantly reduce risk of poor performance, it is not a guaranty
Initial performance testing cycles should focus more on non-code related performance variables
September 20, 2018
Entegration Software

22. Lessons Learnt
Intensive code related performance testing & tuning should take place after custom solutions have been put into testing and application patch sets are frozen
Performance testing should be in the critical path of project plan and performance testing instances should be patched just like the BST instances
Should plan on at least one marathon testing run that extends for 3 or 4 days
September 20, 2018
Entegration Software

23. Contact Information
Ramesh Padmanabhan
Entegration Software
September 20, 2018
Entegration Software