PeopleSoft High Performance and MAA Guidelines on Oracle Exalogic and Oracle Exadata' [CON7985] Latha Krishnaswamy| Director, Product Management | Exalogic Dennis McCormick, Systems Manager at City and County of San Francisco Aaron DeLosreyes, Director, Oracle Technology, Cognizant Oct 2nd, 2014 Oracle Confidential – Restricted

Oracle Confidential – Restricted

Program Agenda PeopleSoft on Exalogic - Anti-pattern traps to avoid…. What is Exalogic Elastic Cloud Software? Exalogic Elastic Cloud Software – PeopleSoft Optimizations PeopleSoft Reference Architecture on Exalogic MAA Best practices PeopleSoft on Exalogic Case Studies – City of San Francisco, Cognizant Demo Summary 1 2 3 4 5 6 7 Oracle Confidential – Restricted

PeopleSoft on Exalogic: Anti-Patterns Observed… Multiple PeopleSoft Environments Port conflict, session state Hardware (F5) & Software Load Balancers (Apache) vs OTD on Exalogic Non-separation of JVMs PIAs for FS, TE, HR – traffic crisscrossing Integration Broker on same JVM as PIA Scheduler logs For recovery & DR, log file share locations and continuous replications Choice of protocol Not TCP over JDBC, but rather SDPoIB JOLT over RDMA - PT 8.53+ PeopleTools mid-tier Shared storage use by different VMs Patching/upgrading multiple VMs COBOL is chatty. Scheduler on Exalogic. PeopleSoft Application database PUM, SYS DB, SES DB, migrate to Exadata Let’s begin our journey to embark on best ways to onboard PeopleSoft workload on Exalogic & Exadata……. Oracle Confidential – Restricted

Program Agenda What is Exalogic Elastic Cloud Software? 1 Exalogic Elastic Cloud Software – PeopleSoft Optimizations PeopleSoft Reference Architecture on Exalogic MAA Best practices PeopleSoft on Exalogic Case Studies – City of San Francisco, Cognizant Demo Summary 2 3 4 5 6 7 Oracle Confidential – Restricted

Exalogic Elastic Cloud System Software Enterprise Manager Middleware and Business Applications WebLogic Coherence Tuxedo Traffic Director Exalogic Control Exabus Integration Exabus Integration Exabus Integration Exabus Integration Oracle Linux Guest OS Physical Oracle Linux/Solaris Oracle VM 3 for Exalogic Exabus Exalogic Elastic Cloud Hardware = Exalogic Elastic Cloud Software

PeopleSoft on Exalogic – Building Blocks Enterprise Manager PeopleSoft Mid-Tier Deployment PIA WebLogic Application Server Tuxedo Scheduler/ Integration Broker Traffic Director Exalogic Control Exabus Integration Exabus Integration Exabus Integration Exabus Integration Oracle Linux Guest OS Physical Oracle Linux/Solaris Oracle VM 3 for Exalogic Exabus Exalogic Elastic Cloud Hardware = Exalogic Elastic Cloud Software

Exalogic Elastic Cloud Software Optimizations for PeopleSoft Workloads Oracle Confidential – Restricted

Tuxedo on Exalogic Increased Performance from Exabus Standard Hardware 7X 13,680 95,595 EECS 1.0 2.0X Exalogic Eliminates BRIDGE process as bottleneck Enables greater scalability of Tuxedo domains 7x throughput No application changes needed BRIDGE Single TCP Connection Over Ethernet Exabus BRIDGE dge Tuxedo Applications

Sockets Direct Protocol (SDP) Support Bypasses TCP/IP Stack Tuxedo Domain 2 Tuxedo Domain 1 WebLogic Server Node 1 Server 1 Server 2 /WS Clients Node 1 Server 3 Server 4 Jolt Clients This diagram shows just some of the places SDP can be utilized. DB Node 2 Server 1 Server 2

Exalogic Optimizations in WebLogic Server JDBC & Data Source Optimizations Active GridLink for RAC, Socket Direct Protocol (SDP) for JDBC I/O & Work Manager Optimizations Scattered reads / gathered writes, optimized self-tuning thread pool Cluster State Replication Optimizations Lazy deserialization, multiple channels for replication SDP Optimizations Network channels, JDBC connectivity Web Application Scalability and Performance Optimizations Oracle HTTP client connection management Faster Cluster Recovery Rapid death detection and failover Documentation: http://docs.oracle.com/middleware/1212/core/ FMWLC/products2.htm#CHDGCFDC Oracle Confidential – Restricted

Exalogic Optimizations in Coherence and Tuxedo Tuxedo Optimizations Efficient algorithm for Self-Tuning Lock Mechanism (SPINCOUNT) Optimized performance of intra-node communication (shared memory instead of IPC queue, reduced buffer copies) Documentation: http://docs.oracle.com/cd/E53645_01/tuxedo/ docs12cr2/exalogic/exalogic.html#wp1092546 Coherence Optimizations Coherence*Web caching performance optimizations Use of optimized point-to-point communications in Coherence clusters IMB transport option on Linux Documentation: http://docs.oracle.com/middleware/1212/cohe rence/COHAG/deploy_checklist.htm#COHAG53 00 Oracle Confidential – Restricted

Oracle Traffic Director – Overview Built-in Application Delivery Controller for Load Balancing over InfiniBand Request, content-based routing Request rate acceleration Request rate & connection limiting Quality of service tuning Built-in HA on engineered systems Dynamic reconfiguration HTTP Reverse proxy Support for SSL 3.0 and TLS 1.0 Web-application firewall Built-in HTTP, TCP load balancing Native Exabus integration SSL & TLS offloading Content caching & HTTP Compression Throughput Security High Performance Quality of Service High Availability Oracle Confidential – Internal

High Performance – PeopleSoft on Exalogic & Exadata Performance Test Results + Anecdotes Commodity http://moafshar.blogspot.com/2013/10/middleware-optimizations-for-exalogic.html Blog Entry URL: https://blogs.oracle.com/wernerwatschke/ Oracle Confidential – Restricted

Program Agenda What is Exalogic Elastic Cloud Software? 1 Exalogic Elastic Cloud Software – PeopleSoft Optimizations PeopleSoft Reference Architecture on Exalogic MAA Best practices PeopleSoft on Exalogic Case Studies – City of San Francisco, Cognizant Demo Summary 2 3 4 5 6 7 Oracle Confidential – Restricted

PeopleSoft on Exalogic (Virtual) & Exadata Reference Architecture vServers on Exalogic Infiniband connection Shared Storage (ZFSSA) Common PeopleTools VM Templates Multiple PIA - WLS Multiple AppServers Multiple Schedulers Multiple DB (App, SES) Multiple Workloads Application Isolation Better consolidation Patch/Upgrade faster Database RAC nodes …… Oracle Confidential – Restricted

PeopleSoft F5, PIA, OTD configuration on Exalogic virtual SSL Configured on F5 and OTD PeopleSoft PIA’s use F5 virtual address for SSL config PIA, Tuxedo, OTD Admin Node servers deployed on Exalogic Infiniband channel OTD Failover groups configured on EoIB Channel OTD used in Active – Passive configuration Oracle Confidential – Restricted

PeopleSoft OVM Template for Exalogic Simplified Provisioning, Easy Patching and High Performance Supports PeopleTools 8.53, 8.54 Rapidly test drive new PeopleTools release features: Fluid UI, Activity Guides Includes PIA and App Server – Weblogic 11g and Tuxedo 10gR3 Application agnostic – can support any PeopleSoft App Tier New template for every patch-set simplifying patching going forward High Performance – parity with bare metal Complete Instance Management Reconfigure and save Gold images

Apps Life Cycle on Exalogic - Simplified Patch apps using OVM Template Apps Patch sets - save 100s of hours and ensure consistency across App, Middleware and Hardware Manage Apps, Middleware, DB, Servers and Storage from single EM console Scale Up and Scale down app instances on Exalogic using EM and Apps OVM Templates Platinum Services resolve issues faster and reduce risk Application Operations EM for Apps Drill Through Views Data Center Operations - EM for Exalogic

Program Agenda What is Exalogic Elastic Cloud Software? 1 Exalogic Elastic Cloud Software – PeopleSoft Optimizations Reference Architecture MAA Best practices PeopleSoft on Exalogic Case Studies – City of San Francisco, Cognizant Demo Summary 2 3 4 5 6 7 Oracle Confidential – Restricted

VMs with shared storage on Exalogic virtual Save Time, Eliminate Mistakes, Follow Best Practices Shared PS_HOME, PS_APP_HOME on ZFSSA Facilitates patching & upgrading Can scale up VMs depending on load Typical Issues during Load Test Port locking Crisscrossing of OTD failover groups for PIAs PeopleSoft MAA Best practices Clear demarcation of VMs for each PIA PIA, AppServer per env i.e. FS, TE, HR Integration Broker on their own VMs OTD Active Passive setup Storage Replication ZFS storage shares scheduled replication Scheduler logs continuous replication PeopleSoft MAA OTN White Paper Oracle Confidential – Restricted

JD Edwards EnterpriseOne COLLABORATE 08 – Denver, Colorado USA Push Button Apps Deployment with Excelerators Save Time, Eliminate Mistakes, Follow Best Practices Delivered as Fixed Price ACS Service Available Now PeopleSoft Commerce EBS (H2FY15) Automated App Install From Days to hours Create Test Dev Cloud Apply MAA Best Practices Reduce manual tasks and errors Push-Button Application Infrastructure Service Production Ready Oracle App per MAA Infrastructure Install & Config Directives App Topology Design Provision VMs Configure App VMs Application Storage Networks Excelerator Exalogic Exadata Test-Dev App Cloud

Program Agenda What is Exalogic Elastic Cloud Software? 1 Exalogic Elastic Cloud Software – PeopleSoft Optimizations Reference Architecture MAA Best practices PeopleSoft on Exalogic Case Studies – City of San Francisco, Cognizant Demo Summary 2 3 4 5 6 7 Oracle Confidential – Restricted

San Francisco’s Exa Story San Francisco Overview Exa Upgrade Objectives and Criteria Exa Upgrade Implementation Timeline Then and Now Performance Maximum Availability Architecture Looking Back: Challenges and Lessons Learned Final Thoughts

City of San Francisco PeopleSoft 9.0 (Ben, HR, TL, Absence, Payroll,eSuites) Original 9.0 Implementation: Aug 2012 25 Environments on HCM 9.0 and HCM 9.2 (Upgrade) Production= 211 GB (compressed) 6 GB growth biweekly 30k employees, 100,000k covered lives, $100m payroll Controller’s eMerge (PeopleSoft) Division = 40 FTE Responsible for application (4 FTE), OS (2 FTE) and hardware Department of Technology: =200 FTE Responsible for database, storage, network and datacenters

Upgrade Project Objectives Refresh Hardware Move to new Production Data Center Move to new Disaster Recovery/Test/Dev Data Center Simplify Architecture

San Francisco’s Exa Implementation Timeline Upgrade Exadatas to 1/4 rack Exalogic new 1/8 rack Inception Excite Process Kickoff May 9th Install at New Production Data Center Dec 12th -20th Install at New Test/DEV/DR Data Center Feb 10-21st Go-Live June 6th Datacenter Lease Signed July 10th RFQ Issued October 25th Vendor Selected November 5th 1G Connection Established Jan 17th Apply July Patch Aug. 15th-28th Apply April Patch May 19-22nd Review and Analyze Procure Install & Configure Test & Develop Deploy Optimize May-13 Jun-13 Jul-13 Aug-13 Sep-13 Oct-13 Nov-13 Dec-13 Jan-14 Feb-14 Mar-14 April-14 May-14 June-14 July-14 Aug-14 Sept-14 Infrastructure Moved to 2 new data centers Built Test/DEV/DR from floor-up Established connectivity Hardware 2 1/8 rack Exadatas 2 1/8 rack Exalogics UCS for Windows Operating/Database System Upgraded from PT 8.5.0 to 8.5.3 Implemented RAC Implemented Active Dataguard Expanded OEM usage Implemented GoldenGate

Reference Architecture Simplify the visio!

Hardware Refresh: Key Criteria Create a stable, scalable base for future work Increase performance Implement MAA/DR Reduce Total Cost of Ownership: How? Simplify architecture across data centers: reduce training time Increased automation via OEM: more time for value added work One vendor: less time troubleshooting Increased performance: less overtime, more time for Q/A Other options considered: new blades, Vblock, hybrid model

Challenge Recommendation Oracle Stated Benefit CCSF Observation Complex Architecture Database Tier Install Exadata Engineered Solution lower cost and risk Has been met or exceeded Complex Architecture Application Tier Install Exalogic Engineered Solution with predefined PeopleSoft templates Has been met or exceeded, some issues with upgrade templates Live Reporting & Robust DR Install Golden Gate Active DataGuard Fast and Flexible Data Replication Single Points of Failure Install Real Application Clusters (RAC) High availability reduce risk Has been fully met Staff overload Leverage Enterprise Manager Automated proactive infrastructure monitoring, Mgmt, and diagnostics In progress, so far so good Leverage PeopleSoft Mgmt Pack Comprehensive, application performance & config Mgmt Updating/Patching system Oracle Support Removes complexity and transfer responsibility to Oracle from project team Has been met, some issues with front line team

MAA: Highlighted Examples Rolling patching: no overtime and no outages for end users Failover: DR test was planned and implemented in a few hours, SunGard test took months of planning time Replaced parts in production chassis during workweek Exachecks: proactive monitoring, great Oracle Support Use of ZFS Exalogic Shares via Vservers to manage images of PeopleSoft HOME & Oracle HOME: makes everything easy

Maximum Availability Component Then Now Comment RAC No Yes Unable to implement on blades ExaCheck MAA Report Regularly Run and track faults/warnings lets us be proactive Active Dataguard With PeopleTools 8.53.09 have secondary to keep online sessions active Templates Night and day difference in server creation turn around times hours not days or months with full procurement Patching Ability to patch during workweek and transfer responsibility (hardware/OS)

Performance More than just run time improvements…. Trouble shooting is much faster and smoother End User Experience: “the pages jump at you” Templates: spun up PeopleSoft 9.2 Environments in ½ a day, previous lifecycle would have been months or weeks Increased from 1,000 end users in June to 3,00o today, with plans to grow to 30,000 in early 2015

Performance Improvements Highlighted Tasks Benchmark time Exadata time Time Gained % change Create Pay Sheet 14 4 10 250% Load Time Data 45 25 20 80% Prelim Calc # 1 48 28 140% Pre Confirm Audit Report 27 3 24 800% Pay Confirmation 62 21 41 195% Absence Process (Entitlement-Calc) 22 12 120% Create Pay Checks  (XML Testing) 5 100% Create Pay Advice  (XML Testing) 23 92% Distribute cost of Time & Labor 38 30 8 27% CCSF Payroll Register report 73 18 54 304% Retirement services files 39 15 62% Talx E-Pay 109 36 72 196% Talx Unemployment file 11 9 75%

Payroll Processing Timeline* Time Savings 7 hours 5.5 hours Payroll Processing Performance Improvements    1.5 hours DataGuard Backup Payroll Processing Timeline*   Tues Wed Thurs Before Hours 6 10 8 end time 7:00 PM 9:00 PM or later 5:00 or later After 5 6:00 PM 6:00 PM or earlier 3:00 PM *excludes application and database processes that run after payroll. Payroll Processing Improvements: how we used the 7 hours Return to more normal work life balance More time for quality assurance and data entry tasks Will return slack time to end user departments: goal is 1:00 PM on Thursday

Looking Back…. Recommendations… Oracle to do… Plan for growth from day one Training should be “just in time” hard to relate to scale of change Don't deviate from best practice; listen don’t do it the old way Develop documentation and procedures that work for your organization there is no one size fits all Oracle to do… Improve documentation for first timers 800 page user manuals are too dense: need quick reference guides Improve Platinum Gateway experience Issues with 8.53.10 template during initial move and build  

Final Thoughts Are we confident we made the right decision? YES Why? Total Cost of Ownership Scalability: ability to scale up and scale down is a game changer One vendor is really, really nice What’s next? Purchased two new Exadata 1/8 racks and new Exalogic 1/8 rack for Oracle Identity Manager and other applications

Maximum Availability Architecture at Cognizant PeopleSoft Systems Aaron De Los Reyes Sr. Director, Oracle Technology Cognizant, Inc Oct 2, 2014

PeopleSoft ESA, Fin, HR on Exalogic MAA Cognizant’s PeopleSoft ESA, Fin, HR on Exalogic MAA Global Large Critical And, got problematic Peak time slowness More and more servers Lack of ability to scale Longer maintenance window

Oracle Exalogic Solution? Exalogic upgrade on web layer Lintel / Weblogic Benefits 50% improved downstream integration processing Server consolidation – 5:1 Maintenance window reduced to few hours Support effort down 50% Better CPU utilization – 3:1

Didn’t quite stop there… Oracle Exadata Performance woes 10-15 slow incidents/month Batch failures / over runs Slow online pages due to DB2 table locking Long running SQLs Inability to scale for 2015+ volume DB2/AIX 3 Options, 1 Pick DB Layer upgrade to Exadata

Implementation, Outcomes & Benefits Strict timelines Conversion of ~6,000 SQLs Constantly changing production system Only <3 days available for cut over Full DR 60 hours cut over DR upgraded same time as production No major post production issues Financials close batch jobs complete 56% faster Payment batch jobs run 93% faster Customer billing batch jobs run 55% faster RTO for DR down rom hours to near-zero Transactions with <1s response 75% from 30% Batch window reduced to ~4 hours from 10+ hours Huge floor, and power saving 24x7 availability

Cognizant’s Application Landscape PeopleSoft Suite Cognizant’s PeopleSoft Implementation for Corporate functions include - PeopleSoft Financials/Enterprise Services Automation - GL, AP, AR, AM, CM, PO , T&E, CA, BI, RM, PC PeopleSoft HCM – Core HR, Absence Management, Time and Labor, Payroll, ePerformance, Self-Service, Workforce Administration, Recruitment, eCompensation, Profile Management PeopleSoft ELM for Cognizant Academy User base – 180K , World wide Database size : ESA – 2 .5 TB, HCM – 1 TB, ELM – 800 GB Peak user load: ESA – 15K bi-weekly, HCM – 40K semi-annually, ELM – 10K (irregular peaks)

Technology Components Architecture Technology Components Deployment for 3 Applications (Financials, HCM, ELM) Primary: App Tier: ½ Rack Exalogic DB Tier: ¼ Rack Exadata (High Performance) Stand by: App Tier: 1/8th Exalogic DB Tier: ½ Rack Exadata (High capacity) Applications: PS HCM 9.1, PS Financials 9.1, PS ELM 9.1 Environment Management Oracle EM for PeopleSoft Applications Monitoring for Health & Perf: Oracle EM for PeopleSoft and Oracle Real User Experience (RUEI)

Maximum Availability Architecture: PeopleSoft Deployment at Cognizant SLA: 99.9% uptime Primary Site: MAA   Tier HA option Weblogic Cluster/Distributed members PIA Distributed across nodes Tuxedo (App) Tuxedo (IB) Master/Slave with distributed domains Tuxedo (Batch) Database Stand-by Synchronization   Tier HA option App Tier Exalogic to Exalogic using ZFS DB Tier Data Guard RPO Near Zero RTO Less than 2 hours

Maximum Availability at Cognizant Tier Server High Availability Type PeopleSoft Web Tier Exalogic Weblogic cluster Load Balanced across multiple nodes Coherence for session replications Active/Active PeopleSoft App Tier PeopleSoft DB Exadata Oracle RAC Stand by Exalogic ZFS File System replication Stand by Exadata Oracle Data Guard Load Balancer F5 Cluster Maintenance SLA Type SLA adherence High Availability EXA Maintenance EXA Patches Rolling upgrade, fully online with no service outage Active mode Database Maintenance Backup, Stats, Patches DB Backup is performed at local DG copy and Production is kept online always PS Weblogic maintenance Recycing of PIA domains Online maintenance only PS App Server maintenance Recycling of App domains Application upgrades Tools upgrades, Bundles, App upgrades **PeopleSoft Lite for critical functions only **Limited service

Summary Exalogic is engineered systems platform ideal for PeopleSoft workloads Software optimizations and shared storage help scale & serve multiple envs Virtual servers house various components of PeopleSoft – PIA, Tuxedo, integration broker, process scheduler… Multiple VMs ensure High Availability & Workload Isolation PeopleSoft MAA best practices ensure lower Recovery times (RTO) PeopleSoft Exalogic Virtual Templates available for PeopleTools release PeopleSoft Excelerator for rapid provisioning using automation Oracle Confidential – Restricted

Oracle Confidential – Restricted