1. DELL CONFIDENTIAL MAXIMIZING PERFORMANCE ON ORACLE 11G SE RAC Wendy Chen, Systems Engineer Naveen Iyengar, Systems Engineer Global Solutions Engineering, Dell Inc.

2. SE VS. EE  Oracle 11g database is available in multiple editions  Enterprise Edition (EE) –Single or clustered servers –No limit on the maximum number of CPU (up to the maximum number of nodes supported in a RAC cluster) –Contains all Oracle database components, and can be further enhanced with the purchase of options and packs  Standard Edition (SE) –Single or clustered servers –Up to a maximum of 4 CPU sockets in the single or clustered servers –Includes selected, but not all the features come with EE –Built from the same code base as EE –Ideally suited to the needs to SMB with enterprise level performance

3. FEATURE AVAILABILITY  A number of features that come with Oracle 11g EE are not available in SE. For example, –Data guard –Rolling upgrades –Online index and table organization –Parallel backup and recovery –Tablespaces point-in-time recovery –Flashback table / transaction / database –Parallel query / statistics gathering / index builds / data pump export and import –Transportable tablespaces –Infiniband support  Before deploying SE, make sure that you do not require any of the non-supported features.

4. PAY-AS-YOU-GROW SCALABILITY  “Pay-as-you-grow” methodology to scale up to four single socket machines

5. TESTING PERFORMANCE CAPABILITIES OF 11G SE RAC  Quest Benchmark Factory TPC-C –A single node Oracle 11.1.0.7 SE database: 100 to 10,000 users –Two-node Oracle 11.1.0.7 SE RAC database: 100 to 10,000 users –Three-node Oracle 11.1.0.7 SE RAC database: 100 to 10,000 users –Four-node Oracle 11.1.0.7 SE RAC database: 100 to 10,000 users ServerUp to four Dell PowerEdge R610 servers with:  A single Intel® Xeon® X5560 quad-core 2.80 GHz CPU with 8M cache, 6.40 GT/s QPI and TURBO and HT mode enabled  24 GB of RAM or 48 GB of RAM, with 4GB or 8GB DIMMs respectively  Four 1Gb Broadcom NetXtreme II NIC ports for iSCSI traffic External StorageTwo Dell TM EqualLogic TM PS4000XV iSCSI storage arrays, each with 16 15K RPM 146GB SAS hard drives Volume ConfigurationThree 220 GB volumes for database files; One 150 GB volume for Flash Recovery area; One 2 GB volume for OCR, CSS, and SPFILE OS and Device DriverOracle Enterprise Linux 5 Update 3  Open iSCSI initiator iscsi-initiator-utils-6.2.0.868-0.18.el5  Device Mapper multipath driver device-mapper-multipath-0.4.7- 23.el5 Storage NetworkTwo stacked Dell PowerConnect 6248 gigabit Ethernet switches for iSCSI SAN Test Software  Quest Benchmark Factory 5.7.1  Oracle 64 bit 11.1.0.7 SE RAC Database ConfigurationUp to four Oracle 11.1.0.7 SE RAC with:  13 GB memory_target on each instance

6. PERFORMANCE MONITORING  CPU utilization: SAR (System Activity Report) nohup sar 1 30000 > ~/sar-4nodes-run195-node1.txt & CPU %user %nice %system %iowait %steal %idle 02:31:43 PM all 0.38 0.00 0.25 4.25 0.00 95.12 02:31:44 PM all 0.25 0.00 0.25 4.12 0.00 95.38 02:31:45 PM all 0.12 0.00 0.50 0.25 0.00 99.13 02:31:46 PM all 0.62 0.00 0.62 1.12 0.00 97.62 02:31:47 PM all 0.87 0.00 1.12 1.99 0.00 96.01 02:31:48 PM all 0.13 0.00 0.38 5.63 0.00 93.87 02:31:49 PM all 0.62 0.00 0.37 1.87 0.00 97.13... 02:33:29 PM all 0.38 0.00 0.12 0.25 0.00 99.25 02:33:30 PM all 0.00 0.00 0.25 0.00 0.00 99.75 02:33:31 PM all 0.25 0.00 0.62 0.25 0.00 98.88 02:33:32 PM all 0.37 0.00 0.25 0.00 0.00 99.38 02:33:33 PM all 0.50 0.00 0.50 1.13 0.00 97.87 02:33:34 PM all 0.25 0.00 0.25 0.25 0.00 99.25

7. PERFORMANCE MONITORING  Physical memory utilization: OS Watcher nohup./startOSW.sh 5 96 & zzz ***Sun Jun 21 23:04:59 CDT 2009 MemTotal: 49434036 kB MemFree: 34270564 kB Buffers: 354892 kB Cached: 12637136 kB SwapCached: 7732 kB Active: 10278772 kB Inactive: 4483804 kB HighTotal: 0 kB HighFree: 0 kB LowTotal: 49434036 kB LowFree: 34270564 kB SwapTotal: 8388600 kB SwapFree: 8239712 kB

8. PERFORMANCE MONITORING  Storage utilization: EqualLogic SAN HeadQuarters

9. TUNING ORACLE MEMORY SIZE  The optimal size of SGA is the point where the marginal benefit of physical I/O reduction begins to decline  The optimal size of PGA is when the estimated PGA over-allocation size is 0 SGA Target Size (M)SGA Size FactorEst DB Time (s)Est Physical Reads 3,1680.381,327,87037,779,644 4,2240.50895,77620,459,187 5,2800.63728,22013,742,069 6,3360.75641,71010,272,675 7,3920.88598,9108,558,231 8,4481.00569,1427,364,453 9,5041.13549,5096,576,457 10,5601.25534,5975,979,936 11,6161.38524,0115,555,743 12,6721.50519,6855,381,942 13,7281.63516,0465,236,863 14,7841.75516,1125,236,863 15,8401.88516,1175,236,863 16,8962.00516,1195,236,863 PGA Target Est (MB) Size Factr W/A MB Processed Estd Extra W/A MB Read/ Written to Disk Estd PGA Cache Hit % Estd PGA Overalloc Count Estd Time 9120.25564.9815.2597.001131,740 1,8240.50564.9815.2597.00771,740 2,7360.75564.9815.2597.00501,740 3,6481.00564.980.00100.00171,694 4,3771.20564.980.00100.0001,694 5,1071.40564.980.00100.0001,694 5,8361.60564.980.00100.0001,694 6,5661.80564.980.00100.0001,694 7,2962.00564.980.00100.0001,694

10. TESTING SE RAC SCALABILITY WITH 24 GB RAM PER NODE – CPU UTILIZATION

11. TESTING SE RAC SCALABILITY WITH 24 GB RAM PER NODE – MEMORY AND STORAGE UTILIZATION

12. TESTING SE RAC SCALABILITY WITH 24 GB RAM PER NODE – OVERALL PERFORMANCE ANALYSIS  As the systems near their maximum memory performance, they progressively utilize more swap space and the database starts to get unstable.  When comparing the CPU graph with the memory graph, at the maximum user load there is still ample CPU capability even once the test fails.  Given that the Oracle memory area (memory_target) is set to 13 GB to minimize physical I/O, even at the maximum 4-node cluster size the storage members were delivering IOPS well below their potential, suggesting that the storage disks were not the bottleneck for the overall performance.

13. TESTING SE RAC SCALABILITY WITH 48 GB RAM PER NODE – CPU UTILIZATION

14. TESTING SE RAC SCALABILITY WITH 48 GB RAM PER NODE – MEMORY AND STORAGE UTILIZATION

15. TESTING SE RAC SCALABILITY WITH 48 GB RAM PER NODE – OVERALL PERFORMANCE ANALYSIS  As the physical memory increased from 24 GB to 48 GB, the system performance was no longer limited by memory, thus the database was able to handle more transactions at higher user loads. Consequently, the PS4000XV members processed more I/O requests and showed much improved IOPS results comparing to the 24 GB configuration.  The corresponding disk latency performance was still within the acceptable range of 20 ms in all test durations of the 48 GB memory configurations, except towards the end of run on the 4-node RAC, when disk latency started to have a delayed response of above 20 ms. This indicates that the PS4000XV showed signs of stress after approximately 7000 user loads on the 4-node RAC.

16. PERFORMANCE CHARACTERISTICS SUMMARY  The 48 GB Oracle RAC cluster scales better than its 24 GB counterpart. For the 24 GB configuration, the TPS measure scales in the range of 40% - 60% with the addition of each node. With 48 GB per node, the results demonstrate a near linear scalability, in the range of 75%-95% and that the benefit of additional memory became more apparent with each additional node.  With 48 GB of physical memory per node, both the CPU and memory resources become limited almost simultaneously. In an ideal situation, all the resources would be constrained simultaneously. Extra CPU capability without complementary memory may not lead to ideal scaling characteristics, and likewise extra memory without CPU capability may be unnecessary.  By adequately balancing memory, CPU and storage you can help make the most out of scaling out an Oracle RAC cluster. A bottleneck in any one component such as storage, CPU, or memory may result in non-optimal scaling. Likewise, scaling up of one component when another component is the bottleneck for the system may not add performance and may be unnecessary.

17. RESOURCES  Dell whitepaper: Maximizing Performance on a Cost Effective Oracle 11g SE RAC Database on Dell EqualLogic PS4000XV iSCSI Storage http://www.dell.com/downloads/global/solutions/Oracle _SE_RAC.pdf?c=us&cs=555&l=en&s=biz http://www.dell.com/downloads/global/solutions/Oracle _SE_RAC.pdf?c=us&cs=555&l=en&s=biz  Dell Oracle Solutions Engineering http://www.dell.com/oracle http://www.dell.com/oracle

18. Q & A