Exadata Distinctives 988 Bobby Durrett US Foods

What is Exadata? Complete Oracle database platform Disk storage system Unique to Exadata – intelligent disk storage CPU, memory, network hardware Operating system Database software Fully installed and configured

V2 Cell Storage servers Intel-based Linux servers with disk drives 14 cell servers per rack 7 on top, 7 on bottom 12 disks per server 3 Infiniband network switches 2 in middle 1 on bottom 8 Intel Linux database servers 4 on bottom, 4 on top

What is standard about Exadata? Off the shelf hardware Intel x86 processors Standard Sun disk drives, memory Infiniband network adapters and switches PCIe Flash Cards Linux operating system Oracle Database 11.2 Enterprise Edition

What is unique about Exadata? Storage Servers have Oracle database specific features Smart Scan – returns only needed data Storage Indexes – only accesses needed disk blocks Hybrid Columnar Compression – fits data into fewer disk blocks Exadata Smart Flash Cache IO Resource Manager

Rows, Columns, and Blocks A SQL table is a collection of rows – one row per sale A row is a list of columns – date, product, customer, amount,… A table is broken up into equal sized blocks each with a number of rows = block size/row size = 8000 bytes/80 bytes = 100 rows per block Results of queries are a subset of the columns and a subset of the rows of a table

Query select sale_date, product_name, customer_name, amount from sales, products, customers where sales.product_number=products.product_number and sales.customer_number=customers.customer_number and sale_date between to_date('01/01/2012','MM/DD/YYYY') and to_date('01/31/2012','MM/DD/YYYY') and product_type = 'Cheese' and customer_state = 'FL';

Plan | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | | 0 | SELECT STATEMENT | | | | 10 (100)| | |* 1 | HASH JOIN | | 4 | 232 | 10 (10)| 00:00:01 | |* 2 | HASH JOIN | | 4 | 136 | 7 (15)| 00:00:01 | |* 3 | TABLE ACCESS STORAGE FULL| PRODUCTS | 2 | 34 | 3 (0)| 00:00:01 | |* 4 | TABLE ACCESS STORAGE FULL| SALES | 4 | 68 | 3 (0)| 00:00:01 | |* 5 | TABLE ACCESS STORAGE FULL | CUSTOMERS | 2 | 48 | 3 (0)| 00:00:01 |

Predicates Predicate Information (identified by operation id): access("SALES"."CUSTOMER_NUMBER"="CUSTOMERS"."CUSTOMER_NUMBER") 2 - access("SALES"."PRODUCT_NUMBER"="PRODUCTS"."PRODUCT_NUMBER") 3 - storage("PRODUCT_TYPE"='Cheese') filter("PRODUCT_TYPE"='Cheese') 4 - storage(("SALE_DATE">=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "SALE_DATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss'))) filter(("SALE_DATE">=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss') AND "SALE_DATE"<=TO_DATE(' :00:00', 'syyyy-mm-dd hh24:mi:ss'))) 5 - storage("CUSTOMER_STATE"='FL') filter("CUSTOMER_STATE"='FL')

Columns Column Projection Information (identified by operation id): (#keys=1) "PRODUCT_NAME"[VARCHAR2,12], "SALE_DATE"[DATE,7], "AMOUNT"[NUMBER,22], "CUSTOMER_NAME"[VARCHAR2,17] 2 - (#keys=1) "PRODUCT_NAME"[VARCHAR2,12], "SALE_DATE"[DATE,7], "AMOUNT"[NUMBER,22], "SALES"."CUSTOMER_NUMBER"[NUMBER,22] 3 - "PRODUCTS"."PRODUCT_NUMBER"[NUMBER,22], "PRODUCT_NAME"[VARCHAR2,12] 4 - "SALE_DATE"[DATE,7], "SALES"."PRODUCT_NUMBER"[NUMBER,22], "SALES"."CUSTOMER_NUMBER"[NUMBER,22], "AMOUNT"[NUMBER,22] 5 - "CUSTOMERS"."CUSTOMER_NUMBER"[NUMBER,22], "CUSTOMER_NAME"[VARCHAR2,17]

SGA and PGA

Direct Path

Comparison of three db server types Compare three types of database servers to demonstrate features unique to Exadata DB server with directly attached disk drives DB server attached to a Storage Area Network (SAN) Exadata With Smart Scan Without Smart Scan

Observations about the three server types DB server with disks looks like Exadata Cell server Indexes Compression memory cache – bypassed at times returns small subset of full data DB server with SAN looks like full Exadata system Storage server High speed network Database servers

More observations about the three server types DB server with SAN differs from full Exadata system Full database blocks copied over SAN network Results only copied over Exadata network Database server caches blocks from SAN Database server does not cache results from cell server Exadata Smart Scan can be turned off Without Smart Scan Exadata works just like DB server with SAN

How to Use Exadata Smart Scan Queries that would run a FULL table scan on a normal database server will run a Smart Scan on Exadata. Normal query tuning involves deciding between FULL and INDEX scans. Use same methods that encourage FULL scans on normal databases to cause Smart Scans on Exadata.

Moving an existing application to Exadata Try these to encourage Smart Scans: ALTER INDEX … INVISIBLE; OPTIMIZER_INDEX_COST_ADJ = (> 100) FULL hint Use this parameter to temporarily disable Smart Scans: CELL_OFFLOAD_PROCESSING=FALSE All of these options can be tested without changing the application

Why use Exadata? New set of performance enhancing features All other Oracle features still available Easy to use new features Reuse existing SQL tuning skills – FULL or INDEX scans

Other Considerations Hierarchy of storage performance Normal RAM – 1000x faster than disk – 10ns Flash Cache – 10x faster than disk – 1 ms Disk – 10 ms Storage Indexes are Automatic You don’t have to do anything to use them Not easy to figure out when they will be used

Other Considerations Decrease SGA, increase PGA Reduce sga_max_size, sga_target Increase pga_aggregate_target OR Set workarea_size_policy = MANUAL Set hash_area_size = 1 gig (large)

Other Considerations Databases tuned for Exadata may not perform well if cloned to normal database server Tables compressed with HCC on Exadata will need to be uncompressed unless you have a tool such as ZFS storage Queries that normally run Smart Scans will run FULL scans Poor performance of TEMP space reallocation If you have an environment with queries that use a large amount of temp space and can run on any RAC node you may get poor performance as space is moved from one node’s sort segment to that of another node Newer X3-8 may address this – 2 node RAC

Other Considerations HCC and tables > 255 columns We hit several bugs related to tables with > 255 columns Newest release resolves most of these Newness issues Getting more stable with latest releases Blogs, books, documentation, training improving More people becoming skilled in Exadata

Training Recommendations RAC class/book/experience Exadata class from Oracle

Questions?