Presentation is loading. Please wait.

Presentation is loading. Please wait.

Oracle Database 11g for Data Warehousing Presenter’s Name Presenter’s Title.

Similar presentations


Presentation on theme: "Oracle Database 11g for Data Warehousing Presenter’s Name Presenter’s Title."— Presentation transcript:

1

2 Oracle Database 11g for Data Warehousing Presenter’s Name Presenter’s Title

3 Agenda Technology Monitoring Information Life-cycle Management (ILM)‏ Oracle Optimized Warehouse Initiative Market

4 Technology

5 Parallel Execution select c.cust_last_name, sum(s.amount_sold)‏ from customers c, sales s where c.cust_id = s.cust_id group by c.cust_last_name ; Data on DiskParallel Servers scan scan scan aggregate Scanners Coordinator join join join aggregate aggregate JoinersAggregators

6 Partitioning – Benefits Large Table Difficult to Manage Partition Divide and Conquer Easier to Manage Improve Performance Composite Partition Better Performance More flexibility to match business needs JAN FEB JAN FEB USA EUROPE ORDERS Transparent to applications

7 Partitioning in Oracle Database 11g Interval Partitioning JAN FEB MAR APR ORDERS JAN FEB ORDERS MAR JAN FEB INVENTORY Partitions are created automatically as data arrives

8 Partitioning in Oracle Database 11g Complete Composite Partitioning Range – range List – list List – hash List – range JAN FEB > ORDERS RANGE-RANGE Order Date by Order Value USA EUROPE > ORDERS LIST-RANGE Region by Order Value USA EUROPE Gold Silver ORDERS LIST-LIST Region by Customer Type    

9 Partitioning in Oracle Database 11g Reference Partitioning ORDERS Line Items Pick Lists Stock Holds Back Orders ORDERS Line Items Pick Lists Stock Holds Back Orders ORDERS Line Items Pick Lists Stock Holds Back Orders ORDERS Line Items Pick Lists Stock Holds Back Orders ORDERS Line Items Pick Lists Stock Holds Back Orders Partition ORDERS by Date JAN FEB MAR APR Inherit partitioning strategy

10 Partitioning in Oracle Database 11g Virtual Column-Based Partitioning ORDERS ORDER_ID ORDER_DATE CUSTOMER_ID US JAN EU FEB EU JAN US JAN US FEB JAN FEB USA EUROPE ORDERS REGION requires no storage Partition by ORDER_DATE, REGION REGION AS (SUBSTR(ORDER_ID,6,2)) US EU US

11 Compression Tables and indexes can be compressed Can be specified on a per-partition basis Typical compression ratio 3:1 Requires more CPU to load data Decompression hardly costs resources Compress for all DML operations Less data on disk Requires less time to read Completely transparent Up To 3X Compression

12 SQL Query Result Cache Store query results in cache Repetitive executions can use cached result Data Warehouse queries Long-running, IO-intensive Expensive computations Return few rows Excellent opportunity for SQL Query Result Cache | Id | Operation | Name | | 0 | SELECT STATEMENT | | | 1 | RESULT CACHE | fz6cm4jbpcwh48wcyk60m7qypu | | 2 | SORT GROUP BY ROLLUP | | |* 3 | HASH JOIN | | etc.

13 SQL Query Result Cache Opportunity Retail customer data (~50 GB)‏ Concurrent users submitting queries randomly Executive dashboard with 12 heavy analytical queries Cache results only at in-line view level 12 queries run in random, different order – 4 queries cached Measure average, total response time for all users 447 s 267 s 186 s No cache 334 s 201 s 141 s Cache 25% 24% Improvement # Users

14 Other Performance Features Transparent to Your Application Materialized Views Transparent rewrites of expensive queries Including rewrites on remote objects Incremental automatic refresh Bitmap Indexes Optimal storage Ideal for star or star look-a-like schemas SQL Access Advisor – based on workload Materialized view advice Index advice Partition advice

15 SQL analytics Bring Algorithms to the Data Not Data to the Algorithms Analytic computations done in the database SQL Analytics OLAP Data Mining Statistics Scalability Security Backup & Recovery Simplicity OLAP Data Mining Statistics

16 Native Support for Pivot and Unpivot SALESREP Q1 Q2 Q3 Q SALESREP QU REVENUE Q Q Q Q Q Q Q Q Q Q Q Q4 310

17 Native Support for Pivot and Unpivot select * from quarterly_sales unpivot include nulls (revenue for quarter in (q1,q2,q3,q4))‏ order by salesrep, quarter ; QUARTERLY_SALES SALESREP Q1 Q2 Q3 Q SALESREP QU REVENUE Q Q Q Q Q Q Q Q Q Q Q Q4 310

18 Native Support for Pivot and Unpivot SALESREP 'Q1' 'Q2' 'Q3' 'Q4' SALES_BY_QUARTER SALESREP QU REVENUE Q Q Q Q Q Q Q Q Q Q Q Q1 260 select * from sales_by_quarter pivot (sum(revenue)‏ for quarter in ('Q1','Q2','Q3','Q4'))‏ order by salesrep ;

19 Transform Data Where Data Resides In-database ETL technology Extract Change Data Capture External Tables SQL*Loader Data Pump Transportable Tablespaces Multi-Table Insert MERGE Distributed Queries Table Functions LoadTransformInsert Partition Exchange Loading DML error logging

20 Capture changes from [redo | archive] logs No changes to source applications Minimal performance impact on source applications Store changes in change tables Provide (bulk) SQL interface to change data OLTP DB Log files Change Data Log Miner and Streams DW Tables SQL, PL/SQL, Java Transform Read-consistent subscription Capture PMOPs Time-based subscription windows Asynchronous Change Data Capture Oracle Database 11g

21 RAC – Scale Incrementally Months 100% 200% 300% WorklodWorklod

22 Automatic Storage Management Storage pool for database files Load-balanced across disks Capacity on demand Add/remove storage on-line Automatic IO load balancing Fault tolerant, high performance Automatically mirrors and stripes Low cost No IO tuning required No volume manager or file system needed

23 Mixed Workloads Concurrent small data loads and queries Looks like... OLTP Oracle's read consistency Readers never block writers Writers never block readers Queries are always consistent and auditable No deadlocks Introduced in Oracle V4 (1982) – major improvements in V6 (1988)‏ report update Rollback Segment Before Image accurate report Budget table

24 Database Resource Manager Protect the system pro-actively Maximum number of concurrent operations Priority-dependent maximum Degree Of Parallelism (DOP)‏ High Priority Sales Analysis 20 users (DOP 10)‏ Medium Priority Ad Hoc Reports 200 users (DOP 4)‏ Low Priority ETL Jobs 200 users (DOP 4)‏

25 Oracle Database Security Marketing Finance Sales  Authenticate  Protect data in transit  Authorize  Access Control  Protect stored data  Audit  Identity Management 

26 Feature Usage for Large-Scale Data Warehouses Source: TB Club Report: A survey of 30 multi-TB Oracle DW’s – data July 2006 Partitioning, parallelism, and compression are the foundation for large-scale data warehousing

27 Monitoring

28 I/O Monitoring Database Control

29

30 Parallel Execution Monitoring Database Control

31 Near Real-Time SQL Monitoring Coming in Grid Control

32 Parallel SQL Monitoring Coming in Grid Control

33 Information Life-cycle Management (ILM)‏

34 Information Lifecycle Management “The policies, processes, practices, and tools used to align the business value of information with the most appropriate and cost effective IT infrastructure from the time information is conceived through its final disposition.” Storage Networking Industry Association (SNIA) Data Management Forum Historical Data Active Data Less Active Data

35 Information Lifecycle Management Orders Q1 Orders Q2 Orders Q3 Orders Q4 Orders Older Orders Active High Performance Storage Tier Less Active Low Cost Storage Tier Historical Online Archive Storage Tier

36 Traditional Storage Approach All data resides on a single storage tier High Performance Storage Tier = $72 per Gb All data on active = $972,000! Active

37 Partitioning is the Foundation for ILM Partition data onto appropriate storage tier Active Less Active Historical High Performance Storage Tier = $72 per Gb Low cost Storage Tier = $14 per Gb Read only Storage Tier = $7 per Gb

38 Partitioning is the Foundation for ILM Move data onto appropriate storage tier 5% Active 35% Less Active 60% Historical High Performance Storage Tier = $72 per Gb Low cost Storage Tier = $14 per Gb Read only Storage Tier = $7 per Gb

39 Partitioning is the Foundation for ILM Reduce storage costs accordingly 5% Active 35% Less Active 60% Historical High Performance Storage Tier = $72 per Gb Low cost Storage Tier = $14 per Gb Read only Storage Tier = $7 per Gb $49,800$67,700$58,000

40 Introduce Compression Reduce storage costs across all tiers 5% Active 35% Less Active 60% Historical $16,600$22,600$19,400 Lets use compression factor of 3 $49,800$67,700$58,000

41 Cost Savings by Storage Tier

42 Oracle Optimized Warehouse Initiative

43 Goals for Oracle data warehouse solutions: Provide superior system performance Provide a superior customer experience

44 Full Range of DW Solution Options Database Options Management Packs Reference Configuration Documented best-practice configurations for data warehousing Benefits: High performance Simple to scale; modular building blocks Industry-leading database and hardware Available today with HP, IBM, Sun, EMC/Dell Flexibility for the most demanding data warehouse Benefits: High performance Unlimited scalability Completely customizable Industry-leading database and hardware Custom Database Options Management Packs Flexibility Pre-configured, Pre-installed, Validated Partitioning RAC Optimized Warehouse Scalable systems pre- installed and pre- configured: ready to run out-of-the-box Benefits: High performance Simple to buy Fast to implement Easy to maintain Competitively priced

45 Market

46 Data Warehouse Market Source: IDC, Worldwide Data Warehousing Tools 2005 Vendor Shares Oracle is the Data Warehousing DBMS Market Leader

47 Leading Scalability Wintercorp VLDB Survey Source: Yahoo! Oracle AT&TDaytona KT-IT GroupDB AT&TDaytona LGR - Cingular Oracle25.20 Amazon.com Oracle24.77 AnonymousDB UPSSMicrosoft19.47 Amazon.com Oracle Nielsen MediaSybase IQ Survey France Telecom Oracle AT&TProprietary SBC Teradata AnonymousDB Amazon.com Oracle13.00 Kmart Teradata Claria Oracle HIRA Sybase IQ FedEx Teradata 9.98 Vodafone Gmbh Teradata Survey SearsTeradata 4.63 HCIA Informix 4.50 Wal-MartTeradata 4.42 Tele DanmarkDB CiticorpDB MCIInformix 1.88 NDC Health Oracle 1.85 Sprint Teradata 1.30 Ford Oracle 1.20 Acxiom Oracle Survey

48 Oracle DW 10+TB Customers (3/2006) Various Platforms and Architectures Acxiom16 TBHP Allstate15 TBSun (RAC)‏ Amazon61 TBHP (RAC)‏ Cellcom14 TBHP CenturyTel 10 TBHP Chase30 TBIBM (RAC)‏ Choicepoint14 TBSun Claria 38 TBSun Experian14 TBSun KTF14 TBHP Cingular25 TBHP Mastercard20 TBIBM (RAC)‏ NASDAQ35 TBSun NexTel 28 TBHP NYSE Group15 TBHP (RAC)‏ Reliance Ltd13 TBSun Starwood 12 TBHP TIM (Italy)12 TBHP (RAC)‏ Turkcell14 TBSun (RAC)‏ UBS AG15 TBSun UPS10 TBHP Yahoo! 130 TBFujitsu Hundreds of Terabyte+ DW Customers!

49 Summary Technology Monitoring Information Life-cycle Management (ILM)‏ Oracle Optimized Warehouse Initiative Market

50 For More Information or oracle.com BI & Data Warehousing

51


Download ppt "Oracle Database 11g for Data Warehousing Presenter’s Name Presenter’s Title."

Similar presentations


Ads by Google