Presentation is loading. Please wait.

Presentation is loading. Please wait.

Perfect solution outstanding experience. ORACLE MATERIALZIED VIEWS A PERFORMANCE TUNING GEM.

Similar presentations


Presentation on theme: "Perfect solution outstanding experience. ORACLE MATERIALZIED VIEWS A PERFORMANCE TUNING GEM."— Presentation transcript:

1 perfect solution outstanding experience

2 ORACLE MATERIALZIED VIEWS A PERFORMANCE TUNING GEM

3 perfect solution outstanding experience Agenda A Game Changer What is a Materialized View Purposes of Materialized Views Enhancing SQL Performance Query Rewrite Real World Example Advanced Query Rewrite Concepts Q/A

4 perfect solution outstanding experience A Game Changer Provide SQL Tuning Services for OLTP and DSS (Data Warehouse) Most Tuning Methods Consist of SQL Redevelopment Converting Correlated Sub Queries to Simply Sub Queries Partitioning/Compression Index Development SQL Baselines Data warehouse tuning cases started to pop up where none of the conventional methods would provide the solution

5 perfect solution outstanding experience A Game Changer Data warehouse performance goals The Report that Started it All Local Fund Budget Report Heavily Utilized Report Running almost 4 minutes Conventional methods would not work Type of ReportGeneral Performance Goal Heavily Utilized5 seconds or less Moderately Utilized30 seconds or less Occasionally Utilized2 Minutes or less Maximum Time Allowed3 Minutes

6 perfect solution outstanding experience A Game Changer Local Fund Budget Report was tuned to return results in about 4 seconds, down from almost 4 minutes How did it happen? The SQL in the report was not touched New indexes were not added The existing table structure behind the report remained the same SOLUTION: Query Rewrite using Materialized Views SOLUTION: Query Rewrite using Materialized Views

7 perfect solution outstanding experience What is a Materialized View First, lets define a View A View stores a SELECT statement and executes the stored SELECT statement when referenced via another SQL statement. View: HR_DEPT_VW SELECT DEPTID,MANAGER_ID FROM DEPT_TBL WHERE CATEGORY = HR View: HR_DEPT_VW SELECT DEPTID,MANAGER_ID FROM DEPT_TBL WHERE CATEGORY = HR SELECT COUNT(1) FROM HR_DEPT_VW Table: DEPT_TBL DEPTID MANAGER_ID CATEGORY Table: DEPT_TBL DEPTID MANAGER_ID CATEGORY

8 perfect solution outstanding experience What is a Materialized View Now, lets define a Materialized View Like a View, a Materialized View stores a SELECT statement, in addition the results of the SELECT are stored in a table. MView: HR_DEPT_MV SELECT DEPTID,MANAGER_ID FROM DEPT_TBL WHERE CATEGORY = HR MView: HR_DEPT_MV SELECT DEPTID,MANAGER_ID FROM DEPT_TBL WHERE CATEGORY = HR Table: DEPT_TBL DEPTID MANAGER_ID CATEGORY Table: DEPT_TBL DEPTID MANAGER_ID CATEGORY MView Table: HR_DEPT_MV DEPTID MANAGER_ID MView Table: HR_DEPT_MV DEPTID MANAGER_ID

9 perfect solution outstanding experience What is a Materialized View Now, lets define a Materialized View Like a View, a Materialized View stores a SELECT statement, in addition the results of the SELECT are stored in a table. SELECT COUNT(1) FROM HR_DEPT_MV MView: HR_DEPT_MV SELECT DEPTID,MANAGER_ID FROM DEPT_TBL WHERE CATEGORY = HR MView: HR_DEPT_MV SELECT DEPTID,MANAGER_ID FROM DEPT_TBL WHERE CATEGORY = HR Table: DEPT_TBL DEPTID MANAGER_ID CATEGORY Table: DEPT_TBL DEPTID MANAGER_ID CATEGORY MView Table: HR_DEPT_MV DEPTID MANAGER_ID MView Table: HR_DEPT_MV DEPTID MANAGER_ID

10 perfect solution outstanding experience Purposes of Materialized Views Replication Easily replicate tables from one database to another Enhancing SQL Performance Facilitate Incremental Warehouse Loading Operations (ETL) Use Materialized View Logs to track data changes in tables, which will be used to identify records needing to be updated, inserted, and deleted in the data warehouse.

11 perfect solution outstanding experience Enhancing SQL Performance How do Materialized Views help SQL Perform Better? Results of the associated query are stored in a table Data can be pre-aggregated and pre-joined, cutting down the resources and time needed to execute the query TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows 2:40.03 Min SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM MV_GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows 0:0.03 Min

12 perfect solution outstanding experience Enhancing SQL Performance There are two ways to utilize MVs to enhance SQL performance Directly referencing the MV as in the example below Using the Oracle Query Rewrite functionality TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM MV_GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows 2:40.03 Min 95 rows 0:0.03 Min

13 perfect solution outstanding experience Query Rewrite What is Query Rewrite? Query rewrite transforms a SQL statement expressed in terms of tables or views into a statement accessing one or more materialized views that are defined on the detail tables. The transformation is transparent to the end user or application, requiring no intervention and no reference to the materialized view in the SQL statement. Because query rewrite is transparent, materialized views can be added or dropped just like indexes without invalidating the SQL in the application code.

14 perfect solution outstanding experience Query Rewrite Example of Query Rewrite TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID Generate Plan

15 perfect solution outstanding experience Query Rewrite Example of Query Rewrite SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID

16 perfect solution outstanding experience Query Rewrite Materialized View DDL CREATE MATERIALIZED VIEW MV_GL_ENTRIES BUILD IMMEDIATE REFRESH COMPLETE ENABLE QUERY REWRITE AS SELECT FISCAL_PERIOD_SID FISCAL_PERIOD_SID,SUM(ENTRY_AMT) ENTRY_AMT FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID REFRESH COMPLETE Tells Oracle that when this MV is refreshed all data will first be removed and then a complete reload will occur. Most MVs created for Query Rewrite will be a complete refresh. REFRESH COMPLETE Tells Oracle that when this MV is refreshed all data will first be removed and then a complete reload will occur. Most MVs created for Query Rewrite will be a complete refresh. ENABLE QUERY REWRITE Tells Oracle that this MV can be considered for Query Rewrite. If this Query Rewrite is Disabled then Oracle will not consider this MV as an option for Query Rewrite. ENABLE QUERY REWRITE Tells Oracle that this MV can be considered for Query Rewrite. If this Query Rewrite is Disabled then Oracle will not consider this MV as an option for Query Rewrite. BUILD IMMEDIATE Tells Oracle to run the SELECT statement and load the MV table right away. The Build can also be deferred. BUILD IMMEDIATE Tells Oracle to run the SELECT statement and load the MV table right away. The Build can also be deferred.

17 perfect solution outstanding experience Query Rewrite The STALENESS Factor A MV can be in one of several STALENESS states FRESH – The data in the MV accurately reflects the data in the base tables STALE – Data in one or more of the base tables has changed and the MV may no longer accurately reflect the data in the base tables UNUSABLE – The MV is currently being refreshed NEEDS_COMPILE – The table structure of one or more of the base tables has changed and the MV needs to be compiled UNKNOWN – One of two situations cause an UNKNOWN status An ALTER MATERIALIZED VIEW….CONSIDER FRESH has been run The MV is based on a PREBUILT TABLE (Advanced Topic) A Materialized View will only be considered for Query Rewrite if its in one of the two STALENESS states OR STALENESS = STALE and the following database parameter is set query_rewrite_integrity=stale_tolerated OR STALENESS = STALE and the following database parameter is set query_rewrite_integrity=stale_tolerated

18 perfect solution outstanding experience Query Rewrite The STALENESS Factor TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID Generate Plan UPDATE GL_ENTRIES SET ENTRY_AMT = 3 WHERE FISCAL_PERIOD_SID = 12

19 perfect solution outstanding experience Query Rewrite The STALENESS Factor TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows TABLE: GL_ENTRIES FISCAL_PERIOD_SID ENTRY_AMT 38,020,294 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows MV: MV_GL_ENTRIES SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID 95 rows SELECT FISCAL_PERIOD_SID,SUM(ENTRY_AMT) FROM GL_ENTRIES GROUP BY FISCAL_PERIOD_SID Generate Plan UPDATE GL_ENTRIES SET ENTRY_AMT = 3 WHERE FISCAL_PERIOD_SID = 12 BEGIN DBMS_MVIEW.REFRESH (MV_GL_ENTRIES); END;

20 perfect solution outstanding experience Query Rewrite The STALENESS Factor Query Rewrite works best for Materialized Views that are based on tables that remain static throughout the day Transactional or OLTP databases typically do not work well due to the constant updating of data that will require constant refreshing of the MV to maintain a FRESH state. Query Rewrite works well with data warehouses

21 perfect solution outstanding experience A Real World Example First, a little lingo Subject Area Dimensions have a one-to-many relationship with the Fact Star Schema Basically, just a bunch of tables with data

22 perfect solution outstanding experience A Real World Example Subject Area

23 perfect solution outstanding experience A Real World Example Local Fund Budget Report Local Fund Budget Report

24 perfect solution outstanding experience A Real World Example Generate Plan Cost 207,000 3:53 Minutes Cost 207,000 3:53 Minutes

25 perfect solution outstanding experience A Real World Example Lets Create an Materialized View Create Materialized View CREATE MATERIALZIED VIEW KUALI_ADMIN.KF_MVA_BUDGET_BALANCE … SELECT BUD.FISCAL_PERIOD_SID …, SUM(BUD.UA_BUD_CY_ANN_EXP) FROM SYSADM.PS_UA_EMPL_JOB_DTL EMP, SYSADM.PS_UA_BUD_ACNT_TYP ACNT, KUALI_ADMIN.KF_D_ORG ORG, KUALI_ADMIN.KF_D_FISCAL_PERIOD FP, KUALI_ADMIN.KF_D_PROJECT PROJ, KUALI_ADMIN.KF_D_OBJECT_CODE OC, KUALI_ADMIN.KF_D_ACCOUNT ACCT, KUALI_ADMIN.KF_F_BUDGET_BALANCE BUD WHERE BUD.ACCOUNT_SID = ACCT.ACCOUNT_SID AND BUD.OBJECT_CODE_SID = OC.OBJECT_CODE_SID AND BUD.PROJECT_SID = PROJ.PROJECT_SID AND BUD.FISCAL_PERIOD_SID = FP.FISCAL_PERIOD_SID AND BUD.UA_BUD_ACCT_DEPT_ORG_SID = ORG.ORG_SID AND BUD.BUD_ACCT_TYP_SID = ACNT.BUD_ACCT_TYP_SID (+) AND BUD.EMPL_JOB_DETAIL_SID = EMP.EMPL_JOB_DTL_SID (+) GROUP BY BUD.FISCAL_PERIOD_SID … Pre-Aggregate Pre-Join

26 perfect solution outstanding experience A Real World Example Generate Plan Cost 207,000 3:53 Minutes Cost 207,000 3:53 Minutes Generate Plan Cost Seconds Cost Seconds

27 perfect solution outstanding experience A Real World Example Generate Plan Cost 207,000 3:53 Minutes Cost 207,000 3:53 Minutes Generate Plan Cost Seconds Cost Seconds Explain Plan without MV Explain Plan with MV Expensive Joins NO Joins MV used 98.5% Less Temp Space for Sorting CBO Cost 207,000 vs 483 CBO Cost 207,000 vs 483

28 perfect solution outstanding experience A Real World Example Another alternative would have been to create a new, summarized Subject Area Benefits of Materialized Views over building additional summarized Subject Areas Materialized Views are more agile and quicker to implement Existing reports need not be updated to point to a new set of tables Report writers will not have to be concerned with choosing the best Subject Area from which to create reports

29 perfect solution outstanding experience Advanced Concepts

30 perfect solution outstanding experience In the Beginning… Due to differences in tables and joins in each report a different MV was created Every time a change was made to the report the MV had to be updated Became a maintenance nightmare Something had to change or MVs were not going to be a viable option Budget MV1 Budget Report 1 Budget Report 2 Budget Report 3 Budget MV2 Budget MV3 Budget MV4

31 perfect solution outstanding experience A New Methodology… Budget Balances MV1 Budget Report 1 Budget Report 2 Budget Report 3 GL Entries MV1 GL Entries Report 1 GL Entries Report 2 GL Entries Report 3 Labor Entries MV1 Labor Entries Report 1 Labor Entries Report 2 Labor Entries Report 3

32 perfect solution outstanding experience How did we do it… Must Know Advanced Concepts Query Delta Joins Materialized View Delta Joins Nested Materialized Views

33 perfect solution outstanding experience Query Delta Joins A query delta join is a join that appears in the query but not in the materialized view. A query delta join will also occur if a field from a dimension is included in the query but is not included in the MV. In order for the join to work, the materialized view must contain the joining key.

34 perfect solution outstanding experience Example Query Delta Join MV Includes Budget Balance Fact joined to Account Dimension MV Includes FK columns to the Object Code and Account Dimensions

35 perfect solution outstanding experience Example Query Delta Join Budget Balances -OBJECT_CODE_SID -ACCOUNT_SID -SUM(UA_BUD_CY_ANN) Account -ACCOUNT_SID -ACCOUNT_BUDGET_SHELL_CD Consists of columns, tables, and joins that all exist in the MV Budget Balances -OBJECT_CODE_SID -ACCOUNT_SID -SUM(UA_BUD_CY_ANN) Account -ACCOUNT_SID -ACCOUNT_BUDGET_SHELL_CD

36 perfect solution outstanding experience Example Query Delta Join Budget Balances -OBJECT_CODE_SID -ACCOUNT_SID -SUM(UA_BUD_CY_ANN) Account -ACCOUNT_SID -ACCOUNT_BUDGET_SHELL_CD This query adds the ACCOUNT_NBR field which does not exist in our test MV Budget Balances -OBJECT_CODE_SID -ACCOUNT_SID -SUM(UA_BUD_CY_ANN) Account -ACCOUNT_SID -ACCOUNT_BUDGET_SHELL_CD -ACCOUNT_NBR The Delta Join is made possible by including the joining key from the FACT in the MV

37 perfect solution outstanding experience Example Query Delta Join Budget Balances -OBJECT_CODE_SID -ACCOUNT_SID -SUM(UA_BUD_CY_ANN) Account -ACCOUNT_SID -ACCOUNT_BUDGET_SHELL_CD The Delta Join is made possible by including the joining key from the FACT in the MV OBJECT_CD and the Object Code Dimension was added to the query Budget Balances -OBJECT_CODE_SID -ACCOUNT_SID -SUM(UA_BUD_CY_ANN) Account -ACCOUNT_SID -ACCOUNT_BUDGET_SHELL_CD Object Code -OBJECT_CODE_SID -OBJECT_CD

38 perfect solution outstanding experience Materialized View Delta Joins A materialized view delta join is a join that appears in the materialized view but not the query. All delta joins in a materialized view are required to be lossless with respect to the result of common joins. (Well get to this concept a little later.)

39 perfect solution outstanding experience Example Materialized View Delta Join Budget Balances -OBJECT_CODE_SID -ACCOUNT_SID -SUM(UA_BUD_CY_ANN) Account -ACCOUNT_SID -ACCOUNT_BUDGET_SHELL_CD MV Delta Join is made possible by guaranteeing a lossless join between the FACT and DIMENSION Budget Balances -SUM(UA_BUD_CY_ANN) This query removes any reference to the Account Dimension, which exists in the MV definition.

40 perfect solution outstanding experience Combined MV and Query Delta Join Budget Balances -OBJECT_CODE_SID -ACCOUNT_SID -SUM(UA_BUD_CY_ANN) Account -ACCOUNT_SID -ACCOUNT_BUDGET_SHELL_CD -Removes the Account Dimension (MV Delta Join) -Adds the Object Code Dimension (Query Delta Join) -Removes the Account Dimension (MV Delta Join) -Adds the Object Code Dimension (Query Delta Join) Budget Balances -OBJECT_CODE_SID -SUM(UA_BUD_CY_ANN) Object Code -OBJECT_CODE_SID -OBJECT_CD

41 perfect solution outstanding experience How are Delta Joins Possible? Table Relationships Must be Configured Properly Lossless Joins Non-Duplicating Joins Dimension Join Keys must be included in the SELECT of the MV definition

42 perfect solution outstanding experience Lossless Joins A lossless join guarantees that the result of common joins is not restricted Budget Balances -ACCOUNT_SID -COUNT(1) Account -ACCOUNT_SID ACCOUNT_SID = ACCOUNT_SID No rows were lost when Budget Balance was joined to Accounts Lossless Join

43 perfect solution outstanding experience Lossless Joins A lossless join guarantees that the result of common joins is not restricted Budget Balances -ACCOUNT_SID -COUNT(1) Employee Detail -EMPL_JOB_DETAIL_SID EMPL_JOB_DETAIL_SID = EMPL_JOB_DTL_SID Not a Lossless Join Rows were lost when Budget Balance was joined to Employee Detail Not a Lossless Join

44 perfect solution outstanding experience Lossless Joins A lossless join guarantees that the result of common joins is not restricted Budget Balances -ACCOUNT_SID -COUNT(1) Employee Detail -EMPL_JOB_DETAIL_SID EMPL_JOB_DETAIL_SID = EMPL_JOB_DTL_SID Not a Lossless Join Outer Joins will guarantee a Lossless Join EMPL_JOB_DETAIL_SID = EMPL_JOB_DTL_SID (+) Lossless Join

45 perfect solution outstanding experience Non-Duplicating Join A non-duplicating join guarantees that the result of common joins will not cause duplicates There must be a many-to-one relationship between the joins from the Fact to the Dimensions Many-to-OneMany-to-Many

46 perfect solution outstanding experience How to let Oracle Know To let Oracle know of the Lossless and Non- Duplicating joins the following must be configured on the tables in the subject area: Primary Key constraints must be created on all Dimensions Foreign Key constraints must be create on Fact join columns Join columns on the Fact must be set to NOT NULL unless that column will be outer joined to the Dimension

47 perfect solution outstanding experience Example Primary Key Constraint PK and FK constraints can cause havoc and slowness in a data warehouse Oracle has provided a way to create the constraints while avoiding the baggage DISABLE – Does not enforce the constraint on rows created after the constraint is created. Good for ETL. NOVALIDATE – Does not enforce the constraint on rows that exist in the table when the constraint is created.

48 perfect solution outstanding experience Example Primary Key Constraint If the PK and FK constraints are in DISABLE and NOVALIDATE mode there is one more step that must be taken in order to let Oracle know we can RELY on these constraints RELY – Tells Oracle that regardless of the state of the constraint, Oracle can assume the constraint is valid. Without this MV Rewrite functionality will not work properly if constraints are not enabled.

49 perfect solution outstanding experience Include Join Columns in MV Subject Area

50 perfect solution outstanding experience Nested Materialized Views Budget Balances MV1 Budget Report 3 Budget Report 4 Budget Report 5 Budget Balances Nested MV1 Budget Balances Nested MV2 Budget Report 1 Budget Report 2 Budget Report 6 Budget Report 7 60 Minute Refresh 2 Minute Refresh 4 Minute Refresh

51 perfect solution outstanding experience Compression Compressing data improves I/O because less blocks of data are read from or written to disk and sent back and forth between disk and database servers I/O is the biggest bottleneck in most Databases so this improves the performance of SELECTs Compression also saves disk space Basic compression is standard with Oracle Enterprise Edition and works well with MVs Compress your MVs

52 perfect solution outstanding experience Compression DataUncompressedCompressed Size in MB (76.5% Compression rate) Seconds to Query Shell Cd = C Seconds to Query Shell Cd = A Seconds to Query all Shell Cd

53 perfect solution outstanding experience Document your MVs We document our MVs in wiki and include: Reports tuned by MV Data Model of the MV Change History Constraints required along with DDLs to create them FK Index Requirements Required Database Grants Materialized View DDL Materialized View Index DDLs

54 perfect solution outstanding experience To Sum it Up Materialized View Query Rewrite is a powerful tool that can dramatically increase query performance Tuning your Warehouse with MVs instead of summarized subject areas is far more efficient and flexible Make sure to understand a few advanced concepts (Delta Joins and Nested MVs) before starting your adventure Use Compression Document your MVs

55 perfect solution outstanding experience Q/A Sean Kelleher Work: Cell: Web:www.tbginc.comwww.tbginc.com Please feel free to contact Sean with any questions


Download ppt "Perfect solution outstanding experience. ORACLE MATERIALZIED VIEWS A PERFORMANCE TUNING GEM."

Similar presentations


Ads by Google