Carl Dudley – University of Wolverhampton, UK Materialized Views Carl Dudley University of Wolverhampton, UK Oracle ACE Director UKOUG Committee

Carl Dudley – University of Wolverhampton, UK Materialized Views Carl Dudley University of Wolverhampton, UK Oracle ACE Director UKOUG Committee carl.dudley@wlv.ac.uk

2 Carl Dudley – University of Wolverhampton, UK Introduction Working with Oracle since 1986 Oracle DBA - OCP Oracle7, 8, 9, 10 Oracle DBA of the Year – 2002 Oracle ACE Director Regular Presenter at Oracle Conferences Consultant and Trainer Technical Editor for a number of Oracle texts UK Oracle User Group Director Member of IOUC Day job – University of Wolverhampton, UK

3 Carl Dudley – University of Wolverhampton, UK Topics Materialized View Features and Syntax Query Rewrite Refreshing Materialized Views Nested Materialized Views Implementing Constraints via Materialized Views Fast and Complete Refresh Performance Multiple Materialized Views Estimating the Size of Materialized Views

4 Carl Dudley – University of Wolverhampton, UK Materialized View Features  Data Warehouses typically contain a few very large tables —Joins and aggregate queries can be very expensive  MVs often contain summary tables of aggregations of a table's data —Pre-calculated results enhance query performance —Usually defined as aggregate views and/or join views For example, a view containing sums of salaries by job within department  Sometimes act as replica snapshots of tables on remote databases —Not covered in this presentation

5 Carl Dudley – University of Wolverhampton, UK Materialized Views Features (continued)  Not normally accessed directly —Queries on base tables are automatically re-written to use MVs  MVs can be nested —MVs built on other materialized views  Transparent to SQL applications —Can be created or dropped without affecting the validity of SQL statements  Consume storage space  Contents must be updated whenever underlying detail tables are modified

6 Carl Dudley – University of Wolverhampton, UK Creating Materialized Views CREATE MATERIALIZED VIEW emv BUILD IMMEDIATE REFRESH COMPLETE ON DEMAND AS SELECT deptno,SUM(sal) FROM emp GROUP BY deptno;  This view is populated with data upon creation  To keep the view synchronised with emp, it must be completely refreshed —This will be done manually on demand (the default)  Queries against emp will not be re-written to use the view (default)

7 Carl Dudley – University of Wolverhampton, UK Creating Materialized Views (continued) CREATE MATERIALIZED VIEW emv BUILD DEFERRED REFRESH FAST ON COMMIT ENABLE QUERY REWRITE AS SELECT deptno,SUM(sal),COUNT(*),COUNT(sal) FROM emp GROUP BY deptno;  Not populated with data until first refresh action takes place —The first refresh must be a complete refresh  To keep the view synchronised, it will undergo fast refresh operations —Refresh happens every time a transaction on emp commits  Queries against emp will be re-written to use the view —Unless the parameter QUERY_REWRITE_ENABLED is set to false

9 Carl Dudley – University of Wolverhampton, UK Query Rewrite  The optimizer compares costs when deciding to use a materialized view —Query is rewritten to use the materialized view —Underlying base tables are not used and performance is normally improved  A few carefully chosen MVs can support a large range of queries

10 Carl Dudley – University of Wolverhampton, UK Query Rewrite  Transformation of a query based on tables to an equivalent query based on one or more materialized views  Several factors affect whether query rewrite will occur 1.Enabling or disabling query rewrite can be achieved via the CREATE or ALTER statement for materialized views REWRITE and NOREWRITE hints in individual SQL statements SELECT /* +NOREWRITE */... The REWRITE_OR_ERROR hint will prevent the processing of a statement if it is not rewritten 2.Rewrite integrity levels Some levels allow the use of stale data 3.Dimensions and constraints Outside the scope of this talk but details are in the Oracle docs

11 Carl Dudley – University of Wolverhampton, UK Controlling Accuracy of Query Rewrite with QUERY_REWRITE_INTEGRITY ALTER SESSION SET QUERY_REWRITE_INTEGRITY = ENFORCED|TRUSTED|STALE_TOLERATED; ENFORCED MV data must be fresh and any constraints used must be enabled and validated TRUSTED Trusts data conforms to constraints that are not validated STALE_TOLERATED Allows stale MVs to be used  Only the ENFORCED level guarantees totally accurate results —MVs will not be used if they are not refreshed after the base table changes

12 Carl Dudley – University of Wolverhampton, UK Checking on the MV Situation  DML on underlying objects causes NEEDS_COMPILE when REFRESH ON DEMAND is specified  A refresh or compile fixes the state SELECT mview_name,refresh_mode,staleness,compile_state FROM user_mviews MVIEW_NAME REFRESH_MODE STALENESS COMPILE_STATE ------------------------ ------------ ------------- ------------- CAL_MONTH_SALES_MV DEMAND NEEDS_COMPILE NEEDS_COMPILE FWEEK_PSCAT_SALES_MV DEMAND STALE VALID SUM_SALES_PSCAT_WEEK_MV COMMIT FRESH VALID ALTER MATERIALIZED VIEW cal_month_sales_mv COMPILE; SELECT mview_name,refresh_mode,staleness,compile_state FROM user_mviews; MVIEW_NAME REFRESH_MODE STALENESS COMPILE_STATE ------------------------ ------------ ------------- ------------- CAL_MONTH_SALES_MV DEMAND STALE VALID FWEEK_PSCAT_SALES_MV DEMAND STALE VALID SUM_SALES_PSCAT_WEEK_MV COMMIT FRESH VALID

13 Carl Dudley – University of Wolverhampton, UK Query Rewrite Performance CREATE MATERIALIZED VIEW sales_prod_mv BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT prod_id,SUM(amount_sold) FROM sales GROUP BY prod_id; SELECT prod_id,SUM(amount_sold) FROM sales GROUP BY prod_id; Sample MV Sample query

14 Carl Dudley – University of Wolverhampton, UK Observing Query Rewrite  Queries may be rewritten to use the materialized view SELECT prod_id,SUM(amount_sold) FROM sales GROUP BY prod_id; Elapsed time : 0.03 seconds --------------------------------------------------------------------------- |Id | Operation | Name |Rows|Bytes | Cost(%CPU)| --------------------------------------------------------------------------- | 0| SELECT STATEMENT | | 10| 1872 | 3 (0)| | 1| MAT_VIEW REWRITE ACCESS FULL| SALES_PROD_MV| 72| 1872 | 3 (0)| --------------------------------------------------------------------------- ----------------------------------------------------------------- |Id | Operation | Name |Rows | Bytes | Cost(%CPU) | ----------------------------------------------------------------- | 0 | SELECT STATEMENT | | 73 | 657 | 1182 (17)| | 1 | HASH GROUP BY | | 73 | 657 | 1182 (17)| | 2 | TABLE ACCESS FULL| SALES | 932K| 8196K| 1038 (5)| -----------------------------------------------------------------  When the view is not present, the base table is scanned Elapsed time : 16.21 seconds

15 Carl Dudley – University of Wolverhampton, UK Did Query Rewrite Occur and Why?  Query rewrite occurs transparently —Check for query rewrite AUTOTRACE dbms_mview.explain_rewrite  dbms_mview.explain_rewrite —Gives reasons WHY the rewrite did not occur The rules governing this are extremely complex —Populates the rewrite_table with information Which materialized views will be used Comparative costings of the rewritten and un-rewritten query —Each account must have its own rewrite_table Created by running the utlxrw script in the admin directory

16 Carl Dudley – University of Wolverhampton, UK Explaining the rewrite: Example One BEGIN DBMS_MVIEW.EXPLAIN_REWRITE ('SELECT prod_id,SUM(amount_sold) FROM sales GROUP BY prod_id'); END; / SELECT message,original_cost,rewritten_cost FROM rewrite_table; MESSAGE ORIGINAL_COST REWRITTEN_COST ------------------------------------------------------------ ------------- -------------- QSM-01009: materialized view,prod_sum_mv, matched query text 180 3  An efficient rewrite occurs because of the exact match of the query and the definition of the view —Most rewrites are more complex  Aliases can be used for MV columns to allow exact match with table columns CREATE MATERIALIZED VIEW emv (e_deptno, d_deptno) ENABLE QUERY REWRITE AS SELECT e.deptno, d.deptno...

17 Carl Dudley – University of Wolverhampton, UK Explaining the rewrite: Example Two BEGIN DBMS_MVIEW.EXPLAIN_REWRITE ('SELECT prod_id,AVG(amount_sold) FROM sales GROUP BY prod_id'); END; / SELECT message,original_cost,rewritten_cost FROM rewrite_table; MESSAGE ORIGINAL_COST REWRITTEN_COST ------------------------------------------ ------------- -------------- QSM-01086: dimension(s) not present or not 0 0 used in ENFORCED integrity mode QSM-01065: materialized view, prod_sum_mv, 0 0 Cannot compute measure, AVG, in the query  Query rewrite cannot occur for the given reasons —No costs are calculated

18 Carl Dudley – University of Wolverhampton, UK Undocumented Query Rewrite Parameters KSPPINM KSPPSTVL -------------------------- -------- _explain_rewrite_mode FALSE _query_rewrite_or_error FALSE _query_cost_rewrite TRUE _query_rewrite_2 TRUE _query_rewrite_1 TRUE _query_rewrite_fudge 90 _query_rewrite_expression TRUE _query_rewrite_jgmigrate TRUE _query_rewrite_fpc TRUE _query_rewrite_drj TRUE _query_rewrite_maxdisjunct 257 _query_rewrite_vop_cleanup TRUE _mmv_query_rewrite_enabled TRUE KSPPINM KSPPSTVL ------------------------------------ ------------ _bt_mmv_query_rewrite_enabled TRUE _pre_rewrite_push_pred TRUE _union_rewrite_for_gs YES_GSET_MVS _query_rewrite_setopgrw_enable TRUE _force_rewrite_enable FALSE _query_mmvrewrite_maxpreds 10 _query_mmvrewrite_maxintervals 5 _query_mmvrewrite_maxinlists 5 _query_mmvrewrite_maxdmaps 10 _query_mmvrewrite_maxcmaps 20 _query_mmvrewrite_maxregperm 512 _query_mmvrewrite_maxmergedcmaps 50 _query_mmvrewrite_maxqryinlistvals 500 _enable_query_rewrite_on_remote_objs TRUE COL ksppinm FORMAT a40 COL ksppstvl FORMAT a40 SELECT ksppinm,ksppstvl FROM x$ksppi i,x$ksppcv v WHERE i.indx = v.indx AND i.ksppinm like '%rewrite%‘ AND SUBSTR(ksppinm,1,1) = '_';

20 Carl Dudley – University of Wolverhampton, UK Refreshing Materialized Views  If data in base tables change, MVs need to be refreshed in one of two ways 1.On transaction commit ( REFRESH ON COMMIT ) MV kept up to date but performance impact on base table transactions 2. On demand ( REFRESH ON DEMAND ) Requires a procedure to be invoked The procedure can take an argument to specify the type of refresh COMPLETE - complete refresh by repopulating FAST - incrementally applies changes to the view FORCE - fast refresh if possible, if not it forces a complete refresh dbms_mview.refresh('emp_mv1')

21 Carl Dudley – University of Wolverhampton, UK Refreshing Materialized Views  Refresh can be accelerated with ATOMIC_REFRESH set to 'FALSE ' —MV is truncated rather then deleted (default value is TRUE on 10g and up) Less undo and redo but MV unavailable during refresh —Allows parallel refresh  Fast Refresh can be accelerated if MV logs are created 'WITH COMMIT SCN' —Snapshots do not have to be built during the refresh —Use is made of all_summap view —Can also set NOLOGGING for the materialized view logs

22 Carl Dudley – University of Wolverhampton, UK Materialized View Logs  Materialized view logs are required for fast refresh  Defined using CREATE MATERIALIZED VIEW LOG on the base table —Not created on the materialized view —Each inserted and deleted row on the base table creates a row in the log —An update may generate two rows in the log for old and new values  Normally need to specify the ROWID clause  Logs that support aggregate materialized views must also contain: 1.Every column in the table that is referenced in the materialized view 2.The INCLUDING NEW VALUES clause —Name of the log is mlog$_ —Specify SEQUENCE if the table is expected to have a mix of inserts/direct- loads, deletes, and updates CREATE MATERIALIZED VIEW LOG ON emp WITH ROWID (deptno,sal) INCLUDING NEW VALUES;

23 Carl Dudley – University of Wolverhampton, UK Tuning Materialized Views for Fast Refresh and Query Rewrite  dbms_advisor.tune_mview helps optimize CREATE MATERIALIZED VIEW statements  Shows how to implement materialized views and any required logs  The output statements for ‘ IMPLEMENTATION’ include: — CREATE MATERIALIZED VIEW LOG statements Creates any missing materialized view logs required for fast refresh — ALTER MATERIALIZED VIEW LOG FORCE statements Fixes any materialized view logs for maybe fast refresh —One or more CREATE MATERIALIZED VIEW statements Could suggest just adding required columns For example, add ROWID column for materialized join view and add aggregate column for materialized aggregate view

24 Carl Dudley – University of Wolverhampton, UK Tuning Materialized Views – Using the Advisor VARIABLE task_cust_mv VARCHAR2(30); VARIABLE create_mv_ddl VARCHAR2(4000); BEGIN :task_cust_mv := 'emp_mv1'; :create_mv_ddl := 'CREATE MATERIALIZED VIEW emp_mv1 REFRESH FAST ENABLE QUERY REWRITE AS SELECT deptno,sum(sal) FROM emp GROUP BY deptno'; DBMS_ADVISOR.TUNE_MVIEW(:task_cust_mv, :create_mv_ddl); END; /  Tasks can be deleted EXEC dbms_advisor.delete_task( ' emp_mv1 ' ) Requires ADVISOR privilege

25 Carl Dudley – University of Wolverhampton, UK Tuning Materialized Views – Examining the Output SELECT statement FROM user_tune_mview WHERE task_name= :task_cust_mv AND script_type='IMPLEMENTATION'; STATEMENT ------------------------------------------------------------------ CREATE MATERIALIZED VIEW LOG ON "SCOTT"."EMP" WITH ROWID, SEQUENCE  "SAL","DEPTNO") INCLUDING NEW VALUES ALTER MATERIALIZED VIEW LOG FORCE ON "SCOTT"."EMP" ADD ROWID, SEQUENCE  "SAL","DEPTNO") INCLUDING NEW VALUES CREATE MATERIALIZED VIEW SCOTT.EMP_MV1 REFRESH FAST WITH ROWID ENABLE QUERY REWRITE AS SELECT SCOTT.EMP.DEPTNO C1, SUM("SCOTT"."EMP"."SAL") M1, COUNT("SCOTT"."EMP"."SAL") M2, COUNT(*) M3 FROM SCOTT.EMP GROUP BY  SCOTT.EMP.DEPTNO  For the view to be fast refreshed 1.A log is recommended because the view will be fast refreshed 2.The view must include a count of the aggregate column and also a count of the rows

26 Carl Dudley – University of Wolverhampton, UK Tuning Materialized Views – a Further Example BEGIN :task_cust_mv := 'emp_mv2'; :create_mv_ddl := 'CREATE MATERIALIZED VIEW emp_mv2 REFRESH FAST ENABLE QUERY REWRITE AS SELECT dept.deptno,dname,sum(sal) FROM emp,dept WHERE dept.deptno = emp.deptno GROUP BY dept.deptno,dname' DBMS_ADVISOR.TUNE_MVIEW(:task_cust_mv, :create_mv_ddl); END; /

27 Carl Dudley – University of Wolverhampton, UK Further Example Output SELECT statement FROM user_tune_mview WHERE task_name= :task_cust_mv AND script_type='IMPLEMENTATION'; STATEMENT -------------------------------------------------------------------- CREATE MATERIALIZED VIEW LOG ON "SCOTT"."DEPT" WITH ROWID, SEQUENCE ("DEPTNO","DNAME") INCLUDING NEW VALUES ALTER MATERIALIZED VIEW LOG FORCE ON "SCOTT"."DEPT" ADD ROWID, SEQUENCE ("DEPTNO","DNAME") INCLUDING NEW VALUES CREATE MATERIALIZED VIEW LOG ON "SCOTT"."EMP" WITH ROWID, SEQUENCE ("SAL","DEPTNO") INCLUDING NEW VALUES ALTER MATERIALIZED VIEW LOG FORCE ON "SCOTT"."EMP" ADD ROWID, SEQUENCE ("SAL","DEPTNO") INCLUDING NEW VALUES CREATE MATERIALIZED VIEW SCOTT.EMP_MV2 REFRESH FAST WITH ROWID ENABLE QUERY REWRITE AS SELECT SCOTT.DEPT.DNAME C1, SCOTT.DEPT.DEPTNO C2, SUM("SCOTT"."EMP"."SAL") M1, COUNT("SCOTT"."EMP"."SAL") M2, COUNT(*) M3 FROM SCOTT.EMP, SCOTT.DEPT WHERE SCOTT.DEPT.DEPTNO = SCOTT.EMP.DEPTNO GROUP BY SCOTT.DEPT.DNAME, SCOTT.DEPT.DEPTNO

28 Carl Dudley – University of Wolverhampton, UK Materialized View Capabilities  dbms_mview.explain_mview is used to explain the capabilities (query rewrite, refresh modes) of an MV —The optional value '6' is a user supplied statement identifier  Places information in mv_capabilities_table —Has one row for each reason why a certain action is not possible —Needs to be built in each investigating account by executing the script utlxmv.sql found in the admin directory —The statement_id column can be used to fetch rows for a given MV EXECUTE dbms_mview.explain_mview ('emp_mv1','6')

29 Carl Dudley – University of Wolverhampton, UK The mv_capabilities_table EXECUTE dbms_mview.explain_mview('emp_mv1','6') SELECT capability_name,possible,msgtxt FROM mv_capabilities_table WHERE statement_id = '6'; CAPABILITY_NAME P MSGTXT ----------------------------- - ----------------------------------------------- PCT N REFRESH_COMPLETE Y REFRESH_FAST Y REWRITE Y PCT_TABLE N relation is not a partitioned table REFRESH_FAST_AFTER_INSERT Y REFRESH_FAST_AFTER_ONETAB_DML N SUM(expr) without COUNT(expr) REFRESH_FAST_AFTER_ONETAB_DML N COUNT(*) is not present in the select list REFRESH_FAST_AFTER_ANY_DML N mv log does not have sequence # REFRESH_FAST_AFTER_ANY_DML N see the reason why REFRESH_FAST_AFTER_ONETAB_DML is disabled REFRESH_FAST_PCT N PCT is not possible on any of the detail tables in the materialized view REWRITE_FULL_TEXT_MATCH Y REWRITE_PARTIAL_TEXT_MATCH Y REWRITE_GENERAL Y REWRITE_PCT N general rewrite is not possible or PCT is not possible on any of the detail tables PCT_TABLE_REWRITE N relation is not a partitioned table

30 Carl Dudley – University of Wolverhampton, UK Restrictions on Fast Refresh on MVs There are many more special case restrictions and rules  Joins… — ROWID s of all tables in FROM list must appear in the SELECT list of the query —Materialized view logs must exist on all tables involved in the join  Aggregates… —Only SUM, COUNT, AVG, STDDEV, VARIANCE, MIN and MAX are supported for fast refresh COUNT(*) must be included in the view —For each aggregate such as AVG(expr), the corresponding COUNT(expr) must be present

31 Carl Dudley – University of Wolverhampton, UK salessales_mv mlog$_sales 25 47 25 47 25 47 62 materialized view Materialized view log build commit 62 Base table 62  REFRESH COMPLETE ON COMMIT does not require logs – Very unlikely to be used Fast Refresh on Commit

32 Carl Dudley – University of Wolverhampton, UK Bulk Loading – Impact on Fast Refresh 1.INSERT /*+ append */ 2.SQL*Loader DIRECT=TRUE  Improved performance as no data written to MV log —Much less redo and undo —Minimal tracking in all_sumdelta Summary information with ROWID ranges

33 Carl Dudley – University of Wolverhampton, UK Flexing the Materialized Views with ON PREBUILT TABLE  MVs cannot be ALTER ed —What if a column width change is required? Could make use of ON PREBUILT TABLE to avoid MV rebuild CREATE TABLE j_g AS SELECT job, COUNT(*) ct FROM emp GROUP BY job; CREATE MATERIALIZED VIEW j_g ON PREBUILT TABLE AS SELECT job, COUNT(*) ct FROM emp GROUP BY job; OBJECT_ID DATA_OBJECT_ID OBJECT_NAME OBJECT_TYPE --------- -------------- ----------- ------------------- 86118 86118 J_G TABLE 86119 J_G MATERIALIZED VIEW DROP TABLE j_g; -- fails due to presence of MV DROP MATERIALIZED VIEW j_g; --succeeds but leaves table intact ALTER TABLE j_g MODIFY job VARCHAR2(40); CREATE MATERIALIZED VIEW j_g ON PREBUILT TABLE AS SELECT job, COUNT(*) ct FROM emp GROUP BY job; —Can also help when fixing 'out of sync' errors with logs and base tables ORA-12034: materialized view log younger than last refresh

35 Carl Dudley – University of Wolverhampton, UK Nested Materialized Views Base table 1Base table 2Base table 3 Join Materialized View Aggregate Materialized View 1 Aggregate Materialized View 2 Aggregate Materialized View 3 Aggregate Materialized View 4 Nested Views

36 Carl Dudley – University of Wolverhampton, UK prod_sales (master) Prod_name Cust_id Amt_sold Bolt 237 22.05 Pin 120 9.23 Bolt 194 3.87 Refreshing Nested MVs  DBMS_MVIEW.REFRESH_DEPENDENT with NESTED = TRUE refreshes all views in a hierarchy that depend on one or more specified base tables sum_prod_sales (nested) Prod_name SUM(Amt_sold) Bolt 25.92 Pin 9.23 Fresh products Prod_id Prod_name 10 Bolt 14 Pin Sales Prod_id Cust_id Amt_sold 10 237 22.05 14 120 9.23 10 194 3.87 Fresh Stale Refresh nested view Prod_name SUM(Amt_sold) Bolt 136.75 Pin 9.23 Stale sales Prod_id Cust_id Amt_sold 10 237 22.05 14 120 9.23 10 194 3.87 10 105 110.83 Update the sales table Refresh the master Prod_name Cust_id Amt_sold Bolt 237 22.05 Pin 120 9.23 Bolt 194 3.87 Bolt 105 110.83

37 Carl Dudley – University of Wolverhampton, UK Nested MVs in Query Rewrite 2. Every master object requires a log containing the ROWID CREATE MATERIALIZED VIEW master_nest_mv BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT p.prod_id, p.prod_name, c.channel_id, c.channel_desc FROM sales s, products p, channels c WHERE s.prod_id = p.prod_id AND s.channel_id = c.channel_id; 1.Create a master materialized view CREATE MATERIALIZED VIEW LOG ON sales WITH ROWID; CREATE MATERIALIZED VIEW LOG ON products WITH ROWID; CREATE MATERIALIZED VIEW LOG ON channels WITH ROWID; 3. The master materialized view also requires a log containing ROWID, each column in the materialized view itself, and all new values CREATE MATERIALIZED VIEW LOG ON master_nest_mv WITH ROWID (prod_id, prod_name, channel_id, channel_desc) INCLUDING NEW VALUES;

38 Carl Dudley – University of Wolverhampton, UK Nested MVs in Query Rewrite (continued) — AUTOTRACE shows evidence of use but does not show the hierarchy ----------------------------------------------------------------------------------- |Id| Operation |Name |Rows|Bytes|Cost(%CPU)| Time | ----------------------------------------------------------------------------------- | 0|SELECT STATEMENT | | 225| 9450| 3 (0) |00:00:01| | 1|MAT_VIEW REWRITE ACCESS FULL|NESTED_PROD_CHAN_MV| 225| 9450| 3 (0) |00:00:01| ----------------------------------------------------------------------------------- CREATE MATERIALIZED VIEW nested_prod_chan_mv BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT channel_desc, prod_name, COUNT(prod_id) FROM master_nest_mv GROUP BY channel_desc, prod_name; 4. Create the nested MV based on the master MV

39 Carl Dudley – University of Wolverhampton, UK Nested MVs in Query Rewrite (continued) SELECT message FROM rewrite_table; MESSAGE ----------------------------------------------------------------- QSM-01151: query was rewritten QSM-01033: query rewritten with materialized view, NESTED_PROD_CHAN_MV QSM-01336: the materialized view you specified (NESTED_PROD_CHAN_MV) was not used to rewrite the query QSM-01033: query rewritten with materialized view, MASTER_NEST_MV BEGIN DBMS_MVIEW.EXPLAIN_REWRITE( 'SELECT c.channel_desc, p.prod_name FROM sales s, products p, channels c WHERE s.prod_id = p.prod_id AND c.channel_id = s.channel_id GROUP BY channel_desc, prod_name ','NESTED_PROD_CHAN_MV'); END; /

41 Carl Dudley – University of Wolverhampton, UK Foreign Keys Referencing Non-unique Columns  Problem :  Need to enforce a foreign key constraint on the occupation column in the empdep table based on data in the emp table —Requires a reference to a non-unique column ( job ) in the emp table  Reference a materialized view carrying only unique values of job EMPNO JOB ----- -------- 7366 SALESMAN 7500 MANAGER 7902 MANAGER 7566 CLERK 7934 SALESMAN JOB -------- SALESMAN MANAGER CLERK ENAME OCCUPATION -------- ---------- WOODS CLERK JOHNSON MANAGER COX CLERK PITT CLERK SPINK SALESMAN DRAPER MANAGER EMP EMP_V1 EMPDEP primary key foreign key

42 Carl Dudley – University of Wolverhampton, UK Enforcing Foreign Keys Without Primary keys CREATE MATERIALIZED VIEW LOG ON emp WITH ROWID,PRIMARY KEY,SEQUENCE(job) INCLUDING NEW VALUES; CREATE MATERIALIZED VIEW emp_v1 REFRESH FAST ON COMMIT ENABLE QUERY REWRITE AS SELECT job,COUNT(*),COUNT(job) FROM emp GROUP BY job; ALTER MATERIALIZED VIEW emp_v1 ADD PRIMARY KEY (job); ALTER TABLE empdep ADD CONSTRAINT empdep_emp_v1 FOREIGN KEY (occupation) REFERENCES emp_v1; UPDATE empdep SET occupation = 'X'; ORA-02291: integrity constraint (SCOTT.EMPDEP_EMP_V1) violated - parent key not found job is not unique in emp job is unique in MV empdep must have only jobs in the emp table

43 Carl Dudley – University of Wolverhampton, UK Enforcing Complex Constraints with Materialized Views  Employee's appraisal date must be within their department's appraisal period EMPNO ENAME JOB MGR HIREDATE SAL COMM DEPTNO APP_DATE ----- ------ --------- ---- --------- ---- ---- ------ --------- 7369 SMITH CLERK 7902 17-DEC-80 800 20 15-MAR-11 7499 ALLEN SALESMAN 7698 20-FEB-81 1600 300 30 15-APR-11 7521 WARD SALESMAN 7698 22-FEB-81 250 500 30 15-APR-11 7566 JONES MANAGER 7839 02-APR-81 2975 20 15 MAR-11 DEPTNO DNAME LOC START_APP_PERIOD END_APP_PERIOD ------ -------- -------- ---------------- -------------- 20 RESEARCH DALLAS 01-FEB-11 01-MAY-11 30 SALES CHICAGO 01-MAR-11 01-JUN-11  MV which returns app_date s with departmental appraisal periods CREATE MATERIALIZED VIEW emp_dept_appr_mv REFRESH FAST ON COMMIT AS SELECT e.deptno,e.app_date e_app_date,d.start_app_period d_start_app_period,d.end_app_period d_end_app_period,e.ROWID e_rowid,d.ROWID d_rowid FROM emp e,dept d WHERE e.deptno = d.deptno constraint checked when changes to the emp table are committed

44 Carl Dudley – University of Wolverhampton, UK Enforcing Complex Constraints with Materialized Views (continued)  Prevent invalid appr_date s appearing in the view ALTER TABLE emp_dept_appr_mv ADD CONSTRAINT emp_dept_appr_mv_chk1 CHECK (e_app_date BETWEEN d_start_app_period AND d_end_app_period) DEFERRABLE; —Constraint is checked on commit rather than on update  Insert a row with an appr_date outside of the departmental period INSERT INTO emp (empno,deptno,app_date) VALUES(1, 20, '01-jan-2013');  Constraint is not enforced until the commit (and refresh) takes place SQL> COMMIT; ORA-12048: error encountered while refreshing materialized View "EMP_DEPT_APPR_MV" ORA-02290: check constraint (emp_dept_appr_mv_chk1) violated

45 Carl Dudley – University of Wolverhampton, UK Testing Complex Constraint Checking with Materialized Views  Could lead to large numbers of rows in the MV when constraint is repeatedly NOT violated —Hence, simply reverse the MV condition to prevent valid data in the view —The MV never collects any data Constraint prevents the invalid data But the MV logs are still populated! AND NOT (e.app_date BETWEEN d.start_app_period AND d.end_app_period)

46 Carl Dudley – University of Wolverhampton, UK Indexes on Materialized Views  A single unique index is often created on an MV when it is built —If the MV is a grouped (aggregate) MV, the index is a function-based Btree and has a name of I_SNAP$_ —Built on virtual RAW column(s) built into the MV —Number of columns in the index is equal to number of grouping columns Names are similar to sys_nc00003$ CREATE MATERIALIZED VIEW sales_mv AS SELECT p.prod_subcategory, t.week_ending_day, SUM(s.amount_sold) AS sum_amount_sold FROM sales s, products p, times t WHERE s.time_id = t.time_id AND s.prod_id = p.prod_id GROUP BY p.prod_subcategory, t.week_ending_day generates two hidden indexed columns in the MV

47 Carl Dudley – University of Wolverhampton, UK Indexes on Grouped Materialized Views  Values in the hidden columns SELECT sys_nc00004$,sys_nc00005$,prod_subcategory,week_ending_day FROM sales_mv; SYS_NC00004$ SYS_NC00005$ PROD_SUBCATEGORY WEEK_ENDING_DAY ------------------------ ---------------- ---------------- --------------- 43616D6572617300 77C6020101010100 Cameras 01-FEB-98 4D6F6E69746F727300 77C6031601010100 Monitors 22-MAR-98 4465736B746F702050437300 77C6030801010100 Desktop PCs 08-MAR-98 43616D636F726465727300 77C6010B01010100 Camcorders 11-JAN-98 43616D636F726465727300 77C6011901010100 Camcorders 25-JAN-98 4163636573736F7269657300 77C6020101010100 Accessories 01-FEB-98 4163636573736F7269657300 77C6030101010100 Accessories 01-MAR-98 486F6D6520417564696F00 77C6010B01010100 Home Audio 11-JAN-98 486F6D6520417564696F00 77C6031601010100 Home Audio 22-MAR-98 : : : : —43 is the ASCII value of ‘C’ in hexadecimal

48 Carl Dudley – University of Wolverhampton, UK Indexes on Non-grouped Materialized Views  If base table has a PK and a one to one mapping of rows with the MV —A normal Btree index is built on the PK within the view Oracle recommends building an index on any ROWID column in the MV and a concatenated index on all key columns of the base tables on the MV (for fast refresh) —If PK on the base table is explicitly named (e.g PK_EMP) The normal Btree index on the MV is named PK_EMP1 and supports a constraint named PK_EMP1  If no primary key on the base table —No index is built on the MV Indexes are not built when the USING NO INDEX clause is used in MV creation

50 Carl Dudley – University of Wolverhampton, UK Managing Update and Refresh Operations No. of rows updated UpdateRefresh FastCompleteFastComplete 100:00.200:00.100:01.400:13.2 10000:00.2 00:02.200:07.3 50000:00.500:00.100:00.200:09.3 100000:00.700:00.100:01.500:07.7 500000:06.500:00.300:01.600:08.0 5000000:58.000:19.100:39.100:10.5 7500000:35.700:22.300:53.500:08.5 10000000:58.900:28.501:40.600:06.1 50000008:52.902:47.712:40.800:07.2  Tests performed on a 918443 row sales table having one MV in fast or complete refresh mode —Updates performed in batch mode (single update statement) on base table

51 Carl Dudley – University of Wolverhampton, UK Updating Tables having Materialized Views Batch Update Performance on Table with Fast or Complete Refresh Materialized View 00:00.0 01:00.0 02:00.0 03:00.0 04:00.0 05:00.0 06:00.0 07:00.0 08:00.0 09:00.0 10:00.0 1 100500 10005000 5000075000 100000500000 No of Rows Updated Time Taken (Minutes) Fast Complete  Fast refresh has the overhead of MV log maintenance

52 Carl Dudley – University of Wolverhampton, UK Fast and Complete Refresh Performance Performance of Fast Vs Complete Refresh 00:00.0 01:00.0 02:00.0 03:00.0 04:00.0 05:00.0 06:00.0 07:00.0 08:00.0 09:00.0 10:00.0 11:00.0 12:00.0 1 100 500 1000 5000 50000 75000 100000 500000 No of Rows Updated Time Taken to Refresh (Minutes) Fast Complete  MV is truncated during a complete refresh —Faster for large scale updates

53 Carl Dudley – University of Wolverhampton, UK Overheads on OLTP - Transaction Test  Repeated updates on a single row BEGIN FOR i in 1..n LOOP UPDATE sales SET amount_sold = amount_sold +.01 WHERE prod_id = 14 AND cust_id = 50840 AND time_id = '03-NOV-01' AND channel_id = 3; COMMIT; END LOOP; END;  sales table has an index to avoid full table scans masking update timings ALTER TABLE sales ADD CONSTRAINT pk_sales PRIMARY KEY (prod_id,cust_id,time_id,channel_id);

54 Carl Dudley – University of Wolverhampton, UK Transaction Overheads - Test  Routine timed with and without MV log on sales —Log was truncated after each test CREATE MATERIALIZED VIEW LOG ON sales WITH ROWID (cust_id,prod_id,channel_id,time_id,amount_sold,quantity_sold) INCLUDING NEW VALUES;  Timings also produced with and without the COMMIT statement

55 Carl Dudley – University of Wolverhampton, UK Timings With and Without MV log and COMMIT Number of iterations (n) Time taken (seconds) With commitWithout commit No MV log MV log present No MV log MV log present 5000.210.250.030.15 10000.340.430.060.25 25000.620.960.230.51 50001.141.890.401.07 75001.672.810.611.58 100002.034.000.792.09 5000011.1018.593.9310.62  Presence of MV log can result in up to 250% performance overhead

56 Carl Dudley – University of Wolverhampton, UK Contention Caused by Materialized Views  Applies to REFRESH ON COMMIT views in OLTP environments —Refresh takes place as part of the OLTP transaction —Aggregate MVs are prone to contention issues —Updates on base tables are 'condensed' into fewer rows in the MV —Causes locking and contention issues as transactions queue to update the grouped row in the MV EMPNO ENAME SAL DEPTNO ----- ------- ---- ------ 7782 CLARK 2450 10 7839 KING 5000 10 7934 MILLER 1300 10 7369 SMITH 800 20 7876 ADAMS 1100 20 7902 FORD 3000 20 7788 SCOTT 3000 20 7566 JONES 2975 20 7521 WARD 1250 30 7499 ALLEN 1600 30 7654 MARTIN 1250 30 7698 BLAKE 2850 30 7844 TURNER 1500 30 7900 JAMES 950 30 CREATE MATERIALIZED VIEW m1 REFRESH FAST ON COMMIT AS SELECT deptno, SUM(sal),COUNT(*), COUNT(sal) FROM emp GROUP BY deptno; DEPTNO SUM(SAL) COUNT(*) COUNT(sal) ------ -------- -------- ---------- 10 8750 3 3 20 10875 5 5 30 940 6 6 Possible contention for locks of type JI

58 Carl Dudley – University of Wolverhampton, UK Using Multiple Materialized Views CREATE MATERIALIZED view prod_high_mv BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT prod_id, SUM(amount_sold) FROM sales WHERE prod_id > 15 GROUP BY prod_id; CREATE MATERIALIZED view prod_low_mv BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT prod_id, SUM(amount_sold) FROM sales WHERE prod_id <= 15 GROUP BY prod_id; PROD_ID SUM(AMOUNT_SOLD) ------- ---------------- 14 733738663 10 50236718.5 15 734982650 11 49879911.7 13 662255842 12 33032602.8 PROD_ID SUM(AMOUNT_SOLD) ---------- ---------------- 17 808449750 16 220330955 19 78746741.5 18 1005874847  Two materialized views each covering only part of the base data —Table contains sales data for prod_id s from 10 to 19

59 Carl Dudley – University of Wolverhampton, UK Using Multiple Materialized Views Execution Plan ----------------------------------------------------------------------------------------- | Id | Operation | Name |Rows | Bytes | Cost (%CPU)| Time | ----------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 10 | 110 | 6 (50)| 00:00:01| | 1 | UNION-ALL | | | | | | | 2 | MAT_VIEW REWRITE ACCESS FULL| PROD_LOW_MV | 6 | 66 | 3 (0)| 00:00:01| | 3 | MAT_VIEW REWRITE ACCESS FULL| PROD_HIGH_MV| 4 | 44 | 3 (0)| 00:00:01| SELECT prod_id, SUM(amount_sold) FROM sales -- WHERE prod_id > 0 GROUP BY prod_id; A WHERE clause appears to be necessary for multiple MVs to be used in pre 11gR2

60 Carl Dudley – University of Wolverhampton, UK Overlapping Materialized Views CREATE MATERIALIZED VIEW prod_low_mv BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT prod_id, SUM(amount_sold) FROM sales WHERE prod_id <= 15 GROUP BY prod_id; PROD_ID SUM(AMOUNT_SOLD) ------- ---------------- 14 733738663 10 50236718.5 15 734982650 11 49879911.7 13 662255842 12 33032602.8 CREATE MATERIALIZED VIEW prod_high_mv BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT prod_id, SUM(amount_sold) FROM sales WHERE prod_id >= 15 GROUP BY prod_id; PROD_ID SUM(AMOUNT_SOLD) ------- ---------------- 15 734982650 17 808449750 16 220330955 19 78746741.5 18 1005874847

61 Carl Dudley – University of Wolverhampton, UK Overlapping Materialized Views  Duplicate row for prod_id = 15 is (correctly) not shown in query output SELECT prod_id, SUM(amount_sold) FROM sales WHERE prod_id > 0 GROUP BY prod_id ORDER BY prod_id; PROD_ID SUM(AMOUNT_SOLD) ------- ---------------- 10 50236718.5 11 49879911.7 12 33032602.8 13 662255842 14 733738663 15 734982650 16 220330955 17 808449750 18 1005874847 19 78746741.5

62 Carl Dudley – University of Wolverhampton, UK Overlapping Materialized Views  But the plan shows a UNION-ALL operation across the two views —This is due to re-aggregation that is required across inline views -------------------------------------------------------------- |Id| Operation | Name | -------------------------------------------------------------- | 0| SELECT STATEMENT | | | 1| VIEW | | | 2| UNION-ALL | | | 3| MAT_VIEW REWRITE ACCESS FULL | PROD_HIGH_MV | | 4| MAT_VIEW REWRITE ACCESS FULL | PROD_LOW_MV | -------------------------------------------------------------- SELECT prod_id,SUM(sa) FROM (SELECT prod_id, SUM(amount_sold) sa FROM sales WHERE prod_id <= 15 GROUP BY prod_id UNION SELECT prod_id, SUM(amount_sold) sa FROM sales WHERE prod_id >= 15 GROUP BY prod_id) GROUP BY prod_id ORDER BY prod_id; Equivalent direct use of MVs generates exact same plan even when ALL is not specified

63 Carl Dudley – University of Wolverhampton, UK Multiple MVs with Different Aggregate Functions  The two views generate same groups —Different aggregate functions CREATE MATERIALIZED view prod_avg_mv BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT prod_id, AVG(amount_sold) FROM sales GROUP BY prod_id; CREATE MATERIALIZED view prod_sum_mv BUILD IMMEDIATE ENABLE QUERY REWRITE AS SELECT prod_id, SUM(amount_sold) FROM sales GROUP BY prod_id; SELECT prod_id, SUM(amount_sold), AVG(amount_sold) FROM sales GROUP BY prod_id; ---------------------------------------------------------------------------- | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | ---------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 72 | 648 | 960 (3)| 00:00:12 | | 1 | HASH GROUP BY | | 72 | 648 | 960 (3)| 00:00:12 | | 2 | TABLE ACCESS FULL| SALES | 500K| 4394K| 940 (1)| 00:00:12 | ---------------------------------------------------------------------------- —Neither view is used

64 Carl Dudley – University of Wolverhampton, UK Multiple MVs with Different Aggregate Functions - a Workaround  Encapsulate the views in the FROM clause —Results from views are hash-joined SELECT a.prod_id, a.sum_prod, b.avg_prod FROM (SELECT prod_id,SUM(amount_sold) sum_prod FROM sales GROUP BY prod_id) a, (SELECT prod_id,AVG(amount_sold) avg_prod FROM sales GROUP BY prod_id) b WHERE a.prod_id = b.prod_id; ----------------------------------------------------------------------------------------- | Id | Operation | Name | Rows| Bytes| Cost (%CPU)| Time | ----------------------------------------------------------------------------------------- | 0 | SELECT STATEMENT | | 72| 2664| 7 (15)| 00:00:01 | | 1 | HASH JOIN | | 72| 2664| 7 (15)| 00:00:01 | | 2 | MAT_VIEW REWRITE ACCESS FULL| PROD_SUM_MV | 72| 792| 3 (0)| 00:00:01 | | 3 | MAT_VIEW REWRITE ACCESS FULL| PROD_AVG_MV | 72| 1872| 3 (0)| 00:00:01 | ----------------------------------------------------------------------------------------- —Both views are used

66 Carl Dudley – University of Wolverhampton, UK Estimating Size of Materialized Views  Materialized views can consume significant amounts of storage  Estimate size of MV with dbms_mview.estimate_mview_size —Estimates number of rows and bytes  Accuracy of the estimate procedure was tested on an original sales table of 918443 rows, joined with customers and products

67 Carl Dudley – University of Wolverhampton, UK The Oracle Block Block 17 Free Space Row 4 Block header Reserved for updates to existing rows 1 2 Row numbers  8K block —Row header occupies approximately 100 bytes —819 bytes of free space reserved for updates (default PCTFREE of 10%) —Inserted data can occupy around 7300 bytes

68 Carl Dudley – University of Wolverhampton, UK Estimate Routine The output from the routine is as follows : Number of rows is : 1020125 Number of bytes is : 126495500 DECLARE nr NUMBER; --number of rows nb NUMBER; --number of bytes BEGIN dbms_mview.estimate_mview_size (stmt_id => 'mv_est', select_clause => 'SELECT c.cust_id,c.cust_city,p.prod_id,p.prod_name FROM sales s, customers c, products p WHERE s.cust_id = c.cust_id AND s.prod_id = p.prod_id', num_rows => nr, num_bytes => nb); dbms_output.put_line('Number of rows is : '||TO_CHAR(nr)); dbms_output.put_line('Number of bytes is : '||TO_CHAR(nb)); END;

69 Carl Dudley – University of Wolverhampton, UK Accuracy of Estimate  On building the view, the number of rows was actually 918443 —This shows that the row estimate of 1020125 is reasonably accurate  Its ROWID s were interrogated to find the actual block storage SELECT COUNT(COUNT(*)) “No. of Blocks” FROM mv_est GROUP BY dbms_rowid.rowid_relative_fno(ROWID),dbms_rowid.rowid_block_number(ROWID); No. of Blocks ------------- 6170  If each block has around 7300 bytes of data, then the number of bytes is 45041000 (7300*6170) —This is wildly different from the estimated value of 126495500

70 Carl Dudley – University of Wolverhampton, UK Checking the Dictionary EXEC dbms_stats.gather_table_stats('SH',' ') SELECT avg_col_len,column_name FROM user_tab_cols WHERE table_name IN ('CUSTOMERS','PRODUCTS') AND column_name IN ('CUST_ID','PROD_ID','CUST_CITY','PROD_NAME'); AVG_COL_LEN COLUMN_NAME ----------- ------------ 5 CUST_ID 10 CUST_CITY 4 PROD_ID 26 PROD_NAME  So, on average, each row should consume 45 + 5 (row overhead) bytes  The total number of occupied blocks should be (45+5)*918443/7300 = 6290 blocks —This is close to the result obtained from the actual ROWID s of the MV (6170)  Perhaps a more accurate value can be obtained from the dictionary

71 Carl Dudley – University of Wolverhampton, UK Results and Conclusions – Query Rewrite  MVs return queries in a fraction of the time  MVs are considered for rewrite even if they contain only part of the data required by the query  MVs are not considered for query rewrite if base table(s) are updated —Unless the session settings are altered to STALE_TOLERATED mode This emphasizes the need for refreshing the views  Multiple materialized views can be used to satisfy a single query — However if the aggregate materialized views do not include the same aggregate function then neither view will be considered  The estimate_mview_size procedure — Extremely accurate in estimating the number of rows an MV will contain — Not so accurate at predicting actual storage consumption

72 Carl Dudley – University of Wolverhampton, UK Results and Conclusions – Refresh Methods  REFRESH FAST —Efficient refresh performance when the numbers of rows affected by a single transaction was small —Due to the presence of the materialized view log —BUT if transaction rate is significant, could have significant effect  REFRESH COMPLETE ON DEMAND suits batch processing environments —Table updates are large and infrequent  REFRESH FAST ON COMMIT suits OLTP environments —WELL perhaps —Regular yet small table updates depending on real-time data

Carl Dudley – University of Wolverhampton, UK Materialized Views Carl Dudley University of Wolverhampton, UK Oracle ACE Director UKOUG Committee carl.dudley@wlv.ac.uk

Carl Dudley – University of Wolverhampton, UK Materialized Views Carl Dudley University of Wolverhampton, UK Oracle ACE Director UKOUG Committee

Similar presentations

Presentation on theme: "Carl Dudley – University of Wolverhampton, UK Materialized Views Carl Dudley University of Wolverhampton, UK Oracle ACE Director UKOUG Committee"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Carl Dudley – University of Wolverhampton, UK Materialized Views Carl Dudley University of Wolverhampton, UK Oracle ACE Director UKOUG Committee

Similar presentations

Presentation on theme: "Carl Dudley – University of Wolverhampton, UK Materialized Views Carl Dudley University of Wolverhampton, UK Oracle ACE Director UKOUG Committee"— Presentation transcript:

Similar presentations

About project

Feedback