We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byDestinee Cast
Modified over 2 years ago
Optimizing and Simplifying Complex SQL with Advanced Grouping Presented by: Jared Still
© 2009/2010 Pythian About Me Worked with Oracle since version 7.0 Have an affinity for things Perlish, such as DBD::Oracle Working as a DBA at Pythian since Jan 2011 Hobbies and extracurricular activities usually do not involve computers or databases. Contact: About this presentation We will explore advanced grouping functionality This presentation just skims the surface Truly understanding how to make use of advanced grouping you will need to invest some time experimenting with it and examining the results. 2
© 2009/2010 Pythian 3
Why talk about GROUP BY? Somewhat intimidating at first It seems to be underutilized The performance implications of GROUP BY are not often discussed 4
© 2009/2010 Pythian 5 GROUP BY Basics GROUP BY does not guarantee a 21:00:47 SQL> select /*+ gather_plan_statistics */ deptno, count(*) 21:00:48 2 from scott.emp 21:00:48 3 group by deptno 21:00:48 4 / DEPTNO COUNT(*) rows selected. Notice the execution plan step is HASH GROUP BY Inline views and/or Subfactored Queries may change results – best not to rely on that behavior. GROUP BY can be HASH or SORT – neither guarantees sorted output
© 2009/2010 Pythian 6 GROUP BY Basics GROUP BY is a SQL optimization Following does 4 full table scans of select /*+ gather_plan_statistics */ distinct dname, decode( d.deptno, 10, (select count(*) from scott.emp where deptno= 10), 20, (select count(*) from scott.emp where deptno= 20), 30, (select count(*) from scott.emp where deptno= 30), (select count(*) from scott.emp where deptno not in (10,20,30)) ) dept_count from (select distinct deptno from scott.emp) d join scott.dept d2 on d2.deptno = d.deptno; DNAME DEPT_COUNT SALES 6 ACCOUNTING 3 RESEARCH 5 3 rows selected.
© 2009/2010 Pythian GROUP BY Basics Use GROUP BY to reduce IO 1 full table scan of select /*+ gather_plan_statistics */ d.dname, count(empno) empcount from scott.emp e join scott.dept d on d.deptno = e.deptno group by d.dname order by d.dname; DNAME EMPCOUNT ACCOUNTING 3 RESEARCH 5 SALES 6 3 rows selected. 7
© 2009/2010 Pythian GROUP BY Basics – HAVING Not used as much as it once was – here’s why It is easily replaced by Subfactored Queries (ANSI CTE: Common Table Expressions ) select deptno,count(*) from scott.emp group by deptno having count(*) > 5; can be rewritten as: with gcount as ( select deptno,count(*) as dept_count from scott.emp group by deptno ) select * from gcount where dept_count > 5; 8
© 2009/2010 Pythian Advanced GB – CUBE() Used to generate cross tab type reports Generates all combinations of columns in with emps as ( select /*+ gather_plan_statistics */ ename, deptno from scott.emp group by cube(ename,deptno) ) select rownum, ename, deptno from emps 9
© 2009/2010 Pythian Advanced GB – CUBE() Notice the number of rows returned? 32 Notice the #rows the raw query actually returned. 56 in GENERATE CUBE in execution plan. Superaggregate rows generated by Oracle with NULL for GROUP BY columns– these NULLS represent the set of all values (see GROUPING() docs). Re-examine output for rows with NULL. For each row, Oracle generates a row with NULL for all columns in CUBE() All but one of these rows is filtered from output with the SORT GROUP BY step. Number of rows is predictable 10
© 2009/2010 Pythian Advanced GB – CUBE() Is CUBE() saving any work in the database? Without CUBE(), how would you do this? gb_6.sql – UNION ALL Notice the multiple TABLE ACCESS FULL steps CUBE() returned the same results with one TABLE scan 11
© 2009/2010 Pythian Advanced GB – CUBE() OK – so what good is it? Using the SALES example schema - Criteria: all sales data for the year sales summarized by product category, aggregates based on 10-year customer age ranges, income levels, summaries income level regardless of age group summaries by age group regardless of income Here’s one way to do 12
© 2009/2010 Pythian Advanced GB – CUBE() Use CUBE() to generate the same UNION ALL 8 seconds 9 table scans CUBE() 4 seconds 4 table scans 2 index scans 13
© 2009/2010 Pythian Advanced GB–Discern SA NULL Look at output from previous SQL – See all those NULLS on CUST_INCOME_LEVEL and AGE_RANGE How should you handle them? Can you use NVL() ? How will you discern between NULL data and Superaggregate Are all those NULL values generated as Superaggregate rows? 14
© 2009/2010 Pythian Advanced GB–GROUPING() Use GROUPING to discern - 0 = data null, 1 = SA null Use with DECODE() or CASE to determine – examine the use of GROUPING() Now we can see which is NULL data and which is SA NULL, and assign appropriate text for SA NULL - Put it to work in our Sales Report “ALL INCOME” and “ALL AGE” where sales are Aggregated on the income regardless of age, and age regardless of income. 15
© 2009/2010 Pythian Advanced GB–GROUPING_ID() GROUPING_ID() takes the idea behind GROUPING() up a notch GROUPING() returns 0 or 1 GROUPING_ID() evaluates expressions and returns a bit vector – arguments correspond to bit GROUPING_ID() generates the GID values GROUPING() illustrates binary bit OK – we made a truth table. What can we do with it? 16
© 2009/2010 Pythian Advanced GB–GROUPING_ID() Use GROUPING_ID() to customize sales report Useful for customizing report without any code change Summaries only Age Range only Income level + summaries etc… Options chosen by user are assigned values that correspond to bit vector used in – examine PL/SQL block Experiment with different values and check output What do you think will happen when all options=0? How would you create this report without advanced grouping? No, I did not write an example – too much work. 17
© 2009/2010 Pythian Advanced GB–ROLLUP() Similar to CUBE() for 1 argument ROLLUP() identical to for 1+N arguments ROLLUP produces fewer redundant 18
© 2009/2010 Pythian Advanced GB–ROLLUP() ROLLUP() – running subtotals without UNION ALL Much like CUBE(), ROLLUP() reduces the database workload Sales Report: All customers that begin with ‘Sul’ subtotal by year per customer subtotal by product category per customer grand 19
© 2009/2010 Pythian Advanced GB–GROUPING SETS Use with This looks just like the CUBE() output from gb_14b.sql But, now we can do things with GROUPING SETS that cannot easily be done with CUBE() Add “Country” to generated data Total by Country and ROLLUP(Region, 20
© 2009/2010 Pythian Advanced GB–GROUPING SETS Combine what has been covered into the sales Sometimes GROUPING SETS produces duplicate rows Last 2 lines of reports are duplicates In this case due to ROLLUP(PROD_CATEGORY) Use GROUP_ID() – its purpose is to distinguish duplicate rows caused by GROUP BY uncomment HAVING clause and rerun to see effect Performance Note: GROUPING SETS is better at reducing workload GROUPING_ID more flexible – no code changes 21
© 2009/2010 Pythian Advanced GROUP BY - Summary Greatly reduce database workload with Advance GROUP BY functionality Greatly reduce the amount of SQL to produce the same results There is a learning curve Start using it! 22
© 2009/2010 Pythian References Oracle 11g Documentation on advanced GROUP BY is quite good Pro Oracle SQL – Apress Advanced SQL Functions in Oracle 10g Functions-Oracle-10G/dp/ X Functions-Oracle-10G/dp/ X 23
© 2009/2010 Pythian 24 Grouping Glossary CUBE() GROUP_ID() GROUPING() GROUPING_ID() GROUPING_SETS() ROLLUP()
© 2009/2010 Pythian Glossary–SUPERAGGRETE ROW GROUP BY extension will generate rows that have a NULL value in place of the value of the column being operated on. The NULL represents the set of all values for that column. The GROUPING() and GROUPING_ID() functions can be used to distinguish these. 25
© 2009/2010 Pythian Glossary – CUBE() GROUP BY extension CUBE(expr1,expr2,…) returns all possible combination of columns passed Demo: gl_cube.sql 26
© 2009/2010 Pythian Glossary – GROUP_ID() Function GROUP_ID() Returns > 0 for duplicate rows Demo: gl_group_id.sql 27
© 2009/2010 Pythian Glossary – ROLLUP() GROUP BY extension ROLLUP(expr1, expr2,…) Creates summaries of GROUP BY expressions Demo: gl_rollup.sql 28
© 2009/2010 Pythian Glossary – GROUPING() Function GROUPING(expr) returns 1 for superaggregate rows returns 0 for non-superaggregate rows Demo: gl_rollup.sql Used in demo to order the rows 29
© 2009/2010 Pythian Glossary – GROUPING_ID() Function GROUPING_ID(expr) returns a number representing the GROUP BY level of a row Demo: gl_grouping_id.sql 30
© 2009/2010 Pythian Glossary – GROUPING SETS GROUP BY Extension GROUPING SETS( expr1, expr2,…) Used to create subtotals based on the expressions page Demo: gl_grouping_sets.sql 31
© 2009/2010 Pythian GROUP BY Bug Malformed GROUP BY statements that worked < may now get ORA-979 not a GROUP BY expression Due to bug # being fixed in Patch can be used to re-institute previous behavior ( must be patched offline – online mode patch is 32
©Brooks/Cole, 2001 Chapter 12 Derived Types-- Enumerated, Structure and Union.
1 Structured Query Language (SQL) Mail: Phone: YM: talim_bansal Blog:
12 Copyright © 2005, Oracle. All rights reserved. Query Rewrite.
10 Copyright © 2005, Oracle. All rights reserved. Dimensions.
1 Advanced SQL Topics Edward Wu. 2 Lesson Objectives Learn how to create and use indexes Create, Alter, and Drop Views Outer / Self Join Nested Queries.
INSERT BOOK COVER 1Copyright © 2011 Pearson Education, Inc. Publishing as Prentice Hall. Exploring Microsoft Office Excel 2010 by Robert Grauer, Keith.
PSSA Preparation. Question 1(no calculator) D Question 2 (no calculator)
A Guide to MySQL 4. 2 Objectives Retrieve data from a database using SQL commands Use compound conditions in queries Use computed columns in queries Use.
13 Copyright © 2005, Oracle. All rights reserved. Using the SQL Access Advisor.
FACTORING Think unfoil Work down, Show all steps ax 2 + bx + c.
8 Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Using Explicit Cursors.
© 2012 National Heart Foundation of Australia. Slide 2.
McDonald’s calendar 2009 January
1 Chapter 1 The Study of Body Function Image PowerPoint Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
25 seconds left….. 24 seconds left….. 23 seconds left…..
OvidSP Flexible. Innovative. Precise. Introducing OvidSP Resources.
Murach's PHP and MySQL, C15© 2010, Mike Murach & Associates, Inc.Slide 1.
3-1 4 Oracle Data Integrator First Project – Simple Transformations: One source, one target.
6 Copyright © 2009, Oracle. All rights reserved. Displaying Data from Multiple Tables Oracle SQL Programming: Orange Coast College.
Copyright © 2008 Cengage Learning Understanding Generalist Practice, 5e, Kirst-Ashman/Hull 1.
11 1 Chapter 11 Database Performance Tuning and Query Optimization Database Systems: Design, Implementation, and Management, Seventh Edition, Rob and Coronel.
Chapter 11 Membrane Structure Essential Cell Biology Third Edition Copyright © Garland Science 2010.
Chapter 15 Intracellular Compartments and Transport Essential Cell Biology Third Edition Copyright © Garland Science 2010.
9 Copyright © 2004, Oracle. All rights reserved. Using DDL Statements to Create and Manage Tables.
1 Displaying Open Purchase Orders (F/Y 11). 2 At the end of this course, you should be able to: –Run a Location specific report of all Open Purchase.
UNITED NATIONS Shipment Details Report – January 2006.
13 Copyright © 2005, Oracle. All rights reserved. Monitoring and Improving Performance.
FIRST COURSE Microsoft Access. XP 2 Organizing Data Your first step in organizing data is to identify the individual fields – The specific value, or content,
Januar 2005 S M T O T F L
1 Prof. Valter Bezerra Dantas
Factor P (8-5ab) 2. 4(d² + 4) 3. 3rs(2r – s) 4. 15cd(1 + 2cd) 5. 8(4a² + 3b²) 6. 12xy(3y – 4x) 7. 5x²y(6x + 7y) 8. 3cd²(3c² - 2d) 9. 15bc³(5b +
1 Copyright © 2011, Oracle and/or its affiliates. All rights reserved.
© Paradigm Publishing, Inc Access 2010 Level 2 Unit 2Advanced Reports, Access Tools, and Customizing Access Chapter 8Integrating Access Data.
Copyright © 2011, Elsevier Inc. All rights reserved. Chapter 2 Author: Julia Richards and R. Scott Hawley.
Talisma CRM © Campaign Overview Mailers Mailing Lists 1Proprietary and Confidential.
BMU - E I 1 Development of renewable energy sources in Germany in
Jeopardy Topic 1Topic Q 1Q 6Q 11Q 16Q 21 Q 2Q 7Q 12Q 17Q 22 Q 3Q 8Q 13Q 18Q 23 Q 4Q 9Q 14Q 19Q 24 Q 5Q 10Q 15Q 20Q 25 Final Jeopardy.
© 2010 Pearson Addison-Wesley. All rights reserved. Addison Wesley is an imprint of Chapter 11: Structure and Union Types Problem Solving & Program Design.
2010 fotografiert von Jürgen Roßberg © Fr 1 Sa 2 So 3 Mo 4 Di 5 Mi 6 Do 7 Fr 8 Sa 9 So 10 Mo 11 Di 12 Mi 13 Do 14 Fr 15 Sa 16 So 17 Mo 18 Di 19.
13 Copyright © Oracle Corporation, All rights reserved. Controlling User Access.
Haas MFE SAS Workshop Lecture 3: Peng Liu Haas School.
Contract Audit Follow-Up (CAFU) 3.5 Pre-Defined & Ad hoc Reports November 2009 ITCSO Training Academy.
11 Copyright © 2005, Oracle. All rights reserved. Refreshing Materialized Views.
Chapter 7 © 2013 Pearson Education, Inc. Publishing as Prentice Hall 1 Modern Database Management 11 th Edition Jeffrey A. Hoffer, V. Ramesh, Heikki Topi.
EU Market Situation for Eggs and Poultry Management Committee 21 June 2012.
Copyright © Cengage Learning. All rights reserved. 1 Functions and Applications.
© 2017 SlidePlayer.com Inc. All rights reserved.