Presentation is loading. Please wait.

Presentation is loading. Please wait.

SQL on Fire! Part 1 Tips and Tricks around SQL. Agenda  Part I  SQL vs. SQL PL example  Error handling  Tables out of nowhere  Pivoting  Aggregation.

Similar presentations


Presentation on theme: "SQL on Fire! Part 1 Tips and Tricks around SQL. Agenda  Part I  SQL vs. SQL PL example  Error handling  Tables out of nowhere  Pivoting  Aggregation."— Presentation transcript:

1 SQL on Fire! Part 1 Tips and Tricks around SQL

2 Agenda  Part I  SQL vs. SQL PL example  Error handling  Tables out of nowhere  Pivoting  Aggregation  Deleting duplicate rows  Alternatives to Min and Max  Part II  Mass deletes  Order processing  Moving rows between tables  Recursion  Merge  Calculating nesting levels Easy Difficult

3 Motivation – The OLTP Mantra  Reduce Codepath  Reduce Logical and Physical I/O  Reduce Network Traffic  Minimize Lock Contention  Avoid Deadlocks High performance starts with the application.

4 SQL vs. SQL PL - Example  DDL CREATE TABLE emp(empID INTEGER NOT NULL salary INTEGER, name VARCHAR(20), deptID INTEGER); CREATE UNIQUE INDEX empIdxPK ON emp(empID) INCLUDE (deptID); ALTER TABLE emp ADD PRIMARY KEY (empID);

5 SQL vs. SQL PL EmpIDSalaryNameDeptID Jones Wang Zhang Smith Kim Chen Jou Yung Hall Zhu Xia200

6 SQL vs. SQL PL  Business request: Give everyone in deptID 100 a 5% raise.

7 SQL vs. SQL PL – First Attempt DECLARE empcur CURSOR FOR SELECT * FROM emp FOR UPDATE; OPEN empcur; emploop: LOOP FETCH empcur INTO vempid, vsalary, vname, vdeptid; IF SQLSTATE = ‘02000’ THEN LEAVE emploop; END IF; IF deptid = 100 THEN SET vsalary = vsalary * 1.05; UPDATE emp SET salary = vsalary WHERE CURRENT OF empcur; END IF; END LOOP emploop; CLOSE empcur; Read whole table Only need few rows

8 SQL vs. SQL PL - Second Attempt DECLARE empcur CURSOR FOR SELECT * FROM emp WHERE deptID = 100 FOR UPDATE; OPEN empcur; emploop: LOOP FETCH empcur INTO vempid, vsalary, vname, vdeptid; SET vsalary = vsalary * 1.05; IF SQLSTATE = ‘02000’ THEN LEAVE emploop; END IF; UPDATE emp SET salary = vsalary WHERE CURRENT OF empcur; END LOOP emploop; CLOSE empcur; Retrieve all columns Use only one column

9 SQL vs. SQL PL - Third Attempt DECLARE empcur CURSOR FOR SELECT salary FROM emp WHERE deptID = 100 FOR UPDATE; OPEN empcur; emploop: LOOP FETCH empcur INTO vsalary; SET vsalary = vsalary * 1.05; IF SQLSTATE = ‘02000’ THEN LEAVE emploop; END IF; UPDATE emp SET salary = vsalary WHERE CURRENT OF empcur; END LOOP emploop; CLOSE empcur; Use cursor Trivial logic

10 SQL vs. SQL PL - Final Attempt UPDATE emp SET salary = salary * 1.05 WHERE deptID = 100; Tips: WHERE deptID IN (100, 200) WHERE deptID IN (SELECT deptID FROM dept WHERE deptlocation = ‘Shenzhen’) SET salary = salary * CASE WHEN name = ‘Zhang’ THEN 1.05 ELSE 1.03 END

11 SQL vs. SQL PL – Aggregation  Return all departments with more than 5 employees: SELECT deptID FROM emp GROUP BY deptID HAVING COUNT(*) > 5  Return the salary of the most highly paid employee by department for departments with more than 5 employees: SELECT deptID, MAX(salary) FROM emp GROUP BY deptID HAVING COUNT(*) > 5

12 Create table if it does not exist BEGIN DECLARE CONTINUE HANDLER FOR SQLSTATE '42710' BEGIN /* Already exists? Ignore */ END; CREATE TABLE T(c1 INT); END Tip 1: There is no need to avoid SQL errors as long as they are handled… Tip 2:.. except when statement rollback implies undo operation from the log

13 SQL PL – DDL cleanup BEGIN DECLARE CONTINUE HANDLER FOR SQLSTATE '42704' BEGIN /* Does not exist? Ignore */ END; DROP TABLE T1; DROP TABLE T2; DROP SEQUENCE S1; END Tip: Lost errors are very hard to find. So be specific and never ignore generic SQLEXCEPTION.

14 SQL PL – Is this table empty? IF EXISTS(SELECT 1 FROM emp WHERE dept = 100) THEN.. END IF; Instead of: IF EXISTS(SELECT 1 FROM emp WHERE empid = 5) THEN UPDATE emp SET salary = salary * 1.05 WHERE empID = 5; END IF; Use: UPDATE emp SET salary = salary * 1.05 WHERE empID = 5; Tip: Don’t be afraid of NOTFOUND (SQLSTATE ‘02000’). NOT FOUND not an error

15 Tables from nowhere  Create tables without INSERT: VALUES (1, 2), (3, 4), …. (10, 11) or VALUES (CAST(? AS INTEGER), CAST(? AS DOUBLE)), (?, ?), … (?, ?)  Use anywhere like INSERT INTO T VALUES (1, 2), (3, 4), (5, 6)

16 VALUES for multi row INSERT CREATE TABLE T(c1 INT, c2 VARCHAR(10)); INSERT INTO T VALUES (?, ?), (?, ?), (?, ?), … (?, ?); Tip: For mass inserts prepare INSERT statements with: 1, 10, 100, 1000, rows of parameter markers each. Execute the biggest that fits the remaining load in a loop.

17 PIVOT and UNPIVOT YearQuarterResults CREATE TABLE Sales(Year INTEGER, Quarter INTEGER, Results INTEGER);

18 PIVOT and UNPIVOT SalesQ1Q2Q3Q CREATE TABLE SalesAgg(year INTEGER, q1 INTEGER, q2 INTEGER, q3 INTEGER, q4 INTEGER);

19 PIVOT SELECT Year, MAX(CASE WHEN Quarter = 1 THEN Results END) AS Q1, MAX(CASE WHEN Quarter = 2 THEN Results END) AS Q2, MAX(CASE WHEN Quarter = 3 THEN Results END) AS Q3, MAX(CASE WHEN Quarter = 4 THEN Results END) AS Q4 FROM Sales GROUP BY Year Tip: Use MAX() because it is supported for all comparable types including strings

20 PIVOT Access Plan: RETURN ( 1) | GRPBY ( 2) | FETCH ( 3) / \ IXSCAN TABLE: SALES ( 4) | INDEX: SALESIDX

21 UNPIVOT SELECT Year, Quarter, Results FROM SalesAgg AS S, LATERAL(VALUES(1, S.q1), (2, S.q2), (3, S.q3), (4, S.q4)) AS Q(Quarter, Results); Tip: Use LATERAL (or TABLE) to allow correlation of S.q* to the left.

22 UNPIVOT Access Plan: RETURN ( 1) | NLJOIN ( 2) / \ TBSCAN TBSCAN ( 3) ( 4) | | TABLE: SALESAGG TABFNC: GENROW

23 Aggregation  Problem: DB2 does not support user defined aggregates  Thoughts: –XMLAGG() provides aggregation without loss of data –Use mathematical rules  Examples: –Aggregate concatenation –Aggregate multiplication

24 XML functions – a primer  XMLAGG() –Aggregates XML values into an XML sequence  XMLELEMENT() –Tags a scalar value and returns XML e.g. XMLELEMENT(NAME ‘x’ 5) => 5  XMLSERIALIZE() –Casts an XML value into a string

25 Aggregate concatenation CREATE TABLE Employee(name VARCHAR(15), dept VARCHAR(15)); NameDept MisoSolutions JohnDevelopment SergeSolutions LeeL3 MarkID JackL3 LilyQuality BerniSolutions

26 Aggregate concatenation DeptNames SolutionsBerni, Miso, Serge DevelopmentJohn L3Jack, Lee IDMark QualityLily

27 Aggregate concatenation SELECT Dept, SUBSTR(Names, 1, LENGTH(names) -1) FROM (SELECT Dept, REPLACE (REPLACE (XMLSERIALIZE (CONTENT XMLAGG(XMLELEMENT(NAME a, name) ORDER BY name) AS VARCHAR(60)), ' ', ''), ' ', ',') AS Names FROM Employee GROUP BY Dept) AS X; Strip last comma Replace end tags with commas Strip start tags XML to VARCHAR Aggregate in order of names Tag a name to become XML

28 Aggregate multiplication CREATE TABLE probabilities(modelVARCHAR(10), event VARCHAR(10), percent FLOAT); ModelEventPercent Boing 737Engine Airbus 320Engine Boing 737Engine Airbus 320Engine Airbus 320Fin0.002 Boing 737Fin0.0018

29 Aggregate multiplication  a = EXP(LOG(a)) X* Y = EXP(LOG(X * Y)) X * Y = EXP(LOG(X) + LOG(Y))  PROD(X i ) i=1..n = EXP(SUM(LOG(X i ) i=1..n )

30 Aggregate multiplication SELECT model, DEC(EXP(SUM(LOG(percent))), 8, 7) AS percent FROM probabilities WHERE event IN (‘Engine 1’, ‘Fin’) GROUP BY model ModelPercent Boing Airbus

31 Retrieving MAXimum row CREATE TABLE emp(name VARCHAR(10), dept VARCHAR(10), salary INTEGER); CREATE INDEX emp_ind ON emp(dept, salary DESC);  Standard using selfjoin: SELECT name, salary FROM emp WHERE salary = (SELECT MAX(salary) FROM emp WHERE dept = ‘SQL Compiler’ AND dept = ‘SQL Compiler’);

32 Retrieving MAXimum row SELECT name, salary FROM emp WHERE dept = 'SQL Compiler' ORDER BY salary DESC FETCH FIRST ROW ONLY; NameDeptSalary FrankRewrite20000 HarrySQL Compiler18000 JanetSQL Compiler19000 GwenSQL Compiler22000 JasonOptimizer21000

33 Retrieving MAXimum row Access Plan: RETURN ( 1) | FETCH ( 2) / \ IXSCAN TABLE: EMP ( 3) | INDEX: EMP_IND

34 Delete duplicate rows CREATE TABLE Inventory(Item INTEGER, Quantity INTEGER, InvDate DATE); CREATE UNIQUE INDEX InvIdx ON Inventory(Item ASC, InvDate DESC);

35 Delete duplicate rows ItemQuantityInvDate Oct Nov Nov Oct Nov Dec Oct Nov Dec

36 Delete duplicate rows ItemQuantityInvDate Nov Nov Dec Nov Dec

37 Delete duplicate rows - classic DELETE FROM Inventory AS D WHERE (Item, InvDate) NOT IN (SELECT Item, InvDate FROM Inventory I WHERE D.Item = I.Item ORDER BY InvDate DESC FETCH FIRST ROW ONLY);

38 Delete duplicate rows - improved  Using OLAP DELETE FROM (SELECT row_number() OVER(PARTITION BY Item ORDER BY InvDate DESC) AS rn FROM Inventory) WHERE rn > 1; Tip: Remove ORDER BY clause if it doesn’t matter which duplicates get eliminated.

39 Delete duplicate rows Access Plan: RETURN ( 1) | DELETE ( 2) / \ FETCH TABLE: INVENTORY ( 3) / \ FILTER TABLE: INVENTORY ( 4) | IXSCAN ( 5) | INDEX: INVIDX

40 Conclusion  Exploit SQL to:  Increase concurrency  Reduce I/O  Reduce code-path  Make the application more readable  SQL provides powerful support Tip: Part II with even meaner examples after the break

41 Serge Rielau IBM SQL on Fire! Part 1


Download ppt "SQL on Fire! Part 1 Tips and Tricks around SQL. Agenda  Part I  SQL vs. SQL PL example  Error handling  Tables out of nowhere  Pivoting  Aggregation."

Similar presentations


Ads by Google