Download presentation

Presentation is loading. Please wait.

Published byYuliana Alley Modified over 2 years ago

1
SQL Top-N and Pagination Pattern Maxym Kharchenko

2
What is top-N Give me the top 10 salaries in the Sales dept Give me the top 10 best selling books Give me the 10 latest orders

3
What is top-N

4
Setup cities Name Null? Type NAME NOT NULL VARCHAR2(100) STATE NOT NULL VARCHAR2(100) POPULATION NOT NULL NUMBER PCTFREE 99 PCTUSED 1

5
Naïve Top-N SELECT name, population FROM cities WHERE rownum <= 5 ORDER BY population DESC; NAME Pop Robertsdale city 5,276 Glen Allen town (pt.) 458 Boligee town 328 Riverview town 184 Altoona town (pt.) 30 Give me the top 5 cities by population Statistics 7 consistent gets

6
Naïve Top-N explained | Id | Operation | Name | Rows | Bytes | Time | | 0 | SELECT STATEMENT | | 5 | 110 | 00:00:01 | | 1 | SORT ORDER BY | | 5 | 110 | 00:00:01 | |* 2 | COUNT STOPKEY | | | | | | 3 | TABLE ACCESS FULL| CITIES | 10 | 220 | 00:00:01 |

7
Correct top-N query SELECT name, population FROM cities ORDER BY population DESC FETCH FIRST 5 ROWS ONLY; SELECT * FROM ( SELECT name, population FROM cities ORDER BY population DESC ) WHERE rownum <= 5; >= 12c <= 11g

8
Correct top-N query: Execution SELECT * FROM ( SELECT name, population FROM cities ORDER BY population DESC ) WHERE rownum <= 5; NAME Pop Los Angeles city 3,792,621 Chicago city (pt.) 2,695,598 Chicago city 2,695,598 New York city (pt.) 2,504,700 Statistics consistent gets

9
Reading, filtering and sorting | Id | Operation | Name | Rows |TempSpc| Time | | 0 | SELECT STATEMENT | | 5 | | 00:01:58 | |* 1 | COUNT STOPKEY | | | | | | 2 | VIEW | | | | 00:01:58 | |* 3 | SORT ORDER BY STOPKEY| | | 1768K| 00:01:58 | | 4 | TABLE ACCESS FULL | CITIES | | | 00:01:54 |

10
Proper data structure | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 5 | 00:00:01 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 10 | 00:00:01 | | 3 | TABLE ACCESS BY INDEX ROWID | CITIES | | 00:00:01 | | 4 | INDEX RANGE SCAN DESCENDING| I_POP | 10 | 00:00:01 | Ordered By: Population Statistics 12 consistent gets CREATE INDEX i_pop ON cities(population);

11
Why index works Colocation Can stop after reading N rows No Sort Ordered By: Population CREATE INDEX i_pop ON cities(population);

12
More elaborate top-N SELECT * FROM ( SELECT name, population FROM cities WHERE state='Florida' ORDER BY population DESC ) WHERE rownum <= 5; NAME Pop Jacksonville city 821,784 Miami city 399,457 Tampa city 335,709 St. Petersburg city 244,769 Orlando city 238,300 Give me the top 5 cities by population in Florida Statistics 264 consistent gets

13
Uncertain nature of filtering WHERE state='Florida' ORDER BY population DESC ) WHERE rownum <= 5; WHERE state='Florida' ORDER BY population DESC ) WHERE rownum <= 200; Ordered By: Population Statistics consistent gets Statistics 264 consistent gets

14
Multi column indexes State Population AKALCOFLAZ Ordered By: State State+Population Not Ordered by: Population WHERE state=FL NOW: Ordered By: Population MAWA where state=FL CREATE INDEX i_state_pop ON cities(state, population);

15
Multicolumn indexes | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 5 | 00:00:01 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 11 | 00:00:01 | | 3 | TABLE ACCESS BY INDEX ROWID | CITIES | 1099 | 00:00:01 | |* 4 | INDEX RANGE SCAN DESCENDING| I_STATE_POP | 11 | 00:00:01 | Predicate Information (identified by operation id): 1 - filter(ROWNUM<=5) 4 - access("STATE"='Florida') Statistics 12 consistent gets

16
Trips to the table | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 5 | 00:00:01 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 11 | 00:00:01 | | 3 | TABLE ACCESS BY INDEX ROWID | CITIES | 1099 | 00:00:01 | |* 4 | INDEX RANGE SCAN DESCENDING| I_STATE_POP | 11 | 00:00:01 | Predicate Information (identified by operation id): 1 - filter(ROWNUM<=5) 4 - access("STATE"='Florida') Statistics 12 consistent gets

17
Index range scan: cost math Window: 500 records 4-5 logical reads ~ 5-10 logical reads ~ logical reads

18
Covering index CREATE INDEX i_state_pop_c ON cities (state, population, name); CREATE INDEX i_state_pop ON cities (state, population); | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 5 | 00:00:01 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 10 | 00:00:01 | |* 3 | INDEX RANGE SCAN DESCENDING| I_STATE_POP_C | 506 | 00:00:01 | Statistics 12 consistent gets Statistics 7 consistent gets

19
Ideal top-N Use the index Make the best index And read only from the index

20
Less than ideal top-N Effect of query conditions Effect of deletes and updates Technicalities

21
Condition better! WHERE active != 'N' ORDER BY order_date DESC ) WHERE rownum <= 10; WHERE active = 'Y' ORDER BY order_date DESC ) WHERE rownum <= 10; Statistics consistent gets Statistics 10 consistent gets CREATE TABLE orders ( … active char(1) NOT NULL CHECK (active IN ('Y', 'N'))

22
Trade WHERE for ORDER BY SELECT * FROM (SELECT * FROM t WHERE a=12 ORDER BY c) ) WHERE rownum <= 10; WHERE a=12 ORDER BY c WHERE a=12 ORDER BY b, c WHERE a=12 AND b=0 ORDER BY c CREATE INDEX t_idx ON t(a, b, c); Statistics 1200 consistent gets Statistics 12 consistent gets Statistics 12 consistent gets

23
Tolerate filtering SELECT * FROM ( SELECT name, population FROM cities WHERE state != 'Florida' ORDER BY population DESC ) WHERE rownum <= 10; Statistics 28 consistent gets

24
Tolerate filtering | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 11 | 00:00:01 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 11 | 00:00:01 | |* 3 | TABLE ACCESS BY INDEX ROWID | CITIES | | 00:00:01 | | 4 | INDEX RANGE SCAN DESCENDING| I_POP | 12 | 00:00:01 | Predicate Information (identified by operation id): filter(ROWNUM<=10) 3 - filter("STATE"<>'Florida')

25
Updates and Deletes cities2 Name Null? Type NAME NOT NULL VARCHAR2(100) STATE NOT NULL VARCHAR2(100) POPULATION NOT NULL NUMBER BUDGET_SURPLUS NOT NULL VARCHAR2(1) CREATE INDEX i2_pop ON cities2(budget_surplus, population, name);

26
Updates and Deletes SELECT * FROM ( SELECT name, population FROM cities2 WHERE budget_surplus='Y' ORDER BY population DESC ) WHERE rownum <= 5 ; | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 5 | 00:00:01 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 12 | 00:00:01 | |* 3 | INDEX RANGE SCAN DESCENDING| I2_POP | | 00:00:01 | Statistics 7 consistent gets

27
Updates and Deletes UPDATE cities2 SET budget_surplus='N' WHERE rowid IN ( SELECT * FROM ( SELECT rowid FROM cities2 ORDER BY population DESC ) WHERE rownum <= 200); | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 5 | 00:00:01 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 12 | 00:00:01 | |* 3 | INDEX RANGE SCAN DESCENDING| I2_POP | | 00:00:01 | Statistics 207 consistent gets

28
Updates and Deletes

29
ALTER TABLE cities2 ADD (version number default 0 NOT NULL); CREATE INDEX i2_vpop ON cities2(budget_surplus, version, population); UPDATE cities2 SET version=1 WHERE budget_surplus='Y' AND version=0; 0 Y 1 Budget_surplus Version Population Y Y Budget_surplus

30
Updates and Deletes | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 1 | 00:00:01 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 1 | 00:00:01 | |* 3 | INDEX RANGE SCAN DESCENDING| I2_VPOP | 1 | 00:00:01 | Statistics 9 consistent gets SELECT * FROM ( SELECT name, population FROM cities2 WHERE budget_surplus='Y' AND version=1 ORDER BY population DESC ) WHERE rownum <= 5;

31
Pagination SELECT * FROM ( SELECT name, population FROM cities WHERE state='Florida' ORDER BY population DESC ) WHERE rownum <= 10; SELECT * FROM ( SELECT name, population, rownum AS rn FROM cities WHERE state='Florida' ORDER BY population DESC ) WHERE rownum <= 20 ) WHERE rn > 10;

32
Dumb Pagination ) WHERE rownum <= 20 ) WHERE rn > 10; Statistics 22 consistent gets ) WHERE rownum <= 30 ) WHERE rn > 20; Statistics 32 consistent gets

33
Smart pagination SELECT * FROM ( SELECT name, population FROM cities WHERE state='Florida' AND population < ORDER BY population DESC ) WHERE rownum <= 10; SELECT * FROM ( SELECT name, population, rownum AS rn FROM cities WHERE state='Florida' ORDER BY population DESC ) WHERE rownum <= 20 ) WHERE rn > 10; Statistics 22 consistent gets Statistics 12 consistent gets

34
Top-N with joins SELECT * FROM ( SELECT c.name as city, c.population, s.capital FROM cities c, states s WHERE c.state_id = s.id AND c.state='Florida' ORDER BY c.population DESC ) WHERE rownum <= 5 / state population state_id name Filter Order By Join Select Driving table: Joined to table: id Join capital Select Use Nested Loops!Build indexes like this!

35
Top-N with joins: Good | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 5 | 00:00:13 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 10 | 00:00:13 | | 3 | NESTED LOOPS | | 10 | 00:00:13 | |* 4 | INDEX RANGE SCAN| I_C | 506 | 00:00:07 | |* 5 | INDEX RANGE SCAN| I_S | 1 | 00:00:01 |

36
Top-N with joins: Bad | Id | Operation | Name | Rows | Time | | 0 | SELECT STATEMENT | | 5 | 00:00:07 | |* 1 | COUNT STOPKEY | | | | | 2 | VIEW | | 10 | 00:00:07 | |* 3 | SORT ORDER BY STOPKEY| | 10 | 00:00:07 | |* 4 | HASH JOIN | | 10 | 00:00:07 | |* 5 | INDEX RANGE SCAN | I_C | 506 | 00:00:07 | |* 6 | INDEX RANGE SCAN | I_S | 1 | 00:00:01 |

37
Gotchas? TMI Too many indexes

38
Thank you!

39
Query conditions WHERE state = 'Florida ' State Population AKALCOFLAZMAWA where state=FL WHERE state != 'Florida ' where state != FL

40
AKALFLMAWA …… GAHI Watch out for DESC/ASC WHERE state >= 'Florida' ORDER BY state, population DESC ) WHERE rownum <= 10 WHERE state >= 'Florida' ORDER BY state, population ) WHERE rownum <= 10 Statistics 12 consistent gets Statistics consistent gets CREATE INDEX i_s_pop ON cities(state, population); + SORTNO SORT

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google