Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Index Internals Julian Dyke Independent Consultant Web Version juliandyke.com © 2005 Julian Dyke.

Similar presentations


Presentation on theme: "1 Index Internals Julian Dyke Independent Consultant Web Version juliandyke.com © 2005 Julian Dyke."— Presentation transcript:

1 1 Index Internals Julian Dyke Independent Consultant Web Version juliandyke.com © 2005 Julian Dyke

2 juliandyke.com 2 Agenda 1.Introduction 2.Block Structure 3.Block Compression 4.Insertion 5.Deletion 6.Coalesce / Rebuild 7.Freelists 8.Virtual Indexes

3 © 2005 Julian Dyke juliandyke.com 3 B*Tree Indexes  Based on modified B*Tree algorithm  Contain branch blocks and leaf blocks  Blocks contain keys and data  Keys maintained in sorted order within blocks  All leaf blocks are at the same depth  All blocks are on average 75% full

4 © 2005 Julian Dyke juliandyke.com 4 Index Types  There are several recent variants of B*tree indexes including TypeIntroduced Bitmap Indexes7.3.2 Index Organised Table8.0 Partitioned Indexes8.0 Reverse Key8.0 LOB Index8.0 Compressed8.1.5 Function-Based Indexes8.1.5 Descending8.1.5 Virtual Indexes8.1.5 Bitmap Join Indexes9.0.1

5 © 2005 Julian Dyke juliandyke.com 5 Limits  Maximum number of B*tree levels is 24  Maximum number of columns is 16 in 7.3 and below; 32 in 8.0 and above  Maximum key lengths vary with release and block size Block Size8.1.79.0.19.2.0 204875815261478 4096157831663118 8192321864466398 1638464981300612958

6 © 2005 Julian Dyke juliandyke.com 6 Leaf Blocks  Every index has a least one leaf block  Each leaf block contains 0 or more rows  Each row contains a key and data  Indexes can be unique or non-unique  Leaf row formats differ for unique and non- unique indexes

7 © 2005 Julian Dyke juliandyke.com 7 Leaf Block Structure 20 bytes 72 bytes 16 bytes 2 bytes per row 4 bytes Block Common Header Transaction Header Index Header Index Leaf Header Index Leaf Rows Tail Free Space Slot Array Block Size 2 bytes

8 © 2005 Julian Dyke juliandyke.com 8 Branch Blocks  Indexes may contain branch blocks  Branch blocks point to other branch blocks or leaf blocks  Branch blocks contain 0 or more rows  Each row has a suffix compressed key and a pointer to the next block  Compressed rows are terminated with 0xFE byte

9 © 2005 Julian Dyke juliandyke.com 9 Branch Block Structure 20 bytesBlock Common Header Transaction Header Index Header Index Branch Header Index Branch Rows Tail Free Space Slot Array 48 bytes 16 bytes 24 bytes 2 bytes per row 4 bytes Block Size 2 bytes

10 © 2005 Julian Dyke juliandyke.com 10 Branch Blocks  Each block has a pointer to the left hand side of the tree. This is part of the header  A branch block containing N rows points to N+1 blocks. S DEU DENENG SCO SPA USAAUS BEL CAN Branch Blocks Root Block Leaf BlocksLevel 0 Level 1 Level 2

11 © 2005 Julian Dyke juliandyke.com 11 Root Block  Every index has one root block  May be a leaf block or a branch block  Can be an empty leaf block  Always the next block after the segment header in the first extent Segment Header Root Block ….. First Extent

12 © 2005 Julian Dyke juliandyke.com 12 BLEVEL versus Height  BLEVEL is the number of branch block levels in the B*tree ANALYZE INDEX i1 COMPUTE STATISTICS; SELECT blevel FROM dba_indexes WHERE index_name = 'I1';  Height is the total number of levels in the B*tree ANALYZE INDEX i1 VALIDATE STRUCTURE; SELECT height FROM index_stats;  Height = BLEVEL + 1

13 © 2005 Julian Dyke juliandyke.com 13 Internal versus External ROWIDs  Internal ROWIDs are used in branch blocks Always 4 bytes.  External ROWIDs are used in leaf blocks BytesDescription 6Local Indexes 8Cluster Indexes 10Global Indexes 32LOB Indexes  In 7.3.4 and below all ROWIDs are 4 bytes  IOTs do not use external ROWIDs

14 © 2005 Julian Dyke juliandyke.com 14 Leaf Rows ROWIDColumn 1Flag Byte Length Byte(s) Lock Byte Length Byte(s) Column 2 Unique Index ROWIDColumn 1Flag Byte Length Byte(s) Lock Byte Length Byte(s) Column 2Length Byte Non-Unique Index  Unique indexes use one byte per row less than non-unique indexes  Each column has one length byte (< 128 bytes); two length bytes otherwise

15 © 2005 Julian Dyke juliandyke.com 15 Non Unique versus Unique Indexes SELECT c01,c02 FROM t1 WHERE c01 IN ( SELECT c01 FROM t2 ); SELECT STATEMENT MERGE JOIN SORT JOIN TABLE ACCESS (FULL) 'T1' SORT JOIN VIEW OF 'VW_NSO_1' SORT (UNIQUE) TABLE ACCESS (FULL) 'T2' SELECT STATEMENT NESTED LOOPS TABLE ACCESS (FULL) 'T1' INDEX (UNIQUE SCAN) 'I2' CREATE INDEX i2 ON t2 (c01); CREATE UNIQUE INDEX i2 ON t2 (c01); Non Unique Index Unique Index

16 © 2005 Julian Dyke juliandyke.com 16 Non Unique Leaf Rows  All leaf rows are stored in sorted order  For non-unique indexes the ROWID is appended to the key to create a unique key  Keys must be effectively unique so that updates can traverse the B*tree directly to the affected leaf block without requiring a scan  For concatenated indexes allows range scans of prefix columns

17 © 2005 Julian Dyke juliandyke.com 17 Non-Unique Leaf Rows Y Y Y Y Y 01 41 E9 A5 00 01 01 41 E9 A5 00 02 01 41 E9 A6 00 00 01 41 E9 A6 00 01 01 41 E9 A6 00 02 Y Y Y Y Y 01 41 E9 A7 00 00 01 41 E9 A7 00 01 01 41 E9 A7 00 02 01 41 E9 A8 00 00 01 41 E9 A8 00 01 Y Y Y Y Y 01 41 E9 A8 00 02 01 41 E9 A9 00 00 01 41 E9 A9 00 01 01 41 E9 A9 00 02 01 41 E9 AA 00 00 01 41 E9 A8 00 02Y 01 41 E9 A7Y  For non-unique indexes ROWIDs may be stored in branch blocks  ROWIDs are suffix compressed where possible

18 © 2005 Julian Dyke juliandyke.com 18 Branch Block Compression  Branch block rows are suffix compressed  Number of branch blocks in an index is determined by  Length of index key and data  Number of leaf blocks  Uniqueness of the leading edge of the key  Number of branch blocks affects index height

19 © 2005 Julian Dyke juliandyke.com 19 Branch Block Compression 000009999990 00009999990 0009999990 000999999 00999999 0999999 999999  A single column 1000000 row CHAR(N) index Key Format Prefix Length Key Length Index Height Branch Blocks Leaf Blocks 0638210990 1739211495 28310312196 39411712821 410412513334 511414414085 612415714706 999999 000000 Constant prefix Monotonic values 000000 to 999999

20 © 2005 Julian Dyke juliandyke.com 20 Branch Block Compression  A single column 1000000 row CHAR(12) index Key Format Prefix Length Key Length Index Height Branch Blocks Leaf Blocks 012310814706 112311614706 212312414706 312413414706 412414114706 512414914706 612415714706 000009999990 999999 000000 Constant prefix 999999XXXXXX 0999999XXXXX Monotonic values 000000 to 999999 XXXXXX Filler XXXX00999999 XXX000999999 0009999990XX X00009999990

21 © 2005 Julian Dyke juliandyke.com 21 Leaf Block Compression  In Oracle 8.1.5 and above, index leaf blocks can be compressed e.g. CREATE INDEX i1 ON t1 (c01,c02,c03) COMPRESS 2;  Each leaf row is split into a prefix and a suffix  The number of columns in the prefix is specified using the COMPRESS clause  Repeating prefix columns within the block are held once in a prefix row  Suffix row has pointer to prefix row  Prefix row has implicit pointers to suffix rows

22 © 2005 Julian Dyke juliandyke.com 22 Compressed Leaf Block Structure Prefix Slot Array Free Space Prefix rows and Suffix rows Suffix Slot Array Tail 20 bytes 72 bytes 16 bytes 2 bytes per row 4 bytes 2 bytes 4 bytes per row Block Common Header Index Leaf Header Index Header Transaction Header Block Size Compression Header 4 bytes

23 © 2005 Julian Dyke juliandyke.com 23 Leaf Block Compression CountryCity FranceParis GermanyBerlin GermanyFrankfurt GermanyMunich ItalyMilan ItalyRome Suffix Slot Array Prefix Slot Array 16001200130014009001000 1500 1700 11001 4 6 1700 1500 1400 1300 1200 1100 1600 1000 900 France Paris Germany Berlin Frankfurt Munich Italy Milan Rome Prefix Row Suffix Row

24 © 2005 Julian Dyke juliandyke.com 24 Leaf Block Compression  Example – single column index Number of Rows Number of Distinct Keys Number of Leaf Blocks (Compressed) Number of Leaf Blocks (Uncompressed) 10000168104 100001068104 1000010069104 10000100075104 10000200083104 10000250086104 100005000105104 10000 141104 CREATE TABLE (c01 CHAR(5)); CREATE INDEX i1 ON t1 (c01) COMPRESS 1;

25 © 2005 Julian Dyke juliandyke.com 25 Insertion Inserting 'ENG','SCO' and 'USA' BeforeAfter Index initially has single empty leaf block at root. Rows are inserted into leaf block ENG SCO USA

26 © 2005 Julian Dyke juliandyke.com 26 Insertion Inserting 'BEL' BeforeAfter ENG SCO USA Root node is now a branch block. Two new leaf nodes created. Leaf rows split between the two new blocks USABEL ENG SCO U

27 © 2005 Julian Dyke juliandyke.com 27 Insertion Inserting 'SPA' BeforeAfter USABEL ENG SCO U SCO SPA USABEL ENG SU New leaf node created. Leaf rows split between the two leaf nodes. New leaf block pointer added to branch block

28 © 2005 Julian Dyke juliandyke.com 28 Insertion Inserting 'AUS' New row added to leaf block. No other blocks affected BeforeAfter SCO SPA USABEL ENG SU SCO SPA USAAUS BEL ENG SU

29 © 2005 Julian Dyke juliandyke.com 29 Insertion Inserting 'CAN' New leaf node created. Leaf rows split between the two leaf nodes. New leaf block pointer added to branch block BeforeAfter SCO SPA USAAUS BEL ENG SU ENG SCO SPA USAAUS BEL CAN ESU

30 © 2005 Julian Dyke juliandyke.com 30 Insertion Inserting 'DEN' New leaf node created. Branch block now full so new branch blocks created. Index now has three levels BeforeAfter S DEU DENENG SCO SPA USAAUS BEL CAN ENG SCO SPA USAAUS BEL CAN ESU

31 © 2005 Julian Dyke juliandyke.com 31 PCTFREE  Only applies to CREATE INDEX statement.  Not used for subsequent inserts CREATE TABLE t1 (c01 NUMBER); Insert 100000 rows CREATE INDEX i1 ON t1 (c01) PCTFREE 50 CREATE TABLE t1 (c01 NUMBER); CREATE INDEX i1 ON t1 (c01) PCTFREE 50; Insert 100000 rows 100000Leaf rows100000 1910Leaf blocks858 1588891Leaf rows length1588891 1909Branch rows857 13Branch blocks7 22738Branch rows length10172

32 © 2005 Julian Dyke juliandyke.com 32 Deletion Deleting 'CAN' Deleted flag is set for leaf row. Leaf block free space count is adjusted. No effect on other blocks ENG S DEU DEN SCO SPA USAAUS BEL CAN S DEU DENENG SCO SPA USAAUS BEL CAN BeforeAfter

33 © 2005 Julian Dyke juliandyke.com 33 Deletion Deleting 'ENG' Deleted flag is set for leaf row. Leaf block free space count is adjusted. Leaf block pointer is not deleted from branch ENG S DE DEN SCO SPA USA U AUS BEL CAN S U DENENG SCO SPA USA ED AUS BEL CAN BeforeAfter

34 © 2005 Julian Dyke juliandyke.com 34 Deletion Inserting 'FRA' Leaf block is reused. Leaf row added to leaf block. No other blocks affected FRA S DEU DEN SCO SPA USAAUS BEL CAN S DEU USA ENGDEN SCO SPA AUS BEL CAN BeforeAfter

35 © 2005 Julian Dyke juliandyke.com 35 Coalesce Before S DEU DENENG SCO SPA USAAUS BEL CAN After S DEN ENG SCO SPA USA AUS BEL CAN D ALTER INDEX index1 COALESCE; Leaf rows are merged in leaf blocks. Branch blocks are updated where necessary

36 © 2005 Julian Dyke juliandyke.com 36 Rebuild Before S DEU DEN ENG SCO SPA USAAUS BEL CAN After ALTER INDEX index1 REBUILD; DEN ENG SCO SPA USA AUS BEL CAN D SP Index is rebuilt in a new segment. All blocks are modified

37 © 2005 Julian Dyke juliandyke.com 37 Freelists  Index blocks can be returned to the freelist after all rows have been deleted  Empty index blocks can be recycled  Leaf blocks may be reused as branch blocks and vice versa  Blocks will not be reused until existing blocks on the freelist have been used  Empty blocks are initially returned to the transaction freelist. They are subsequently moved to the master freelist.

38 © 2005 Julian Dyke juliandyke.com 38 Freelists Deleting all rows from blocks 723 to 726 BeforeAfter 726730727725724723 728729 722 726730727725724723 728729 722 Blocks are inserted in transaction freelist. Branch block is not modified in case rows are re-inserted again Master Freelist Transaction Freelist 723724 725 726 730 Master Freelist Transaction Freelist 730731

39 © 2005 Julian Dyke juliandyke.com 39 Freelists Inserting new rows BeforeAfter Block 730 is full and is removed from master freelist. New rows added to block 731 726730727725724723728729722 Master Freelist Transaction Freelist 723724 725 726 730 726731727725724723728729722 Master Freelist Transaction Freelist 723724 725 726 730 731

40 © 2005 Julian Dyke juliandyke.com 40 Freelists Inserting new rows BeforeAfter Block 731 is full and is removed from master freelist. Blocks 723 to 726 moved from transaction freelist to master freelist. New rows added to block 726 Master Freelist Transaction Freelist 727726730725724723728729722731 Master Freelist Transaction Freelist 723724 725 726 731727725724723728729722730 731 726725 724 723

41 © 2005 Julian Dyke juliandyke.com 41 Freelists Inserting new rows BeforeAfter Block 726 is full and is removed from master freelist. New rows added to block 725 Leaf block 724 becomes a new branch block Master Freelist Transaction Freelist 727726730725724723728729722731 725724 723 Master Freelist Transaction Freelist 727726730725724723728729722731 726725 724 723

42 © 2005 Julian Dyke juliandyke.com 42 Freelists Inserting new rows BeforeAfter Blocks 725 and 724 are removed from master freelist. Branch block 728 is moved to transaction freelist New rows added to block 723 727726730725 724 723 729 722 731 Master Freelist Transaction Freelist 728 723 727726730725 724 723 728729 722 731 Master Freelist Transaction Freelist 725724 723

43 © 2005 Julian Dyke juliandyke.com 43 Freelists Inserting new rows BeforeAfter Block 723 is full and is removed from master freelist. Branch block 728 becomes a leaf block New rows added to block 728 727726730725724723729722731 Master Freelist Transaction Freelist 728 727726730725724723729722731 Master Freelist Transaction Freelist 728 723 728

44 © 2005 Julian Dyke juliandyke.com 44 Virtual Indexes  In 8.1.5 and above it is possible to create virtual indexes  Virtual indexes have a data dictionary definition, but no associated segment  Effectiveness of new indexes can be tested by generating theoretical execution plans  The CBO will consider virtual indexes if the hidden parameter _use_nosegment_indexes is set to true

45 © 2005 Julian Dyke juliandyke.com 45 Virtual Indexes CREATE TABLE t1 AS SELECT * FROM dba_objects WHERE ROWNUM < 1000; ANALYZE TABLE t1 COMPUTE STATISTICS;  Consider the following analysed table CREATE INDEX i1 ON t1 (owner, object_name) NOSEGMENT; EXECUTE DBMS_STATS.GENERATE_STATS (USER,'I1');  A virtual index can be created using the NOSEGMENT keyword  Statistics must be generated for the new index based on the existing statistics for the table

46 © 2005 Julian Dyke juliandyke.com 46 Virtual Indexes SELECT object_id FROM t1 WHERE owner = USER AND object_name = 'T1'; ALTER SESSION SET "_use_nosegment_indexes" = TRUE; SELECT STATEMENT Optimizer = CHOOSE TABLE ACCESS (FULL) OF 'T1' SELECT STATEMENT Optimizer = CHOOSE TABLE ACCESS (BY INDEX ROWID) INDEX (RANGE SCAN) OF 'I1'  The statement  generates the plan  the same statement generates a different plan  With the hidden parameter enabled

47 © 2005 Julian Dyke juliandyke.com 47 Thank you for your interest For more information and to provide feedback please contact me My e-mail address is: info@juliandyke.com My website address is: www.juliandyke.com


Download ppt "1 Index Internals Julian Dyke Independent Consultant Web Version juliandyke.com © 2005 Julian Dyke."

Similar presentations


Ads by Google