1. Understanding Indexes
Craig Martin
10/18/2012
Understanding Indexes
We will take a look a level deeper than most people?s understanding of how indexes work. You will learn how to tell where an index will be most effective, the information that is really stored in an index, and some tricks to get indexes to do things you might not know were possible.

2. Understanding Indexes
An index is an optional structure, associated with a table or table cluster, that can sometimes speed data access.
- Oracle® Database Concepts
11g Release 2 (11.2)

3. Understanding Indexes
An index is an optional structure, associated with a table or table cluster, that can sometimes speed data access.
Indexes are independent data structures. They take up their own space. They can be added or dropped, and it has no effect on the table.
- Oracle® Database Concepts
11g Release 2 (11.2)

4. Optional Structure Index Types: B-tree indexes B-tree cluster indexes
Bitmap and bitmap join indexes
Application domain indexes
- B-tree indexes
These indexes are the standard index type. They are excellent for primary key and highly-selective indexes. Used as concatenated indexes, B-tree indexes can retrieve data sorted by the indexed columns.
- B-tree cluster indexes
This type of index is used to index a table cluster key. Instead of pointing to a row, the key points to the block that contains rows related to the cluster key.
- Bitmap and bitmap join indexes
In a bitmap index, an index entry uses a bitmap to point to multiple rows. In contrast, a B-tree index entry points to a single row. A bitmap join index is a bitmap index for the join of two or more tables. See "Bitmap Indexes".
- Application domain indexes
This type of index is created by a user for data in an application-specific domain. The physical index need not use a traditional index structure and can be stored either in the Oracle database as tables or externally as a file. See "Application Domain Indexes".

5. Optional Structure Index Types: B-tree indexes B-tree cluster indexes
Bitmap and bitmap join indexes
Application domain indexes
B-trees, short for balanced trees, are the most common type of database index.

6. Optional Structure
A B-tree index is an ordered list of values divided into ranges. A B-tree index has two types of blocks: branch blocks for searching and leaf blocks that store values. The upper-level branch blocks of a B-tree index contain index data that points to lower-level index blocks. Each of these entries points to a leaf block that contains key values that fall in the range.
A B-tree index is balanced because all leaf blocks automatically stay at the same depth. Thus, retrieval of any record from anywhere in the index takes approximately the same amount of time.

7. Optional Structure root block
The height of the index is the number of blocks required to go from the root block to a leaf block. The branch level is the height minus 1. This example index has a height of 3 and a branch level of 2.

8. Optional Structure
Branch blocks store the minimum key prefix needed to make a branching decision between two keys. This technique enables the database to fit as much data as possible on each branch block. The branch blocks contain a pointer to the child block containing the key. The number of keys and pointers is limited by the block size.
The leaf blocks contain every indexed data value and a corresponding rowid used to locate the actual row. Each entry is sorted by (key, rowid). Within a leaf block, a key and rowid is linked to its left and right sibling entries. The leaf blocks themselves are also doubly linked.

9. Understanding Indexes
An index is an optional structure, associated with a table or table cluster, that can sometimes speed data access.
The purpose of an index is to provide a more efficient access path to specific data
- Oracle® Database Concepts
11g Release 2 (11.2)

10. Data Access Broken into data blocks
Table Structure
Broken into data blocks
Multiple rows stored in single block
When row deleted, space left for future rows
Order not necessarily maintained
To read, start at beginning and continue until you hit high water mark
Typical block size is 8K. The number of rows that can fit on a single block depends on the number of columns, size of data in each column, the block size, and parameters like pctfree.

11. Data Access Root B-Tree Index Branch1 Branch2 Branch3 Leaf1 Leaf2
Index points to exact location of each row. Can be thought about like an Index of a book. The pages of the book are similar to the blocks of a table. The words (values) in the index tell you what page (block) each occurrence is located.

12. Data Access Rowid Globally Unique Identifier for a row
Contains Physical address of row in the table
Example: AAA0tPAAsAAAD44AAM
Format: OOOOOOFFFBBBBBBRRR
data object number = identifies database segment
datafile number = tablespace relative datafile
data block = datafile relative block number
row = row number in block
data object number
data block
datafile number
row

13. Data Access Types of Index Scans FULL INDEX SCAN FAST FULL INDEX SCAN
INDEX RANGE SCAN
INDEX UNIQUE SCAN
INDEX SKIP SCAN
FULL INDEX SCAN
The database reads the entire index in order
FAST FULL INDEX SCAN
Full index scan in which the database accesses the data in the index itself without accessing the table, and the database reads the index blocks in no particular order.
INDEX RANGE SCAN
Ordered scan of an index where more than one rowid could be returned
INDEX UNIQUE SCAN
Only 1 rowid could be returned
INDEX SKIP SCAN
Database "skips" through a single index as if it were searching separate indexes. Useful when non-leading column specified in predicate, and leading column has few distinct values.

14. Data Access
setup

15. Data Access
look at statistics for table and index

16. Data Access
31 consistent gets = 28 blocks + 3 for segment headers

17. Data Access
3 consistent gets = 1 root block + 1 leaf block + 1 data block

18. Data Access
3 consistent gets = 1 root block + 1 leaf block + 1 data block

19. Data Access
4 consistent gets = 1 root block + 1 leaf block + 2 data block

20. Data Access
How to tell that the values are on different data blocks

21. Data Access
4 consistent gets = 1 root block + 2 leaf block + 1 data block

22. Data Access
46 consistent gets = 1 root block + 20 leaf block + 25 data block

23. Data Access
46 consistent gets = 1 root block + 20 leaf block + 25 data block

24. Data Access
31 consistent gets = 28 data blocks + 3 for segment headers

25. Data Access
Full Table Scan constant, where Index gets worse as more rows selected

26. Data Access Advantages of Full Table Scan
Allows use of multiblock read
Index will read one block per I/O
Puts blocks at the end of the buffer cache, so they are the first blocks to age out
Index will put all blocks at the front of the buffer cache, which could cause frequently used blocks to get pushed out
Full table scans have their advantages so beneficial when a large number of blocks being selected.

27. Understanding Indexes
An index is an optional structure, associated with a table or table cluster, that can sometimes speed data access.
The data matters
- Oracle® Database Concepts
11g Release 2 (11.2)

28. Sometimes
setup

29. Sometimes
look at statistics for table and index

30. Sometimes
31 consistent gets = 28 data blocks + 3 for segment headers

31. Sometimes
31 consistent gets = 28 data blocks + 3 for segment headers

32. Sometimes
3 consistent gets = 1 root block + 1 leaf block + 1 data block

33. Sometimes
97 consistent gets = 1 root block + 1 leaf block + 95 visits to data blocks

34. Sometimes
9628 consistent gets = 1 root block + 20 leaf block visits to data blocks

35. Sometimes
The order of the data makes no difference in the Full Table Scan, but makes a huge difference in the efficiency of the index

36. Very much related is clustering factor
Very much related is clustering factor. Richard Foote, one of the worlds leading experts on indexes, has very clear explanation.

43. Sometimes
How do we know which columns make good candidates?

44. Sometimes
We can use rowid to tell!

45. Sometimes
We can use rowid to tell!

46. Understanding Indexes
Index Options

47. Composite Index id - val - rowid 1 - 2 - rowid1 1 - 3 - rowid2
A composite index contains multiple columns. The structure is exactly the same, except the leaf blocks store both columns values.

48. Composite Index create index index_name on table(col1, col2);
select col2, col1
from table
where col1 = 12;
col1 - col2 - rowid
rowid1
rowid2
rowid3
rowid4
rowid5
rowid6
rowid7
rowid8
rowid9
rowid10
rowid11
If a query can be completed using just the information stored in the index, Oracle will not even go to the table.

49. Function Based Index select * from objects
where upper(object_name) = 'DUAL';
create index obj_idx3 on
objects(UPPER(object_name));
But what if you need to use a function? For example if you have an application that needs to provide case insensitive matching? The simple solution is a function based index.

50. Function Based Index select object_name from objects
where upper(object_name) = 'DUAL';
Will query use the Objects table at all? Or can we get everything out of just the index? The answer is that we still need to hit the table.

51. Function Based Index
In a Function Based Index, values are computed and stored in index
TABLE
dual
Dual
DUAL
dUaL
INDEX
DUAL - rowid1
DUAL - rowid2
DUAL - rowid3
DUAL - rowid4
Only the computed values are stored, so to get the original value, you need to go back to table.

52. Usability
Indexes are usable (default) or unusable. An unusable index is not maintained by DML operations and is ignored by the optimizer. An unusable index can improve the performance of bulk loads. Instead of dropping an index and later re-creating it, you can make the index unusable and then rebuild it. Unusable indexes and index partitions do not consume space. When you make a usable index unusable, the database drops its index segment.

53. Visibility
Indexes are visible (default) or invisible. An invisible index is maintained by DML operations and is not used by default by the optimizer. Making an index invisible is an alternative to making it unusable or dropping it. Invisible indexes are especially useful for testing the removal of an index before dropping it or using indexes temporarily without affecting the overall application.

54. Compression
Oracle Database can use key compression to compress portions of the primary key column values in a B-tree index or an index-organized table. Key compression can greatly reduce the space consumed by the index

55. NOSORT
You can specify NOSORT to indicate to the database that the rows are already stored in the database in ascending order, so that Oracle Database does not have to sort the rows when creating the index. If the rows of the indexed column or columns are not stored in ascending order, then the database returns an error.

56. REVERSE
The bytes of the index key are reversed, for example, 103 is stored as 301. The reversal of bytes spreads out inserts into the index over many blocks. Prevents contention issues from hot blocks

57. DESC
Stores data on a particular column or columns in descending order

58. Understanding Indexes

59. Nulls and Indexes
If all columns in index are null, then Oracle does not index that row
If any column in index is not null, then Oracle will index the row
Indexes only contain rows where at least one value is not null. This is because typically you don't look for nulls, and not storing the rows saves space.

60. Nulls and Indexes select * from orders where processed = 'N' null 'Y'
We can use this knowledge to our advantage. If you have a scenario where you frequently just hit one value, you don't need to store all of the other values in the table. You could just make the values null instead. This will make for a much smaller and more efficient index.
'N'

61. Nulls and Indexes Combine this knowledge with Function Based Index:
create index not_processed_idx on orders (
case when processed = 'N' then processed end );
create index not_processed_idx2 on orders (
case when processed = 'N' then product_id end,
case when processed = 'N' then order_id end
Using a function based index, it is possible to index a subset of the rows. You can even index multiple columns for only a subset of values of another.

62. Nulls and Indexes select object_id from objects; select object_id
where object_id is not null;
select *
from objects
where owner_id not in (
select owner_id
from owners
where owner_id is not null );
select *
from objects
where owner_id not in (
select owner_id
from owners );
Downside is that you can't use index only in situations where you otherwise would, even if no nulls actually exist in the table

63. Nulls and Indexes alter table objects modify object_id not null;
alter table owners
modify owner_id not null;
Best solution is to alter the table to not allow nulls

64. Nulls and Indexes select * from objects where object_id is null;
create unique index obj_idx1 on objects(object_id);
When I first started, I fully expected the index to be used to solve this query

65. Nulls and Indexes
It was quickly apparent that the "is null" clause did not utilize the index.

66. Understanding Indexes
Index Tricks and Tips

67. Index Tips and Trips select * from objects where object_id is null;
Later, I learned that it actually is possible to create an index that will be used with this query.
create index obj_idx7 on objects(object_id, 1);

68. Index Tips and Trips
The execution plan shows this index is used to find null values.

69. Index Tips and Trips select * from objects where object_id = 12305;
Typically you would expect this situation to utilize the index
create unique index obj_idx1 on objects(object_id);

70. Index Tips and Trips
Execution plan shows Full Table Scan. Predicate information shows that Oracle is applying to_number function due to implicit conversion, because the object_id column is actually a VARCHAR.

71. Index Tips and Trips select * from objects where object_id = '12305';
When using the correct datatype, the index is used as expected.

72. Index Tips and Trips
Sometimes functions are used when they don't need to be
select *
from objects
where created_date >= trunc(sysdate)
and created_date < trunc(sysdate + 1);
select *
from objects
where trunc(created_date) = trunc(sysdate);
Frequently I see indexes not being utilized because functions are being used in SQL when there is really no need for them. Rewriting the query to not use the function allows for the index to be used.

73. Index Tips and Trips
The execution plan shows the FBI can be used to retrieve the data

74. Index Tips and Trips select * from objects
where upper(object_name) like 'DU%';
create index obj_idx3 on objects(UPPER(object_name));
Will the wildcard character cause the index to not be used?

75. Index Tips and Trips
No, the index is still used.

76. Index Tips and Trips select * from objects
where upper(object_name) like '%UAL';
What if the wildcard is in the beginning of the string?

77. Index Tips and Trips
In that case, the index will not be used

78. Index Tips and Trips create index obj_idx2 on objects(created_date);
select *
from objects
where last_updated_date = trunc(sysdate)
and created_date = trunc(sysdate);
create index obj_idx2 on objects(created_date);
An index is created on both columns in the where clause. Will both of these indexes be used?
create index obj_idx5 on objects(last_updated_date);

79. Index Tips and Trips
Only one of the indexes is used

80. Index Tips and Trips select * from objects where object_name = 'DUAL';
create index obj_idx8
on objects(owner, object_name);
select *
from objects
where object_name = 'DUAL';
We filter on the 2nd column in the index

81. Index Tips and Trips
Even though the column is part of the index, the index is not used

82. Index Tips and Trips select * from objects where object_name = 'DUAL';
create index obj_idx9
on objects(object_name, owner);
select *
from objects
where object_name = 'DUAL';
Now flip the order of the columns, so object_name is not the leading column in the index.

83. Index Tips and Trips
With the new order, Oracle uses the index

84. Index Tips and Trips col1 - col2 - rowid 1 - 2 - rowid1 1 - 3 - rowid2
Index sorted in order by first column, then second, etc. To find all the rows with a particular value in the second column, you would have to look across all the values for the first column which could exist on every block of the index, which is inefficient. Possible exception is an Index Skip Scan, which typically is only used if there are just a few distinct values for the first column.

85. References Oracle Documentation - Concepts Guide!
Oracle Index Internals - Rebuilding The Truth by Richard Foote
When to Use an Index - Cary Millsap

86. Questions?