1. DB-08: A Day in the Life of a Type II Record Richard Banville Progress Fellow

2. 2© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Agenda n Update a record n Details

3. 3© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Major Shared-Memory Structures Buffer pool (-B) Lock Table (-L) BI Buffers (-bibufs)AI Buffers (-aibufs) Transaction Table (-n) Usrctl (-n) Srvctl (-Mn) Object Cache (-omcache)

4. 4© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record DO TRANSACTION: FIND customer WHERE cust.name = “Dopey”. cust.name = “Smarty” cust.city = “Orlando” cust.state = “FL”. END. A Simple Program Begin Transaction. Find Index Entry via chosen index Lock & Retrieve record

5. 5© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record DO TRANSACTION: FIND customer WHERE cust.name = “Dopey”. cust.name = “Smarty” cust.city = “Orlando” cust.state = “FL”. END. A Simple Program Update record buffer w/changed fields Make any index changes Save record changes

6. 6© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record DO TRANSACTION: FIND customer WHERE cust.name = “Dopey”. cust.name = “Smarty” cust.city = “Orlando” cust.state = “FL”. END. A Simple Program Commit the updates Release or downgrade locks

7. 7© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Details Part I Getting the Record Begin Transaction. Find Index Entry via chosen index Lock & Retrieve record

8. 8© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Start Transaction n Acquire a transaction entry n Begin note deferred Transaction Table (-n) Usrctl (-n) User Id 9Trans Id 2433405

9. 9© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Finding the Record n Lookup record by area/rowid n Rowid stored in index –Associates known value with rowid –Know values are ordered within index n Language picks index –Choice based on statement n For fast access –have index that matches predicate

10. 10© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Index Entry n An index entry consists of two parts: –key (one or more components) –rowid of the record containing the key Key Component(s) Name Rec-Id Bashful 256 Doc 3341 Dopey 9794 Grumpy 9806 Happy 9876 Sleepy 7776 Sneezy 512 Key Component(s) Cust-Num Rec-Id 100 512 101 7776 102 3341 103 9876 104 9794 105 9806 106 256

11. 11© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... B-tree Layout Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... root internal leaf Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... records Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Need Area & Root Block to start traversal (Name Index)

12. 12© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Disk Storage Files Storage Areas Schema AreaArea 7Area 8 Extent Table A Table BCust TblTable D Index A-1 Index B-1Name Idx Cust# IdxSRep IdxIndex D-1Index D-2 Logical Physical Languages Storage Engine

13. 13© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Disk Storage Files Storage Areas and Object Mapping Schema AreaArea 7Area 8 Extent Table A Table B Cust Tbl Table D Index A-1 Index B-1 Name IdxCust# IdxSRep IdxIndex D-1Index D-2 Logical Physical Mapping Languages Storage Engine

14. 14© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Object Mapping Cache replacing Hash Table LRU Chain “oldest” “newest” aging Area 6Area 7Area 8Area 9 T1I1T3I3T2I2T4I4 Array of cache entries (unordered) Each entry contains area info & root block

15. 15© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Performance (omlru simplified) replacing Hash Table LRU Chain “oldest” “newest” aging Omlru avoidance when –omsize proper Area 6Area 7Area 8Area 9 T1I1T3I3T2I2T4I4 Find I2/T2 Find I1/T1 Find I3/T3 Find I2 …Find I1/T1

16. 16© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record We know the Index’s area We know the rowid of the index root block Let’s find it so we can traverse the index

17. 17© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Find Buffer in Buffer Pool replacing Hash Table (-hash) LRU Chain “oldest” “newest” aging MMMM MMMM Checkpoint Queue MMMM Page Writer Queue database Array of (–B) buffer pool entries (unordered) May need to flush data on page-in (if APW not @ peak performance)

18. 18© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Effect of -Bp replacing Hash Table (-hash) LRU Chain “oldest” “newest” aging MMMM User 1 –Bp LRU Chain User 2 –Bp LRU Chain

19. 19© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Find the “Dopey” Entry Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... root internal leaf Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... records Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter TopReserved Free Space o o o... Compressed Index Entries... BotInde x No. Num Entries Bytes Used... Compressed Index Entries... Dummy Entry... Compression effects number of blocks traversed

20. 20© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Free List Active & Queued (Waiting) Lock Entries Hash Table (-lkhash) The Lock Table (-L) X X SQ Row-id Table-id Owner Lock State Next Pointer Lock Table Entry S S X X Need to share lock the record (even though the record wasn’t located yet!) (9794) (1) (Shared) (User Id 9)

21. 21© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record We know the Record’s rowid In the interest of time, assume we’ve done the object mapping and attempted the lookup in the buffer pool Let’s go get the record

22. 22© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Reading a Record from Disk * 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 10 0 0 0 Block numbers 1 to 8,388,608 Row numbers 0 to 255 ROWID prostrct list : d “Customer":8,64 x_7.d1 f 384 d “Customer":8,64 x_7.d2 f 3200 d “Customer":8,64 x_7.d3 Looking for ROWID 9794, Table 1, Area 8 Find _Area where _Area-number = 8. Display _Area-recbits _Area-Blocksize. 6 8192

23. 23© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Reading a Record from Disk Find _Area where _Area-number = 8. Display _Area-recbits _Area-Blocksize. prostrct list : d “Customer":8,64 x_7.d1 f 384 d “Customer":8,64 x_7.d2 f 3200 d “Customer":8,64 x_7.d3 Looking for ROWID 9794, Table 1, Area 8 6 8192 * 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 Block numbers 1 to 33,554,432 Row numbers 0 to 63 ROWID 9794 * 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 1 1 0 0 1 0 0 0 0 1 0 Block: 153 Row: 2

24. 24© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Get the Record Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter Free Space Free Dirs. Rec 0 OffsetRec 1 Offset Rec 2 OffsetRec n Offset Num Dirs. Free Space Used Data Space Record 0 Record 1 Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter Free Space Free Dirs. Rec 0 OffsetRec 1 Offset Rec 2 OffsetRec n Offset Num Dirs. Free Space Used Data Space Record 0 Record 1 Language Record Buffer Find the record and assemble all the pieces. Rowid 9794 Rowid 2828 Row 2 in Block 153 of area 8

25. 25© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Details Part II Perform Updates Update record buffer w/changed fields Make any index changes Save record changes

26. 26© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Update Fields in Record Buffer n New row is larger than original –Field values stored variable size n Need more space –We can handle it new original Language Record Buffer State Name

27. 27© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Free List (-L) Active & Queued (Waiting) Lock Entries Hash Table (-lkhash) Upgrade the Lock to EX X X SQ Row-id Table-id Owner Lock State Next Pointer Lock Table Entry S S X X (9794) (1) S S

28. 28© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Free List (-L) Active & Queued (Waiting) Lock Entries Hash Table (-lkhash) Lock Conflict! X X SQ S S X X XQ Queued entry - We must wait (Watch out for deadlock!) (-lkwtmo)

29. 29© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Free List (-L) Active & Queued (Waiting) Lock Entries Hash Table (-lkhash) Now We Got It X X SQ S S X X XQ X X

30. 30© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Key Component(s) Name Rec-Id Bashful 256 Doc 3341 Dopey 9794 Grumpy 9806 Happy 9876 Sleepy 7776 Sneezy 512 Non-unique Key Updating an Index Entry Key Component(s) Name Rec-Id Bashful 256 Doc 3341 Dopey 9794 Grumpy 9806 Happy 9876 Sleepy 7776 Smarty 9794 Sneezy 512 Non-unique Key “Dopey” goes “Smarty” Key Component(s) Name Rec-Id Bashful 256 Doc 3341 Dopey -2433405 Grumpy 9806 Happy 9876 Sleepy 7776 Smarty 9794 Sneezy 512 Unique Key

31. 31© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Update the Record Blocks Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter Free Space Free Dirs. Rec 0 OffsetRec 1 Offset Rec 2 OffsetRec n Offset Num Dirs. Free Space Used Data Space Record 0 Record 1 Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter Free Space Free Dirs. Rec 0 OffsetRec 1 Offset Rec 2 OffsetRec n Offset Num Dirs. Free Space Used Data Space Record 0 Record 1 Language Record Buffer All buffer pool actions replayed Rowid 9794 Rowid 2828 Reallocate space fragment by fragment OOPs, out of space!

32. 32© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Need Space? 200 300 400 200 Record block free chain for Customer table Cust Object Block Cust Object Block 4K 2K 8K Record block free chain for Order table Order Object Block Order Object Block Objects are selfish with their space allocation chains

33. 33© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Need Space? 200 300 400 200 Record block free chain for Customer table Cust Object Block Cust Object Block Blocks on RM free chain governed by Record Packing Factor Records/Block, Create Limit (75/150), Toss Limit (150/300) Blocks checked, recycled or tossed is limited (internal boundaries) Example: Need 200 bytes Toss RecycleTake/ Toss

34. 34© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Need Space? 200 300 400 200 Record block free chain for Customer table Cust Object Block Cust Object Block Blocks on RM free chain governed by Record Packing Factor Records/Block, Create Limit (75/150), Toss Limit (150/300) Blocks checked, recycled or tossed is limited (internal boundaries) Example: Need 300 bytes Toss Recycle

35. 35© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Cluster Size Matters 200 300 400 200 Record block free chain for Customer Table Cust Object Block Cust Object Block 8K Area Control Object Area Control Object 8K Area Cluster Free Chain

36. 36© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Cluster Size Matters 200 300 400 200 Record block free chain for Customer Table Cust Object Block Cust Object Block 8K Area Control Object Area Control Object 8K Area Cluster Free Chain Cluster free chain empty? Raise area HWM or extend area.

37. 37© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Connect All Fragments Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter Free Space Free Dirs. Rec 0 OffsetRec 1 Offset Rec 2 OffsetRec n Offset Num Dirs. Free Space Used Data Space Record 0 Record 1 Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter Free Space Free Dirs. Rec 0 OffsetRec 1 Offset Rec 2 OffsetRec n Offset Num Dirs. Free Space Used Data Space Record 0 Record 1 Language Record Buffer Rowid 9794 Rowid 2828 Block’s DBKEYTypeChainBackup Ctr Next DBKEY in ChainBlock Update Counter Free Space Free Dirs. Rec 0 OffsetRec 1 Offset Rec 2 OffsetRec n Offset Num Dirs. Free Space Used Data Space Record 0 Record 1 Rowid 10240

38. 38© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Details Part III Make it real Commit the updates Release or downgrade locks

39. 39© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record End Transaction Transaction Table (-n) Usrctl (-n) User Id 9 Trans Id 2433405 Release Transaction Entry Write Commit Note/do commit processing

40. 40© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record S S S S EQ Free List Active & Queued (Waiting) Lock Entries Hash Table (-lkhash) Release/Downgrade Locks X X SQ X X Awaken users waiting on lock Release exclusive lock SQ EQ SQ X X EQ

41. 41© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record In Summary n YOU: Simple Update n WE: Lots of work n You can Enhance the Process

42. 42© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Questions?

43. 43© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record Thank you for your time!

44. 44© 2005 Progress Software Corporation DB-08: A Day in the Life of a Type II Record