DB-03: A Tour of the OpenEdge™ RDBMS Storage Architecture Richard Banville Technical Fellow

2© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture What’s New: Type II Storage Areas n Block clustering –Alleviates object fragmentation –Improves I/O efficiency n Concurrent Space Allocation –Area space Allocation –Object space allocation n Improved object manipulation –Table scan without index –Fast object deletion

3© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Agenda n Physical Layout n Advantages n A Usage Example n Summary

4© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Type II Area Block Clusters Block Cluster:  8, 64, or 512 adjacent blocks  Configured in.st file  Applied via prostrct  Fixed size for area  Unit of space allocation for objects  Blocks within cluster are “non-social”

5© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Type II Area Objects Object are made up of one or more block clusters (Objects: tables, indexes, lobs, area control)

6© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Type II Area Objects Clusters chained together for fast access (Objects: tables, indexes, LOBS, area control) Allows table scan without an index Allows fast table delete

7© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture What Else Is Different w/Type II n Area HWM increased a cluster at a time –More efficient block formatting –Concurrent Space Allocation n Database extend –MAX(64, cluster size) n Other –Reduced fragmentation and scatter –Allows more advanced tools to be designed

8© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Object Block (Type I Area): Space Allocation Chains Mixed Object Block Mixed Rec Block Mixed Rec Block Mixed Rec Block Mixed Rec Block Free Block Free Block Free Block Free Block Record Free Chain Idxdel Block Idxdel Block Idxdel Block Unique indexes Only Total blocks, HWM

9© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Object Block (Type II Area): Area Free Space Allocation Chains Area Object Block Free Block Free Block Free Block Free Block Area Control Object Total blocks, Cluster HWM Free Cluster List (for Area)

10© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Object Block (Type II Area): Space Allocation Chains Index Object Block Rec Free Block Rec Free Block Rec Free Block Rec Free Block Free Block Free Block Free Block Free Block Idxdel Block Idxdel Block Idxdel Block Unique indexes Only Table 2 Object Block Rec Free Block Rec Free Block Rec Free Block Rec Free Block Table 1 Object Block Total blocks, Cluster HWM Total blocks, Cluster HWM Total blocks, Cluster HWM

11© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture This Is Important Stuff n Indexes –Index delete chain maintenance –Index blocks are clustered together n Mixed Areas –Vast improvements multi table or multi index areas –Won’t have index and record blocks intertwined n Reduced fragmentation and scatter –Fewer Dump and Loads! n More efficient I/O

12© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Record Space Allocation n Record Packing Factor –Records stored variable length –Space allocated from record block free chains –RPF regulates record block free chain content n Definitions –Records per Block : (Blocksize / mean rec size) –Create limit: rec block free space required for a created record’s expansion. Also the minimum rec fragment size. (75/150) –Toss limit: free space required for a rec block to remain on the record block free chain (150/300)

13© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Changing RPF n Records Per Block –Changed via.st file –Granularity n Value per area n Create/Toss limits – _proutil C setTableTossLimit value –Granularity Values per area in Type I storage area Values per object in Type II storage area

14© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture RPF Suggestions n Change Create Limit if: –Fragmentation occurs due to record updates of newly created records –You expect to see 1 fragment but get 2 n Change Toss Limit if fragmentation occurs due to record updates of existing records –You expect 1 or 2 fragments but get 3 or 4 n Coordinate with RPB –(Blocksize / mean rec size) n Don’t change if you have no reason to

15© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Overall Type II Layout Area Control Object Object #1 Object #2 Cluster Free List Object #3 Area Data (Free & in use)

16© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Agenda n Physical Layout n Advantages n A Usage Example n Summary

17© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Other Advantages of Type II Storage Areas n Block level Check sum –Identifies corrupt blocks prior to data change –Allows for larger I/O in future n Storage –Rowids stored as 64 bits –Variable length block header size n Object information stored in block header –Allows for improved maintenance –Improves database repair operations

18© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Storage Management Advantages n Improvements through organization n Efficient block formatting –Cluster at a time –Fewer bi/ai notes written n Bottlenecks Resolved –Concurrent space allocation –Optimistic buffer and index locking protocols n Migration Path –Can use both Type I & Type II in same database

19© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Root Level 1 Level 2 Level 3 (Leaf) - Leaf entries contain pointer to record Records - Cursor maintains info or last key accessed Table Scan via B-tree

20© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Select * from Customer; - Cursor maintains info of last record accessed - I/O Sequential through cluster 2 nd Cluster 1 st Cluster 3 rd Cluster 4 th Cluster

21© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Fast Object Delete Area Control Object Object #1 Object #2 Cluster Free List Object #3 Area Data (Free & in use)

22© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Fast Object Delete Area Control Object Object #2 Cluster Free List Object #3 Area Data (Free & in use)

23© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Fast Object Delete Area Control Object Object #2 Cluster Free List Object #3 Area Data (Free & in use)

24© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture OpenEdge 10 Temp tables n Released in 10.0b –Fast delete –Fast delete/create on empty n Enhanced in 10.0b02 –Avoid delete/create –Avoid I/O when formatting n Hybrid Type I & II Storage Area –Index Objects Type I –Other Objects Type II –8 Block Clusters n Suggestions – “empty temp-table ” –-tmpbsize 1, -tmpbsize 8 –Better performance with increased –Bt

25© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Agenda n Physical Layout n Advantages n A Usage Example n Summary

26© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Best Practices for Use n Physical –Include Striping (RAID or do it yourself) –File extent Location n Schema –Separate index and table data –Multi table area for small, medium & large records –Records per block properly set for each area n Growth –Always have a variable length extent –Enable large files

27© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Location, Location, Location b /bi/exampleDB.b1 f b /bi/exampleDB.b2 f b /bi/exampleDB.b3 # d "Schema Area":6,64 /db/exampleDB.d1 # d “Customer Indexes":7,1;8 /db/exampleDB_7.d1 f d “Customer Indexes":7,1;8 /db/exampleDB_7.d2 # d “Customer Data":8,128;64 /db/exampleDB_8.d1 f d “Customer Data":8,128;64 /db/exampleDB_8.d2

28© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Cluster Size b /bi/exampleDB.b1 f b /bi/exampleDB.b2 f b /bi/exampleDB.b3 # d "Schema Area":6,64 /db/exampleDB.d1 # 8 d “Customer Indexes":7,1;8 /db/exampleDB_7.d1 f d “Customer Indexes":7,1;8 /db/exampleDB_7.d2 # 64 d “Customer Data":8,128;64 /db/exampleDB_8.d1 f d “Customer Data":8,128;64 /db/exampleDB_8.d2

29© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Records Per Block b /bi/exampleDB.b1 f b /bi/exampleDB.b2 f b /bi/exampleDB.b3 # d "Schema Area":6,64 /db/exampleDB.d1 # 1 d “Customer Indexes":7,1;8 /db/exampleDB_7.d1 f d “Customer Indexes":7,1;8 /db/exampleDB_7.d2 # 128 d “Customer Data":8,128;64 /db/exampleDB_8.d1 f d “Customer Data":8,128;64 /db/exampleDB_8.d2

30© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Multi Object Areas d “Large Record Indexes":9,1;8 /db/exampleDB_9.d1 f d "Large Record Indexes":9,1;8 /db/exampleDB_9.d2 # d “Large Record Tables":10,16;64 /db/exampleDB_10.d1 f d “Large Record Tables":10,16;64 /db/exampleDB_10.d2 # d “Small Record Indexes":11,1;8 /db/exampleDB_11.d1 f d “Small Record Indexes":11,1;8 /db/exampleDB_11.d2 # d “Small Record Tables":12,256;64 /db/exampleDB_12.d1 f d “Small Record Tables":12,256;64 /db/exampleDB_12.d2

31© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Cluster Size: Fast Growing Tables 64 d “Misc Indexes":13,1;64 /db/exampleDB_13.d1 f d “Misc Indexes":13,1;64 /db/exampleDB_13.d2 # 512 d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d1 f d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d2 f d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d3 # a /ai/exampleDB.a1 f a /ai/exampleDB.a2 f a /ai/exampleDB.a3 f …

32© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Records Per Block: Fast Growing Tables 1 d “Misc Indexes":13,1;64 /db/exampleDB_13.d1 f d “Misc Indexes":13,1;64 /db/exampleDB_13.d2 # 64 d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d1 f d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d2 f d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d3 # a /ai/exampleDB.a1 f a /ai/exampleDB.a2 f a /ai/exampleDB.a3 f …

33© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture AI File Location d “Misc Indexes":13,1;64 /db/exampleDB_13.d1 f d “Misc Indexes":13,1;64 /db/exampleDB_13.d2 # d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d1 f d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d2 f d “Fast Growing Tables":14,64;512 /db/exampleDB_14.d3 # a /ai/exampleDB.a1 f a /ai/exampleDB.a2 f a /ai/exampleDB.a3 f …

34© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture In Summary n More efficient layout n Better Performance n Migration path n Foundation for the future

35© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Questions?

36© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture Thank you for your time!

37© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture

38© 2005 Progress Software Corporation DB-03: The OpenEdge™ RDBMS Storage Architecture OpenEdge 10 RDBMS Advanced Storage Architecture The following Progress courses cover related subject matter. Please visit: for course descriptions and relevant curriculum maps. n Database Administration