OpenEdge Standard Storage Areas Standards are like toothbrushes – everybody agrees you should have one, but no one wants to use yours. -- unknown

A Few Words about the Speaker Tom Bascom; Progress user & roaming DBA since 1987 Partner, White Star Software, LLC Expert consulting services related to all aspects of Progress and OpenEdge. tom@wss.com Partner, DBAppraise, LLC Remote database management service for OpenEdge. Simplifying the job of managing and monitoring the world’s best business applications. tom@dbappraise.com I have been working with Progress since 1987 … and today I am both President of DBAppraise; The remote database management service… where we simplify the job of managing and monitoring the worlds best business applications; and Vice President of White Star Software; where we offer expert consulting services covering all aspects of Progress and OpenEdge.

Who is White Star Software? The oldest and most respected independent OpenEdge DBA consulting firm in the world! The world’s top independent OpenEdge DBAs! Author of ProTop, the #1 FREE OpenEdge Database Monitoring Tool: http://dbappraise.com/protop.html

Standards ISO Standard “Standards” Comic http://xkcd.com/927

Progress’ CCS Common Component Specification: https://community.progress.com/products/directions/common_component/f/208 Key Point: These are community driven efforts! Progress participates and supports but does not dictate. Last night…

The Schema Area Thou Shalt Not Put User Data In The Schema Area! BEST PRACTICES INVOLVING THE SCHEMA AREA http://knowledgebase.progress.com/articles/Article/P106007

The Problem The default Storage Area for tables, indexes and LOBs is the Schema Area. It is also the ONLY area available in a default, empty database. Progress advises that the Schema Area not be used to store user tables, indexes and LOBs. It should only contain system data! All of the advanced features of the OpenEdge database rely upon type 2 storage areas. Without specific guidance nearly all new applications, conversions, upgrades and migrations leave the user objects in the schema area. The only advice from Progress is that no user objects (tables, indexes or large objects) should be assigned to the schema area. Yet, by default, the only area available is the schema area and, by default, the schema area is the area that newly created objects are assigned to. As a result, nearly all new applications, conversions, upgrades and migrations leave the user objects in the schema area. All of the advanced features of the OpenEdge database rely upon type 2 storage areas. The schema area is always a type 1 area and objects in the schema area are, therefore, unable to participate in these advanced features.

Great! Now what? Currently application partners and other developers have no guidance to help them determine an appropriate set of storage areas for their applications. An explicit set of recommended minimum storage areas will help partners and developers to easily assign their storage objects to appropriate storage areas. This, in turn, will position their applications to be able to immediately leverage advanced features of the database without painful post-deployment maintenance (such as dumping and re-loading). A side effect is that revenue opportunities for Progress and partners will thus be more easily realized because a barrier to adoption has been removed!

Kbase Articles, PUG Talks, Videos, Forums… There are dozens – maybe even hundreds of excellent sources for great advice on how to create good type 2 storage areas! Programmers, architects and part-time DBAs are overwhelmed So decisions don’t get made, nothing is done and yet another database gets created with everything in the schema area 

Kbase Articles, PUG Talks, Videos, Forums… There are dozens – maybe even hundreds of excellent sources for great advice on how to create good type 2 storage areas! Programmers, architects and part-time DBAs are overwhelmed So decisions don’t get made, nothing is done and yet another database gets created with everything in the schema area  Which is an excellent source of consulting opportunities!

Type 2 Storage Areas Always use Type 2 storage areas (areas with 8, 64 or 512 blocks per cluster are Type 2 areas). Never put any objects (tables, indexes, or LOBs) in the Schema Area. Do not mix objects of different types (e.g. tables and indexes) in the same storage area: Create a storage area for tables. Create a storage area for indexes. Create a separate storage areas for LOBs, if any. Create a storage area for the data of tables with word indexes, if any. Create a storage area for each very large table. For each of those, create a storage area for the indexes of that table.

That’s Nice But… It still requires users to act!

We Need A There should be a Standard for This! Currently application partners and other developers have no guidance to help them determine an appropriate set of storage areas for their applications. An explicit set of recommended minimum storage areas will help partners and developers to easily assign their storage objects to appropriate storage areas. This, in turn, will position their applications to be able to immediately leverage advanced features of the database without painful post-deployment maintenance (such as dumping and re-loading). A side effect is that revenue opportunities for Progress and partners will thus be more easily realized because a barrier to adoption has been removed!

Project Proposal Define a minimum collection of Storage Areas for an application using an OpenEdge database

The Solution An explicit set of recommended minimum storage areas. That will help partners and developers to easily assign their storage objects to appropriate storage areas. While remaining flexible and extensible for more advanced customers.

Solution Constraints The “out of the box” solution MUST NOT: require coding changes! degrade performance significantly increase disk space usage require other parameter or tuning changes require end user decision making prevent end users from adding their own storage areas

Solution Benefits Greatly increases db scalability at no cost Will position applications to be able to immediately leverage advanced features of the database Without painful post-deployment maintenance (such as dumping and re-loading) While remaining flexible and extensible for more advanced customers

Deliverables Structure file defining the recommended Storage Areas White paper describing: the areas and how they should be used how to safely extend the areas for advanced users scripts for converting existing schema-only database definitions to the new spec

Option 1 A “one size fits all” storage area: d "data":10,128;8 .

Option 2 Discrete data and index storage areas: d "data":10,128;8 . d "indexes":11,128;8 .

Option 3 Discrete data, index and LOB storage areas: d "data":10,128;8 . d "indexes":11,128;8 . d "largeObjects":12,128;512 .

Option 4 Discrete data, index, LOB and word-index data storage areas: d "data":10,128;8 . d "indexes":11,128;8 . d "largeObjects":12,128;512 . d "wordIndexData":13,128;8 . http://knowledgebase.progress.com/articles/Article/000030594 -datascanthreads <n>: the number of concurrent threads used to scan table data associated with the indexes being rebuilt. When specified, -datascanthreads invokes a multi-threaded data scan that spawns <n> threads for the data scan phase. The data being scanned, must meet all of the following criteria for a multi-threaded data scan to occur: - The data area being scanned must be a Type II area - No index within the data area being scanned is also being rebuilt - No index associated with the data area being scanned is a word index - Index rebuild must be run with the "sort" option, that is answering "y" when asked if you have enough room for sorting. If any requirement is not met, the data scan is performed using the original single threaded mechanism. If not specified, the –datascanthreads value defaults to 0, also indicating the original single threaded mechanism.

Option 5 (“gilding the lily”) Discrete data, index, LOB, word-index data and “B2” storage areas: d "data":10,128;8 . d "indexes":11,128;8 . d "largeObjects":12,128;512 . d "wordIndexData":13,128;8 . d "b2Data":14,128;8 . d "b2Indexes":15,128;8 . http://knowledgebase.progress.com/articles/Article/000030594 -datascanthreads <n>: the number of concurrent threads used to scan table data associated with the indexes being rebuilt. When specified, -datascanthreads invokes a multi-threaded data scan that spawns <n> threads for the data scan phase. The data being scanned, must meet all of the following criteria for a multi-threaded data scan to occur: - The data area being scanned must be a Type II area - No index within the data area being scanned is also being rebuilt - No index associated with the data area being scanned is a word index - Index rebuild must be run with the "sort" option, that is answering "y" when asked if you have enough room for sorting. If any requirement is not met, the data scan is performed using the original single threaded mechanism. If not specified, the –datascanthreads value defaults to 0, also indicating the original single threaded mechanism.

Cluster Size (CSZ) CSZ must be at least 8 in order to be a type 2 storage area Disk allocation of different CSZ for empty objects (8KB db blocks): CSZ 8 = 64KB CSZ 64 = 512KB CSZ 512 = 4096KB or 4MB CSZ 8 minimizes the growth of empty databases If an area is designated for LARGE objects then CSZ 512 makes sense

Rows Per Block 8KB block, 256 rows per block = 30 bytes of user data per record: http://knowledgebase.progress.com/articles/Article/P81745 LOB areas: RPB = 1? http://knowledgebase.progress.com/articles/Article/P88663 Advantage to RPB = 1 is with growing LOBS

Fixed Length or Variable Extents? All variable is very simple Fixed length will require decision making and management

After-image extents? Creating them does not result in AI being enabled But it does move you one step closer # add_ai.st – add 8 variable ai extents a .

Other Guidelines and Considerations?

Questions? And now we have some time for questions…

Thank You!

Parking lot http://standardsinsight.com/ieee_company_detail/what-are-standards-why-are-they-important http://www.etsi.org/standards/why-we-need-standards http://www.bsigroup.com/en-GB/standards/benefits-of-using-standards/