Presentation is loading. Please wait.

Presentation is loading. Please wait.

OPS-10: Moving V8/V9 RDBMS to OpenEdge ® 10 Rob Marshall Principal Solutions Consultant.

Similar presentations


Presentation on theme: "OPS-10: Moving V8/V9 RDBMS to OpenEdge ® 10 Rob Marshall Principal Solutions Consultant."— Presentation transcript:

1 OPS-10: Moving V8/V9 RDBMS to OpenEdge ® 10 Rob Marshall Principal Solutions Consultant

2 © 2008 Progress Software Corporation2 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 What’s in OpenEdge RDBMS? BLOB, CLOB Datetime, Datetime-TZ INT64 (no conversion) Type II Storage Areas Fast Drop & Temp tables Increased shmem –B 1 billion Internal algorithmic enhancements Buffers, Locks, Indexing Improved APW scanning Auto Record Defrag Enhanced Txn Backout New Defaults Log File New format Significant events Improved management Db I/O by User by Object Database Describe 64 bit Rowids 64 bit Sequences 64 bit Integer Datatype Large Index Key Entries (1970) 32,000 areas 8 TB Shmem Performance Visibility Large Database Support Datatype Support

3 © 2008 Progress Software Corporation3 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 What’s in OpenEdge RDBMS? Online Schema adds Sequences Tables Fields Indexes w/fast fill Online space management Enabled/Disable AI online Enable AI Mgmt online HA Clusters Bundled Index Rebuild By area, table, active status.st file syntax checker AI Management Multi threaded utilities idxbuild, binary dump Binary Dump without index Binary Load Performance Index Fix with NO-LOCK SSL Auditing High Availability Security Maintenance

4 © 2008 Progress Software Corporation4 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10  General Migration Strategy  The Fast and Easy Upgrade  Physical Upgrade  Tuning Opportunity Agenda

5 © 2008 Progress Software Corporation5 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Basic Steps  Preparation Truncate BI, Disable AI, 2PC, and Replication (V9) Backup (V8/9)  Install Install OpenEdge Don’t need to uninstall V8/9 *Don’t overwrite your current Progress Directory  Upgrade Upgrade DB to OpenEdge 10 Do your backups !!!! Recompile/Re-deploy your ABL code if client is OpenEdge  Run the Application and Test, Test and….. Test First detail plan, review plan, test plan THEN execute…

6 © 2008 Progress Software Corporation6 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 The Application  Safe  Install OpenEdge on your existing test machine  proutil -C conv910*  Recompile application* and test  Fast and Loose  Upgrade a remote or local client site  Recompile application* and test  The Rollout Upgrade remote systems with OpenEdge  Remote client server, Remote Application servers  “re-deploy” newly built application (PROPATH) and test * ABL code needs to be recompiled/re-deployed only when upgrading the client to R10. In 3-tier configurations (with AppServer™) the client could still be V9. Not possible via SQL or V8. * You will have to convert a V8 db to V9 before converting to OpenEdge

7 © 2008 Progress Software Corporation7 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10  General Migration Strategy  The Fast and Easy Upgrade  Physical Upgrade  Tuning Opportunity Agenda

8 © 2008 Progress Software Corporation8 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Deployment  Preparation Truncate BI Disable AI, 2PC, Replication (V9) Backup database (V8/9) Validate backup  Install OpenEdge 10 on server machine And everywhere else! (Clients must be upgraded before the server can be)  Recompile and re-deploy application (if need be) Client/Server V9 to OpenEdge 10 disallowed V9 Clients to R10 AppServer to R10 Database is allowed, No SQL V9 to OpenEdge permitted

9 © 2008 Progress Software Corporation9 Connectivity: Progress ® V9 and OpenEdge 10  ABL mixed configurations: Progress V9 and OpenEdge 10 are supported One version back (clients to servers) One version forward ( clients to Application Server )  SQL must match Refer to product by product information for further details Client Application Server Database 9109 9 99 9 NEW in OpenEdge 10

10 © 2008 Progress Software Corporation10 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Database Conversion  Run conversion utility _proutil -C conv910 –B 512  Conversion runs in 5-minutes or less  Basically just a schema upgrade No changes to user records or indexes No changes to physical structures  Backup database  Re-start database and application

11 © 2008 Progress Software Corporation11 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10  You are now “Good To Go” with OpenEdge  No physical changes to existing user data Data remains in Type I storage areas Data fragmentation exists as before  Optimize physical layout when time permits...

12 © 2008 Progress Software Corporation12 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10  General Migration Strategy  The Fast and Easy Upgrade  Physical Upgrade  Tuning Opportunity Agenda

13 © 2008 Progress Software Corporation13 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Storage Areas?  Performance, Scalability & Maintenance  Take advantage of new features  No adverse application effects Physical reorg does NOT change the application Object location is abstracted from the language by an internal mapping layer Different physical deployments can run with the same compiled r-code

14 © 2008 Progress Software Corporation14 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 How to Get There  Preparation (same as before) Truncate BI, disable AI, backup, validate, install…  Before Physical Reorg Upgrade database to OpenEdge 10 –conversion utility –prostrct create (a must if changing block size)  Physical Updates (no r-code changes required) Separate schema from user data Create new storage areas –Specify records per block –Specify Type II cluster sizes

15 © 2008 Progress Software Corporation15 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 How to Get There  Physical Reorg Spread data out amongst new areas Move indexes  Online options vs offline options Database block size changes are offline  After Reorg Reclaim Unused Space –Truncate old data area –Delete old data area

16 © 2008 Progress Software Corporation16 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 How to Get There  In Place (same database) Transformation done all at once or over time Add new storage areas to existing database Migrate data from old areas to new Reclaim space from old areas  New database Transformation done in one maintenance window Dump old data Create new database Load into new database Prodel old database  Mix of Option #1 and Option #2 (custom) Create new database Move data from old database to new database Reclaim space by deleting old database

17 © 2008 Progress Software Corporation17 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Getting started Separate user data from schema

18 © 2008 Progress Software Corporation18 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Moving schema tables Separate Schema from user data (in place): proutil -C mvsch (offline operation) Schema Area Old Default Area Renames existing schema area

19 © 2008 Progress Software Corporation19 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Moving schema tables Separate Schema from user data: proutil -C mvsch (offline operation) Old Default Area Schema Area Renames existing schema area Creates new schema area Moves schema Tables & Indexes Schema Area

20 © 2008 Progress Software Corporation20 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Moving schema tables Separate Schema from user data: proutil -C mvsch (offline operation) Old Default Area Schema Area Renames existing schema area Creates new schema area Moves schema Tables & Indexes Schema Area You move data To new areas User Area User Area User Area User Area You truncate Old Default Area

21 © 2008 Progress Software Corporation21 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Physical Changes: Location, Location, Location  Create.st file with new layout Set records per block  Use Type II Storage Areas Tables 64 or 512 block clusters Indexes 8 or 64 block clusters d “Cust/Bill Indexes“:7,1;8 /d_array2/myDB_7.d1 f 512000 d “Cust/Bill Indexes“:7,1;8 /d_array2/myDB_7.d2 # d “Customer Data“:8,16;64 /d_array2/myDB_8.d1 f 1024000 d “Customer Data“:8,16;64 /d_array2/myDB_8.d2 # d “Billing Data“:9,32;512 /d_array2/myDB_9.d1 f 1024000 d “Billing Data“:9,32;512 /d_array2/myDB_9.d2

22 © 2008 Progress Software Corporation22 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Physical Changes  Validate first prostrct add new.st -validate  Then update prostrct add new.st OR: prostrct addonline new.st The Structure file format is valid. (12619)

23 © 2008 Progress Software Corporation23 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Moving Tables and Indexes 1.Table move and Index move Online (by primary index) 2.Dump and Load (D&L) With or without index rebuild Application must be offline  Suggestion: Mix of option #1 and #2 1 st purge/archive unneeded data Table move small tables (number of blocks) D&L everything else 3 Options for Data Movement

24 © 2008 Progress Software Corporation24 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Option #1: Table/Index Move  Advantages: Online (with no-lock access) Dump & load in one step Schema is automatically updated No object # changes to deal with Parallelism  Disadvantages: Only No-lock accessible during move Moves but doesn’t “rebuild” indexes Too slow for large tables Changes are logged! –BI growth may be of concern Pros and Cons

25 © 2008 Progress Software Corporation25 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Table Move proutil -C tablemove [ owner-name. ] table-name table-area [ index-area ]  Move/reorg a table by its primary index  Move a table AND its indexes Preferred performance  Fast for small tables

26 © 2008 Progress Software Corporation26 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Index Move proutil -C idxmove [ owner-name. ] table-name. index-name area-name  Move an index from one area to another  Does NOT alter/optimize index block layout  Fast but does not “rebuild” indexes  Online but changes to owning table blocked

27 © 2008 Progress Software Corporation27 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Option #2: Dump and Load  Textual Data ASCII dump ASCII load Bulk load followed by index rebuild  Binary Binary dump Binary load –With index rebuild –Followed by index rebuild  Custom (Textual or raw) D&L with triggers Buffer-Copy / Raw-data-transfer / Export/Import Can be tricky, you may want help 3 Dump and Load Flavors

28 © 2008 Progress Software Corporation28 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Dump and Load General Strategy  Create new database structure Add to existing DB New database  Run tabanalys  Dump table data sequence values, _User table  Data definitions Dump definitions Modify storage area locations Load definitions  Load table data  Build indexes (if needed) [10.1C can specify pack]  Run tabanalys  Backup * If you have the disk space, creating a new db saves time

29 © 2008 Progress Software Corporation29 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Dumping the data

30 © 2008 Progress Software Corporation30 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Dictionary Data Dump  Database Admin tool  OR: run prodict/dump_d.p(,, ).  Advantages: Fast and Easy Parallel No endian issues  Disadvantages: 2 GB File size limit** Pre 10.1C Can’t choose dump order Have to dump entire table Must ensure no one changes table between D&L

31 © 2008 Progress Software Corporation31 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Using Binary Dump  Advantages: Fastest and Easy No 2 GB file size limit No endian issues Can choose dump order (by index) Can dump table data in portions Multi threaded (10.1B) Can dump multiple tables concurrently (parallel)  Disadvantages: Must ensure no table changes between D&L (unless using triggers as well)

32 © 2008 Progress Software Corporation32 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Binary Dump Specified proutil -C dumpspecified ‘field- value1 AND operator value2’ -preferidx  10.1B03 allows multiple values  Switches table.field MUST be lead participant in index Valid operators: LT, GE, LE, GT, EQ -preferidx determines specific index to use -index, -thread are ignored  Performance Threaded is preferred Can run in parallel with many unique values  Cautions Avoid using descending indexes There is a risk of missing a range

33 © 2008 Progress Software Corporation33 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Binary Dump Specified Finding the median value define variable i as integer no-undo. define variable max-recs as integer initial 0 no-undo. define variable median as integer no-undo. for each mytable NO-LOCK use-index : max-recs = max-recs + 1. end. median = max-recs / 2. do i = 1 to median: find next mytable NO-LOCK use-index. end. display i substr(field1, 1, 192).

34 © 2008 Progress Software Corporation34 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Binary Dump Threaded proutil -C dump -index -thread 1 -threadnum -dumpfile -Bp 64  -index Choose index based on read order -index 0 –Faster dump, slower read –Assumes coming from Type II  -thread indicates threaded dump # threads automatic (# CPUs) –threadnum max of # CPUs * 2 Threads only available in multi user mode Workgroup only supports 1 thread  -dumpfile used as input for load

35 © 2008 Progress Software Corporation35 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Don’t forget SQL SQL Tables and Views These need to be dumped and loaded as well You can use different commands to move the information: sqlschema sqldump sqlload

36 © 2008 Progress Software Corporation36 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Data Dump Completed. Reorganize the Area/Object Configuration

37 © 2008 Progress Software Corporation37 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Dump & Modify data definitions  Use Data administration tool  OR: run prodict/dump_df.p(‘ALL’, ‘.df’, ‘ ’). If using bulk load: run prodict/dump_fd.p(‘ALL’, ‘.fd’).

38 © 2008 Progress Software Corporation38 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Dump & Modify data definitions  Update.df files Optionally delete old table Change table’s area information  Delete/Drop tables  Load data definitions Data administration tool OR: run prodict/load_df.p(“.df"). ADD TABLE "mytable2" AREA “Old Default Area" DROP Table “mytable2” ADD TABLE "mytable2" AREA “New Data Area"

39 © 2008 Progress Software Corporation39 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Alternative Data Definition Modification  Truncate objects for fast move/delete proutil -C truncate area “Old Default Area” Warns then deletes data (but NOT schema)  Rebuild/activate empty indexes (if moving) proutil -C idxbuild inactiveindexes Can be done ONLINE, not just the build but the activate  Move “empty” tables/indexes to new area proutil -C tablemove [ index-area ] If all data in area dumped…

40 © 2008 Progress Software Corporation40 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Load the data back in (finally). Remember to load all data files. Be sure to validate data loaded.

41 © 2008 Progress Software Corporation41 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Loading Things to consider...  Enable large file support In the Operating System (ulimit) In the Filesystem / volume groups In the Database

42 © 2008 Progress Software Corporation42 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Bulkload proutil -C bulkload -B 1000 –i –Mf 10  Data input from dictionary or custom data dump Mentioned here for completeness only  Drawbacks: 2 GB file limit (pre 10.1C) Loads one table at a time (single user) Does not insert index entries –Requires index rebuild as separate step No advantage over other loads Slower than all other loads

43 © 2008 Progress Software Corporation43 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Dictionary Load  Data Administration Tool  OR: run prodict/load_d.p(‘table1’, ‘table1.d’).  Data input from dictionary or custom data dump 2 GB file limit per load (pre 10.1C)  Load data in parallel (to separate tables) Inserts index entries Index tree not perfect  Performance close to binary load + index rebuild (when loading multiple tables)

44 © 2008 Progress Software Corporation44 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Binary Load proutil -C load.bd [build]  Load to new or truncated area Truncated rather than “emptied”  Parallel load to different tables Same or different areas without scatter! When using Type II Areas  Optionally load with build indexes Somewhat better performance

45 © 2008 Progress Software Corporation45 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Binary Load proutil -C load.bd -dumplist  Dump List File: /usr1/db/mytable.bd /usr1/db/mytable2.bd /usr1/db/mytable3.bd  Must load ALL dumps (.db, db2,.db3, …) From a threaded dump or dumpspecified…

46 © 2008 Progress Software Corporation46 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Tuning the Process  Dump with –RO, high –B and/or -Bp Dump on index with fewest # blocks (if possible)  Load with High –B, –r** or –i** BIW, 1.5 APW’s per CPU, Very Large BI clusters with 16K BI blocks No AI/2PC  Spread data, BI and temp files across disks / controllers Tune for high (non-recoverable) activity ** only use -r & -i when complete data recovery possible

47 © 2008 Progress Software Corporation47 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 After the Load proutil -C idxbuild [ all |table | area schema | activeindexes | inactiveindexes] [- thread n] [-threadnum n] [-T ] [-TM n] [–TB ] [-B n] [-SG n] [-SS ] [-pfactor n]  Many new idxbuild choices  Helpful parameters -SG 64 (sort groups) -SS filename (file containing sort file list) -TM 32 (merge buffers) -TB 31 (temp block size) -B 1000  Run tabanalys validate # records  Backup your database Build Indexes (where applicable)

48 © 2008 Progress Software Corporation48 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Reclaim space proutil -C truncate area Warns then deletes data proutil -C truncate area Only truncates “empty” areas (but all of them)  Area logically truncated (option #2)  Extents can be deleted prostrct remove d For areas that were emptied…

49 © 2008 Progress Software Corporation49 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 After the Load Don’t Forget.... Run UPDATE STATISTICS for SQL/ODBC Think your done…

50 © 2008 Progress Software Corporation50 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Agenda  General Migration Strategy  The Fast and Easy Upgrade  Physical Upgrade  Tuning Opportunity

51 © 2008 Progress Software Corporation51 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Tuning Opportunity  -Bt (temp tables are Type II storage areas) – client parameter  10.1B+ changes default Temp table block size From 1K to 4K –tmpbsize 1 restores old behavior – client parameter  Monitor BI Cluster Size BI notes are bigger in OpenEdge 10  BI grow

52 © 2008 Progress Software Corporation52 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 In Summary  Conversion is quick Physical Upgrade at your leisure Lots of physical re-org options  Rollout can be simple  10,000+ customers on OpenEdge

53 © 2008 Progress Software Corporation53 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Relevant Exchange Sessions  OPS-1: DBA 101 - How Healthy is Your Database Today?  OPS-8: Alerts, Alarms, Pages and Harbingers of Trouble…  OPS-14: Effective OpenEdge Database Configuration  OPS-18: Data Management and Platforms Roadmap  OPS-19: What is IPv6 and Why Should I Care  OPS-20: Data Management and Platforms Info Exchange  OPS-23: OpenEdge Performance Basics  OPS-28: A New Spin on Some Old Latches

54 © 2008 Progress Software Corporation54 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Questions ?

55 © 2008 Progress Software Corporation55 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10 Thank You

56 © 2008 Progress Software Corporation56 OPS-10: Moving V8/V9 RDBMS to OpenEdge 10


Download ppt "OPS-10: Moving V8/V9 RDBMS to OpenEdge ® 10 Rob Marshall Principal Solutions Consultant."

Similar presentations


Ads by Google