1. B2: What’s New in 10.1 RDBMS? So many features, so few releases Richard Banville Progress Fellow

2. © 2008 Progress Software Corporation 2 Agenda  High Availability Problem Avoidability Visibility Scalability  Maintainability The database “ilities”

3. © 2008 Progress Software Corporation 3 Database Consistency Checking Index orderId in order for recid 2010 could not be deleted. Wrong key in idx 10 for record 2010. Invalid size of an index entry. Seen these messages before?

4. © 2008 Progress Software Corporation 4 Database Consistency Checking Invalid RM block for area 10 rmdoins: pbk->free went negative dbkey 4096 bkwrite: bktbl dbk 4096 not equal to bkbuf dbk -1234 bkaddr called with negative blkaddr: -1234 Or how about these…

5. © 2008 Progress Software Corporation 5 Database Consistency Checking Index orderId in order for recid 2010 could not be deleted. (1422) Wrong key in idx 10 for record 2010. (14227) Invalid size of an index entry. (2670) Seen these messages before?

6. © 2008 Progress Software Corporation 6 Database Consistency Checking Invalid RM block for area 10 (10809) rmdoins: pbk->free went negative dbkey 4096 area 10 (822) Record continuation not found, fragment recid 4096 area 10. (10831) bkwrite: bktbl dbk 4096 not equal to bkbuf dbk -1234 (645) bkaddr called with negative blkaddr: -1234 (762) Or how about these…

7. © 2008 Progress Software Corporation 7 Database Consistency Checking  Shared memory overwrite protection -MemCheck  Physical block consistency checking -DbCheck -AreaCheck “area name” -Index Check “index name” -TableCheck ‘table name” Stop problems before they happen

8. © 2008 Progress Software Corporation 8 Database Consistency Checking  Database startup parameter  Managed via promon R&D Admin Functions 8. Block level consistency check Enabling the consistency checks Current consistency check status: 1. -MemCheck: enabled 2. -DbCheck: enabled 3. -AreaCheck in area: "customer" enabled 4. -IndexCheck: disabled 5. -TableCheck: disabled Enter the option to enable/disable a consistency check:

9. © 2008 Progress Software Corporation 9 Database Consistency Checking  Memory checking: unnoticeable impact  Block level checking Performance impact Current consistency check status: 1. -MemCheck: enabled 2. -DbCheck: enabled 3. -AreaCheck in area: "customer" enabled 4. -IndexCheck: disabled 5. -TableCheck: disabled Enter the option to enable/disable a consistency check: ~5% < 1%

10. © 2008 Progress Software Corporation 10 Database Consistency Checking  dbtool block consistency checking 5. Read or Validate Database Block(s)  Validation levels 0: Block header info only 1: Record header & record size 2: Record overlap checking 3. Record Validation (logical validation) Looking for existing inconsistencies online

11. © 2008 Progress Software Corporation 11 Online Tools  Online dbanalys - Includes chanalys info  Tools now online proutil -C chanalys [ -scan ] -scan: fewer locks, less consistent proutil -C idxcheck Idxcheck validation levels –Physical consistency –Keys to records –Records to keys –Validate key order –Locks associated tables More analysis available online contention

12. © 2008 Progress Software Corporation 12 The roll forward process myDb myDb.bak

13. © 2008 Progress Software Corporation 13 The roll forward process myDb ai ftp

14. © 2008 Progress Software Corporation 14 The roll forward process myDb Hot Standby ai ftp ai X SYSTEM ERROR: Attempt to read block 18446744073709550382 which does not exist in area 8, database x. ** Save file named core for analysis by Progress Software Corporation. Roll forward

15. © 2008 Progress Software Corporation 15 Ai Verify rfutil -C aiverify Partial: ai block and note header validation –Increases reliability of archived ai files Full: partial + note data validation –Identifies point in time recovery Running –At ai switch or on ai archival –Just before roll forward of extent  Preferably on hot standby Ai validation before application

16. © 2008 Progress Software Corporation 16 Roll forward verification myDb Hot Standby ai ftp ai X Hot Stand by: Validate/fix production db Re-base hot standby Recovery Scenario: Roll forward to transaction rfutil myDb -C aiverify full rlNoteVerify: Note dbkey is negative -1234. (14099) Trid: 358 code = RL_CXINS version = 2 (12528)

17. © 2008 Progress Software Corporation 17 More tools for high availability  “Online” backup of replication target (foundational work) Normal operating state required SHR schema lock on source ai file stores changes until complete  EMC’s SRDF* certification Real time copies of logical data volumes Data replication Remote backup Replication enhancements *Symmetrix Remote Data Facility (SRDF) Disaster recovery Fail over/fail back

18. © 2008 Progress Software Corporation 18 The problem Out of free shared memory. Use -Mxs to increase. Lock table overflow, increase -L on server. Have you ever seen these error messages… or heard these… Why can’t you improve the buffer pool hit ratio on the database? The recovery subsystem is a bottle neck. Look at that BI buffer wait %.

19. © 2008 Progress Software Corporation 19 Increase startup parameters online proutil -C increaseto  : -L, -B, -bibufs, -aibufs, -Mxs  Increase, not decrease  Resource restrictions apply  New shared memory segments Security restrictions –Servers: automatically attach quickly –Self serve: attach w/db action over time Segment size Increase startup parameters without database restart

20. © 2008 Progress Software Corporation 20 Increase startup parameters online Increasing available locks online proutil myDb -C increase -L 10000 Waiting for broker connection to newly added shared memory segments. Usr Name Type Pid 7 richb ABL 5957 The connections above have not attached to recently added shm segments. Do you wish to recheck? (y/n) (n): Increase params aborted because of shared memory allocation issue. (y): Increase params increasing lock table size (-L) from 1025 to 10016.

21. © 2008 Progress Software Corporation 21 Agenda  High Availability Problem Avoidability Visibility Scalability  Maintainability The database “ilities”

22. © 2008 Progress Software Corporation 22 Promon  Promon R&D Status 17. Servers By Broker A more organized view of existing data Better organized server grouping Sv Pend. Cur. Max. Port No Pid Type Protocol Logins Users Users Users Num 0 15275 Login TCP 5 0 0 15 2053 2 15501 Auto TCP 1 0 1 15 1025 3 15509 Auto TCP 1 0 1 15 1026 4 15511 Auto TCP 1 0 1 15 1027 5 15514 Auto TCP 1 0 1 15 1028 1 15381 Login TCP 5 0 0 5 2051 7 15609 Auto TCP 1 0 1 5 1030 8 15617 Auto TCP 1 0 1 5 1031 9 15629 Auto TCP 1 0 1 5 1032 10 15638 Auto TCP 1 0 1 5 1033

23. © 2008 Progress Software Corporation 23 Promon  Promon R&D Other Displays 7. Total Locks per User  User type display – _Connect-ClientType Improved user information User Name Type PID TTY Total Record SHR/EXCL... 5 richb SELF/ABL 15494 /dev/pts/16 1 1 1 11 richb SELF/ABL 16101 /dev/pts/13 3 3 0 24 richb REMC/SQLC 15530 2 1 0 48 richb REMC/WTA 20182 mysystem 2 2 0 100 richb REMC/ABL 20183 mysystem 2 2 0 101 richb SQFC 20100 mysystem 5 4 0 150 richb REMC/APSV 20101 mysystem 1 1 0 175 richb SELF/APSV 20102 mysystem 1 1 0

24. © 2008 Progress Software Corporation 24 Statement Caching  List recent client statements Promon R&D Status 18. Client Database-Request statement Cache –By user/server/all users current and future. –Last line or entire stack –ABL info obtained from DEBUG-LIST output .i’s are in-lined What code is executing against my database

25. © 2008 Progress Software Corporation 25 Statement Caching Where is that ABL code executing User number : 24 User name : richb User type : REMC/ABL Login date/time : 03/06/08 15:30 Statement caching type : ABL Program Stack Statement caching last updated : 03/06/08 15:35 Statement cache information : 39 : proc7 /usr1/richb/x.p 26 : proc6 /usr1/richb/x.p 22 : proc5 /usr1/richb/x.p 18 : proc4 /usr1/richb/x.p 14 : proc3 /usr1/richb/x.p 10 : proc2 /usr1/richb/x.p 6 : proc1 /usr1/richb/x.p 3 : /usr1/richb/x.p

26. © 2008 Progress Software Corporation 26 Statement Caching What’s that SQL code executing User number : 23 User name : richb User type : REMC/SQLC Login date/time : 03/06/08 15:42 Statement caching type : SQL Statement Statement caching last updated : 03/06/08 15:42 Statement cache information : select count(*) from pub.customer

27. © 2008 Progress Software Corporation 27 Statement Caching proutil -C updatevst Need to load new schema fields  _Connect vst _Connect-CachingType _Connect-CacheLastUpdate _Connect-CacheInfoType _Connect-CacheLineNumber[32] _Connect-CacheInfo[32] VST support - _Connect

28. © 2008 Progress Software Corporation 28 Agenda  High Availability Problem Avoidability Visibility Scalability  Maintainability The database “ilities”

29. © 2008 Progress Software Corporation 29 Scalability  Large file support for bulk load (> 2Gb) Independent of DB large file status  IPv6 support More ip addresses –only 30% ip addrs left (7 yrs) Routing improvements Required by government contracts Configuration –-ipver IPv4 (default) or IPv6 –Property file: ipver= –Explorer option I want more

30. © 2008 Progress Software Corporation 30 Scalability  IPv6 support Mixed mode (dual stack) –IPv6 can convert IPv4 address –Not available on windows Avoiding confusion –-minport/-maxport –use IPv6 configured –H hostnames I want more

31. © 2008 Progress Software Corporation 31 Internal Performance Improvements  “clean” shutdown indicator Avoids long redo in roll forward Last 2 clusters always redone  Improved read operation concurrency Latch enhancements and usage optimization  Multi-user bi activity optimization Avoid rollback “block jump” notes Fewer notes written The need for speed

32. © 2008 Progress Software Corporation 32 Agenda  High Availability Problem Avoidability Visibility Scalability  Maintainability The database “ilities”

33. © 2008 Progress Software Corporation 33 Roll forward protection myDb ai myDb.bak

34. © 2008 Progress Software Corporation 34 Roll forward protection myDb ai X ** The database was last changed Mon Apr 1 15:37:38 2008. ** The after-image file expected Mon Apr 1 15:33:45 2008. ** Those dates don't match, so you have the wrong copy of one of them. roll forward open /usr1/x.a4 error: -1. Roll forward

35. © 2008 Progress Software Corporation 35 Roll forward protection myDb ai X In the.lg file: rfutil -C roll forward session end. Single-user session begin for richb on /dev/pts/101. Begin Physical Redo Phase at 256.

36. © 2008 Progress Software Corporation 36 Roll forward protection rfutil -C roll forward oplock  Prevents “stray” database connections Prostrct add allowed  Automatic disablement At roll forward completion  Explicit disablement rfutil -C roll opunlock Recovers db Stops the roll forward process Non interruptible roll forward

37. © 2008 Progress Software Corporation 37 Roll forward protection myDb ai - Access to the database during roll forward process is not allowed because it will modify the database. - Write access to the database will not be allowed until the roll forward operations have completed. rfutil -C roll forward oplock –a myDb.a1 Connection attempts:

38. © 2008 Progress Software Corporation 38 Index Rebuild  Index rebuild packing factor Max % of space used Avoids costly index block splits proutil -C idxbuild –pfactor  Examine utilization % in idxanalys More control

39. © 2008 Progress Software Corporation 39 Index Fix Interface  Idxfix uses idxbuild interface Select indexes to fix by –Table, schema, area or activation state Specific choices Select one of the following: All (a/A) - Fix all the indexes Some (s/S) - Fix only some of the indexes By Area (r/R) - Fix indexes in selected areas By Schema (c/C) - Fix indexes by schema owners By Table (t/T) - Fix indexes in selected tables By Activation (v/V) - Fix selected active or inactive indexes

40. © 2008 Progress Software Corporation 40 Space Allocation  Rec space search depth (10.1b05) New depth search algorithm R&D  Administrative Functions Menu  Adjust Latch Options  2. Record Free Chain Search Depth Factor: 5 -recspacesearchdepth Alter free space search of record free chain

41. © 2008 Progress Software Corporation 41 SQL Stored Procedures  64 bit JVM availability Java ™ 1.5 certification Can use same drivers Additional schema –_SysProcBin, _SysProcText 32 bit databases have schema already Databases created in 64 bit environment proutil -C enablestoredproc 64 bit stored procedure support.

42. © 2008 Progress Software Corporation 42 Binary Dump Specified - Improved proutil -C dumpspecified AND  Option values: GT, GE, LT, LE, EQ  Dump specific ranges  Improved parallelism proutil db –C dumpspecified cust.custnum GE 5 and LE 100 /dumpdir Binary dump specified with “between” range dumping

43. © 2008 Progress Software Corporation 43 In Summary We’ve made it even easier…  to achieve high availability  to see what’s going on  to maintain

44. © 2008 Progress Software Corporation 44 Relevant Exchange Sessions  OPS-1: How Healthy is Your Database Today?  OPS-18: Data Management Roadmap  OPS-19: What’s IPV6 and Why Should I Care?  OPS-28: A New Spin on Some Old Latches

45. © 2008 Progress Software Corporation 45 Questions ?

46. © 2008 Progress Software Corporation 46 What’s here that is also in 10.1B03?  Database Consistency Checking -memCheck, 0DbCheck, -AreaCheck, -IndexCheck, - TableCheck  Dbtool: Read or Validate Database Blocks  Rfutil: aiverify roll forward oplock  Promon: Servers by Broker Total Locks per User  Record lock removal on NO-LOCK reads

47. © 2008 Progress Software Corporation 47 Thank You

48. © 2008 Progress Software Corporation 48