1. McNaughton-McKay Progress 9.1D Training Jeffrey A. Brown - Technical Support Consultant (jefbrown@nxtrend.com)

2. Agenda: SX.enterprise concepts Progress 9.x architecture Marc/scripts/configuration Maintenance

3. SX.enterprise Concepts

4. Client/server vs self serving clients Appserver

5. SX.enterprise Concepts Double click on SX.e icon –Prowin32 Database connection via code –Connecting to database –pf … –Autoserver connection Login box Status window – Appserver connection

6. User 1 _mprosrv Shared Memory prowin32.db

7. User 1AutoServer.db prowin32 _mprosrv Shared Memory

8. User 1 prowin32.db AutoServer _mprosrv Shared Memory

9. User 1 prowin32 AppServer.db AutoServer _mprosrv Shared Memory

10. Appserver Discussion

11. User 1 AppServerDatabase Appserver Discussion

12. User requests customer address

13. User 1 AppServerDatabase Appserver Discussion

14. User requests a price for a specific customer

15. User 1 AppServerDatabase Appserver Discussion

16. User 1 AppServerDatabase Appserver Discussion

17. User 1 AppServerDatabase Appserver Discussion

18. Appserver Discussion - Summary Users are connected to Appserver and Database Small requests do not use Appserver Larger requests go through Appserver Need to have large pipe between Database Server and Appserver Server

19. Progress 9.x Architecture

20. Progress 9.x Architecture - Summary Admin Server NameServer AppBrokers –AppServers Database Brokers –AutoServers

21. Progress 9.x Architecture.db

22. Progress 9.x Architecture Admin Server.db /rd/bin/admin.start

23. Progress 9.x Architecture Admin Server.db Name Server Starts

24. Progress 9.x Architecture Admin Server DB Broker.db Name Server Starts /rd/bin/dbbroker.start

25. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server Starts /rd/bin/appbroker.start

26. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server Registers

27. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1

28. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Connects

29. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Starts Connects

30. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Disconnects

31. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Connects

32. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Connects

33. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Disconnects

34. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Connects

35. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server Starts Connects

36. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server Disconnects

37. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server Connects

38. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server

39. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server

40. Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server

41. Progress 9.x Architecture Firewall considerations –-minport –-maxport

42. Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall

43. Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall

44. Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall

45. Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall Cannot connect

46. Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall

47. Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall Port 1025+

48. Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall -minport -maxport

49. Progress 9.x Architecture - Summary Admin Server is the key Name Server directs users to App Brokers App Brokers spawn App Servers Database Brokers spawn Auto Servers Firewall considerations –-minport & -maxport –Nameserver - UDP

50. NxTrend Standard Ports

51. Most are defined in MARC Should be defined in the /etc/services file Test ports are the same using 7200 range

52. NxTrend Standard Ports nxt7100/tcp# SX database appsrv7120/tcp# SX appserver twl7160/tcp# TWL twlappsrv7161/tcp# TWL appserver twlsxappsrv7162/tcp# TWL SX appserver nxtadmsrv7180/tcp# adminserver port nxtadmsrvadmin7181/tcp# adminserver admin port nxtnamesrv7182/tcp# nameserver port

53. Storage Areas and Extents

54. Progress v8.x - Extents Extent Disk Storage - Files Extent

55. Progress v8.x – Database Extents nxt.db nxt.d1 nxt.d2 nxt.d3 nxt.d4 nxt.d5 nxt.d6 nxt.d7 nxt.d8 nxt.d9 nxt.d10

56. Progress v9.1x - Storage Areas Area 6Area 7Area 8 Extent Disk Storage - Files Table ATable B Index A-1Index B-1 Table C Table D Index C-1 Index D-1Index E-1 Table E Area 51 Extent Index E

57. Progress v9.1x – Database Extents Schema Area (7) trhead (9) trlines (11) trother (13) trsm (15) trstat (17) trtrans (19) trcust (8) trhead_idx (10) trlines_idx (12) trother_idx (14) trsm_idx (16) trstat_idx (18) trtrans_idx (20) trcust_idx

58. Progress v9.1x – Database Extents nxt.dbnxt_10.d1nxt_14.d1nxt_18.d1 nxt.d1nxt_10.d2nxt_14.d2nxt_18.d2 nxt_7.d1nxt_10.d3nxt_14.d3nxt_18.d3 nxt_7.d2nxt_11.d1nxt_15.d1nxt_19.d1 nxt_7.d3nxt_11.d2nxt_15.d2nxt_19.d2 nxt_8.d1nxt_11.d3nxt_15.d3nxt_19.d3 nxt_8.d2nxt_12.d1nxt_16.d1nxt_20.d1 nxt_8.d3nxt_12.d2nxt_16.d2nxt_20.d2 nxt_9.d1nxt_12.d3nxt_16.d3nxt_20.d3 nxt_9.d2nxt_13.d1nxt_17.d1 nxt_9.d3nxt_13.d2nxt_17.d2 nxt_13.d3nxt_17.d3

59. Progress v9.1x - Increased Size Limits Database size: 16,384,000 gigabytes Before-Image size: 32,768 gigabytes Number of areas: 1,000 Database block size: 8,192 bytes Extent Size: 2 gigabytes * *Unless db and filesystem are large file enabled

60. MARC / Scripts & Configs

61. Marc Demo new Marc

62. New Scripts / Script Templates

63. New Scripts and New Names Many new scripts have been added to MARC’s script base Many more scripts had a name change to standardize names and functions

64. The Progress Admin Server These scripts help manage the Progress Admin and Name Servers admin.start admin.shut admin.query nameserver.query

65. The Database These scripts help manage the database brokers dbbroker.start dbbroker.shut dbbroker.query These are database utilities dbstats truncate.bi

66. The Application Broker These scripts help manage the Progress Application Brokers appbroker.start appbroker.shut appbroker.query

67. Parameter Files The following are a new set of parameter files for client connections client.pf editor.pf batch.pf singleuser.pf connect.pf

68. The Environment These scripts help manage this particular environment nxt.env nxtall.start nxtall.shut

69. Enterprise To start SX.enterprise, now use sxe sxe sxee sxeqde

70. The Report Manager These are the Report Manager scripts rptmgr.start rptmgr.shut

71. Binary Dump and Load These scripts are for Dump and Loads easybdsareabd bddriver.psareabddriver.p bddumpall.pbddumparea.p bdloadall.pbdloadarea.p bdscript.psareabdscript.p

72. After Imaging These scripts are for AI aimage.start aimage.shut aimage.backedup aimage.list aimage.archive aimage.new aistats rollforward.ai truncate.ai

73. Backups These are new standard backup scripts backup.data backup.full backup.online

74. RxServer RxServer scripts are now generated by MARC rxsfax.sh rxsprint.sh emailout.sh

75. Other Scripts These don’t fit anywhere else rdclean sassi.sh shutuser showuser config_rpt

76. TWL TWL scripts have been mostly rewritten MARC generates all of the needed TWL scripts TWL scripts now follow a standard naming convention

77. Additional TWL Application Broker These scripts help manage the Application Broker that manages TWL connections to the SX.enterprise database twlappbroker.start twlsxappsrv twlappbroker.shut twlsxappsrv twlappbroker.query twlsxappsrv TWL uses the twlappbroker scripts which are generated from TWLMARC. These twlappbrokers are for connections to the TWL database.

78. TWL RF Units These scripts are used to start the TWL application on RF units twlrf twlentrf twlentrf.pf

79. TWL End of Day These scripts are used to run TWL End of Day processing. These are usually called from the backup scripts The first script performs processing that requires a single-user connection to the DB while the second performs tasks using a normal multi-user connection to the DB twleod1 twleod2

80. TWL Interfaces The following scripts are used to manage the TWL interfaces twlsend.starttwlrcvpack.sh twlsend.shtwlrcv.sh twlsend.shuttwlrcv.shut twlsend.querytwlrcv.query

81. The Templates The next section describes the new Script Generator in MARC

82. MARC Generate Scripts MARC Databases -> Configure -> Generate Scripts Select the database(s) to generate scripts for Select the scripts to generate

83. Script Template All scripts generated by MARC are stored as templates These templates contain the form of the final script Critical portions of a template contain tags These tags get substituted with real values to generate a script

84. MARC Generate Scripts All of the templates reside in subdirectories under /rd/marc/scripts/template All templates must end with a.tpl These subdirectories are: base dg hp ibm sco intchan taxware twl

85. All of the templates in base are added to the list One system directory is included based on the selected machine type in Installations -> Configure (e.g. ibm ) If optional products are setup in MARC, then those templates are also included in the list (e.g. twl ) The custom directory is always checked first and overrides a standard template of the same name MARC Generate Scripts

86. Script templates may also be in /rd/marc/scripts/cust in the same subdirectories Copy the template from the directory under /rd/marc/scripts/template Custom Scripts

87. Template Tags There are two types of template tags –System tags (defined by MARC) –Custom tags (defined by the user) Tags are names enclosed in ‘ ’ –Example: The convention is to use tag names that are all capital letters

88. System Tags System tags are defined during the installation of MARC System tags contain values that MARC can get from the information in its own database System Tags can not be edited directly with the tag editor

89. Custom Tags Users may use the Tag Editor to create and edit tags. Custom tags are created by users or may be in some delivered scripts (e.g. rxsprint.sh) Values for the tags can be assigned in two ways: –By answering the prompt when MARC finds an undefined tag –By using the tag editor in Databases -> Configure -> Tag Editor

90. The Tag Editor The tag editor allows the user to: –Create new custom tags –Modify existing custom tags –Delete custom tags

91. Example Template Here is a simple template echo “Hello, world!” Top two lines are reserved for formatting SCRIPTNAME is what the script will be called DESTDIR tells where the script will get created * Future (new to SX.e 4.0):

92. Script Naming All scripts should be prefaced with the environment name Use ENV before the script name to substitute in the environment name in the MARC that is generating the script

93. Destinations The destination directory can be a path The destination can be one of these predefined directories BIN OPSYS RD EXEC

94. Building A Script Here is a simple template “helloworld.sh.tpl” echo “Hello, world!” Should be created in the../cust/base directory Generating script to /rd/bin/

95. Script Templates - Summary All scripts will be generated by MARC and are stored as templates Should no longer hand edit scripts Script issues will begin by regenerating all scripts New scripts are easily added and existing ones easily updated

96. Progress 9.x Properties Files

97. No database parameter files for Progress 9.x Admin server reads from “properties” files. The properties files are stanza based. Marc has been rebuilt to handle modifications to these files. You may also use the Progress Explorer Tool to make modifications. DO NOT edit these files by hand!

98. Progress 9.x Properties Files V8 used many parameter files (/rd/opsys/nxtdb.pf) $DLC/properties/conmgr.properties –Database Broker –All Databases Registered $DLC/properties/ubroker.properties –Admin Server –Name Server –App Broker –Etc. DO NOT edit these files by hand!

99. Progress 9.x Properties Files Utilities to help validate your changes: ubconfig nsconfig asconfig (-name appsrv) dbconfig These utilities are in $DLC/bin

100. Report Managers/ Queues

101. In Graphical SX.e all printing is handled by Report Manager. New –param options for Report Manager Define different Report Managers for each Queue Prevents Report Managers from being too busy to pick up “on demand” jobs. Provides method of running jobs sequentially

102. Report Managers/Queues -param _d- Demand Jobs Only. _lXXX - Log File Name where “XXX” is the full path of the log file. _uXXXX- The report manager will only process jobs for the specified user. _pXXX - Pause Time, where “XXX” is the pause time in seconds. Can be the first or second parameter, but must precede the _q parameter where both are used. Do not set the pause time to zero. _qXXX- Queue Name where “XXX” is the name of a print queue. _x - This report manager will not support the blank queue.

103. Report Managers/Queues rptmgr - poepp User 1 rptmgr - oerd rptmgr - poepp rptmgr - arrp rptmgr - smrs

104. Report Managers/Queues Setup the queue in SX.e Copy the rptmgr.start script to a new script example: rptmgrOE.start Modify the new rptmgr.start command line –Sample -param with pause time of 5 seconds and queue name of OE that only picks up jobs for this queue: -param _p5_qOE_x Modify the nxtall.start to include the new rptmgr.start

105. Report Managers/Queues rptmgrOE User 1 rptmgrSM - smrs rptmgrPO - poepp rptmgr – porr rptmgr - arrp _p5_qOE_x _p5_qSM_x _p5_qPO_x _p5 _p5_d

106. Disconnecting Users - shutuser script

107. Using shutuser & showuser Risks in shutting a user out Never use kill -9

108. User 1 _mprosrv _progres u Shared MemoryVirtual tables u1.db ltp Syntax:# shutuser PID

109. User 1 _mprosrv prowin32 Server u Shared MemoryVirtual tables u1.db ltp Syntax: # shutuser 0001sys

110. Progress 9.x Log Files

111. New log files in Progress 9.x $DLC/bin/proenv –$WRKDIR –exit out when done Most logs are found in /rd/tmp Some are in /db/sort

112. Progress 9.x Log Files admserv.log –exceptions.log –default.log –AdminServerSystemOut.log NS1.ns.log appsrv.broker.log / appsrv.server.log rptmgr#### rptmgr.err

113. Progress 9.x Log Files dbstats.log –Summary –Empty blocks –Free blocks

114. Progress Monitor Utility - promon

115. Promon Progress Monitor utility $DLC/bin/promon Useful for performance monitoring as well as troubleshooting Database broker must be up to access promon Hidden menu R&D There are also hidden options in the hidden menus

116. Promon - Main Menu 1 - User control –used to see who is logged in 3 - Block Access –used to see number of records accessed –useful in troubleshooting hung/slow queries 4 - Record Locking Table –useful in troubleshooting locking issues related to code

117. Promon - Main Menu - Continued 5 - Activity –The most useful screen –The performance monitor screen –Displays number of transaction commits –Displays number of checkpoints –Displays number of buffers flushed –Displays percent APW writes –Displays percent BIW writes –Displays percent Buffer Hits (buggy)

118. Promon - Main Menu - Continued 5 - Activity - Continued –Displays the database size (includes fixed extents) –Displays the BI file size (includes extents) –Displays amount of DB Broker Shared Memory –Displays number of users logged in and how attached –Displays number of APW’s that are running –Displays number of AutoServers that are running

119. Promon - Main Menu - Continued 6 - Shared Resources –Displays db parameters –Displays lock table high water mark 7 - Database Status –Displays database block size –Displays the database high water mark –Displays number of free blocks below highwater mark –Displays BI block & cluster size –Displays database creation date (if multi-volume)

120. Promon - R&D Main Menu 1 - Status Displays 2 - Activity Displays 3 - Other Displays 4 - Administrative Functions

121. Promon - R&D, 1 - Status Displays 3 - Servers –Displays status information for Login Brokers (LB) & Auto Servers (AS) –Displays PID for active LB & AS –Displays active/inactive –Displays number of login attempts –Displays number of current logins –Displays maximum number of logins –Displays port numbers for active LB & AS

122. Promon - R&D, 1 - Status Displays - Continued 4 - Processes/Clients … –2 - Blocked Clients Used for troubleshooting “deadly embrace” locks Useful for troubleshooting “hung” processes –3 - Active Transactions Useful for troubleshooting unusual BI file growth Displays active transactions per user Displays information about each active transaction

123. Promon - R&D, 2 - Activity Displays 1 - Summary –Displays same info as Main Menu option #5 3 - Buffer Cache –use to correctly calculate Buffer Hits % 5 - BI Log –Displays BI file activity –useful in troubleshooting unusual BI file growth 13 - Other –Displays Wait on Semaphore information –useful in tuning -spin

124. Promon - R&D, 3 - Other Displays 1 - Performance Indicators –Useful in performance tuning 2 - I/O Operations by Process –Displays historical record read information per process –useful in troubleshooting lengthy processes 4 - Checkpoints –Displays historical checkpoint statistic information –useful in tuning checkpoint performance

125. Promon - R&D, 4 - Administrative Functions 4 - Adjust Latch Options –Used to tune -spin on the fly –Can only use this if -spin is set to at least 1

126. AI & Online Backups

127. After-Imaging (AI)

128. After-Imaging Progress’ after-imaging feature lets you recover a database if you lose the database or before image (BI) files. AI DB

129. Roll-Forwarding/ point-in-time recovery With roll-forward recovery, Progress uses data in the AI files to automatically reprocess all the transactions since the last good backup.

130. After-Imaging - How does it work? DB BI AI Shared Memory

131. After-Imaging - How does it work? DB BI AI Shared Memory

132. After-Imaging - How does it work? DB BI AI Shared Memory

133. After-Imaging - How does it work? DB BI AI Shared Memory

134. After-Imaging - How does it add up? AI files + Restored DB = DB just before the crash

135. DB BI AI Crashed: DB After-Imaging – How does recovery work?

136. DB BI AI Delete: DB & BI After-Imaging – How does recovery work?

137. DB BI AI Restored: DB & BI

138. After-Imaging – How does recovery work? DB BI AI Rollforward: AI to DB

139. After-Imaging – How does recovery work? DB BI AI Shared Memory No Data Lost!!!

140. Progress After-Imaging - The Setup

141. After-Imaging - The setup 5 fixed AI extents in /ai –round-robin the fixed ai extents aimage.archive moves full extents to /aisave Extent 1 Extent 3 Extent 2 Extent 5 Extent 4

142. After-Imaging - The key to it’s success /ai/aisave nxt.a1 nxt.a2 nxt.a3 nxt.1 nxt.2 nxt.3 nxt.4 nxt.5 Extent 1 Extent 2 Extent 3 Extent 1 Extent 2

143. After-Imaging - aistall DB startup parameter that halts the database The /db/nxt.lg and /rd/tmp/aimage.log files will tell you why the db “locked up/stalled”. All database activity will freeze Un-halt/un-stall the database by running /rd/bin/aimage.archive The database will take off right where it left off without losing any data.

144. After-Imaging – The AI Timeline

145. Must maintain the timeline After backup, immediately turn on AI After restoring, immediately roll in the ai files The.ai file stores/logs the database datestamp The database must match the.ai file datestamp

146. After-Imaging – The AI Timeline Backup DB Switch to next.ai file Start AI DB Start Switch to next.ai file Turning AI On nxt.a2nxt.a3nxt.a4

147. After-Imaging – The AI Timeline Backup DB The Crash Switch to next.ai file DB Start The Crash nxt.a2nxt.a3

148. After-Imaging – The AI Timeline Restore The DB Just before DB crashed Roll in next.ai file Roll in first.ai file The Recovery nxt.a1 nxt.a2 Backup DB DB Start Truncate.bi file Start AI Roll in next.ai file nxt.a3

149. After-Imaging – The AI Timeline Common Mistakes Backup DB Switch to next.ai file Start AI DB Start nxt.a2nxt.a3 Truncate.bi file The Crash

150. After-Imaging – The AI Timeline Common Mistakes Restore The DB Roll in first.ai file nxt.a1 DB Start Roll in first.ai file nxt.a1

151. After-Imaging Recovery Copy (cp –p) the /ai/nxt.a* files to /aisave –Must do the copy before restoring the database or you Will Lose Data –They may contain data needed to recover the database Restore the database Roll-forward all of the /aisave/nxt.# and /aisave/nxt.a# files with /rd/bin/rollforward.ai The /aisave/aisave.log will tell you which /aisave/nxt.# file to start with. Do Not Do Anything to the database before rolling in the ai files!

152. After-Imaging Summary Must maintain the timeline Use the aimage.archive if your database “stalls” No harm comes by attempting to roll the wrong file Rollforward won’t let you roll the wrong file

153. Progress’ Backup Utility - probkup

154. Probkup - Benefits Backs up all necessary database files Block-by-block integrity check of database Block-by-block crc check of back up Supports Online backups

155. Probkup - How it works Quiets/Halts the database updates Backs up the.bi file Un-quiets the database Backs up the database block-by-block Verifies the database by doing a crc check of the backup files Sends a return code for success or error

156. 123 1234567891011 Database Blocks Backup Blocks Probkup - Online Backups

157. 123 1234567891011 Database Blocks Backup Blocks 10 Probkup - Online Backups

158. 123 1234567891011 Database Blocks Backup Blocks 10 45678911 Probkup - Online Backups

159. Probkup - Standards This is the only supported online backup utility NxTrend only supports online backups to disk NxTrend only supports full backups not incremental NxTrend requires the database to come down once a week for offline maintenance AI is not required for probkup or vice-versa

160. Probkup – backup.data daily A cron job for backup.data calls backup.online backup.online backs up the database to disk backup.online verifies the backup w/crc check backup.online sends return code to backup.data backup.data sends the disk backup and other files to tape via cpio The database did not come down!!

161. Probkup – backup.full weekly A cron job for backup.full calls backup.online backup.online backs up the database to disk backup.online verifies the backup w/crc check backup.online sends return code to backup.data backup.full shuts the database down backup.full runs offline maintenance utilities backup.full starts the database backup.full sends the disk backup and other files to tape via cpio

162. AI & Online Backups - Summary A great insurance policy The standard of the future…

163. Business Continuity - DRP Build a good DRP and test it Keep more than one copy of the DRP on hand.

164. Review: Question & Answer Session

165. Questions ? ? ? ? ? ? ? ? ? ? ? ?

166. McNaughton-McKay Progress 9.1D Training