McNaughton-McKay Progress 9.1D Training Jeffrey A. Brown - Technical Support Consultant

Agenda: SX.enterprise concepts Progress 9.x architecture Marc/scripts/configuration Maintenance

SX.enterprise Concepts

Client/server vs self serving clients Appserver

SX.enterprise Concepts Double click on SX.e icon –Prowin32 Database connection via code –Connecting to database –pf … –Autoserver connection Login box Status window – Appserver connection

User 1 _mprosrv Shared Memory prowin32.db

User 1AutoServer.db prowin32 _mprosrv Shared Memory

User 1 prowin32.db AutoServer _mprosrv Shared Memory

User 1 prowin32 AppServer.db AutoServer _mprosrv Shared Memory

Appserver Discussion

User 1 AppServerDatabase Appserver Discussion

User requests customer address

User 1 AppServerDatabase Appserver Discussion

User requests a price for a specific customer

User 1 AppServerDatabase Appserver Discussion

User 1 AppServerDatabase Appserver Discussion

User 1 AppServerDatabase Appserver Discussion

Appserver Discussion - Summary Users are connected to Appserver and Database Small requests do not use Appserver Larger requests go through Appserver Need to have large pipe between Database Server and Appserver Server

Progress 9.x Architecture

Progress 9.x Architecture - Summary Admin Server NameServer AppBrokers –AppServers Database Brokers –AutoServers

Progress 9.x Architecture.db

Progress 9.x Architecture Admin Server.db /rd/bin/admin.start

Progress 9.x Architecture Admin Server.db Name Server Starts

Progress 9.x Architecture Admin Server DB Broker.db Name Server Starts /rd/bin/dbbroker.start

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server Starts /rd/bin/appbroker.start

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server Registers

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Connects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Starts Connects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Disconnects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Connects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Connects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Disconnects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 Auto Server Connects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server Starts Connects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server Disconnects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server Connects

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server

Progress 9.x Architecture Admin Server App Broker DB Broker.db Name Server User 1 App Server Auto Server

Progress 9.x Architecture Firewall considerations –-minport –-maxport

Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall

Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall

Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall

Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall Cannot connect

Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall

Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall Port 1025+

Progress 9.x Architecture DB Broker.db User 1 Auto Server Firewall -minport -maxport

Progress 9.x Architecture - Summary Admin Server is the key Name Server directs users to App Brokers App Brokers spawn App Servers Database Brokers spawn Auto Servers Firewall considerations –-minport & -maxport –Nameserver - UDP

NxTrend Standard Ports

Most are defined in MARC Should be defined in the /etc/services file Test ports are the same using 7200 range

NxTrend Standard Ports nxt7100/tcp# SX database appsrv7120/tcp# SX appserver twl7160/tcp# TWL twlappsrv7161/tcp# TWL appserver twlsxappsrv7162/tcp# TWL SX appserver nxtadmsrv7180/tcp# adminserver port nxtadmsrvadmin7181/tcp# adminserver admin port nxtnamesrv7182/tcp# nameserver port

Storage Areas and Extents

Progress v8.x - Extents Extent Disk Storage - Files Extent

Progress v8.x – Database Extents nxt.db nxt.d1 nxt.d2 nxt.d3 nxt.d4 nxt.d5 nxt.d6 nxt.d7 nxt.d8 nxt.d9 nxt.d10

Progress v9.1x - Storage Areas Area 6Area 7Area 8 Extent Disk Storage - Files Table ATable B Index A-1Index B-1 Table C Table D Index C-1 Index D-1Index E-1 Table E Area 51 Extent Index E

Progress v9.1x – Database Extents Schema Area (7) trhead (9) trlines (11) trother (13) trsm (15) trstat (17) trtrans (19) trcust (8) trhead_idx (10) trlines_idx (12) trother_idx (14) trsm_idx (16) trstat_idx (18) trtrans_idx (20) trcust_idx

Progress v9.1x – Database Extents nxt.dbnxt_10.d1nxt_14.d1nxt_18.d1 nxt.d1nxt_10.d2nxt_14.d2nxt_18.d2 nxt_7.d1nxt_10.d3nxt_14.d3nxt_18.d3 nxt_7.d2nxt_11.d1nxt_15.d1nxt_19.d1 nxt_7.d3nxt_11.d2nxt_15.d2nxt_19.d2 nxt_8.d1nxt_11.d3nxt_15.d3nxt_19.d3 nxt_8.d2nxt_12.d1nxt_16.d1nxt_20.d1 nxt_8.d3nxt_12.d2nxt_16.d2nxt_20.d2 nxt_9.d1nxt_12.d3nxt_16.d3nxt_20.d3 nxt_9.d2nxt_13.d1nxt_17.d1 nxt_9.d3nxt_13.d2nxt_17.d2 nxt_13.d3nxt_17.d3

Progress v9.1x - Increased Size Limits Database size: 16,384,000 gigabytes Before-Image size: 32,768 gigabytes Number of areas: 1,000 Database block size: 8,192 bytes Extent Size: 2 gigabytes * *Unless db and filesystem are large file enabled

MARC / Scripts & Configs

Marc Demo new Marc

New Scripts / Script Templates

New Scripts and New Names Many new scripts have been added to MARC’s script base Many more scripts had a name change to standardize names and functions

The Progress Admin Server These scripts help manage the Progress Admin and Name Servers admin.start admin.shut admin.query nameserver.query

The Database These scripts help manage the database brokers dbbroker.start dbbroker.shut dbbroker.query These are database utilities dbstats truncate.bi

The Application Broker These scripts help manage the Progress Application Brokers appbroker.start appbroker.shut appbroker.query

Parameter Files The following are a new set of parameter files for client connections client.pf editor.pf batch.pf singleuser.pf connect.pf

The Environment These scripts help manage this particular environment nxt.env nxtall.start nxtall.shut

Enterprise To start SX.enterprise, now use sxe sxe sxee sxeqde

The Report Manager These are the Report Manager scripts rptmgr.start rptmgr.shut

Binary Dump and Load These scripts are for Dump and Loads easybdsareabd bddriver.psareabddriver.p bddumpall.pbddumparea.p bdloadall.pbdloadarea.p bdscript.psareabdscript.p

After Imaging These scripts are for AI aimage.start aimage.shut aimage.backedup aimage.list aimage.archive aimage.new aistats rollforward.ai truncate.ai

Backups These are new standard backup scripts backup.data backup.full backup.online

RxServer RxServer scripts are now generated by MARC rxsfax.sh rxsprint.sh out.sh

Other Scripts These don’t fit anywhere else rdclean sassi.sh shutuser showuser config_rpt

TWL TWL scripts have been mostly rewritten MARC generates all of the needed TWL scripts TWL scripts now follow a standard naming convention

Additional TWL Application Broker These scripts help manage the Application Broker that manages TWL connections to the SX.enterprise database twlappbroker.start twlsxappsrv twlappbroker.shut twlsxappsrv twlappbroker.query twlsxappsrv TWL uses the twlappbroker scripts which are generated from TWLMARC. These twlappbrokers are for connections to the TWL database.

TWL RF Units These scripts are used to start the TWL application on RF units twlrf twlentrf twlentrf.pf

TWL End of Day These scripts are used to run TWL End of Day processing. These are usually called from the backup scripts The first script performs processing that requires a single-user connection to the DB while the second performs tasks using a normal multi-user connection to the DB twleod1 twleod2

TWL Interfaces The following scripts are used to manage the TWL interfaces twlsend.starttwlrcvpack.sh twlsend.shtwlrcv.sh twlsend.shuttwlrcv.shut twlsend.querytwlrcv.query

The Templates The next section describes the new Script Generator in MARC

MARC Generate Scripts MARC Databases -> Configure -> Generate Scripts Select the database(s) to generate scripts for Select the scripts to generate

Script Template All scripts generated by MARC are stored as templates These templates contain the form of the final script Critical portions of a template contain tags These tags get substituted with real values to generate a script

MARC Generate Scripts All of the templates reside in subdirectories under /rd/marc/scripts/template All templates must end with a.tpl These subdirectories are: base dg hp ibm sco intchan taxware twl

All of the templates in base are added to the list One system directory is included based on the selected machine type in Installations -> Configure (e.g. ibm ) If optional products are setup in MARC, then those templates are also included in the list (e.g. twl ) The custom directory is always checked first and overrides a standard template of the same name MARC Generate Scripts

Script templates may also be in /rd/marc/scripts/cust in the same subdirectories Copy the template from the directory under /rd/marc/scripts/template Custom Scripts

Template Tags There are two types of template tags –System tags (defined by MARC) –Custom tags (defined by the user) Tags are names enclosed in ‘ ’ –Example: The convention is to use tag names that are all capital letters

System Tags System tags are defined during the installation of MARC System tags contain values that MARC can get from the information in its own database System Tags can not be edited directly with the tag editor

Custom Tags Users may use the Tag Editor to create and edit tags. Custom tags are created by users or may be in some delivered scripts (e.g. rxsprint.sh) Values for the tags can be assigned in two ways: –By answering the prompt when MARC finds an undefined tag –By using the tag editor in Databases -> Configure -> Tag Editor

The Tag Editor The tag editor allows the user to: –Create new custom tags –Modify existing custom tags –Delete custom tags

Example Template Here is a simple template echo “Hello, world!” Top two lines are reserved for formatting SCRIPTNAME is what the script will be called DESTDIR tells where the script will get created * Future (new to SX.e 4.0):

Script Naming All scripts should be prefaced with the environment name Use ENV before the script name to substitute in the environment name in the MARC that is generating the script

Destinations The destination directory can be a path The destination can be one of these predefined directories BIN OPSYS RD EXEC

Building A Script Here is a simple template “helloworld.sh.tpl” echo “Hello, world!” Should be created in the../cust/base directory Generating script to /rd/bin/

Script Templates - Summary All scripts will be generated by MARC and are stored as templates Should no longer hand edit scripts Script issues will begin by regenerating all scripts New scripts are easily added and existing ones easily updated

Progress 9.x Properties Files

No database parameter files for Progress 9.x Admin server reads from “properties” files. The properties files are stanza based. Marc has been rebuilt to handle modifications to these files. You may also use the Progress Explorer Tool to make modifications. DO NOT edit these files by hand!

Progress 9.x Properties Files V8 used many parameter files (/rd/opsys/nxtdb.pf) $DLC/properties/conmgr.properties –Database Broker –All Databases Registered $DLC/properties/ubroker.properties –Admin Server –Name Server –App Broker –Etc. DO NOT edit these files by hand!

Progress 9.x Properties Files Utilities to help validate your changes: ubconfig nsconfig asconfig (-name appsrv) dbconfig These utilities are in $DLC/bin

Report Managers/ Queues

In Graphical SX.e all printing is handled by Report Manager. New –param options for Report Manager Define different Report Managers for each Queue Prevents Report Managers from being too busy to pick up “on demand” jobs. Provides method of running jobs sequentially

Report Managers/Queues -param _d- Demand Jobs Only. _lXXX - Log File Name where “XXX” is the full path of the log file. _uXXXX- The report manager will only process jobs for the specified user. _pXXX - Pause Time, where “XXX” is the pause time in seconds. Can be the first or second parameter, but must precede the _q parameter where both are used. Do not set the pause time to zero. _qXXX- Queue Name where “XXX” is the name of a print queue. _x - This report manager will not support the blank queue.

Report Managers/Queues rptmgr - poepp User 1 rptmgr - oerd rptmgr - poepp rptmgr - arrp rptmgr - smrs

Report Managers/Queues Setup the queue in SX.e Copy the rptmgr.start script to a new script example: rptmgrOE.start Modify the new rptmgr.start command line –Sample -param with pause time of 5 seconds and queue name of OE that only picks up jobs for this queue: -param _p5_qOE_x Modify the nxtall.start to include the new rptmgr.start

Report Managers/Queues rptmgrOE User 1 rptmgrSM - smrs rptmgrPO - poepp rptmgr – porr rptmgr - arrp _p5_qOE_x _p5_qSM_x _p5_qPO_x _p5 _p5_d

Disconnecting Users - shutuser script

Using shutuser & showuser Risks in shutting a user out Never use kill -9

User 1 _mprosrv _progres u Shared MemoryVirtual tables u1.db ltp Syntax:# shutuser PID

User 1 _mprosrv prowin32 Server u Shared MemoryVirtual tables u1.db ltp Syntax: # shutuser 0001sys

Progress 9.x Log Files

New log files in Progress 9.x $DLC/bin/proenv –$WRKDIR –exit out when done Most logs are found in /rd/tmp Some are in /db/sort

Progress 9.x Log Files admserv.log –exceptions.log –default.log –AdminServerSystemOut.log NS1.ns.log appsrv.broker.log / appsrv.server.log rptmgr#### rptmgr.err

Progress 9.x Log Files dbstats.log –Summary –Empty blocks –Free blocks

Progress Monitor Utility - promon

Promon Progress Monitor utility $DLC/bin/promon Useful for performance monitoring as well as troubleshooting Database broker must be up to access promon Hidden menu R&D There are also hidden options in the hidden menus

Promon - Main Menu 1 - User control –used to see who is logged in 3 - Block Access –used to see number of records accessed –useful in troubleshooting hung/slow queries 4 - Record Locking Table –useful in troubleshooting locking issues related to code

Promon - Main Menu - Continued 5 - Activity –The most useful screen –The performance monitor screen –Displays number of transaction commits –Displays number of checkpoints –Displays number of buffers flushed –Displays percent APW writes –Displays percent BIW writes –Displays percent Buffer Hits (buggy)

Promon - Main Menu - Continued 5 - Activity - Continued –Displays the database size (includes fixed extents) –Displays the BI file size (includes extents) –Displays amount of DB Broker Shared Memory –Displays number of users logged in and how attached –Displays number of APW’s that are running –Displays number of AutoServers that are running

Promon - Main Menu - Continued 6 - Shared Resources –Displays db parameters –Displays lock table high water mark 7 - Database Status –Displays database block size –Displays the database high water mark –Displays number of free blocks below highwater mark –Displays BI block & cluster size –Displays database creation date (if multi-volume)

Promon - R&D Main Menu 1 - Status Displays 2 - Activity Displays 3 - Other Displays 4 - Administrative Functions

Promon - R&D, 1 - Status Displays 3 - Servers –Displays status information for Login Brokers (LB) & Auto Servers (AS) –Displays PID for active LB & AS –Displays active/inactive –Displays number of login attempts –Displays number of current logins –Displays maximum number of logins –Displays port numbers for active LB & AS

Promon - R&D, 1 - Status Displays - Continued 4 - Processes/Clients … –2 - Blocked Clients Used for troubleshooting “deadly embrace” locks Useful for troubleshooting “hung” processes –3 - Active Transactions Useful for troubleshooting unusual BI file growth Displays active transactions per user Displays information about each active transaction

Promon - R&D, 2 - Activity Displays 1 - Summary –Displays same info as Main Menu option #5 3 - Buffer Cache –use to correctly calculate Buffer Hits % 5 - BI Log –Displays BI file activity –useful in troubleshooting unusual BI file growth 13 - Other –Displays Wait on Semaphore information –useful in tuning -spin

Promon - R&D, 3 - Other Displays 1 - Performance Indicators –Useful in performance tuning 2 - I/O Operations by Process –Displays historical record read information per process –useful in troubleshooting lengthy processes 4 - Checkpoints –Displays historical checkpoint statistic information –useful in tuning checkpoint performance

Promon - R&D, 4 - Administrative Functions 4 - Adjust Latch Options –Used to tune -spin on the fly –Can only use this if -spin is set to at least 1

AI & Online Backups

After-Imaging (AI)

After-Imaging Progress’ after-imaging feature lets you recover a database if you lose the database or before image (BI) files. AI DB

Roll-Forwarding/ point-in-time recovery With roll-forward recovery, Progress uses data in the AI files to automatically reprocess all the transactions since the last good backup.

After-Imaging - How does it work? DB BI AI Shared Memory

After-Imaging - How does it work? DB BI AI Shared Memory

After-Imaging - How does it work? DB BI AI Shared Memory

After-Imaging - How does it work? DB BI AI Shared Memory

After-Imaging - How does it add up? AI files + Restored DB = DB just before the crash

DB BI AI Crashed: DB After-Imaging – How does recovery work?

DB BI AI Delete: DB & BI After-Imaging – How does recovery work?

DB BI AI Restored: DB & BI

After-Imaging – How does recovery work? DB BI AI Rollforward: AI to DB

After-Imaging – How does recovery work? DB BI AI Shared Memory No Data Lost!!!

Progress After-Imaging - The Setup

After-Imaging - The setup 5 fixed AI extents in /ai –round-robin the fixed ai extents aimage.archive moves full extents to /aisave Extent 1 Extent 3 Extent 2 Extent 5 Extent 4

After-Imaging - The key to it’s success /ai/aisave nxt.a1 nxt.a2 nxt.a3 nxt.1 nxt.2 nxt.3 nxt.4 nxt.5 Extent 1 Extent 2 Extent 3 Extent 1 Extent 2

After-Imaging - aistall DB startup parameter that halts the database The /db/nxt.lg and /rd/tmp/aimage.log files will tell you why the db “locked up/stalled”. All database activity will freeze Un-halt/un-stall the database by running /rd/bin/aimage.archive The database will take off right where it left off without losing any data.

After-Imaging – The AI Timeline

Must maintain the timeline After backup, immediately turn on AI After restoring, immediately roll in the ai files The.ai file stores/logs the database datestamp The database must match the.ai file datestamp

After-Imaging – The AI Timeline Backup DB Switch to next.ai file Start AI DB Start Switch to next.ai file Turning AI On nxt.a2nxt.a3nxt.a4

After-Imaging – The AI Timeline Backup DB The Crash Switch to next.ai file DB Start The Crash nxt.a2nxt.a3

After-Imaging – The AI Timeline Restore The DB Just before DB crashed Roll in next.ai file Roll in first.ai file The Recovery nxt.a1 nxt.a2 Backup DB DB Start Truncate.bi file Start AI Roll in next.ai file nxt.a3

After-Imaging – The AI Timeline Common Mistakes Backup DB Switch to next.ai file Start AI DB Start nxt.a2nxt.a3 Truncate.bi file The Crash

After-Imaging – The AI Timeline Common Mistakes Restore The DB Roll in first.ai file nxt.a1 DB Start Roll in first.ai file nxt.a1

After-Imaging Recovery Copy (cp –p) the /ai/nxt.a* files to /aisave –Must do the copy before restoring the database or you Will Lose Data –They may contain data needed to recover the database Restore the database Roll-forward all of the /aisave/nxt.# and /aisave/nxt.a# files with /rd/bin/rollforward.ai The /aisave/aisave.log will tell you which /aisave/nxt.# file to start with. Do Not Do Anything to the database before rolling in the ai files!

After-Imaging Summary Must maintain the timeline Use the aimage.archive if your database “stalls” No harm comes by attempting to roll the wrong file Rollforward won’t let you roll the wrong file

Progress’ Backup Utility - probkup

Probkup - Benefits Backs up all necessary database files Block-by-block integrity check of database Block-by-block crc check of back up Supports Online backups

Probkup - How it works Quiets/Halts the database updates Backs up the.bi file Un-quiets the database Backs up the database block-by-block Verifies the database by doing a crc check of the backup files Sends a return code for success or error

Database Blocks Backup Blocks Probkup - Online Backups

Database Blocks Backup Blocks 10 Probkup - Online Backups

Database Blocks Backup Blocks Probkup - Online Backups

Probkup - Standards This is the only supported online backup utility NxTrend only supports online backups to disk NxTrend only supports full backups not incremental NxTrend requires the database to come down once a week for offline maintenance AI is not required for probkup or vice-versa

Probkup – backup.data daily A cron job for backup.data calls backup.online backup.online backs up the database to disk backup.online verifies the backup w/crc check backup.online sends return code to backup.data backup.data sends the disk backup and other files to tape via cpio The database did not come down!!

Probkup – backup.full weekly A cron job for backup.full calls backup.online backup.online backs up the database to disk backup.online verifies the backup w/crc check backup.online sends return code to backup.data backup.full shuts the database down backup.full runs offline maintenance utilities backup.full starts the database backup.full sends the disk backup and other files to tape via cpio

AI & Online Backups - Summary A great insurance policy The standard of the future…

Business Continuity - DRP Build a good DRP and test it Keep more than one copy of the DRP on hand.

Review: Question & Answer Session

Questions ? ? ? ? ? ? ? ? ? ? ? ?

McNaughton-McKay Progress 9.1D Training