Presentation is loading. Please wait.

Presentation is loading. Please wait.

OPS-12: Caring for an Ailing AppServer ™ Roy Ellis Principal QA Engineer.

Published byTamara Bennett Modified over 9 years ago

Similar presentations

Presentation on theme: "OPS-12: Caring for an Ailing AppServer ™ Roy Ellis Principal QA Engineer."— Presentation transcript:

1 OPS-12: Caring for an Ailing AppServer ™ Roy Ellis Principal QA Engineer

2 © 2008 Progress Software Corporation2 OPS-12: Caring for an Ailing AppServer Agenda  Anatomy of the AppServer  Diagnosing Problems  Help Treating your AppServer AppServer Broker ClientsClients AgentsAgents Agent clien t Listening Port

3 © 2008 Progress Software Corporation3 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer What is the OpenEdge ® AppServer?

4 © 2008 Progress Software Corporation4 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer  Broker Java TM process Handles communication from clients to agents Manages client and agent pools Writes to.broker.log Or What is the Progress ® AppServer?.broker.log

5 © 2008 Progress Software Corporation5 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer  Agent “C” process Handles requests and replies to the clients Communicates “STATUS” to broker Writes to.server.log Or What is the Progress AppServer?.server.log

6 © 2008 Progress Software Corporation6 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer  State-aware  State-reset  Stateless  State-free 4 Operating Modes One to one Many to few - Default

7 © 2008 Progress Software Corporation7 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer  Client to Broker gets redirected to Agent State-aware State-reset  Client to Agent through the Broker Stateless State-free 2 Connection Models

8 © 2008 Progress Software Corporation8 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer 2 Connection Models State-aware State-reset clien t AppServer Broker Agent 2 Connection Models Stateless State-free

9 © 2008 Progress Software Corporation9 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer AppServer Broker Thread Pools AppServer Broker Listening Thread L-3090 Client Threads C-0001 C-0002 C-0003 C-0004 C-0005 C-0006 … Server Threads S-0001 S-0002 S-0003 S-0004 S-0005

10 © 2008 Progress Software Corporation10 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer Broker communication with Agents Broker starts agent and waits for agent to respond, status set to: STARTING Once started the agent reports to broker and broker sets status to: AVAILABLE Client request agent, is redirected to agent port, broker sets status to: CONNECTING Client connects to agent, agent reports to broker, broker sets status to: CONNECTED Agent completes request, notifies broker, broker sets status to: AVAILABLE

11 © 2008 Progress Software Corporation11 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer [08/04/04@11:41:31.334-0400] P-001268 T-009968 2 AS AS Application Server connected with connection id: 172.30.1.47::asbroker1::3090::b0d80cae72::1191a17d1f0:-7ffc. (8358) [08/04/04@11:41:31.334-0400] P-001268 T-009968 2 AS 4GLTRACE Run performCardAuth "John Doe 1234567890 Visa -H localhost -S 6789" [Main Block - authcard.p] [08/04/04@11:41:31.334-0400] P-001268 T-009968 2 AS 4GLTRACE Run connectAuthSvc "-H localhost -S 6789 " [performCardAuth - authcard.p] [08/04/04@11:41:31.350-0400] P-001268 T-009968 2 AS 4GLTRACE Run createAuthMsg "John Doe 1234567890 Visa " [performCardAuth - authcard.p] [08/04/04@11:41:31.350-0400] P-001268 T-009968 2 AS 4GLTRACE Run sendAuthMsg "1000 Name=John Doe;Number=1234567890;Type=Visa " [performCardAuth - authcard.p] [08/04/04@11:42:07.397-0400] P-001268 T-009968 2 AS 4GLTRACE Run disconnectAuthSvc "1000" [performCardAuth - authcard.p] [08/04/04@11:42:07.428-0400] P-001268 T-009968 2 AS AS Application Server disconnected. (8360) Reading Log Files [08/04/04@11:41:31.334-0400] P-001268 T-009968 2 AS AS Application Server connected with connection id: 172.30.1.47: :asbroker1::3090::b0d80cae72::1191a17d1f0:-7ffc. (8358) AS 4GLTRACE Run connectAuthSvc “-H localhost -S 6789” [performCardAuth - authcard.p]

12 © 2008 Progress Software Corporation12 OPS-12: Caring for an Ailing AppServer Anatomy of the AppServer  AdminServer - Required JAVA process Manages all OpenEdge Brokers and Servers Writes to admserv.log  NameServer - Optional JAVA process Traffic manager from clients to AppServers Writes to.ns.log Supporting Processes

13 © 2008 Progress Software Corporation13 OPS-12: Caring for an Ailing AppServer Agenda  Anatomy of the AppServer  Diagnosing Problems  Help Treating your AppServer

14 © 2008 Progress Software Corporation14 OPS-12: Caring for an Ailing AppServer Diagnosing Problems $DLC/bin/asbman -name asbroker1 -start OpenEdge Release 10.1C01 as of Fri May 9 21:58:30 EDT 2008 Connecting to Progress AdminServer using rmi://localhost:20931/Chimera (8280) Searching for asbroker1 (8288) Connecting to asbroker1 (8276) Unable to start asbroker1 (8297) Startup Problems

15 © 2008 Progress Software Corporation15 OPS-12: Caring for an Ailing AppServer Diagnosing Problems ===================================================================== [08/05/18@20:05:25.933-0400] P-000283 T-Main 1 --- --- /usr1/stat/progress/101c/wrk/asbroker1.broker.log opened. [08/05/18@20:05:25.939-0400] P-000283 T-Main 1 --- --- Logging level set to =2 [08/05/18@20:05:25.940-0400] P-000283 T-Main 1 --- --- Log entry types activated: UBroker.Basic,[08/05/18@20:05:26.020-0400] P-000283 T-Main 2 UB Basic ubroker version: v101c (07-May-08) (8038) [08/05/18@20:05:26.314-0400] P-000283 T-L-9090 2 UB Basic Started listenerthread: L- 9090 (8043) [08/05/18@20:05:26.394-0400] P-000283 T-L-9090 1 UB ---------- Could not listen on port: 9090 (8044) [08/05/18@20:05:26.394-0400] P-000283 T-L-9090 1 UB ---------- java.net.BindException: Address already in use (errno:226) [08/05/18@20:05:26.394-0400] P-000283 T-L-9090 1 UB ---------- at java.net.PlainSocketImpl.socketBind(Native Method) [08/05/18@20:05:26.394-0400] P-000283 T-L-9090 1 UB ---------- at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359) [08/05/18@20:05:26.394-0400] P-000283 T-L-9090 1 UB ---------- at java.net.ServerSocket.bind(ServerSocket.java:325) [08/05/18@20:05:26.394-0400] P-000283 T-L-9090 1 UB ---------- at java.net.ServerSocket. (ServerSocket.java:186) Startup Problems -.broker.log

16 © 2008 Progress Software Corporation16 OPS-12: Caring for an Ailing AppServer Diagnosing Problems $DLC/bin/asbman -name asbroker1 -start Connecting to Progress AdminServer using rmi://localhost:20931/Chimera (8280) Searching for asbroker1 (8288) Connecting to asbroker1 (8276) Starting asbroker1. Check status. (8296) $DLC/bin/asbman -name asbroker1 -query Connecting to Progress AdminServer using rmi://localhost:20931/Chimera (8280) Searching for asbroker1 (8288) Connecting to asbroker1 (8276) Broker: asbroker1 not running (8313) Startup Problems

17 © 2008 Progress Software Corporation17 OPS-12: Caring for an Ailing AppServer Diagnosing Problems [08/05/18@20:15:21.807-0400] P-000435 T-S-0004 2 UB Basic Started server: /usr1/stat/progress/101c/dlc/bin/_proapsv -logginglevel 2 -logfile /usr1/stat/progress/101c/wrk/asbroker1.server.log -ubpid 435 -Ms 1 -logname asbroker1 -logentrytypes ASPlumbing,DB.Connects - logthreshold 0 -numlogfiles 3 -ASID 4 –ubpropfile /usr1/stat/progress/101c/dlc/properties/ubroker.properties -svrefresh - ipver IPv4 -db mydatabase (8108) [08/05/18@20:15:22.210-0400] P-000435 T-S-0004 1 UB ---------- Message from server : Startup Procedure Failed (8113) [08/05/18@20:15:22.241-0400] P-000435 T-S-0004 1 UB ---------- IOException reading message from server : java.io.EOFException: pipe to server broken (8117) [08/05/18@20:15:22.241-0400] P-000435 T-S-0004 2 UB Basic ERROR: cannot start server. (8100) Startup Problems -.broker.log

18 © 2008 Progress Software Corporation18 OPS-12: Caring for an Ailing AppServer Diagnosing Problems [08/05/18@20:29:13.991-0400] P-000667 T-000000 1 AS -- ** There is no server active for database mydatabase on -H cruz -S 6543. (704) [08/05/18@20:29:13.991-0400] P-000667 T-000000 1 AS -- Error initializing the application server. (5479) [08/05/18@20:29:13.992-0400] P-000667 T-000000 2 AS AS Application Server Shutdown. (5476) [08/05/18@20:29:15.001-0400] P-000668 T-000000 1 AS -- ** There is no server active for database mydatabase on -H cruz -S 6543. (704) [08/05/18@20:29:15.001-0400] P-000668 T-000000 1 AS -- Error initializing the application server. (5479) [08/05/18@20:29:15.002-0400] P-000668 T-000000 2 AS AS Application Server Shutdown. (5476) Startup Problems -.server.log

19 © 2008 Progress Software Corporation19 OPS-12: Caring for an Ailing AppServer Diagnosing Problems [08/04/03@13:03:33.470-0400] P-008824 T-004332 2 AS AS Starting application server for asbroker1. (5560) [08/04/03@13:03:33.501-0400] P-008824 T-004332 2 AS AS Application Server Startup. (5473) [08/04/03@13:03:33.814-0400] P-008824 T-004332 1 AS -- C:\tmp\ex08\roy\eg1\asstart.r Database sports2000 not connected. (1006) [08/04/03@13:03:33.814-0400] P-008824 T-004332 1 AS -- C:\tmp\ex08\roy\eg1\asstart.r startup procedure ended with an ERROR condition. (8025) [08/04/03@13:03:33.814-0400] P-008824 T-004332 2 AS AS Application Server Shutdown. (5476) Startup Problems -.server.log

20 © 2008 Progress Software Corporation20 OPS-12: Caring for an Ailing AppServer Diagnosing Problems  All “INITIAL” agents must start or the AppServer will fail to start Set initial agents to start to zero –“initialSrvrInstance=0” in ubroker.properties (IPv6 enabled on a machine can cause agents not to communicate with broker)  Add “–Djvmstart.debug=1” to the “jvmargs=“ in AdminServerPlugins.properties View $WRKDIR/jvmStart.log file Startup Problems

21 © 2008 Progress Software Corporation21 OPS-12: Caring for an Ailing AppServer Diagnosing Problems Broker Name : asbroker1 Operating Mode : State-reset Broker Status : ACTIVE Broker Port : 3090 Broker PID : 25319 Active Servers : 5 Busy Servers : 0 Locked Servers : 0 Available Servers : 5 Active Clients (now, peak) : (0, 6) Client Queue Depth (cur, max) : (0, 11) Total Requests : 124 Rq Wait (max, avg) : (259 ms, 11 ms) Rq Duration (max, avg) : (5543 ms, 24 ms) PID State Port nRq nRcvd nSent Started Last Change 25320 CONNECTED 02003 000020 000020 000020 May 19, 2008 06:28 May 19, 2008 07:28 25321 AVAILABLE 02018 000031 000031 000031 May 19, 2008 06:28 May 19, 2008 07:28 25323 AVAILABLE 02019 000014 000014 000014 May 19, 2008 06:28 May 19, 2008 07:28 25322 CONNECTED 02006 000026 000026 000026 May 19, 2008 06:28 May 19, 2008 07:28 25324 AVAILABLE 02021 000033 000033 000033 May 19, 2008 06:28 May 19, 2008 07:28 Phantom Agents

22 © 2008 Progress Software Corporation22 OPS-12: Caring for an Ailing AppServer Diagnosing Problems ps -ef | grep _proapsv statqa 25324 25319 0 06:28:47 ? 0:24 /usr1/stat/progress/101c/dlc/bin/_proapsv -logginglevel 2 -logf statqa 25323 25319 0 06:28:46 ? 0:56 /usr1/stat/progress/101c/dlc/bin/_proapsv -logginglevel 2 -logf statqa 25321 25319 0 06:28:45 ? 0:55 /usr1/stat/progress/101c/dlc/bin/_proapsv -logginglevel 2 –log Phantom Agents protrace.25320 protrace.25322

23 © 2008 Progress Software Corporation23 OPS-12: Caring for an Ailing AppServer Diagnosing Problems  Look at log files.broker.log.server.log Database log files where agent is connected  Increase logging levels srvrLoggingLevel=4 (extended) srvrLogEntryTypes=4GLTrace –UBNet - New for 10.1C Diagnosing Application problems

24 © 2008 Progress Software Corporation24 OPS-12: Caring for an Ailing AppServer Diagnosing Problems Broker Name : asbroker1 Operating Mode : Stateless Broker Status : ACTIVE Broker Port : 3090 Broker PID : 25319 Active Servers : 1 Busy Servers : 0 Locked Servers : 0 Available Servers : 1 Active Clients (now, peak) : (0, 1) Client Queue Depth (cur, max) : (0, 1) Total Requests : 11 Rq Wait (max, avg) : (11 ms, 11 ms) Rq Duration (max, avg) : (24 ms, 11 ms) PID State Port nRq nRcvd nSent Started Last Change 25320 SENDING 02003 000001 000001 00001 Apr 4, 2008 11:40 Apr 4, 2008 11:41 Hanging Agents

25 © 2008 Progress Software Corporation25 OPS-12: Caring for an Ailing AppServer Diagnosing Problems [08/04/04@11:41:31.334-0400] P-001268 T-009968 2 AS AS Application Server connected with connection id: 172.30.1.47::asbroker1::3090::b0d80cae72::1191a17d1f0:-7ffc. (8358) [08/04/04@11:41:31.334-0400] P-001268 T-009968 2 AS 4GLTRACE Run performCardAuth "John Doe 1234567890 Visa -H localhost -S 6789" [Main Block - authcard.p] [08/04/04@11:41:31.334-0400] P-001268 T-009968 2 AS 4GLTRACE Run connectAuthSvc "-H localhost -S 6789 " [performCardAuth - authcard.p] [08/04/04@11:41:31.350-0400] P-001268 T-009968 2 AS 4GLTRACE Run createAuthMsg "John Doe 1234567890 Visa " [performCardAuth - authcard.p] [08/04/04@11:41:31.350-0400] P-001268 T-009968 2 AS 4GLTRACE Run sendAuthMsg "1000 Name=John Doe;Number=1234567890;Type=Visa " [performCardAuth - authcard.p] [08/04/04@11:42:07.397-0400] P-001268 T-009968 2 AS 4GLTRACE Run disconnectAuthSvc "1000" [performCardAuth - authcard.p] [08/04/04@11:42:07.428-0400] P-001268 T-009968 2 AS AS Application Server disconnected. (8360) Diagnosing Agent problems

26 © 2008 Progress Software Corporation26 OPS-12: Caring for an Ailing AppServer Agenda  Anatomy of the AppServer  Diagnosing Problems  Help Treating your AppServer

27 © 2008 Progress Software Corporation27 OPS-12: Caring for an Ailing AppServer Help Treating your AppServer  -query Get status and history of AppServer broker and agents  -kill Shutdown an AppServer with busy or hanging agents May need to run twice in a row asbman

28 © 2008 Progress Software Corporation28 OPS-12: Caring for an Ailing AppServer Help Treating your AppServer  Available on PSDN  ABL source code available  Read and Sort multiple log files  Unsupported LogRead

29 © 2008 Progress Software Corporation29 OPS-12: Caring for an Ailing AppServer Help Treating your AppServer  Don’t connect using the startup parameters Any database down to a single database will cause your AppServer agents to be unresponsive  Connect using the “CONNECT” call in the srvrStartupProc= srvrConnectProc= srvrActivateProc=  Use “IF CONNECTED” ABL Database Connections

30 © 2008 Progress Software Corporation30 OPS-12: Caring for an Ailing AppServer Help Treating your AppServer  New in 10.1B  Described in ubroker.properties.README  Read the CAUTION! Use only when directed by Tech Support  Periodically force the shutdown of all agents and allow new agents to start when requested Lifespan

31 © 2008 Progress Software Corporation31 OPS-12: Caring for an Ailing AppServer Help Treating your AppServer OpenEdge Management

32 © 2008 Progress Software Corporation32 OPS-12: Caring for an Ailing AppServer D I S C L A I M E R Under Development  The next slide includes information about potential future products and/or product enhancements.  What I am going to say reflects our current thinking, but the information contained herein is preliminary and subject to change. Any future products we ultimately deliver may be materially different from what is described here. D I S C L A I M E R

33 © 2008 Progress Software Corporation33 OPS-12: Caring for an Ailing AppServer Help Treating your AppServer  New in 10.2A  Described in ubroker.properties.README  List of all clients known to the AppServer, includes: Host/IP Port Connected Agent More! List AppServer Connections

34 © 2008 Progress Software Corporation34 OPS-12: Caring for an Ailing AppServer Help Treating your AppServer  Technical Support  Knowledge Base (kbase)  PSDN Help from Progress

35 © 2008 Progress Software Corporation35 OPS-12: Caring for an Ailing AppServer AppServer Broker In Summary  Understand your AppServer  Diagnose your Problem  Care for your Ailing AppServer Agent clien t Listening Port ClientsClients AgentsAgents

36 © 2008 Progress Software Corporation36 OPS-12: Caring for an Ailing AppServer Relevant Exchange Sessions  OPS-15: What was Happening with my Database, AppServer, Operating System… Yesterday, Last Month, Last Year?  SOA-38: OpenEdge Middleware Road Map and Info Exchange  OPS-19: What is IPv6 and Why Should I care?  OPS-13: Building and Deploying a Highly Available Application

37 © 2008 Progress Software Corporation37 OPS-12: Caring for an Ailing AppServer Questions ?

38 © 2008 Progress Software Corporation38 OPS-12: Caring for an Ailing AppServer Thank You

39 © 2008 Progress Software Corporation39 OPS-12: Caring for an Ailing AppServer

Download ppt "OPS-12: Caring for an Ailing AppServer ™ Roy Ellis Principal QA Engineer."

Similar presentations

B7: Caring for an Ailing AppServer™

B7: Caring for an Ailing AppServer™
What’s that user doing? Rich Banville / Richard Shulman June 7, 2011 The OpenEdge Statement Cache.

What’s that user doing? Rich Banville / Richard Shulman June 7, 2011 The OpenEdge Statement Cache.
DEV-2: Getting Started with OpenEdge® Architect – Part I

DEV-2: Getting Started with OpenEdge® Architect – Part I
TDPS Wireless v1.8.1900. Enhancements E1 - Multi load E2 - Driver time scheduler.

TDPS Wireless v Enhancements E1 - Multi load E2 - Driver time scheduler.
OPS-7: Migrating your Distributed Application from V9 to OpenEdge ® 10 with (Almost) No Downtime Roy Ellis Principal QA Engineer.

OPS-7: Migrating your Distributed Application from V9 to OpenEdge ® 10 with (Almost) No Downtime Roy Ellis Principal QA Engineer.
DONE02: Where Did You Go Wrong? Diagnostics and Troubleshooting in OpenEdge Kris Murphy Senior Solution Engineer.

DONE02: Where Did You Go Wrong? Diagnostics and Troubleshooting in OpenEdge Kris Murphy Senior Solution Engineer.
DEV-13: You've Got a Problem, Here’s How to Find It

DEV-13: You've Got a Problem, Here’s How to Find It
Scale Up Access to your 4GL Application using Web Services

Scale Up Access to your 4GL Application using Web Services
Integrating OpenEdge® Applications with SonicMQ®

Integrating OpenEdge® Applications with SonicMQ®
A CHAT CLIENT-SERVER MODULE IN JAVA BY MAHTAB M HUSSAIN MAYANK MOHAN ISE 582 FALL 2003 PROJECT.

A CHAT CLIENT-SERVER MODULE IN JAVA BY MAHTAB M HUSSAIN MAYANK MOHAN ISE 582 FALL 2003 PROJECT.
Hands-On Microsoft Windows Server 2003 Networking Chapter 7 Windows Internet Naming Service.

Hands-On Microsoft Windows Server 2003 Networking Chapter 7 Windows Internet Naming Service.
Care and Feeding of the Pacific Application Server for OpenEdge in Production David Cleary Principal Software Engineer Progress Software.

Care and Feeding of the Pacific Application Server for OpenEdge in Production David Cleary Principal Software Engineer Progress Software.
DEV-14: Understanding and Programming for the AppServer™

DEV-14: Understanding and Programming for the AppServer™
What’s new in Progress Developer Studio for OpenEdge 11.0?

What’s new in Progress Developer Studio for OpenEdge 11.0?
NovaBACKUP 10 xSP Technical Training By: Nathan Fouarge

NovaBACKUP 10 xSP Technical Training By: Nathan Fouarge
ARCH-6: UML Modeling with Enterprise Architect Phillip Magnay Technical Architect.

ARCH-6: UML Modeling with Enterprise Architect Phillip Magnay Technical Architect.
A New Object Model for WebSpeed and HTTP

A New Object Model for WebSpeed and HTTP
Server Design Discuss Design issues for Servers Review Server Creation in Linux.

Server Design Discuss Design issues for Servers Review Server Creation in Linux.
Advanced Workflow. Agenda 1. Understanding the Overall Architecture 2. Setting up Outlook for Workflow 3. Launching Workflow from Applications 4. Getting.

Advanced Workflow. Agenda 1. Understanding the Overall Architecture 2. Setting up Outlook for Workflow 3. Launching Workflow from Applications 4. Getting.
OPS-17: Utilizing Firewalls - In the Reign of Fire

OPS-17: Utilizing Firewalls - In the Reign of Fire

Similar presentations

Ads by Google