1. Basic CA-ESP Workload Automation Course – Agent Overview

2. What You Will Learn What is an ESP System Agent
Basic Agent Architecture
How ESP System Agents Interact With CA ESP Workload Automation Engine
Basic Configuration Options
Defining non-mainframe workload
Value that ESP Agents can provide
ESP Agent for Databases

3. Product Components

4. ESP Agents Lean, non-invasive conduits (Approx 50MB disk space)
Extends CA ESP Workload Automation’s solutions across a variety of operating systems and ERP environments
Unlimited scalability, throughput
Each platform has unique ESP Agent
Manage through a single point of control, will be integrated with CA Job Management products (CA-7, CA-Autosys)

5. ESP System Agents - Scalability
Volumes and Concurrency
Large number of concurrent processes
Lab tested at 1,000 concurrent jobs, largest known field implementation 500 concurrent jobs
Highly scalable
Can keep up with volumes manager can handle

6. What is an ESP Agent? Processes work Notifies Manager of job status
Started
Running
Complete/Failed
Stores output from jobs in a spool file
Allows users to control workload
Waits for work
Persistence

8. Workload Objects Distributed OS Integration ERP Integration
File Management
Agent Monitoring
Web App Integration
OS Resource Integration
Database Integration
Mainframe Integration

9. Who Controls the ESP Agent?
Schedulers
Schedule jobs regardless of platform & Have control over entire enterprise
Operations
Monitor entire enterprise & Control every production job
System Administrators
Install ESP Agent & Maintain ESP Agent

10. ESP System Agent Architecture
ESP System Agents for UNIX Based Platforms are started processes.
ESP System Agents for Microsoft Windows are installed as Windows Services
All ESP System Agents Utilize SUN Microsystems JAVA Runtime Environment as the Base Architectural Component
All ESP System Agent Functions are Created as JAVA Plug-In’s

11. ESP System Agent Architecture
Spawns & Tracks Submitted Commands & Scripts
Processes & Monitors File Trigger & FTP Requests
Processes Machine Resource Monitoring Requests
Processes EJB & JMS Publish Subscribe Requests
Processes TCP/IP Communications & SNMP Requests
Job Execution JAVA Plug-In
File Monitoring JAVA Plug-In
Object Monitoring JAVA Plug-In
J2EE Execution JAVA Plug-IN
Agent Management JAVA Plug-In
SUN Java Runtime Environment Base Code

12. Communication
ESP Managers and ESP System Agents communicate asynchronously using message queues.
ESP Managers and Agents communicate by sending Automated Framework Messages or AFMs.
Most Agent commands deal with the control of these AFMs.

13. Communication
ESP Managers and ESP System Agents have Sender and Receiver Ports.
The receiver listens on a predefined TCP/IP port.
When the sender has messages to transmit, it connects to the receiver’s port, sends the messages, and then closes the connection.

14. Communication ESP Managers and Agents have sender and receiver ports.
Each Agent has one dynamic sender port and one receiver port.
The ESP Host can have multiple receiver ports (for example, to separate encrypted and non-encrypted message traffic) and has one dynamic sender port for each connected Agent.

15. ESP Agent Basic Communications

16. Communication Automated Framework Messages include:
ESP System Agent Name
Command or Script to Execute
UserId to Execute the Workload Under
Arguments for the Script or Command
Environment Variables
Job Execution Status

17. Communications Sample AFM Messages:
CM_DE54 DLEIGH_WIN_AGENT WINNT1/WLMDEMO5.20/MAIN State EXEC SetStart Status(Executing at DLEIGH_WIN_AGENT) Jobno(1860) User(DLEIGH_WIN_AGENT) Host(DLeigh)
CM_DE54 DLEIGH_WIN_AGENT WINNT1/WLMDEMO5.20/MAIN State COMPLETE Cmpc(0) SetEnd User(DLEIGH_WIN_AGENT) Host(DLeigh)
CM_DE54 DLEIGH_WIN_AGENT CYBDL01K/WLMDEMO3.3/MAIN State SUBERROR Failed SetEnd Status(Command file not found) Cmpc(20007) User(DLEIGH_WIN_AGENT) Host(DLeigh)

18. Basic Configuration
All ESP Agent Configuration Settings are stored in a single file called agentparm.txt
This file is created by the installation process and can be manipulated using any text editor
This Configuration File Defines the Name of the Agent, Ports Used, and Other Configurations that Pertain to Other Functions (FTP, J2EE, SNMP, etc.)

19. Basic Configuration Sample agentparm.txt File:
# ESP System Agent for Microsoft Windows parameters #
# Log
log.level=5
log.maxsize=
# Agent name
agentname=DLEIGH_WIN_AGENT
# Communications
communication.managerid_1=CM_DE54
communication.manageraddress_1=lparc
communication.managerport_1=6666
communication.monitorobject_1=DLEIGH_WIN_AGENT/AGENTMON1.0/MAIN
communication.managerid_2=ESPRESSO_CALYPSO_47500
communication.manageraddress_2=calypso
communication.managerport_2=47507
communication.monitorobject_2=DLEIGH_WIN_AGENT/AGENTMON2.0/MAIN

20. Basic Configuration Sample agentparm.txt File:
communication.managerid_3=ESPRESSO_ELVIS_7500
communication.manageraddress_3=elvis
communication.managerport_3=7507
communication.monitorobject_3=DLEIGH_WIN_AGENT/AGENTMON3.0/MAIN
communication.inputport=9900
communication.prefixlevel=2 #
# Security
security.filename=D:/Cybermation/ESP System Agent R6.1/security.txt
security.keystorage=D:/Cybermation/ESP System Agent R6.1/keys.txt
security.cryptkey=0x
security.level=off
# Initiators
# There will be separate line for each pair of <class, number of initiators>
# initiators.class_N, where N is number running from 1 to ...
# Class is user definable, (should be the same here and in AFM)
# Soft shutdown mode waits for all jobs to be completed
initiators.class_1=Default,1000

21. Basic Configuration Sample agentparm.txt File: # Persistence #
persistence.gcinterval=10000
persistence.level=2
# Core parameters (for plugins)
core.address=localhost
core.port=35800
# General characteristics
spooldir=./spool
COLD_START=false
# Runner plugin parameters
runnerplugin.managerport=35801
runnerplugin.requestport=35802
# Start JVMs
oscomponent.attachjvm=true

22. Basic Configuration Sample agentparm.txt File: #Path to the JRE
oscomponent.javapath=D:/Cybermation/ESP System Agent R6.1/jre/bin
#type of jvm (used only if attachjvm=true)
oscomponent.jvm=client
#What plugins we want to start by the core Java agent
plugins.start_internal_1=runner
plugins.start_internal_2=fileMon
plugins.start_internal_3=objmon
plugins.start_internal_4=management
plugins.start_internal_5=ftp
plugins.start_internal_6=j2ee
plugins.start_internal_7=router #
# SNMP specific
management.snmp.mibfile=D:/Cybermation/ESP System Agent R6.1/cybermib.txt
management.snmp.host=DLeigh
management.snmp.port=162
management.snmp.community=public

23. Basic Configuration Sample agentparm.txt File: # # FTP specific
# ftp.noserver=false
ftp.serverport=21
#ftp.client.ssl=true
#ftp.client.ssl.truststore=
#ftp.client.ssl.truststore.password=055A55EB863D2A5D
#ftp.server.ssl=true
#ftp.server.ssl.keystore=
#ftp.server.ssl.keystore.password=C8B98BFA BC0
# Windows Service name
oscomponent.servicename=ESP System Agent for Microsoft Windows R6.1
oscomponent.servicedisplayname=ESP System Agent for Microsoft Windows R6.1

24. Successful Communication depends on…
Agent Information that must match…
Agentparm.txt Topology in ESP dSeries
agentname
Name
communication.inputport
Agent Port number
communication.managerid
Manager Instance Host name
communication.manageraddress
DNS name or IP Address of dSeries Server
coomunication.managerport
ESP Server Manager Port
security.cryptkey
Encryption key used from server to agent

25. Test Your Memory
What Type of Workload Can an ESP System Agent Execute?
How Does the ESP System Agent Communicate with an ESP Server?
Windows Scripts, Windows Commands, FTP Processes, Machine Resource Monitors, J2EE Based Processes, etc…
Utilizing Automated Framework Messages over TCP/IP

26. Test Your Memory What is the Receiver Port Used for?
What Job States Can the ESP System Agent for Windows Send to an ESP Server?
The receiver listens on a predefined TCP/IP port. When the sender has messages to transmit, it connects to the receiver’s port, sends the messages, and then closes the connection.
READY, EXEC, COMPLETE, FAIL, SUBERROR, & SUBDELAY

27. Additional Automation capabilities

28. Approaches for use in your environment
Event-Driven Workload Automation
“Batch” Environment Monitoring
Run Book Automation

29. Event-Driven Automation
Trigger off new applications based on monitoring capabilities of the R7 Agent, using ESP Alerts:
Error message being written to a log file
A service coming online
Server’s CPU stuck at 100%
ESP:dSeries currently implemented as Jobs, also several monitors implemented as Event Triggers

30. “Batch” Environment Monitoring
The ability to monitor critical environmental dependencies of an ESP Application to ensure successful completion
Example: Find out you are going to run out of disk space hours before the server dies
Example: If you know you’re going to start running a job on your SAP server at 2am, and the database server for that job crashes at 10pm, why wait 4 hours for everyone to be surprised?
(and getting called in the middle of the night)
Simply another step towards better managed SLA’s

31. Run Book Automation
Latest fad in IT Operations that many of you, have been doing for years
Focus on managing (executing, controlling, monitoring) IT Operations tasks in a controlled, workflow-style manner much like workload automation
Example: Shutting down certain processes on a server, running maintenance, starting back up the processes
You can, naturally, do much of this with your existing ESP Scheduler and ESP Agents

32. FTP Workload Objects

33. Automating File Transfers
To schedule FTP workload, use the FTP_JOB job type.
You can automate file transfer with an FTP job. The job can use an existing FTP server or the Agent’s in-built FTP server.
Note: To use the Agent as an FTP server, you need to configure the Agent during installation or set the Agent parameter ftp.noserver to false. The Agent configured as an FTP server does not support anonymous file transfers

34. Automating File Transfers
In this sample, a file called /temp/cyberftp txt is downloaded from a UNIX machine (rem_unix) then copied to a local machine, a Windows PC. Note that the two locations include a complete path statement. After the download is complete, the job completes in ESP dSeries:

35. Properties for ‘Monitor’ Workload Objects

36. Adding Monitoring Capabilities to ESP
Starting with Release 6 of the ESP System Agent, new monitoring capabilities were introduced:
Windows
Windows Event Log Monitoring
Windows Services Monitoring
Windows/UNIX/OS400
Disk Space Monitoring
Process Monitoring
TCP/IP Address/Port Monitoring
CPU Monitoring
Text File Monitoring
J2EE
JMS Publish/Subscribe to Queues and Topics

37. Windows Services What is it?
This job type allows you to monitor Windows Services on a local machine
Sample Definition:

38. Windows Services
Service Name corresponds to the name of the Service as identified in the Services Application

39. Windows Services
Status refers to the state you with the Service to be in for the monitor to complete
Valid Statuses include: RUNNING, STOPPED, CONTINUE_PENDING, PAUSE_PENDING, START_PENDING, STOP_PENDING, EXISTS, NOTEXISTS
Can be coded with either WAIT or NOW. Now will return a COMPLETE/FAIL immediately, while WAIT will remain until the condition is met or the JOB is forced complete

40. Windows Event Log What is it?
A job to monitor the Event Log of a Local Windows Server. It can monitor any one of three types of logs
Application Log (Programs)
System Log (System Components, e.g. a Driver)
Security Log (Security Events like an invalid login, file access)

41. Windows Event Log

42. Windows Event Log
Sample Job Definition

43. Windows Event Log
EVENTLOG values are generally Application, Security, or System
EVENTTYPE values may be ERROR, WARN, INFO, AUDITS, AUDITF (AUDITS and AUDITF related to EVENTLOG Security only)
EVENTSOURCE value is typically generated by software vendor. Values with spaces requires quotes
EVENTCATEGORY represents a classification by the EVENTSOURCE, in this case, ‘Norton AntiVirus’ is catagorizing this as a Disk event.

44. Disk Space
What is it?
Allows you to find out how much disk space has been used or is free
Can be expressed in MB, GB, or as a Percentage
Can be CONTINUOUS monitored using an ESP Alert

45. IP Monitoring What is it?
It allows you to monitor specific IP addresses or IP Address/Port combinations to validate that network resources are accessible and listening ports are available
Valid Status include RUNNING and STOPPED and are monitored for
immediate state NOW or monitored until condition is met WAIT

46. CPU Monitoring What is it? Why would I want to do this?
Allows you to monitor CPU utilization to determine success/failure criteria of your job monitor
Why would I want to do this?
Provides you with a means of ensuring that the machine has sufficient CPU available before submitting workload to it
If a machine is in a continuously busy state, allows you to fire off notifications
CPU Monitoring is also tied into Physical Resources, which can be used for physical load balancing (required HPO for ESP:mSeries)
How is CPU utilization measured?
Windows uses CPU as recorded by the system
UNIX uses Load Average

47. CPU Monitoring
This example completes if the CPU Usage is greater than 80%.
Can be used to send warnings to System Administrators.

48. Text File Monitoring
Allows you to monitor INSIDE a text file for a specific string of text for a matching value
Monitoring has a great deal of flexibility with Regular Expressions

49. Text File Monitoring
This example monitors the file by lines for a text string, but only the first 20 lines

50. Text File Monitoring
Example using Regular Expressions:

51. Properties for JMS Workload Objects
JMS Monitoring

52. JMS Subscribe What is it? Java Messaging Service
A Java-based standard for connecting to, and communicating with, message brokers such as MQ Series, Sonic, WebLogic, or Tibco
Allows you to monitor a JMS Queue or Topic for a specific message that matches your criteria

53. JMS Subscribe JMSS_JOB job type Subscribe to Topics or Queues
Receive
JMS Provider
ESP
Agent
Topic or
Queue
Acknowledge
JMSS_JOB job type
Subscribe to Topics or Queues
Filter results

54. JMS Subscribe

55. JMS Subscribe A few uncommon terms INITIAL_CONTEXT CONNECTION_FACTORY
JNDIUSER

56. ESP Agent for Databases

57. ESP Database Agent
Provides integration to a variety of database platforms
Utilizes the agent plug-in architecture to adapt to the System Agent core
Written entirely in JAVA and the JDBC API

58. ESP Database Agent Features
SQL
Parameters
Events
Results
Results SP
TRG SP - +
TRG
SQL Updates and Queries
Stored Procedures
Table Monitor
Data Triggers

59. Event Driven SQL Statements
Event-driven execution of SQL commands i.e. Insert, update, delete, etc.
Success criteria specified through regular expressions
Output stored to file
ESP Agent for Databases
Results
SQL

60. Invoking Stored Procedures
Event-driven execution of stored procedures
Retrieve output values generated by the stored procedure
ESP Agent for Databases SP SP

61. Monitor Database to Trigger an ESP Event
Monitor a table for a net change in size
i.e. number records added or deleted which satisfy a user defined condition
Complete or selected (columns) records can be defined
One-time or continuous alerts
ESP Agent for Databases - +

62. ESP Agent for Databases
Database Triggers
Very granular notification of changes to a database table
Database generates the events as they occur
Excellent performance, with very low overhead
Generate an event for every record added, deleted, or modified which satisfies a user defined criteria
One-time or continuous alerts
ESP Agent for Databases
Events
TRG
TRG

63. Database Agent Technical Details
Supported databases
Oracle 9i & up
Microsoft SQL Server 2000 & up
IBM DB2

64. Database Agent Examples
This example uses SQL to Insert a line into a Table called ‘stores’ in the ‘pubs’ database

65. Questions?