Download presentation
Presentation is loading. Please wait.
Published byLily Mason Modified over 7 years ago
1
Creighton Hicks hicksc@us.ibm.com
Overview and Direction of the IBM Tivoli System Automation Product Family Creighton Hicks
2
Agenda Business Case Tivoli System Automation Product Family
Tivoli System Automation for z/OS Tivoli System Automation for Multiplatforms Base Pre-Canned Policy Templates Tivoli System Automation for Multiplatforms End-to-End What’s next on the Scope
3
Business Issue 40% 20% On demand challenges Customer pressures
Downtime unaffordable Heterogeneous by nature Different HW/SW platforms Cross-cluster application dependencies Complexity Customer pressures No end-to-end automation Cost and availability issues due to multiple automation and operations teams Education requirements Automation implementation and maintenance costs Rapid change of I/T infrastructure Brokerage Retail Sales Pay per view 1 2 3 4 5 6 7 Cost per hour for being Down in M$ Standish Group Application Failures Operator Errors Technology Failures 40% 20% Outage Causes IDC 2001 On this chart, you see how outages and complexity can affect your business. You will figure out that those pains are very, very similar to a lot of the issues we are trying to solve by our performance and availability products.The main topic is that our customers, especially in the on demand business environment, can really no longer afford down time. So it’s not just a question about losing business, it’s a question about losing customers and losing reputation. You may all remember, for example, the eBay crash two or three years ago where they got a huge revenue hit, I think it was about $3 million to $5 million. They got a huge loss of reputation, and their stock value also decreased a lot. for example, What you see here is that the costs in brokerage houses are about $6.5 million for one hour of being down. A lot of our customers are starting to deploy, new work loads, like mySAP, like WebSphere , and like customer relationship management tools. They have a need here to react very quickly to business needs. So they get changes very fast and they have to react very fast to get those stuff deployed on the systems. Those new work loads are often complex by themselves, already. Complexity is further increased by prerequisites like messaging, networking, directory and database access and by the need to make them web-based. Therefore a lot of workloads run on multiple platforms platforms. So you have applications running partly on a zSeries Box and partly on a Unix system or an xSeries or Windows system. This also creates problems with respect to operations skills. Based on the tools and the processes customers use currently, it’s very hard to figure out how the failure of a resource, let’s assume that a certain component in an SAP system crashes or your database is no longer working, impacts your whole business transaction and what the right steps are to get back on line. So , do you just have to restart the component or do you have to bring down the whole application and start right then. That is all very complex. You face a lot of situations where the people working on those problems don’t have really the understanding of all of the relationships of such a application resource, how to repair it and what it means to the business. That is also proven by several studies. On the left side, you see a pie chart and that 40% of all issues are really caused by operator errors and complexity is the main reason for that one here. So overall, it all comes down basically to a cost of ownership, speed to market, and competitiveness issue. So you either have to spend a lot of money, or you will lose customers against the competition, and lose your business. Therefore, we need to tackle this problem. Automation is the way to address some of those issues. Loss of business Loss of customers – the competition is just a mouse click away Loss of credibility, brand image and stock value
4
Towards a System Automation Family
SA MP (End-to-End Autom.) Cross Cluster Automation TSA for Multiplatforms SA MP 1.2 SA MP 2.2 One Product SA MP (Base) HA for Distributed Systems SA for Linux 1.1 SA MP 2.1 SA z/OS Automation and HA ESCON Manager SA z/OS ISCF ACA/ISCF TSCF AOC GDPS AOC 1.2 AOC 1.4 SA z/OS 2.2 SA z/OS 3.1 AOC 1.1 AOC 1.3 SA z/OS 2.1 SA z/OS 2.3 SA z/OS 3.2 AF/REMOTE ’87 ’88 ’ ’91 ’92 ’93 ’94 ’1995 ’96 ’97 ’98 ’ ’01 ’02 ’03 ’ ’06 ’07
5
Landscape “HA View” – Today often Bottom Up
Sample Bank HA Scope SAP DB2 Scripts,... CRM System HA Scope HA Scope J2EE Appl DB2 WAS Scripts,... WebSrv Trading System
6
Enterprise View – Top Down
Sample Bank CRM HA Scope SAP DB2 Dependency CRM System Trading System HA Scope HA Scope J2EE Appl DB2 WAS Dependency WebSrv Trading System
7
Automation and High Availability with IBM Tivoli System Automation for z/OS
8
SA z/OS’s Architecture
Automation Manager Knows sysplex wide configuraiton Drives automation decisions Implements an expert system tuned for automation concepts Applies automation concepts with a high abstraction agains abstract resources: Several group types Several dependency types Automation Agent Starts, stops, and monitors resources Abstract Resource Model Automation Manager CF Agent
9
High Availability with IBM Tivoli System Automation for Multiplatforms (Base Component)
10
Automation Manager RSCT
SA MP (base) - Mainframe-like High Availability for the Distributed World Exploits proven, leading pSeries & zSeries technology: AIX’s cluster infrastructure RSCT (Reliable Scalable Cluster Technology) provides functions like heartbeat, resource monitoring and control, … z/OS’ Automation Manager Expert system (hidden to the user) which drives automation decisions based on an automation policy Automation Manager Automation Manager Recovery RM Recovery RM Recovery RM Globalized Resources RSCT RMC RMC RMC GblRes RM FileSys RM Config RM .. other RM GblRes RM FileSys RM Config RM .. other RM GblRes RM FileSys RM Config RM .. other RM HAGS HAGS HAGS HATS HATS HATS Node 1 Node 2 Node 3
11
Tivoli System Automation – Basic Functionality
Request Changes Monitor Events Resources & Relationships & Rules Discovery Resources Relationships Automation Policy correlation rules Evaluate relationship Apply state change Find actions to reach goal Submit Action Translate event to a state Translate Action Decide Monitor and Control M A P E
12
SA MP (Base) Towards a Distributed World
PC Systems “classical” UNIX Servers zSeries Solaris on SUN-servers AIX and Linux on pSeries HP-UX on HP-servers Linux on BladeCenter Linux on iSeries Linux on PCs (xSeries and non-IBM PCs) Windows on PCs (xSeries and non-IBM PCs) Linux on zSeries z/OS on zSeries Currently under discussion SA MP (base) Currently under discussion SA MP (base) SA z/OS SA MP (base) SA MP (base) SA MP (base)
13
Simple Example: How do I automate my Cluster?
Step I: Define and start the cluster preprpnode charmhost1 charmhost2 mkrpdomain democluster charmhost1 charmhost2 startrpdomain democluster Step II: Define resources Step IIa: Define the WebServer resource mkrsrc IBM.Application Name="WebServer" StartCommand="/opt/IBMHTTPServer/bin/apachectl start" StopCommand="/opt/IBMHTTPServer/bin/apachectl stop" MonitorCommand="/root/build/FVT/Utils_386/ingcmon USERID root -PATH /opt/IBMHTTPServer/bin/httpd" MonitorCommandPeriod=5 MonitorCommandTimeout=5 NodeNameList="{'charmhost1','charmhost2'}" UserName="root" RunCommandsSync=0 Step IIb: define the WebIP resource mkrsrc IBM.ServiceIP Name="WebIP" IPAddress=" " NetMask=" " NodeNameList="{'charmhost1','charmhost2', '}" Step III: Define automation policy Step IIIa: Create a resource group DemoRG with members WebServer and WebIP mkrg DemoRG addrgmbr -m T -g DemoRG IBM.Application:WebServer addrgmbr -m T -g DemoRG IBM.ServiceIP:WebIP Step IIIb: Define a relationship (as example) mkrel -p DependsOn –S IBM.ServiceIP:WebIP -G IBM.Application:WebServer charmhost1 charmhost2 Resource Group: DemoRG WebIP Floating Resource: WebIP DependsOn WebServer Floating Resource: WebServer
14
Resource Types and Policy Elements
“Simple” Resourcess Serial fixed resource Serial floating resource Resource Group Is a collection of resources which are treated as one logical instance Is used to start, stop, and monitor Group status is an aggregation of its members‘s status Members can be Resources and Resource Groups Equivalency Relationships For start/stop sequence: StartAfter, StopAfter For dependend resources: DependsOn, DependsOnAny, and ForcedDownBy For placement constraints: Collocated, AntiCollocated, Affinity, AntiAffinity, IsStartable “Everything is a resource” Resource Group:RG_DB2 Floating Resource: ServiceIP Service IP Service IP DependsOn Floating Resource: DB2 DB2 DB2 DependsOn DependsOn Floating Resource: Mount Point Mount Point Mount Point Equivalency: Network NIC NIC NIC NIC Node 1 Node 2
15
Resource Types Serial Fixed = unique resource in the cluster
Serial Floating = one instance is started at one time. Fixed B Fixed D Serial Fixed Fixed A Fixed C Float E Fixed E1 Fixed E2 FixedE3 Serial Floating Float F Fixed F1 Fixed F2 Node 1 Node 2 Node 3
16
“chrg –o online <rg>” “chrg –o offline <rg>”
Resource Group Is a collection of resources which are treated as one logical instance Is used to start, stop, and monitor Start/stop of a group affects all its members Group status is the aggregation of its members’ status Resource Group Members can be Resources Resource Groups Start/Stop “chrg –o online <rg>” “chrg –o offline <rg>” Monitor “lsrg –g <rg>” RG_WebAppl Web Server Service IP File System
17
Resource Groups - Nesting
Start/Stop/Monitor “chrg –o online <rg>” “chrg –o offline <rg>” “lsrg –g <rg>” RG_AirlineReservation RG_WebAppl RG_XX RG_DB Web Server RG_XX1 DB Instance Appl C Appl E Service IP DB Data Appl D File System Service IP
18
Relationships Type I: Start sequence relationship
StartAfter StopAfter Type II: DependsOn relationships DependsOn DependsOnAny ForcedDownBy Type III: Location relationships Collocated AntiCollocated Affinity AntiAffinity IsStartable
19
Start Sequence Relationship
StartAfter Start sequencing Start Sequence Policy Appl A Start Sequencing Appl A StartAfter Service IP Service IP
20
DependsOn Relationships
Main Idea: Appl A depends on the function of Appl B If B is gone, A does not work DependsOn Start sequencing Force down behavior Stop sequencing Implicit Collocation Stop Sequence Force Down Start Sequence Policy on 1,2 Appl A Appl A Appl A DependsOn File System File System on 1,2 File System
21
DependsOn Relationships
Start sequencing Stop sequencing Force down behavior Implicit collocated DependsOnAny Policy on 1,2 Appl A DependsOn on 1,2 File System Policy on 1,2 Appl A DependsOnAny on 3,4 DB2
22
Location Relationships
Hard location constraints: Collocated relationship Resources must run on the same location AntiCollocated relationship Resource must run on different locations Soft location constraints Affinity relationship Resources should run on the same location AntiAffinity relationship Resources should run on different locations
23
Collocated Relationship
App App Policy IP IP on 1,2,3,4 Node 1 Node 2 App Collocated IP App on 1,2,3,4 IP Node 3 Node 4
24
AntiCollocated Relationship
Web Web Policy on 1,2,3,4 Node 1 Node 2 Web AntiCollocated SAP on 1,2,3,4 SAP Web Node 3 Node 4
25
Flexible Source/Target Definition
Possible to define relationships between... resource groups, resources, equivalencies ...in any combination RG_B RG_A RG_C Appl A Appl D Appl E Appl B Appl F
26
Cross Node Relationships
Relationships can be defined between resources running on different systems in the cluster RG_Dist1 Fixed B Fixed A Float E Fixed E1 Fixed E2 FixedE3 RG_Dist2 Fixed C RG_Dist2 Fixed D FixedF Fixed G Node 1 Node 2 Node 3
27
Pre-Canned Policy Templates
28
Why do we need Pre-canned Policy Templates?
“ Well, TSA MP provides very powerful automation policy concepts with a very high abstraction level. So, why are pre-canned policy templates needed? ” Good reasons for Pre-canned Policy Templates Customer asks for solutions not products Pre-canned policies simplify customer’s life and helps to save time E.g.: No policy development, less testing, fast enablement,… A customer often has no knowledge about application-specific HA requirements Needed to define automation policy A customer might not have knowledge about HA clustering
29
Pre-Canned Policy Template: An Example
Step I: Define and start the cluster preprpnode charmhost1 charmhost2 mkrpdomain democluster charmhost1 charmhost2 startrpdomain democluster Step II: Define resources Step IIa: Define the WebServer resource mkrsrc IBM.Application Name="WebServer" StartCommand="/opt/IBMHTTPServer/bin/apachectl start" StopCommand="/opt/IBMHTTPServer/bin/apachectl stop" MonitorCommand="/root/build/FVT/Utils_386/ingcmon USERID root -PATH /opt/IBMHTTPServer/bin/httpd" MonitorCommandPeriod=5 MonitorCommandTimeout=5 NodeNameList="{'charmhost1','charmhost2'}" UserName="root" RunCommandsSync=0 Step IIb: define the WebIP resource mkrsrc IBM.ServiceIP Name="WebIP" IPAddress=" " NetMask=" " NodeNameList="{'charmhost1','charmhost2', '}" Step III: Define automation policy Step IIIa: Create a resource group DemoRG with members WebServer and WebIP mkrg DemoRG addrgmbr -m T -g DemoRG IBM.Application:WebServer addrgmbr -m T -g DemoRG IBM.ServiceIP:WebIP Step IIIb Define a relationship (as example) mkrel -p DependsOn –S IBM.ServiceIP:WebIP -G IBM.Application:WebServer Step II (New): Use Pre-Canned Policy Template Step IIa: Fill out Configuration File Step IIb: Trigger Policy Generator charmhost1 charmhost2 Pre-Canned Policy Template Resource Group: DemoRG Configuration File # SAMBA example script_dir="/usr/sbin/rsct/sapolicies/samba" prefix="SA-samba-" daemonpath="/usr/lib/samba/classic" nodes="node-1 node-2" ip_1=" , " nieq_1="eth0:node-1,eth0:node-2" data_work="/mnt WebIP Floating Resource: WebIP Application-specific Policy Generator (script provided by IBM) DependsOn WebServer Floating Resource: WebServer
30
Using Pre-canned Policy Templates – For Free
Are provided by IBM for a growing set of applications Are “plug and play” Are easy to adjust to customer specific requirements Are tested by IBM Contain all resource definitions and policy definitions Provide a detail documentation Pre-canned policy templates are for free !!!! If no pre-canned policy is available for your specific application IBM helps Define your own policy
31
Pre-canned Automation Policy Templates (Linux / AIX)
Data Management DB2 8.x ESE DB2 8.x ESE DPF DB2 8.x HADR DB2 7.x WE, EE Oracle 9i Oracle 8i WebSphere WebSphere Application Server 6.0 WebSphere MQ Tivoli Products Tivoli Workload Scheduler CCMDB / TADDM Tivoli Storage Manager (TSM) TSM Client Tivoli Enterprise Console 3.8 SAP SAP Replicated Enqueue environment SAP Application Server Shared File Systems NFS Server NFS Client Samba Groupware Sendmail 8.11 Web Servers Apache Web Server IBM HTTP Server Currently under development: WebSphere Application Server 5.1 DP for mySAP DRBD SA MP End-to-End Component Tivoli Provisioning Manager
32
Combining Pre-canned Policy Templates
“Complete Application” mySAP + DB2 TSA MP’s powerful policy elements allow easy integration (e.g. nested resource groups, or cross-node relationships) Consider templates as a tool kid which can build up a “complete application” Start, stop, and monitor mySAP Pre-Canned Policy Template DependsOn DB2 Pre-Canned Policy Template Mount Point Service IP DB2 Floating Resource: Mount Point Floating Resource: DB2 Floating Resource: ServiceIP Resource Group: RG_DB2 DependsOn
33
Pre-Canned Policy Template for Oracle
34
Pre-canned Policy Template for DB2
TSA MP’s pre-canned policy is DB2’s default HA solution on Linux Shipped with every DB2 V8.2 CD (Linux) Includes a free TSA MP License for a two-node cluster Pre-canned policy template supports: DB2 8.x ESE, DB2 8.x ESE DPF (multi-partition) DB2 8.x HADR and DB2 7.x WE, EE Documentation Available: Highly Available DB2 Universal Database v8.x ESE/DPF using IBM Tivoli System Automation for Multiplatforms Automating DB2 HADR Failover using IBM Tivoli System Automation for Multiplatforms
35
DB2 Pre-Canned Policy Template
Resource Group: RG_DB2 Step I: Define and start cluster preprpnode <node1> <node2> ... mkrpdomain <domainname> <node1> <node2> ... startrpdomain <domainname> Step II: Use Pre-canned Policy Template cd /opt/IBM/dbs/V8.1/ha/salinux ./regdb2salin –a <DB2 Instance> m <MountPoint> i <IP Address> (Exception: The DB2 pre-canned policy template works with options instead of a config file) Floating Resource: ServiceIP Service IP Service IP DependsOn Floating Resource: DB2 DB2 DB2 DependsOn DependsOn Floating Resource: Mount Point Mount Point Mount Point Equivalency: Network NIC NIC NIC NIC Node 1 Node 2
36
Pre-canned Policy Template for SAP: (ABAP, J2EE, XI, or ADIN)
XI_ABAP_LOP_ENQ ES ServiceIP DependsOn MS GW CO SE StartAfter, IsStartable, AntiCollocated XI_ABAP_LOP_ENQREP ERS Collocated IfNotOffline StartAfter db2_db2lop_0-rg DB2 Instance DependsOn DB2_Mount ServiceIP XI_ABAP_LOP_siccps06_DV.. XI_ABAP_LOP_siccps07_D00 ABAP AS siccps07 StartAfter StartAfter StartAfter SAP_SYS_ROUTER ServiceIP DependsOn SAPROUTER SA-nfsserver-rg SAPNFS DependsOn SHARED DATA ServiceIP StartAfter Resource Group floating Resource (siccps06,siccps07) fixed Resource
37
Pre-canned Policy Template for TWS
Resource Group: TWS_mdm-rg IBM.Application: TWS-MDM-rs DependsOn Equivalency: TWS_server-equ IBM.Application: TWS-server1 IBM.Application: TWS-server1 IBM.Application: TWS-server2 . . . IBM.Application: TWS-server-rs IBM.Application: TWS-Appserv1-rs Resource Group: tws#-rg DependsOn IBM.Application: TWS-server-rs IBM.Application: TWS-Appserv1-rs Resource Group: tws#-rg DependsOn IBM.Application: TWS-server-rs IBM.Application: TWS-Appserv1-rs Resource Group: tws#-rg DependsOn . . . This is a pictorial representation of the SA MP policy. The double lined circles are resources that can move between nodes – In this case nodeA and nodeB. The single lined circles are fixed to a single node. The arrows represent a dependency from a resource to another. Here we have: Two fixed resource groups each with a TWS server resource (core processes: batchman, jobman, netman, mailman) that depends on the appserver (new to 8.3). The TWS appserver then depends on DB2. The equivalency and MDM floating resource allow exactly one functional TWS server to be promoted to be the master domain manager for the TWS network. The db2 resources are abstracted in the dark green square at the bottom. This solution leverages the already widely used HADR policies for SA MP. SA MP is the recommended HA solution for DB2 on both Linux and AIX. DependsOnAny Resource Group: db2hadr_TWS-rg From DB2 HADR Pre-canned Policy
38
Cross Cluster Automation with IBM Tivoli System Automation for Multiplatforms (End-to-End Automation Component)
39
Enterprise View – Top Down
Sample Bank CRM HA Scope SAP DB2 Dependency CRM System Trading System HA Scope HA Scope J2EE Appl DB2 WAS Dependency WebSrv Trading System
40
TSA MP E2E Automation Web based GUI (ISC based)
Main features Monitoring and problem analysis: Show… Operational status, location, resource details Actively show status changes Problems with resources Resource dependencies Topology Clusters and systems Group membership trees Operator instructions per resource Automation logs per domain (cluster) Filtered views … matching a specified name filter located on a specific system with errors or warnings with operator requests Operational tasks Activate/Deactivate Policy for an E2E domain Start/Stop resources Reset a resource from a non-recoverable error Exclude/Include a node from the automation Move a resource (selection in choice group)
41
Automation Concepts - Resource States
Each Resource has the following states: Observed State – Describes the current runtime state of the resource. For example: Online, Offline, Starting, Stopping, … Desired State – Describes the intended target state which is expressed within the automation policy or a result of requests. Possible values: Online or Offline Operational State – Gives more details about the current automation processing on this resource. For example: StartRequestPending Compound State – Provides a traffic-light indication to the operator if he has to react on some situation. Possible value: OK, Warning, Error, Fatal
42
Resource Types and Policy Elements
Reference Resource is a reference to a resource in a cluster Resource Group Basic Group Choice Group Relationships StartAfter, StopAfter, ForceDownBy SA MP (End-to-End Automation Component) Browser Client CRM SAP Ref StartAfter DB2 Ref Adapter Adapter SA MP (base) TSA z/OS SAP DB2 SAP Cluster (AIX/pSeries with SA MP) Sysplex I (Sysplex with TSA z/OS)
43
Architecture System Automation End-to-End Server
TSA AO Automation Engine Daemon process Requests and events from TSA AO engine WebSphere Application Server 6 TSA AO Automation Access EJBs WebSphere Portal Server H Requests against TSA AO engine T Browser Client T P TSA for Application Operations GUI Database S . .. Adapter Plug-in for TSA MP SA MP (Linux) e r v TSA AO Generic Adapter e Integrated Solutions Console (ISC) r . .. Adapter Plug-in for TSA MP SA MP (AIX) TSA AO Generic Adapter Data Warehouse TSA AO Generic Adapter JMS . .. Adapter Plug-in for TSA z/OS TSA. z/OS Common Event Infrastructure (CEI) TEC System Automation End-to-End Server
44
SA MP (End-to-End Automation Component) - Cross Cluster Automation
Adapter MSCS * New with SA MP 2.2 Adapter * New with SA MP 2.2 HACMP Adapter SA MP SA z/OS Adapter SA MP AIX Adapter Adapter SA MP z/OS on zSeries Other HA cluster prod. SA MP AIX Other HA cluster prod. Linux on PCs, BladeCenter Linux on zSeries Solaris HP-UX Linux on POWER (p,i) Windows PCs ( Windows / Linux ) UNIX ( AIX, Linux, Solaris, HP-UX ) zSeries ( Z/OS, zLinux)
45
Policy with Resources, Groups and Relationships
A simple XML Document describes your IT Environment Easy to edit – use any existing XML Editor System Automation can deal with changes in your IT Environment. Add Applications to Automation Create new groups Adapt relationships No Programming skill required ! <AutomationPolicy version="1.0"> <PolicyInformation> <PolicyName> End-to-End Demo Policy</PolicyName> <PolicyAuthor>Michael Atkins</PolicyAuthor> <PolicyDescription> This document contains a demo policy </PolicyDescription> </PolicyInformation> <ResourceReference name=“Ref A"> <DesiredState>Online</DesiredState> <Description> This is the reference to Application A </Description> <Owner>Bob Owens</Owner> <InfoLink> <ReferencedResource> <AutomationDomain> AIX_Cluster </AutomationDomain> <Name>Appl A</Name> <Class>ResourceGroup</Class> </ReferencedResource> </ResourceReference ….. <Relationship> <Source> Ref A </Source> <Type> StartAfter </Type> <Target> DB Ref </Target> </Relationship> <ResourceGroup name=“E2E_Resource_Group" > This is the group starting my Web-Service </Description> <Member> Ref A </Member> <Member> DB Ref </Member> </ResourceGroup> </AutomationPolicy> E2E_Resource_Group Ref A DB Ref
46
What’s next on the Scope?
47
Tivoli System Automation Roadmap
48
Need More Information? Creighton Hicks hicksc@us.ibm.com
Please contact me: Creighton Hicks
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.