© 2008 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. BSM(OMI) 9.2X ENRICHMENT MANAGER AND TROUBLESHOOTING Remzi Cirit AMX BSM(OMi) SWAT Team Based Presentation Dermer Maria

© Copyright 2012 Hewlett-Packard Development Company, L.P.2 AGENDA –An Introduction to CMS and RTSM –An Introduction to Enrichments –Enrichments & the Enrichment Manager –The Enrichment Process –Creating Enrichments – Step by Step –The Enrichment TQL –Troubleshooting

© Copyright 2012 Hewlett-Packard Development Company, L.P.3 ENRICHMENT RULES CAN BE USED FOR SEVERAL PURPOSES: –Adding new CIs and relationships to the RTSM. –Deleting specific CI instances from the RTSM. –Updating the attribute values of specific CI instances in the RTSM. –Enrichment query nodes and relationships differ from other query nodes and relationships in that they are conceptual deductions that represent real relationships and CIs that cannot be discovered automatically by the discovery process. 3

© Copyright 2012 Hewlett-Packard Development Company, L.P.4 INTRODUCTION TO UCMDB

© Copyright 2012 Hewlett-Packard Development Company, L.P.5 HP CMDB TERMINOLOGY –HP Configuration Management Database (CMDB) is called: The Universal Configuration Management Database (uCMDB) –One of Universal CMDB’s key differentiators is its native integration with the HP Data Flow application –Universal CMDB federates to trusted data sources and power- consuming applications to form a Configuration Management System (CMS)

© Copyright 2012 Hewlett-Packard Development Company, L.P.6 HP CMDB TERMINOLOGY (CONT.) –In Business Service Management 9x the CMDB is called the Run-time Service Model (RTSM) –Another Synonym that was used in version 9.0 is ODB or Operational Database. ODB is a deprecated term for RTSM –In an environment of multiple CMDBs the term CMS is used to describe the data store and its processes. CMS is the central RTSM server and the authority for configuration management in a solution spanning across multiple CMDBs.

© Copyright 2012 Hewlett-Packard Development Company, L.P.7 AN EXAMPLE OF A VIEW

© Copyright 2012 Hewlett-Packard Development Company, L.P.8 RTSM OVERVIEW (CONT.) The RTSM also contains: –The TQL query definitions that are used to query and retrieve data from the RTSM, for presentation in the views –The configuration item type (CIT) model – A repository for the CI types used to define the CIs and their relationships –The object repositories for the additional data added to CIs in Service Health and Service Level Management, such as KPIs, shortcut menus, and tooltips

© Copyright 2012 Hewlett-Packard Development Company, L.P.9 POPULATING THE RTSM –Dynamic discovery performed by many data collectors –Synchronization with other systems such as a CMS RTSM BPM RUMDiagTV BPI App Service Transactions SiSOMNNMi J2EE.Net Virtualization Databases Middleware App, Service Txns J2EE servers Web servers Databases J2EE.Net Databases Middleware Mainframe Business Processes BP step BP scope Transaction J2EE Web servers Databases Virtualization AD Exchange J2EE Virtualization Clustering Routers Switches L2 connectivity DDMa Discovery Red border indicates discovery data source.

© Copyright 2012 Hewlett-Packard Development Company, L.P.10 RTSM FEATURES – DATA ENRICHMENT –Enrichments provide the functionality to automate processes of addition, update and deletion of CIs and relationships

© Copyright 2012 Hewlett-Packard Development Company, L.P.11 RTSM MODELING TAB –Modeling enables you to logically organize and manage the content of your IT Universe model. –RTSM provides tools needed to build and present logical displays using a hierarchical tree format that represents your IT organization.

© Copyright 2012 Hewlett-Packard Development Company, L.P.12 MODELING TAB OVERVIEW –IT Universe Manager enables you to manage the content of your IT Universe model. The following tasks represent the basic functionality of the application: Creating CIs without creating relationships – You can define new CIs that are added to the CMDB individually, so that no new relationships are created. Creating CIs with relationships for them – You can define new CIs and attach them to existing CIs, so defining a new CI and a new relationship in the CMDB. Adding existing CIs and creating relationships for them – You can attach CIs to an existing CI, so creating new relationships for existing CIs. Deleting CIs– You can delete CIs from the CMDB –Modeling Studio is a tool for creating and managing views. You can create your own views, or work with the out-of-the-box (factory) views provided with your BSM installation. The IT Universe model in the configuration management database can be very large, holding thousands of CIs. A view enables you to build a subset of the overall IT Universe model, containing only those CIs relating to a specific area of interest. You can define your own views to display only the information that is relevant to your organization’s business needs.

© Copyright 2012 Hewlett-Packard Development Company, L.P.13 MODELING TAB OVERVIEW (CONT.) –Impact Analysis Manager: Enables you to define the interdependencies between components in your IT infrastructure to simulate how infrastructure changes can impact your system. By creating Impact rules based on TQL queries, you can determine the true origin of a problem (root cause) and its business impact. –CI Type Manager This page enables you to view the information in the CI Type model, which contains the definitions of all CITs defined in the system and the relationships that define the connections between them Each CIT has its own attributes, as well as the attributes inherited from its parent CIT.

© Copyright 2012 Hewlett-Packard Development Company, L.P.14 MODELING TAB OVERVIEW (CONT.) Enrichment Manager: This page enables you to define Enrichment rules. These rules, which are based on specific TQL queries, can be used for several purposes, including: –Adding new CIs and relationships to the RTSM –Deleting specific CI instances from the RTSM –Updating the attribute values of specific CI instances in the RTSM Enrichment query nodes and relationships differ from other query nodes and relationships in that they are conceptual deductions that represent real relationships and CIs that cannot be discovered automatically by the discovery process. Enrichment query nodes and relationships are created as part of a TQL query, whose other TQL query nodes are regular ones, meaning TQL query nodes that already exist in the RTSM.

© Copyright 2012 Hewlett-Packard Development Company, L.P.15 THE ENRICHMENT MANAGER

© Copyright 2012 Hewlett-Packard Development Company, L.P.16 AN INTRODUCTION TO ENRICHMENTS –Enrichments provide the functionality to automate processes of addition, update and deletion of CIs and relationships –Enrichments are based on TQLs –Together with other discovery, web services and manual processes Enrichments take an important part in the population of the RTSM. Business Service Node Node element Usage Compositio n Condition Node Containment Result Enrichment

© Copyright 2012 Hewlett-Packard Development Company, L.P.17 AN ENRICHMENT RULE CONSISTS OF A SPECIALIZED TQL THAT CONTAINS CITS AND RELATIONSHIPS ON WHICH AN OPERATION IS DEFINED. –An Enrichment Rule consists of a specialized TQL that contains CITs and relationships on which an operation is defined. They differ from other TQL nodes and relationships as they allow for modifications to be applied to the CMDB. These modifications correspond to conceptual deductions that represent real relationships and CIs that cannot be discovered automatically by the discovery process. –Enrichment rules are used to add, update and delete CIs and relationships. They may be used to add CIs that represent non- discoverable information such as lines of business or business user CIs. They may also update infrastructure CIs to include contact information and locations. Additionally, enrichment rules may be used to delete redundant information in the CMDB. –Once the CI or relationship is updated by an enrichment, it is treated the same as any other information stored in the CMDB. 17

© Copyright 2012 Hewlett-Packard Development Company, L.P.18 AN INTRODUCTION TO THE ENRICHMENT MANAGER –The Enrichment Manager, within the Modeling section of the RTSM provides the functionality to manage, create, edit, activate and deactivate enrichment rules. –Accessing the ITUM is accomplished through: RTSM  Enrichment Manager

© Copyright 2012 Hewlett-Packard Development Company, L.P.19 THE ENRICHMENT MANAGER USER INTERFACE 19 The TQL Editor The Enrichment Explorer CI Type Selector CI Type Selector The Information Pane

© Copyright 2012 Hewlett-Packard Development Company, L.P.20 EXAMPLE OF AN ATTRIBUTE CONDITION THAT ENABLES A NODE TO BE IDENTIFIED BY ITS IP ADDRESS: –This attribute condition is the first attribute condition you need to define in the Attribute tab to enable the query node of type node to be identified by its IP Address. –Attribute name — Node is Complete –Operator — Equal –Value — False –This is the second attribute condition you need to define in the Attribute tab to enable the query node of type node to be identified by its IP address. –Attribute name — Node is Complete –Operator — Is null –Value — Value cannot be changed –Define the Enrichment Rule 20

© Copyright 2012 Hewlett-Packard Development Company, L.P.21 ENRICHMENT COMPONENTS –The TQL query, of an “Enrichment” type –Enrichment defined actions –Note: There is no rollback for all enrichments, therefore, enrichments should be tested in the TEST environment and the RTSM should be backed up prior to use Once Enrichments are saved and set to active- They begin to run automatically

© Copyright 2012 Hewlett-Packard Development Company, L.P.22 SCHEDULING ENRICHMENTS –The scheduler is accessed via ODB Administration > Administration > Scheduler

© Copyright 2012 Hewlett-Packard Development Company, L.P.23 SCHEDULING ENRICHMENTS (CONT.) Job Definition Schedule Definition Action Definition

© Copyright 2012 Hewlett-Packard Development Company, L.P.24 ENRICHMENT STEPS –The Enrichment rule defines the structure and properties of the enrichment. This includes the name, description and associated TQL. 1. The TQL query retrieves the result set of CIs and relationships on which the enrichment operations act. 2. The enrichment nodes and relationships, also called target nodes and relationships, are the targets of the enrichment operations. 3. The enrichment operations are update, add and delete. They are applied to the enrichment nodes and relationships. 24

© Copyright 2012 Hewlett-Packard Development Company, L.P.25 ENRICHMENTS: THE ENRICHMENT PROCESS –Back up RTSM DB –Create the Enrichment Definition –Verify the query and results –Define the operation and values –Activate and Save –Schedule It –If the Enrichment is to run periodically (not often) – Remove the Enrichment Rule after use 25 Day 3

© Copyright 2012 Hewlett-Packard Development Company, L.P.26 ENRICHMENTS: THE ENRICHMENT PROCESS –By default the rule is inactive –You can check the Rule is Active checkbox during the process or later by right click on the rule and select properties 26 Day 3

© Copyright 2012 Hewlett-Packard Development Company, L.P.27 ENRICHMENTS STEP BY STEP #1: CREATING AN ENRICHMENT –Name –Description –Based on TQL –Active Flag –TQL Priority –TQL Description 27 Day 3

© Copyright 2012 Hewlett-Packard Development Company, L.P.28 –To create a new Enrichment rule, click the New icon. In the Create New Enrichment Rule dialog, add the following information: –Name: name of the enrichment rule. –Description: an optional brief description of the purpose of the enrichment. –Based on TQL: this field takes the name of the Enrichment rule by default. To change the name, select the default name and type the new name over it. If an Enrichment TQL already exists, select the checkbox. A list will be shown in place of the name edit field. Browse the list for the name of the Enrichment TQL and select it. –Active: unchecked by default. As soon as the Enrichment rule is saved, it will be executed against the CMDB. It is recommended to keep the rule inactive until such time as you are ready to run the rule. –TQL Priority: sets the TQL execution priority. –TQL Description: an optional description of the TQL. 28

© Copyright 2012 Hewlett-Packard Development Company, L.P.29 ENRICHMENTS STEP BY STEP #2: CONFIGURING THE TQL QUERY 29 Day 3 Nodes Relationships Parameters –Attribute –Cardinality To build the TQL query, select the TQL Editing pane option. Add nodes and relationships to define the query. Remember, the purpose of the query is to define the set of CIs and relationship on which the enrichment operations will act.

© Copyright 2012 Hewlett-Packard Development Company, L.P.30 ENRICHMENTS STEP BY STEP #2A: CONFIGURING THE TQL QUERY 30 Day 3 The following limitations exist in Enrichment TQLs:  TQLs may not contain any self relationships  TQL structure cannot form a closed circuit (loop)  All nodes must be linked (no unconnected nodes)  There must be two nodes at the minimum  All Nodes to be enriched must be required and must have a cardinality of at lease 1..*

© Copyright 2012 Hewlett-Packard Development Company, L.P.31 NOTE –Any node to which an enrichment operation is applied must be a required element. The node cannot be optional. That is, it cannot be linked to other nodes with a cardinality of 0..* or 0... That is to say a cardinality of 0.. is not allowed in the TQL. Only that it is not allowed on the node to which the operation will be applied. –NOTE: In mathematics, the cardinality of a set is a measure of the "number of elements of the set". For example, the set A = {2, 4, 6} contains 3 elements, and therefore A has a cardinality of 3. –There are two approaches to cardinality – one which compares sets directly using bijections and injections, and another which uses cardinal numbers. 31

© Copyright 2012 Hewlett-Packard Development Company, L.P.32 ENRICHMENTS STEP BY STEP #3: CONFIGURING THE ENRICHMENT 32 Day 3 Switch to Enrichment Mode Create Enrichment –Update/Delete Relationship/Node –Add Node/Relationship Select Attributes to be updated and set values(if Relevant)

© Copyright 2012 Hewlett-Packard Development Company, L.P.33 BY ATTRIBUTE: –By Attribute: set the value of the selected attribute based on the value of another attribute. Select the node from which to pull the attribute information. All nodes and relationships of the TQL will appear in the list. Select the attribute that will be used to update the information. In the case of string type values, you may also use a regular expression. –Advanced: is only available for string values. Allows you to use a combination of constants, attribute values, and regular expressions to build the attribute value for the target node. 33

© Copyright 2012 Hewlett-Packard Development Company, L.P.34 ENRICHMENTS STEP BY STEP #4: ADDITIONAL DATA USAGE CAPABILITIES 34 Day 3 By Attribute: o Allows for the usage of data from other nodes in the TQL o Allows for the use of regular expressions to capture data Advanced Options o Additional Data Manipulation Options

© Copyright 2012 Hewlett-Packard Development Company, L.P.35 ENRICHMENTS STEP BY STEP #5: TQL AND RESULT DATA VERIFICATION 1.Calculate TQL Result Count 2.View Instances/Preview 35 Day 3

© Copyright 2012 Hewlett-Packard Development Company, L.P.36 ADDING NODE – CREATING CIS USING ENRICHMENTS. STEP 6 –It is common to use enrichments for creation of new CIs. To define creation of new CI, switch Enrichment Manager to Enrichment Mode. Any new Query Node added in Enrichment Mode defines creation of new CI. When creating CIs we have to follow general rules of CIs creation. –Key attributes – values for key attributes need to be defined, RTSM server uses these attributes to calculate Unique Identifier for each new CI. Creation of CI without these attributes is not possible. –Mandatory fields – some fields can be defined as mandatory in CIT manager. RTSM will not create a CI if these attributes are not populated with values. –Use right click on target nodes (query nodes added in Enrichment Mode), choose Update Query Node to get to the Update Query Node dialog. Define values for the key and required attributes. –Deleting Node – Deleting CIs –Enrichment is very useful when doing batch deletion of CIs. Create TQL that will return CIs meant for deletion under one or more nodes. Right click on the query node and choose Delete Relation/Query Node. All the CIs represented by this Query Node will get deleted as soon as the enrichment rule becomes active. –Same rules and workflows apply to relationships as well. 36

© Copyright 2012 Hewlett-Packard Development Company, L.P.37 ENRICHMENTS STEP BY STEP #6: ADDING AND DELETING NODES 37 Day 3 Enrichments also allow for the addition and deletion of Nodes and Relationships. The following rules apply to such Enrichments: Adding Nodes: Requires the definition of a relationship to the node to be added Requires the node to be created, having the necessary keys to be supplied Deleting Nodes: Requires the definition of the nodes/relationships to be deleted in the TQL

© Copyright 2012 Hewlett-Packard Development Company, L.P.38 –J2EE –Layer 2 –Mainframe –MQ Series –Network –Siebel –SOA –IIS – WebService –SAP –Software –Multi Process 38 Day 3 ENRICHMENTS STEP BY STEP #7: OOTB ENRICHMENTS

© Copyright 2012 Hewlett-Packard Development Company, L.P.39 ENRICHMENTS STEP BY STEP #8: MANAGING ENRICHMENTS Enrichment Rules may be organized in the Rules Explorer. As in the other managers, the explorer is used to store the rules in a manner that they are straight forward to find and maintain. New folders may be added and rules may be moved between folders by dragging and dropping them into the desired folder. Rules may be copied by using the Save As option. Enrichment Rules are activated and deactivated from the Rules explorer. To do so, select Properties from the right-click context menu. In the properties window select the Active checkbox to activate the rule or deselect the Active checkbox to deactivate the rule. To import a rule from XML, select the XML button on the Explorer toolbar. From the browse dialog, browse the xml file that contains the rule. It is important to note that the TQL must be present within the system prior to importing the rule. This may be checked in Query Manager. Rules may be exported to XML by using the right-click context menu for the selected rule. 39 Day 3

© Copyright 2012 Hewlett-Packard Development Company, L.P.40 Day 3 The Scheduler is accessed through the Administration Section of uCMDB ENRICHMENTS: SCHEDULING ENRICHMENTS Any enrichment may be scheduled to run through the Scheduler. A scheduled Enrichment rule is activated by the scheduler at the designated run time. Access Scheduler from the Administration area of uCMDB.

© Copyright 2012 Hewlett-Packard Development Company, L.P.41 ENRICHMENTS: SCHEDULING ENRICHMENTS (CONT.) *Job Definition *Action Definition*Scheduling 41 Day 3

© Copyright 2012 Hewlett-Packard Development Company, L.P.42 ADD JOB BUTTON IN SCHEDULER. – In order to create a scheduling for enrichment rule click on Add Job button in Scheduler. –In Job Definition type name and Job Definition, click on New Action button. –In Action Definition dialog choose Run an Enrichment Rule. –In Action Definition choose the Enrichment Rules from the list. –Close the Action Definition dialog and define the time and/or interval to run the selected enrichment rules. 42

© Copyright 2012 Hewlett-Packard Development Company, L.P.43 Enrichments allow for the automated update/deletion/additon of CIs and their relationships in the RTSM Enrichments allow for the use of “inferred” data from other CIs included in the TQL Enrichments are powerful and dangerous, therefore all enrichments require verification & testing in a separate environment prior to deployment ENRICHMENTS - SUMMARY 43 Day 3

© Copyright 2012 Hewlett-Packard Development Company, L.P.44 TROUBLESHOOTING 44

© Copyright 2012 Hewlett-Packard Development Company, L.P.45 DEBUG 45

© Copyright 2012 Hewlett-Packard Development Company, L.P.46 CMDB.PROPERTIES –log4j.category.cmdb.enrichment=DEBUG,cmdb.enrichment.appender –log4j.category.com.hp.ucmdb.api.server.modeling=${loglevel},cmdb.enri chment.appender –log4j.category.com.mercury.topaz.cmdb.server.enrichment=DEBUG,cm db.enrichment.detailed.appender –log4j.category.com.mercury.topaz.cmdb.shared.enrichment=DEBUG,cm db.enrichment.detailed.appender 46

© Copyright 2012 Hewlett-Packard Development Company, L.P.47 LOGS –cmdb.enrichment.detailed.log –cmdb.enrichment.log – :57:59,825 Calculator Task-0] INFO appender.calculateEnrichment(202) - deletehosts -- Calculation is required – :57:59,843 Calculator Task-0] INFO appender.calculateEnrichment(209) - deletehosts -- Start calculate enrichment related to pattern: deletehosts with version – :57:59,858 Calculator Task-0] INFO appender.fillBulk(50) - deletehosts -- enrichment definition remove 2 object/s – :57:59,893 Calculator Task-0] INFO appender.calculateEnrichment(235) - deletehosts -- Finished calculate enrichment related to pattern: deletehosts 47

© Copyright 2012 Hewlett-Packard Development Company, L.P.48 CALCULATE AND REMOVES IT – :05:00,129 [QuartzScheduler_Worker-1] INFO appender.calculateAdHocEnrichment(109) - deletehosts -- Calculate ad hoc enrichment – :05:00,133 [QuartzScheduler_Worker-1] INFO appender.calculateAdHocEnrichment(121) - deletehosts -- Finished ad hoc calculate enrichment related to pattern: deletehosts 48

© Copyright 2012 Hewlett-Packard Development Company, L.P.49 UPDATE CI ATTRIBUTE – :25:46,851 INFO appender.addEnrichmentDefinition(133) - updatedomainname -- add enrichment definition – :30:00,216 [QuartzScheduler_Worker-3] INFO appender.calculateAdHocEnrichment(109) - updatedomainname -- Calculate ad hoc enrichment – :30:00,264 [QuartzScheduler_Worker-3] INFO appender.fillBulk(50) - updatedomainname -- enrichment definition update 2 object/s – :30:00,284 [QuartzScheduler_Worker-3] INFO appender.calculateAdHocEnrichment(121) - updatedomainname -- Finished ad hoc calculate enrichment related to pattern: updatedomainname 49