Gennaro Tortone, Sergio Fantinel – Bologna, LCG-EDT Monitoring Service DataTAG WP4 Monitoring Group DataTAG WP4 meeting Bologna –

Gennaro Tortone, Sergio Fantinel – Bologna, Summary monitoring of Grid elements GLUE schema EDG-WP4 monitoring framework Discovery process tasks

Gennaro Tortone, Sergio Fantinel – Bologna, Monitoring of grid elements (1/2) LOW LEVEL measurements CPU load memory usage disk usage (per partition) network activity number of processes number of users (UI) … Computing Element Storage Element Worker Node Resource Broker Information Index Replica Manager Replica Catalog […] SERVICE checks gatekeeper gsiftp gris gdmp RB/LB … “GRID” measurements number of total CPUs number of free CPUs number of running jobs number of waiting jobs SE free disk space …

Gennaro Tortone, Sergio Fantinel – Bologna, Monitoring of grid elements (2/2) sources of information LOW LEVEL measurements -> plugins/sensors installed on each machine (published through GRIS) SERVICE checks -> sensors installed on monitoring server GRID measurements -> sensors installed on monitoring server ALL => DataBase aggregate information (monitoring server side) per VO per site …

Gennaro Tortone, Sergio Fantinel – Bologna, GLUE schema Conceptual model of grid resources to be used as a base schema of the GIS (Grid Information Service) for discovery and monitoring purposes model of computing resources (CE) model of storage resources (SE) model of relationships among them (close CE/SE) Implementation status (v. 1.0) (for Globus MDS) LDAP schema (DataTAG WP4.1) information providers (CE/SE) GLUE schema extension to include all monitoring metrics done – “host level” added to GLUE schema

Gennaro Tortone, Sergio Fantinel – Bologna,

EDG-WP4 monitoring framework It provides a client (Monitoring Sensor Agent - MSA) running sensors (Monitoring Sensors - MS) on each node to monitor, and a central server (Fabric Monitoring Server - fmonServer) to collect data. The server receives samples as they are measured by MSA, and stores them in a flat file / Oracle database The client is provided with a sensor (sensorLinuxProc) which uses /proc file system to measure various basic quantities on Linux (CPU load, network, etc).

EDG-WP4 monitoring framework local farm element computing element Gennaro Tortone, Sergio Fantinel – Bologna,

GRIS (GLUE schema)WP4 fmonserver computing element information providers farm monitoring archive run ldif output write read WP4 monitoring agent worker node /proc filesystem WP4 sensor run read metric output WP4 monitoring agent worker node /proc filesystem WP4 sensor run read metric output information index GIIS (GLUE schema) monitoring server discovery servicemonitoring service ldap query web interface Monitoring DB Gennaro Tortone, Sergio Fantinel – Bologna,

Discovery process Through the GIIS, via LDAP, we can obtain the CE/SE available at a specific time. With the use of a DB we can compare the infos from the GIIS against the past history of the availability of the resources (an object can be new, disappeared, re-available) Trough the GRIS of the CE/SE we can obtain SITE/HOSTS infos (we can reiterate the discovery process at site level to get site resources/infos – ex: CEIDs, WNs, Network Adapters, supported transfer protocols,…)

Gennaro Tortone, Sergio Fantinel – Bologna, Discovery process: base schema Monitoring DB Monitorig Server GIIS GRIS GIIS Server Computing Element/ Storage Element SQL 1: LDAP Query 2: available CE/SE 3: LDAP Query 4: CEIDs, WNs, Steps 3,4 repeated for every CE/SE LDAP

Gennaro Tortone, Sergio Fantinel – Bologna, Tasks (1/3) identify the requirements for Grid monitoring done – Grid monitoring analysis draft [with some LCG inputs] (available on evaluation of existing monitoring tools (sensors) to use as “first monitoring layer” on each grid-element done – tools evaluated: EDG-WP4 fabric-monitoring tool (fmon) client-server model very easy to use very easy to install (one RPM – without dependencies) highly customizable (time interval for each metric, …) it is very easy to add a new metric historical archive database in Oracle/plain-text format

Gennaro Tortone, Sergio Fantinel – Bologna, Tasks (2/3) extension of the WP4 fabric-monitoring tool (fmon) to include other monitoring metrics done – (all metrics added are available on GLUE schema extension to include all monitoring metrics done – “host level” added to GLUE schema development of information-providers “to fill” the GLUE host level extension – done definition of database structure to store snapshot/historical monitoring data – done

Gennaro Tortone, Sergio Fantinel – Bologna, Tasks (3/3) automatic resource discovery using MDS infrastructure and GLUE schema – in progress development of a web interface to display various “grid-views” (per VO, per site, etc.) – in progress