Monitoring and Information Services Technical Group Report

Slides:



Advertisements
Similar presentations
EGEE-II INFSO-RI Enabling Grids for E-sciencE The gLite middleware distribution OSG Consortium Meeting Seattle,
Advertisements

FP7-INFRA Enabling Grids for E-sciencE EGEE Induction Grid training for users, Institute of Physics Belgrade, Serbia Sep. 19, 2008.
Plateforme de Calcul pour les Sciences du Vivant SRB & gLite V. Breton.
A conceptual model of grid resources and services Authors: Sergio Andreozzi Massimo Sgaravatto Cristina Vistoli Presenter: Sergio Andreozzi INFN-CNAF Bologna.
Open Science Grid Frank Würthwein UCSD. 2/13/2006 GGF 2 “Airplane view” of the OSG  High Throughput Computing — Opportunistic scavenging on cheap hardware.
OSG End User Tools Overview OSG Grid school – March 19, 2009 Marco Mambelli - University of Chicago A brief summary about the system.
TeraGrid Information Services December 1, 2006 JP Navarro GIG Software Integration.
Open Science Grid Software Stack, Virtual Data Toolkit and Interoperability Activities D. Olson, LBNL for the OSG International.
OSG Public Storage and iRODS
OSG Operations and Interoperations Rob Quick Open Science Grid Operations Center - Indiana University EGEE Operations Meeting Stockholm, Sweden - 14 June.
Next Steps Guy Warner
OSG Services at Tier2 Centers Rob Gardner University of Chicago WLCG Tier2 Workshop CERN June 12-14, 2006.
OSG Middleware Roadmap Rob Gardner University of Chicago OSG / EGEE Operations Workshop CERN June 19-20, 2006.
The Disk Resource Manager A Berkeley SRM for Disk Cache Implementation and Testing Experience on the OSG Jorge Luis Rodriguez IBP/Grid Mini-Workshop Apr.
Mark L. Green, Ph.D. Grid Computational Scientist Monitoring and Information Services Architecture and Deployment University at Buffalo The State University.
Managing Computational Activities on the Grid - from Specifications to Implementation: The GLUE 2 information model OGF25, 2nd March 2009, Catania Balázs.
Publication and Protection of Site Sensitive Information in Grids Shreyas Cholia NERSC Division, Lawrence Berkeley Lab Open Source Grid.
INFSO-RI Enabling Grids for E-sciencE SA1: Cookbook (DSA1.7) Ian Bird CERN 18 January 2006.
May 8, 20071/15 VO Services Project – Status Report Gabriele Garzoglio VO Services Project – Status Report Overview and Plans May 8, 2007 Computing Division,
A. Sim, CRD, L B N L 1 OSG Applications Workshop 6/1/2005 OSG SRM/DRM Readiness and Plan Alex Sim / Jorge Rodriguez Scientific Data Management Group Computational.
G RID M IDDLEWARE AND S ECURITY Suchandra Thapa Computation Institute University of Chicago.
Overview of Monitoring and Information Systems in OSG MWGS08 - September 18, Chicago Marco Mambelli - University of Chicago
DOSAR Workshop, Sao Paulo, Brazil, September 16-17, 2005 LCG Tier 2 and DOSAR Pat Skubic OU.
GridPP Deployment & Operations GridPP has built a Computing Grid of more than 5,000 CPUs, with equipment based at many of the particle physics centres.
Partnerships & Interoperability - SciDAC Centers, Campus Grids, TeraGrid, EGEE, NorduGrid,DISUN Ruth Pordes Fermilab Open Science Grid Joint Oversight.
EGEE-II INFSO-RI Enabling Grids for E-sciencE EGEE and gLite are registered trademarks State of Interoperability Laurence Field.
CEOS WGISS-21 CNES GRID related R&D activities Anne JEAN-ANTOINE PICCOLO CEOS WGISS-21 – Budapest – 2006, 8-12 May.
Open Science Grid An Update and Its Principles Ruth Pordes Fermilab.
INFSO-RI Enabling Grids for E-sciencE OSG-LCG Interoperability Activity Author: Laurence Field (CERN)
Ruth Pordes November 2004TeraGrid GIG Site Review1 TeraGrid and Open Science Grid Ruth Pordes, Fermilab representing the Open Science.
US LHC OSG Technology Roadmap May 4-5th, 2005 Welcome. Thank you to Deirdre for the arrangements.
Conference name Company name INFSOM-RI Speaker name The ETICS Job management architecture EGEE ‘08 Istanbul, September 25 th 2008 Valerio Venturi.
Portal Update Plan Ashok Adiga (512)
6/23/2005 R. GARDNER OSG Baseline Services 1 OSG Baseline Services In my talk I’d like to discuss two questions:  What capabilities are we aiming for.
Site Validation Session Report Co-Chairs: Piotr Nyczyk, CERN IT/GD Leigh Grundhoefer, IU / OSG Notes from Judy Novak WLCG-OSG-EGEE Workshop CERN, June.
Status Organization Overview of Program of Work Education, Training It’s the People who make it happen & make it Work.
The OSG and Grid Operations Center Rob Quick Open Science Grid Operations Center - Indiana University ATLAS Tier 2-Tier 3 Meeting Bloomington, Indiana.
Adrian Jackson, Stephen Booth EPCC Resource Usage Monitoring and Accounting.
DataTAG is a project funded by the European Union DataTAG WP4 meeting, Bologna 29/07/2003 – n o 1 GLUE Schema - Status Report DataTAG WP4 meeting Bologna,
Testing and integrating the WLCG/EGEE middleware in the LHC computing Simone Campana, Alessandro Di Girolamo, Elisa Lanciotti, Nicolò Magini, Patricia.
Interoperability Activity Frederick Luehring Indiana University July 20, 2005 OSG Consortium Meeting.
INFSO-RI SA2 ETICS2 first Review Valerio Venturi INFN Bruxelles, 3 April 2009 Infrastructure Support.
1 Open Science Grid: Project Statement & Vision Transform compute and data intensive science through a cross- domain self-managed national distributed.
OSG Status and Rob Gardner University of Chicago US ATLAS Tier2 Meeting Harvard University, August 17-18, 2006.
SAM architecture EGEE 07 Service Availability Monitor for the LHC experiments Simone Campana, Alessandro Di Girolamo, Nicolò Magini, Patricia Mendez Lorenzo,
Monitoring and Information Services Core Infrastructure (MIS-CI) Service Description Mark L. Green OSG Integration Workshop at UC Feb 15-17, 2005.
Grid Deployment Technical Working Groups: Middleware selection AAA,security Resource scheduling Operations User Support GDB Grid Deployment Resource planning,
Building on virtualization capabilities for ExTENCI Carol Song and Preston Smith Rosen Center for Advanced Computing Purdue University ExTENCI Kickoff.
Defining the Technical Roadmap for the NWICG – OSG Ruth Pordes Fermilab.
DGAS Accounting – toward national grid infrastructures HPDC workshop on Monitoring, Logging and Accounting, (MLA) in production Grids 10/06/2009, Munich.
Grid Colombia Workshop with OSG Week 2 Startup Rob Gardner University of Chicago October 26, 2009.
OSG User Group August 14, Progress since last meeting OSG Users meeting at BNL (Jun 16-17) –Core Discussions on: Workload Management; Security.
Open Science Grid Interoperability
Jean-Philippe Baud, IT-GD, CERN November 2007
Bob Jones EGEE Technical Director
First proposal for a modification of the GIS schema
Regional Operations Centres Core infrastructure Centres
Open Science Grid Progress and Status
Interoperability Activity
Management of Virtual Machines in Grids Infrastructures
GGF OGSA-WG, Data Use Cases Peter Kunszt Middleware Activity, Data Management Cluster EGEE is a project funded by the European.
Grid Service Monitoring Working Group
Artem Trunov and EKP team EPK – Uni Karlsruhe
Management of Virtual Machines in Grids Infrastructures
A conceptual model of grid resources and services
20409A 7: Installing and Configuring System Center 2012 R2 Virtual Machine Manager Module 7 Installing and Configuring System Center 2012 R2 Virtual.
Leigh Grundhoefer Indiana University
Pierre Girard ATLAS Visit
Monitoring and Information Services
Sergio Andreozzi Laurence Field Balazs Konya
Presentation transcript:

Monitoring and Information Services Technical Group Report Mark L. Green, Ph.D. Co-Chair, Monitoring and Information Services Technical Group

Outline Current State VDT 1.3.6 Validation OSG 0.1.6 Validation From MIS standpoint looks good Buffalo has a problem with 64-bit machine OSG 0.1.6 Validation GIP/MDS Configuration Digesting GIP documentation and Schema specification Need to form documentation for proper configuration Script based Should static information GIP requires be part of OSG minimum monitoring requirements? Ongoing work Currently formulating Roadmap for OSG MIS with the LCG/EGEE Grid Deployment Team

Need for MIS-CI Activity Formation Outline Monitoring and Information Services - Core Infrastructure (MIS-CI) The Monitoring and Information Services Technical Group has determined the need for a well defined MIS-CI Activity. This has also been discussed with several other Technical Groups and Activities including Deployment, Operations, Interoperable and Provisioning. We have identified several potential interested contributors for the Activity already. The Activity co-chairs are (pending formal acceptance): Mark L. Green LCG Representative (Markus, Laurence, Software Development Team Member)

MIS-CI Scope MIS-CI Information Gathering Scope Gatekeeper Statistics Fine Grain Load Information Memory Processes Etc. Service Statistics Gatekeeper GridFTP SRM Site Functional Tests VO Based Disk Utilization Batch Job Statistics Worker Node Validation

MIS-CI Scope MIS Compute Element and Storage Element Resource Repository Lightweight with minimal resource loading and requirements Self monitoring Expandable Common schema with unique name space Virtual organization based aggregation Timestamps

MIS-CI Scope MIS Compute Element and Storage Element Information Gathering Lightweight information gathering Self monitoring Expandable Modular design Cron based implementation

MIS-CI Scope MIS Compute Element and Storage Element Information Gathering Lightweight information gathering Self monitoring Expandable Modular design Cron based implementation

MIS-CI Mission Re-design the proposed MIS-CI system architecture to include the following capabilities: Inter-operational with European EGEE Grid Inter-operational with partner grids TeraGrid SURA SHARCNET Etc. Expand and submit current OSG minimum monitoring requirements documents Incorporate storage resource monitoring

MIS-CI Architecture SQL-Lite Database Remote VO Historical Repository Local Backup User Resource Gatekeeper jobmanager-mis -profile (default) jobs gridftp diskspace -accounting policy environment software VO statistics security Discovery Service MIS-CI Profiles Schema Eval Cron Self- Monitor accounting ? MIS-CI Consumer Throttling MIS-CI Resource Throttling Custom Remote Repository Grid Scheduler OSG Auditing/Accounting

MIS-CI Level of Effort Indiana GOC Binghamton LCG/EGEE Grid Deployment Center for Computational Research SUNY Buffalo Stakeholder 1.5 FTE Indiana GOC 0.5 FTE Binghamton 1.0 FTE LCG/EGEE Grid Deployment 0.25 FTE (pending) Others Estimated requirement 2.0 FTE