WP4-install status update

Slides:



Advertisements
Similar presentations
LNL M.Biasotto, Roma, 22 novembre Some notes on WP6 INFN Testbed.
Advertisements

LCFG overview and future Michael George University of Liverpool
Andrew McNab - Manchester HEP - 2 May 2002 Testbed and Authorisation EU DataGrid Testbed 1 Job Lifecycle Software releases Authorisation at your site Grid/Web.
German Cancio – WP4 developments Partner Logo WP4-install plans WP6 meeting, Paris project conference
ASIS et le projet EU DataGrid (EDG) Germán Cancio IT/FIO.
27-29 September 2002CrossGrid Workshop LINZ1 USE CASES (Task 3.5 Test and Integration) Santiago González de la Hoz CrossGrid Workshop at Linz,
Status of Globus activities within INFN (update) Massimo Sgaravatto INFN Padova for the INFN Globus group
Partner Logo German Cancio – WP4-install LCFG HOW-TO - n° 1 WP4 hands-on workshop: EDG LCFGng exercises
WP4-install task report WP4 workshop Barcelona project conference 5/03 German Cancio.
AUTOBUILD Build and Deployment Automation Solution.
C. Loomis – Testbed Status – 28/01/2002 – n° 1 Future WP6 Tasks Charles Loomis January 28, 2002
DataGrid is a project funded by the European Commission under contract IST IT Post-C5, Managing Computer Centre machines with Quattor.
EDG LCFGng: concepts Fabric Management Tutorial - n° 2 LCFG (Local ConFiGuration system)  LCFG is originally developed by the.
1 Linux in the Computer Center at CERN Zeuthen Thorsten Kleinwort CERN-IT.
October, Scientific Linux INFN/Trieste B.Gobbo – Compass R.Gomezel - T.Macorini - L.Strizzolo INFN - Trieste.
Olof Bärring – WP4 summary- 6/3/ n° 1 Partner Logo WP4 report Status, issues and plans
quattor NCM components introduction tutorial German Cancio CERN IT/FIO.
EDG WP4: installation task LSCCW/HEPiX hands-on, NIKHEF 5/03 German Cancio CERN IT/FIO
Partner Logo DataGRID WP4 - Fabric Management Status HEPiX 2002, Catania / IT, , Jan Iven Role and.
Olof Bärring – WP4 summary- 4/9/ n° 1 Partner Logo WP4 report Plans for testbed 2
Partner Logo German Cancio – WP4-install LCFG HOW-TO - n° 1 LCFGng configuration examples Updated 10/2002
1 The new Fabric Management Tools in Production at CERN Thorsten Kleinwort for CERN IT/FIO HEPiX Autumn 2003 Triumf Vancouver Monday, October 20, 2003.
05/29/2002Flavia Donno, INFN-Pisa1 Packaging and distribution issues Flavia Donno, INFN-Pisa EDG/WP8 EDT/WP4 joint meeting, 29 May 2002.
Quattor-for-Castor Jan van Eldik Sept 7, Outline Overview of CERN –Central bits CDB template structure SWREP –Local bits Updating profiles.
First attempt for validating/testing Testbed 1 Globus and middleware services WP6 Meeting, December 2001 Flavia Donno, Marco Serra for IT and WPs.
German Cancio – WP4 developments Partner Logo System Management: Node Configuration & Software Package Management
Large Farm 'Real Life Problems' and their Solutions Thorsten Kleinwort CERN IT/FIO HEPiX II/2004 BNL.
Deployment work at CERN: installation and configuration tasks WP4 workshop Barcelona project conference 5/03 German Cancio CERN IT/FIO.
20-May-2003HEPiX Amsterdam EDG Fabric Management on Solaris G. Cancio Melia, L. Cons, Ph. Defert, I. Reguero, J. Pelegrin, P. Poznanski, C. Ungil Presented.
G. Cancio, L. Cons, Ph. Defert - n°1 October 2002 Software Packages Management System for the EU DataGrid G. Cancio Melia, L. Cons, Ph. Defert. CERN/IT.
DataGRID WPMM, Geneve, 17th June 2002 Testbed Software Test Group work status for 1.2 release Andrea Formica on behalf of Test Group.
Maite Barroso – WP4 Barcelona – 13/05/ n° 1 -WP4 Barcelona- Closure Maite Barroso 13/05/2003
Lemon Monitoring Miroslav Siket, German Cancio, David Front, Maciej Stepniewski CERN-IT/FIO-FS LCG Operations Workshop Bologna, May 2005.
Installing, running, and maintaining large Linux Clusters at CERN Thorsten Kleinwort CERN-IT/FIO CHEP
SPMA & SWRep: Basic exercises HEPiX hands-on, NIKHEF 5/03 German Cancio
Software Management with Quattor German Cancio CERN/IT.
Olof Bärring – WP4 summary- 4/9/ n° 1 Partner Logo WP4 report Plans for testbed 2 [Including slides prepared by Lex Holt.]
22nd April 2002 Steve Traylen, RAL, 1 LCFG Installation Steve Traylen. LCFG – A tool for installation and configuration. UK HEP SYSMAN,
DGC Paris WP2 Summary of Discussions and Plans Peter Z. Kunszt And the WP2 team.
C. Aiftimiei, E. Ferro / January LCFGng server installation Cristina Aiftimiei, Enrico Ferro INFN-LNL.
Olof Bärring – EDG WP4 status&plans- 22/10/ n° 1 Partner Logo EDG WP4 (fabric mgmt): status&plans Large Cluster.
E. Ferro, CNAF, april Enrico Ferro INFN-LNL Installation of a LCFG server.
German Cancio – WP4 developments Partner Logo WP4-install progress CERN, 19/6/2002 for WP4-install.
Maite Barroso - 10/05/01 - n° 1 WP4 PM9 Deliverable Presentation: Interim Installation System Configuration Management Prototype
ASIS + RPM: ASISwsmp German Cancio, Lionel Cons, Philippe Defert, Andras Nagy CERN/IT Presented by Alan Lovell.
Andrew McNab - Globus Distribution for Testbed 1 Globus Distribution for Testbed 1 Andrew McNab, University of Manchester
The EDG Testbed The European DataGrid Project Team
15-Feb-02Steve Traylen, RAL WP6 Test Bed Report1 RAL/UK WP6 Test Bed Report Steve Traylen, WP6 PPGRID/RAL, UK
Linux Configuration using April 12 th 2010 L. Brarda / CERN (some slides & pictures taken from the Quattor website) ‏
Status of Globus activities Massimo Sgaravatto INFN Padova for the INFN Globus group
CERN 19/06/2002 Kickstart file generator Andrea Chierici (INFN-CNAF) Enrico Ferro (INFN-LNL) Marco Serra (INFN-Roma)
Quattor tutorial Introduction German Cancio, Rafael Garcia, Cal Loomis.
PDS4 Project Report PDS MC F2F University of Maryland Dan Crichton March 27,
Partner Logo Olof Bärring, WP4 workshop 10/12/ n° 1 (My) Vision of where we are going WP4 workshop, 10/12/2002 Olof Bärring.
Fabric Management: Progress and Plans PEB Tim Smith IT/FIO.
Managing Large Linux Farms at CERN OpenLab: Fabric Management Workshop Tim Smith CERN/IT.
Quattor: An administration toolkit for optimizing resources Marco Emilio Poleggi - CERN/INFN-CNAF German Cancio - CERN
StoRM: a SRM solution for disk based storage systems
DataGrid Quality Assurance
Status of Fabric Management at CERN
Germán Cancio CERN IT/FIO LCG workshop, 24/3/04
SPI external software build tool and distribution mechanism
Project Center Use Cases
Generator Services planning meeting
German Cancio CERN IT .quattro architecture German Cancio CERN IT.
Towards automation of computing fabrics using tools from the fabric management workpackage of the EU DataGrid project Maite Barroso Lopez (WP4)
Porting LCG to IA64 Andreas Unterkircher CERN openlab May 2004
The EU DataGrid Fabric Management Services
Server Management and Automation Windows Server 2012 R2
Linux Cluster Tools Development
Presentation transcript:

WP4-install status update WP4 workshop 12/02 German Cancio, Enrico Ferro

Agenda Release 2 work since last workshop Release 3 work progress Outlook

LCFGng (I) Release 2 work: concentrate on making LCFGng stable and available on RH7, while preparing the field for Release 3 Prepare LCFGng / RH62 for deployment on EDG testbeds: documentation, RPM lists, component configuration lists Port LCFGng to RH72 … and then RH73 ;-( Migration documentation: how to go from LCFG to LCFGng Template testbed configurations made available

LCFGng (II) PXE support and web interface Updaterpms ‘light’ EDG LCFGng supports now PXE. Looks for the XML profiles available to detect which nodes could be installed Uses HTTP/cgi to acknowledge installation completion Web interface to select if and which installation type Updaterpms ‘light’ Updaterpms can be configured to respect local installation Separate ‘first installation’ from ‘update’ phases during RPM installs 99% of the updaterpms failures were due to badly packaged RPM’s. These errors can be avoided now with the two-phase installation. Replace populate-server by Steve Traylen’s edg-updaterep

LCFGng (III) Wrote new components (AFS, xinetd,..), distribute ‘external’ components (Monitoring, WP2 RC..) CDB->Kickstart translator prototype Others clean up pending bugzilla reports Enable autobuild Write unit tests Teaching: “How to write LCFG components” tutorial at CERN LCFGng tutorial at CERN Coordination workshops in October CERN-Edinburgh (2 days) CERN-INFN (1.5 days)

LCFGng (IV) Main issues: RH release and updates were not available on Marianne -> relying on external repositories, which lead to inconsistencies as these change over time! Late decision in deciding which RH 7 version to take. When we had EDG LCFG already available for RH72, it was decided to go for RH73.  Ongoing support for “old” LCFG. Neither the production nor the development testbeds have been migrated yet to LCFGng. Thus, new components have been made available both for LCFG and LCFGng, which has lead developers to write them in shell – but shell component support will be discontinued in R3! Updaterpms is not available for rpm 4.0.4, which is the rpm version used by RH73! We are using rpm4.0.2 instead. A special package was built to solve some dependancy problem. Anyway it is working

Release 3 developments WP4-install has been working on the following R3 developments: Software Distribution: Software Repository (SWrep) Software Package Management Agent (SPMA) Transactional RPM front-end (rpmt) Global Configuration Schema

SD Architecture Component GUI swrep client SPMA List of SW Repository Central Config DB SD Architecture Desktops Component GUI Authorized users ACL based, using ssh authentication “desired” configuration Local Config file swrep client SPMA Add/delete/update packages (RPM, pkg) List of Installed pkgs Transaction set SW Repository packages rpmt/pkgt HTTP, NFS, FTP filesystem Package files RPM/PKG db external WP4-install

Current status (I) SWrep: A first version of the SWrep is available for tests in CVS. Currently, manages RPM packages. Can be extended to manage Solaris PKG packages. SW manager’s access is based on ACL’s and SSH authentication. Packages can be uploaded via SSH tunnel or using HTTP. SW client access is done by exporting the directory with the packages over HTTP, NFS, FTP, … Documentation: http://cern.ch/wp4-install/R3/SPM/SoftwareRepository.pdf

Current Status (II) Transactional RPM (rpmt): Prototype extensions for supporting per-package flags (no dependencies, no scripts) However, this implied a higher complexity in the rpmt tool We also found out that most of the LCFG per-package flags are not necessary Thus, we have decided to drop per-package flag support for the future. CERN Solaris team has released a first version of an equivalent tool for PKG, ‘pkgt’.

Current Status (III) Software Package Manager Agent (SPMA): First pre-alpha version being tested internally Early tests at CERN using it for ASIS package distribution Modular plug-in design for packagers current implementation supports RPM CERN Solaris team looking at providing an PKG plug-in (interfacing to pkgt)

Current Status (IV) Global Configuration Schema: Revised large parts of the initial Global Configuration Schema Specially concentrating on the /system and /software trees /software sub-tree ‘owned’ by WP4-install Propose functions for accommodating Software Package lists in the HLDL language Proposal found at: http://cern.ch/wp4-install/R3/GlobalSchema

Outlook Future work will be focused on Release 3 developments Node Configuration Management developments (next slide) Component support libraries Component porting to new framework when available We are a bit behind schedule, but we hope to have a first R3 solution ready for tests in Spring next year Development, support and teaching for legacy systems is an extremely manpower consuming issue, which has to be weighted against the need for new developments Political issues may interfere with and/or slow down developments.. PACMAN

NCM Architecture XML server client DBM Cache registration & notification “Low-level” API NVA API cdisp Invocation Component libs SUE sysmgt Logging Template processor Monitoring interface Components Components Configure() Components WP4-config WP4-install