NICOS System of Nightly Builds for Distributed Development Alexander Undrus CHEP’03.

Slides:

Advertisements

Similar presentations

Configuration management

Advertisements

University of Southampton Electronics and Computer Science M-grid: Using Ubiquitous Web Technologies to create a Computational Grid Robert John Walters.

Week 6: Chapter 6 Agenda Automation of SQL Server tasks using: SQL Server Agent Scheduling Scripting Technologies.

1 OBJECTIVES To generate a web-based system enables to assemble model configurations. to submit these configurations on different.

2004 Cross-Platform Automated Regression Test Framework Ramkumar Ramalingam, Rispna Jain IBM Software Labs, India.

ManageEngine TM Applications Manager 8 Monitoring Custom Applications.

Report Distribution Report Distribution in PeopleTools 8.4 Doug Ostler & Eric Knapp 7264.

Automated Tests in NICOS Nightly Control System Alexander Undrus Brookhaven National Laboratory, Upton, NY Software testing is a difficult, time-consuming.

Slide 1 of 9 Presenting 24x7 Scheduler The art of computer automation Press PageDown key or click to advance.

Apache Jakarta Tomcat Suh, Junho. Road Map Tomcat Overview Tomcat Overview History History What is Tomcat? What is Tomcat? Servlet Container.

8/17/2015CS346 PHP1 Module 1 Introduction to PHP.

Linux Operations and Administration

Apache Tomcat Web Server SNU OOPSLA Lab. October 2005.

Linux Operations and Administration

FALL 2005CSI 4118 – UNIVERSITY OF OTTAWA1 Part 4 Web technologies: HTTP, CGI, PHP,Java applets)

Abstract The automated multi-platform software nightly build system is a major component in the ATLAS collaborative software organization, validation and.

Framework for Automated Builds Natalia Ratnikova CHEP’03.

About Dynamic Sites (Front End / Back End Implementations) by Janssen & Associates Affordable Website Solutions for Individuals and Small Businesses.

CHEP2000 February 2000 Impact of Software Review and Inspection Doris Burckhart CERN ATLAS DAQ/EF-1 Back-end software.

INTERNET APPLICATION DEVELOPMENT For More visit:

Rsv-control Marco Mambelli – Site Coordination meeting October 1, 2009.

Nightly Releases and Testing Alexander Undrus Atlas SW week, May

 To explain the importance of software configuration management (CM)  To describe key CM activities namely CM planning, change management, version management.

03/27/2003CHEP20031 Remote Operation of a Monte Carlo Production Farm Using Globus Dirk Hufnagel, Teela Pulliam, Thomas Allmendinger, Klaus Honscheid (Ohio.

Oracle Data Integrator Workflow Management: The Packages.

The LCG SPI project in LCG Phase II CHEP’06, Mumbai, India Feb. 14, 2006 Andreas Pfeiffer -- for the SPI team

1 3. Computing System Fundamentals 3.1 Language Translators.

Nightly System Growth Graphs Abstract For over 10 years of development the ATLAS Nightly Build System has evolved into a factory for automatic release.

Computer Emergency Notification System (CENS)

The huge amount of resources available in the Grids, and the necessity to have the most up-to-date experimental software deployed in all the sites within.

Organization and Management of ATLAS Nightly Builds F. Luehring a, E. Obreshkov b, D.Quarrie c, G. Rybkine d, A. Undrus e University of Indiana, USA a,

Alexander Richards, UCL 1 Atlfast and RTT (plus DCube) Christmas Meeting 18/12/2007.

20/09/2006LCG AA 2006 Review1 Committee feedback to SPI.

Feedback from the POOL Project User Feedback from the POOL Project Dirk Düllmann, LCG-POOL LCG Application Area Internal Review October 2003.

General rules 1. Rule: 2. Rule: 3. Rule: 10. Rule: Ask questions ……………………. 11. Rule: I do not know your skill. If I tell you things you know, please stop.

Alex Undrus – Nightly Builds – ATLAS SW Week – Dec Preamble: Code Referencing Code Referencing is a vital service to cope with 7 million lines of.

Marco Cattaneo - DTF - 28th February 2001 File sharing requirements of the physics community  Background  General requirements  Visitors  Laptops 

Configuring and Troubleshooting Identity and Access Solutions with Windows Server® 2008 Active Directory®

J.P. Wellisch, CERN/EP/SFT SCRAM Information on SCRAM J.P. Wellisch, C. Williams, S. Ashby.

Module: Software Engineering of Web Applications Chapter 2: Technologies 1.

INFSO-RI Enabling Grids for E-sciencE ARDA Experiment Dashboard Ricardo Rocha (ARDA – CERN) on behalf of the Dashboard Team.

G.Govi CERN/IT-DB 1 September 26, 2003 POOL Integration, Testing and Release Procedure Integration  Packages structure  External dependencies  Configuration.

ASP. ASP is a powerful tool for making dynamic and interactive Web pages An ASP file can contain text, HTML tags and scripts. Scripts in an ASP file are.

A. Aimar - EP/SFT LCG - Software Process & Infrastructure1 SPI Software Process & Infrastructure for LCG Project Overview LCG Application Area Internal.

Yannick Patois - Datagrid Software Repository Presentation - March, n° 1 Datagrid Software Repository Presentation CVS, packages and automatic.

ASP-2-1 SERVER AND CLIENT SIDE SCRITPING Colorado Technical University IT420 Tim Peterson.

Alex Undrus – Shifters Meeting – 16 Oct ATLAS Nightly System Integration LS1 Ugrade SIT Task Force Objective: increase efficiency, flexibility,

Experiment Support CERN IT Department CH-1211 Geneva 23 Switzerland t DBES Andrea Sciabà Hammercloud and Nagios Dan Van Der Ster Nicolò Magini.

SPI NIGHTLIES Alex Hodgkins. SPI nightlies  Build and test various software projects each night  Provide a nightlies summary page that displays all.

NA61/NA49 virtualisation: status and plans Dag Toppe Larsen Budapest

Geant4 is a toolkit to simulate the passage of particles through matter, and is widely used in HEP, in medical physics and for space applications. Ongoing.

INFSO-RI Enabling Grids for E-sciencE Using of GANGA interface for Athena applications A. Zalite / PNPI.

Alex Undrus – GRID Testing – 18 Oct Nightlies Testing on the GRID: Status Oct Purposes: Moderate scale production for quick validation (when.

Session 11: Cookies, Sessions ans Security iNET Academy Open Source Web Development.

Online Data Monitoring Framework Based on Histogram Packaging in Network Distributed Data Acquisition Systems Tomoyuki Konno 1, Anatael Cabrera 2, Masaki.

Maite Barroso – WP4 Workshop – 10/12/ n° 1 -WP4 Workshop- Developers’ Guide Maite Barroso 10/12/2002

Comments on SPI. General remarks Essentially all goals set out in the RTAG report have been achieved. However, the roles defined (Section 9) have not.

GLAST CHEP 03 March T.Burnett1/14 User-friendly tools for Managing Software Development Projects User-friendly tools for Managing Software Development.

Use of CMT in LHCb CMT Workshop, LAL (Orsay) 28 th February - 1 st March 2002 P. Mato / CERN.

11 DEPLOYING AN UPDATE MANAGEMENT INFRASTRUCTURE Chapter 6.

Software Release Build Process and Components in ATLAS Offline Emil Obreshkov for the ATLAS collaboration.

SQL Database Management

Integrating ArcSight with Enterprise Ticketing Systems

How to Contribute to System Testing and Extract Results

NA61/NA49 virtualisation:

LCGAA nightlies infrastructure

GLAST Release Manager Automated code compilation via the Release Manager Navid Golpayegani, GSFC/SSAI Overview The Release Manager is a program responsible.

PHP / MySQL Introduction

Database Driven Websites

Presentation transcript:

NICOS System of Nightly Builds for Distributed Development Alexander Undrus CHEP’03

CHEP'03 - Alexander Undrus - NICOS system2 Purpose of NICOS NIghtly COntrol System Provide flexible nightly build system for coordination of distributed development Test new software submissions Send immediate feedback to developers Facilitate multi-person, multi-platform development of code Ensure maximum stability of build processes

CHEP'03 - Alexander Undrus - NICOS system3 Flexible Build System Based on experience gained in software management of large HEP projects (CLEO, ATLAS) Operates on UNIX-like platforms (adaptation for Windows is planned) Works with popular release tools (CMT, SCRAM, SRT) Provides options for version management Allows optional number of releases in a cycle

CHEP'03 - Alexander Undrus - NICOS system4 Software Testing Unit tests supported by release tools Success is determined by the return value of a test Integrated tests defined at NICOS configuration. Success criteria: Critical patterns in the output Comparison of histograms (under development)

CHEP'03 - Alexander Undrus - NICOS system5 Informing Developers NICOS dynamic web pages  Detailed status of builds is displayed on the project web page  Build summary web pages display results for individual packages and integrated tests Responsible developers are notified about problems via

CHEP'03 - Alexander Undrus - NICOS system6 Stability of builds Build process is divided into steps Current step is displayed on the project web page If any step fails, NICOS Controller tries to restart from the point of failure Time limits are imposed for execution of steps NICOS job can be automatically killed if runs overtime

NICOS Organization NICOS Controller NICOS Job Error Analysis Testing Project Make Project Setup Code Checkout Packages Database Packages Database Project Database Project Database NICOS Webmaster Nicos WebServer CHEP’03 - Alexander Undrus - NICOS system 7

CHEP'03 - Alexander Undrus - NICOS system8 NICOS Project Database XML-like file with configuration parameters Each build step is associated with Markup tag Markup tag with parameters is followed by commands for particular step scram b NICOS_MAIL=“no”

CHEP'03 - Alexander Undrus - NICOS system9 NICOS Packages Database Names of packages Method of CVS tag selection (for each package) Exact tag Latest CVS tag Latest CVS tag with restrictions (e.g. starting with some string) External script Addresses of responsible developers

CHEP'03 - Alexander Undrus - NICOS system10 NICOS Web Server (available in 0.2 version of NICOS) Based on PHP scripts Version Administrator Change conventions for tag selections Change addresses of developers Add/Delete packages Project Configurator Configure project from internet NICOS Job Controller (in 0.3 version of NICOS) Show status of builds, stop/start builds All actions require admin privileges

NICOS Web Pages CHEP’03 - Alexander Undrus - NICOS system 11 NICOS Project Page Build Results Project Configuratio n Page NICOS Job Controller Project Configurator NICOS Version Administrator

Example of NICOS Project Web Page CHEP’03 - Alexander Undrus - NICOS system 12

Example of NICOS Build Results Web Page CHEP’03 - Alexander Undrus - NICOS system 13

Example of NICOS Job Controller Page CHEP’03 - Alexander Undrus - NICOS system 14

Example of NICOS Version Administrator Page CHEP’03 - Alexander Undrus - NICOS system 15

CHEP'03 - Alexander Undrus - NICOS system16 Schedule Three phases of NICOS development 1.Basic functionalities 2.NICOS Web Master, replacing shell scripts 3.NICOS Smart Configurator, consider porting to Windows JanFebMarAprMayJunJulySepOctNovDec Version 1 Version 2 Version 3

CHEP'03 - Alexander Undrus - NICOS system17 NICOS projects Atlas Nightly Builds Since autumn packages Several platforms and OS About hundred developers POOL nightly builds Since February 2003

CHEP'03 - Alexander Undrus - NICOS system18 Further Information ATLAS nightly builds web page atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/ OO/dist/nightlies/www/nightly_builds.html atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/ OO/dist/nightlies/www/nightly_builds.html NICOS web page os/index.html os/index.html POOL nightly builds web page atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/ OO/dist/nightlies/poolwww/index.html atlas.web.cern.ch/Atlas/GROUPS/SOFTWARE/ OO/dist/nightlies/poolwww/index.html