The NorduGrid toolkit user interface Mattias Ellert Presented at the 3 rd NorduGrid workshop, Helsinki, 2002-05-23.

Slides:

Advertisements

Similar presentations

EGEE is a project funded by the European Union under contract IST EGEE Tutorial Turin, January Hands on Job Services.

Advertisements

© 2007 IBM Corporation IBM Global Engineering Solutions IBM Blue Gene/P Job Submission.

Grid Resource Allocation Management (GRAM) GRAM provides the user to access the grid in order to run, terminate and monitor jobs remotely. The job request.

The Grid Constantinos Kourouyiannis Ξ Architecture Group.

Condor-G: A Computation Management Agent for Multi-Institutional Grids James Frey, Todd Tannenbaum, Miron Livny, Ian Foster, Steven Tuecke Reporter: Fu-Jiun.

Southgreen HPC system Concepts Cluster : compute farm i.e. a collection of compute servers that can be shared and accessed through a single “portal”

06/08/10 PBS, LSF and ARC integration Zoltán Farkas MTA SZTAKI LPDS.

Dr. David Wallom Use of Condor in our Campus Grid and the University September 2004.

Building a computational cluster managed by ARC NorduGrid Alexey N. Makarov Saint-Petersburg State University, Department of Physics.

A Grid Resource Broker Supporting Advance Reservations and Benchmark- Based Resource Selection Erik Elmroth and Johan Tordsson Reporter ： S.Y.Chen.

Sun Grid Engine Grid Computing Assignment – Fall 2005 James Ruff Senior Department of Mathematics and Computer Science Western Carolina University.

GridScape Ding Choon Hoong Grid Computing and Distributed Systems (GRIDS) Lab. The University of Melbourne Melbourne, Australia WW Grid.

K.Harrison CERN, 23rd October 2002 HOW TO COMMISSION A NEW CENTRE FOR LHCb PRODUCTION - Overview of LHCb distributed production system - Configuration.

Grids and Globus at BNL Presented by John Scott Leita.

The NorduGrid project: Using Globus toolkit for building Grid infrastructure presented by Aleksandr Konstantinov Mattias Ellert Aleksandr Konstantinov.

Click to add text SAP XBP 3.0 exploitation TWS Education + Training.

Resource Management Reading: “A Resource Management Architecture for Metacomputing Systems”

Introduction to information theory

The SAM-Grid Fabric Services Gabriele Garzoglio (for the SAM-Grid team) Computing Division Fermilab.

The ATLAS Production System. The Architecture ATLAS Production Database Eowyn Lexor Lexor-CondorG Oracle SQL queries Dulcinea NorduGrid Panda OSGLCG The.

A Java Based Prototype Grid User Interface Janice Drohan Project Supervisor: Prof. Peter Clarke.

Overview of the NorduGrid Information System Balázs Kónya 3 rd NorduGrid Workshop 23 May, 2002, Helsinki.

Elisabetta Ronchieri - How To Use The UI command line - 10/29/01 - n° 1 How To Use The UI command line Elisabetta Ronchieri by WP1 elisabetta.ronchieri.

Introduction to NorduGrid ARC / Arto Teräs Slide 1(16) Introduction to NorduGrid ARC Arto Teräs Free and Open Source Software Developers' Meeting.

Grid Computing - AAU 14/ Grid Computing Josva Kleist Danish Center for Grid Computing

Bigben Pittsburgh Supercomputing Center J. Ray Scott

WP9 Resource Management Current status and plans for future Juliusz Pukacki Krzysztof Kurowski Poznan Supercomputing.

Grid Job and Information Management (JIM) for D0 and CDF Gabriele Garzoglio for the JIM Team.

Advanced Searches Using History Advanced Searches What? For a given session, a list of Standard Format Past Searches is automatically saved each time.

K. Harrison CERN, 20th April 2004 AJDL interface and LCG submission - Overview of AJDL - Using AJDL from Python - LCG submission.

NorduGrid Architecture and tools CHEP2003 – UCSD Anders Wäänänen

Computational grids and grids projects DSS,

1 st December 2003 JIM for CDF 1 JIM and SAMGrid for CDF Mòrag Burgon-Lyon University of Glasgow.

INFSO-RI Enabling Grids for E-sciencE Workload Management System Mike Mineter

GLOBAL GRID FORUM 10 Workflows in PROGRESS and GridLab environments Michał Kosiedowski.

The PROGRESS Grid Service Provider Maciej Bogdański Portals & Portlets 2003 Edinburgh, July 14th-17th.

Stuart Wakefield Imperial College London Evolution of BOSS, a tool for job submission and tracking W. Bacchi, G. Codispoti, C. Grandi, INFN Bologna D.

Network Queuing System (NQS). Controls batch queues Only on Cray SV1 Presently 8 queues available for general use and one queue for the Cray analyst.

Resource Brokering in the PROGRESS Project Juliusz Pukacki Grid Resource Management Workshop, October 2003.

PROGRESS: ICCS'2003 GRID SERVICE PROVIDER: How to improve flexibility of grid user interfaces? Michał Kosiedowski.

22 nd September 2003 JIM for CDF 1 JIM and SAMGrid for CDF Mòrag Burgon-Lyon University of Glasgow.

Author - Title- Date - n° 1 Partner Logo EU DataGrid, Work Package 5 The Storage Element.

Using the EMI testbed ARC middleware Marek Kočan University of P. J. Šafárik, Košice.

The NorduGrid Information System Balázs Kónya GGF July, 2002, Edinburgh.

© Geodise Project, University of Southampton, Geodise Middleware & Optimisation Graeme Pound, Hakki Eres, Gang Xue & Matthew Fairman Summer 2003.

Scheduling in HPC Resource Management System: Queuing vs. Planning Matthias Hovestadt, Odej Kao, Alex Keller, and Achim Streit 2003 Job Scheduling Strategies.

Review of Condor,SGE,LSF,PBS

Job Management DIRAC Project. Overview  DIRAC JDL  DIRAC Commands  Tutorial Exercises  What do you have learned? KEK 10/2012DIRAC Tutorial.

APST Internals Sathish Vadhiyar. apstd daemon should be started on the local resource Opens a port to listen for apst client requests Runs on the host.

SAN DIEGO SUPERCOMPUTER CENTER Inca Control Infrastructure Shava Smallen Inca Workshop September 4, 2008.

ATLAS Data Challenge on NorduGrid CHEP2003 – UCSD Anders Wäänänen

High-Performance Computing Lab Overview: Job Submission in EDG & Globus November 2002 Wei Xing.

© Geodise Project, University of Southampton, Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003.

Timeshared Parallel Machines Need resource management Need resource management Shrink and expand individual jobs to available sets of processors Shrink.

Korea Workshop May GAE CMS Analysis (Example) Michael Thomas (on behalf of the GAE group)

Portable Batch System – Definition and 3 Primary Roles Definition: PBS is a distributed workload management system. It handles the management and monitoring.

STAR Scheduling status Gabriele Carcassi 9 September 2002.

PROGRESS: GEW'2003 Using Resources of Multiple Grids with the Grid Service Provider Michał Kosiedowski.

Application examples Oxana Smirnova (Lund, EPF) 3 rd NorduGrid Workshop, May23, 2002.

Clarens Toolkit Building Blocks for a Simple TeraGrid Gateway Tutorial Conrad Steenberg Julian Bunn, Matthew Graham, Joseph Jacob, Craig Miller, Roy Williams.

Manchester Computing Supercomputing, Visualization & eScience Seamless Access to Multiple Datasets Mike AS Jones ● Demo Run-through.

STAR Scheduler Gabriele Carcassi STAR Collaboration.

Active-HDL Server Farm Course 11. All materials updated on: September 30, 2004 Outline 1.Introduction 2.Advantages 3.Requirements 4.Installation 5.Architecture.

Job Management Beijing, 13-15/11/2013. Overview Beijing, /11/2013 DIRAC Tutorial2  DIRAC JDL  DIRAC Commands  Tutorial Exercises  What do you.

NorduGrid Architecture EDG ATF Meeting CERN – June 12 th 2002 Anders Wäänänen.

Assumptions What are the prerequisites? … The hands on portion of the workshop will be on the command-line. If you are not familiar with the command.

Oxana Smirnova, Jakob Nielsen (Lund University/CERN)

Workload Management System

Chapter 5: CPU Scheduling

Oxana Smirnova (Lund, EPF) 3rd NorduGrid Workshop, May23, 2002

Presentation transcript:

The NorduGrid toolkit user interface Mattias Ellert Presented at the 3 rd NorduGrid workshop, Helsinki,

05/24/02 1 Mattias Ellert, The NorduGrid toolkit user interface The User Interface ● The NorduGrid toolkit user interface consists of a set of commands that can be executed from the command line ● ngsub - for job submission ● ngstat - to obtain the status of jobs and clusters ● ngcat - to display the stdout or stderr of a running job ● ngget - to retrieve the result from a finished job ● ngkill - to kill a running job ● ngclean - to delete a job from a remote cluster ● ngsync

05/24/02 1 Mattias Ellert, The NorduGrid toolkit user interface Job Submission ● The ngsub command is used to submit a job to a NorduGrid cluster ● The job and the resources needed to run it should be described using the extended resourse specification language (XRSL) - see Oxana's talk ● The XRSL string can be given as an argument: ngsub '&(executable=myprog.exe)' ● or be read from a file: ngsub -f myjob.xrsl

05/24/02 1 Mattias Ellert, The NorduGrid toolkit user interface Brokering - where do jobs go? ● The user interface contains a broker that decides to which queue at which cluster a job should be submitted ● The decision is made based on information about the different clusters obtained from the MDS servers on the clusters - see Balázs's talk about information providers

05/24/02 1 Mattias Ellert, The NorduGrid toolkit user interface Brokering algorithm ● The status of the cluster and the queue must be 'active'. ● The user must be authorized to use the cluster and the queue ● The cluster's and queue's characteristics must match the requirements specified in the xrsl string (max CPU time, required free diskspace, installed software etc)

05/24/02 1 Mattias Ellert, The NorduGrid toolkit user interface Brokering algorithm (cont) ● From all queues that fulfills the criteria one is chosen randomly, with a weight proportional to the number of free CPUs available for the user in each queue ● If the are no available CPUs in any of the queues, the job is submitted to the queue with the lowest number of queued job per processor

05/24/02 1 Mattias Ellert, The NorduGrid toolkit user interface User's Manual ● A user's manual for the NorduGrid toolkit user interface describing the commands and all their options is available from the NorduGrid website

05/24/02 1 Mattias Ellert, The NorduGrid toolkit user interface Examples ● The demo later today will give plenty of examples on how to use the different user interface commands with various options