Unified scripts ● Currently they are composed of a main shell script and a few auxiliary ones that handle mostly the local differences. ● Local scripts.

Slides:



Advertisements
Similar presentations
Operating Systems Operating system is the “executive manager” of all hardware and software.
Advertisements

1 OBJECTIVES To generate a web-based system enables to assemble model configurations. to submit these configurations on different.
Bookshelf.EXE - BX A dynamic version of Bookshelf –Automatic submission of algorithm implementations, data and benchmarks into database Distributed computing.
OpenVMS System Management A different perspective by Andy Park TrueBit b.v.
Grid and CDB Janusz Martyniak, Imperial College London MICE CM37 Analysis, Software and Reconstruction.
LUNARC, Lund UniversityLSCS 2002 Transparent access to finite element applications using grid and web technology J. Lindemann P.A. Wernberg and G. Sandberg.
Browsers and Servers CGI Processing Model ( Common Gateway Interface ) © Norman White, 2013.
C Pre-Course Workshop Terence Lee QT406 -> PQ607
Linux Networking CIS Why Linux/Unix? Configurability ▫Customizable System to satisfy unique needs. Scalability ▫Able to serve an increasing number.
DIRAC API DIRAC Project. Overview  DIRAC API  Why APIs are important?  Why advanced users prefer APIs?  How it is done?  What is local mode what.
Analysis of SQL injection prevention using a proxy server By: David Rowe Supervisor: Barry Irwin.
Distributed File Systems
Bigben Pittsburgh Supercomputing Center J. Ray Scott
9 Chapter Nine Compiled Web Server Programs. 9 Chapter Objectives Learn about Common Gateway Interface (CGI) Create CGI programs that generate dynamic.
1 In the good old days... Years ago… the WWW was made up of (mostly) static documents. –Each URL corresponded to a single file stored on some hard disk.
03/27/2003CHEP20031 Remote Operation of a Monte Carlo Production Farm Using Globus Dirk Hufnagel, Teela Pulliam, Thomas Allmendinger, Klaus Honscheid (Ohio.
Towards a unified Cyberaide architecture Fugang Wang May 29, 2009.
TELE 301 Lecture 10: Scheduled … 1 Overview Last Lecture –Post installation This Lecture –Scheduled tasks and log management Next Lecture –DNS –Readings:
CS 390 Unix Programming Summer Unix Programming - CS 3902 Course Details Online Information Please check.
Silberschatz, Galvin and Gagne  Operating System Concepts Chapter 3: Operating-System Structures System Components Operating System Services.
UNIX Commands. Why UNIX Commands Are Noninteractive Command may take input from the output of another command (filters). May be scheduled to run at specific.
Process Control. Module 11 Process Control ♦ Introduction ► A process is a running occurrence of a program, including all variables and other conditions.
Website Development with PHP and MySQL Saving Data.
Stuart Wakefield Imperial College London Evolution of BOSS, a tool for job submission and tracking W. Bacchi, G. Codispoti, C. Grandi, INFN Bologna D.
Introduction Advantages/ disadvantages Code examples Speed Summary Running on the AOD Analysis Platforms 1/11/2007 Andrew Mehta.
A Brief Documentation.  Provides basic information about connection, server, and client.
Framework of Job Managing for MDC Reconstruction and Data Production Li Teng Zhang Yao Huang Xingtao SDU
Dynamic web content HTTP and HTML: Berners-Lee’s Basics.
EGI-InSPIRE RI EGI-InSPIRE EGI-InSPIRE RI Direct gLExec integration with PanDA Fernando H. Barreiro Megino CERN IT-ES-VOS.
Guide to Linux Installation and Administration, 2e1 Chapter 11 Using Advanced Administration Techniques.
Get Rid of Cron Scripts Using Events Sheeri Cabral Senior DB Admin/Architect,
Module 8 : Configuration II Jong S. Bok
Pipeline Introduction Sequential steps of –Plugin calls –Script calls –Cluster jobs Purpose –Codifies the process of creating the data set –Reduces human.
Shell Interface Shell Interface Functions Data. Graphical Interface Graphical Interface Command-line Interface Command-line Interface Experiments Private.
File Systems cs550 Operating Systems David Monismith.
8 Chapter Eight Server-side Scripts. 8 Chapter Objectives Create dynamic Web pages that retrieve and display database data using Active Server Pages Process.
Form Processing Week Four. Form Processing Concepts The principal tool used to process Web forms stored on UNIX servers is a CGI (Common Gateway Interface)
1 DIRAC Job submission A.Tsaregorodtsev, CPPM, Marseille LHCb-ATLAS GANGA Workshop, 21 April 2004.
ISG We build general capability Introduction to Olympus Shawn T. Brown, PhD ISG MISSION 2.0 Lead Director of Public Health Applications Pittsburgh Supercomputing.
CP476 Internet Computing Perl CGI and MySql 1 Relational Databases –A database is a collection of data organized to allow relatively easy access for retrievals,
Liudmila Stepanova SINP-MSU team Testing gLite Worker Nodes LCG Dubna, Jul 26, 2007.
Input Output Techniques Programmed Interrupt driven Direct Memory Access (DMA)
STAR Scheduling status Gabriele Carcassi 9 September 2002.
STAR Scheduler Gabriele Carcassi STAR Collaboration.
Advanced topics Cluster Training Center for Simulation and Modeling September 4, 2015.
WebScan: Implementing QueryServer 2.0 Karl Geiger, Amgen Inc. BRS NA UG August 1999.
MESA A Simple Microarray Data Management Server. General MESA is a prototype web-based database solution for the massive amounts of initial data generated.
Nate Anderson So, You’ve Inherited an OnBase System.
Compute and Storage For the Farm at Jlab
GRID COMPUTING.
Introduction to Dynamic Web Programming
U.S. ATLAS Grid Production Experience
Quick Look on dCache Monitoring at FNAL
Work report Xianghu Zhao Nov 11, 2014.
EXCEPTION HANDLING IN SERVER CLIENT PROGRAMMING
ETL Job Scheduler Job Database Server User Interface Scheduler
Shell Script Assignment 1.
DUCKS – Distributed User-mode Chirp-Knowledgeable Server
Life Cycle Objectives By James Lee and Matt Higgins
(Chapter 2) John Carelli, Instructor Kutztown University
CCR Advanced Seminar: Running CPLEX Computations on the ISE Cluster
Building Web Applications
Initial job submission and monitoring efforts with JClarens
Good Testing Practices
Michael P. McCumber Task Force Meeting April 3, 2006
Chapter 2: Operating-System Structures
Periodic Processes Chapter 9.
CS122B: Projects in Databases and Web Applications Winter 2019
Chapter 2: Operating-System Structures
Overview of Computer system
Presentation transcript:

Unified scripts ● Currently they are composed of a main shell script and a few auxiliary ones that handle mostly the local differences. ● Local scripts are sourced to modify local variables eg whether to modify.bashrc or to add a modules-env module or to modify the.softenv etc. ● Later on, based on application asked, the input file may be checked for keywords

● Currently only gaussian has a separate script that is sourced – would like to better incorporate this in main script for maintenability.Scheduler specific preample is created ● Scheduler submit script is created – Includes the uberftp commands at the end of the script ● Job submitted ● The jobid is sent back to the calling script by means of a "unique" keyword in the output

TODOs ● Radium.ncsa needs to be included in the unified scripts ● External user implementation ● Input files should be scanned further for consistency and for keywords like Gamess “Memory” etc - does it make sense to have this done on clusterscripts or client

● What part of the server scripts and cluster scripts will still be used in six months when we go to web services ● (relates to server scripts too) We have the specifications.cfg and the mysql database and the "local scripts' - should we simplify/unify by only using the mysql database? Comments/Questions

● Clean up of ccguser directory including scratch – For each job now there are 2 dirs – Use of local $TMPDIR at OSC (local disk space)?

● Improve debugging (relates to server scripts) – Need to separate the error log of the cgi scripts to job specific error log (eg add the “$PROCESS_ID” at the end of the log file to make it specific for the job) - but then we ll need to clean up those logs ● Is it worth centralizing all the cluster-scripts by keeping them and executing them on the GMS server instead of in clusters? – Will easy modifications – One less place to look at and worry about