Presentation on theme: "Dominik Stokłosa Pozna ń Supercomputing and Networking Center, Supercomputing Department INGRID 2008 Lacco Ameno, Island of Ischia, ITALY, April 9-11 Workflow."— Presentation transcript:
Dominik Stokłosa Pozna ń Supercomputing and Networking Center, Supercomputing Department INGRID 2008 Lacco Ameno, Island of Ischia, ITALY, April 9-11 Workflow management in Remote Instrumentation Infrastructure – based on e-VLBI experiences
Introduction to the e-VLBI Introduction to the e-VLBI PSNC in EXPReS - FABRIC PSNC in EXPReS - FABRIC Grid – VLBI design & technology Grid – VLBI design & technology Summary Summary Outline
Introduction to the e-VLBI VLBI is a technique, in which physically independent and widely separated radio telescopes observe the same region of sky simultaneously, in order to generate very high-resolution continuum and spectral-line images of cosmic radio sources Telescopes are usually separated by thousands of kilometres Data from each telescope are digitally sampled and stored locally, using high-capacity magnetic tape systems and magnetic disk-array systems Data are sent and correlated at the central point (JIVE) The total flow of data into the central processor is approximately 10- 100 Terabytes per single observation, after processing this is reduced to 10-100 Gbytes.
Introduction to EXPReS EXPReS – the objective is to create a production-level electronic VLBI (e-VLBI) service, in which the radio telescopes are reliably connected to the central data processor at JIVE via a high-speed optical-fibre communication network. Project Details Three years, started March 2006 International collaboration Funded at 3.9 million EUR FP6, Contract #026642
PSNC in EXPReS EXPReS – a Real-time e-VLBI Radio Telescope - JRA1: Future Arrays of Broadband Radio-Telescopes on Internet Computing (FABRIC) - Grid – VLBI collaboration - Grid Workflow management - Grid Routing Creating solution for incorporating Grid resources for distributed correlation using existing infrastructure.
Once upon a time (1) Everything was slow Telescopes collected data on tapes Sent via postal mail Hard drive arrays slightly improved the situation The entire cycle could easily require 6 months or more
Once upon a time (2) Hardware correlator; the EVN MkIV data correlator at JIVE dedicated, purpose designed/built hardware a super computer; ~100 T ops/sec
Today / In the near future Data can be transferred over the network Each stage of the process can be speeded up GRID resources Software correlator e-VLBI - electronic VLBI
WFM – phase 1 Definition of radio telescopes – automatically based on the observation schedule
WFM – phase 2 Definition of file servers (each file sever is responsible for capturing data from RT)
WFM – phase 3 Definition of correlation nodes and data flows between components
WFM – properties Definition of resource properties
WFM (1) The WFM application has several purposes: Workflow creation – a user is given a list of building blocks. Each block represents the available resource. It is the users responsibility to design and construct the experiments workflow from the available building blocks. Workflow submission – the scenario prepared in the previous step can be submitted to the system. Workflow management and monitoring – allows to manage the executed workflow.