Presentation is loading. Please wait.

Presentation is loading. Please wait.

Dominik Stokłosa Pozna ń Supercomputing and Networking Center, Supercomputing Department INGRID 2008 Lacco Ameno, Island of Ischia, ITALY, April 9-11 Workflow.

Similar presentations


Presentation on theme: "Dominik Stokłosa Pozna ń Supercomputing and Networking Center, Supercomputing Department INGRID 2008 Lacco Ameno, Island of Ischia, ITALY, April 9-11 Workflow."— Presentation transcript:

1 Dominik Stokłosa Pozna ń Supercomputing and Networking Center, Supercomputing Department INGRID 2008 Lacco Ameno, Island of Ischia, ITALY, April 9-11 Workflow management in Remote Instrumentation Infrastructure – based on e-VLBI experiences

2 Introduction to the e-VLBI Introduction to the e-VLBI PSNC in EXPReS - FABRIC PSNC in EXPReS - FABRIC Grid – VLBI design & technology Grid – VLBI design & technology Summary Summary Outline

3 Introduction to the e-VLBI VLBI is a technique, in which physically independent and widely separated radio telescopes observe the same region of sky simultaneously, in order to generate very high-resolution continuum and spectral-line images of cosmic radio sources Telescopes are usually separated by thousands of kilometres Data from each telescope are digitally sampled and stored locally, using high-capacity magnetic tape systems and magnetic disk-array systems Data are sent and correlated at the central point (JIVE) The total flow of data into the central processor is approximately Terabytes per single observation, after processing this is reduced to Gbytes.

4 Introduction to EXPReS EXPReS – the objective is to create a production-level electronic VLBI (e-VLBI) service, in which the radio telescopes are reliably connected to the central data processor at JIVE via a high-speed optical-fibre communication network. Project Details Three years, started March 2006 International collaboration Funded at 3.9 million EUR FP6, Contract #026642

5 PSNC in EXPReS EXPReS – a Real-time e-VLBI Radio Telescope - JRA1: Future Arrays of Broadband Radio-Telescopes on Internet Computing (FABRIC) - Grid – VLBI collaboration - Grid Workflow management - Grid Routing Creating solution for incorporating Grid resources for distributed correlation using existing infrastructure.

6 Once upon a time (1) Everything was slow Telescopes collected data on tapes Sent via postal mail Hard drive arrays slightly improved the situation The entire cycle could easily require 6 months or more

7 Once upon a time (2) Hardware correlator; the EVN MkIV data correlator at JIVE dedicated, purpose designed/built hardware a super computer; ~100 T ops/sec

8 Today / In the near future Data can be transferred over the network Each stage of the process can be speeded up GRID resources Software correlator e-VLBI - electronic VLBI

9 e-VLBI System - architecture

10 e-VLBI System – control flow

11 Workflow Manager module (WFM)

12 CCF – correlator configuration file

13 WFM – phase 1 Definition of radio telescopes – automatically based on the observation schedule

14 WFM – phase 2 Definition of file servers (each file sever is responsible for capturing data from RT)

15 WFM – phase 3 Definition of correlation nodes and data flows between components

16 WFM – properties Definition of resource properties

17 WFM (1) The WFM application has several purposes: Workflow creation – a user is given a list of building blocks. Each block represents the available resource. It is the users responsibility to design and construct the experiments workflow from the available building blocks. Workflow submission – the scenario prepared in the previous step can be submitted to the system. Workflow management and monitoring – allows to manage the executed workflow.

18 WFM (2) Obrazek z WFM -> przeplywy danych

19 Link definition

20 Type definition Radio telescope File server Correlator

21 Properties definition Dynamic forms Parameters stored in Json format

22 PSNC in EXPReS EXPReS – a Real-time e-VLBI Radio Telescope - JRA1: Future Arrays of Broadband Radio-Telescopes on Internet Computing (FABRIC) - Grid – VLBI collaboration - Grid Workflow management - Grid Routing Creating solution for incorporating Grid resources for distributed correlation using existing infrastructure.

23 Thank you for your attention EXPReS is made possible through the support of the EC, 6 th FP, Contract #026642


Download ppt "Dominik Stokłosa Pozna ń Supercomputing and Networking Center, Supercomputing Department INGRID 2008 Lacco Ameno, Island of Ischia, ITALY, April 9-11 Workflow."

Similar presentations


Ads by Google