EURO-VO: GRID and VO Lofar Information System Design OmegaCEN Kapteyn Institute TARGET- Computing Center University Groningen Garching, 10 April 2008 Lofar Information System Design OmegaCEN Kapteyn Institute TARGET- Computing Center University Groningen Garching, 10 April 2008 Edwin A. Valentijn EURO-VO DCA GRID and the VO

EURO-VO: GRID and VO (20-)30-80 MHz and MHz -EoR - transients -Survey -astroparticles -EoR - transients -Survey -astroparticles

EURO-VO: GRID and VO

E-Lofar: from experiment to observatory

EURO-VO: GRID and VO LOFAR in short Large distributed radio telescope –In NL 2008: initial roll out 20 stations –In NL 2009: 18 central (24 Gbps) + 18 remote (2 Gbps) phased array antenna stations each ~4 soccerfields size –Full scale aperture synthesis array, extends 100 km Two main bands –High Band ~ 7,500 tiles 120 – 240 MHz –Low Band ~ 7,500 dipoles 20 – 80 MHz Digital Radio 40 MHz processing bandwidth –Extreme tunability in time/space/frequency –Large instantaneous sky coverage Fibre network, Software Correlator Run as a broad common-user observatory –Making the transition now from “project” to “observatory” Large distributed radio telescope –In NL 2008: initial roll out 20 stations –In NL 2009: 18 central (24 Gbps) + 18 remote (2 Gbps) phased array antenna stations each ~4 soccerfields size –Full scale aperture synthesis array, extends 100 km Two main bands –High Band ~ 7,500 tiles 120 – 240 MHz –Low Band ~ 7,500 dipoles 20 – 80 MHz Digital Radio 40 MHz processing bandwidth –Extreme tunability in time/space/frequency –Large instantaneous sky coverage Fibre network, Software Correlator Run as a broad common-user observatory –Making the transition now from “project” to “observatory”

E-LOFAR Participants GLOW: -- Effelsberg station delivered -- Garching, Potsdam, Tautenburg stations ordered --Jülich station federal funding proposal submitted LOFAR-UK: – Chilbolton, Cambridge, Jodrell, Edinburgh – 1 station funded, site to be chosen --3 more stations Sweden: – Onsala station funded + Poland, Italy, Austria, Bulgaria, Ireland, France GLOW: -- Effelsberg station delivered -- Garching, Potsdam, Tautenburg stations ordered --Jülich station federal funding proposal submitted LOFAR-UK: – Chilbolton, Cambridge, Jodrell, Edinburgh – 1 station funded, site to be chosen --3 more stations Sweden: – Onsala station funded + Poland, Italy, Austria, Bulgaria, Ireland, France EURO-VO: GRID and VO

Wires in the field EURO-VO: GRID and VO

Station based processing –Receiver –Filter –Beamformer –Buffer –Receiver –Filter –Beamformer –Buffer –Receiver –Filter –Beamformer –Buffer –Receiver –Filter –Beamformer –Buffer –10 GbE –1 GbE –Used for 1 beam & station cross correlation Input data rate: ~ 460 Gbps Output data rate: ~ 2 Gbps Processing capacity: ~ 1.5 Tmul/s Storage capacity: 96 Gbyte

Lofar: Data Stream EURO-VO: GRID and VO

Lofar Storage

Lofar Long term (>2weeks) archive and user enabling ADD EURO-VO: GRID and VO

STREAM of MetaData Design: data driven vs. compute driven from the field to central processor Distributed – p2p / federated –Communities –Dataservers – own + EGEE GRID –Compute servers – own+ EGEE GRID AstroWISE EURO - VO Design: data driven vs. compute driven from the field to central processor Distributed – p2p / federated –Communities –Dataservers – own + EGEE GRID –Compute servers – own+ EGEE GRID AstroWISE EURO - VO EURO-VO: GRID and VO

LoWISE design

Lofar-WISE: Data Storage –Groningen –SARA Amsterdam EURO-VO: GRID and VO

IBM – GPFS filesystem Connect to AstroWISE –Storage table IBM – GPFS filesystem Connect to AstroWISE –Storage table EURO-VO: GRID and VO

LoWise- concepts All Tier 1s on an equal footing Centrally operated GPFS distributed filesystem File retrieval –Db viewer (web) –Prompt (db query) –Pathname – data server –gLite – Grid ftp Single distributed db- with all Meta and source data –Valid --Quality -- ready for publication flags Visibility of data –Owner –Project –Archive --EURO-VO Authentication / Assigned resources –AstroWISE Oracle system All Tier 1s on an equal footing Centrally operated GPFS distributed filesystem File retrieval –Db viewer (web) –Prompt (db query) –Pathname – data server –gLite – Grid ftp Single distributed db- with all Meta and source data –Valid --Quality -- ready for publication flags Visibility of data –Owner –Project –Archive --EURO-VO Authentication / Assigned resources –AstroWISE Oracle system EURO-VO: GRID and VO

Processing - options Tier 0 – CP Tier 1 – re-processing –Local clusters –EGEE GRID Tier 2 –On Tier 1 –Local anarchy Tier 0 – CP Tier 1 – re-processing –Local clusters –EGEE GRID Tier 2 –On Tier 1 –Local anarchy EURO-VO: GRID and VO

AstroWISE-Up

EURO-VO: GRID and VO –VST - Virtual Survey Telescope

EURO-VO: GRID and VO –

EURO-VO: GRID and VO

Astro-WISE: Services EURO-VO: GRID and VO

TARGET diagram EURO-VO: GRID and VO

QC - calibration scientist monitoring

EURO-VO: GRID and VO Extreme data lineage

Astro-WISE: Grid Computing –selection of Target in Astro-Wise –storage element defined a priori –EGEE computing EURO-VO: GRID and VO

LoWise and Euro-VO EURO-VO: GRID and VO

GRID- EuroVO –Awe visualization

EURO-VO: GRID and VO –AI – Cultural Heritage

EURO-VO: GRID and VO /Monk