Presentation on theme: "THE PON-SCOPE GRID INFRASTRUCTURE P.I. – Giuseppe Marrucci Astrophysics – Longo Electromagnetism - Franceschetti High energy Physics - Merola Computer."— Presentation transcript:
THE PON-SCOPE GRID INFRASTRUCTURE P.I. – Giuseppe Marrucci Astrophysics – Longo Electromagnetism - Franceschetti High energy Physics - Merola Computer sciences - Russo Mathematics - Murli Materials and environment - Barone Medicine and Genetics - Salvatore Social and Human sciences - Zollo
250 CPU quadri-processori Xeon Quad-core EM64T Clovertown E5320, 1.86 GHz, 2x4MB cache, supportati da 8 GB di RAM, 1066 FSB, con 2 dischi SAS 36GB, controller RAID SAS 3 Gb/s. Storage: 100 HDD FC / 300GB (30TB) + 100 HDD SATA 2 da 500GB (50TB), expandable up to 120TB & 240 HD 4 cpu dual core 16GB RAM Rete non blocking with 240 porte 10/20 gigabit infiniband full redundant, 12 PS banda aggregata 12 Gigabit, 2 gateway fiber channel-infiniband with 2 gates FC4 - 8 gigabit. 230 nodi in infiniband; 20 blade forfibre channel connectivity. PON-SCOPE Preexistence Campus GRID Astrogrid SM LHC Tier 2 (INFN) For a total 0f 1 k-CPUs
Astrophysics in GRID-SCOPE Astroparticles Simulations of CR and induced showers – F. Guarino Simulation of NEMO - G. Barbarino Simulations of Gravitational Waves from various astrophysical sources VIRGO – L. Milano Cosmology Simulations of cosmic string signatures on CMB – G. Longo Primordial Nucleosynthesis – G. Miele VO related activity Pipeline for survey data (VST-Tube + 2dphot) – de Carvalho, Grado, La Barbera, Longo Integration of ASTROGRID with GRID-SCOPE (VO-Tech broker) Extractor (image segmentation) – O. Laurino Data Mining Supervised and unsupervised data mining tools Photometric redshifts QSO and AGN search & classification Cluster identification & characterization
G. Longo (PI) M. Brescia (INAF - PM) S. Cavuoti (applications) A. Corazza (models and algorithms) R. DAbrusco (applications)G. dAngelo (documentation, GRID) N. Deniskina (GRID – VO interface)M. Garofalo (applications) O. Laurino (System, Applications)A. Nocella (UML software engineering) G. Riccio (Applications)B. Skordovski (models) External Members C. Donalek (Caltech-CRAC)G. Djorgovski (Caltech-CRAC) The VO-Neural Team http://people.na.infn.it/~astroneural/ http://people.na.infn.it/~astroneural/
THE PROBLEM Grid Launcher (N.V. Deniskina) allows to launch GRID-applications using the ASTROGRID Workbench and to transfer the results from the GRID-UI to the data storage of Astrogrid. VO is an environment open to a wide community & the GRID IS NOT (access through personal certificates) Time consuming tasks cannot be run from VO users unless the security problem is solved (or fooled….)
GRID-Launcher (N.V. Deniskina) Workbench of the user GRID SCOPE AstroGRID MySpace (data storage) UI RB SE WNCE data result
The workflow of the job is following: 1. Grid_launcher a) takes the user input from Workbench of Astrogrid; b) collects all files, tabs and programs needed; c) wraps them in an archive and sends it to the Scope-GRID UI. (The Authentication on Scope is done by public keys exchange). 2. The Scope UI receives data and JDL program from "GRID_launcher", unpacks them and translates them to Grid job format. 3. Once GRID job jdl file is ready, "GRID_launcher" starts it in Grid (from a AstroGrid node); periodically checks the status; and then (when job is finished) retrieves the results. 5. "GRID_launcher" receives the data archive, unpacks it and puts the results into the MySpace data storage of AstroGRID.
GRID launcher has been implemented and tested on : VONeural_MLP: supervised clustering VONeural_SVM: supervised clustering Sextractor: for survey data processing SWARP - is a program to resample and co-add FITS images using any arbitrary astrometric projection defined in the WCS standard.
Tests on scientific cases (done and in progress) Photometric redshifts of SDSS galaxies (DAbrusco et al. ApJ, 2007) uses VONeural_MLP Classification of AGN in UKIDS+SDSS data (Dabrusco et al. 2008, MNRAS in press) Using VONeural_SVM Search for LSB in SDSS data using NEXT (in progress, Laurino)
SDSS-DR4/5 – GG trainingvalidationTest set 60%, 20%, 20% MLP, 1(5), 1(18) 0.01<Z<0.250.25<Z<0.50 99.6 % accuracy MLP, 1(5), 1(23)MLP, 1(5), 1(24) rob = 0.206 rob = 0.234 Interpolation of systematic errors Phot Z for SDSS General Galaxy sample at least 30 experiments (10-12 h/each) training on 350.000 objects 12 features results for 32.000.000 objects
σ z = 0.02 Redshifts for 30 million galaxies Photometric redshifts for 30 million SDSS galaxies
Looking for AGN candidates in SDSS+UKIDS 3-D PCA PPS
SDSS UKIDSS preprocessing clustering labeling BoK results PPS/SVM NEC dendrogram Cluster optimization 1 experiment/one node ca. 11 days Looking for AGN candidates in SDSS+UKIDS 3-D PCA
Applicazione 2 con SVM Miglior Risultato: 81.5% PON-SCOPE GRID Infrastructure (110 nodes PON NA-CA-CT) lg 2 (gamma) lg 2 (C)
What types of programs we plan to launch to GRID? There are typical astronomical tasks that need long-time calculations: 1) all types of numerical simulations., 2) image reduction (+, -, statistic, calibration)., 3) search of astronomical solution (astrometry calibration)., 4) photometry calibration., 5) determination of luminosity function of the galaxies., 6) photometric redshift., 7) source selection (clustering, determination of the type of the object) (mushroom as an example)., These tasks can be as AG programs (or other VO programs ) inside of AG as user's programs. These tasks can be the sequence of programs (not one program).