Virtual organizations in astronomy and beyond Prof. Giuseppe Longo Chair of Astrophysics - Department of Physical Sciences University of Napoli Federico II – Italy National Institute of Astrophysics – Napoli Unit Tblisi, March

The Exponential Growth of Information in Astronomy Total area of 3m+ telescopes in the world in m 2, total number of CCD pixels in Megapix, as a function of time. Growth over 25 years is a factor of 30 in glass, 3000 in pixels. Gigapixel arrays are a reality,hence optical and near infrared surveys are becoming common Space missions archives are being federated Old datasets (space and ground based instruments) are being federated Estimated 1 TB per day in 2008

Massive numerical simulations Distributed computing (PB per simulation) Astronomy, more than other sciences is facing a Major Data Avalanche ( … a true tsunami…) Large survey projects from ground and from space (past, ongoing, future) Distributed data repositories Data are not where the users are PetaBytes of data / week GRID INFRASTRUCTURE Data federation of MDS Adoption of standards and common onthologies Data analysis and interpretation Need for a new generation of tools (A.I. based) capable to work in a distributed environment International Virtual Observatory Alliance

The distributed environment Once the VO’a will come operationals, there will be no need to have locally powerful computing facilities,

Federation of existing and new databases through adoption of common standards Network access to the databases To provide the user with user friendly access to all federated data To allow the user to access distributed computing facilities and to exploit all available data withouth moving the data but the codes (… data remain at data centers where the expertise is) To open entirely new paths to discovery process in astronomy (but not only!) What are some of the goals of VO’s

VO are the most democratic tool ever implemented by any scientific community. Data repositories are mostly public (either immediately or after proprietary period of observers) Data analysis and data mining tools are available to the international community through a distributed computing environment Every one can contribute (either with new data or with new SW-tools) Once the VO be implemented, new – top level science will be at the “fingers” of any competent scientist who has minimal computing facilities and a good access to the WWW

What is being done in Napoli 1 – The surveys VLT Survey Telescope (Napoli,ESO) P.I: Prof. M. Capaccioli 2.5 m diameter - OPTICAL 1x1 sq deg f.o.v. 16 k x 16k CCD mosaic (optical) New technology Adaptive optics 0.2 arcsec psf Operational end GB raw data/night Nobel laureate R. Giacconi visiting VST factory

VLT site, cerro Paranal (Chile)

Omegacam French – Netherlands – Italy consortium 16 k x 16 k array CCd mosaic Ready Data processing pipeline European FP6 network ASTROWISE Real time storage and processing of the VST data What is being done in Napoli 2 – The detector

What is being done in Napoli 3 – The computing Campus GRID nodes 150 TB storage (IBM, DEC - Alpha, etc.) 16 GBaud optical fibers backbone Recently evolved into PON - SCOPE 3.6 M€ (8.2 M€ total) for Hardware (512 boards with 4 CPU’s) Financed by Italian Government Operational end 2007

What is being done in Napoli 4 – The Data mining Draco Project building the GRID infrastructure for the Italian VO 400 k€ - MIUR Euro – VO, VO-Tech European Virtual Observatory Technological Infrastructures European Infrastructures for VO (UK, D, I, F, etc.) 6.6 M€ - EU VO- Neural (Napoli lead) Building Data Mining and Visualization for Massive Data Sets in a Distributed Environment Cost- Action 283 EU

Parameter space of incredibly high dimensionality (N>>100) Complex parameter space

Example 1: panchromatic view of the universe X Opt. radio IR. Crab Nebula: SN 1054 a.C.

Example 2: a new way to do conventional astronomy Selection of quasar candidates from a 3 band photometric survey

Example: exploring a 3D Parameter Space Given an arbitrary parameter space: Data Clusters Points between Data Clusters Isolated Data Clusters Isolated Data Groups Holes in Data Clusters Isolated Points Nichol et al Slide courtesy of Robert CalTech.

Example: 21-D parameter space VO- Neural Probabilistic Principal Surfaces Negative ENtropy Clustering + Dendrogram

Multiwavelenght – multiepoch – multinstrument data (federation of databases) hence there is a strong need for a new generation of data processing, data visualization and data-mining tools These tools must be largely based on Artificial Intelligence Interoparibility is a must (Plastic is a standard) Many probles to be solved : Missing data (bew data models are needed) Parallelization of existing codes Sensibilization of the community through selected scientific cases (astrophysics, bioinformatic, marketing, etc.) THESE TOOLS ARE OF WIDE APPLICATION: bioinformatics, geophysics (environment, stratigraphy, etc.), business (stock market, marketing strategies, etc.). Therefore interdisciplinarity is a must!

We (UK, F, I, D, USA, India) intend to pursue the above tasks using the following instruments: National funds and private companies EU funds through new COST Action and ITN Eventually through RI US funds through NSF Conferences and Schools for young students (dissemination is CRUCIAL) NEW POTENTIAL PARTNERS ARE ENCOURAGED TO CONTACT ME : Plate or digital archives of astronomical data Other types of scientific data Advanced programming and mathematical know-how’s