1. www.g-vo.org The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs

2. www.g-vo.org Overview The Virtual Observatory –Motivation –Approach The IVOA –Standards –Examples, demos private demos are always possible ! GAVO –Project –Results –Outlook Theory in the VO: –The Millennium Simulation database

3. www.g-vo.org Astronomical data sets are large, and getting larger Simulations as well (courtesy Simon White) (courtesy Alex Szalay)

4. www.g-vo.org Most data is (or could be) online Google (http://www.google.com/Top/Science /Astronomy/Data_Archives/)http://www.google.com/Top/Science /Astronomy/Data_Archives/ MPE (http://www.mpe.mpg.de/AstR/ )http://www.mpe.mpg.de/AstR/

5. www.g-vo.org Opportunities So, the Internet is (or could be) the world’s best telescope: –It has data on every part of the sky –In every measured spectral band: optical, x-ray, radio.. –As deep as the best instruments (2 years ago). –It is up when you are up –It’s a smart telescope: links objects and data to literature on them –It even contains truly virtual data (courtesy Alex Szalay)

6. www.g-vo.org

7. Problems Data formats not uniform Interfaces not uniform Data sets very large Data set far away (makes them larger) How to find data of interest? Computers have difficulty understanding web pages

8. www.g-vo.org Promises (/hopes) of the VO. To make results of astronomical research, more readily available to the community –data and applications To facilitate their –communication, reuse, comparison, combination, checking

9. www.g-vo.org How ? Online availability of datasets and applications Standardisation of –publication and discovery mechanisms –description/meta-data (FITS ++ ) –selection mechanisms –form(at)s of transmitted data –data access protocols Why standardisation?

10. www.g-vo.org Babylonian confusion SDSS ROSAT 2MASS FIRST GAIA N x M

11. www.g-vo.org Why standardisation

12. www.g-vo.org We need a common language ROSAT FIRST GAIA SDSS2MASS N + M

13. www.g-vo.org Standardise on: Discovery Data description (“meta-data”) Data formats Protocols (Web) Application Interfaces Query language

14. www.g-vo.org Why (standardised) protocols? VO service oriented. Data sets too large to move : –You can FTP 1 MB in 1 sec (optimistically) –You can FTP 1 GB in min (= 1 $/GB) –You can FTP 1 TB in 2 days and 1K$ –You can FTP 1 PB in 3 years and 1M$ (numbers from Alex Szalay) –SFTP even slower Remote filtering required to reduce size data sets “Move analysis to data”: services Allow programmatic access to services

15. www.g-vo.org

16. VO’s esperanto

17. www.g-vo.org Warning VO can not (and does not aim to) be everything to everyone Users will have to be able to visit the underlying data in all gory detail: provenance Even then standardisation helps Agreement is hard to come by (see FITS) Problems are hard ! VO is still partially a research project.

18. www.g-vo.org The International Virtual Observatory Alliance (IVOA) The International Virtual Observatory Alliance (IVOA) Facilitate the international coordination and collaboration necessary for the development and deployment of the tools, systems and organizational structures necessary to enable the international utilization of astronomical archives as an integrated and interoperating virtual observatory.

19. www.g-vo.org Alliance of national VOs

20. www.g-vo.org Working and Interest Groups Working and Interest Groups WGs –Standards and Process: how the IVOA works –Resource registry: where to register and discover resources –Data Modeling: how to describe data sets –Semantics: how to understand one another –VOTable: standard format for tabular data sets –Data Access Layer: very simple data access services –VO Query Language: more sophisticated data access –Grid and web services: programmatic accessibility –Applications: stand alone, and together –VOEvent: astronomical telegrams in XML IGs –Theory: virtual observations for virtual universes –Open GridForum Astro-RG: VObs and virtual organisations –Data Curation and Preservation: how not to loose your data

21. www.g-vo.org Publication standard: Resource Registry Implementations: –Searchable Registry at STScISearchable Registry at STScI –NCSA Publishing Registry –ESAVO Full Registry Client tools: –VOExplorer (AstroGrid) –Splat, SpecView, Aladin

22. www.g-vo.org Messaging standard: VOTable http://www.ivoa.net/twiki/bin/view/IVOA/IvoaVOTable XML format for tabular data Column definitions included with data Binary streams and FITS included Understood by many client tools (see applications set) –TOPCAT –VOPlot –Aladin –Many others

23. www.g-vo.org VOTable example

24. www.g-vo.org Data Access Protocols I Simple protocols for discovering and retrieving data sets Generally HTP-GET based: –Base url: http://www.g-vo.org/ssa?action=doQuery&archiveid=ssa.service.cdfs –Query parameters: POS=53.161,-27.832&SIZE=0.2&BAND=0/&TIME=,&FORMAT=ALL Returns (heavily) annotated list of spectra with access URL for retrieving them.

25. www.g-vo.org Data Access Protocols II Examples from GAVO, for more usage examples see tool demosGAVO –Simple cone search (source catalogues)Simple cone search ROSAT source catalogues and photon event lists) –Simple Image AccessSimple Image Access ROSAT pointed observations and All-Sky Survey fields –Simple Spectral AccessSimple Spectral Access Chandra Deep Field South optical follow up

26. www.g-vo.org Registry + DAL protocols: Interoperability Standard services, once registered, can be found by client tools …

27. www.g-vo.org Interoperability ….and executed together (too many ROSAT results to show all here!)

28. www.g-vo.org Interoperability … and shown together

29. www.g-vo.org Applications, tools VO aware tools: –Images: Aladin –Source lists and tables: TOPCAT, VOPlot –Spectra: Splat, SpecView, VOSpec –3D: VisIVO Application interoperability: Plastic –Messaging standard –Tying TOPCAT to Aladin to Splat to … –Uses VOTable to send data from one app to another –All on your desktop

30. www.g-vo.org GAVO

31. GAVO: VO in Germany BMBF funded In 2 nd phase of financing Institutes/people –AIP: M. Steinmetz (Co-I), I. Nickelt –ARI-ZAH: PI J. Wambsganss (PI), M. Demleitner, U. Stampa –MPE: W. Voges (Co-I), J.Kim, G.Lemson –Uni Tϋbingen: T. Rauch –TUM-Informatik: B. Gufler

32. www.g-vo.org GAVO tasks Investigate archiving and IVOA compatible publication Investigate data mining techniques Participate in -, and represent IVOA efforts Special emphasis on theory in the VO GAVO-expertise center and archive @ARI Support for custom services Info/help on all things VO for Germany (Please come by !)

33. www.g-vo.org GAVO services, standard and custom SCS and SIAP for ROSAT results (J.Zuther)ROSAT Cross matcher tool (used by S. Komossa) SSA for CDFS spectra (G. Szokoly)CDFS Publication A. Finoguenov’s A3266 data.A3266 SSA for TMAP synthetic spectra (T. Rauch, Tubingen)SSA Various virtual telescopes (Planck@MPA, K.Dolag, J. Blaizot)virtual telescopes RASS source classifier (Class-X team) Cluster finder (P. Schuecker et al) –Now in AstroGrid-D (A. Carlson)

34. www.g-vo.org GAVO current and planned projects GAVO@ARI –Archive for small science results –Feedback on all things GAVO –Hopefully long term Collaborative tool –Automated upload and annotation of data –Wrap with IVOA protocols –Pre-publication collaboration capabilities Simulation registry

35. www.g-vo.org Theory in the VO

36. www.g-vo.org Observations in the VO Most VO efforts concentrate on observational data sets –simple observables: photons detected at a certain time from a certain area on the sky –long history of archiving –pre-existing standards (FITS) –valuable over long time (digitising 80 yr old plates) Standards observationally biased –common sky: cone search, SIAP, region –common objects: XMatch –data models: characterisation of sky/time/energy(/no polarisation yet)

37. www.g-vo.org Theory in the VO: issues Simulations not so simple –complex observables –no standardisation (not even HDF5) –archiving ad hoc, for local use Moore’s law makes useful lifetime relatively short: few years later can do better Current IVOA standards somewhat irrelevant –no common sky –no common objects –requires data models for content, physics, code

38. www.g-vo.org The Millennium database + web server Post-processing results only SQLServer database Web application –portal: http://www.mpa-garching.mpg.de/millennium/http://www.mpa-garching.mpg.de/millennium/ –public DB access: http://www.g-vo.org/Millenniumhttp://www.g-vo.org/Millennium –private access: http://www.g-vo.org/MyMillenniumhttp://www.g-vo.org/MyMillennium Access methods –browser with plotting capabilities through VOPlot applet –wget + IDL or R –TOPCAT plugin

39. www.g-vo.org Usage statistics