1. 2 Dec 2004 Technology Open House 2004 1 The National Virtual Observatory T HE US N ATIONAL V IRTUAL O BSERVATORY Bob Hanisch Gretchen Greene Wil O’Mullane

2. 2 Dec 2004 Technology Open House 2004 2 Abstract The National Virtual Observatory has been under development for three years, and is now at the point of making its first release of applications software and tools to the astronomy community. The team will describe the main technology development areas of the VO and demonstrate some of the tools that will be released for general community use in conjunction with the January AAS meeting. We will also briefly describe the software development toolkit that we assembled for the NVO Summer School, which includes software libraries and sample code for VO-compatible data publication and retrieval.

3. 2 Dec 2004 Technology Open House 2004 3 Astronomy is facing a data avalanche Multi-Terabyte (soon: multi- Petabyte) sky surveys and archives over a broad range of wavelengths Billions of sources, hundreds of attributes per source 1 nanoSky (HDF-S) 1 microSky (DPOSS)

4. 2 Dec 2004 Technology Open House 2004 4 The changing face of observational astronomy Large digital sky surveys are becoming dominant source of data in astronomy: > 100 TB, growing rapidly –SDSS, 2MASS, DPOSS, GSC, FIRST, NVSS, RASS, IRAS, QUEST, GALEX, SST; CMBR experiments; Microlensing experiments; NEAT, LONEOS, and other searches for Solar system objects –Digital libraries: ADS, astro-ph, NED, CDS, NSSDC –Observatory archives: HST, CXO, space and ground-based –Future: PanSTARRS, LSST, and other synoptic surveys; astrometric missions, GW detectors Data sets orders of magnitude larger, more complex, more homogeneous than in the past Roughly 1 TB/Sky/band/epoch –Human Genome is < 1 GB, Library of Congress ~ 20 TB

5. 2 Dec 2004 Technology Open House 2004 5 Toward a “new astronomy” Past: Observations of small, carefully selected samples (often with a priori prejudices) of objects in one or a few wavelength bands Future: Multi-wavelength data for millions of objects, allowing us to –Discover significant patterns from the analysis of statistically rich and unbiased image/catalog databases (e.g., Gunn-Peterson effect in high-z quasars) –Understand complex astrophysical systems via confrontation between data and sophisticated numerical simulation Discovering new phenomena and patterns in these datasets will require simultaneous access to multi-wavelength archives, advanced visualization and statistical analysis tools

6. 2 Dec 2004 Technology Open House 2004 6 Motivation National Academy of Sciences Decadal Survey recommended NVO as highest priority small (<$100M) project “ Several small initiatives recommended by the committee span both ground and space. The first among them—the National Virtual Observatory (NVO)—is the committee’s top priority among the small initiatives. The NVO will provide a “virtual sky” based on the enormous data sets being created now and the even larger ones proposed for the future. It will enable a new mode of research for professional astronomers and will provide to the public an unparalleled opportunity for education and discovery.” —Astronomy and Astrophysics in the New Millennium, p. 14

7. 2 Dec 2004 Technology Open House 2004 7 History 1990s: NASA establishes wavelength-oriented science archive centers; multiple large ground-based digital sky survey projects initiated (following 1988 Astrophysics Data System report) April 1999, Decadal Survey Panel on Theory, Computation, and Data Discovery met in Los Alamos –Szalay, Prince, and Alcock coin the name “National Virtual Observatory” November 1999, NVO organizational workshop at JHU February 2000, 2 nd NVO workshop at NOAO-Tucson June 2000, conference held at Caltech, “Towards a Virtual Observatory” June 2000, ad hoc steering committee formed February 2001, AASC/NAS report “Astronomy and Astrophysics in the New Millennium” released April 2001, proposal submitted to NSF ITR program, 17 collaborating organizations, led by A. Szalay (JHU) September 2001, NSF announces proposal selection January 2003, first NVO science prototypes shown at Seattle AAS

8. 2 Dec 2004 Technology Open House 2004 8 What is the Virtual Observatory… and what it is not… The VO is: A set of international standards to share complex data A modular set of tools to work with distributed data A simple environment to publish data to An essential part of the research astronomer’s toolkit A catalyst for world-wide access to astronomical archives A vehicle for education and public outreach The VO is not: A replacement for building new telescopes and instruments A centralized repository for data A data quality enforcement organization

9. 2 Dec 2004 Technology Open House 2004 9 Who is the National Virtual Observatory? US NVO development project, funded by NSF Information Technology Program and managed by NSF Astronomy Division, is entering 4 th year of 5-year project Funding is $10M+ over the 5 years 17 organizations (astro, CS, IT) involved –JHU (PI Alex Szalay), STScI, Caltech (Astronomy, IPAC, CACR), HEASARC, SAO, NRAO, NOAO, NCSA, SDSC, FNAL, USNO, et al. Collaboration being extended to Gemini Science Archive, LSST, Keck

10. 2 Dec 2004 Technology Open House 2004 10 International collaboration NVO is co-founder of the International Virtual Observatory Alliance IVOA now has 15 member projects Adopted a standards process based on W3C Forum for technical development, working groups, discussion and sharing of experience http://ivoa.net

11. 2 Dec 2004 Technology Open House 2004 11 Exposure NVO Summer School (Sept. 2004, Aspen) trained 40 students and software developers in VO tools and technologyNVO Summer School Special Session scheduled for January 2005 AAS meeting: Astronomical Research with the Virtual Observatory; includes several papers based on Summer School projects (environments of radio galaxies, starburst galaxies)Astronomical Research with the Virtual Observatory IAU Joint Discussion on Future Large Telescopes and the Virtual Observatory (July 2003, Sydney)IAU Joint Discussion on Future Large Telescopes and the Virtual IAU Symposium being proposed for 2006 (Prague)

12. 2 Dec 2004 Technology Open House 2004 12 Science prototypes Science demonstrations show capabilities of new infrastructure, motivate and guide technical developments. For example: –Data discovery, multi-λ comparisons –Search for brown dwarfs –Galaxy morphologies in clusters –Globular cluster simulations

13. 2 Dec 2004 Technology Open House 2004 13 Science results Padovani et al. (2004)) demonstrates that VO tools are mature enough to produce cutting-edge science results by exploiting astronomical data beyond classical identification limits (R  25)Padovani et al. (2004)

14. 2 Dec 2004 Technology Open House 2004 14 Science results McGlynn et al. (2004) classified all unidentified ROSAT WGACAT objects using VO data access methods to cross- correlate multi- wavelength catalogs –Technique applied to find candidate X-ray binaries and now to SDSS photometric catalog More than 400 papers related to “virtual observatory” in ADS

15. 2 Dec 2004 Technology Open House 2004 15 Technology development Resource registries Data models Content description (UCDs) Data access layer (SIAP, SSAP, cone search) VO Query Language (ADQL, OpenSkyQuery) VOTable Grid and Web Services Activities can be followed on IVOA web (http://ivoa.net)http://ivoa.net

16. 2 Dec 2004 Technology Open House 2004 16 Software toolkit Assembled software development toolkit for NVO Summer School –AXIS (Java web services) –ANT (Java-based software build tool) –Mirage, Topcat, VOPlot applications –Sample data files STScI Web Services course (http://chart.stsci.edu/twiki/bin/view/Main/ STWebServicesCource)http://chart.stsci.edu/twiki/bin/view/Main/ STWebServicesCource See http://us-vo.org/summer- school/proceedings/software/index.htmlhttp://us-vo.org/summer- school/proceedings/software/index.html Summer School proceedings available at http://us- vo.org/summer-school/proceedings/index.cfmhttp://us- vo.org/summer-school/proceedings/index.cfm

17. 2 Dec 2004 Technology Open House 2004 17 Capabilities Tools and applications available at http://us-vo.org/http://us-vo.org/ Data location and access –Resource registry (yellow pages) has thousands of catalogs and archives and is easily extendedResource registry –DataScope uses registry to locate data and allow user to display or download data of interest; can cache results on popular objects or for transient eventsDataScope Spectrum and filter database and analysis toolsSpectrumfilter HST/MAST VO services Catalog cross-correlation (test version)Catalog cross-correlationtest version SDSS web services Dynamic source-list generation and cross-correlation Visualization of tabular, spectral, and imaging datatabularspectralimaging Source classification Try these yourself at http://chart.stsci.edu/twiki/bin/view/Main/AASJanuary2005 http://chart.stsci.edu/twiki/bin/view/Main/AASJanuary2005

18. 2 Dec 2004 Technology Open House 2004 18 Future VO-enabled science January 2005: software release in conjunction with AAS, international data access through standard registries –Applications you have just seen January 2005 AVO demos focusing on dynamic generation of SEDs for galaxies and comparisons with stellar spectral evolution models, and search for stars in transition from AGB to PN (unidentified IR sources); seeJanuary 2005 AVO demos TeraGrid-based analyses (CPU time already allocated): standard “sky atlas” spatial sampling and data federation, galaxy SED fitting, image mosaicing, quasar spectral modeling, N-point correlation function for SDSS galaxies, CMB modelingTeraGrid-based analyses

19. 2 Dec 2004 Technology Open House 2004 19 Future VO-enabled science NVO applications for 2005-2006 TBD in consultation with Science Steering Committee –Dynamic time series analysis, period fitting –“VO-Google” –Fast data inventory service –Flux-recovery service –Image registration and subtraction services –VO integration with legacy software systems (web service interfaces, data access) –Datamining and data federation on increasingly large, distributed databases

20. 2 Dec 2004 Technology Open House 2004 20 STScI/JHU activities NVO project management Technical development –Resource metadata and registry services –OpenSkyNodes (GALEX, HDF, UDF, GOODS) –HST/MAST catalog and image services with VOTable support and direct links to VOPlot –Links to/from the literature, ADS Collaboration with GOODS team on catalog and image cutout services Collaboration with JHU/SDSS personnel; joint technical discussions, augmented by co-location in Bloomberg building Subscribe to nvo@stsci.edu (majordomo)nvo@stsci.edu

21. 2 Dec 2004 Technology Open House 2004 21 Ear to the ground NSF and NASA working to create joint program; draft RFP could be available as soon as next spring –Agency support is firm, but not unwavering Continuing to build community support from the ground up –Demonstrations –Software releases –Summer School –EPO partnerships

22. 2 Dec 2004 Technology Open House 2004 22 Next steps Submitted white paper “The National Virtual Observatory: From Framework to Facility” to NSF and NASA this past summer –Suggests responsibilities and scope for eventual NVO operational (distributed) facility –Suggests smaller, tighter collaboration –Describes several possible management models, advocates management by consortium –Urges creation of joint NSF/NASA/other program to provide single point of contact for funding

23. 2 Dec 2004 Technology Open House 2004 23 Summary $10M committed in US, >$40M worldwide, to VO development Active international community is working and meeting regularly to establish the VO Major archives and catalogs available through VO and more coming Refereed research papers utilizing VO now beginning to appear

24. 2 Dec 2004 Technology Open House 2004 24