11 July 2002NVO EPO Workshop1 The National Virtual Observatory Robert Hanisch Space Telescope Science Institute Project Manager NSF NVO Project (with liberal borrowing from NVO Project collaborators)

11 July 2002NVO EPO Workshop2 The National Virtual Observatory National Academy of Sciences Decadal Survey recommended NVO as highest priority small (<$100M) project Several small initiatives recommended by the committee span both ground and space. The first among themthe National Virtual Observatory (NVO)is the committees top priority among the small initiatives. The NVO will provide a virtual sky based on the enormous data sets being created now and the even larger ones proposed for the future. It will enable a new mode of research for professional astronomers and will provide to the public an unparalleled opportunity for education and discovery. (p.14)

11 July 2002NVO EPO Workshop3 What is the Virtual Observatory? Real Virtual sky archives, catalogs, published literature telescope Internet detectors, computer programs instruments site the users desktop telescope control, instrument control, data acquisition, data processing, data storage astronomers, technicians, engineers, programmers, support staff, …

11 July 2002NVO EPO Workshop4 The Virtual Observatory… Provides observers with access to all archived astronomical data as if it were stored on the local computer Provides tools to locate and retrieve data of interest, regardless of where it is stored Provides tools to compare data from different real telescopes and instruments Provides computational services and data management services on a supercomputer scale

11 July 2002NVO EPO Workshop5 The Exponential Growth of Information in Astronomy Total area of 3m+ telescopes in the world in m 2, total number of CCD pixels in Megapix, as a function of time. Growth over 25 years is a factor of 30 in glass, 3000 in pixels. Moores Law growth in CCD capabilities/size Gigapixel arrays are on the horizon Improvements in computing and storage will track the growth in data volume Data growth rate implies that data storage cannot be centralized Data volume and complexity are increasing!

11 July 2002NVO EPO Workshop6 The Exponential Growth of Information in Astronomy Total area of 3m+ telescopes in the world in m 2, total number of CCD pixels in Megapix, as a function of time. Growth over 25 years is a factor of 30 in glass, 3000 in pixels. Moores Law growth in CCD capabilities/size Gigapixel arrays are on the horizon Improvements in computing and storage will track the growth in data volume Data growth rate implies that data storage cannot be centralized Data volume and complexity are increasing!

11 July 2002NVO EPO Workshop7 Discoveries When and where are discoveries made? –Always at the edges and boundaries –Going deeper, using more colors…. –Physicists make many measurements and discard most; Astronomers make many measurements and find wealth in their entirety and combination (J. Ostriker, 6/14/02) Metcalfes law –Utility of computer networks grows as the number of possible connections: O(N 2 ) VO: Federation of N archives –Possibilities for new discoveries grow as O(N 2 ) Current sky surveys have proven this –Very early discoveries from SDSS, 2MASS, DPOSS

11 July 2002NVO EPO Workshop8 New Science Rare and exotic objects –Very high redshift quasars –Brown dwarfs –Time-variable objects, transient events: distant supernovae and microlensing –Dark matter in the galactic halo –Variable stars –Asteroids

11 July 2002NVO EPO Workshop9 New Science An example of a possible new type of a phenomenon, which can be discovered through a systematic exploration of the T ime Domain : A normal, main-sequence star which underwent an outburst by a factor of > 300. There is some anecdotal evidence for such megaflares in normal stars. The cause, duration, and frequency of these outbursts is currently unknown. Will our Sun do it? A new generation of synoptic sky surveys may provide the answers -- and uncover other new kinds of objects or phenomena.

11 July 2002NVO EPO Workshop10 New Science Structure and evolution of the universe –Proper statistical comparison between local and distant samples –Cluster surveys as tracer of large-scale structures –Automated detection of arc-shaped objects to locate gravitational lenses

11 July 2002NVO EPO Workshop11 New Science A fully digital Galaxy –Star catalogs (stellar evolution, stellar dynamics) –Interstellar medium –Role of close encounters and influence on star formation –Comparison with theoretical models and simulations

11 July 2002NVO EPO Workshop12 New Science Census of active galactic nuclei –Systematic searches for black holes –Panchromatic approach to circumvent obscuration problems Search for extra-solar planets –Search for planet transits in much larger data samples (Large Synoptic Survey Telescope), bolstered by supporting astrometric data Theoretical astrophysics –Globular cluster modeling –Galaxy mergers –Evolution of large-scale structure

11 July 2002NVO EPO Workshop13 New Technologies Standardizing access to distributed data –Web Services: XML: Extensible Markup Language SOAP: Simple Object Access Protocol WSDL: Web Services Description Language RDF: Resource Description Framework UDDI: Universal Description, Discovery and Integration Standardizing distributed computing –Grid Services Custom configure remote computing dynamically Build your own remote computer, and discard Virtual Data: new data sets on demand The Grid the tools for building the semantic web "The Semantic Web is an extension of the current web in which information is given well-defined meaning, better enabling computers and people to work in cooperation." -- Tim Berners-Lee, James Hendler, Ora Lassila, The Semantic WebThe Semantic WebScientific American, May 2001

11 July 2002NVO EPO Workshop14 New Technologies Standardizing access to distributed data –Web Services: XML: Extensible Markup Language SOAP: Simple Object Access Protocol WSDL: Web Services Description Language RDF: Resource Description Framework UDDI: Universal Description, Discovery and Integration Standardizing distributed computing –Grid Services Custom configure remote computing dynamically Build your own remote computer, and discard Virtual Data: new data sets on demand The Grid the tools for building the semantic web "The computational grid is analogous to the electric power grid. Grid computing allows to couple geographically distributed resources and offers consistent and inexpensive access to resources irrespective of their physical location or access point. It enables sharing, selection, and aggregation of a wide variety of geographically distributed computational resources (such as supercomputers, compute clusters, storage systems, data sources, instruments, people), thus allowing them to be used a single, unified resource for solving large-scale compute and data intensive computing applications. --

11 July 2002NVO EPO Workshop15 New Technologies Standardizing access to distributed data –Web Services: XML: Extensible Markup Language SOAP: Simple Object Access Protocol WSDL: Web Services Description Language RDF: Resource Description Framework UDDI: Universal Description, Discovery and Integration Standardizing distributed computing –Grid Services Custom configure remote computing dynamically Build your own remote computer, and discard Virtual Data: new data sets on demand The Grid the tools for building the semantic web

11 July 2002NVO EPO Workshop16 How Will It Work? Define commonly used `atomic services Build higher level toolboxes/portals on top We do not build `everything for everybody Use the rule: –Define the standards and interfaces –Build the framework –Build the 10% of services that are used by 90% –Let the users build the rest from the components

11 July 2002NVO EPO Workshop17 NSF NVO Project NSF ITR project, Building the Framework for the National Virtual Observatory is a collaboration of 17 funded and 3 unfunded organizations –Astronomy data centers –National observatories –Supercomputer centers –University departments –Computer science/information technology specialists PI and project director: Alex Szalay (JHU) CoPI: Roy Williams (Caltech/CACR) $10M award for five-year period, beginning 1 Nov 01 –4-5% for EPO coordination, plus in-kind contributions

11 July 2002NVO EPO Workshop18 Project Management NSF CISE + AST External Review Committee PI/Project Director: Szalay Co-PI/Chief Architect: Williams Executive Committee Data Centers Project Scientist Project Manager System Architect E&O Coordinator Technical Working Group Science Working Group InfrastructureActivities Local/Distant Universe Digital Milky Way Rare/Exotic Objects AGN Census Extra-Solar Planets Science Prototypes Theoretical Astrophysics Portals/Workbenches Metadata Standards Grid Services/Testbed Data Models DataAccess/Resources Data Providers E&O Coordinator Project Manager

11 July 2002NVO EPO Workshop19 Education & Outreach Integral part of project Emphasis is on development of partnerships Kick-start with this workshop –Understand requirements on NVO services from perspective of formal education, informal education, commercial/corporate, and public outreach content developers

11 July 2002NVO EPO Workshop20 Education/Outreach Partners Association of Science- Technology Centers International Planetarium Society National Air and Space MuseumSilicon Graphics (Digital Planetarium) Spitz (Electric Sky)Maryland Space Grant Consortium Gettysburg College (Project CLEA) UC Berkeley American Museum of Natural History

11 July 2002NVO EPO Workshop21 Milestones Nov 2001 – Jan 2002: Established project structure April 15, 2002: VOTable V1.0 May 1, 2002: 50+ ConeSearch services registered May 8, 2002: Defined initial science demos June 13, 2002: Formed International VO Alliance July 11-12, 2002: EPO Workshop Nov 15, 2002: Internal testing of science demos January 2003: Initial science demonstrations (AAS) August 2003: Intermediate NVO science demos (IAU)

11 July 2002NVO EPO Workshop22 Critical Path Science demonstrations –Identified, scoped, and scheduled Service registry issues –International coordination User interface issues –Retrofit existing portals EPO requirements –Impact on metadata standards

11 July 2002NVO EPO Workshop23 Role of Science Prototypes Keep focus on user and science needs Identify most common services Verify standardization efforts Encourage data providers to participate Demonstrate to community that NVO tools will –arrive soon –will be useful for everybody –can evolve incrementally First science demos planned for January 2003

11 July 2002NVO EPO Workshop24 Initial Science Prototypes Brown-Dwarf search –Distributed query across several archives –Correlations with non-detections –Example of typical NVO search Gamma-Ray burst –Event follow up service –Exercise in standards compliance/interoperabilty Galaxy evolution in clusters –On-the-fly image analysis and pattern recognition –Exercise in grid computing

11 July 2002NVO EPO Workshop25 SkyQuery Distributed Query tool using a set of services Feasibility study, built in 6 weeks from scratch –Tanu Malik (JHU CS grad student) –Tamas Budavari (JHU astro postdoc) Won 2 nd prize in Microsoft.NET Contest Allows queries like: SELECT o.objId, o.ra, o.r, o.type, t.objId FROM SDSS:PhotoPrimary o, TWOMASS:PhotoPrimary t WHERE XMATCH(o,t)<3.5 AND AREA(181.3,-0.76,6.5) AND o.type=3

11 July 2002NVO EPO Workshop26

11 July 2002NVO EPO Workshop27 ConeSearch Search for catalog objects around a point Returns data in VOTable format Requires a registered profile Point of the exercise –A learning experience –Existing archives test and implement VOTable –Understand service description issues Automated test and verification In less than two weeks 7 groups, 50 services Cross-Identification service built on top

11 July 2002NVO EPO Workshop28

11 July 2002NVO EPO Workshop29 International Collaboration European initiatives underway –Astrophysical Virtual Observatory funded by European Commission (3.3 million, three years) –AstroGrid, funded by UK e-science program (£5 million, three years) Other international efforts starting: –Canada (C$4M recently approved), India, Japan, Chile, Germany, Russia, Australia International VO roadmap announced at Garching VO conference, 10 June 2002 International VO Alliance formed, 13 June 2002 Regular telecons among NVO, AVO, and AstroGrid leadership Frequent technical contacts among partners

11 July 2002NVO EPO Workshop30 Summary NSF ITR NVO project is one of four major and numerous other small VO-related initiatives now underway world-wide NVO is adopting, adapting, or developing necessary technology as derived from science requirements NVO project is dealing with many of the management challenges that will face the ultimate VO organization NVO (and only NVO) has emphasis on EPO as integral component of project