Presentation is loading. Please wait.

Presentation is loading. Please wait.

2004-09-13NVO Summer School, Aspen Center for Physics1 National Virtual Observatory Applications Software Development Summer School: Introduction and Overview.

Similar presentations


Presentation on theme: "2004-09-13NVO Summer School, Aspen Center for Physics1 National Virtual Observatory Applications Software Development Summer School: Introduction and Overview."— Presentation transcript:

1 2004-09-13NVO Summer School, Aspen Center for Physics1 National Virtual Observatory Applications Software Development Summer School: Introduction and Overview Bob Hanisch Project Manager, US NVO Project Space Telescope Science Institute T HE US N ATIONAL V IRTUAL O BSERVATORY

2 2004-09-13NVO Summer School, Aspen Center for Physics2 Motivation NVO development now three years along…software environment is sufficiently well-developed that the community can start to build VO-enabled applications Want community to start to see the VO as an important toolkit for doing astronomical research and public outreach/education Want feedback from the community on how to improve and extend VO tools Want you to be our agents, our collaborators

3 2004-09-13NVO Summer School, Aspen Center for Physics3 Faculty The faculty is drawn from the NVO Project development team Programmers and programmer/scientists with experience in development of web-based applications and services Introductions…

4 2004-09-13NVO Summer School, Aspen Center for Physics4 Faculty Tamas Budavari, The Johns Hopkins University, Baltimore

5 2004-09-13NVO Summer School, Aspen Center for Physics5 Faculty Dave De Young, National Optical Astronomy Observatories, Tucson

6 2004-09-13NVO Summer School, Aspen Center for Physics6 Faculty Matthew Graham, Caltech, Pasadena

7 2004-09-13NVO Summer School, Aspen Center for Physics7 Faculty Gretchen Greene, Space Telescope Science Institute, Baltimore

8 2004-09-13NVO Summer School, Aspen Center for Physics8 Faculty Bob Hanisch, Space Telescope Science Institute, Baltimore

9 2004-09-13NVO Summer School, Aspen Center for Physics9 Faculty Tom McGlynn, NASA Goddard Space Flight Center, Greenbelt, MD

10 2004-09-13NVO Summer School, Aspen Center for Physics10 Faculty Maria Nieto-Santisteban, The Johns Hopkins University, Baltimore

11 2004-09-13NVO Summer School, Aspen Center for Physics11 Faculty Wil OMullane, The Johns Hopkins University, Baltimore

12 2004-09-13NVO Summer School, Aspen Center for Physics12 Faculty Ray Plante, National Center for Supercomputing Applications, Urbana, IL

13 2004-09-13NVO Summer School, Aspen Center for Physics13 Faculty Doug Tody, National Radio Astronomy Observatory, Socorro, NM

14 2004-09-13NVO Summer School, Aspen Center for Physics14 Faculty Roy Williams, Caltech, Pasadena (Roy)

15 2004-09-13NVO Summer School, Aspen Center for Physics15 Thanks to… NSF and NASA for their financial support Aspen Center for Physics, president Dave De Young, for agreeing to provide the facility and Jane Kelly for helping with all contractual and logistical matters Shelly Meyett, Ranpal Gill, and Sadie Lingham for registration, catering, travel, and all other matters of logistical support

16 2004-09-13NVO Summer School, Aspen Center for Physics16 Logistical Matters Breakfast and lunch to be provided at ACP, except no lunch on Wednesday (free afternoon). Summer School dinner Wednesday evening at the Mezzaluna Restaurant (see city maps) Wireless network available here at ACP and at Aspen Meadows ACP bicycles are available See also http://chart.stsci.edu/twiki/bin/view/Main/ LogisticalInformation, or ask Ranpal or Janehttp://chart.stsci.edu/twiki/bin/view/Main/ LogisticalInformation

17 2004-09-13NVO Summer School, Aspen Center for Physics17 About the Program Day 1: Overview and tour of science capabilities (Magical Mystery Tour) Day 2: How the VO worksstandards, protocols, web services, clients and servers (A Hard Days Night) Day 3: The registry (how to publish and find things) and the Grid (how to Compute), plus free time (Strawberry Fields Forever) Day 4: A look inside some existing VO science applications, followed by participants forming teams and working on their own projects (With a Little Help from My Friends) Day 5: Participants show their work (Ticket to Ride)

18 2004-09-13NVO Summer School, Aspen Center for Physics18 Caveats The VO is a work in progress. Standards and interfaces are in development and changing. Future versions will be similar to current ones, but code may break and have to be updated. Software libraries and tools have not yet been used by many people outside of the development projectsyou will probably uncover bugs and be able to make things fail. Please tell us what works and what doesnt. This is our first Summer School. We dont know if we have the right balance of topics and time. We will try to adjust as we go along. Well take a few minutes for feedback at the end of each day. Participants have diverse interests and backgrounds. Material may be advanced for some, simple for others. When we get to doing hands-on work, we ask that the more experienced software people lend a hand.

19 2004-09-13NVO Summer School, Aspen Center for Physics19 Your Project Start thinking NOW about a project you might like to do at the end of the week. We will form teams of ~4 people each to work together on these projects. Some suggestions are posted on the SS web. On Friday the faculty will review the presentations, and select what they consider to be the best project. One member of that project team (chosen by the group) will get an expense-paid trip to the San Diego AAS meeting (January 2005) and will be invited to speak in the special session on VO Applications.

20 2004-09-13NVO Summer School, Aspen Center for Physics20 NVO: History and Motivation

21 2004-09-13NVO Summer School, Aspen Center for Physics21 History 1990s: NASA establishes wavelength-oriented science archive centers; multiple large ground-based digital sky survey projects initiated April 1999, Decadal Survey Panel on Theory, Computation, and Data Discovery met in Los Alamos –Szalay, Prince, and Alcock coin the name National Virtual Observatory November 1999, NVO organizational workshop at JHU February 2000, 2 nd NVO workshop at NOAO-Tucson June 2000, conference held at Caltech, Towards a Virtual Observatory June 2000, ad hoc steering committee formed February 2001, AASC/NAS report Astronomy and Astrophysics in the New Millennium released April 2001, proposal submitted to NSF ITR program, 17 collaborating organizations, led by A. Szalay (JHU) September 2001, NSF announces proposal selection January 2003, first NVO science prototypes shown at Seattle AAS

22 2004-09-13NVO Summer School, Aspen Center for Physics22 Decadal Survey Recommendation National Academy of Sciences Decadal Survey recommended NVO as highest priority small (<$100M) project Several small initiatives recommended by the committee span both ground and space. The first among themthe National Virtual Observatory (NVO)is the committees top priority among the small initiatives. The NVO will provide a virtual sky based on the enormous data sets being created now and the even larger ones proposed for the future. It will enable a new mode of research for professional astronomers and will provide to the public an unparalleled opportunity for education and discovery. Astronomy and Astrophysics in the New Millennium, p. 14

23 2004-09-13NVO Summer School, Aspen Center for Physics23 What the Virtual Observatory is… A suite of international standards for the discovery, exchange, intercomparison, and analysis of network-accessible astronomical data A data access and analysis environment that exploits the emerging computation/software/data Grid A framework for data processing that enables and encourages the re-use of algorithms A tool for science planning: Identify gaps in coverage of parameter space. Which new missions, instruments, experiments will have largest impact? A catalyst for world-wide access to astronomical archives A routinely used tool of the research astronomer A vehicle for education and public outreach

24 2004-09-13NVO Summer School, Aspen Center for Physics24 What the Virtual Observatory is not… A replacement for building new telescopes and instruments A centralized repository for data A data quality enforcement organization

25 2004-09-13NVO Summer School, Aspen Center for Physics25 Astronomy is Facing a Data Avalanche Multi-Terabyte (soon: multi- Petabyte) sky surveys and archives over a broad range of wavelengths Billions of detected sources, hundreds of measured attributes per source 1 nanoSky (HDF-S) 1 microSky (DPOSS)

26 2004-09-13NVO Summer School, Aspen Center for Physics26 The Changing Face of Observational Astronomy Large digital sky surveys are becoming the dominant source of data in astronomy: > 100 TB, growing rapidly –Current examples: SDSS, 2MASS, DPOSS, GSC, FIRST, NVSS, RASS, IRAS; CMBR experiments; Microlensing experiments; NEAT, LONEOS, and other searches for Solar system objects … –Digital libraries: ADS, astro-ph, NED, CDS, NSSDC –Observatory archives: HST, CXO, space and ground-based –Future: QUEST2, LSST, and other synoptic surveys; GALEX, SIRTF, astrometric missions, GW detectors Data sets orders of magnitude larger and more complex than in the past Roughly 1 TB/Sky/band/epoch (1 arcsec pixels, 1 byte per pixel) –Human Genome is < 1 GB, Library of Congress ~ 20 TB

27 2004-09-13NVO Summer School, Aspen Center for Physics27 Science of a Qualitatively Different Nature Statistical astronomy done right –Precision cosmology, Galactic structure, stellar astrophysics –Discovery of significant patterns and multivariate correlations –Poissonian errors unimportant Systematic exploration of the observable parameter spaces –Searches for rare or unknown types of objects and phenomena –Low surface brightness universe, the time domain –Confronting massive numerical simulations with massive data sets

28 2004-09-13NVO Summer School, Aspen Center for Physics28 Precision Cosmology and a better marriage of theory and observations DPOSS clustersNumerical simulation

29 2004-09-13NVO Summer School, Aspen Center for Physics29 Multi-Wavelength Data paint a more complete (and more complex!) picture of the universe Infrared emission from interstellar dust Smoothed galaxy density map

30 2004-09-13NVO Summer School, Aspen Center for Physics30 A Panchromatic Approach to the Universe… …reveals a more complete physical picture The resulting complexity of data translates into increased demands for data analysis, visualization, and understanding

31 2004-09-13NVO Summer School, Aspen Center for Physics31 The Time Domain Megaflares on normal main sequence stars (DPOSS)

32 2004-09-13NVO Summer School, Aspen Center for Physics32 Faint, Fast Transients The Time Domain

33 2004-09-13NVO Summer School, Aspen Center for Physics33 NVO: Technology and Infrastructure

34 2004-09-13NVO Summer School, Aspen Center for Physics34 Registries NVO Resource Discovery Computational Services Virtual Data ConeSearch SIAP, SSAP VOTable FITS, GIF,… Catalogs, Archives, Collections, Models ADQL, OSQ NVO Data Access Layer Queries Responses Portals, User Interfaces, Tools VOPlotMirageTopcatconVOTDIS HTTP, Web, & Grid Services NVO Architecture Data Models, UCDs, Metadata Day 1

35 2004-09-13NVO Summer School, Aspen Center for Physics35 Registries NVO Resource Discovery Computational Services Virtual Data ConeSearch SIAP, SSAP VOTable FITS, GIF,… Catalogs, Archives, Collections, Models ADQL, OSQ NVO Data Access Layer Queries Responses Portals, User Interfaces, Tools VOPlotMirageTopcatconVOTDIS HTTP, Web, & Grid Services NVO Architecture Data Models, UCDs, Metadata Day 2

36 2004-09-13NVO Summer School, Aspen Center for Physics36 Registries NVO Resource Discovery Computational Services Virtual Data ConeSearch SIAP, SSAP VOTable FITS, GIF,… Catalogs, Archives, Collections, Models ADQL, OSQ NVO Data Access Layer Queries Responses Portals, User Interfaces, Tools VOPlotMirageTopcatconVOTDIS HTTP, Web, & Grid Services NVO Architecture Data Models, UCDs, Metadata Day 3

37 2004-09-13NVO Summer School, Aspen Center for Physics37 NVO Architecture Registry Layer Existing Data Centers Data Services Semantics (UCD) SIAP, SSAP VOTable FITS, GIF,… OpenSkyQuery SkyQueryVOPlot OASIS conVOT Topcat Mirage AladinDIS Disks, Tapes, CPUs, Fiber Grid Middleware SRB, Globus, OGSA SOAP, GridFTP data mining visualization image source detection Digital Library Other registries XML, DC, METS OAI ADS My Space storage services Databases, Persistency, Replication Virtual Data Workflow (pipelines) Discover Compute Publish Collaborate Authentication & Authorization crossmatch HTTP ServicesSOAP Services Grid Services stateless, registered & self-describing & persistent, authenticated Portals, User Interfaces, Tools Compute Services Bulk Access interfaces to data

38 2004-09-13NVO Summer School, Aspen Center for Physics38 VOTable Reached international agreement on VOTable V1.0 specification in April 2002 XML-based standard with in-line data or links to external data Utilized for basic catalog and image access protocols Merges AstroRes heritage with XML flexibility Complements FITS Multiple I/O libraries available (Java, Perl, C++, C#)

39 2004-09-13NVO Summer School, Aspen Center for Physics39 Data Models Data modeling effort aimed at defining basic data types and relationships among them High-level entities: image, spectrum, time series, catalog Low-level entries: quantity, resolution, time of observation Interfaces and protocols for other VO services derived from DM relationships

40 2004-09-13NVO Summer School, Aspen Center for Physics40 Data Access Layer Data Access Layer is mediator between NVO data requests and data delivery Defined Cone Search protocol and have ~100 implementations Defined Simple Image Access Protocol (SIAP) and have 20+ implementations OpenSkyQuery provides database interface to OpenSkyNodes with extended SQL-like VO Query Language Specification for Simple Spectral Access Protocol in development

41 2004-09-13NVO Summer School, Aspen Center for Physics41 Resource Metadata and Registry Resource Metadata describes NVO data collections, services; this metadata is collected into a Registry Resource Identifiers are component of resource metadata; have agreed on syntax Using Open Archive Initiative protocols for metadata harvesting Now focusing on query mechanisms and general updating/synching options Prototype registry utilized in science demonstration, Data Inventory Service

42 2004-09-13NVO Summer School, Aspen Center for Physics42 Unified Content Descriptors UCDs provide common data dictionary for describing contents of catalogs CDS initiative, now broadened to international VO discussion Current discussion focusing on structure and extensibility

43 2004-09-13NVO Summer School, Aspen Center for Physics43 VO Query Language Working on minimal extensions to SQL to support astronomical queries (e.g., spatial proximity) Astronomical Data Query Language Defining standard query service based on SDSS SkyQuery: OpenSkyNode and OpenSkyQuery Investigating higher-level query languages; natural language Xquery

44 2004-09-13NVO Summer School, Aspen Center for Physics44 Grid and Web Services Increasing number of web services (cone search and SIAP wrappers, for example) Registry services will be implemented as web services Prototyped use of Grid in galaxy morphology science demonstration ROME (Remote Object Management Environment) project provides stateful user interface to Grid-based or other compute- intensive applications Working closely with Grid community to understand progress on Grid services, e.g., OGSA, and to determine best time to adopt

45 2004-09-13NVO Summer School, Aspen Center for Physics45 NVO-Enabled Science

46 2004-09-13NVO Summer School, Aspen Center for Physics46 Science Prototypes Science prototypes guide and validate technical initiatives NVO Year 1 –Brown dwarf candidate search –Gamma-ray burst follow-up –Galaxy morphology measurement (utilizing computational grid) Year 1.5 –Data Inventory Service Year 2 –Data Inventory Service with registry-based resource selection –Access to theoretical simulation (globular cluster) and virtual observations European VO project –Type 2 (obscured) quasars: 40 new candidates found –Galactic star formation regions

47 2004-09-13NVO Summer School, Aspen Center for Physics47 Data Inventory Service: NVO DataScope Positions of HST and Chandra observations for GRB010222 Scientific Motivation: Rapid collection of multi-wavelength imaging, catalog and observation data following an interesting transient event is essential. This service can also be used as a general tool to quickly access all data available on any patch of sky for any science use. Data Resources: Multi-wavelength data from any number of sites (currently 13 different sites) sampling energies from X-ray to radio, and including images, object lists, and catalogues of observations. What the VO Brings: Integration and organization of a variety of data sources into an easily comprehensible information set. Scalability to an arbitrary number of data providers. Integrates data with multiple data visualization services. Enabling Technologies: Standard protocols to remote services such as Cone Search and Simple Image Access, standardized VOTables for data retrieval transformation, and standardized semantics encoded as Uniform Content Descriptors (UCDs). Resource registry. Future Prospects: Customization and quality control of resources searched; more sophisticated use of metadata.

48 2004-09-13NVO Summer School, Aspen Center for Physics48 Data Inventory Service

49 2004-09-13NVO Summer School, Aspen Center for Physics49 Data Inventory Service

50 2004-09-13NVO Summer School, Aspen Center for Physics50 Data Inventory Service

51 2004-09-13NVO Summer School, Aspen Center for Physics51 Brown Dwarf Candidate Search Scientific Motivation: The search for brown dwarfs has been revolutionized by the latest deep sky surveys. A key attribute to discovering brown dwarfs is the federation of many surveys over different wavelengths. Such matching of catalogs is currently laborious and time consuming. This matching problem is generic to many areas of astrophysics. Data Resources: Sloan Digital Sky Survey (SDSS) Early Data Release (15 million objects) 2-Micron All Sky Survey (2MASS) 2 nd Incremental Point Source Catalog (162 million objects) What the VO Brings: Today, doing datasets is user-intensive and is replicated by many different users. Also, the correlation of these two datasets can take years of CPU time if not done correctly. The NVO brings two key aspects to this problem. First, it removes the need for the user to download large data to their machine, making direct use of distributed data. Second, the matching algorithm used here is computa-tionally efficient and designed to give answers in minutes rather than hours; results can be returned to the user in real-time. Sloan z magnitudes vs. 2MASS J magnitudes, with brown dwarf candidates in red. Data are from the SDSS Early Data Release and 2MASS 2nd Incremental Release. Future Prospects: Catalog matching of large datasets is a generic problem in astrophysics. Therefore, making the matching facility available to any user for use on any dataset will greatly enhance the productivity of scientists. Standard I/O formats allow developers to create tools to use the matched data and easily integrate with existing visualization and analysis tools (anomaly detector). Bringing these data together on remote machines with enough CPU to perform analysis (Grid technology) will allow cross-comparisons of unprecedented scale.

52 2004-09-13NVO Summer School, Aspen Center for Physics52 As a T dwarf becomes cooler (i.e., methane and water absorptions increase) or more distant… –SDSS detects it only at z band –2MASS detects it only at J band

53 2004-09-13NVO Summer School, Aspen Center for Physics53 Demo Leads to Discovery! New brown dwarf candidate confirmed spectroscopically with Keck Observatory

54 2004-09-13NVO Summer School, Aspen Center for Physics54 Galaxy Morphology in Clusters Scientific Motivation: Investigate the dynamical state of galaxy clusters and galaxy evolution within the context of large-scale structure. Use galaxy morphology as a probe of dynamical history by calculating, for each galaxy in a cluster: Surface brightness Concentration index Asymmetry index These parameters are analyzed with other indicators such as magnitude, color, peculiar velocity, position in cluster, and cluster large-scale structure. Data Resources:Computing Resources: Chandra X-ray image (SAO/CXC)USC/ISI ROSAT image (GSFC/HEASARC)UW-Madison/NCSA DSS image (STScI/MAST)Fermilab Galaxy cluster catalogs (NED) CNOC1 cluster images and catalogs (CADC) What the VO Brings: Distributed data access and Grid-based computing make possible for the first time effective integration of multiple datasets and real-time computing. Integration of data from diverse sources is enabled by standardized data objects and standardized remote computing services. Flexibility of access means that further NVO-compliant images and catalogs can be added easily. Users can select their visualization portal (Aladin, OASIS, DS9). Enabling Technologies: VOTable, NVO-compliant catalog and image access, standard semantics, Grid computing infrastructure. Future Prospects: Dynamic discovery and selection of image, catalog, and computing resources. User-selection of analysis tools and ability to publish data to the NVO framework.

55 2004-09-13NVO Summer School, Aspen Center for Physics55 Galaxy Morphology in Clusters

56 2004-09-13NVO Summer School, Aspen Center for Physics56 Galaxy Morphology in Clusters

57 2004-09-13NVO Summer School, Aspen Center for Physics57 Galaxy Morphology in Clusters

58 2004-09-13NVO Summer School, Aspen Center for Physics58 Globular Cluster Simulations

59 2004-09-13NVO Summer School, Aspen Center for Physics59 Globular Cluster Simulations

60 2004-09-13NVO Summer School, Aspen Center for Physics60 Spectral Database Browser

61 2004-09-13NVO Summer School, Aspen Center for Physics61 Spectral Database Browser

62 2004-09-13NVO Summer School, Aspen Center for Physics62 Spectral Database Browser

63 2004-09-13NVO Summer School, Aspen Center for Physics63 Spectral Database Browser

64 2004-09-13NVO Summer School, Aspen Center for Physics64 Spectral Database Browser

65 2004-09-13NVO Summer School, Aspen Center for Physics65 NVO: Broader Context and Vision

66 2004-09-13NVO Summer School, Aspen Center for Physics66 International VO Alliance

67 2004-09-13NVO Summer School, Aspen Center for Physics67 International VO Alliance The IVOA brings together the astronomers, developers, and managers of the VO initiatives world-wide –Agreements on standards for data access (VOTable, catalog queries, image retrieval, resource descriptions, etc.) –Coordination of development activities –Sharing of software –Sharing of experience 15 participating organizations: Astrogrid, AVO, US-NVO, VO-Australia, VO-Canada, VO-China, VO-France, VO- Germany (GAVO), VO-Spain, VO-Hungary, VO-India, VO- Italy (DRACO), VO-Japan, VO-Korea, VO-Russia http:www.ivoa.net

68 2004-09-13NVO Summer School, Aspen Center for Physics68 International VO Alliance

69 2004-09-13NVO Summer School, Aspen Center for Physics69 VO and Future Large Facilities Software and data management are major components of future large facilitiesVO infrastructure should help mitigate costs Maximum scientific return from future large facilities depends on ease of comparison with other data, availability of standard data products Education and outreach enabled by VO provides agencies, taxpayers with visible return on investment

70 2004-09-13NVO Summer School, Aspen Center for Physics70 The VO Vision The VO is the semantic web for astronomy (Tim Berners-Lee) The VO democratizes astronomical research The VO brings the universe to your desktop –The professional astronomer –Graduate students –Undergraduates –K-12 –Amateurs –The public http://us-vo.org

71 2004-09-13NVO Summer School, Aspen Center for Physics71 On with the tour…

72 2004-09-13NVO Summer School, Aspen Center for Physics72 On with the tour…


Download ppt "2004-09-13NVO Summer School, Aspen Center for Physics1 National Virtual Observatory Applications Software Development Summer School: Introduction and Overview."

Similar presentations


Ads by Google