Presentation is loading. Please wait.

Presentation is loading. Please wait.

NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 1 Virtual Observatories...or how to do your research from a beach in the Bahamas rather.

Published byAusten Bryan Modified over 8 years ago

Similar presentations

Presentation on theme: "NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 1 Virtual Observatories...or how to do your research from a beach in the Bahamas rather."— Presentation transcript:

1

2 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 1 Virtual Observatories...or how to do your research from a beach in the Bahamas rather than stuck up a cold mountain in the middle of the night...

3 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 2 Who I am John Taylor: jdt@roe.ac.ukjdt@roe.ac.uk Work on the AstroGrid and VOTech Virtual Observatory projects Member of Edinburgh's Wide Field Astronomy Unit – curates catalogue and image data from SuperCOSMOS and (soon) WFCAM and (less soon) VISTA Software engineer NOT Astronomer

4 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 3 Outline The data deluge Why science is changing The scale of the problem and technical issues Catalogues How to use them VOTables, FITS files Visualisation tools The Virtual Observatory AstroGrid Demonstration

5 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 4 I: The astronomer's world is changing... Previously an Astronomer was a “radio astronomer” or a “x-ray astronomer”. Now you need to be an expert in all wavelengths. The move towards Survey Astronomy Federating data from different instruments. Vast quantities of auto-generated data See: http://grist.caltech.edu/sc4devo/What_is_the_VO.pdf http://grist.caltech.edu/sc4devo/What_is_the_VO.pdf

6 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 5

7 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 6 Supernova Remnant Cassiopeia-A – a 300 year old Supernova Shocks seen in the X- ray Chandra image Heavy elements seen in the optical Dust shows in the IR Images from Chandra Science Centre Slide: Nic Walton, AstroGrid Project Scientist Mapping e¯s in the magnetic field as revealed by Radio data

8 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 7 Data Deluge (Ex-)Concorde (15 km) CD stack with 1 year LHC data (~ 20 km) SuperCOSMOS: 2.5 Tb WFCAM: 14Tb per year VISTA: 4Tb per week Physicists, Astronomers, Biologists all facing the same problem Steve Lloyd: http://www.gridpp.ac.uk /talks/Data_Deluge_Bru nel_2005.ppt

9 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 8 Nearby Spiral Galaxy x-ray optical radio

10 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 9 II: Catalogue and Image Servers Cone searches Data formats: FITS and VOTables Visualising the data: Topcat and Aladin More sophisticated searches using SQL Demo of the SuperCOSMOS Science Archive

11 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 10

12 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 11 Example: SuperCosmos Science Archive Scans of the UK Schmidt telescope plates Catalogue data & Image data 3 wavebands, 2 epochs Over 1 billion objects 2.5 terabytes in size

13 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 12 Data Formats FITS – the astronomer's jpeg images/spectra/tables Includes metadata VOTable “xml” format for tables The VO standard UCDs describe what a quantity “is”

14 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 13 Visualising the data TOPCAT For viewing and manipulating tables, especially VOTables Create new columns Filter the sources Output the table in different formats Aladin Good for viewing image files Overlay catalogue tables Create RGB images Access data archives such as Vizier Aims to be a general “Virtual Observatory” portal

15 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 14 Complex queries: SQL SQL is a general language for querying databases select ra,dec,mag FROM sources where ra>0 and ra -45 and dec 23 give me ra, dec and mag values (use * to get everything) table name (each db will have several) the following conditions are true

16 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 15 SQL II Plenty of resources on the web for learning SQL e.g. http://www.w3schools.com/sql/default.asphttp://www.w3schools.com/sql/default.asp There is an astronomy variant of SQL called ADQL (astronomy data query language) It's basically SQL with a few extra bits thrown in (a CIRCLE keyword for doing cone searches, and an XMATCH keyword for cross-matching data between databases) [Cross Matching: Taking two or more different database tables from (maybe) different sources and joining them together based on their RA and DEC]

17 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 16 Some databases SIMBAD - Basic data on objectshttp://simbad.u- strasbg.fr/Simbadhttp://simbad.u- strasbg.fr/Simbad SuperCOSMOS Science Archive http://surveys.roe.ac.uk/ssa/ SDSS (Sloan Digital Sky Survey) http://www.sdss.org/ http://www.sdss.org/ 2mass (2 micron all sky survey) http://www.ipac.caltech.edu/2mass/ http://www.ipac.caltech.edu/2mass/ XMM http://xmm.vilspa.esa.es/external/xmm_data_acc/x sa/index.shtml http://xmm.vilspa.esa.es/external/xmm_data_acc/x sa/index.shtml

18 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 17 Databases of databases Vizier http://vizier.u-strasbg.fr/viz-bin/VizieRhttp://vizier.u-strasbg.fr/viz-bin/VizieR Datascope http://heasarc.gsfc.nasa.gov/cgi- bin/vo/datascope/init.plhttp://heasarc.gsfc.nasa.gov/cgi- bin/vo/datascope/init.pl NED (Nasa Extragalactic Database) http://nedwww.ipac.caltech.edu/ http://nedwww.ipac.caltech.edu/ OpenSkyQuery http://openskyquery.net/Sky/skysite/ http://openskyquery.net/Sky/skysite/

19 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 18 This presentation will be available at: http://wiki.astrogrid.org/pub/Main/JohnTaylor/neon.sxi

20 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 19 A recap on yesterday Is the SSA now behaving? Last night's observing group: how to use SQL to get your list of galaxies from SDSS select g.ra,g.dec,g.objId, s.z from Galaxy g, ELRedShift s where g.specObjID=s.specObjID and s.z>0 and s.z 0 and.........

21 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 20 III: Virtual Observatories: What's a virtual observatory? What do you need to make one? AstroGrid – glue for the Euro-VO Demo

22 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 21 The International Virtual Observatory

23 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 22 Virtual Observatory “The VO vision can be summed up as the desire to make all archives speak the same language − all searchable and analysable by the same tools, all data sources accessible through a common interface, all data held in distributed databases that appear as one.” Andy Lawrence, 09/2003 Take all the world's Astronomical data, and all the world's applications and computing power and make it appear that they reside on the user's desktop.

24 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 23 Science Cases – Brown Dwarf Location Brown Dwarfs are cool, faint and (if can see them) close Roughly speaking: Query optical and infrared catalogues for objects that are “bright” in the infra red, but not in the optical and have large proper motions. Requires use of more than one catalogue More: “The AstroGrid 10” http://wiki.astrogrid.org/bin/view/Astrogrid/Science Problems http://wiki.astrogrid.org/bin/view/Astrogrid/Science Problems

25 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 24 AstroGrid Components PAL CEA

26 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 25 Example workflow: calculating photometric redshifts

27 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 26 AstroGrid Components

28 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 27 Demo

29 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 28 Contacts and info Contacts: John Taylor jdt@roe.ac.uk Institute for Astronomy, Royal Observatoy of Edinburgh Background Info on AstroGrid http://www.astrogrid.org Download the Euro-VO AstroGrid software from: http://software.astrogrid.org

30 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 29 What now? AstroGrid Release 1 is up and running now http://www.astrogrid.org/release http://wiki.astrogrid.org/bin/view/Astrogrid/ReleaseV1Pages

31 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 30 Locating data and applications IVO-standard registry Registries harvest from each other Xml-based query language ADQL

32 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 31 Workflow Orchestrate services Queries and workflows are designed using the portal – other clients on the way Work is run remotely and asynchronously Archives searched and results manipulated Results are stored in a virtual file system Queries and workflows can be re-used and shared

33 A PPARC funded project AstroGrid Intro & Demo John Taylor Institute for Astronomy, Edinburgh

34 NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 33 Building a VO - Layer Diagram

Download ppt "NEON Obs School 11-Aug-2005 Archival Data and Virtual Observatories 1 Virtual Observatories...or how to do your research from a beach in the Bahamas rather."

Similar presentations

VO/IVOA and The Astronomy Community Dave De Young NOAO.

VO/IVOA and The Astronomy Community Dave De Young NOAO.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.

September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.

September 13, 2004NVO Summer School1 VO Protocols Overview Tom McGlynn NASA/GSFC T HE US N ATIONAL V IRTUAL O BSERVATORY.
VO Standards – Catalog Access Tamás Budavári Johns Hopkins University.

VO Standards – Catalog Access Tamás Budavári Johns Hopkins University.
John Cunniffe Dunsink Observatory Dublin Institute for Advanced Studies Evert Meurs (Dunsink Observatory) Aaron Golden (NUI Galway) Aus VO 18/11/03 Efficient.

John Cunniffe Dunsink Observatory Dublin Institute for Advanced Studies Evert Meurs (Dunsink Observatory) Aaron Golden (NUI Galway) Aus VO 18/11/03 Efficient.
What does LOFAR have to do with the Virtual Observatory (VO)? LOFAR Science Day 16 December 2003 Melbourne David Barnes The University of Melbourne.

What does LOFAR have to do with the Virtual Observatory (VO)? LOFAR Science Day 16 December 2003 Melbourne David Barnes The University of Melbourne.
The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.

The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.
VO-DAS Chenzhou CUI Chao LIU, Haijun TIAN, Yang YANG, etc National Astronomical Observatories, CAS.

VO-DAS Chenzhou CUI Chao LIU, Haijun TIAN, Yang YANG, etc National Astronomical Observatories, CAS.
MAST-VizieR/NED cross correlation tutorial 1. Introduction Figure 1: Screenshot of the MAST VizieR Catalog Search Form. or enter here as object class:

MAST-VizieR/NED cross correlation tutorial 1. Introduction Figure 1: Screenshot of the MAST VizieR Catalog Search Form. or enter here as object class:
A PPARC funded project Tony Linde Programme Manager eScience meets eFrameworks 28 th April 2006 NeSC, Edinburgh.

A PPARC funded project Tony Linde Programme Manager eScience meets eFrameworks 28 th April 2006 NeSC, Edinburgh.
AstroGrid: Virtual Observatory Service Nicholas Walton AstroGrid: Virtual Observatory Service Nicholas Walton AstroGrid and Euro-VO Technology Centre Project.

AstroGrid: Virtual Observatory Service Nicholas Walton AstroGrid: Virtual Observatory Service Nicholas Walton AstroGrid and Euro-VO Technology Centre Project.
Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.

Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.
Evanthia Hatziminaoglou, ESO - Garching Astronomy with Virtual Observatories, Pune 15-19 October 2007 VO Tools.

Evanthia Hatziminaoglou, ESO - Garching Astronomy with Virtual Observatories, Pune October 2007 VO Tools.
ADASS, London Sep 23-26, 2007 VOExplorer : Visualising Data Discovery in the Virtual Observatory Jonathan Tedds University of Leicester/AstroGrid/ EuroVO.

ADASS, London Sep 23-26, 2007 VOExplorer : Visualising Data Discovery in the Virtual Observatory Jonathan Tedds University of Leicester/AstroGrid/ EuroVO.
Leicester Database & Archive Service J. D. Law-Green, J. P. Osborne, R. S. Warwick X-Ray & Observational Astronomy Group, University of Leicester What.

Leicester Database & Archive Service J. D. Law-Green, J. P. Osborne, R. S. Warwick X-Ray & Observational Astronomy Group, University of Leicester What.
AstroGrid Group 7: Teemu Toivola Tero Viitala. Problem several separate databases no common interface between databases difficulties of joining related.

AstroGrid Group 7: Teemu Toivola Tero Viitala. Problem several separate databases no common interface between databases difficulties of joining related.
Aus-VO: Progress in the Australian Virtual Observatory Tara Murphy Australia Telescope National Facility.

Aus-VO: Progress in the Australian Virtual Observatory Tara Murphy Australia Telescope National Facility.
Cool white dwarfs in the Sloan & SuperCOSMOS Sky Surveys Nigel Hambly, Wide Field Astronomy Unit, IfA, University of Edinburgh.

Cool white dwarfs in the Sloan & SuperCOSMOS Sky Surveys Nigel Hambly, Wide Field Astronomy Unit, IfA, University of Edinburgh.
S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCDs - metadata for astronomy Sébastien Derriere François Ochsenbein Thomas Boch CDS, Observatoire astronomique.

S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCDs - metadata for astronomy Sébastien Derriere François Ochsenbein Thomas Boch CDS, Observatoire astronomique.
A long tradition. e-science, Data Centres, and the Virtual Observatory why is e-science important ? what is the structure of the VO ? what then must we.

A long tradition. e-science, Data Centres, and the Virtual Observatory why is e-science important ? what is the structure of the VO ? what then must we.

Similar presentations

Ads by Google