S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCDs - metadata for astronomy Sébastien Derriere François Ochsenbein Thomas Boch CDS, Observatoire astronomique de Strasbourg

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Context: data and metadata in astronomy Astronomers have to deal with large amounts of heterogeneous data: ● articles, publications ● catalogues (tabular data) ● spectroscopy ● images, sky surveys Data are produced by many different sources: how can we compare them ?

S. Derriere et al., ESSW03 Budapest, 2003 May | | | | | 0.09| 0.19| 9.99| | | | | | | | 0.07| 0.17| 9.99| | | | | | | | 0.04| 0.11| 9.99| | | | | | | | 0.18| 0.19| 9.99| | | | | | | | 0.10| 9.99| 9.99| | | | | | | | 0.09| 0.18| 9.99| | | | | | | | 9.99| 0.27| 0.22| | | | | | | | 0.05| 0.13| 9.99| | | | | | | | 0.04| 0.08| 9.99| | | | | | | | 0.04| 0.10| 9.99| | | | | | | | 0.05| 0.09| 9.99| | | | | | | | 0.15| 0.23| 9.99| | | | | | | | 0.06| 0.09| 0.12| | | | | | | | 0.13| 9.99| 9.99| | | | | | | | 9.99| 0.21| 9.99| | | | | | | | 0.06| 0.09| 0.14| | | | | | | | 0.19| 9.99| 9.99| | | | | | | | 0.11| 0.20| 9.99| | | | | | | | 0.10| 0.14| 9.99| | | | | | | | 0.13| 0.18| 9.99| | | | | | | | 0.06| 0.11| 0.16| | | | | | | | 0.11| 0.20| 9.99| | | | | | | | 0.07| 0.12| 0.18| | | | | | | | 0.10| 0.13| 0.22| | | | | | | | 0.11| 9.99| 9.99| | | | | | | | 0.05| 0.12| 9.99| | | | | | | | 9.99| 0.19| 9.99| | | | | | | | 0.10| 0.18| 9.99| | | | | | | | 0.06| 0.09| 0.12| | | | | | | | 9.99| 0.23| 0.27| | | | | | | | 0.05| 0.12| 9.99| | | | | | | | 9.99| 0.24| 9.99| | | | | | | | 0.17| 0.19| 9.99| | | | | | | | 0.05| 0.08| 0.09| | | | | | | | 0.20| 9.99| 9.99| | | DECmagKRAmagJmagI Astronomical catalogues (tables):

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Astronomical tables Description with column names (labels) is not enough! The VizieR service, hosted by the Centre de Données astronomiques de Strasbourg (CDS) is a collection of over 3000 different catalogues. Catalogues can contain several tables. They may contain up to a few hundred columns; the largest has 1 billion rows. Tables come from various origins : ● electronic publication of tables in journals ● digitization of old data (photographic plates) ● recent observations with digital detectors

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Description of tables In the VizieR service, a standardized description is provided for each catalogue (ReadMe file), but: ● Heterogeneous origins, formats and descriptions. ● Each author/consortium is free to label columns with its own « column names » (120 names for a V magnitude in VizieR) and to use fancy units. ● A priori, no automated comparisons between different tables are possible. How to search by content ? « Find catalogues containing proper motion measurements, or Johnson B magnitudes.... »

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Unified Content Descriptors UCDs (Unified Content Descriptors) have been created to provide a semantic description of column contents. Bottom-up origin: manual exploration of hundreds of VizieR catalogues by P. Ortiz For each new concept found, a new UCD was created. UCDs are simple strings, with a short explanation: PHOT_EXTINCTION_ISM Interstellar extinction. POS_GAL_LAT Galactic latitude

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Unified Content Descriptors ~1500 UCDs are sufficient to describe the contents of the 100,000 columns in VizieR. UCDs were built in a 4-level hierarchical structure. But this structure is not mandatory. It is only: - a convenient way of grouping similar elements from a given point of view - a specification of the context (make implicit information explicit) UCDs are not a data model, nor an ontology!

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Usage and tools Some tools have been developed to demonstrate how UCDs can be used to select catalogues, compare data, etc... Tools to manipulate UCDs include a browser to explore the tree, and the search of UCDs by plain text.

S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCD browser

S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCD browser

S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCD browser

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Find relevant catalogues with UCDs

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Search UCD from plain text Ex: magnitude B

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Allows contents-based catalogue selection

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Automated comparisons If two columns have the same UCD, they can be compared! UCD+units = Automated columns conversions

S. Derriere et al., ESSW03 Budapest, 2003 May 20

In the VO context A Virtual Observatory for astronomy: ● Several projects (AVO, NVO, AstroGrid,...) ● Provide astronomers with tools to query, combine and exploit all the available data. ● International Virtual Observatory Alliance (IVOA) We all want INTEROPERABILITY between the different services. First milestone: VOTable 1.0 XML standard for exchanging astronomical tabular data Carry metadata together with data: including UCD

S. Derriere et al., ESSW03 Budapest, 2003 May 20

In the VO context Because UCDs were created from a huge basis of astronomical data, they offer a very good coverage of the semantic domain. They are already used in several services for the VO: Cone search (NVO initiative): retrieve sources around a given position on the sky in some catalogue. ➢ ucd= "ID_MAIN" ➢ ucd= "POS_EQ_RA_MAIN" ➢ ucd= "POS_EQ_DEC_MAIN" Returns VOTable (possibly with UCDs).

S. Derriere et al., ESSW03 Budapest, 2003 May 20 In the VO context Aladin (interactive sky atlas): change the display for a catalogue.

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Perspectives for UCDs in the VO UCDs offer a standard way to describe the semantic contents. They need to be improved, to have more flexibility, and cover some new domains. A few changes will occur in the near future: ● There will be a list of core UCDs, validated by the IVOA. ● Namespaces will be allowed to define some new UCDs ● Use of atoms (modifiers): ivoa:PHOT_JHN_V,ERROR Help people to assign UCDs to their own datasets

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Assignation of UCDs The original descriptions of elements can consist of: ● a name ● a unit ● a description Catalog---Catalogue (H=Hipparcos) (H0) HIP---Identifier (HIP number) (H1) Proxy---Proximity flag (H2)Note on Proxy: this flag RAhms---Right ascension in h m s, ICRS (J ) (H3) DEdms---Declination in deg ' ", ICRS (J ) (H4) Vmagmag? Magnitude in Johnson V (H5) VarFlag---? Coarse variability flag (H6)Note on VarFl

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Assignation of UCDs

S. Derriere et al., ESSW03 Budapest, 2003 May 20 Perspectives for UCDs in the VO UCDs will be used: ● In the registries (descriptions of resources and services); ● In Data Models (attached to attributes of OO-models) --- already done for the IDHA image data model; ● In VOQL, the VO query language (XML or SQL-based? not defined yet !). Towards an ontology: UCDs will be used as a starting point to build an astronomical ontology (proposal submitted for collaboration with IT labs).