VO Query Language GSFC XML Group Ed Shaya Brian Thomas Kirk Borne

May IVO Cambridge VOQL Requirements Provide a means for users to submit general requests for astronomical information from a distributed set of repositories. Allow for the science use cases. Easy to learn and use: – Hide from the user obvious but tedious steps – May require several levels o f language with only the top level being easy. Allow for web form entry. Independent of internal arrangement of data at repositories. Plug-n-play metadata and ontology. Span a distributed set of heterogeneous services. – Each VO query can transform to multiple queries in local dialects. – Workflow of interactions between registries, services, and user. – Integration of multiple responses

May IVO Cambridge More VOQL Requirements Easy to parse and transform into other forms Extensible – Sites can extend query language through local namespaces – VO namespace can add language elements into the future.

May IVO Cambridge XML Query Language Compatible XML and Human-Readable versions Xquery is a superset of Xpath Based on Quilt, XQL, and XML-QL – Quilt is based on Object Query Langauge (OQL) – OQL is based on Structured Query Language (SQL) If,then,else: case switch: basic functions: define new functions FLWR (for, let, where, return) for $i in (1 to 3) let $j := (1 to $i) Results in: $i = 1, $j = 1 $I = 2, $j = (1,2) $I =3, $j = (1,2,3)

May IVO Cambridge XQuery Continued for $s in document('‘bright_stars.xml'')/*/id_main let $b := document('‘photometry.xml'')/*/star[name = $s]/band where count ($b) > 1 return $i for $j in (2 to count($b)) $b[$j]/value - $b[$j-1]/value

May IVO Cambridge

May IVO Cambridge OLAP/XMLA On-line Analytical Processes Reduces bandwidth/time of data out Statistical Package add on to Databases Analysis of DataCubes – Hierarchy of Axis Values Years, Months, Days, Hours, minutes Degrees, minutes, seconds Interior, core, mantle, atmosphere, mesosphere, exosphere

May IVO Cambridge JVO Query Language – Naoki Yasuda Retrieves catalog data and images from multiple data servers via a single user interface Extension of SQL – Catalog.UCD – Box(Point(c1.ra,c1.dec), width1,height1) – XMATCH(c1,c2,!c3,…)< 3 arcsec – Select Catalog: Keyword1 & Keyword2 Select by [[MAX|MIN](PROPERTY) | ALL] [NAME] – Area : [inside|outside] area0 Area1 [overlap|union] area2 | shape SHAPE: box, circle, oval, triangle,point DIFF(x.obs_date, y.obs_date) > 30 days

May IVO Cambridge Data mining Beyond finding data; intense data filtering, conditioning, knowledge synthesis. Grid Services? – Principal Component Analysis – Iterative solutions – Genetic algorithms – Maximum-likelihood functions – Neural nets – Decision trees – Cluster analysis – Regression analysis

May IVO Cambridge Data Objects Dataset – Tables Fields – Units – Class (UCD) – Range – Values – Images Axes Coordinate Maps Data Values – Spectra Wavelength Intensity

May IVO Cambridge ADQL Obtain Data Sets – By bibliographic query Author, date published, title, journal, volume – By description Keywords, abstract, mission name Obtain tables – By title, table #, field names – By Xpath /LocalGroup/[galaxy=“M31”]/region7/v-band – Obtain table data by UCDs or field names – Min/max of range, regular expression Obtain N-cube data – Subset by axis values, – subset by ra,dec, radius or more generally Func(axes1..)

May IVO Cambridge Astronomy Data Query Language (ADQL)

May IVO Cambridge ADQL/Query Schema

May IVO Cambridge Knowledge Based Query Class  Instance  Objects Property (V-band)  Instance  value (-1.4) – Measurement property values are Data – Modifier (aperture)  Instance  value (3 arcsec) Modifier (inequality)  Instance  value (before, not) – Aggregate property – member, region, component Values are bags of objects – SubclassOf property – subclass has restricted property value range or restricted list of properties. Property Space – N-properties form a space. A bit of math is needed to relate values.

May IVO Cambridge Problem Statement Language: Root

May IVO Cambridge PSL Constraint

May IVO Cambridge PSL AstroObject

May IVO Cambridge AC : The Astrographic Catalogue on the Hipparcos System 1275 I/275 The AC Catalogue AC : The Astrographic Catalogue on the Hipparcos System S E Urban T E Corbin G L Wycoff E Hoeg C Fabricius V V Makarov Astron. J AJ U Dataset Schema

May IVO Cambridge <keywords xml:base= parentListURL="adc_keywordList.html"> Positional data Astrographic zones Surveys The AC is a revised version of the 1997 release of the AC 2000 (Cat. ). It was decided that the availability of an improved reference catalogue and the inclusion of photometry from the Tycho-2 catalogue would be sufficient to warrant a complete re- reduction of the data and a new distribution of the catalogue. The AC catalog contains positions of 4,621,751 stars at the average epoch of plate exposures for each star (average 1907). Dataset Continued

May IVO Cambridge Case Study 0: Setting up the Query Return RA, Dec, Vmag for stars with 13<Vmag<15 and 10:12:53.5<RA<13:13:43 and 18:38:00<DE< 18:40:00. PSL: ?vmag \ ?ra ?de

May IVO Cambridge Case Study 0: Mapping Query to Metadata Search for tables with metadata that satisfy: – Object/[class=“star”] –search-> keyword, description – –search-> field/UCD, name – –search-> field/UCD, name – –search-> field/UCD, name – Property/range –search-> field/min and field/max or coverage attributes For all such tables, return: ?vmag, ?ra, ?de Also, return for group with Vmag info.

May IVO Cambridge Problem Statement Language (PSL) Begin Request Constraint Find astronomical objects with the following properties: AND these properties 1. Name: assign to var1 2. Class is "cluster of galaxies | galaxy cluster" 3. Measurement quantities satisfy: a. X-ray brightness > 3.3E7Jy : assign to var2 1. Time interval of measurement: 1998Y-1999Y Using the above variables satisfy, the math formulae: 1. (var2 + var3) < (var1 – log[var4]) OR these constraints [several constraints for which one must be true etc ] Return a table with the following sequence of fields: var1 var2 End Request PSL Pull down AndConstrainties, Andproperties PSL Pull down AndConstrainties, Andproperties Property Name Pull Down Name, Class, etc. Property Name Pull Down Name, Class, etc. MathML Pull down *,-,/,+,sum,avg,, etc MathML Pull down *,-,/,+,sum,avg,, etc

May IVO Cambridge Brian Thomas’ Infrastructure

May IVO Cambridge Tony Linde’s Infrastructure VO activity – User User – Problem Assistant – service to help user state the problem Problem Assistant – Ontology – terms and relationships derived from existing data Ontology Workflow – to retrieve data, merge it, analyze it, reduce it Workflow Registry – lists all services and their high level metadata Registry Job Control – decides which jobs and when Job Control Data Centre – receiver of query for all internal data sources Data Centre Data Source Service – uses translator to restate query Data Source Translator – from data query language to implemented service Translator Languages – Problem Statement Language (PSL) Problem Statement Language (PSL) – Workflow Language (WFL) Workflow Language (WFL) – Astronomical dataset Query Language (ADQL) Astronomical dataset Query Language (ADQL) – Ontology Query Language (OQL) Ontology Query Language (OQL) – Registry Query Language (RQL) Registry Query Language (RQL)

May IVO Cambridge Conclusion Metadata should clearly distinguish between values that are property values and those that are modifiers of properties. Then, a mapping from a natural(ish) scientific knowledge based language (PSL) to a request language for data-center common items (ADQL) is possible. A federated system with a VO-wide vocabulary plus specialized (local) namespaces is best for getting started right away and permitting for evolution.