GAVO – after one year H.-M. Adorf & the GAVO team.

Slides:

Advertisements

Similar presentations

World Wide Telescope mining the Sky using Web Services Information At Your Fingertips for astronomers Jim Gray Microsoft Research Alex Szalay Johns Hopkins.

Advertisements

ESA Data Integration Application Open Grid Services for Earth Observation Luigi Fusco, Pedro Gonçalves.

Publishing catalogues: HIPASS and SUMSS Australian Virtual Observatory 2003 Workshop 17th-18th November 2003 Sydney David Barnes, Brett Beeson, Travis.

The Australian Virtual Observatory e-Science Meeting School of Physics, March 2003 David Barnes.

ATLAS/LHCb GANGA DEVELOPMENT Introduction Requirements Architecture and design Interfacing to the Grid Ganga prototyping A. Soroko (Oxford), K. Harrison.

Lehrstuhl Informatik III: Datenbanksysteme Grid-based Data Stream Processing in e-Science 1 Richard Kuntschke 1, Tobias Scholl 1, Sebastian Huber 1, Alfons.

VO-DAS Chenzhou CUI Chao LIU, Haijun TIAN, Yang YANG, etc National Astronomical Observatories, CAS.

The Virtual Observatory in Germany and abroad A status report with examples/demos from GAVO and other national VOs.

Development of China-VO ZHAO Yongheng NAOC, Beijing Nov

László Dobos 1,2, Tamás Budavári 2, Nolan Li 2, Alex Szalay 2, István Csabai 1 1 Eötvös Loránd University, Budapest,

Searching for Tidal Streams in SDSS Chinese Virtual Observatory 刘超中国科学院国家天文台.

1 Software & Grid Middleware for Tier 2 Centers Rob Gardner Indiana University DOE/NSF Review of U.S. ATLAS and CMS Computing Projects Brookhaven National.

ADASS, London Sep 23-26, 2007 VOExplorer : Visualising Data Discovery in the Virtual Observatory Jonathan Tedds University of Leicester/AstroGrid/ EuroVO.

Slide 1 of 9 Presenting 24x7 Scheduler The art of computer automation Press PageDown key or click to advance.

Aus-VO: Progress in the Australian Virtual Observatory Tara Murphy Australia Telescope National Facility.

QCDgrid Technology James Perry, George Beckett, Lorna Smith EPCC, The University Of Edinburgh.

OpenAlea An OpenSource platform for plant modeling C. Pradal, S. Dufour-Kowalski, F. Boudon, C. Fournier, C. Godin.

CERN - IT Department CH-1211 Genève 23 Switzerland t Monitoring the ATLAS Distributed Data Management System Ricardo Rocha (CERN) on behalf.

National Center for Supercomputing Applications GLORIAD Science Applications Astronomy – Virtual Observatories Global Climate Change Richard M. Crutcher.

11/27/2003IVOA Small Projects Meeting China-VO Data Access Service Based on OGSA Jian Sang National Astronomical Observatory of China Chinese Virtual.

INAF - National Institute for Astrophysics The National Institute for Astrophysics coordinates and participates in the Astronomy and Astrophysics (A&A)

Automated Classification of X-ray Sources R. J. Hanisch, A. A. Suchkov, R. L. White Space Telescope Science Institute T. A. McGlynn, E. L. Winter, M. F.

The Japanese Virtual Observatory (JVO) Yuji Shirasaki National Astronomical Observatory of Japan.

The GAVO Cross-Matcher Application Hans-Martin Adorf, Gerard Lemson, Wolfgang Voges GAVO, Max-Planck-Institut für extraterrestrische Physik, Garching b.

WSRF Supported Data Access Service (VO-DAS)‏ Chao Liu, Haijun Tian, Dan Gao, Yang Yang, Yong Lu China-VO National Astronomical Observatories, CAS, China.

Theory in the German Astrophysical VO Summary: We show results of efforts done within the German Astrophysical Virtual Observatory (GAVO). GAVO has paid.

Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.

The ASDC SED Builder Milvia Capalbi (INAF-ASDC) in collaboration with Paolo Giommi (ASI-ASDC), Giulia Stratta (INAF-ASDC), Roberto Primavera (ElsagDatamat)

Hello!. International Virtual Observatory Alliance Ajit Kembhavi, IUCAA, Pune.

Prototype system of the Japanese Virtual Observatory The Japanese Virtual Observatory (JVO) aims at providing easy access to federated astronomical databases.

QCDGrid Progress James Perry, Andrew Jackson, Stephen Booth, Lorna Smith EPCC, The University Of Edinburgh.

JVO JVO Portal Japanese Virtual Observatory (JVO) Prototype 2 Masahiro Tanaka, Yuji Shirasaki, Satoshi Honda, Yoshihiko Mizumoto, Masatoshi Ohishi (NAOJ),

4th EGEE user forum / OGF 25 Catania, TheoSSA on AstroGrid-D Iliya Nickelt (AIP / GAVO), Thomas Rauch (IAAT / GAVO), Harry Enke (AIP.

DOMENICO TALIA (joint work with M. Cannataro, A. Congiusta, P. Trunfio) DEIS University of Calabria ITALY Grid-Based Data Mining and.

Theory in the Virtual Observatory Gerard Lemson, GAVO.

Contents 1.Introduction, architecture 2.Live demonstration 3.Extensibility.

Assembly and Classification of Spectral Energy Distributions – A New VO Web Service Hans-Martin Adorf, GAVO, Max-Planck-Institut für extraterr. Physik,

Spectral Analysis in the VO Thomas Rauch, Iliya Nickelt and the GAVO and AstroGrid-D Teams.

ILDG Middleware Status Chip Watson ILDG-6 Workshop May 12, 2005.

Federation and Fusion of astronomical information Daniel Egret & Françoise Genova, CDS, Strasbourg Standards and tools for the Virtual Observatories.

Wiss. Beirat AIP, ClusterFinder & VO-Methods H. Enke German Astrophysical Virtual Observatory ClusterFinder VO Methods for Astronomical Applications.

Workshop Garching, June 27 – July Statistical Cross-Matching Across Distributed Archives H.-M. Adorf & GAVO Team MPI f. extraterrestrische Physik.

A PPARC funded project Workflow and Job Control in Astrogrid Jeff Lusted Dept Physics and Astronomy University of Leicester.

Moby Web Services Iván Párraga García MSc on Bioinformatics for Health Sciences May 2006.

Japanese Virtual Observatory Project Abstract : The National Astronomical Observatory of Japan (NAOJ) started the Japanese Virtual Observatory (JVO) project.

CMU-CS lunch talk, Gerard Lemson1 Computational and statistical problems for the Virtual Observatory With contributions from/thanks to: GAVO.

GES 2007, The German Astrophysical Virtual Observatory (GAVO) Knowledge Networking for Astronomy in Germany and abroad Gerard Lemson 1,2, Wolfgang.

Who are we ? what is a VO ? what is a Grid ? how do we get there ? Andy Lawrence S.P.I.E. Hawaii Aug 2002 AstroGrid

The International Virtual Observatory Alliance (IVOA) interoperability in action.

Enabling Grids for E-sciencE Astronomical data processing workflows on a service-oriented Grid architecture Valeria Manna INAF - SI The.

German Astrophysical Virtual Observatory Overview and Results So Far W. Voges, G. Lemson, H.-M. Adorf.

Japanese Virtual Observatory (JVO) National Astronomical Observatory of Japan (NAOJ) Contact Address: Demo 1: GRID-based database federation.

Pan-STARRS PS1 Published Science Products Subsystem Presentation to the PS1 Science Council August 1, 2007.

Strasbourg, EURO-VO DCA First Board Meeting (GA)VO projects at MPG(E) and participation in VO-DCA (tbcd) Wolfgang Voges, Gerard Lemson.

12 Oct 2003VO Tutorial, ADASS Strasbourg, Data Access Layer (DAL) Tutorial Doug Tody, National Radio Astronomy Observatory T HE US N ATIONAL V IRTUAL.

Web based spectrum databases and utilities László Dobos Tamás Budavári István Csabai MAGPOP kick-off meeting, January Cassis.

© Geodise Project, University of Southampton, Geodise Middleware Graeme Pound, Gang Xue & Matthew Fairman Summer 2003.

AHM04: Sep 2004 Nottingham CCLRC e-Science Centre eMinerals: Environment from the Molecular Level Managing simulation data Lisa Blanshard e- Science Data.

Matthew Farrellee Computer Sciences Department University of Wisconsin-Madison Condor and Web Services.

IVOA Small Projects Meeting Application to the science S. Honda, Y. Shirasaki, M. Tanaka and JVO team National Astronomical Observatory of Japan.

VO Data Access Layer IVOA Cambridge, UK 12 May 2003 Doug Tody, NRAO.

MSF and MAGE: e-Science Middleware for BT Applications Sep 21, 2006 Jaeyoung Choi Soongsil University, Seoul Korea

JAFER Toolkit Project Oxford University 1 JAFER Java-based high level Z39.50 toolkit Matthew Dovey; Colin Tatham; Antony Corfield; Richard Mawby Oxford.

Wednesday NI Vision Sessions

JENAM 2008 Theory Standards for the Virtual Observatory SimDB + SimDAP.

Hongbin Li 11/13/2014 A Debugger of Parallel Mutli- Agent Spatial Simulation.

VO-Neural / Data Mining Exploration

German Astrophysical Virtual Observatory

Datamining VizieR and SDSS

Presentation transcript:

GAVO – after one year H.-M. Adorf & the GAVO team

ADA III GAVO’s Pillars Publication and federation of datasets including theoretical ones Spatial matching & classification & visualization (NGSE) Grid computing

ADA III The GAVO Team PI: Wolfgang Voges, MPE, Garching CoI: Matthias Steinmetz, AIP, Potsdam Hans-Martin Adorf, MPE Harry Enke, AIP Gerard Lemson, MPE

ADA III Collaborations MPE, Garching (Peter Schuecker) AIP Potsdam Canadian Virtual Obs. (CVO), Victoria Universitaetssternw. Muenchen (USM) MPA, Garching (Planck, …)

ADA III GAVO Work Areas Publication of ROSAT datasets REST-based SCS service (HTTP GET in; XML, CSV, HTML out) SOAP-based SCS service MCMCS Download Manager Fuzzy spatial matcher Grid-based detection of galaxy clusters ROSAT matcher/classifier service

ADA III ROSAT All-Sky Survey (RASS) BSC, FSC & Photons VO-compatibility: Input: Simple cone search Output: VOTable, … New features Exposure times Galactic & Cartesian coordinates Healpix indices

ADA III SCS

ADA III Result

ADA III VOTable

ADA III MCMCS Download Manager Tool … … accesses registry at JHU User … … selects up to 70 distributed catalogues … specifies one or more sky-locations Tool … … queries remote catalogues … retrieves datasets (VOTables) for further processing

ADA III MCMCS Download Manager

ADA III MCMCS Download Manager

ADA III Workflow

ADA III Fuzzy spatial matcher Essential for data mining Prototype features  Performs “fuzzy” match between pairs of source lists from different catalogues  Computes probability of real match based on astrometric uncertainties Moving matcher into production use  Currently exported to Canadian Virtual Observatory  Feeding matches to ClassX classifier Long-term goal: assemble an SED

ADA III Fuzzy spatial matcher (ctd)

ADA III Grid computing What is Grid-computing? Globus Toolkit, SGE, EJBs WebServices Detection of galaxy clusters using RASS photons, SDSS RASS source classification using RASS B/FSC, SUMSS, NVSS, USNO

ADA III Workflow

ADA III Galaxy-cluster detection Search for galaxy clusters in heterogeneous databases RASS photons SDSS Software FORTRAN 90 code (data analysis) Perl scripts (data/grid management) Globus Toolkit

ADA III Galaxy-cluster detection Components Setup script  Sets up the directory structure  Distributes the software across the grid Execution script  Triggers the parallel execution on the compute nodes Data collection script  Retrieves the results from the compute nodes

ADA III GAVO Mini-GRID

ADA III Status Perl scripts Upgrading the RASS Photon DB Soon integrate the SDSS mirror copy

ADA III RASS-source classifier ClassX: international collaboration  T.A. McGlynn, A.A. Suchkov, E.L. Winter, R.J. Hanisch, R.L. White, F. Ochsenbein, S. Derriere, W. Voges, M.F. Corcoran Requires data from several sky-surveys  X-ray: ROSAT BSC+FSC  Optical: USNO, (SDSS)  Infrared: (2MASS)  Radio: FIRST, NVSS, SUMSS Distributed multi-wavelength VO-application

ADA III Data Flow

ADA III PoseQuery

ADA III DisplayQueryResult

ADA III DisplayMatchResult

ADA III The matching problem

ADA III DisplayXMatchResult

ADA III DisplayClassXResult

ADA III RASS-source classifier State & session management Using Java Servlet session mechanism Internal abstract state machine Strong points End-to-end VO-application Works with lists of objects / sky-coordinates Accesses several distributed databases Uses deterministic matcher (at Vizier and SDSS) Uses probabilistic matcher (at GAVO) Uses data analysis service (at NASA)

ADA III RASS-source classifier Future work Use IVOA-standards or protocols  VOTable  WSDL, SOAP-based Web Services  Add graphics Multi-threaded Modularize, generalize and make S/W configurable

ADA III RASS-source classifier Open questions Is an HTML form-based GUI sufficiently powerful to control a complex grid application? What are the user expectations wrt the workflow? Is there a standard workflow that one could use / agree upon?

ADA III Conclusion GAVO is a small, but ambitious project Two Grid applications Detection of galaxy clusters  Globus Toolkit Classification of ROSAT sources  Based on WebServices

ADA III References GAVO Mid-Term Report, 90 pages, (soon) Detection of X-ray Clusters of Galaxies by Matching RASS Photons and SDSS Galaxies within GAVO, P. Schuecker, H. Boehringer, W. Voges, A&A (submitted) Automated classification of ROSAT sources using heterogeneous multiwavelength catalogs, T.A. McGlynn et al., ApJ (submitted)