Combined use of AMDA, CAA, QSAS and CL based on time-table exchange. C. Jacquey, V. Génot, E. Budnik, M. Bouchemit, M. Gangloff, CDPP/CESR B. Cecconi,

Slides:



Advertisements
Similar presentations
AMDA service Visualisation module Analysis module Time Table manager Local database Remote database Download module.
Advertisements

The CDPP Centre de Données de la Physique des Plasmas Plasma Physics Data Centre.
V. Génot, C. Jacquey, E. Budnik, R. Hitier, M. Bouchemit, M. Gangloff Toulouse, France R. Conseil, D. Heulet, and C. Huc CNES Science archives.
Solar and STP Physics with AstroGrid 1. Mullard Space Science Laboratory, University College London. 2. School of Physics and Astronomy, University of.
Operating a Virtual Observatory Raymond J. Walker, Jan Merka, Todd A. King, Thomas Narock, Steven P. Joy, Lee F. Bargatze, Peter Chi and James Weygand.
Computational Physics Kepler Dr. Guy Tel-Zur. This presentations follows “The Getting Started with Kepler” guide. A tutorial style manual for scientists.
Development of a Community Hydrologic Information System Jeffery S. Horsburgh Utah State University David G. Tarboton Utah State University.
SpaceGRID and EGSO Satu Keski-Jaskari Maria Vappula Parallal Computing – Seminar
BinX and Astronomy Bob Mann Institute for Astronomy and National e-Science Centre.
Numerical Grid Computations with the OPeNDAP Back End Server (BES)
FILING SYSTEMS Research Data Management. Filing is more than saving files, it’s making sure you can find them later in your project. Naming Directory.
Time Table exchange QSAS / CL / CAA / AMDA CESR, 25/26 feb
HDMC VxO SPASE June 2008 VO-oriented activities of the CDPP M. Gangloff, E. Budnik, M. Bouchemit C. Jacquey, V. Génot, N. André, B. Cecconi, B. Lavraud.
S. Derriere et al., ESSW03 Budapest, 2003 May 20 UCDs - metadata for astronomy Sébastien Derriere François Ochsenbein Thomas Boch CDS, Observatoire astronomique.
AMDA and CLweb Two complementary web-based services for spacecraft data exploitation with help from LESIA and support from CNES and CNRS Benoit Lavraud,
DM_PPT_NP_v01 SESIP_0715_AJ HDF Product Designer Aleksandar Jelenak, H. Joe Lee, Ted Habermann Gerd Heber, John Readey, Joel Plutchak The HDF Group HDF.
Flexible tools for integrating observations and models Johan De Keyser Emmanuel Gamby Belgian Institute for Space Aeronomy.
Functions and Demo of Astrogrid 1.1 China-VO Haijun Tian.
THEMIS launch meeting Cape Canaveral Feb , 2007 C. Jacquey, V. Génot, E. Budnik, R. Hitier, M. Bouchemit CDPP/CESR, Toulouse, France AMDA Automated.
Solar Terrestrial Physics: ESA SpaceGRID Chris Perry, Rutherford Appleton Laboratory 21st March 2003.
Database structure for the European Integrated Tokamak Modelling Task Force F. Imbeaux On behalf of the Data Coordination Project.
The Virtual Radiation Belt Observatory (ViRBO) and tools for radiation belt science R.S. Weigel Department of Computational and Data.
The SPASE Data Model: Standard Metadata for Space Science Data Description J. Thieman, T. King, A. Roberts, J. King, C. Harvey, and P. Richards Oct. 24,
AMDA, Automated Multi-Dataset Analysis: A web-based service provided by the CDPP. Jacquey 1 C., V. Génot 1, E. Budnik 2, R. Hitier 3, M. Bouchemit 1, M.
Summary of distributed tools of potential use for JRA3 Dugan Witherick HPC Programmer for the Miracle Consortium University College.
ViRBO and Autoplot. ViRBO Overview Virtual Radiation Belt Observatory Facilitate Search, Access, and Visualization of Radiation Belt Data Search – Develop.
International Conference on Comparative Planetology: Venus - Earth - Mars ESTEC, Noordwijk, The Netherlands, Thursday 14 May :10:00, Abstract #
Strasbourg astronomical Data Centre (DS) Françoise GENOVA.
Autoplot Overview Autoplot developed originally for ViRBO Virtual Observatory, then adopted by VMO, and RBSP instrument and other teams.
Influence of solar wind density on ring current response.
The european ITM Task Force data structure F. Imbeaux.
Federation and Fusion of astronomical information Daniel Egret & Françoise Genova, CDS, Strasbourg Standards and tools for the Virtual Observatories.
_______________________________________________________________CMAQ Libraries and Utilities ___________________________________________________Community.
The CoRoT ground-based complementary archive The CoRoT ground-based complementary archive Monica Rainer, Ennio Poretti M. Rosa Panzera, Angelo Mistò INAF.
Solar and STP science with AstroGrid Silvia Dalla School of Physics & Astronomy, University of Manchester A PPARC funded project.
SPASE: Metadata Interoperability in the Great Observatory Environment Jim Thieman Todd King Aaron Roberts Joe King AGU Joint Assembly May 23, 2006.
Presented by Scientific Annotation Middleware Software infrastructure to support rich scientific records and the processes that produce them Jens Schwidder.
AMDA, Automated Multi-Dataset Analysis: A web-based service provided by the CDPP usable for THEMIS data exploitation. THEMIS SWT, Boulder March 23-25,
Solar and space physics datasets within a Virtual Observatory: the AstroGrid experience Silvia Dalla * and Nicholas A Walton  * School of Physics & Astronomy,
AstroGrid Solar/STP planning meeting Agenda: Helioscope Preparing for Solar-B Time-series viewing application IVOA and time series A PPARC funded project.
The Virtual Magnetospheric Observatory VMO/U Ray Walker Todd King Steven Joy.
The CDPP (Centre de Données de Physique des Plasmas) is the french national centre of natural plasma data. Development of new tools and interoperability.
Metadata Mòrag Burgon-Lyon University of Glasgow.
© Federal Statistical Office, Institute for Research and Development in Federal Statistics, Elmar Wein Federal Statistical Office Concepts, materials and.
HDF EOS Workshop David Han Code
SolarFlows Dr. Gabriele Pierantoni (TCD). Contents What is Heliophysics ? How could workflows help ? Some examples What we are doing...
Plasma Node demonstrators 1/Registry : –Set of XML descriptors of planetary plasma data (MAPSKP, VEX, MEX ) –Compliant with the SPASE data model
Mireille Louys et al., OV-France Theory WG meeting, LYON, June DALIA : an observation vs simulation comparison frame work What could be a Model.
SWT Overview 1 UCB, Nov15/16, 2006 Mirror site and services at CESR and CDPP Christian Jacquey, CDPP, CESR, Emmanuel Penou, CESR,
Welcome to the PRECIS training workshop
CCSDS Meeting data Archive Ingest - January 2007 CNES 1 CCSDS - MOIMS Area Data Archive Ingest WG CNES Report Colorado Springs meeting – January 2007 Claude.
The Virtual Heliospheric Observatory and Distributed Data Processing T.W. Narock 1,2, A. Szabo 2, A. Davis 3 1. L3 Communications,
VxO Kickoff Meeting - May 22, 2006 The Evolving Heliophysics Data Environment: “VxO Kickoff” Chuck Holmes Joe Bredekamp May 22, 2006.
SPDF Science Advisory Group - September 29-30, 2005 Page 12/24/2016 9:09:48 PM Services of the Space Physics Data Facility (SPDF) / Sun-Earth Connection.
STEREO and the Virtual Heliospheric Observatory Tom Narock 1,2 Adam Szabo 1 Jan Merka 2 (1) NASA/Goddard Space Flight Center (2) L3 Communications, GSI.
Instance Discovery and Schema Matching With Applications to Biological Deep Web Data Integration Tantan Liu, Fan Wang, Gagan Agrawal {liut, wangfa,
Qtran Software Updates. Qtran Qtran Handles CAA Time Range data type (becomes 2 Epoch values in CDF) Single record of fill values written into empty.
OVT status Yuri Khotyaintsev Swedish Institute of Space Physics, Uppsala and the OVT team ovt.irfu.se November 18, 2015.
NVS New Zealand National Vegetation Survey. What is NVS? NVS (National Vegetation Survey) – New Zealand’s largest archive facility for plot-based vegetation.
ECLAT Cluster Footprint Mapping. ECLAT meeting March 2012 WP330 Cluster footprint mapping 1.Cluster footprint mapping using statistical magnetospheric.
1 OEAW Contribution to ECLAT R. Nakamura, W. Baumjohann, M. Volwerk Institut fu ̈ r Weltraumforschung der Österreichischen Akademie der Wissenschaften.
R. Maggiolo 1, M. Echim 1,2, D. Fontaine 3, A. Teste 4, C. Jacquey 5 1 Belgian Institute for Space Aeronomy (IASB-BIRA); 2 Institute.
InSilicoLab – Grid Environment for Supporting Numerical Experiments in Chemistry Joanna Kocot, Daniel Harężlak, Klemens Noga, Mariusz Sterzel, Tomasz Szepieniec.
8 th Cross-Calibration Workshop, Kinsale, Ireland, October 20081/14 Draft Replies and Actions to the Recommendations of the Review Panel for Final.
Solar System Plasma data archiving in France
The IMPEx Data Model and Protocol
Requirements Swarm Cal/Val Team Workshop 18/06/2014
Reportnet 3.0 Database Feasibility Study – Approach
Presentation transcript:

Combined use of AMDA, CAA, QSAS and CL based on time-table exchange. C. Jacquey, V. Génot, E. Budnik, M. Bouchemit, M. Gangloff, CDPP/CESR B. Cecconi, CDPP/LESIA C. Perry, CAA team T. Allen, A. Rochel, QSAS team E. Penou, J.A. Sauvaud, CESR Cluster Workshop, Tenerife, March 12, 2008 This talk is related to: -Poster (Jacquey et al.) this evening -Talks on CAA (C. Perry), QSAS (T. Allen) and CL (E. Penou)

Some aspects of the studies of space physics ·We need to analyse measurements obtained by several observatories ·We need to analyse multi-instrument data ·We need to compare the observations to simulation results We need to extract and treat a lot of data coming from multiple resources  This takes time and energy ·Dynamical processes  The time is the very first parameter ·The studies consists of analysis/characterisation of events treated in case studies or in statistical studies · Creating Time-Tables (or event list) is a common step of the work

Time-Tables (Event Lists)  Time-Table produced by the users  Time-Tables coming from statistical studies (e.g. magnetopause crossing lists) sometimes made available to the community (e.g. the Frey’s substorm list)  Time-Table coming from search tool based on orbitographic or conjonction criteria (e.g. the Query tool of the SPDF/SSCWeb)  Time-Table coming from search tool based on mathematical criteria applied to the content of the data (e.g. AMDA of CDPP)  Time-Table coming from visual search tool (e.g. AMDA of CDPP) Exchanging Time-Tables between data centres, services and tools?  saving time and energy. Increase of the scientific return.

AMDA Automated Mutiple Dataset Analysis  Web based service  Transparent access to data  AMDA local database (CLUSTER, ACE, THEMIS, GEOTAIL, WIND, …, ISEE)  External database (CDAWeb, MAPSKP, THEMIS/CESR, …)  Produces and exploits time-tables Visualisation editor Parameter editor Download data Conditional search External data Visual search Time-Table manager AMDA DOES NOT WORK WITH InternetExplorer and Safari AMDA is public (

Others DARTS SSL THEMIS database Others … SPDF SSCWeb  Time-Tables + many services GAIA Virtual Observatories VSPO, VMO, VHO, VIRBO, … CL QSAS Resources: data centres, services, tools CAA Validated high resolution CLUSTER data Plots and tools Mirror THEMIS database CDPP database AMDA  Time-Tables CDPP CDAWeb

Science Case: study and characterisation of the local properties of the cross-tail current  Based on CLUSTER data when it encircles the neutral sheet. (i)Current density, profile, gradient (ii)Thickness, location, motion (iii)Distribution function (i+ and e-), flows, … (iv)Turbulence/wave level and spectra (v)…  Influence factor: (i)Solar Wind pressure (ii)IMF (iii)Geomagnetic activity: -State of the ionosphere. -Oxygen abundance -Ring current intensity

Use Case: the main steps Access and extraction of ACE/WIND/GEOTAIL Interplanetary data Access and extraction of ground (AE, AL, SYM, DST) data Access and extraction of modelisation or simulation data (Tsyganenko, CCMC, …) Selection of the periods of NS encircling condition User tools Basic treatments Formatting Data merging Time shifting Results Catalogues Maps Models Selection of the periods of NS encircling condition AMDA Access and extraction of ground (AE, AL, SYM, DST) data Access and extraction of ACE/WIND/GEOTAIL Interplanetary data CDAWeb, CDPP, AMDA, DARTS, … CAA Access and extraction of CLUSTER data Access and extraction of CLUSTER data Specific tools Specific tools CL Generic tools Generic tools CL, QSAS or AMDA

Creation of time-tables on AMDA/CDPP ÞTT1, time-table listing the periods when the conditions are fulfilled ÞTT2, extended time-table

Conditional search

Visualisation

Visual search

Managing Time-Tables Text or XML VOTable

Parameter editor

Data dowload Data merging

AMDA -Automated search -Visual search  Time-Tables CAA Validated high resolution CLUSTER data CL QSAS Standardised sub-databases Other? CDAWeb DARTS CDPP/CESR THEMIS database SSL THEMIS database Other? Time-Tables First step: producing sub-databases corresponding to the time-tables Time-Tables Data, parameters User tools

How to do that?  Standardisation of the time-tables  Standardisation of the organisation of the sub- databases to be produced  Interface implementation on CAA for reading the time- table and producing sub-databases  Interface implementation on the tools QSAS and CL  Standardisation of the time-table description

Step 2: catalogues T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … T_StartT_EndP1P1 …PNPN Q1Q1 …QMQM … AMDA, SSCWeb or other services CAA, CDAweb, CDPP or other data centres QSAS, CL, AMDA or other tools

Time-table: T1 – T2 …. AMDA USER Computation of P1,..,PN CAA Catalogue: T1, T2, P1, …, PN Sub-database Catalogue: T1, T2, P1, …, PN, Q1, …,QN Future: Archiving the catalogues for availability to the community QSAS or CL CDPP CDAweb DARTS USER Catalogues and associated tools available at the data centres

Conclusion  Exchange of standardised Time-Tables between data centres, services and tools should allow the researcher - to save large amount of time and energy - to exploit more extended data amount - to increase the scientific return of the data  First step: extraction and exploitation of sub-databases corresponding to the Time-Tables  Second step: creation and exploitation of catalogues, similarly to astronomy and solar physics  This project starts with CAA, QSAS, CL and AMDA, but of course, other new participant is welcome  The specifications of this project need the input of the potential users  Poster this evening in order to collect suggestions and ideas. Stable version: Test version:

Use cases Use case = simulation of studies which could be performed by the researchers in order to:  identify and analyse their needs  specify the user requirements  Imagine, define and specify the system to be developed The developers of the system need the input from the community Obvious requirement: standardisation  Standardisation of the data format ÞCEF, CDF, NetCDF, (FITS) ÞBut beyond that, standard way for representing data of different classes (particle, waves, Time-Tables, …)  Standardisation of the data description  SPASE (Space Physics Archive Search and Extract, )

Step 2: sub-database extraction ÞOn CAA, CDAweb, DARTS, CDPP, AMDA, … Requirements: %Standard format for time-tables %Interface for reading and managing the time-tables %User interface for selecting data %Optionally, basic on the fly treatment (e.g. data merging, SW propagation shifting, …) %Standardised management of the multiple file output ·Filename ·Descriptor ·Product recovering

download

Pi-Pj/(Xi-Xj)

Step 3: plot production on CL, QSAS, AMDA %INPUT: ▪ time table ▪ corresponding sub-database %Manipulation of the time-table %User edited plot %Launch in batch mode Requirements:  Standardisation (i) of the time-table and (ii) of the data  Plot editor  Batch mode  Standardised management of the plot files ▪ Filename ▪ Descriptor ▪ Product recovering

AMDA Automated Mutiple Dataset Analysis Created by the CDPP: Stable version: Testing version: Contact:

Example: event search Old fashion: “paper” search AE, ALISEE-data IMP-8 AE, AL ISEE MAG ISEE- electron IMP8 MAG IMP8 PLA Current fashion: web Event search takes time and energy CDAWeb SPDF UCLA CAA CDPP And others

Automated Mutiple Dataset Analysis AMDA: the goals Multiple Dataset: the study of the (multi-scale) dynamics of plasma objects requires to perform the integrated analysis of multi-point and multi-instrument data Automated (or semi-automated) Analysis: help for  Search, characterisation, classification of events  Extensive exploitation of the vast database (statistics)  “historical” studies (example: through solar cycles)  Building catalogues  Building virtual constellations  Time-Table (EventList) production

AMDA, fact sheet Built in IDL, C, FORTRAN, Javascript Works on standardised and “simple” data, NetCDF Includes models (Tsyganenko, Shock, MP, NS, …) Uses external data (now from CDAWeb and soon from MAPSKP, IWF, THEMIS, …) Forum for user feedback Twiki for information (development, bugs, …) exchange Already in use for scientific studies AMDA DOES NOT WORK WITH InternetExplorer and Safari AMDA is public

Functionalities  Automated access to data  Access to and on-the-flight ingestion of external data (test version only)  Data merging  Data download  Data visualisation  User edited parameter computation on the data  Automated conditionnal search  Visual search Production of Time-Tables

Local Database AMDA System CDAWeb CDPP MAPSKP AMDA/”External Data” others

Needed development on AMDA  Temporal functions (mean, regression, dispersion estimator, …)  User pluggins  Continuous SW propagation shifting  Scatter plots, map plots, …  Your suggestion  ….

Conclusion and questions  CAA, QSAS, CL and CDPP teams are starting a collaboration for providing to the researchers ways to save time and energy for performing their studies  Other services and database are invited to join this effort  We need input, comments, suggestions from the potential users.