1. WfMS and external systems Katarzyna Bylec PSNC

2. Agenda Introduction Pre-corelation ▫ North Star ▫ NRAO SCHED ▫ Vlbeer FTP ▫ Log2vex ▫ drudg Correlation ▫ Field system ▫ Telescope visualisation ▫ Software correlator Post-correlation ▫ j2ms2 ▫ 2bitVVx ▫ Diagnostic tools ▫ tConvert ▫ Crude images creation ▫ EVN data archive WP7 Meeting – Poznań- 19-04-2010

3. Introduction External systems ▫ Existing or to-be-implemented APIs ▫ Defined by Use Cases ▫ Included directly in KIWI Portal ▫ Accessed via WfMS Server Goals of this presentation ▫ Decide if KIWI/NEXPReS will use those services ▫ What tools can be used? ▫ Are there APIs defined? ▫ Are those services remotelly available? Glossary ▫ WfMS – Workflow Manager Service WP7 Meeting – Poznań- 19-04-2010

4. Pre-correlation: North Star Web-based Proposal Submission Tool Usage: ▫ By Prinicipal Investigator ▫ Creation and submission of proposal ▫ Information of proposal status Interactivity/automatization ▫ Human interaction needed WfMS integration ▫ Should it be integrated in KIWI Portal? ▫ Web-page redirecting WP7 Meeting – Poznań- 19-04-2010

5. Pre-correlation: NRAO SCHED Usage: ▫ By Prinicipal Investigator ▫ Produce VEX file for observation Interactivity/automatization ▫ Have to be reviewed by JIVE staff WfMS integration ▫ What’s the input? Can the creation be automated? ▫ Is it accessible as a remote service? Can it be integrated in KIWI Portal (directly)? ▫ Should it be installed at WfMS? WP7 Meeting – Poznań- 19-04-2010

6. Pre-correlation: vlbeer FTP FTP repository of schedules Usage: ▫ By all ▫ stores VEX schedules ▫ password-based upload ▫ anonymous download Interactivity/automatization ▫ Access - automated ▫ Files - have to be reviewed by JIVE staff WfMS integration ▫ Should it be integrated in KIWI Portal (directly FTP access)? ▫ Import from Vlbeer to KIWI Portal ▫ An idea: alternative repository managed by WfMS? WP7 Meeting – Poznań- 19-04-2010

7. Pre-correlation: log2vex Usage: ▫ By Correlation Operator ▫ adds infromation to VEX file:  clock errors  recorded media  Earth Orientation Parameters (EOPs) Interactivity/automatization ▫ It can be automated - ? WfMS integration ▫ WfMS can run log2vex using input data from user (or can it be done without user basing on some const. data?) WP7 Meeting – Poznań- 19-04-2010

8. Pre-correlation: drudg Usage: ▫ By Telescope Operator ▫ Input: VEX file ▫ Output: Procedure and Snap files ▫ can be run interactively or in a batch mode Interactivity/automatization ▫ Human interaction needed – to review the files ▫ Automated – with set of oscilator voltage values known WfMS integration ▫ WfMS integration – run by WfMS and results presented for acceptation? WP7 Meeting – Poznań- 19-04-2010

9. Correlation: Field system Station Control Software Usage: ▫ By Telescope Operator ▫ Mainly monitoring ▫ Telescope control? Interactivity/automatization ▫ Human interaction needed WfMS integration ▫ DO WE NEED THAT? (Security, time) ▫ Integration of Remote Tool (WP5’s Task 3) into KIWI Portal? ▫ RPC call to FS functionalities ▫ What’s the difference between FS and Station Control Software? WP7 Meeting – Poznań- 19-04-2010

10. Field System - remote Remote Access – Web Services ▫ RPC-style  C/C++  ONC RPC implementation  UNIX only client generation tool ▫ If FS will expose telescope/observation controll API  WfMS can forward it to KIWI Portal  Will TOs acctually use it when they have full FS access?  WfMS can start/stop observation - ??? Remote Access – Rich Client Application ▫ C++ wxWidgets ▫ very hard/impossible to integrate it into KIWI Portal WP7 Meeting – Poznań- 19-04-2010

11. Correlation: Telescopes visualisation Usage: ▫ Functionality in KIWI Portal ▫ Visualisation of resources (parsing VEX) ▫ Monitoring Interactivity/automatization ▫ automated WfMS integration ▫ Tool needed (the same as used in EXPReS? JIVE?) Event – Location - Date

12. Correlation: Correlator … will be discussed tomorrow. WP7 Meeting – Poznań- 19-04-2010

13. Post-correlation: j2ms2 & 2bitVVx Usage ▫ j2ms2:  translates correlation output to APIS++/CASA MeasurementSet ▫ 2bitVVx:  Correction of amplitudes (Vleck correction) Interactivity/automatization ▫ Can be automated - ? WfMS integration ▫ Can be run by WfMS after correlation ▫ Are those programs available as remote services? ▫ Documentation and programs needed WP7 Meeting – Poznań- 19-04-2010

14. Post-correlation: diagnostic tools Usage ▫ Suite of programs run on the MeasurementSet ▫ Assures that correlation was done correctly ▫ In e-VLBI re-correlation is not possible Interactivity/automatization ▫ Can be automated - ? WfMS integration ▫ What tools are in this suite? ▫ Can they be run by WfMS? ▫ Are they accessible as a remote services? WP7 Meeting – Poznań- 19-04-2010

15. Post-correlation: tConvert Usage ▫ Converts MeasurementSet into the FITS-IDI data format ▫ FITS-IDI is the de facto standard for VLBI data reduction Interactivity/automatization ▫ Can be automated - ? WfMS integration ▫ Can tConvert be run by WfMS? ▫ Is it accessible as a remote services? WP7 Meeting – Poznań- 19-04-2010

16. Post-correlation: crude images Usage ▫ Pipeline script (Python/ParselTongue) ▫ Creates crude images of observed sources ▫ Generates diagnostic plots Interactivity/automatization ▫ Can be automated - ? ▫ Can diagnostic plots be handled by WfMS (user needed?) WfMS integration ▫ Can it be run by WfMS? ▫ Is accessible as remote services? WP7 Meeting – Poznań- 19-04-2010

17. Post-correlation: EVN data archive Usage ▫ http://www.jive.nl/select-experiment http://www.jive.nl/select-experiment ▫ Upload of correlation results ▫ Access to data Interactivity/automatization ▫ Can be automated - ? WfMS integration ▫ Is there API available? ▫ Are they accessible as remote services? ▫ What is taken as an input? ▫ What parameters describe uploaded data? WP7 Meeting – Poznań- 19-04-2010