1. ASPERA WorkshopLyon – 8 October 2010 1 Grid for control of Astronomical Instrumentation C. Vuerli, G.Taffoni, F. Pasian INAF – National Institute of Astrophysics, Italy

2. ASPERA WorkshopLyon – 8 October 2010 2 Goal Gridification of Telescopes and more in general of astronomical instrumentation to remotely monitor, control and process data in quasi-real-time Remote monitoring of telemetry and of scientific data under acquisition Remote control: astronomers interact with astronomical instrumentation to drive the acquisition data process Quasi-real-time data processing: acquisition of data and a first processing of them is performed simultaneously

3. ASPERA WorkshopLyon – 8 October 2010 3 Rationale Astronomical Observing facilities produce very large data sets Access to Grid infrastructures make possible the on-line processing of data as it requires considerable computing power and storage capacity Technological advances of astronomical instrumentation takes place frequently and usually implies enhancements of the volumes of produced data A more efficient collaboration between astronomers and their convenient access to the research facilities could improve the quality of astronomical research

4. ASPERA WorkshopLyon – 8 October 2010 4 Rationale It is highly desirable a better exploitation of astronomical instrumentation to tackle dead times due to adverse conditions for scheduled observations Costs could dramatically decrease given that astronomers do not have to travel to remote observing facilities

5. ASPERA WorkshopLyon – 8 October 2010 5 The targeted Integrated System Grid-UI The GRID Grid-GUI IE QE DBMS CE SE CE

6. ASPERA WorkshopLyon – 8 October 2010 6 The “DYNACORE” Ancestor An EU funded project to provide scientists and astronomers with a powerful tool for remote collaboration in experiments or observations requiring one or more remote facilities. One of the main objectives was the development of a teleoperation system which can be easily adapted and optimized for a particular instrument.

7. User/User Interaction Virtual Operations Room User/Instrument Interaction Virtual Laboratory DYNACORE Server Instrument/Instrument Interaction

8. ASPERA WorkshopLyon – 8 October 2010 8 The GRID.IT Project GRID.IT: National multi-disciplinary project to validate the Italian Grid infrastructure INAF participated with 3 applications Access to Astronomical Databases via Grid Porting on Grid the VST Pipelines Monitoring/Control of astronomical instrumentation

9. ASPERA WorkshopLyon – 8 October 2010 9 Query DB specific driver G-DSE and G-ICS gatekeeper JobManagerQueryManager JobProcess QueryProcess Scheduler p-in Pbs/LSF query plug-in GRAM GIS RDBMS MDS GRIS Ldap ldif Grid Providers (snmp) Instrument Control System ICSManager ICS plug-in ICSProcess ICS specific driver ICS RDBMS

10. ASPERA WorkshopLyon – 8 October 2010 10 LCG-2: DB + ICS enabled middleware Computing cluster Network resourcesData storage Operating system Local scheduler File system User accessSecurityData transfer Information schema Resource Broker Data managementApp monitoring system User interfaces Applications Hardware System software “Basic” services “Collective” services Application level services HPSS, CASTOR… Scientific Linux NFS, … PBS, Condor, LSF,… VDT (Condor, Globus, GLUE) + XMLDB + XMLICS EU DataGrid +Glite + G-DSE + G-ICS Information system RDBMS MySQL Ora, etc. ICS Instruments

11. ASPERA WorkshopLyon – 8 October 2010 11 Grid Instrument Control System Local Resource Manager ICS Grid Access Enabled GRAM Protocol Grid info system MDS LDAP Job Manager ICSC Manager ICS Instance Man ICSC Driver ICS Instance Worker Node Meta Machine GANGLIA snmpd ICS Synchro CS Tel CS Synchro ODBC Manager DSE Instance Man ODBC Driver DSE Instance DSE Internal DB User DB

12. ASPERA WorkshopLyon – 8 October 2010 12 What about QE and IE? Some development for QE continued at INAF (after an unsuccessful EC project proposal) through some national (INAF) funded projects and a prototype for G-DSE was implemented The development of IE instead was frozen and never resumed due to lack of funds

13. ASPERA WorkshopLyon – 8 October 2010 13 What opportunities now? DORII (Deployment of Remote Instrumentation Infrastructure) builds on experience gained in previous European projects (in particular GridCC) Designs and sets up an extended infrastructure with involvement of users in three main areas (Earthquake, Environmental Sciences, Experimental Sciences)

14. ASPERA WorkshopLyon – 8 October 2010 14 DORII The DORII e-Infrastructure is mainly based on the EGEE (and now on EGI) infrastructure and on gLite middleware To deal with the interactivity requirements of the applications, the DORII e-Infrastructure integrates a selection of services built by the Interactive European Grid Project (int.eu.grid) The gLite middleware offers the main grid services such as information, job management, data management and security control

15. ASPERA WorkshopLyon – 8 October 2010 15 DORII Architecture The users and the instruments interact via the Instrument Element The VCR is a Grid portal which enables registered users to interactively access the DORII Grid resources The g-Eclipse framework provides tools that customize Grid user applications, manage Grid resources and support the development cycle of new software applications To access all the resources and services of the DORII e-nfrastructure, a suitable Common Library will be soon available

16. ASPERA WorkshopLyon – 8 October 2010 16 The Instrument Element (IE) The Instrument Element (IE) represents the virtualization of the scientific data sources, mainly digital instrumentation and sensors IE is implemented as an open source Java middleware that handles the tasks of remote control and safe access to the instrumentation. In particular, it handles the tasks of user authentication and authorization remote operations concurrent access to devices locking and reservation of instrumentation during an experiment publishing of alarms and events generated by the instrumentation monitoring of the device variables a simple connection to the Glite's Grid resources.

17. ASPERA WorkshopLyon – 8 October 2010 17 The IE Architecture

18. ASPERA WorkshopLyon – 8 October 2010 18 The Instrument Element (IE) The IE has been introduced as a novel grid component through the QE+IE project proposal In the context of the GRIDCC project, and by the end of the project it has been deployed in a number of pilot applications The two models of IE differ from the point of view of architecture and implementation. From the functional point of view they are similar

19. ASPERA WorkshopLyon – 8 October 2010 19 The Instrument Element (IE) IE exposes to the outside world a web service interface that integrates the Glite's authentication module based on the delegation of the client's X509 proxy certificates The delegated proxy is further used to access the traditional grid resources, in particular the Storage Elements The actual device, or rather its control system, is linked to the middleware through a protocol adapter called the Instrument Manager (IM)

20. ASPERA WorkshopLyon – 8 October 2010 20 The Instrument Manager (IM) An IM consists of a set of Java classes and a single XML description file that fully describes the device An instrument is described by its state machine transactions regular commands attributes (variables) operating parameters. The XML device descriptor is used by the developer to list the commands, attributes, parameters, initial and final states, input and output types, measurement units, attribute, parameter value boundaries, etc.

21. ASPERA WorkshopLyon – 8 October 2010 21 More about the IE The IE offers the support for saving the instrumentation output directly to the remote grid storage or uploading of some parameters to the device from the remote grid storage using the Grid-FTP protocol Instrument variables (attributes) may be monitored synchronously (through the polling mechanism) Asynchronously (more efficient) using the Java Message System (JMS). JMS is also used to publish the alarms generated by the device or the events that occurred during the operations IE allows for the plugging of the scientific instrumentation into a complex data processing workflow pipeline consisting of traditional grid resources like storage capacities and processing power in a safe and standardized manner IE also allows for remote operations and monitoring even of complex equipments

22. ASPERA WorkshopLyon – 8 October 2010 22 The Virtual Control Room The Virtual Control Room is an open source Grid portal based on Gridsphere and Web 2.0 technologies. Easy discovering, browsing and using GLite Grid resources are its most significant features Upon registration, the VCR allows its users to safely create a long term proxy certificates on their VO's MyProxy server. From that moment on, the VCR becomes a SSO portal to the Grid. As users log in with their username and password, short term (usually 12-hour) proxy certificates are created. Such certificates are then used for authentication at VO's computing and storage resources. The VCR allows users to easily access resources of the underlying middleware and to exploit its services.

24. ASPERA WorkshopLyon – 8 October 2010 24 The Virtual Control Room VCR is a front end to the Instrument Element (IE) thus allowing for remote control of the scientific instrumentation and adopting the same security scheme (proxy delegation) as for the storage and computing resources SSL tunneling allows users to safely access the not yet gridified devices or complex legacy instruments VCR is also a collaborative environment that offers a set of groupware tools in support of a scientific teamwork. Tools include an e-logbook, an integrated chat, a wiki-like help system and people, and resource browsers VCR integrates with the third party tools like Skype VCR may be adopted as a ready-to-use virtual collaboratory or as an extendible framework for developing advanced, application-specific collaboratories. A set of libraries are provided with the VCR software development kit that supports the implementation of application- specific plug-ins

26. ASPERA WorkshopLyon – 8 October 2010 26 The RISGE-RG in OGF The RISGE (Remote Instrumentation Services in Grid Environment) RG aims at bringing together various existing approaches in defining remote access interfaces to sophisticated laboratory equipment, as well as to come up with use cases that can dictate the requirements for integrating scientific instruments with the Grid. One of the purposes of this group is to bring together people involved in the field of remote instrumentation to document experiences, identify best practices and develop informational documents.

27. ASPERA WorkshopLyon – 8 October 2010 27 References The IE and the VCR in GridCC and DORII have been developed at ELETTRA SCpA operating the Synchrotron Radiation Machine in Trieste, Italy http://www.elettra.trieste.it Roberto Pugliese roberto.pugliese@elettra.trieste.it

28. ASPERA WorkshopLyon – 8 October 2010 28 References http://www.gridcc.org/viewnote.php?id=1812 http://www.dorii.eu RINGrid project (remote instrumentation) http://www.ringrid.eu http://www.ringrid.eu Interactive Europran Grid project http://www.i2g.eu/ http://www.i2g.eu/ g-Eclipse project (software frameworks for application developers) http://www.geclipse.org/ http://www.geclipse.org/ http://forge.gridforum.org/sf/projects/risge-rg

29. ASPERA WorkshopLyon – 8 October 2010 29 End of Presentation