1. Elettra Virtual Collaboratory: the evolution of a Virtual Laboratory Software from a simple web application to the GRIDCC Roberto Pugliese, Alessandro Busato, Alessio Curri, Enrico Mariotti, Daniele Favretto, Fulvio Billè, Roberto Borghes, Fabio Asnicar, Valentina Chenda, Laura Del Cano, Lawrence Iviani, Michele Turcinovich and the GRIDCC collaboration Sincrotrone Trieste – ELETTRA Instruments and Sensors on the Grid IEEE Conference on e-Science and Grid Computing Melbourne, Australia, on 5-8 December 2005

2. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Outline The Elettra Virtual Collaboratory (EVC)  EVC @ Work The BIOXHIT project  Virtual Collaborative System (VCS) The EUROTeV project  The Global Accelerator Network  The Multipurpose Virtual Laboratory (MVL) Evolving EVC to meet VCS and MVL requirements The GRIDCC project  The Multipurpose Collaborative Environment (MCE) Evolving EVC with MCE

3. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it What is a Collaboratory? The term “collaboratory” was coined by William Wulf by merging the words collaboration and laboratory, and defined as “... Center without walls, in which researchers can perform their research without regard to geographical location - interacting with colleagues, accessing instrumentation, sharing data and computational resource, and accessing information in digital libraries”.

4. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it What is a Collaboratory? In particular, the core capabilities that constitute a collaboratory can be seen as technologies to link:  People to people (e.g., electronic mail, and tools for data conferencing, such as VRVS)  People to information (e.g., the World Wide Web and digital libraries)  People to facilities (e.g., status of remote instruments) to enhance utilization by expanding access to resources

5. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it What is the Elettra Virtual Collaboratory (EVC)? EVC is an example of virtual laboratory, a system which allows a team of researchers distributed anywhere in the world to perform a complete experiment on the equipped beamlines and experimental stations of Elettra. User at ELETTRA Team Member at Home Lab Remote Collaborator Equipment Control Data CPU COLLABORATORY

6. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it What is Elettra?

7. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it EVC usage scenarios: Cristallography “by mail” EVC allows biologists to send by mail protein crystals which will be ananlized at the Xray Diffraction beamline by the beamline staff. Collected data and results are accessible via EVC and results can be downloaded as soon they are available.

8. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it EVC in action: a web portal EVC is based on the “web portal” metaphor All you need is a browser EVC supports four different user categories:  Visitors  Normal users  Project leaders  Staff

9. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Collaborating to an EVC project Scientists working to an EVC project can use many project related collaboration tools EVC presents an adaptive interface changing to suite the category and expertise level of the user

10. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Collaboration Tools: EVC chat EVC chat is “project centered”: there is a different channel for each project Usual chat feature are extended in order to allow exchange of  Drawings  scientific images  graphical annotations

11. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Collaboration tools:scientific visualisation Scientists can browse, visualise and process remotely scientific data in real-time as soon as the data is collected

12. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Collaboration Tools: telepresence The different video streams of the equipped experimental stations can be selected and viewed even through a slow connection Movable cameras can be controlled via web by the project leader

13. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Collaboration tools remote computing Legacy software is normally not web enabled EVC uses VNC to web enable legacy apps.  It is small and simple, sharable and open  Can be tunnelled via ssh VNC can be used as a fast integration tool

14. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Collaboration Tools: Remote Beamline Control and Supervision Beamwatch presents a synoptic view of the beamlines Autorised people can thus operate remotely on the beamline intrumentation

15. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it EVC Architecture EVC has 2 main components:  the application server and  a set of nodes The application server is running the portal application, the user and project database; the application server activates actions implemented by agents running in the nodes or requests services to external systems node1node2 nodek-1 nodek application server DB

16. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it EVC facts EVC project started on June 2001 and finished on June 2003. The first prototype was installed on the Xray Diffraction beamline of ELETTRA on June 2002 EVC is now operating on all the beamline and experimental station of ELETTRA and acts as the web interface to the Elettra Scientific Computing Environment (instruments, computing farms, storage) EVC was presented at SMAU2002, NOBUGS2002 and SMAU2003, HCI2003, NOBUGS2004 EVC development staff is partecipating in many EU founded projects under FP6 (BIOXHIT, IA-SFS/JRA1, GRIDCC, EUROTeV/GAN)

17. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it BIOXHIT Virtual Collaboratory System The BIOXHIT project which will develop an integrated platform for high-throughput structure determination ELETTRA is developing the Virtual Collaboratory System a Virtual Organization (VO) connecting all the European laboratories doing research in the field of structural genomics.

18. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it BIOXHIT VCS scenario Supporting Services VO user crystallisation beamline 1 beamline 2 beamline 3 Processing Farm Processing Farm Data Storage VCS will be used to implement a widley distributed Virtual Organisation (VO) connecting all the stations and Laboratories involved in the BIOXHIT project

19. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it VCS Node @ EMBL-Hamburg

20. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it EUROTeV/GANMVL In the EUROTeV project the design study of the International Linear Collider ELETTRA is developing the Multipurpose Virtual Laboratory, the core tool to implement the Global Accelerator Network, a VO connecting all the international laboratories doing research in the field of Accelerators. Remote control of an accelerator facility has the potential of revolutionizing the mode of operation and the degree of exploitation of large experimental physics facilities. The first prototype of the system planned by April 2005 will allow the remote control of ELETTRA storage ring from DESY.

21. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it MVL @ Work

22. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it MVL @ Work

23. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it www.lightsources.org

24. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it VCS/MVL Virtual Organisation

25. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Evolving EVC to implement VCS / MVL In this architecture every institute in the VO should have a VCS Node. The node can support more stations (e.g. a data collection station, a cristallisation, a control room station, a movable station etc). Stations can share resources and tools. The remote collaborator will use his PC equipped with a web browser and if the case with a projector. All the communication (AS-to-AS and AS-to-LN) is done via webservices secured with X.509 certificates by mutual authentication Legacy applications are integrated using VNC if they do not have a web interface or ssh tunnels and proxy or redirection if they already have a web interface.

26. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Evolving EVC to implement VCS / MVL All the communication between the Application Servers located in the distributed laboratories is done via webservices (Axis implementation). All the Local Nodes run a Local Node Server. Comunication between the AS and the LN is done via webservices (gSOAP implementation). The systems are also equipped by a Management Station which allows easy configuration and maintenance via web browser. VCS and can be considered a sort of integration platform. Scripts stored in the AS database are transferred to the local nodes, executed and the results returned to the user via the AS.

27. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it The GridCC Project Instruments GridComputational Grid +

28. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it GRIDCC project in 3 steps Development of generic Grid middleware, based on existing building blocks (Grid Services) which will allow the remote control and monitoring instrumentation such as distributed systems. Testing of the middleware on challenging applications to validate it both in terms of functionality and quality of service:  European Power Grid  Geo-hazards  Remote Operation of an Accelerator Facility  High Energy Physics Experiment  … Dissemination of the new software technology to encourage a wide range of enterprises to evaluate and adopt our Grid-oriented approach to real-time control and monitoring of remote instrumentation.

29. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it The MCE and the GRIDCC landscape Supporting Services Virtual Ctrl. Room Virtual Ctrl. Room Diagnostics Instrument 1 Instrument 2 Instrument 3 The MCE is a software to implement Virtual Control Rooms, i.e., multi-user, collaboration-supporting interfaces to a widely distributed control system with access to grid-enabled computing and data storage facilities StorageElement ComputingElement

30. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it GridCC Architecture Information Monitor Service Computing Element Computing Element Computing Element Storage Elements Storage Elements Storage Element Global Problem Solver Virtual Control Room Virtual Control Room Security Service AutS TGS PolR Instrument Element Instrument Element Instrument Element Collaborative Service Exec. Service WfMS WMS AgrS

31. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it The Multipurpose Collaborative Environment a groupware providing general purpose services and interfaces to support collaboration among researchers and operators, to control remote instrumentation, and other tasks related with experimental activities will be used to implement the VCR for the different GRIDCC pilot applications through customization and integration with application-specific services. based on a core groupware application (providing, e.g., authentication, management of the VO users and instruments) and a set of plug-ins:  General purpose (e.g., chat, notebook, video conference)  Specific to the particular application (e.g., accelerator control, specific instrument control)

32. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Design and Development Approach State of the art evaluation  Collaboration support tools for scientific experimental activities  Available technologies Derivation of general requirements  through use cases, interviews, exchanges with other related projects (e.g., EUROTeV) Prototyping  Discussions over interface sketches  Incremental development of functional prototypes

33. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Design Choices Web-based portal interface  no installation, porting, clients available almost anywhere  Minimal system requirements: Web browser, JRE for applets Exploit portlet technology  Why? manageable integration of application-specific functionalities within the MCE  Current prototypes are based on a modified version of the GridSphere framework, developed under the GridLab EU project  Collaboration with the GridSphere team to extend and improve the framework to our needs was established and some of our developments are already in GridSphere codebase

34. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Login page

35. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Personalisation

36. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it eLogbook

37. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it VO view: people browser and chat

38. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Resource browser and Instrument control

39. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it More Instruments Control

40. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Video/Phone Conference: VRVS/Skype

41. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Desktop Sharing

42. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Execution Services: job submission

43. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Execution Services: File Access

44. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it Evolving EVC with MCE EVC project started in 2001 and has now more than 4 years of operations EVC software has been improved in these years moving from a simple single facility web application to a multi-facility integration platform based on webservices We are currently refactoring EVC in order to migrate to the GRIDCC MCE middleware.

45. Instruments and Sensors on the Grid Melbourne, Australia, December 2005 Roberto Pugliese pugliese@elettra.trieste.it GRIDCC Architecture IE VCR IE ExeS IE CE IE SE IMS SecS PS VCR WfMS WMS AgrS AutS TGS PolR VIGS ACM IMDM IMSpx RS LPS CollabS