1. EGI Technical Forum Amsterdam, 16 September 2010 Sylvain Reynaud

2. JSAGA2 Plan SAGA quick overview JSAGA focuses –uniform usage of middleware –ease of extension –efficiency and scalability –control of the API behavior –operating-system independency JPySAGA Example applications Perspectives

3. JSAGA3 SAGA quick overview SAGA : the specification –a recommendation of Open Grid Forum –a Simple API –for using Grid middleware execution management –job submission, monitoring... data management –transfer, list files, search… service discovery … –in user Applications implementations (or wrappers) exist for C++, Java and Python S A G A See next talk ! independent of middlewares => use middlewares in a uniform way consistent "look-and-feel" => use functionalities in a uniform way

4. JSAGA4 The JSAGA implementation a Java implementation of the SAGA specification focuses on –uniform usage of middleware…. (beyond what they natively support) –ease of extension………………. –efficiency and scalability………. –control of the API behavior........ (configure binding rather than trying) –operating-system independency (tested on ) Under LGPL license (thanks to the design of adaptor interfaces)

5. JSAGA5 Uniform usage of existing grids

6. JSAGA6 user applications VOMS WMS gsiftp Gatekeep. Globus JSAGA GridFTP Example – uniform usage SAGA job SAGA jobout.txtout.txt

7. JSAGA7 user applications VOMS WMS gsiftp Example – what happens behind EGEE proxy + attributes translate submit job poll status delegate transfer DGrid proxy adapt + translate submit job receive status transfer –to any protocol Gatekeep. Globus JSAGA CREAM-CELCG-CE Unicore WMS WMS Globus GK RSL VOMS VOMS modified job wrapper script GridFTP SAGA job D-Grid EGEE JDLout.txtout.txt

8. JSAGA8 Layered software architecture JSAGA interface implementation

9. JSAGA9 Layered software architecture Used by end users Used by end users –web portals, GUI, CLI… Used by application developers Used by application developers –a single uniform interface –object oriented, high-level Implemented by adaptors Implemented by adaptors –each "way" to implement a given feature has 1 interface –service oriented, low-level Implemented by middleware Implemented by middleware –each technology has its own interfaces user applications middleware APIs core engine JSAGA adaptors JSAGA adaptors interfaces adaptors interfacesJSAGA SAGA SAGA Java binding interface implementation

10. JSAGA10 listen status changes transfer output Layered software architecture user applications GridFTP Globus GK adaptor get status get output upload data JSAGA core engine JSAGA adaptor interfaces adaptor Used by end users Used by end users –web portals, GUI, CLI… Used by application developers Used by application developers –a single uniform interface –object oriented, high-level Implemented by adaptors Implemented by adaptors –each "way" to implement a given feature has 1 interface –service oriented, low-level Implemented by middleware Implemented by middleware –each technology has its own interfaces

11. JSAGA11 Layered software architecture user applications GridFTP Globus GK adaptor get status get output upload data JSAGA core engine JSAGA adaptor interfaces Used by end users Used by end users –web portals, GUI, CLI… Used by application developers Used by application developers –a single uniform interface –object oriented, high-level Implemented by adaptors Implemented by adaptors –each "way" to implement a given feature has 1 interface –service oriented, low-level Implemented by middleware Implemented by middleware –each technology has its own interfaces listen status changes transfer output

12. JSAGA12 Layered software architecture user applications GridFTP Globus GK adaptor get status get output upload data JSAGA core engine JSAGA adaptor interfaces listen status changes Middleware API can be used more efficiently Middleware API can be used more efficiently –engine selects best interface for each user request Developing new adaptors is easier Developing new adaptors is easier –just wrap functionalities supported by middleware API transfer output

13. JSAGA13 LogicalPhysical Supported technologies Minimal dependencies on external libraries external tools –e.g. no gLite-UI operating system –tested on planned construction done Data Execution Security InMemCredGlobusG. LegacyG. RFC820MyProxyVOMSJKSSSHLogin / pwdX509gatekeepergLite-WMSwsgramunicore6creamnaregisshlocal rnslfnsrbirodscatalogrbyteiogsiftpsrmsrbirodshttphttpssftpftpfilezipcache Grid techno. Others

14. JSAGA14 C Python Jython? JPySAGA Python Bindings for SAGA Java SAGAJSAGA Java GAT Java Bindings for SAGA Java-based Python SAGA wrapper SAGA-C++ C++ Bindings for SAGA JySAGA a user application legacy python bind. Boost wrapper SAGA-C++Py a user application

15. JSAGA15 Example applications DIRAC ( D istributed I nfrastructure with R emote A gent C ontrol ) the pilot jobs system of LHCb experiment Pandora Gateway the grid platform of the company a Grid/Cloud Computing Business Services Provider JUX (Java Universal eXplorer) a multi-protocols extensible file explorer

16. JSAGA16 Example applications Elis@ web portal for industrial and academic production grids JJS (Java Job Submission) tools for submitting jobs to EGI resource selection based on QoS observed while submitting jobs SimExplorer software for simulation experiment management includes a workflow engine that uses JSAGA to run simulations on grids jCAE (Java CAE) Computer Aided Engineering with support for distributed computing facilities/

17. JSAGA17 Perspectives Develop new plug-ins –NorduGrid middleware ARC plug-in in progress… –LRMS (DRMAA-based) Grid Engine –LRMS (CLI-based/ssh) torque Grid Engine Develop API extensions –Grid-RPC (from SAGA core) –Service Discovery API (a SAGA extension) with plug-ins for technologies –BDII with plug-ins for schemas –GLUE –CIM

18. JSAGA18 Perspectives Job submission –principle describe user's job submit it to the grid –middleware select the execution host –OGF standards BES, DRMAA, SAGA –examples Remote Procedure Call (RPC) –principle deploy user's service on grid call a procedure –middleware select the service instance –OGF standards Grid-RPC, SAGA –examples jobGrid-RPC

19. JSAGA19 Future direction of interop. Client and server sides interoperability are complementary –Server-side interoperability efforts (in particular standardization) are essential to slow down middleware APIs diversity –But can we imagine server-side interoperability for all kind of distributed resources ? HPC/HTC academic/industrial grids international/regional grids –Server-side interoperability depends on middleware and resource providers only with client-side interoperability, (advanced) users can integrate themselves resources they need to access …and share their contribution with others ! clusters, SSH server, local host RPC grids, desktop grids, clouds next technologies…

20. JSAGA20 Questions ?