1. 1 www.lpds.sztaki.hu/pgportal pgportal@lpds.sztaki.hu Advanced features of the P-GRADE portal Peter Kacsuk, Gergely Sipos Peter Kacsuk, Gergely Sipos MTA SZTAKI Tamás Kiss University of Westminster

2. 2 Contents Solving Grid interoperability Supporting collaborative Grid application development Classification of Grid portals

3. 3 Grid interoperability

4. 4 Grid interoperability 1 P-GRADE portal supports job submission in 2 nd generation Grids: –LCG – EGEE –GT2 – UK NGS and US OSG –NorduGrid P-GRADE Portal enables: –Connecting 2 nd generation production Grids

5. 5 User P-GRADE Portal v2.3 SZTAKI Portal Server Connecting 2 nd generation production Grids LCG-2 broker Certificate server Proxy Workflow Manchester Leeds UK NGS GT2 EGEE-VOCE LCG-2 Job Budapest Poznan Brno Supports both direct and brokered job submission

6. 6 Grid interoperability 2 P-GRADE Portal with GEMLCA supports legacy code service invocation in 2 nd and 3 rd generation Grids: –LCG, gLite – EGEE –GT2, GT4 – UK NGS Combination of jobs and legacy code services enables P-GRADE Portal to support: –Extending 2 nd generation Grids with legacy code services

7. 7 Extending 2 nd generation Grids with legacy code services Job Repository (GEMLCA) Manchester User Leeds GEMLCA P-GRADE Portal UoW Portal Server Executable NGS GT2 Poznan Budapest EGEE/VOCE Executable

8. 8 Grid interoperability 3 Combination of jobs and legacy code services enables P-GRADE Portal to support: –Transition from 2 nd generation Grids to 3 rd generation Grids: Extending 2 nd generation production Grids with 3 rd generation volunteer Grid sites Connecting 2 nd generation production Grids with 3 rd generation volunteer Grids –Connecting 2 nd generation and 3 rd generation production Grids

9. 9 GEMLCA Repository Manchester Westminster User Oxford P-GRADE NGS GEMLCA Portal UoW Portal Server UoW site – GT4 with GEMLCA Service Service Invocation Executable Extending 2 nd generation Grids with 3 rd generation volunteer Grid sites UK NGS GT2

10. 10 Extending the NGS with legacy code services and GT4 Resources GT2 Job submission to Rutherford GT4 Service Invocation at UoW GEMLCA Legacy code submitted to Oxford GT2 Job submission to Manchester

11. 11 GEMLCA Repository Manchester User Leeds P-GRADE NGS GEMLCA Portal UoW Portal Server NGS GT2 Grid WestFocus GT4 Grid UoW Brunel Service Invocation Extending 2 nd generation production Grids with 3 rd generation volunteer Grids Executable

12. 12 Connecting 2 nd generation and 3 rd generation production Grids Job Repository (GEMLCA) Manchester User Leeds GEMLCA P-GRADE Portal UoW Portal Server Executable NGS GT2 WestFocus GT4 Grid UoW Brunel Poznan Budapest EGEE LCG/gLite Executable Service Invocation Executable

13. 13 Plan: Connecting major Grids by GEMLCA and the P-GRADE Portal In test phaseAvailable

14. 14 Regional coverage of using P-GRADE Portal + GILDA For education & training

15. 15 Conclusion: Easy-to-use, technology-neutral Grid portal for e-Scientists The P-GRADE Portal hides differences of Grids Globus 2 and 4 – LCG2 – gLite Grid interoperability Transparent switching between Grid technologies Graphical tools for application development, execution and monitoring Sequential & parallel components can be integrated into large Grid applications Direct or broker-based resource allocation Your code does not have to contain grid specific calls

16. 16 Collaborative work support

17. 17 Grid usage Grid Grid middleware and resources Portal BrowserSpecial client Grid portal Execute application Get results Define application Grid app.

18. 18 The problem Grids have been designed for highly complex applications (The overhead is too big to execute small programs…) How can a single person define highly complex applications?

19. 19 The solution today… Grid Portal BrowserSpecial client Execute application Get results Define application Grid app. E-mail, phone, skype, MSN, …

20. 20 The solution today… Grid Portal BrowserSpecial client Execute application Get results Define application Grid app. E-mail, phone, skype, MSN, … How can I integrate the knowledge of many people into a single and consistent entity in the shortest time?

21. 21 Similar problems have been already solved… Concurrent editing of Web pages: WIKI, MediaWiki Concurrent editing of documents: Online collaboration tool in MS Word Concurrent editing of tables: Online collaboration tool in MS Excel …

22. 22 The solution: collaborative Grid portals Grid portals that support –concurrent engineering of applications –concurrent management of application execution Grid Portal Grid application

23. 23 The Collaborative P-GRADE Portal Grid P-GRADE Portal BrowserSpecial clientBrowserSpecial client submit results edit

24. 24 The Collaborative P-GRADE Portal Grid P-GRADE Portal BrowserSpecial clientBrowserSpecial client submit results edit No time-consuming communication … …only efficient, workflow- oriented collaboration!

25. 25 Collaborative Grid workflows Collaborativ e workflow Integrate knowledge – development phase Generate and distribute results – execution phase

26. 26 An e-Science example Collaborativ e workflow Topology of the graph Input protein molecules Experiment results Protein folding algorithm

27. 27 A business example Collaborativ e workflow Graph topology Input for the market prediction application Results of the prediction workflow Market prediction model

28. 28 Collaborative workflow development –The portal has to guarantee that the result is a consistent workflow –Protect against data loss  locking mechanisms –As real-time as possible!  to increase parallelism during development

29. 29 Workflow development model Locking workflow components (jobs) exclusively First come first served Only the owner can edit the component

30. 30 Workflow development model Locking workflow components (jobs) exclusively First come first served

31. 31 Workflow development model Locking workflow components (jobs) exclusively First come first served

32. 32 Screenshot Locked for me Locked for somebody else Unlocked

33. 33 Portal server Workflow editor #2 Requesting a lock Workflow editor #1 Lock request Lock accepted

34. 34 Portal server Synchronizing the different views Workflow editor #1 Push updated local view Workflow editor #2 Pull global view

35. 35 Result of the collaborative development Job data Job data Grid resource

36. 36 Executing collaborative workflows 1.Concurrent control requests It can happen in single-user case too (multiple browsers are opened) 2.Different users’ proxies to be used to execute different workflow nodes Grid 1 Grid 2 Grid 3 P1P1 P3P3 P2P2

37. 37 Conclusions Extend the single-user version of the P-GRADE Portal: –Concurrent engineering of workflow-based grid applications –Application execution by multiple users in a concurrent fashion –Prototype version already available

38. 38 Classification of Grid portals

39. 39 Two orthogonal aspects of Grid portals usage Number of accessible Grids –Single isolated Grid (SI) –Multiple isolated Grids (MI) –Multiple collaborative Grids (MC) Collaborative vice not collaborative usage –Multiple isolated users (MI) –Multiple collaborative users (MC)

40. 40 Portal classification Multiple Isolated users (MIxx) Multiple Collaborative users (MCxx) Single Isolated Grid (xxSI) MISI portals (current portals) MCSI portals Multiple Isolated Grids (xxMI) MIMI portalsMCMI portals Multiple Collaborative Grids (xxMC) MIMC portalsMCMC portals

41. 41 LCG-2 P-GRADE-Portal London Rome Athens MISI Portal: P-GRADE portal 1.0

42. 42 Portal classification Multiple Isolated users (MIxx) Multiple Collaborative users (MCxx) Single Isolated Grid (xxSI) MISI portals (current portals) MCSI portals Multiple Isolated Grids (xxMI) MIMI portalsMCMI portals Multiple Collaborative Grids (xxMC) MIMC portalsMCMC portals Multi-Grid portals

43. 43 GridLab SEE-Grid P-GRADE-Portal MIMI Portal: P-GRADE portal (2.1) London Rome Athens

44. 44 Portal classification Multiple Isolated users (MIxx) Multiple Collaborative users (MCxx) Single Isolated Grid (xxSI) MISI portals (current portals) MCSI portals Multiple Isolated Grids (xxMI) MIMI portalsMCMI portals Multiple Collaborative Grids (xxMC) MIMC portalsMCMC portals

45. 45 GridLab SEE-Grid P-GRADE-Portal London Rome Athens MIMC Portal: P-GRADE portal (2.1)

46. 46 Portal classification Multiple Isolated users (MIxx) Multiple Collaborative users (MCxx) Single Isolated Grid (xxSI) MISI portals (current portals) MCSI portals Multiple Isolated Grids (xxMI) MIMI portalsMCMI portals Multiple Collaborative Grids (xxMC) MIMC portalsMCMC portals

47. 47 SEE-Grid P-GRADE-Portal MCMC Portal: P-GRADE portal (3.0) GridLab London Rome Athens

48. 48 MI users WF GUIMC users WF GUI SI Grid WF manager MI Grids WF manager MC Grids WF manager Summary of classification P-GRADE 2.3 P-GRADE 3.0+

49. 49 Conclusions xxMC portals: execute a workflow simultaneously in many Grids P-GRADE portals 2.1 - 2.3 already work as xxMC portal providing service for several production Grids: –HunGrid –SEE-Grid –VOCE –UK NGS www.lpds.sztaki.hu/pgportal MCxx portal: Enables to collaborate through workflow applications! Collaborative P-GRADE Portal: Prototype was demonstrated at SC05 exhibition

50. 50 Thank you for the attention! Questions… kacsuk@sztaki.hu www.lpds.sztaki.hu