BalticGrid-II Project The BalticGrid-II Final Review, Brussels, 26 th May, Joint Research Activity Enhanced Application Services on Sustainable e-Infrastructure Bartek Palak Poznan Supercomputing and Networking Center

The BalticGrid-II Final Review, Brussels, 26 th May, Outline JRA activity of BalticGrid-I Recommendation from BG-I final review JRA activity description Objectives Description of tasks Summary of work done Plans for the next period Conclusions

The BalticGrid-II Final Review, Brussels, 26 th May, JRA activity of BalticGrid-I Account Service Level Agreement Markets and Dynamic Account Management Partners: EENet, IMCS UL, KTH, PSNC, VU Objectives  Investigate, prototype mechanisms for SLA security and enforcement  Design and develop account management system  Development of products: – VUS – Virtual User System – DGAS - Distributed Grid Accounting System – Tycoon – approach for better Service Level Agreement (SLA) Outcome  VUS and DGAS proved its quality by running on production sites.  Several insurmountable issues have been found with the Tycoon system – Unreliable tool, HP changed the policy, no open source any more – The decision of BG is not to use this system due to this issues

The BalticGrid-II Final Review, Brussels, 26 th May, Recommendations from BG-I final review Recommendations:  the consortium will need to provide end-users with the necessary capabilities to use the grid infrastructure effectively […]  following on from the above point, JRA1 and related work are going to become increasingly important and the reviewers recommend that the consortium should continue work in this area in future projects; Recommendations implementation: Activity focuses on providing services enabling efficient and intuitive usage of Grid resources

The BalticGrid-II Final Review, Brussels, 26 th May, JRA Activity 7% of project resources 31,5 PMs Partners  Vilnius University  Vilnius Gediminas Technical Univeristy  University Poznan Supercomputing and Networking Center (coordination)

The BalticGrid-II Final Review, Brussels, 26 th May, Objectives Extending the e-Infrastructure with advanced services necessary from the user, application and administrator point of view - the GUI level Enhancements of existing services enabling user-friendly and intuitive access to Grid environment Providing support for the new research communities well defined in NA3 (Application Identification and Collaboration) and supported by SA3 (Application Integration and Support)

The BalticGrid-II Final Review, Brussels, 26 th May, 2010 Description of tasks Providing user-level support for Grid advanced data management Adaptation of task flow definition and management mechanisms Providing visualization tools for scientific computing in Grid Development of a user-friendly environment used for cooperation of user groups and applications A clear vision of components development: Migrating Desktop advanced grid interface Visualisation of specific computing results Gridcom groupware

The BalticGrid-II Final Review, Brussels, 26 th May, MJRA1.1 Design phase closed (1) Design phase closed (MJRA1.1) Definition of works planned within activity  Establishing co-operation rules  Definition of relations to other activities  Design of development directions Performance analysis of visualization tools in GRID environment  Deployment of existing data visualization tools – Visualization toolkit VTK – Turnkey package ParaView (based on VTK) – Visualization system LitVizG – gVid/i2gLogin software (and VTK version 4.4.2)  Visualisation tools performance tests

The BalticGrid-II Final Review, Brussels, 26 th May, Design phase closed (MJRA1.1) – continued Milestone reached on time (PM04) Milestone summarized in two reports:  DJRA1.1 “Performance analysis of sequential and parallel visualization in GRID environment” (PM03)  DJRA1.2 “Design phase, interoperability and definition of cooperation with other activities: SA1, NA3, SA3 and plan for education trainings in co-operation with NA2” (PM04) MJRA1.1 Design phase closed (2)

The BalticGrid-II Final Review, Brussels, 26 th May, First prototype ready (MJRA 1.2) Enhancement of BalticGrid-I NA3 project components to be compatible with Grid standards Providing support for advanced data management  File manager (Grid Commander) deployed as an independent network application as an independent network application  Restructuring of The Migrating Desktop  Mechanism for easy integration of various file systems  Changes in implementation of existing file systems  Support for Grid data access mechanisms Gridcom groupware development  The first deployable release  Gridcom recovering after system restart  Administrative commands for system configuration  Error handling improvements  Security enhancements MJRA1.2 First prototype ready (1)

The BalticGrid-II Final Review, Brussels, 26 th May, First prototype ready (MJRA 1.2) - continued Deployment of the software in BG-II testbed for visualization Visualization tools deployed considering outcome of performance analysis. All tools are based on VTK to minimize development efforts with gLite compatible architecture.  ParaView – an open-source software suitable for parallel visualization and parallel rendering of large datasets.  Visualization e-service VizLitG is designed for convenient access and visualization of remote data files located in grid.  VTK applications interactively run on grid by means of GVid software. BG-II users can employ full power of huge toolkit functionality and flexible interactivity  VTK applications interactively run on grid by means of GVid software. BG-II users can employ full power of huge toolkit functionality and flexible interactivity. Milestone reached on time (PM09) The first prototype details described in report:  DJRA1.3 - “First prototype […]” (PM09) MJRA1.2 First prototype ready (2)

The BalticGrid-II Final Review, Brussels, 26 th May, Migrating Desktop Handling gLite parametric jobs  Changes of job description structures  Support for parametric jobs in submission mechanisms  Job monitoring mechanisms - improvements  Providing support for list of parameters MJRA Second prototype ready (1)

The BalticGrid-II Final Review, Brussels, 26 th May, Migrating Desktop MD - VizLitG integration  VizLitG integrated as MD file viewer (HDF5) Definition and management of task flow  Evaluation of existing tools  Adoption of the Kepler workflow orchestration system  Publishing Kepler for BalticGrid-II users Preparation of training materials  Recorded demos  Tutorials  User guides MJRA Second prototype ready (2)

The BalticGrid-II Final Review, Brussels, 26 th May, MJRA Second prototype ready (3) Visualization Tools Adaptation of e-Service VizLitG to BG-II user needs:  Enhanced interactivity including VTK widgets,  Improved security (Message Authentication over SSL). Open-source package ParaView v3.6.1:  Developed unstructured HDF5 file reader for decomposed data files of pilot applications,  Distributed GPU rendering employed on grid. Advanced functionality of VisPartDEM:  Parallel visualization of large particle systems,  Visualization of initial defects and propagating cracks,  Visualization of safety margin forces.

The BalticGrid-II Final Review, Brussels, 26 th May, Gridcom New: User Access Control  Owner and many read-only users  User management UI  Handy for admins and colleagues New: Documentation Brochure, article and movie New: Official Gridcom Web Site MJRA Second prototype ready (4) Danila: Rewrite this page pls, to make it more descriptive. E.g. „User management UI” – creation? Enhancement? Design?

The BalticGrid-II Final Review, Brussels, 26 th May, Milestone reached on time (PM18) The second prototype details described in raport:  DJRA1.4 - “Second prototype […]” (PM18) MJRA Second prototype ready (5)

The BalticGrid-II Final Review, Brussels, 26 th May, MJRA Final version of services (1) MJRA1.4 Final version of services (PM24 ) Preparation of final release of components User support – publishing of user guides, tutorials Survey of services for user-level support of Grid data management Migrating Desktop  Evaluation of Migrating Desktop Framework  Optimization of developed mechanisms  User support – publishing of user guides, tutorials Gridcom  Documentation

The BalticGrid-II Final Review, Brussels, 26 th May, MJRA Final version of services (2) MJRA1.4 Final version of services (PM24) Assessment of performance and flexibility of visualization tools Open-source package ParaView:  Analysis of special purpose OpenGL shaders,  Performance tests (Done)  Applications (Planned 3, Done 3) Grid Visualization e-Service VizLitG:  Performance tests (Done)  Applications (Planned 3, Done 3) Visualization toool VisPartDEM:  Light version of tool and GUI,  Performance tests,  Applications (Planned 3, Done 2). Arnas: Update

The BalticGrid-II Final Review, Brussels, 26 th May, Summary of work done - usage All JRA tools are designed to be general Most components could be used with no integration Examples of current and planned usage:  ParaView : – Hopper discharge problem (Particle Technology, Engineering Modelling Tasks) /Integrated/, – Dam break problem (Computational Fluid Dynamics, Engineering Modelling Tasks) /Integrated/, – Oil filter problem (Computational Fluid Dynamics, Engineering Modelling Tasks) /Planned/.  VizLitG: – Tri-axial compaction problem (Particle Technology, Engineering Modelling Tasks) /Integrated/, – Dam break problem (Computational Fluid Dynamics, Engineering Modelling Tasks) /Integrated/, – Convective transport problem (Computational Fluid Dynamics, Engineering Modelling Tasks) /Planned/,  VisPartDEM: – Tri-axial compaction problem (Particle Technology, Engineering Modelling Tasks) /Integrated/, – Hopper discharge problem (Particle Technology, Engineering Modelling Tasks) / Integrated /, – Crack propagation problem (Particle Technology, Engineering Modelling Tasks) /Planned/.  Gridcom – MDS (Multidimensional scaling with city-block distances) HIROMB (operational hydrodynamic model for Lithuanian Baltic Sea coastal area) SHYFEM/AQUABC (Hydrodinamic-ecological model parameter sensitivity analysis)… /Integrated/ – GAUSSIAN NWCHEM SBIO NEURO PAR VIBRO /Planned/  Migrating Desktop – GAMESS, SYNTSPEC (stellar spectra modelling), SemTi-Kamols (natural language processing), CoPS (Complex Comparision of Protein Structures), ANSYS /Integrated/ – GAUSSIAN, deMon, PADEEA /Planned/ Arnas, Danila – update your parts

The BalticGrid-II Final Review, Brussels, 26 th May, Presentations & Publications (1) EGEE’08 conference (Istanbul, Turkey)  Live demo "Using the BalticGrid-II infrastructure: SemtiKamols - a linguistic analyser of Latvian language„ EGEE UF’09 conference (Catania, Italy)  Demo “CoPS - The Complex Comparison of Protein Structures supported by grid”  Talk ‘The Migrating Desktop as a framework for grid applications” NDTCS-2009 (Vilnius, Lithuania):  Presentation “Grid visualization e-service VizLitG” 1st International Conference on Parallel, Distributed and Grid Computing for Engineering, (Pécs, Hungary)  Presentation “Computation and Visualization of Poly- Dispersed Particle Systems on gLite Grid”

The BalticGrid-II Final Review, Brussels, 26 th May, Presentations & Publications (2) XVIIth seminar of Lithuanian Association of Computational Mechanics, (Lithuania)  Invited presentation “Scientific computations, graphical environments and visualization in grid environment“ BalticGrid-II Summer School (Moletai, Lithuania)  Lecture “Enhanced Application Services on Sustainable eInfrastructure“ EU DORII Summer School (Linz, Austria)  Tutorial “Migrating Desktop - intuitive graphical interface to Grid resources” EGEE’09 conference (Barcelona, Spain)  Live demo - "Migrating Desktop - Intuitive Interface to Grid Resources"  Live demo "Visualization e-services and tools for grid build by gLite"

The BalticGrid-II Final Review, Brussels, 26 th May, Presentations & Publications (3) Publications:  R. Kačianauskas, A. Maknickas, A. Kačeniauskas, D. Markauskas, R. Balevičius. Parallel discrete element simulation of poly-dispersed granular material. Advances in Engineering Software, 41(1), 2010, 52–63.  A. Kačeniauskas, R. Kačianauskas, A. Maknickas and D. Markauskas. Computation and Visualization of Poly-Dispersed Particle Systems on gLite Grid Civil-Comp Proceedings, vol. 90, ISSN , Proc. of 1st International Conference on Parallel, Distributed and Grid Computing for Engineering (Eds. B.H.V. Topping and P. Iványi), ISBN , Civil-Comp Press, Stirlingshire, United Kingdom, 2009, p [ISI Proceedings]  A Kačeniauskas, R. Pacevič, T. Katkevičius, A. Bugajev. Grid visualization e-service VizLitG. CDROM Proc. of 13th International Workshop on New Approaches to High-Tech: Nano Design, Technology, Computer Simulations (Eds. A. Melker, V. Nelayev, J. Tamuliene), Vilnius, Vol. 13, 2009,  A Kačeniauskas, R. Pacevič, T. Katkevičius. Dam break flow simulation on grid Dam break flow simulation on grid. CDROM Proc. of 13th International Workshop on New Approaches to High-Tech: Nano Design, Technology, Computer Simulations (Eds. A. Melker, V. Nelayev, J. Tamuliene), Vilnius, Vol. 13, 2009,  A. Kačeniauskas. Mass conservation issues of moving interface flows modelled by the interface capturing approach. Mechanika, ISSN , 2008, 69(1), p

The BalticGrid-II Final Review, Brussels, 26 th May, Presentations & Publications (4) Disseminative materials:  JRA technical brochure: Enhanced Application Services on Sustainable e-Infrastructure  Technical brochure: “Grid visualization tool VisPartDEM”  Technical brochure: “Grid visualization e-service VizLitG”  The poster: “Grid Visualization e-Service VizLitG”  Technical brochure: “ParaView on BalticGrid-II”  The poster: “ParaView on BalticGrid-II” Arnas: Paste here one/two brochures, pls Danila: Any diss materials? Arnas: Paste here one/two brochures, pls Danila: Any diss materials?

The BalticGrid-II Final Review, Brussels, 26 th May, 2010 Beyond the BalticGrid-II User support for JRA products Further development: – Migrating Desktop – PL-Grid (NGI), Euforia (EU) – Gridcom - ( – visualization tools – LitGrid (NGI), …(?) Add plans of development/support of Gridcom, viz tools, pls

The BalticGrid-II Final Review, Brussels, 26 th May, Conclusions The works are proceeding smoothly as planned  Good cooperation within activity  All planned activity objectives fullfilled  The all milestones achieved on time  The deliverables completed on time Prototypes delivered  Services prepared and deployed  Prototype evaluated by SA3 Effective cooperation with other project activities  NA2 – support for educational and disseminative activities  NA3, SA3 – support for the users No technical problems that affected objectives Plans

The BalticGrid-II Final Review, Brussels, 26 th May, Thank you for your attention!