1. European Middleware Initiative (EMI)
Alberto Di Meglio (CERN)
Project Director

2. Outline Project objectives Administrative structure
Technical structure
The EMI Middleware
Relationships and Collaborations
Conclusions
EMI Overview

3. State of the Art and Objectives
EMI Overview

4. A European Vision Tomorrow Today Sustainability Persistence
Interoperability
Easier Access
Today
EMI Overview

5. What is EMI?
The European Middleware Initiative (EMI) project represents a close collaboration of the three major middleware providers - ARC, gLite and UNICORE, together with other software providers - to establish a sustainable model to support, harmonise and evolve the grid middleware for deployment in EGI and other distributed e-Infrastructures
EMI Overview

6. Objectives
Consolidate the existing middleware distribution streamlining services and components to meet and exceed the operational requirements of EGI and other distributed computing infrastructures Develop and standardize the middleware services/functionality following the requirement of EGI and other DCIs, fully integrating virtualization, MPI, monitoring interfaces and embracing new computing models like clouds
Consolidate
Evolve
EMI Overview

7. Objectives
Reactively and proactively maintain the middleware distribution to provide users with increasingly user-friendly, maintainable, reliable, stable, and scalable software Promote the EMI achievements in the user communities, defining and implementing standards when needed, contributing to the establishment of a sustainable model
Support
Promote
EMI Overview

8. Project Governance
EMI Overview

9. Partners (23)
EMI Overview

10. Administrative Structure
Collaboration Board (CB)
ECB
Project Director
Technical Director
Project Executive Board (PEB)
Project Technical Group (PTG)
EMI Overview

11. Project Structure Administrative and Technical Management
Dissemination and Communication
Quality Assurance
Maintenance and Support
Integration, Interoperability and Testbeds
Development and Evolution
EMI Overview

12. Technical Structure
EMI Overview

13. Technical Areas Information Services Data Management Job Management
Security
Quality Assurance
Maintenance and Support
Integration, Interoperabilty and Testbeds
Development and Evolution
EMI Overview

14. Product Teams
Product teams are the services implementation teams within each area responsible to deliver software releases and all associated material.
They perform the required technical tasks from design to release through implementation, testing, certification and 3rd-level support as part of one or more Work Package and according to what specified by the Project Technical Group and the QA policies.
Product Teams are flexible, in the sense that they can be formed or closed as the corresponding products are introduced or obsoleted in EMI and allow adding or removing services as needed even from external contributors
They provide a transparent and direct method to assign responsibility for a service
EMI Overview

15. Product Teams Job Management Data Management Information Services
Security
EMI Release x.y.z-r
x.y.z-r
Quality Assurance
Maintenance and Support
Integration, Interoperabilty and Testbeds
Development and Evolution
EMI Overview

16. Software Engineering PT Release Stress tests Scalability tests
Functional tests
Regression tests
Deployment tests
Packages
Unit tests
Build
Develop
Design
Quality Assurance
Maintenance and Support
Definition
Execution
Integration, Interoperabilty and Testbeds
Development and Evolution
EMI Overview

17. Quality Assurance Improve Release Stress tests Scalability tests
Functional tests
Regression tests
Deployment tests
Packages
Unit tests
Build
Develop
Design
ISO 12207
(Software Lifecycle Processes) ISO/IEC (Software Maintenance)
ISO/IEC (Software Testing) ISO 9126 (Software Quality)
Quality Assurance
Maintenance and Support
Integration, Interoperabilty and Testbeds
Development and Evolution
Monitor
EMI Overview

18. Releases and Release Policies
Major releases: once or twice per year, may contain non-backward compatible changes
Minor releases: a few times per year, fully backward compatible, may contain new functionality
Revisions: every week or two weeks, only bug fixes
Emergency: as needed, only specific bug fixes, use emergency release procedures
Service Level Agreements
EMI Overview

19. The Middleware
EMI Overview

20. Selection Criteria EMI includes:
Middleware services and components from the participating providers
All services that are or will be in production by next year
EGI, PRACE operations must be fully supported
New services that are supposed to replace existing services
New services satisfying identified new user needs
EMI requirements are user-driven, satisfy the needs of the user communities (Archaeology, Astronomy, Astrophysics, Civil Protection, Computational Chemistry, Earth Sciences, Finance, Fusion, Geophysics, High Energy Physics, Life Sciences, Multimedia, Material Sciences, etc.)
EMI Overview

21. Day-0 Distribution Job Management Data Management Information Services
Grid Manager
A-REX
libarclient
ng* job CLI
arc* job CLI
JURA
TSI
XNJS
UAS-Compute
OGSA-BES
URC
UCC
HILA
WMS LB
CREAM
MPI
Classic SE
libarcdata2
ng* data CLI
arc*data CLI
Chelonia
dCache
StoRM
UAS-Data
DPM
lcg_utils
DPM/DICOM LFC
FTS
Hydra
AMGA
Classic Infoserver
Classic Infoindex
ALIS
ISIS
libarcclient (infosys)
Grid Monitor
CIS
Service Registry
BDII
lcg-info and lcg-infosites
Service Discovery API
CEMon
Messaging
APEL
HED security API
arcproxy
Charon
Gateway
XUUDB
XACML Entity
UVOS
VOMS/VOMS-Admin
voms-proxy-*
ARGUS
SLCS
SCAS/LCAS/LCMAPS
gLite-Exec (glExec)
gLite-Delegation Client
Shibboleth SLCS Client
Gridsite
CGSI_gSoap
delegation-Java
Trustmanager/Util-Java
Proxyrenewal
Job Management
Data Management
Information Services
Security
EMI Overview

22. Harmonization Targets
Data Handling Unification
APIs and Clients Harmonization
Standardization
(PGI, SRM, SAML/XACML, US/RUS, GLUE,etc)
Common Service Registry
Common Security Framework
EMI Overview

23. New Development
New services or functionality are introduced as needed based on:
New user requirements
Need to evolve existing services or replace older technologies to follow the growing needs of the infrastructures
Special focus on integration of virtualization (grids on clouds), monitoring interfaces, support for easy construction of gateways
Trade between new research and stability of the infrastructure operations
Large-scale realistic testing, phase-out/migration plans, support policies
EMI Overview

24. Relationships
EMI Overview

25. EGI EGI MCB, SA2 (MU), SA3, SA4 3rd-level support Releases Specs
Requirements
Repositories, tests, certification tools, etc
PTG
EMI Overview

26. User Communities SSCs EGI SSCs MCB, SA2 (MU), SA3, SA4 SSCs
3rd-level support Releases
Specs
Requirements
Repositories, tests, certification tools, etc
PTG
EMI Overview

27. Interoperability partners, commercial partners,
Collaborations
SSCs
SSCs
EGI
MCB, SA2 (MU), SA3, SA4
SSCs
3rd-level support Releases
Specs
Requirements
Repositories, tests, certification tools, etc
PTG
Works with EMI
Interoperability partners, commercial partners,
EMI Overview

28. Exploitation and Sustainability
Successful deployment in EGI, PRACE and other infrastructures of the EMI middleware
Collaboration with standardization bodies
Evolution of existing standards
Definition of new standards
Inclusion of middleware in OS distributions
Packaging requirements, support, conflict management
Service Level Agreements
Move towards a customer/provider model that could in time become a real commercial relationship
EMI Overview

29. Open Collaborations EMI is not a private club
Collaborations with additional software providers are welcome and sought after
“Works with EMI” Program
Collaborating providers need to follow the same rules as the EMI partners for the software to be included in EMI releases
Openness is necessary to converge towards usable common standards and seriously interoperating solutions within and outside Europe
Industrial collaborations are welcome
EMI Overview

30. Conclusions
EMI Overview

31. Conclusions
EMI is built on the strengths of the major European middleware providers
To provide a reliable, usable, scalable, interoperable middleware distribution
It is designed to be the major middleware provider for EGI, PRACE and other DCIs
Support for reliable operations
Evolve the services as the communities grow
It’s an important step forward in the establishment of a true pan-European sustainable, persistent research infrastructure
EMI Overview

32. Thank you