1. Introduction to EGI-Engage and WP3 - e-Infrastructure Commons
Diego ScardaciEGI.eu/INFN
EGI.eu/INFN
12/5/2018
EGI-Engage – First Monthly Meeting

2. EGI-Engage – First Monthly Meeting
Outline
The EGI-Engage project
Open Science Commons
EGI-Engage WP3 & Open Science Commons
EGI-Engage WP3 – e-Infrastructure commons
Objectives
Tasks
Deliverables & Milestones
Software Licence – EGI-Engage rules
Work organisation
WP3 Development Roadmap
F2F meeting
3/19/2015
EGI-Engage – First Monthly Meeting

3. EGI-Engage – First Monthly Meeting
3/19/2015
EGI-Engage – First Monthly Meeting

4. EGI-Engage – First Monthly Meeting
EGI-Engage and WP3
EGI-Engage (March 2015/August 2017)
43 partners (35 funded), 1161 PMs, 30 months
8m Euro EC contribution
Mission: accelerate the implementation of the Open Science Commons
AAI
Service Registry & Marketplace
Accounting
Operational Tools
Resource Allocation
EGI-Engage DoA:
3/19/2015
EGI-Engage – First Monthly Meeting

5. EGI-Engage Competence Centers
BBMRI: data and compute infrastructure that will allow a biobank federation for the storage of big data, parallel data analysis (EGI/EUDAT pilot)
DARIAH: federated cloud storage for high availability of data exploitation, science gateway (grid&cloud), real-time search application
EISCAT_3D: archiving and processing of data (joint EGI/EUDAT pilot)
ELIXIR: enable a federated data analysis infrastructure for representative use cases
EPOS: AAI, IaaS/PaaS/SaaS federated cloud services for data exploitation
Structural biology/INSTRUCT: computation services for big data
LifeWatch: Processing tools for biodiversity data, depositing of citizen scientist data, on demand virtual laboratories in the federated cloud
Disaster Mitigation: Enable applications from this domain on DCIs and facilitate access through gateways
3/19/2015
EGI-Engage – First Monthly Meeting

6. EGI-Engage – First Monthly Meeting
Open Science Commons
3/19/2015
EGI-Engage – First Monthly Meeting

7. EGI-Engage – First Monthly Meeting
Open Science Commons
3/19/2015
EGI-Engage – First Monthly Meeting

8. EGI-Engage – First Monthly Meeting
3/19/2015
EGI-Engage – First Monthly Meeting

9. Open Science Commons Pillars
Data
subject matter for research
dealt with according to the principles of open access and open science
maintaining trust and privacy for researchers.
Digital services/e-Infrastructures
technical services supporting researchers
integrated services and interoperable infrastructures across Europe and the world.
Scientific instruments
equipment and collaborations which generate scientific data (from small to large scale)
Knowledge
The human networks
3/19/2015
EGI-Engage – First Monthly Meeting

10. EGI-Engage – First Monthly Meeting
EGI Engage WP3
&
Open Science Commons
3/19/2015
EGI-Engage – First Monthly Meeting

11. New EGI technical architecture
3/19/2015
EGI-Engage – First Monthly Meeting

12. WP3 & Open Science Commons
Development of the EGI Core Infrastructure Platform and evolution towards e-Commons
AAI  wider adoption of federated IDM, Attribute Authorities, policies and pilots
Requirements, policy, technology, experimentation, adoption
EGI Marketplace  discoverability of services
Accounting, monitoring
Technical, policy and operations harmonization including security operations
3/19/2015
EGI-Engage – First Monthly Meeting

13. e-Infrastructure commons
EGI Engage WP3
e-Infrastructure commons
3/19/2015
EGI-Engage – First Monthly Meeting

14. EGI-Engage WP3 e-Infrastructure commons
Coordinate the development of the e-Infrastructure Commons
User-driven services
Full interoperability with other e-Infrastructures and Ris
Objectives:
Methods for authentication and authorisation in the EGI ecosystem.
Simplify the access to the infrastructure
Technological innovation
Service Registry and Marketplace
3/19/2015
EGI-Engage – First Monthly Meeting

15. EGI-Engage WP3 e-Infrastructure commons
Evolve the EGI accounting system
Big data
New types of accounting metric (e.g. data accounting)
New portal
Support new technologies
Satisfy new requirements emerging from service providers and user communities.
Define interfaces to integrate infrastructures
3/19/2015
EGI-Engage – First Monthly Meeting

16. WP3 Partners, Tasks and Effort
Activity Manager: Diego Scardaci (EGI.eu/INFN)
Partners involved: INFN, EGI.eu, SWING, CESNET, CSIC, CNRS, GRNET, SRCE, SURFsara, CYFRONET, STFC
Effort: 153 PM
Task
Effort
Leader
JRA1.1
Authentication and Authorisation Infrastructure
M3-M27
24.5 PM
Christos Kanellopoulos /GRNET
JRA1.2
Service Registry and Marketplace
M1-M30
22.5 PM
Dean Flanders/SWING
JRA1.3
Accounting
33 PM
Stuart Pullinger/STFC
JRA1.4
Operations Tools
56 PM
Cyril Lorphelin/CNRS
JRA1.5
Resource Allocation –
e-GRANT
17 PM
Tomas Szepieniec
/CYFRONET
3/19/2015
EGI-Engage – First Monthly Meeting

17. EGI-Engage – First Monthly Meeting
Task JRA1.1
Leader: Christos Kanellopoulos (GRNET)
Partners involved: GRNET, EGI.eu, CESNET, NIKHEF, STFC
Effort: 24.5 PM
Objectives:
Explore approaches to easier safe management of user credentials
Identify requirements for user authentication against (web) applications
Develop authentication solutions for use cases (CCs)
Explore eduGain, eduRoam, or STORK
Alternative identity-vetting approaches to current practices (e.g. IGTF IOTA)
Liaise with other projects focusing on AAI (AARC)
3/19/2015
EGI-Engage – First Monthly Meeting

18. EGI-Engage – First Monthly Meeting
Task JRA1.2
Leader: Dean Flanders (SWING)
Partners involved: SWING, IASA, INFN, CYFRONET, STFC
Effort: 22.5 PM
Objectives:
Design and the development of a proof-of-concept of a Service Registry and a Marketplace
Service Registry:
list of all the services offered by EGI
available for consultation by the EGI customers
Marketplace:
allow customers to acquire services (through a payment or for free)
EGI.eu as broker/intermediary
Prototype
3/19/2015
EGI-Engage – First Monthly Meeting

19. EGI-Engage – First Monthly Meeting
Task JRA1.3
Leader: Stuart Pullinger (STFC)
Partners involved: STFC, CESGA
Effort: 33 PM
Objectives:
Evolve the accounting system to manage big data
Include new types of data accounting
record who accesses data, how often, how much is transferred and where to
Extend the current accounting measurement:
Cloud, Storage and GPGPU accounting
Redesign the accounting portal
unified views in the portal for different kind of resources
Expose a complete API to gather accounting data from the system
3/19/2015
EGI-Engage – First Monthly Meeting

20. EGI-Engage – First Monthly Meeting
Task JRA1.4
Leader: Cyril Lorphelin (CNRS)
Partners involved: CNRS, CESNET, GRNET, AUTH, SRCE, STFC
Effort: 56 PM
Objectives:
Ops portal, GOCDB, ARGO, messaging, security monitoring
Implement a modular architecture to manage AAI
Make tools able to serve any research infrastructure
Evolve and improve their support for cloud resources
Define interfaces towards analogue tools belonging to other e-Infrastructures or Ris
Expose internal data through a REST API interface
3/19/2015
EGI-Engage – First Monthly Meeting

21. EGI-Engage – First Monthly Meeting
Task JRA1.5
Leader: Tomasz Szepieniec (CYFRONET)
Partners involved: CYFRONET
Effort: 17 PM
Objectives:
e-Grant development:
to cover the complete SLA lifecycle
to publish information about EGI resources on the EGI Service Registry and Marketplace
to support evolution towards a pay-for-use model
to enable customers to jointly request resources owning to different infrastructures.
3/19/2015
EGI-Engage – First Monthly Meeting

22. Deliverable in Horizon 2020
distinct output of the project, meaningful in terms of the project's overall objectives and constituted by a report, a document, a technical diagram, a software
deliverable classification:
R: Document, report (excluding the periodic and final reports)
DEM: Demonstrator, pilot, prototype, plan designs
DEC: Websites, patents filing, press & media actions, videos, etc.
OTHER: Software, technical diagram, etc.
Milestone
means control points in the project that help to chart progress
3/19/2015
EGI-Engage – First Monthly Meeting

23. Deliverable classification Impact in our work
Deliverables
Report (R):
classical deliverable
managed as EGI-InSPIRE
Demonstrator (DEM) and Software Release (OTHER):
each prototype and software release is a deliverable
release schedule already defined in the DoA
they should be properly documented
a delay on releasing a software will be visible by the project officer and reviewers
Any document to the commission? More info in the next weeks
3/19/2015
EGI-Engage – First Monthly Meeting

24. Deliverables and Demonstrators releases
Documented in the WP3 wiki pages
For each release the following information should be provided:
Release number
Release notes
Link to the documentation (users & admins)
Link to the installation package
Link to the release tag in the source code repository
3/19/2015
EGI-Engage – First Monthly Meeting

25. Software License EGI-Engage Rules
Software code, tools and interfaces developed in EGI-Engage will be released as open source code
full access via source code repositories such as GitHub or SourceForge.
Software developers will be able to choose their preferred source code repository (EGI-Repository as catch-all)
ensure that the contribution is openly accessible
add the metadata information needed to enable reuse
communicate the URL to the consortium
Repo location documented in D1.2 “Quality Plan”.
EGI-Engage software code, tools and interfaces will be published under a license of CC-BY type
all software dependencies compatible with the same type of license
3/19/2015
EGI-Engage – First Monthly Meeting

26. EGI-Engage – First Monthly Meeting
WP3 Deliverables N.
Title PM
Type
Owner
D3.1
Technical design of the new Accounting Portal and implementation plan
M06
Report
CSIC
D3.2
Design of the EGI Service Registry and Marketplace
M12
SWING
D3.3
Accounting Repository release
Other
STFC
D3.4
First release of the Operational tools
CNRS
D3.5
Analysis on techniques to manage big data on the EGI accounting system
M15
D3.6
First release of the new Accounting Portal deployed in production
M14
D3.7
First release of the EGI Service Registry and Marketplace prototype
M18
Dem.
D3.8
First data accounting prototype
M20
D3.9
Identity Management for Distributed User Communities report
M24
GRNET
D3.10
Second release of the new Accounting Portal deployed in production
3/19/2015
EGI-Engage – First Monthly Meeting

27. EGI-Engage – First Monthly Meeting
WP3 Deliverables N.
Title PM
Type
Owner
D3.11
Second release of the Operational tools
M24
Other
CNRS
D3.12
Second release of the Accounting Repository
STFC
D3.13
Report on Data Accounting
Report
D3.14
Second release of the EGI Service Registry and Marketplace prototype
M26
Dem.
SWING
D3.15
Second data accounting prototype
M28
D3.16
Final report on EGI Service Registry and Marketplace
M30
D3.17
Final release of the Accounting Repository
D3.18
Final release of the Operational tools
D3.19
Final release of the new Accounting Portal deployed in production
CSIC
3/19/2015
EGI-Engage – First Monthly Meeting

28. EGI-Engage – First Monthly Meeting
WP3 Milestones N.
Title PM
M3.1
Operational tools development roadmap agreed   
M04
M3.2
User requirements on data accounting collected
M12
M3.3
Operational tools development roadmap revised
M16
M3.4
Pilot services and best practices to enable federated AAI solutions released
M3.5
The first version of the EGI Marketplace is demonstrated
M18
M3.6
The second version of the EGI Marketplace is demonstrated
M26
3/19/2015
EGI-Engage – First Monthly Meeting

29. Software Releases – Schedule (1/2)N.
Title PM
Owner
D3.3
Accounting Repository release (OTHER)
M12
STFC
D3.4
First release of the Operational tools (OTHER)
CNRS
D3.6
First release of the new Accounting Portal deployed in production (OTHER)
M14
CSIC
D3.7
First release of the EGI Service Registry and Marketplace prototype (DEM)
M18
SWING
D3.8
First data accounting prototype (DEM)
M20
D3.10
Second release of the new Accounting Portal deployed in production (OTHER)
M24
D3.11
Second release of the Operational tools (OTHER)
3/19/2015
EGI-Engage – First Monthly Meeting

30. Software Releases – Schedule (2/2)N.
Title PM
Owner
D3.12
Second release of the Accounting Repository (OTHER)
M24
STFC
D3.14
Second release of the EGI Service Registry and Marketplace prototype (DEM)
M26
SWING
D3.15
Second data accounting prototype (DEM)
M28
D3.17
Final release of the Accounting Repository (OTHER)
M30
D3.18
Final release of the Operational tools (OTHER)
CNRS
D3.19
Final release of the new Accounting Portal deployed in production (OTHER)
CSIC
3/19/2015
EGI-Engage – First Monthly Meeting

31. EGI-Engage – First Monthly Meeting
WP3 work organisation
Mailing lists:
Task leaders mailing lists (tbd)
Task mailing list (tbd)
Indigo WP3:
RT queues:
egi-engage-wp3: internal actions
requirements: user requirements
3/19/2015
EGI-Engage – First Monthly Meeting

32. EGI-Engage – First Monthly Meeting
WP3 work organisation
Monthly meeting:
Organised ‘by tasks’
Wiki pages:
Main wp3 page:
Wiki page for each task:
Releases & Roadmap
Reporting:
Bi-weekly report by Monday CoB (before each AMB)
Contributions collected by task leaders and published in the wiki
First report on 30 of March:
3/19/2015
EGI-Engage – First Monthly Meeting

33. Development Coordination
discussions and announcement related to Engage project (deliverables, reports etc)
mandatory for EGI-Engage partners
for performance reporting as core service and other stuff related OLAs
mandatory for core services providers
for any communication between ops tools and technical discussions
all operations tools used in production or developed to become used in production
announcing releases, meetings, technical discussions
3/19/2015
EGI-Engage – First Monthly Meeting

34. EGI-Engage – First Monthly Meeting
Development Roadmap
Development Roadmap
to be updated quarterly
Each PT should fill its table in the task wiki page
Macro activities (no technical details)
3/19/2015
EGI-Engage – First Monthly Meeting

35. EGI-Engage – First Monthly Meeting
EGI Engage WP3
F2F meeting
Amsterdam March
3/19/2015
EGI-Engage – First Monthly Meeting

36. EGI-Engage – First Monthly Meeting
F2F meeting
14-15 April in Amsterdam
Indico page:
One session for each task:
Content to be defined in collaboration with the task leaders
Collecting users requirements
Please, register ASAP
3/19/2015
EGI-Engage – First Monthly Meeting

37. EGI-Engage – First Monthly Meeting
Open Actions
Task leaders: Deputy for each task leader
Task leaders: Mailing list for each task (if needed)
Re-use existing mailing list when possible (e.g. marketplace)
Task leaders: F2F
Session contents
All: People
Complete the excel file I sent you
Provide the effort for each WP3 member
All: Dev Roadmap
Define Roadmap
Fill in related tables in the WP3 wiki pages
Task leaders: Reporting:
First bi-weely report in the wiki by Monday 28 of March CoB:
All: F2F - Register to the meeting in indigo
3/19/2015
EGI-Engage – First Monthly Meeting

38. EGI-Engage – First Monthly Meeting
Questions ?
3/19/2015
EGI-Engage – First Monthly Meeting