1. EUDAT Site and Service Registry
Mark van de Sanden, SURFsara
Johannes Reetz, RZG
Registry RDA plenary
San Diego
March 11, 2015
eudat.eu

2. EUDAT Collaborative Data Infrastructure
EUDAT generic data service provider storage, workflows, processing, archive
deposit
Community Repositories
(thematic data centres)
access
4/12/2019

3. EUDAT Collaborative Data Infrastructure
EUDAT front office
Repository or Data Deposit
e.g. B2DROP, B2SHARE
EUDAT back offices, data service provider storage, workflows processing, archive
deposit
Repositories
access
4/12/2019

4. The central registry in EUDAT
Provides information about the Configuration of the Collaborative Data Infrastructure, its organisational and technical components and how they are grouped.
The (static) information is used for different scenarios such as monitoring and notification services, data policy management services, other information services, for building contact lists of site, security, service contacts and more.
4/12/2019

5. Requirements for the Site and Service* registry
Provide detailed configuration information about sites, service components, service groups with the flexibility to extend the list of attributes (e.g. further certificates)
Security and safety: Role management with a fine grained access and authorization
Ability to define separate administrative domains
High availability
Browsable web interface
API for queries – preferably REST based
Ability to schedule services for downtimes
Production-ready sustainable solution, strong user base
*) „Service“ in terms of Service Oriented Architecture generally means a group of service components (latter also called „service endpoints“)
4/12/2019

6. EUDAT registry based on the EGI GoCDB
Provides Static information about Service Providers (Sites), Service Endpoints and the provided Services in EUDAT.
Any Service Provider and its Services can be registered and properly described.
GLUE2 as basic information model, but any additional property can be added to the description of any site, service component and service group. This guarantees sufficient flexibility.
Developed by EGEE, STFC and EGI
EUDAT2020 will contribute to the further development
Other community infrastructures might be interested to use this technology to manage the configuration of their own e-Infrastructure
4/12/2019

7. Data Infrastructure as Configuration of Services
provided by Data Repositories and Generic Service providers
Distributed Services
REGISTRY Configuration management DB for Sites and Services
GoCDB5
EUDAT generic data service (back offices)
EUDAT front offices (Repository or Deposit)
Community Repositories (front offices)
4/12/2019

8. EUDAT (phase 1) site/service registration model repositories and generic data centre services are registered in one administrative zone
Community Repository (front office)
site or service admin
puts/updates information
EUDAT front office (e.g. B2SHARE, B2DROP)
EUDAT generic data service (back office)
Registry
user or
service
retrieves information
via web/REST interface
4/12/2019

9. registered sites (organisations)
4/12/2019

10. Organisational details (here site RZG)
4/12/2019

11. Service endpoints provided by a site
4/12/2019

12. Service with standard and extension properties
extendable set of attributes
4/12/2019

13. List of Service Groups currently defined
4/12/2019

14. Service Group B2SAFE for CLARIN Comprising all relevant front service components
Supervisor for the SoA
(gets notified about failures)
Relevant PID services
Repository services
Generic storage services
Service Group administrators
Optional property extensions
4/12/2019
Scheduled downtimes

15. Monitoring of B2SAFE_CLARIN service group
4/12/2019

16. REGISTRY Sites, Services, ServiceGroups
Service Groups can be used to define «Islands of Trusted Service Endpoints»
Enables communities to pre-define the domain of sites and services where their data is allowed to be stored, managed and processed.
creg.eudat.eu
REGISTRY Sites, Services, ServiceGroups
Community
EUDAT
3rd EUDAT Conference - 16

17. GoCDB REST API (as used by EUDAT)
Method
Description
Protected
get_cert_status_changes
changes to certification statuses Y
get_cert_status_date
certification status for production sites and change date
get_downtime
downtimes for sites and services N
get_service_endpoint
service endpoints and associated information
get_service_types
valid service types and associated description
get_service_group
service groups and the service endpoints under those groups
get_service_group_role
service groups and the roles held by users over these groups
get_site
site information including contacts, grouped by site
get_site_contacts
persons (and associated info) having a role at site level
get_site_list
sites with minimal associated information
get_site_security_info
Returns security contact information for sites
get_user
Returns a user or a list of users with associated details and roles
get_project_contacts
persons (and associated info) having a role over a project.
$wget --no-check-certificate -O result.xml '
4/12/2019
more about the GoCDB API

18. API Example: query for service groups
<SERVICE_GROUP>…..</SERVICE_GROUP>
<SERVICE_GROUP PRIMARY_KEY="30G0">
<NAME>B2SAFE_CLARIN</NAME>
<DESCRIPTION>Service endpoints which belong to B2SHARE for all CLARIN repositories</DESCRIPTION>
<MONITORED>Y</MONITORED>
<GOCDB_PORTAL_URL>
<SERVICE_ENDPOINT>……</SERVICE_ENDPOINT>
…..
<EXTENSIONS/>
</SERVICE_GROUP>
4/12/2019

19. API Example: service component with extensions
<SERVICE_ENDPOINT PRIMARY_KEY="24G0"> <PRIMARY_KEY>24G0</PRIMARY_KEY>
<HOSTNAME>irods0-eudat.esc.rzg.mpg.de</HOSTNAME>
<GOCDB_PORTAL_URL>
<HOST_OS>Linux</HOST_OS> <HOST_ARCH>amd64</HOST_ARCH>
<BETA>Y</BETA> <SERVICE_TYPE>b2safe.irods</SERVICE_TYPE>
<HOST_IP> </HOST_IP>
<CORE/>
<IN_PRODUCTION>Y</IN_PRODUCTION>
<NODE_MONITORED>Y</NODE_MONITORED>
<SITENAME>RZG</SITENAME>
<COUNTRY_NAME>Germany</COUNTRY_NAME>
<COUNTRY_CODE>DE</COUNTRY_CODE>
<ROC_NAME>EUDAT_REGISTRY</ROC_NAME>
<URL>irods://irods0-eudat.esc.rzg.mpg.de:1247</URL>
<EXTENSIONS>
<EXTENSION><LOCAL_ID>101</LOCAL_ID><KEY>irods_path</KEY><VALUE>/vzRZGE/eudat/</VALUE></EXTENSION>
<EXTENSION><LOCAL_ID>82</LOCAL_ID><KEY>irods_resource</KEY><VALUE>cacheResc</VALUE></EXTENSION>
<EXTENSION><LOCAL_ID>81</LOCAL_ID><KEY>irods_zone</KEY><VALUE>vzRZGE</VALUE></EXTENSION>
</EXTENSIONS>
</SERVICE_ENDPOINT>
4/12/2019

20. EUDAT2020 site/service registration model EUDAT offers community repositories (sites, services) to register and manage the information in separate administrative domains
Community Repositories
Administrative
domain A
Registry
Domain B
Roles
Domain admins
Site admins
Service group admins C
EUDAT admin domain
EUDAT Repository
(e.g. B2SHARE)
EUDAT back office service provider
4/12/2019

21. EUDAT2020 (phase 2) dealing with distinguished community e-Infrastructures
Community Repositories
Community back office
Registry
Domain A
Domain B
Community X
Infrastructure
Registry
Hub
Domain X C
EUDAT admin domain
EUDAT Repository
EUDAT back office service provider
4/12/2019

22. Wishlist for the Site and Service registry
Registry service must be trustworthy (auditable): the authenticity of information and its sources must be ensured.
PIDs for service components; stable, unique IDs are needed for accounting records, service configuration descriptions, …
Some additional standard attributes required: certification (trust) levels, Licenses, SLA and OLA levels
Some critical database entries, e.g. quality indicators, require genuine signatures and authentic binding (e.g. certificates)
Some dynamic information should be made available (e.g for workflows): quotas, available resources,
Modification of entries should be possible via REST API (e.g. scheduled downtimes, rather dynamic information)
History (log) about changes (to be able to track who changed when which entry)
4/12/2019

23. 4/12/2019

24. Some service components (endpoints)
4/12/2019

25. Example: Service Group for B2SAFE Comprising all front service components (endpoints) of a SoA
4/12/2019