EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number Marios Chatziangelou, et al. Institute of Accelerating Systems and Applications (IASA) EGI Applications Database Status report

25/12/15 Capabilities Insert footer here A community driven, central service that stores and provides:  software solutions (in the form of native software and/or virtual appliances), originated from almost every scientific area/discipline  reference of scientific datasets (under development)  the programmers and scientists responsible for them  the publications derived from the registered items (SW, VA & datasets) Acts as distribution medium, for:  native software packages (RPMs, DEBs, TARs) exposed through the UMD Community Repository to the public  virtual machine images by using the HEPiX virtualization vmcatcher/vmcaster technology

35/12/15 Latest features Support for VO-wide image lists Full integration with the Grid/Cloud information system (BDII) and the GOCDB Service Extend the AppDB authentication system by supporting federated AAI through eduGAIN Software Appliances Integration with external metadata repositories Reference datasets (pilot activity)

45/12/15 VO wide image lists (1)  Editable only by VO managers  VO wide image lists always private  Site subscriptions using the vmcatcher  A personal access token is required  All site admins already have or should get a personal access token Simple workflow  The user submits one or more VAs, the user notifies the VO manager, the VO manager dicides whether he will include the VA version or not to a VO wide image list  The user updates a VA (=> new VA version), the system notifies the VO manager(s) that alerady includes the VA in their VO-wide image lists.....  Sites retrieve the new VA version and publish the details into the BDII

55/12/15 VO wide image lists (2) 13 Virtual Organizations and/or Projects are making use of the system 21 Cloud providers already support the AppDB Cloud Marketplace Some metrics:

65/12/15 Full integration with BDII & GOCDB Sites & Resource Providers - correlated information from GOCDB & top-BDII Contact information Sitename Description URIs to GOCDB for more details List of offered vAppliances and VMIs Usage details per offered VMIs Site endpoint Template ID OCCI ID

75/12/15 Extend authentication system: eduGAIN Support authentication using federated Identity providers through eduGAIN. First use case, EBI-EMBL. 1.select the “Federation (eduGAIN)” option for login 2.pick the EBI-EMBL IdP 3.and login into the AppDB with their EBI-EMBL credentials Liberates the users from the necessity of either obtaining a new account or for using X509 certificates.

85/12/15 Software Appliances Main purpose: deliver a Software-as-a-Service solution Technically speaking: SAs are Virtual Appliances and Contextualization scripts pairs In practice: SAs is a solution for installing and/or parametrize 'any' software module within a base Virtual Machine, by using a contextualization script ( = is the first script being executed while a virtual machine it is instantiated/started)

95/12/15 Integration with external metadata repositories Projects and Organizations – acquired from the openAIRE Integration with openAIRE (through the OAI-PMH protocol-interface) Separate AppDB sub-service for managing the high volume of metadata periodically fetched from openAIRE Over 33k Projects and over 27.5k Organizations Many types of relations can be made, between the registered software, vAppliance, person and the fetched projects and/or organizations

105/12/15 Reference datasets – pilot (1) In the context of Life Sciences Data Replication VT, AppDB is being extended into a dataset registry Initial focus is on Life Sciences reference datasets –Integration with the Elixir Tools and Data Services Registry is in the works –Support for other disciplines (e.g. HEP) will be considered in the near future, based on feedback and demand The AppDB dataset registry preview can be accessed from the new “datasets” tab on the development server instance

115/12/15 Reference datasets – pilot (2) Entries in the dataset list view can be grouped by version Every entry is tagged as either primary or derived

125/12/15 Reference datasets – pilot (3) Primary datasets represent original datasets, as posted by the provider –create/edit access available to Admins, Managers, and Dataset Managers Derived datasets are based on a primary dataset but –only a part of the information is kept (subset based on structure), –or only a part of the data entries are selected (subset based on quantity) –create/edit access to them may be open to all users Key characteristics recorded for each dataset include –name, description, disciplines, homepage link, licensing, and a version list Each dataset version may host one or more locations where data can be accessed Locations may be tagged as master or replica, and contain information about –the format in which the dataset is provided (e.g. FASTA, XML, CSV, etc.) –the interface / connection type (e.g. REST, SOAP, SQL, etc.) –the organization or EGI site which provides the location

135/12/15 Reference datasets – pilot (4)

145/12/15 Next steps Still on requirements gathering phase.  Stable requirements: –Finalize the dataset pilot related developments (expected on end of August) –Integrate the AppDB with the Elixir Registry (SW & VA & datasets) –EGI-Engage DoW: “The EGI Applications Database (AppDB) will evolve from its current role as catalogue of applications and virtual machines (VM) to include a graphical user interface allowing authorized users to perform basic VM management operations”. Open item: federated AAI  Draft requirements: –Support for ‘Containers’ i.e. Docker, technology –Developments with regards to the supported VMI formats (i.e. OVA, qcow2, etc.) –Extensions and/or developments that will needed, towards the EGI Marketplace activity/task –Integration with the EGI Accounting repository for fetching VM usage related metrics –Performance improvements (i.e. database clustering, light refactoring of the client-side scripting/javascript etc)

Thank you for your attention. Questions? EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number