The DRIVER Project Paolo Manghi ISTI - National Research Council, Italy

EC: the “Knowledge Infrastructure” Vision Build and maintain environments where content and functionality resources can be openly shared to serve the needs of different user communities e-Infrastructures is the term commonly used to refer to such environments, including both people and technology

DRIVER Project: the Mission Focus: EC Open Access mandates for research output Networking: developing standards and organisational and business models for a sustainable network of European strategic partners in the Open Access community Technical: building an infrastructure for aggregating Open Access research publications from Digital repositories in Europe and making it available for elaboration to any interested community

4 DRIVER Project: the Objectives Develop an environment for integrating existing national, regional, or thematic repositories Create a production-quality European digital repository infrastructure Prepare the future expansion and upgrade of the digital repository infrastructure across Europe Identify and promote the use of a relevant set of standards Raise Open Access awareness among user communities Slide from Prof. Yannis Ioannids, NKUA, Greece

D-NET Technology The DRIVER Project solution 5

Digital Repository Internet OAI-PMH metadata Repository Dspace Greenstone ePrints OpenDLib …. UI Query Ingest

DRIVER’s technical goals Construction and maintenance of the European Information Space for Open Access research publications Arbitrary number of applications consuming the Information Space User portals Consuming applications Key-feature: evolving requirements

Aggregative Digital Library Systems (ADLS) ADLSs Aggregation system: maintaining and populating an Information Space by aggregating content from a collection of OAI-PMH Repositories Custom application: providing community-specific functionalities via Web User Interfaces Well known examples BASE (Germany) DAREnet (Netherlands) OAIster (USA) Others…

Typical ADLS architecture One institution, one community BASE (Germany) DAREnet (Netherlands), OAIster (USA) Others… OAI-PMH … Aggregator Index UI … Store … Search Portal Aggregation system Info Space

The DRIVER project goals Many institutions, many communities Construction of the European Information Space for Open Access research publications Arbitrary number of repositories (data providers) Data curation Arbitrary number of applications consuming the Information Space Community portals Community Information Spaces (comm. metadata format) Etc. Highly evolving requirements!

Typical ADLSs Vs DRIVER requirements OAI-PMH … Aggregator Index UI … Applications Aggregation system Store … Search OAI-PMH New Institution Site Manual maintenance cost New Fun Info Space New UI Index Store New Info Space Aggregator OAI-PMH

Typical ADLSs Vs DRIVER requirements Drawbacks Limited customizability E.g. pre-defined input and target metadata formats High-cost software extensibility E.g, new functionality, new Information Spaces “Manual” repository management Registration, harvesting, curation (XSLT), etc… “Manual” administration for robustness and scalability E.g., store and index replicas Issue: sustainability in the long term 12

DRIVER’s D-Net An Infrastructural approach Service-Oriented Architecture Web Service, service registration, subscription&notification, etc. Distributed running environment Administered by one responsible organization (RO) Used for by participating organizations (POs) to collaboratively build ADLSs ADLSs as service-oriented applications

D-NET’s Service Kit: ADLS components Repositories FS, FTP, NFS Data Sources Web UI Service Web UI Service Recomm. Service Recomm. Service Community Service Community Service User Profile Service User Profile Service Search Service Search Service Data Management Data Management OAI-PMH Service OAI-PMH Service Index Service Index Service Browse Service Browse Service Store Service Store Service OAI-PMH Harvester Service OAI-PMH Harvester Service Information Service Information Service Manager Service Manager Service Authz&Authn Service Authz&Authn Service Collection Service Collection Service Validator Service Validator Service Feature Extraction Service Feature Extraction Service Similarity Service Similarity Service Transformation Service Transformation Service Compound Object Service Compound Object Service Citation Service Citation Service XML Import Service XML Import Service Object Packaging Service Object Packaging Service Repository Man Service Repository Man Service ResultSet Service ResultSet Service End User Functionality Enabling Personalization Service Personalization Service

DRIVER’s D-Net An Infrastructural approach ADLS construction Service compositionality (“LEGO approach”) functionality in “isolation”; e.g. index, storage, aggregation Service customizability e.g., functionality independent from metadata formats Service distribution ADLS components can be distributed over the network Service sharing Hardware and services Application “autonomicity” Services can be orchestrated automatically to accomplish certain tasks 15

ADLSs in D-Net OAI-PMH Repository Index Search Index UI … OAI-PMH … … Enabling Layer Middleware UI Search Index Store Harvester User Profiling … Others Harvester Service Kits Trasformator Store Content Resources Dynamic, distributed Run-time Infrastructure RO PO Repository

Compositionality and customizability Harvester OAI-PMH IndexUISearch IndexUISearchStore Index Transformer Store Index UI Search Metadata Formats IndexUISearch

Sharing of functionality Trasformator Index Search Index UI … Enabling Layer Middleware UI Search Index Trasformator User Profiling … Others Harvester UI Store Dynamic, distributed Run-time Infrastructure Service Kits RO PO OAI-PMH Repository OAI-PMH … … Content Resources Repository

Sharing of content Index Search Index UI … Enabling Layer Middleware UI Search Index Aggregator User Profiling … Others UI Search Store Dynamic, distributed Run-time Infrastructure Service Kits RO PO TrasformatorHarvester OAI-PMH Repository OAI-PMH … … Content Resources Repository

Sharing of content Index Search Index UI … Enabling Layer Middleware UI Search Index Aggregator User Profiling … Others UI Search Index Store Service Kits Dynamic, distributed Run-time Infrastructure RO PO TrasformatorHarvester OAI-PMH Repository OAI-PMH … … Content Resources Repository

D-Net’s benefits Towards software sustainability... Customizability Parametric services and compositionality Openness Designed to integrate new functionality Scalability and Robustness By distribution and replicas Sharing Cost optimization (services and hardware) Autonomic behaviour Reduced maintenance and administration cost Repository management tools GUIs for harvesting and aggregation

D-Net Software Toolkit Software packages Open Source Apache License Release v1.0 (production) and v1.2 (beta) Release 2.0 (beta): Enhanced Publication

D-NET and standards Service Resources are implemented as Web Services and accessed through the corresponding Web Service Interface Parameters calls are enveloped into SOAP messages The Enabling Services are also compatible with REST XML is the lingua-franca for the whole system Resource internal status, i.e. Resource profiles, are represented as XML files conforming to a given schema Profiles are kept into the Information Service, whose underlying engine is an Exist XML engine

D-NET and standards Subscription and Notification Service Any Service can subscribe to events regarding any DRIVER Resource: creation, deletion, and specific action accomplished by a resource The Subscription and Notification mechanism is compliant with the OASIS Standards WS Base Notification 1.3 and WS Topics 1.3 Authorization and Authentication Service offers security contexts to all Resources according to the Access Control Markup Language standard (XACML)

DR IVE R revi ew Wa rsa w Ma y 15t h D-NET standardization Information Service system mediation All relevant resources register their profile into the IS; e.g. Services, collections, indexes, users, etc. Services can access system relevant information through the IS, in a common standard way, with no need to statically know the locations of other Services ResultSet mechanism Standard interfaces and tools for data exchange By reference or value Paging modes, transformation, caching

Example: DRIVER production infrastructure - D-Net’s release v1.2 Enabling Layer Data Layer Functionality Layer Administrators End users Advanced User Interfaces National portals EU Open Access Repositories

DRIVER production infrastructure Current status v1.2 Content About 250+ harvested repositories (more than twice to come) over 33 countries from Europe and beyond 2,200,000+ open access publication metadata records Services Production release: 36 service running instances over 9 nodes located at CNR, UNIBI, ICM and NKUA

D-Net’s uptake European Film Gateway EC project OpenAIRE EC pilot Experimentation of deployment of new infrastructure instances China, India, Portugal, Belgium, Spain, Slovenia Upcoming: Greece and Bulgaria

Technical Team ISTI: Istituto di Scienze e Tecnologie Informatiche, Centro Nazionale delle Ricerche, Pisa, Italy ICM: Interdisciplinary Centre for Mathematical and Computational Modeling, Uniwesytet Warszawski, Poland NKUA: Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, Greece UNIBI: Universität Bielefeld, Germany

D-NET’s interoperability 30

Data interoperability Integrating heterogeneous data sources Content (structure and semantics) Access Protocols Tools Mediation Accessing heterogeneous data sources (standard protocols) Normalization of data, using the same exchange formats (XML) Transformation Transforming data of different data models (structure and semantics) into data of other data models Examples: cleaning, enriching 31

Service interoperability Adoption of Standards Web service, XML, Security, etc Internal policies ResultSets, Service Profiles, Discovery mechanisms, etc Best practices Design of “customizable” services Objectives Minimizing D-NET software learning curve Maximizing reusability of D-NET components Facilitaning integration of software 32

Credits Paolo Manghi ISTI - CNR Speaker’s Contact Details DRIVER II Project Supported by European Commission