Rapid Prototyping of Semantic Mash-Ups through Semantic Web Pipes Danh Le-Phuoc, Axel Polleres, Manfred Hauswirth, Giovanni Tummarello 1, Christian Morbidoni 2 1 Digital Enterprise Research Institute, National University of Ireland, Galway 2 Univ. Politecnica delle Marche, Ancona, Italy WWW Summarized and presented by Hwang Inbeom, IDS Lab., Seoul National University

Copyright  2010 by CEBT Semantic Web Pipes  Generalized semantic web application development framework Supports fast development of semantic data mash-ups Preserving desirable properties – Abstraction, encapsulation, component-orientation, code re-usability, and maintainability Introduces concept of ‘pipe’ to semantic web application development 2

Copyright  2010 by CEBT Introduction  Semantic web applications Requires the combination and integration of RDF data Limited software support Lack of standard programming paradigms  Generic web applications in contrast Have many sophisticated abstractions and tools Well-supported rapid development is available 3

Copyright  2010 by CEBT Introduction (contd.)  Use of SPARQL processors as a solution? A lot of classical software development problems remain – Error-prone, hard to debug, uncontrollable workflow, … Hard to apply agreed architectural styles ‘Pure’ SPARQL has obvious limitations 4

Copyright  2010 by CEBT Introduction (contd.)  Current situation of semantic web application development is similar to that of generic web development community before … 3-tier model for database-oriented web applications Web development tools to support the architecture 5

Copyright  2010 by CEBT The Key Problems  Increasing amount of RDF data … Is fragmented May be incomplete, incorrect or contradicting Partly follows ontologies, often with wrongly or inconsistently used ontologies  Need for standard application development framework 6

Copyright  2010 by CEBT Authors’ Approach: Semantic Web Pipes  We need to sanitize RDF data before integration Decompose the data integration into several flows Handle each flows in parallel  Flexible architectural style For fast development of reliable data-intensive applications using RDF data Based on classical pipe abstraction 7

Copyright  2010 by CEBT Yahoo Pipes  Mash-up application development framework  Users can implement customized services and information streams through the combination of data sources (RSS feeds)  Applications are published to the user community They can be reused and combined to form new pipes 8

Copyright  2010 by CEBT Concept of the Pipe  Functional block of encapsulated operations With several number inputs and a desired output  Composes and processes set of RDF sources by means of pipelined special purpose operators  Decomposing an overall data-integration and processing task into a smaller sub-queries 9 Pipe Encapsulated operations Processed / Integrated data flow RDF RDF data sources

Copyright  2010 by CEBT An Motivating Example  Aggregation of data about Tim Berners-Lee from various sources 10 Changing URI of TBL used in DBLP into his URI used in FOAF Aggregated RDF data

Copyright  2010 by CEBT An Motivating Example (contd.)  Pure SPARQL equivalent of the example 11

Copyright  2010 by CEBT Benefits  Modular design Reusable components – A complete pipe can be used by design of other pipes, as a data source Easy to debug – Each components can be independently executed More intuitive development procedure  Convenient query optimization e.g. Performing reasoning task only to required parts in the execution steps 12

Copyright  2010 by CEBT Operators of Semantic Web Pipes  Base operators for … Data fetching Transforming Reasoning SPARQL processing 13 Generalized form Data fetchingTransformingReasoningSPARQL processing

Copyright  2010 by CEBT SPARQL Processing Operators  Three types of input Datasets – FROM, FROM NAMED clauses SPARQL query Variable bindings  Variables in the query bound to input are replaced by given XML / text  Query processor iterates over all variable binding combinations Simple nested FOR loops – Cartesian product of all values 14

Copyright  2010 by CEBT Operators of Semantic Web Pipes (contd.)  Merge and split operator for workflow control Merge – Takes arbitrary number of RDF graphs as input – Produces a RDF graph composed of the merge of its inputs Split – Single input is cloned to arbitrary number of outputs 15

Copyright  2010 by CEBT An Example of Semantic Web Pipe ?isbn… “A”… “B”… “C”… 16 CONSTRUCT { … ?price … } FROM … WHERE { ?isbn :price ?price … } “A” “B” “C”

Copyright  2010 by CEBT System Design and Implementation  Being developed as an open source project: DERI Pipes  Important components Web-based pipe editor Pipes repository – Pipes are stored as XML files 17

Copyright  2010 by CEBT System Design and Implementation (contd.)  Sharing pipes As resource provider of other pipes Increasing reusability 18 Published pipes list

Copyright  2010 by CEBT Evaluation  Test case studies Three test cases – ‘Tim Berners-Lee on the Semantic Web’ – ‘Friends’ publications’ – ‘SIOC aggregation RSS feed’ Comparison of LOCE(Line Of Code Equivalent) between – Java implementation – Pure SPARQL implementation – Semantic Web Pipes 19

Copyright  2010 by CEBT Evaluation (contd.)  Cognitive Dimensions of Notations Subjective test composed by a set of terms and concepts which have established themselves as important – Abstraction gradient: Can fragments be encapsulated? – Consistency: How much part of the language can be inferred from known parts? – Error-proneness: Does design notation induce “careless mistakes”? – Hidden dependencies: Does dependencies indicated in both directions well? – Premature commitment: Do programmers have to decide without sufficient information? – Progressive evaluation: Can partially-complete program be executed? – Role expressiveness: Do components have readability? – Viscosity: How much effort is required to perform a single change? – Visibility and juxtaposability: How much part of code is visible to readers? 20

Copyright  2010 by CEBT Evaluation (contd.)  Performance evaluation Performance results are not significant factor, because they are dependent to the implementation – However, it is helpful to discuss in general the model related aspects of the pipe execution performance and which optimizations can be applied Many optimization opportunities – Execution of multiple branches in parallel – Query processing can be optimized manually – Full or partial results can be cached 21

Copyright  2010 by CEBT Conclusions and Discussions  Conclusions Semantic Web Pipes remedies current situation of semantic web application development A Semantic Web Pipe consumes online data and each published pipe itself becomes a semantic web source that can be used for others  Discussions Good inspiration and implementation Number of applications can be expressed is limited due to lack of expressiveness It is hard to apply this programming paradigm to real world applications – As well as Yahoo Pipes 22