Enabling Access to Sound Archives through Integration, Enrichment and Retrieval WP2 – Media Semantics and Ontologies

12 Month Review Meeting Project # Introduction to Workpackage  Objectives  O1-Modelling semantics for media data  O2-Representing semantics for media data  O3-Ontology management environment  Proof-of-Concept for Ontology-based searching

12 Month Review Meeting Project # O1 – Identifying semantic Requirements  Scope and usage  Media description  Representation of Content Structure (e.g., segmentations, sequence in time)  Representation of Content Description (e.g., low-level features – tempo, chord)  Knowledge representation formalism  Reasoning support  Easy annotation  Openness Reference: aceMedia multimedia ontology framework

12 Month Review Meeting Project # O1 - Music Ontology  Overview  Merging existing related ontologies  Developed by QMUL  Cover the major requirements  Widely-adopted  Four core MO components  FRBR  FOAF  Event  Timeline

12 Month Review Meeting Project # O1 - Ontology Modularization  Modularization is the partition of the knowledge infrastructure into accurate ontological modules in order to model different aspects of multimedia content.  Allow easy to use and easy to reuse (without changes applied to one affecting the other)  Interoperation and Integration (ontology mediation)  Support for different level of applications  Easy to link multimedia ontologies to domain-specific ontologies

12 Month Review Meeting Project # O1 - Modularized EASAIER Ontology  Top module  basic concepts and relations  concepts and relations in core modules are specializations of top module ones  Core modules  Audio module  concept shared and further specialized by core audio modules (Speech, Music)  Music module  integration of the Music Ontology  Speech module  Video module  Domain ontologies  store special knowledge needed for describing the content of multimedia files e.g. Politicians, Sports etc.

12 Month Review Meeting Project # O1 - Content of the top level ontology module  Central concepts  File, Segment—the primary bearers of multimedia metadata  Low level feature—automatically extractable characteristics of segments  Algorithm—feature extractor procedures  Event—represented, or recording events

12 Month Review Meeting Project # O1 - Content of the speech ontology module  Speech related features and their extractor algorithms  Special concepts and instances for relational features

12 Month Review Meeting Project # O2 – Semantic Representation  Basic Requirement (wish lists from W3C OWL working group)  Ontology sharing (ability to import other ontologies)  Ontology versioning (support changes or evolution)  Interoperability (facilating easy mapping of various ontologies)  Ontology reasoning (providing various reasoning support, simple reasoning is desirable for the reality)  Expressiveness and scalability (a good balance of both, layered structure is important)  Easy to use in syntax and semantics (tools, clear documentation)  Compatibility with other standards  Internationalization

12 Month Review Meeting Project # O2 - WSML  Jos de Bruijn, Stijn Heymans: Translating Ontologies from Predicate-based to Frame-based Languages. In Proceedings of Second International Conference on Rules and Rule Markup Languages for the Semantic Web (RuleML 2006)  Web Service Modeling Language  Language for describing ontologies and Web services  European initiative lead by DERI  Family of formal Web languages layered on top of XML and RDF:  WSML-Core  WSML-DL  WSML-Flight  WSML-Rule  WSML-Full

12 Month Review Meeting Project # O3 – Ontology Management Environment  WSMT  Mainly developed by DERI  It is collaborative EU efforts  Graphical ontology editing, querying, mapping For more details, see WSMT demo

12 Month Review Meeting Project # Deliverables and Milestones  D2.1 Report on metadata management infrastructure and ontology language for media objects  Due Month 24 (May 2008)  In good progress. Works are continuously documented in the internal notes  Milestones  M1-Month6: Semantic requirements established  M2-Month12: Ontology languages selected (WSML and OWL). Further improved integration of WSML DL with OWL DL  Progress  M1 and M2 have been successfully achieved.  Heading for M3 (Month18):  Demo the ontology management infrastructure (WSMT demo)  Demo the functionalities of the infrastructure (Proof-of-the-Concept for ontology-based searching)

12 Month Review Meeting Project # Contributions and Connections with Other Workpackages WP2: Media Semantics and Ontologies WP4: Sound Object Representation WP3: Retrieval Systems WP5: Enriched Access Tools WP6:Intelligent Interfaces  WP4: metadata requirements are considered during the ontology design  WP5: ontology is used to access/query related data  WP3: ontology is used for retrieval system.  WP6: ontology is used for dynamic interface generation

12 Month Review Meeting Project # Upcoming Work Plan Months  Speech ontology  Further development and enriching  Integration with the music ontology  Further mappings of metadata schema to the EASAIER ontology  Further constructing ontology managment infrastructure (Milestone 3 – Month18)  Integration with Instance Store  Further extension and improvement of WSML DL to OWL DL  Testing WSMT

Demo1 Ontology Editing and Population

12 Month Review Meeting Project # Archive publication: Hotbed example Hotbed Database Music Ontology Querying Publishing Extending Instruments Taxonomy Hotbed RDF the Semantic Archivist Query Interface Sound Access tools Features Extraction, Visualization,...

Demo 2 - Proof-of-the-concept Ontology based searching

12 Month Review Meeting Project # Artist Search  Aggregation of music artist information from multiple web sources  Ontology based search:  MusicBrainz data mapped to the MusicOntology  MusicBrainz Web Service:  allows to retrieve artist URI by literal based search  Retrieve corresponding RDF document(s)  process with SPARQL queries (e.g. resolve relationships)  Web2.0 Mashups:  Retrieve data (videos, images) from external sources  utilize RSS Feeds, APIs etc. from Youtube, LyricWiki, Google  more accurate results using references from MusicBrainz RDF data

12 Month Review Meeting Project # Artist Search “Beatles” WS Interface process data...

12 Month Review Meeting Project # Artist Search

12 Month Review Meeting Project # Instrument Reasoning  Reasoning over HOTBED (Scottish Music Archive) instrument scheme  HOTBED database lifted to Music Ontology using D2R mapping  Extended with an Instrument taxonomy  Subsumption reasoning:  Retrieve instrument tree  Search for persons that play an instrument  Subclass relations: resolve persons playing more specific instruments  Example: Wind-Instrument < WoodWind < Flute

12 Month Review Meeting Project # Rules  Infer new knowledge with rules  UseCase: The Velvet Underground discography  Available information:  Membership durations  Album release dates  „Founders“ of the band ?  Members of the band during the release of the earliest album  Albums and corresponding members