1. A Stepwise Modeling Approach for Individual Media Semantics Annett Mitschick, Klaus Meißner TU Dresden, Department of Computer Science, Multimedia Technology Group „Applications of Semantic Technologies“ (AST' 06)

2. 06.10.2006AST 20062 Outline Introduction Approach: -Semantics for Personal Media Management -Information Instantiation and Integration -Implemented System Architecture Conclusion and Future Work

3. 06.10.2006AST 20063 Introduction Media Management @ Home: -Growing amount of digital assets (text, images, video, audio) complex, high-dimensional -User: non-/semi-professional individual conceptualizations usually minimum efforts to annotate/organize -Tools mostly apply administrative (objective) metadata  vs. user interpretation/conceptualization Goal: automated modeling of essential semantic descriptions from existing metadata to ease annotation work

4. 06.10.2006AST 20064 Introduction Solutions: -Automated feature extraction and indexing techniques to manage complexity of huge multimedia collections low-level features  Similarity-based Search (Query-by-Example, etc.) higher-level features  Pattern Recognition/Classification (objects, people, genres, etc.)  requires prototype features (user context, user feedback)  “Semantic Gap” -Media descriptions based on common metadata standards to enable interoperability & comparability MPEG-7 Multimedia Description Schema RDF/OWL representation for semantic modeling -Semantic Desktop Personal Information Management based on Semantic Web Technologies

5. 06.10.2006AST 20065 Approach :: Semantics for Personal Media Management Knowledge representation -Allow description of background-knowledge (context information like events, activities, actors, locations, etc.) -Allow integration of domain models and individual extensions (e.g. integration of MPEG-7 MDS, FOAF for description of people, etc.) -Manageable, straightforward  for non-professionals ABC [Hunter, et al.] -“facilitate interoperability between metadata ontologies“, “conceptual basis for automated mapping …”, “guidance to communities…” -Base classes to attach domain-specific entities  MPEG-7 MDS -Temporal semantics and related actualities (who, what, when, where)  evolution/transition of objects

6. 06.10.2006AST 20066 Approach :: Semantics for Personal Media Management Acquisition of as much information as possible from content + context -File information, administrative metadata, structural features, etc. Information sources depend on application context -naming, availability/accessibility, semantics, etc. of features/attributes e.g. “Date Time Original”/”Date Created”/…, “Creator”/”Author”/…, etc. -properties might be relevant, or redundant in some context e.g. “EXIF:ExposureTime” and “EXIF:ShutterSpeedValue”

7. 06.10.2006AST 20067 First step: -Apply sets of mapping rules to construct media-centric description (MDS) -Mapping rules expandable regarding supported metadata schemes (  developer) Approach :: Information Instantiation and Integration Make=FUJIFILM Model=FinePix F601 [(urn:Exif:Make rdf:value ?j) (urn:Exif:Model rdf:value ?k) (urn:Filepath rdf:value ?l) hashCode(?l, ?t) makeTemp(?x) -> (?t mds:creationTool ?x) (?x mds:make ?j) (?x mds:model ?k) (?x rdf:type mds:Device)] FUJIFILM FinePix F601... FUJIFILM FinePix F601

8. 06.10.2006AST 20068 Approach :: Information Instantiation and Integration Next steps: -Apply derivation rules (schema level) to establish relationships to actualities and temporalities (  developer, user) -Apply classification rules (instance level) to refine description model regarding known pattern (  user)

9. 06.10.2006AST 20069 Approach :: Implemented System Architecture Design rationale: -Automated import and indexing of media items triggered by file system actions -Registry of media type specific analyzing components  providing metadata and feature extraction -Processing of semantics and ontology model wrapped by generic model API  independent from RDF/OWL storage and querying solution (Jena, Sesame, …) -Extensible, flexible service architecture  plug-in based setup, extension at run-time (OSGi service platform)

10. 06.10.2006AST 200610 Conclusion and Future Work Our approach: -Part of the K-IMM (Knowledge through Intelligent Media Management) project  managing private media collections with the help of semantic technologies -Stepwise modeling of resource descriptions to separate distinct problems of construction of media semantics Building objective media descriptions from various sources Deriving subjective, user-oriented semantics Future Work: -Strategies to extend and refine basic rule sets by means of user feedback, context capture -Distribution of modeling steps (Web Services/Agents) -Detailed, comprehensive evaluation

11. 06.10.2006AST 200611 Thank you for your attention! Annett Mitschick annett.mitschick@inf.tu-dresden.de Multimedia Technology Group, Department of Computer Science, TU Dresden http://www-mmt.inf.tu-dresden.de