information retrieval

perspectives: information retrieval application(s) - surveillance psychological - information overload experimental - search engines algorithmic - (media) similarity metrics system - unified (media) repository presentation - user profile / integration search - precision & recall commercial - applications & certification

 image query -- all images with this person  audio query -- identity of speaker  text query -- all transactions with BANK Inc.  video query -- all segments with victim  complex queries -- convicted murderers with BANK transactions  heterogeneous queries -- photograph + murderer + transaction  complex heterogeneous queries -- in contact with + murderer + transaction retrieval media types text, images, audio, video, structured data

 precision -- how many answers are correct  recall -- how many of the right documents are returned precision = ( returned and relevant ) / returned recall = ( returned and relevant ) / relevant

term/documentd0d1d2 snacks100 drinks103 rock-roll011 frequency table indexing

video v, frame f f has associated objects and activities objects and activities have properties video content object instance: (oid,os,ip)  object-id -- oid  object-schema -- os = (fd,fi)  set of statements -- ip: name = v and name = v IN f video

frameobjectsframe-dependent properties 1Janehas(briefcase), at(path) -housedoor(closed) -briefcase 2Janehas(briefcase), at(door) -Dennisat(door) -housedoor(open) -briefcase

objectframe-independent propertiesvalue Janeage 35 height 170cm houseaddress... color brown briefcasecolor black size 40 x 31

 segment retrievals -- exchange of briefcase  object retrievals -- all people in v:[s,e]  activity retrieval -- all activities in v:[s,e]  property-based -- find all videos with object oid query language for video libraries SQL, XML, logic-based,...

media abstractions

 state -- smallest chunk of media data  feature -- any object in a state  attributes -- characteristics of objects  feature extraction map -- to identify content  relations -- to capture state-dependent information  (inter)relations between 'states' or chunks media abstraction  VO = { (O_i,Q_i,C_i) | 1 <= i <= k } where  C_1,...,C_k -- mutually exclusive conditions  Q_1,...,Q_k -- queries  O_1,...,O_k -- objects virtual objects

user interface

augmented virtuality

information retrieval: issues retrieval -- indexing & success metrics video -- (temporal) description logic applications -- Create a VR that realizes a digital dossier for a work of a particular artist. A digital dossier represents the information that is available for a particular work of art, or a collection of works, of a particular artist. digital dossier

perspectives: information retrieval application(s) - digital dossier psychological - focus experimental - user interaction algorithmic - (information) access system - unified presentation space presentation - embodied agents search - semantic annotation commercial - future systems