NATIONAL TECHNICAL UNIVERSITY OF ATHENS Image, Video And Multimedia Systems Laboratory Background
NATIONAL TECHNICAL UNIVERSITY OF ATHENS Automatic Image Annotation Input image Automatic segmentationDesired result
NATIONAL TECHNICAL UNIVERSITY OF ATHENS Tool for: Ground truth construction Semi-automatic image annotation Support of: Automatic segmentation Manual, user driven region merging Export of segmentation masks and textual annotation Image Annotator Tool
NATIONAL TECHNICAL UNIVERSITY OF ATHENS Visual Descriptor Ontology MPEG-7(XML Schema) defines visual descriptors by specifying their components In VDO (RDFS), descriptors are defined through relations with their components Descriptors related to higher – level concepts through inference rules Rules define spatio-temporal constraints
NATIONAL TECHNICAL UNIVERSITY OF ATHENS Knowledge- Assisted Analysis Tool Developed in collaboration with CERTH-ITI
NATIONAL TECHNICAL UNIVERSITY OF ATHENS KAA Results 0 Sea Person Sea Sand Sky 1 …
NATIONAL TECHNICAL UNIVERSITY OF ATHENS Approach: Graph-based representation of images Semantic vs Syntactic: regions are assigned fuzzy set of labels instead of low-level features Modification of traditional segmentation algorithms to operate on labelled regions Simultaneous image segmentation and region labeling Target: Solve oversegmentation problems Assign labels with confidence values to regions Link labels with concepts existing in ontologies Semantic Segmentation
NATIONAL TECHNICAL UNIVERSITY OF ATHENS Sea is oversegmented People have been incorrectly merged with the sand RSST segmentation Semantic RSST segmentation Region is assigned to a fuzzy set of labels: {rock/0.89,sand/0.46} Sea segments are merged correctly Semantic Segmentation
NATIONAL TECHNICAL UNIVERSITY OF ATHENS Visual Attention & Classification Generate visual saliency maps Detect foreground / background Select most representative regions for classification, based on saliency Lower Classification error OriginalSaliency Map Background Detection