Complex Networks for Representation and Characterization of Images For CS790g Project Bingdong Li 9/23/2009

Outline Background Motivation Current States (CS): – Representation – Characterization Using examples from – Backes, Casanova, and Brunos Approach using local information – Kim, Faloutsos and Heberts Approach using global information Comparison of Two Approaches Summary Questions and Comments

Background: Complex Network Source: cs790: complex network lecture

Background: Image Source: CS674 Image Processing Lecture

Background: Image Processing Source: CS674 Image Processing Lecture

Background: Image Representation Source: CS674 Image Processing Lecture

Outline Background Motivation

Belief: Computer vision is one of the most difficult problem remains, how can we represent and characterize image in the way of complex network so that we analysis it? For a given problem, if it can be described in the way of mathematics, it is half way to solve the problem.

Outline Background Motivation Current States (CS): – Representation – Characterization Using examples from – Backes, Casanova, and Brunos Approach using local information

CS: Backes Approach Construction of graph, – Vertices: points of shape boundary are modeled as fully connected network, – Weight: the Euclidean distance d – through a sequential thresholds T l (d< T l ), the fully connected network becomes a dynamic complex network, the topological features of the growth of the dynamic network are used as a shape descriptor (or signature)

CS: Backes Approach

Properties of the complex network – High clustering coefficient – The small world property

CS: Backes Approach

Dynamic evolution signature F: T T where T ini and T Q, respectively, the initial and final threshold

CS: Backes Approach Characterization – Degree descriptor k μ average degree, K k max degree

CS: Backes Approach Evolution by a threshold T=0.1,.15,.20

CS: Backes Approach Process of extraction of degree descriptor from an Image

CS: Backes Approach Advantage of Degree Descriptors – Rotation and scale inveriance – Noise tolerance – Robustness

CS: Backes Approach Representation of rotate invariance

CS: Backes Approach Representation of scale invariance

CS: Backes Approach Characterization – Joint Degree descriptor Is the concatenation of the entropy(H), energy(E), and average joint degree(P) at each instant threshold T

CS: Backes Approach Advantage of Joint Degree Descriptors – Rotation and scale inveriance – Noise tolerance – Robustness – Normalization of vertex is irrelevant because the joint degree concerns the probability distribution P(ki,k)i

CS: Backes Approach

Weakness of Backes Approach: – Initial and final threshold

Outline Background Motivation Current States (CS): – Representation – Characterization Using examples from – Backes, Casanova, and Brunos Approach using local information – Kim, Faloutsos and Heberts Approach using global information

CS: Kims Approach Construct Visual Similarity Network (VSN) – Vertices (V): features of from training images – Edges (E): link features that matched across images – Weights (W): consistence of correspondence with all other correspondences in matching image I a and I b VSN = (V, E, W)

CS: Kims Approach Construction of VSN – Vertices: can be any unit of local visual information. In this approach, features detected using Harris-Affine point detector and the SIFT descriptor

CS: Kims Approach Construction of VSN – Edges: established between features in different images. Spectral matching algorithm is used to each pair of image (I a, I b ) A new edge is established between feature a i and b j

CS: Kims Approach

Construction of VSN – Edge weights – M n*n is a spare weight matrix, M(a i, b j ) is the weight value

A small part of VSN

CS: Kims Approach Characterization – Ranking of information Remove noisy Measure the importance P is the PageRank vector

CS: Kims Approach Characterization – Structural similarity similar nodes are highly likely to exhibit similar link structures in the graph p.4 The similarity is computed by using link analysis algorithm

CS: Kims Approach Characterization Link analysis algorithm Given a VSN G, a node a i, the neighborhood subgraph Gai either pointed to a i or point to by a i M, the adjacency matrix of G a i.

CS: Kims Approach The left image is extracted features, the right image shows top20% high-ranked features

CS: Kims Approach Weakness of Kims Approach – Using threshold in computing edge weights – Mystery constant α =0.1 – Category partition to pre-determined K groups – The difference of objects appearance in the training data set is too big, make the conclusion weak

Outline Background Motivation Current States (CS): Comparison of Two Approaches

Comparison Backess Approach – Unsupervised approach – using local information – Dynamic complex network – More task on complex network, less work on image processing Kims Approach – Supervised approach – using global information – Static complex network – More work on image processing, less work on complex network Both using threshold, but Backes approach based on initial and final value,

Outline Background Motivation Current States (CS): Comparison of Two Approaches Summary

In both approaches using complex network for representation and characterization of image, – provide a unique way for object classification and analysis, – present better results than traditional and state- of-art methods, – demonstrate the potential of complex network analysis to computer vision.

Questions and Comments

Thanks