Outline Multilinear Analysis M. A. O. Vasilescu and D. Terzopoulos, “Multilinear Independent Components Analysis,” CVPR 2005

Motivations Natural images are generated by the interaction of multiple factors related to scene structure, illumination, and imaging November 10, 2018 Computer Vision

Motivations November 10, 2018 Computer Vision

Separating Styles from Content November 10, 2018 Computer Vision

Separating Styles from Content November 10, 2018 Computer Vision

Separating Styles from Content November 10, 2018 Computer Vision

Bilinear Model Suppose that we want to represent both style s and content c with vectors of parameters Let ysc denote a K-dimensional observation vector in style s and content class c November 10, 2018 Computer Vision

How to Learn the Bases and Coefficients Learning is done by minimizing total squared error over the entire training set Again the solution is given by a SVD decomposition November 10, 2018 Computer Vision

How to Learn the Bases and Coefficients For symmetric models, we also minimize the total squared error over the training set This is minimized using an iterative procedure November 10, 2018 Computer Vision

Classification Example November 10, 2018 Computer Vision

Classification Example When the model is trained on 10 faces and tested on the remaining one 1-NN achieves an accuracy of 53.9%±4.3% The proposed method achieves 73.9%±6.7% when parameters are determined automatically It achieves 80.6%±7.5% when optimal parameter values are used November 10, 2018 Computer Vision

Extrapolation November 10, 2018 Computer Vision

Extrapolation November 10, 2018 Computer Vision

Translation November 10, 2018 Computer Vision

Multilinear Analysis Tensors are multilinear mappings over a set of vector spaces An order N tensor is given by Mode-n vectors are given by They result in a mode-n flattening November 10, 2018 Computer Vision

Multilinear Analysis November 10, 2018 Computer Vision

Multilinear Analysis November 10, 2018 Computer Vision

Multilinear Analysis Mode-n product of a tensor and matrix is given by A tensor of n factors November 10, 2018 Computer Vision

Mode-n SVD November 10, 2018 Computer Vision

TensorFaces Weizmann face image database 28 male subjects photographed in 5 viewpoints, 4 illuminations, and 3 expressions Images are aligned to a reference face using a global rigid optical flow and then downsized by a factor of 3 and cropped, yielding a total of 7943 pixels per image within the elliptical cropping window November 10, 2018 Computer Vision

Weizmann Face Dataset November 10, 2018 Computer Vision

TensorFaces The dataset is then represented by a tensor of order 5 November 10, 2018 Computer Vision

TensorFaces November 10, 2018 Computer Vision

TensorFaces November 10, 2018 Computer Vision

TensorFaces November 10, 2018 Computer Vision

TensorFaces The bases for TensorFaces are given by November 10, 2018 Computer Vision

Face Recognition First experiment TensorFaces are trained on an ensemble comprising images of 23 people, captured from 3 viewpoints (0,±34 degrees), with 4 illumination conditions (center, left, right, left + right) It is tested on other images in this 23 person dataset acquired from 2 different viewpoints (±17 degrees) under the same 4 illumination conditions In this test scenario, the PCA method recognized the person correctly 61% of the time while TensorFaces recognized the person correctly 80% of the time. November 10, 2018 Computer Vision

Face Recognition The second experiment In a second experiment, TensorFaces is trained on images of 23 people 5 viewpoints (0,±17,±34 degrees), 3 illuminations (center light, left light, right light) Tested on the 4th illumination (left + right) PCA yielded a poor recognition rate of 27% while Tensorfaces achieved a recognition rate of 88% November 10, 2018 Computer Vision

Dimension Reduction November 10, 2018 Computer Vision

N-Mode Orthogonal Iteration Algorithm November 10, 2018 Computer Vision

Multilinear ICA Similar to MPCA, MICA is done using an n-mode ICA algorithm November 10, 2018 Computer Vision

Different ICA Architectures Architecture I ICA computes independent components of DT Architecture II ICA computes independent components of D November 10, 2018 Computer Vision

MICA November 10, 2018 Computer Vision

MICA November 10, 2018 Computer Vision

MICA November 10, 2018 Computer Vision

Recognition Experiment November 10, 2018 Computer Vision

TensorTextures Similar idea can be used for image synthesis November 10, 2018 Computer Vision

TensorTextures November 10, 2018 Computer Vision

TensorTextures November 10, 2018 Computer Vision

TensorTextures November 10, 2018 Computer Vision

TensorTextures November 10, 2018 Computer Vision

TensorTextures November 10, 2018 Computer Vision

TensorTextures November 10, 2018 Computer Vision

Comparison November 10, 2018 Computer Vision