2 Computer VisionImage processing is a precursor to Computer Vision – making a computer understand and interpret what’s in an image or video.3D shapeRoboticsRecognitionTrackingCategorisation / RetrievalSegmentation
3 Innate Face Recognition Ability Face recognition almost instantaneousHighly invariant to pose, scale, rotation, lighting changesCan handle partial occasions and changes in ageAnd we can do all this for faces of several thousand individualsWho is this?
4 Fully Automated Face Recognition Face DetectionRecognitionImage / VideoIdentityFace DetectionColourMotionTemplateFiduciary points
5 Face Detection Detect face and facial landmarks Australian National UniversityDetect face and facial landmarksRegister (align) face using an image transform
6 Face Recognition approaches Geometric featuresRecognition possible at low resolution and at high noise levels because geometrical features such as nose width and eye separation are used Automated extraction of facial geometric features is very hardTemplate matchingExtract facial regions (matrix of pixels) and compare with that of known individuals Need templates for different face poses/views Very high dimensional data
7 High Dimensional Correlated Data Images as a high dimensional vectorA typical image used for image processing will be 512x512= dimension vector!(Registered) face images are highly correlatedI1,2I … .Face imagesI1,1Image space (high dimensional space of all possible images)
8 Transform Face images to a ‘Face Space’ Face space (low dimensional space of face images)U1U2U3Face imagesThe basis vectors (Ui) of face space are the Principal Components of face images.These Principal Components (PCs) correspond to the directions of greatest variation of the face dataset.The first PC corresponds to the direction (and scale) of greatest variation, the 2nd PC corresponds to the orthogonal direction with the second greatest variation and so on …Embedding a face image vector into face space is done by calculating the vector’s projection onto the Principal Components.Mean face imagef = U * (I - A)Change dimensionality of f by changing that of UFace imageEmbedded faceEigenfaces
9 Principal Component Analysis (Karhunen-Loeve transform)PCA – find the Principal Components of a datasetFinding the Eigenvectors and eigenvalues of the covariance matrix (Eigen decomposition)The covariance matrix contains the relationships (correlations) between the variables of the vectors. Variances along the diagonals and covariance between each pair of variables in other positions
10 PCA (continued) Covariance matrix for images For images will run out of memory because for a dataset of say 512x512 images X*XT will be a x matrix. If it has double precision (64bit) values C will be 512GB!!!Therefore for image we instead compute the reduced covariance matrixIf we have n datapoints c will be nxn. As n is normally much smaller than the dimensionality of the images the matrix c is computableFind the Eigenvectors and eigenvalues of c, solve the following linear equationsYou can get Matlab to solve equations to give the eigenvectors and eigenvaluesEigenvalues (degree of variation)Reduced covariance matrixEigenvectors (Principal Compoments!!)>> [V,lamda]=eig(c);
11 PCA (continued) Eigenvectors (Eigenfaces) of covariance matirx Another way to compute PCA – Singular Value Decomposition methodSVD decomposes an mxn matrix X to a productX = U * S * VT Don’t need to compute covariance matrixYou can get Matlab to solve these equations as well to get U, S and VDiagonal matrix with eigenvalues along diagonalLeft singular vectors (eigenvectors of X*XT)Right singular vectors (eigenvectors of XT*X)>> [U,S,V]=svd(X);Use eigen decomposition with the reduced covariance matrix if you run out of memory with svd (seems to work upto 100x100 images)
12 Eigenfaces 1 to 9 Average face (data not mean subtracted!) High eigenvalue eigenfaces seem to capture lighting and other global intensity changes
13 Lower eigenvalue eigenfaces are better for recognition Eigenfaces 10 to 18
14 Transform into ‘Face Space’ Projectionf = U * (I - A)U1U2U3Transform known faces to face spaceFace space
15 Reconstruction F = UT * U * (I - A) Transform back to image space Mean subtracted imageTransform to face spaceVery high compression!Face space with 28 eigenvectors100x100x8bits = bytes compress to 28*32 bits=112 bytesOnly compress facesCan know whether image contains a face by calculating reconstruction errorFace detection method!
16 ClassifiersClassify an unknown face into one of the known face classesImportant conceptsInter-class (between-class) variance Intra-class (within-class) variance Need a metric to measure distance for class membership?
17 MetricA non-negative function g(x,y) describing the "distance" between neighbouring points for a given set and satisfies(1) g(x,y)+g(y,z)≥g(x,z) (triangle inequality)(2) g(x,y)=g(y,x) (symmetry)(3) g(x,x)=0, as well as g(x,y)= x=y
18 Euclidean distance (~300BC) Mahalanobis distance (1936)Superior to Euclidean distance because it takes distribution of the points (correlations) into accountThe distance between two N dimensional points scaled by the statistical variation in each component of the point.Decision surface
19 Mahalanobis classifier Need many representative examples from each class to compute covariance matrix
20 Recap Image processing precursor to Computer Vision Face Detection Images are high dimensional correlated dataPrincipal Component AnalysisEigen decompositionSingular Value DecompositionEigenfaces for recognition, compression, face detectionClassifiers and Metrics
21 QuestionsWill performance improve if we used orientated edge responses of the face image for recognition? Why?A popular approach uses orientated responses only from certain specific points on the face image (say edge of nose, sides of eyes). What are the advantages/disadvantages of this approach?PCA maximises the variance between vectors embedded in the eigenspace. What does LDA (Linear Discriminant Analysis) do?