Download presentation

Presentation is loading. Please wait.

Published byCamden Buttry Modified about 1 year ago

1
Computer vision: models, learning and inference Chapter 18 Models for style and identity

2
Identity and Style 22Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Identity differs, but images similar Identity same, but images quite different

3
Structure 33Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Factor analysis review Subspace identity model Linear discriminant analysis Non-linear models Asymmetric bilinear model Symmetric bilinear model Applications

4
Factor analysis review 44Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Generative equation: Probabilistic form: Marginal density:

5
Factor analysis 55Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

6
Factor analysis review 66Computer vision: models, learning and inference. ©2011 Simon J.D. Prince E-Step: M-Step:

7
Disadvantages This description of the data does not account for identity For images which have the same style (e.g., pose, lighting), we expect faces which have the same identity to lie in a similar part of the space 7

8
Factor analysis vs. Identity model 88Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Each color is a different identity multiple images lie in similar part of subspace

9
Subspace identity model 99 Generative equation: Probabilistic form: Marginal density: J-th training example of i-th individual same linear combination different noise term between-individual variation within-individual variation

10
Subspace identity model 10 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

11
Factor analysis vs. subspace identity 11 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Factor analysisSubspace identity model

12
Learning subspace identity model 12 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince E-Step: M-Step:

13
Subspace identity model 13 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Different identities

14
Subspace identity model 14 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince The within-individual noise comprises whatever cannot be explained by the identity

15
Face verification! 15 Inference by comparing models, 0 : different identities 1: same identity, Posterior probability:

16
Inference by comparing models 16 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Model 1 – Faces match (identity shared): Model 2 – Faces don’t match (identities differ): Both models have standard form of factor analyzer face verication

17
Inference by comparing models 17 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Compute likelihood (e.g. for model zero) where Compute posterior probability using Bayes rule

18
Face Recognition Tasks PROBE … GALLERY ? 1. CLOSED SET FACE IDENTIFICATION … GALLERY PROBE ? NO MATCH 2. OPEN SET FACE IDENTIFICATION PROBE ? NO MATCH 3. FACE VERIFICATION 4. FACE CLUSTERING ? 18Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

19
Inference by comparing models 19 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

20
Limitations of identity subspace model 20 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

21
Structure 21 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Factor analysis review Subspace identity model Linear discriminant analysis Non-linear models Asymmetric bilinear model Symmetric bilinear model Applications

22
Probabilistic linear discriminant analysis 22 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Generative equation: Probabilistic form: the style of this face, which differs for each instance

23
Probabilistic linear discriminant analysis 23 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

24
Learning 24 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince E-Step – write out all images of same person as system of equations – Has standard form of factor analyzer joint posterior probability distribution over all of the hidden variables M-Step – write equation for each individual data point – Has standard form of factor analyzer – Use standard EM equation

25
Probabilistic linear discriminant analyis 25 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

26
Inference 26 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Model 1 – Faces match (identity shared): Model 2 – Faces don’t match (identities differ): Both models have standard form of factor analyzer Compute likelihood in standard way

27
Example results (XM2VTS database) 27 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Building more complex models is worth the time and effort

28
Structure 28 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Factor analysis review Subspace identity model Linear discriminant analysis Non-linear models Asymmetric bilinear model Symmetric bilinear model Applications

29
Non-linear models (mixture) 29 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Mixture model can describe non- linear manifold. Introduce variable c i which represents which cluster belongs to To be the same identity, must also belong to the same cluster

30
Non-linear models (kernel) 30 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Pass hidden variable through non-linear function f[ ]. Leads to kernelized algorithm !!!!!! it is no longer possible to marginalize over the hidden variables !!!!!! it is no longer possible exactly to compare model likelihoods directly in the inference Approximate

31
Structure 31 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Factor analysis review Subspace identity model Linear discriminant analysis Non-linear models Asymmetric bilinear model Symmetric bilinear model Applications

32
How to solve the situation when the style of the data may change considerably 32 any given frontal face has more in common visually with other non- matching frontal faces than it does with a matching probe face. Model Styles differently

33
Asymmetric bilinear model 33 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Introduce style variable s ij indicates conditions in which data was observed Example: lighting, pose, expression face recognition Asymmetric bilinear model Introduce style variable s indicates conditions in which data was observed Example: lighting, pose, expression face recognition

34
Asymmetric bilinear model 34 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

35
Asymmetric bilinear model 35 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Generative equation: Probabilistic form: Marginal density:

36
Learning 36 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince E-Step: M-Step:

37
Asymmetric bilinear model 37 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

38
Inference – inferring style 38 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Likelihood of style Prior over style Compute posterior over style using Bayes’ rule

39
Inference – inferring identity 39 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Likelihood of identity Prior over identity Compute posterior over identity using Bayes’ rule

40
Inference – comparing identities 40 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Model 1 – Faces match (identity shared): Model 2 – Faces dont match (identities differ): Both models have standard form of factor analyzer Compute likelihood in standard way, combine with prior in Bayes rule

41
Inference – Style translation 41 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Compute distribution over identity Generate in new style

42
Structure 42 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Factor analysis review Subspace identity model Linear discriminant analysis Non-linear models Asymmetric bilinear model Symmetric bilinear model Applications

43
Symmetric bilinear model 43 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Generative equation: Probabilistic form: Mean can also depend on style...

44
Symmetric bilinear model 44 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

45
Multilinear models 45 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Extension of symmetric bilinear model to more than two factors e.g.,

46
Structure 46 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Factor analysis review Subspace identity model Linear discriminant analysis Non-linear models Asymmetric bilinear model Symmetric bilinear model Applications

47
Face recognition 47 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

48
Synthesizing animation 48 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince

49
Discussion 49 Computer vision: models, learning and inference. ©2011 Simon J.D. Prince Generative models Explain data as combination of identity and style factors In identity recognition, we build models where identity was same or different Other forms of inference such as style translation also possible

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google