1. Statistical Modeling and Learning in Vision --- cortex-like generative models Ying Nian Wu UCLA Department of Statistics JSM, August 2010

2. http://www.stat.ucla.edu/~ywu/ActiveBasis Matlab/C code, Data Outline Primary visual cortex (V1) Modeling and learning in V1 Layered hierarchical models

3. Source: Scientific American, 1999 Visual cortex: layered hierarchical architecture V1: primary visual cortex simple cells complex cells bottom-up/top-down

4. Simple V1 cells Daugman, 1985 Gabor wavelets: localized sine and cosine waves Transation, rotation, dilation of the above function

5. image pixels V1 simple cells respond to edges

6. Complex V1 cells Riesenhuber and Poggio,1999 Image pixels V1 simple cells V1 complex cells Local max Local sum Larger receptive field Less sensitive to deformation

7. Independent Component Analysis Bell and Sejnowski, 1996 Laplacian/Cauchy

8. Hyvarinen, 2000

9. Sparse coding Olshausen and Field, 1996 Laplacian/Cauchy/mixture Gaussians

10. Inference: sparsification, non-linear lasso/basis pursuit/matching pursuit mode and uncertainty of p(C|I) explaining-away, lateral inhibition Sparse coding / variable selection Learning: A dictionary of representational elements (regressors)

11. Olshausen and Field, 1996

12. Restricted Boltzmann Machine Hinton, Osindero and Teh, 2006 P(I|C) P(C|I): factorized no-explaining away hidden, binary visible

13. Energy-based model Teh, Welling, Osindero and Hinton, 2003 Features, no explaining-away Maximum entropy with marginals Exponential family with sufficient stat Zhu, Wu, and Mumford, 1997 Wu, Liu, and Zhu, 2000 Markov random field/Gibbs distribution

14. Zhu, Wu, and Mumford, 1997 Wu, Liu, and Zhu, 2000

15. Source: Scientific American, 1999 Visual cortex: layered hierarchical architecture bottom-up/top-down What is beyond V1? Hierarchical model?

16. Hierchical ICA/Energy-based model? Larger features Must introduce nonlinearities Purely bottom-up

17. P(I,C) = P(C)P(I|C) P(C)  P(J,C) I C I J Discriminative correction by back-propagation Unfolding, untying, re-learning Hierarchical RBM Hinton, Osindero and Teh, 2006

18. Hierarchical sparse coding Attributed sparse coding elements transformation group topological neighborhood system Layer above : further coding of the attributes of selected sparse coding elements

19. Hierarchical sparse coding Active basis Wu, Si, Fleming, Zhu, 2007 Residual  generalization

20. Shared matching pursuit 1.Local maximization in step 1: complex cells, Riesenhuber and Poggio,1999 2.Arg-max in step 2: inferring hidden variables 3.Explaining-away in step 3: lateral inhibition Wu, Si, Fleming, Zhu, 2007

21. Active basis Two different scales

22. Putting multiple scales together

23. More elements added Residual images

25. Statistical modeling orthogonal Conditional independence of coefficients Exponential family model Strong edges in background Wu, Si, Gong, Zhu, 2010

27. ……

29. Detection by sum-max maps Wu, Si, Gong, Zhu, 2010

30. Image pixels V1 simple cells V1 complex cells Local max Local sum Complex V1 cells Riesenhuber and Poggio,1999 Larger receptive field Less sensitive to deformation

31. SUM-MAX maps (bottom-up/top-down) Local maximization: complex cells Riesenhuber and Poggio,1999 Gabor wavelets: simple cells Olshausen and Field, 1996 SUM2 operator: what “cell”?

32. Bottom-up detection Top-down sketching SUM1 MAX1 SUM2 arg MAX1 Sparse selective connection as a result of learning Explaining-away in learning but not in inference Bottom-up scoring and top-down sketching

33. Adjusting Active Basis Model by L2 Regularized Logistic Regression By Ruixun Zhang L2 regularized logistic regression  re-estimated lambda’s Conditional on: (1) selected basis elements (2) inferred hidden variables (1) and (2)  generative learning Exponential family model, q(I) negatives  Logistic regression Generative learning without negative examples Discriminative correcting of conditional independence assumption (with hugely reduced dimensionality)

34. Learning from non-aligned training images

36. EM mixture

37. MNIST

38. Active bases as part-templates Split bike template to detect and sketch tandem bike

39. Is there an edge here? Is there an edge nearby? Is there a wheel here? Is there a wheel nearby? Is there a tandem bike here? Soft scoring instead of hard decision

40. Learning part templates or visual words

41. Shape script model Shape motifs: elementary geometric shapes Si and Wu, 2010

43. Layers of attributed sparse coding elements