Bilinear Classifiers for Visual Recognition Hamed Pirsiavash, Deva Ramanan, Charless Fowlkes Computational Vision Lab, University of California Irvine, CA USA Main ideas: -Treat image features and templates as matrices or tensors rather than simply “vectorizing” them -Learn a low rank template by performing alternating minimization using standard linear SVM solver -Share a subset of parameters (subspace) across different problems (transfer learning) Linear SVM: For a given set of training pairs: Experiment 1: Pedestrian detection on INRIA motion dataset f X n ; y g m i n W L ( ) = 1 2 T r + C P a x ; ¡ y X n s = 1 4 £ 6 , f 8 d 5 Regularizer Constraints Was Linear: f ( X ) = T r W > Detection function: Now it is Bilinear: f ( X ) = T r W s > It is biconvex so we use coordinate descent with off-the-shelf linear SVM solver in the loop Learning: Change of basis: ~ W f = A 1 2 T , X n ¡ s m i n ~ W f L ( ; s ) = 1 2 T r + C P a x ¡ y X Experiment 2: Human action recognition on UCF sports dataset A window on sample image Sample template Advantages : -Rank constraint provides natural regularization. -10X runtime speedup with no loss in performance. (Fewer number of convolutions in detection) -Learn shared subspaces across multiple classes or training sets to allow for transfer learning. (e.g., share subspace between “human detector” and “cat detector”) Features Linear (0.518) (Not always feasible) PCA on features (0.444) Bilinear (0.648) 1 x = 2 6 4 . 3 7 5 n y f £ 1 Features Feature vector Vectorizing (traditional) n f n y n f Joint minimization: Our method X = 2 6 4 : . 3 7 5 n y x £ f m i n ~ W f P = 1 L ( ; s ) Feature matrix n x PCA to initialize Experiment 3: Sharing subspace on UCF sports dataset (Using only two training samples for each class) W = s T f Rank restriction: d · m i n ( s ; f ) Train Train Train where n s = y x W 1 s W 2 s Iteration 1 Refined at Iteration 2 W m s n f d Average classification rate 2 6 4 : . 3 7 5 n s £ f = d h i n f Train Train Train Train Coordinate decent iteration 1 Coordinate decent iteration 2 Independent bilinear (C=0.01) 0.222 0.289 Joint bilinear (C=0.1) 0.269 0.356 W f W 1 f W 2 f W m f Related work : -Tenenbaum & Freeman’s bilinear model (Neural Computation’00) -Wolf et al’s low-rank SVM paper (CVPR’07) -Ando and Zhang's transfer learning paper (JMLR'05) W T f W W s Low dim template Subspace W = s T f