1. Compression of CNNs Mooyeol Baek Xiangyu Zhang, Jianhua Zou, Xiang Ming, Kaiming He, Jian Sun: Efficient and Accurate Approximations of Nonlinear Convolutional Networks. Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo, Taelim Choi, Lu Yang, Dongjun Shin: Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications.

2. Motivation It’s practically important to accelerate the test-time computation of CNNs. CNN filters can be approximately decomposed into a series of smaller filters by row-rank approximation.

3. Approaches Zhang et al. Kim et al. m m c n d k k c m m c 1 1 c m m c’ n n d’ 1 1 n d n n 1 1 k k c’ n n

4. Efficient and Accurate Approximations of Nonlinear Convolutional Networks. Xiangyu Zhang, Jianhua Zou, Xiang Ming, Kaiming He, Jian Sun

5. Contribution Low-rank approximation minimizing the reconstruction error of nonlinear responses. Asymmetric reconstruction to reduce the accumulated error of multiple approximated layers. Empirical observation of PCA energy to select proper rank.

6. Low-rank Approximation m m c n d k k c n n d’ 1 1 n

7. Low-rank Approximation Relaxation

8. Asymmetric Reconstruction Uses non-approximate responses to reduce the accumulated error of multiple approximated layers. OriginalApproximated

9. Rank Selection

10. Experiments [1] Linear vs. Nonlinear

11. Experiments [2] Symmetric vs. Asymmetric

12. Experiments [3] Rank selection

13. Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications. Yong-Deok Kim, Eunhyeok Park, Sungjoo Yoo, Taelim Choi, Lu Yang, Dongjun Shin

14. Contribution One-shot whole network compression scheme which consists of simple three steps: 1.Rank selection (Variational Bayesian matrix factorization) 2.Low-rank tensor decomposition (Tucker decomposition) 3.Fine-tuning.

15. Tensor Decomposition Tucker decomposition

16. Tensor Decomposition Zhang et al. Kim et al. m m c n d k k c m m c 1 1 c m m c’ n n d’ 1 1 n d n n 1 1 k k c’ n n

17. Fine-tuning

18. Experiments [1]

19. Experiments [2]