1. Generative Adversarial Nets ML Reading Group Xiao Lin Jul. 22 2015

2. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville and Y. Bengio. "Generative adversarial nets." NIPS, 2014.

3. Overview Problem: Generate adversarial examples Approach: Two player game Theory: Discriminative learning of distribution Potential application “Generative” Neural Nets Improving classification performance

4. Adversarial: a bit of background Visualizing HoG features “Why did my detector fail?” arXiv 2012 ICCV 2013 Visualizing CNNs arXiv 2013 ECCV 2014 CNNs can go wildly wrongarXiv 2013 Generative Adversarial NetsNIPS 2014 Google Deep Dream Facebook Eyescream CVPR 2015 ICLR 2015

5. A bit of background Visualizing HoG features “Why did my detector fail?” arXiv 2012 ICCV 2013 Visualizing CNNs arXiv 2013 ECCV 2014 CNNs can go wildly wrongarXiv 2013 Generative Adversarial NetsNIPS 2014 Google Deep Dream Facebook Eyescream CVPR 2015 ICLR 2015

6. A bit of background Visualizing HoG features “Why did my detector fail?” arXiv 2012 ICCV 2013 Visualizing CNNs arXiv 2013 ECCV 2014 CNNs can go wildly wrongarXiv 2013 Generative Adversarial NetsNIPS 2014 Google Deep Dream Facebook Eyescream CVPR 2015 ICLR 2015

7. A bit of background Visualizing HoG features “Why did my detector fail?” arXiv 2012 ICCV 2013 Visualizing CNNs arXiv 2013 ECCV 2014 CNNs can go wildly wrongarXiv 2013 Generative Adversarial NetsNIPS 2014 Google Deep Dream Facebook Eyescream CVPR 2015 ICLR 2015

8. A bit of background Visualizing HoG features “Why did my detector fail?” arXiv 2012 ICCV 2013 Visualizing CNNs arXiv 2013 ECCV 2014 CNNs can go wildly wrongarXiv 2013 Generative Adversarial NetsNIPS 2014 Google Deep Dream Facebook Eyescream CVPR 2015 ICLR 2015

9. A bit of background Visualizing HoG features “Why did my detector fail?” arXiv 2012 ICCV 2013 Visualizing CNNs arXiv 2013 ECCV 2014 CNNs can go wildly wrongarXiv 2013 Generative Adversarial NetsNIPS 2014 Google Deep Dream Facebook Eyescream CVPR 2015 ICLR 2015

10. A bit of background Visualizing HoG features “Why did my detector fail?” arXiv 2012 ICCV 2013 Visualizing CNNs arXiv 2013 ECCV 2014 CNNs can go wildly wrongarXiv 2013 Generative Adversarial NetsNIPS 2014 Google Deep Dream Facebook Eyescream CVPR 2015 ICLR 2015

11. Adversarial Framework Discriminative model “Police” Learns to determine whether a sample is from the model distribution of the generative model or the data distribution Generative model A team of “counterfeiters” trying to produce fake currency Try to fool the discriminative model with its model distribution Until the counterfeits are indistinguishable from the genuine articles Now the generative model generates a distribution indistinguishable from the data distribution

12. Related work (Table 2) Given examples. Learn model params Observe part of the example Infer the rest Generate examples according to model distribution Given example. Compute probability Design a model family with parameter θ

13. Flu Allergy Sinus Headache Nose=t Slide Credit: Dhruv Batra

14. Flu Allergy Sinus Headache Nose=t Slide Credit: Dhruv Batra

15. Generalized Denoising Autoencoder Y. Bengio, L. Yao, G. Alain and P. Vincent. "Generalized denoising auto-encoders as generative models." NIPS, 2013.

16. Approach: Objective

17. 0/1=D(x)x=G(z) x 1: From data or 0: fake ones from G z, aka random noise Fake x

18. Approach: Optimization D G Data

19. Approach: Optimization D G Data Optimize D Improve G Eventually

20. Approach: Optimization

21. Approach: Convergence Best D given G: right in the middle of data and G

22. Approach: Convergence Best G: G=data and V=-log4

23. Approach: Convergence DG Data Optimize D Improve G Eventually

24. Results

37. Problems D must be in sync with G Train G to optimal => all output collapse to 1 point Tuning parameters “Enough capacity” Multi-mode distributions

38. Future work