1. Generalizing Backpropagation to Include Sparse Coding David M. Bradley (dbradley@cs.cmu.edu)dbradley@cs.cmu.edu and Drew Bagnell Robotics Institute Carnegie Mellon University

2. Outline Discuss value of modular and deep gradient based systems, especially in robotics Introduce a new and useful family of modules Properties of new family –Online training with non-gaussian priors E.g. encourage sparsity, multi-task weight sharing –Modules internally solve continuous optimization problems Captures interesting nonlinear effects such as inhibition that involve coupled outputs Sparse Approximation –Modules can be jointly optimized by a generalization of backpropagation

3. Deep Modular Learning systems Efficiently represent complex functions –Particularly efficient for closely related tasks Recently shown to be powerful learning machines –Greedy layer-wise training improves initialization Greedy module-wise training is useful for designing complex systems –Design and Initialize modules independently –Jointly optimize the final system with backpropagation Gradient methods allow the incorporation of diverse data sources and losses Y. LeCun, L. Bottou, Y. Bengio and P. Haffner: Gradient-Based Learning Applied to Document Recognition, 1998 Y. Bengio, P. Lamblin, H. Larochelle, “Greedy layer-wise training of deep networks.”, NIPS 2007 G. Hinton, S. Osindero, and Y. Teh, “A fast learning algorithm for deep belief networks.”, Neural Computation 2006

4. Mobile Robot Perception RGB Camera NIR Camera Ladar Lots of unlabeled data Hard to define traditional supervised learning data Target task is defined by weakly-labeled structured output data

5. Perception Problem: Scene labeling Motion Planner Cost for each 2-D cell

6. Goal System Labeled 3-D points Camera Laser Labelme Webcam Data Labelme Observed Wheel Heights IMU data Object Classification Cost Lighting Variance Cost Human-Driven Example Paths Proprioception Prediction Cost Ground Plane Estimator Max Margin Planner Classification Cost Point Classifier Data Flow Gradient Motion plans

7. New Modules Modules that are important in this system require two new abilities –Induce new priors on weights –Allow modules to solve internal optimization problems

8. Standard Backpropagation assumes L2 prior Gradient descent with convex loss functions: Small steps with early stopping imply L 2 regularization –Minimizes a regret bound by solving the optimization: –Which bounds the true regret M. Zinkevich, “Online Convex Programming and Generalized Infinitesimal Gradient Ascent”, ‘03

9. KL-divergence –Useful if many features are irrelevant –Approximately solved with exponentiated gradient descent multi-task priors (encourage sharing between related tasks) Alternate Priors Bradley and Bagnell 2008 Argyriou and Evgeniou, “Multi-task Feature Learning”, NIPS 07

10. L 2 Backpropagation Module (M 2 ) + Loss Function c a Module (M 3 )Module (M 1 ) b Input Loss Function

11. With KL prior modules Module (M 2 ) + Loss Function c a Module (M 3 )Module (M 1 ) b Input Loss Function

12. General Mirror Descent Module (M 2 ) + Loss Function c a Module (M 3 )Module (M 1 ) b Input Loss Function

13. New Modules Modules that are important in this system require two new abilities –Induce new priors on weights –Allow modules to solve internal optimization problems interesting nonlinear effects such as inhibition that involve coupled outputs Sparse Approximation

14. Inhibition Input Basis

15. Inhibition Input Basis Projection

16. Inhibition Input Basis KL-regularized Optimization

17. Sparse Approximation Assumes the input is a sparse combination of elements, plus observation noise –Many possible elements –Only a few present in any particular example True for many real-world signals Many applications –Compression (JPEG), Sensing (MRI), Machine Learning Produces effects observed in biology –V1 receptive fields, Inhibition Tropp et al. “Algorithms For Simultaneous Sparse Approximation”, 2005 Olhausen and Field, “Sparse Coding of Natural Images Produces Localized, Oriented, Bandpass Receptive Fields”, Nature 95 Doi and Lewicki, “Sparse Coding of natural images using an overcomplete set of limited capacity units”, NIPS 04 Raina et al. “Self Taught Learning: Transfer Learning from unlabeled data”, ICML ’07

18. Sparse Approximation Semantic meaning is sparse Visual Representation is Sparse (JPEG)

19. MNIST Digits Dataset 60,000 28x28 pixel handwritten digits –10,000 reserved for a validation set Separate 10,000 digit test set

20. Sparse Approximation Basis Coefficients (w 1 ) Input Reconstruction Error (Cross Entropy) r 1 =Bw Error gradient

21. Sparse Approximation KL-regularized Coefficients on a KL-regularized Basis Input Output

22. Sparse Coding Basis Coefficients (w (i) ) Input Reconstruction Error (Cross Entropy) r=Bw (i) Training Examples Minimize over W and B

23. Optimization Modules L1 Regularized Sparse Approximation L1 Regularized Sparse Coding Lee et al. “Efficient Sparse Coding Algorithms”, NIPS '06 Reconstruction Loss Regularization Term Not Convex Convex

24. KL-regularized Sparse Approximation Unnormalized KL Reconstruction Loss Since this is continuous and differentiable, at the minimum we have: Differentiating both sides with respect to B, and solving for the k th row we get:

25. Preliminary Results KL sparse coding with backpropagation KL improves classification performance Backpropagation further improves performance L1 sparse coding

26. Main Points Modular, gradient based systems are an important design tool for large scale learning systems Need new tools to include a family of modules that have important properties Presented a generalized backpropagation technique that –Allow priors that encourage, e.g. sparsity (KL prior): uses mirror descent to modify weights –Uses implicit differentiation to compute gradients through modules (e.g. sparse approximation) that internally solve optimization Demonstrated work-in-progress on building deep, sparse coders using generalized backpropagation

27. Acknowledgements The Authors would like to thank the UPI team, especially Cris Dima, David Silver, and Carl Wellington DARPA and the Army Research Office for supporting this work through the UPI program and the NDSEG fellowship