1. Deep Boltzman machines Paper by : R. Salakhutdinov, G. Hinton Presenter : Roozbeh Gholizadeh

2. Outline  Problems with some other methods!  Energy based models  Boltzmann machine  Restricted Boltzmann machine  Deep Boltzmann machine

3. Problems with other methods!  Supervised learning need labeled data.  Amount of information restricted by labels!  Finding and knowing abnormalities before ever seeing them such as some conditions in a nuclear power plant.  So Instead of learning p(label | data) learn p(data)

4. Energy Based Models

5. Boltzmann machine  Markov random field (MRF) with hidden variables.  Undirected edges representing dependency. Weights can be assigned.

6.  Conditional distributions over hidden and visible units:

7. Learning process  Parameters update:  Exact maximum likelihood learning is intractable.  Use Gibbs sampling to approximate.  Run 2 separate Markov chains to approximate them.

8. Boltzman Machine Learning Procedure

9. Restricted Boltzmann Machine

10. Stochastic approximation procedure (SAP)

11. Why go deep?

12.  Deep architectures are representationally efficient, fewer computational units for same function.  Allow for showing a hierarchy.  Non-local generalization  Easier to monitor what is being learn and guide the machine.

13. Deep Boltzmann Machine  Undirected connection between all layers.  Conditional distributions over visible and hidden:”

14. Pretraining (greedy layerwise)

15. MNIST dataset

16. NORB Misclassification Error rate: DBM : 10.8%, SVM:11.6%, logistic regression: 22.5%, K-nearest neighbors : 18.4%

17. Thank you!