1. Laboratory for Social & Neural Systems Research (SNS) PATTERN RECOGNITION AND MACHINE LEARNING Institute of Empirical Research in Economics (IEW) 22-09-20101 Computational Neuroeconomics and Neuroscience

2. Course schedule 22-09-2010 Computational Neuroeconomics and Neuroscience 2 Date Topic Chapter 13-10-2010 Density Estimation, Bayesian Inference 2 Adrian Etter, Marco Piccirelli, Giuseppe Ugazio 20-10-2010 Linear Models for Regression 3 Susanne Leiberg, Grit Hein 27-10-2010Linear Models for Classification 4 Friederike Meyer, Chaohui Guo 03-11-2010 Kernel Methods I: Gaussian Processes 6 Kate Lomakina 10-11-2010 Kernel Methods II: SVM and RVM 7 Christoph Mathys, Morteza Moazami 17-11-2010 Probabilistic Graphical Models 8 Justin Chumbley Date Topic Chapter 13-10-2010 Density Estimation, Bayesian Inference 2 Adrian Etter, Marco Piccirelli, Giuseppe Ugazio 20-10-2010 Linear Models for Regression 3 Susanne Leiberg, Grit Hein 27-10-2010Linear Models for Classification 4 Friederike Meyer, Chaohui Guo 03-11-2010 Kernel Methods I: Gaussian Processes 6 Kate Lomakina 10-11-2010 Kernel Methods II: SVM and RVM 7 Christoph Mathys, Morteza Moazami 17-11-2010 Probabilistic Graphical Models 8 Justin Chumbley

3. Course schedule 22-09-2010 Computational Neuroeconomics and Neuroscience 3 Date Topic Chapter 24-11-2010 Mixture Models and EM 9 Bastiaan Oud, Tony Williams 01-12-2010 Approximate Inference I: Deterministic Approximations 10 Falk Lieder 08-12-2010 Approximate Inference II: Stochastic Approximations 11 Kay Brodersen 15-12-2010 Inference on Continuous Latent Variables: PCA, Probabilistic PCA, ICA 12 Lars Kasper 22-12-2010 Sequential Data: Hidden Markov Models, Linear Dynamical Systems 13 Chris Burke, Yosuke Morishima Date Topic Chapter 24-11-2010 Mixture Models and EM 9 Bastiaan Oud, Tony Williams 01-12-2010 Approximate Inference I: Deterministic Approximations 10 Falk Lieder 08-12-2010 Approximate Inference II: Stochastic Approximations 11 Kay Brodersen 15-12-2010 Inference on Continuous Latent Variables: PCA, Probabilistic PCA, ICA 12 Lars Kasper 22-12-2010 Sequential Data: Hidden Markov Models, Linear Dynamical Systems 13 Chris Burke, Yosuke Morishima

4. Sandra Iglesias Laboratory for Social & Neural Systems Research (SNS) CHAPTER 1: PROBABILITY, DECISION, AND INFORMATION THEORY Institute of Empirical Research in Economics (IEW) 22-09-20104 Computational Neuroeconomics and Neuroscience

5. Outline -Introduction -Probability Theory -Probability Rules -Bayes’Theorem -Gaussian Distribution -Decision Theory -Information Theory 22-09-2010 Computational Neuroeconomics and Neuroscience 5

6. Pattern recognition 22-09-2010 Computational Neuroeconomics and Neuroscience 6 computer algorithms  automatic discovery of regularities in data use of these regularities to take actions such as classifying the data into different categories classify data (patterns) based either on -a priori knowledge or -statistical information extracted from the patterns

7. Machine learning 22-09-2010 Computational Neuroeconomics and Neuroscience 7 'How can we program systems to automatically learn and to improve with experience?' the machine is programmed to learn from an incomplete set of examples (training set) the core objective of a learner is to generalize from its experience

8. Polynomial Curve Fitting 22-09-20108 Computational Neuroeconomics and Neuroscience

9. Sum-of-Squares Error Function 22-09-20109 Computational Neuroeconomics and Neuroscience 

10. Plots of polynomials 22-09-201010 Computational Neuroeconomics and Neuroscience

11. Over-fitting Root-Mean-Square (RMS) Error: 22-09-201011 Computational Neuroeconomics and Neuroscience

12. Regularization Penalize large coefficient values 22-09-201012 Computational Neuroeconomics and Neuroscience M = 9

13. Regularization: vs. 22-09-201013 Computational Neuroeconomics and Neuroscience M = 9

14. Outline -Introduction -Probability Theory -Decision Theory -Information Theory 22-09-2010 Computational Neuroeconomics and Neuroscience 14

15. Probability Theory Uncertainty Probability theory:  consistent framework for the quantification and manipulation of uncertainty 22-09-2010 Computational Neuroeconomics and Neuroscience 15 Noise on measurements Finite size of data sets

16. Probability Theory Marginal Probability Conditional Probability Joint Probability 22-09-201016 Computational Neuroeconomics and Neuroscience

17. Probability Theory 22-09-201017 Computational Neuroeconomics and Neuroscience i = 1, …,M j = 1, …,L n ij : number of trials in which X = x i and Y = y j c i : number of trials in which X = x i irrespective of the value of Y r j : number of trials in which X = x i irrespective of the value of Y

18. Probability Theory Marginal Probability Conditional Probability Joint Probability 22-09-201018 Computational Neuroeconomics and Neuroscience

19. Probability Theory Marginal Probability Conditional Probability Joint Probability 22-09-201019 Computational Neuroeconomics and Neuroscience

20. Probability Theory Marginal Probability Conditional Probability Joint Probability 22-09-201020 Computational Neuroeconomics and Neuroscience

21. Probability Theory Sum Rule 22-09-201021 Computational Neuroeconomics and Neuroscience

22. Probability Theory Product Rule 22-09-201022 Computational Neuroeconomics and Neuroscience

23. The Rules of Probability Sum Rule Product Rule 22-09-201023 Computational Neuroeconomics and Neuroscience

24. Bayes’ Theorem 22-09-201024 Computational Neuroeconomics and Neuroscience T. Bayes (1702-1761) P.-S. Laplace (1749-1827) p(X,Y) = p(Y,X)

25. Bayes’ Theorem posterior  likelihood × prior 22-09-201025 Computational Neuroeconomics and Neuroscience T. Bayes (1702-1761) P.-S. Laplace (1749-1827) Polynomial curve fitting problem

26. Probability Densities 22-09-201026 Computational Neuroeconomics and Neuroscience

27. Expectations Expectation for a discrete distribution: 22-09-201027 Computational Neuroeconomics and Neuroscience Expectation for a continuous distribution: Expectation of f(x) is the average value of some function f(x) under a probability distribution p(x)

28. The Gaussian Distribution 22-09-201028 Computational Neuroeconomics and Neuroscience

29. Gaussian Parameter Estimation Likelihood function 22-09-201029 Computational Neuroeconomics and Neuroscience

30. Maximum (Log) Likelihood 22-09-201030 Computational Neuroeconomics and Neuroscience

31. Curve Fitting Re-visited 22-09-201031 Computational Neuroeconomics and Neuroscience

32. Maximum Likelihood Determine by minimizing sum-of-squares error,. 22-09-201032 Computational Neuroeconomics and Neuroscience

33. Outline -Introduction -Probability Theory -Decision Theory -Information Theory 22-09-2010 Computational Neuroeconomics and Neuroscience 33

34. Decision Theory Used with probability theory to make optimal decisions Input vector x, target vector t Regression: t is continuous Classification: t will consist of class labels Summary of uncertainty associated is given by Inference problem: is to obtain from data Decision problem: make specific prediction for value of t and take specific actions based on t Inference stepDecision step Determine either or. For given x, determine optimal t. 22-09-201034 Computational Neuroeconomics and Neuroscience

35. Medical Diagnosis Problem X-ray image of patient Whether patient has cancer or not Input vector x: set of pixel intensities Output variable t: whether cancer or not C1 = cancer; C2 = no cancer General inference problem is to determine which gives most complete description of situation In the end we need to decide whether to give treatment or not  Decision theory helps do this 22-09-201035 Computational Neuroeconomics and Neuroscience

36. Bayes’ Decision How do probabilities play a role in making a decision? Given input x and classes C k using Bayes’ theorem Quantities in Bayes theorem can be obtained from p(x,Ck) either by marginalizing or conditioning with respect to the appropriate variable 22-09-2010 Computational Neuroeconomics and Neuroscience 36

37. Minimum Expected Loss Example: classify medical images as ‘cancer’ or ‘normal’ Unequal importance of mistakes Loss or Cost Function given by Loss Matrix Utility is negative of Loss Minimize Average Loss Decision Truth 22-09-201037 Computational Neuroeconomics and Neuroscience Regions are chosen to minimize

38. Why Separate Inference and Decision? Classification problem  broken into two separate stages: – Inference stage: training data is used to learn a model for – Decision stage: posterior probabilities used to make optimal class assignments Three distinct approaches to solving decision problems 1. Generative models: 2. Discriminative models 3. Discriminant functions 22-09-201038 Computational Neuroeconomics and Neuroscience

39. Generative models 1. solve inference problem of determining class-conditional densities for each class separately and the prior probabilities 2. use Bayes’ theorem to determine posterior probabilities 3. use decision theory to determine class membership 22-09-2010 Computational Neuroeconomics and Neuroscience 39

40. Discriminative models 1. solve inference problem to determine posterior class probabilities 2. Use decision theory to determine class membership 22-09-2010 Computational Neuroeconomics and Neuroscience 40

41. Discriminant functions 1. Find a function f(x) that maps each input x directly to a class label e.g. two-class problem: f (·) is binary valued f =0 represents C1, f =1 represents C2  Probabilities play no role 22-09-2010 Computational Neuroeconomics and Neuroscience 41

42. Decision Theory for Regression Inference step Determine Decision step For given x, make optimal prediction, y(x), for t Loss function: 22-09-201042 Computational Neuroeconomics and Neuroscience

43. Outline -Introduction -Probability Theory -Decision Theory -Information Theory 22-09-2010 Computational Neuroeconomics and Neuroscience 43

44. Information theory Quantification of information Degree of surprise: highly improbable  a lot of information highly probable  less information certain  no information Based on probability theory Most important quantity: entropy 22-09-2010 Computational Neuroeconomics and Neuroscience 44

45. Entropy 22-09-201045 Computational Neuroeconomics and Neuroscience H[x] p(x) 0 Entropy is the average amount of information expected, weighted with the probability of the random variable  quantifies the uncertainty involved when we encounter this random variable

46. The Kullback-Leibler Divergence 22-09-201046 Computational Neuroeconomics and Neuroscience Non-symmetric measure of the difference between two probability distributions Also called relative entropy

47. Mutual Information 22-09-201047 Computational Neuroeconomics and Neuroscience Two sets of variables: x and y If independent: If not independent:

48. Mutual Information 22-09-201048 Computational Neuroeconomics and Neuroscience Mutual information  mutual dependence  shared information  related to the conditional entropy

49. Course schedule 22-09-2010 Computational Neuroeconomics and Neuroscience 49 Date Topic Chapter 22-09-2010 Probability, Decision, and Information Theory 1 13-10-2010 Density Estimation, Bayesian Inference 2 20-10-2010 Linear Models for Regression 3 27-10-2010Linear Models for Classification 4 03-11-2010 Kernel Methods I: Gaussian Processes 6 10-11-2010 Kernel Methods II: SVM and RVM 7 17-11-2010 Probabilistic Graphical Models 8 24-11-2010 Mixture Models and EM 9 01-12-2010 Approximate Inference I: Deterministic Approximations 10 08-12-2010 Approximate Inference II: Stochastic Approximations 11 15-12-2010 Inference on Continuous Latent Variables: PCA, Probabilistic PCA, ICA 12 22-12-2010 Sequential Data: Hidden Markov Models, Linear Dynamical Systems 13 Date Topic Chapter 22-09-2010 Probability, Decision, and Information Theory 1 13-10-2010 Density Estimation, Bayesian Inference 2 20-10-2010 Linear Models for Regression 3 27-10-2010Linear Models for Classification 4 03-11-2010 Kernel Methods I: Gaussian Processes 6 10-11-2010 Kernel Methods II: SVM and RVM 7 17-11-2010 Probabilistic Graphical Models 8 24-11-2010 Mixture Models and EM 9 01-12-2010 Approximate Inference I: Deterministic Approximations 10 08-12-2010 Approximate Inference II: Stochastic Approximations 11 15-12-2010 Inference on Continuous Latent Variables: PCA, Probabilistic PCA, ICA 12 22-12-2010 Sequential Data: Hidden Markov Models, Linear Dynamical Systems 13