1. Special Topics In Scientific Computing
Pattern Recognition & Data Mining
Lect2: Probability and One Example of Model Fitting

2. Ref:

3. Probability Theory
Apples and Oranges

4. Probability Theory Marginal Probability Conditional Probability
Joint Probability

5. Probability Theory
Sum Rule
Product Rule

6. The Rules of Probability
Sum Rule
Product Rule

7. Bayes’ Theorem
posterior  likelihood × prior

8. Probability Densities

9. Transformed Densities

10. Expectations Conditional Expectation (discrete)
Approximate Expectation
(discrete and continuous)

11. Variances and Covariances

12. The Gaussian Distribution

13. Gaussian Mean and Variance

14. The Multivariate Gaussian

15. Gaussian Parameter Estimation
Likelihood function

16. Maximum (Log) Likelihood

17. Model Fitting

18. Polynomial Curve Fitting

19. Sum-of-Squares Error Function

20. 0th Order Polynomial

21. 1st Order Polynomial

22. 3rd Order Polynomial

23. 9th Order Polynomial

24. Over-fitting
Root-Mean-Square (RMS) Error:

25. Polynomial Coefficients

26. Data Set Size:
9th Order Polynomial

27. Data Set Size:
9th Order Polynomial

28. Regularization
Penalize large coefficient values

29. Regularization:

30. Regularization:

31. Regularization: vs.

32. Polynomial Coefficients

33. Curve fitting: Statistical View

34. Maximum Likelihood
Determine by minimizing sum-of-squares error,

35. Predictive Distribution

36. Predictive Distribution

37. MAP: A Step towards Bayes
Determine by minimizing regularized sum-of-squares error,

38. Bayesian Curve Fitting

39. Bayesian Predictive Distribution