Presentation is loading. Please wait.

Presentation is loading. Please wait.

Regression “A new perspective on freedom” TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAA A A A A AAA A A.

Similar presentations


Presentation on theme: "Regression “A new perspective on freedom” TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAA A A A A AAA A A."— Presentation transcript:

1 Regression “A new perspective on freedom” TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAA A A A A AAA A A

2 Classification

3 ? CatDog

4 Cleanliness Size

5 ? $$$$$$$$$$

6 Regression

7 $ $$ $$$ $$$$ Price Top speed x y

8 Regression Data Goal: given, predict i.e. find a prediction function

9 Nearest neighbor

10 Nearest neighbor To predict x –Find the data point x i closest to x –Choose y = y i + No training – Finding closest point can be expensive – Overfitting

11 Kernel Regression To predict X –Give data point x i weight –Normalize weights –Let e.g.

12 Kernel Regression [matlab demo]

13 Kernel Regression + No training + Smooth prediction – Slower than nearest neighbor – Must choose width of

14 Linear regression

15 Temperature [start Matlab demo lecture2.m] Given examples Predict given a new point Temperature

16 Temperature Linear regression Prediction

17 Linear Regression Error or “residual” Prediction Observation Sum squared error

18 Linear Regression n d Solve the system (it’s better not to invert the matrix)

19 Minimize the sum squared error Sum squared error Linear equation Linear system

20 LMS Algorithm (Least Mean Squares) where Online algorithm

21 Beyond lines and planes everything is the same with still linear in

22 Linear Regression [summary] n d Let For example Let Minimize by solving Given examples Predict

23 Probabilistic interpretation Likelihood

24 Overfitting [Matlab demo] Degree 15 polynomial

25 Ridge Regression (Regularization) Effect of regularization (degree 19) with “small” Minimize Solve Let

26 Probabilistic interpretation Likelihood Prior Posterior

27 Locally Linear Regression

28 [source: Global temperature increase

29 Locally Linear Regression To predict X –Give data point x i weight –Let e.g.

30 Locally Linear Regression + Good even at the boundary (more important in high dimension) – Solve linear system for each new prediction – Must choose width of To minimize Solve Predict where

31 [source: Locally Linear Regression Gaussian kernel 180

32 [source: Locally Linear Regression Laplacian kernel 180

33 L1 Regression

34 Sensitivity to outliers High weight given to outliers Influence function

35 L 1 Regression Linear program Influence function

36 Spline Regression Regression on each interval

37 Spline Regression With equality constraints

38 Spline Regression With L 1 cost

39 To learn more The Elements of Statistical Learning, Hastie, Tibshirani, Friedman, Springer


Download ppt "Regression “A new perspective on freedom” TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAA A A A A AAA A A."

Similar presentations


Ads by Google