Daphne’s Approximate Group of Students

Outline Linear Regression Unregularized L2 Regularized What is a GP? Prediction with a GP Relationship to SVM Implications What does this mean?

Linear Regression Predicting Y given X Y = wtx + n w_ml = argmax y[m+1] = w_mltx[m+1]

L2 Regularized Lin Reg L2 Regularized (Gaussian Prior on w) Y = wtx + n w ~ N(0,S) w_map = argmax blah + ||w||^2

What is a random process? It’s a prior over functions

What is a Gaussian Process? It’s a prior over functions that generalized a Gaussian Random Vector Prior over Y(x) ~ N(0,I)

Alternate Definition The thing with Euler’s equation

This is weird Not used to thinking of prior over Ys Or are we? We ARE used to thining about prior over w What prior over y does this induce

Math P(w) -> P(Y) Wow! This became a Gaussian Process!

Prediction with a GP Predict y*[m+1] given y[1]…y[m] We get a covariance = error bars Wow! This prediction is the same as w_map but we get error bars!

Generalize that shit - Covariance Functions Note that we have a thing here that is defined by C(x1,x2) which can be kernelized Has to be pos semidefinite Is a kernel function

Relationship to SVM

Example

How do we reconcile these views? Does this change anything?