Self Taught Learning : Transfer learning from unlabeled data Presented by: Shankar B S DMML Lab Rajat Raina et al, CS, Stanford ICML 2007

Semi-supervised learning, transfer learning etc Semi-supervised Learning Uses Unlabeled data; Assumes unlabeled data can be assigned supervised learning task’s class label Transfer Learning Transfer of knowledge from one supervised task to other; Requires labeled data from a different but related task Self-taught learning Uses unlabeled data; Does not require unlabeled data to have same generative distribution or class as the supervised learning task’s data

Self Taught Learning 1. Learn high level feature representation using unlabeled data Eg: Random unlabeled images also will contain basic visual patterns (like edges) that are similar to images (like that of elephant) which needs to be classified 2. Apply the representation to the labeled data and use it for classification

Sparse coding algorithm – Learning higher level representations Given unlabeled data Optimize the following where are the basis vectors and are the activations