Testing the Manifold Hypothesis Hari Narayanan University of Washington In collaboration with Charles Fefferman and Sanjoy Mitter Princeton MIT.

Slides:

Advertisements

Similar presentations

This algorithm is used for dimension reduction. Input: a set of vectors {Xn є }, and dimension d,d

Advertisements

Statistical Machine Learning- The Basic Approach and Current Research Challenges Shai Ben-David CS497 February, 2007.

Informational Complexity Notion of Reduction for Concept Classes Shai Ben-David Cornell University, and Technion Joint work with Ami Litman Technion.

Shortest Vector In A Lattice is NP-Hard to approximate

Distance Preserving Embeddings of Low-Dimensional Manifolds Nakul Verma UC San Diego.

. The sample complexity of learning Bayesian Networks Or Zuk*^, Shiri Margel* and Eytan Domany* *Dept. of Physics of Complex Systems Weizmann Inst. of.

. On the Number of Samples Needed to Learn the Correct Structure of a Bayesian Network Or Zuk, Shiri Margel and Eytan Domany Dept. of Physics of Complex.

A general agnostic active learning algorithm

A Discussion On.  The Paper by Dmitry Berenson and Siddhartha S Srinivasa here proves the probabilistic completeness of RRT based algorithms when planning.

1 Heat flow and a faster Algorithm to Compute the Surface Area of a Convex Body Hariharan Narayanan, University of Chicago Joint work with Mikhail Belkin,

Nonlinear Unsupervised Feature Learning How Local Similarities Lead to Global Coding Amirreza Shaban.

The loss function, the normal equation,

By : L. Pour Mohammad Bagher Author : Vladimir N. Vapnik

Support Vector Machines (SVMs) Chapter 5 (Duda et al.)

Northwestern University Winter 2007 Machine Learning EECS Machine Learning Lecture 13: Computational Learning Theory.

Measuring Model Complexity (Textbook, Sections ) CS 410/510 Thurs. April 27, 2007 Given two hypotheses (models) that correctly classify the training.

The Multiple Regression Model Hill et al Chapter 7.

Computational Learning Theory

1 Motion Planning for Multiple Robots B. Aronov, M. de Berg, A. Frank van der Stappen, P. Svestka, J. Vleugels Presented by Tim Bretl.

Approximate Nearest Neighbors and the Fast Johnson-Lindenstrauss Transform Nir Ailon, Bernard Chazelle (Princeton University)

Dasgupta, Kalai & Monteleoni COLT 2005 Analysis of perceptron-based active learning Sanjoy Dasgupta, UCSD Adam Tauman Kalai, TTI-Chicago Claire Monteleoni,

Scalable Training of Mixture Models via Coresets Daniel Feldman Matthew Faulkner Andreas Krause MIT.

Bayesian belief networks 2. PCA and ICA

Algorithm Evaluation and Error Analysis class 7 Multiple View Geometry Comp Marc Pollefeys.

Computational Learning Theory

Statistical Learning Theory: Classification Using Support Vector Machines John DiMona Some slides based on Prof Andrew Moore at CMU:

Clustering & Dimensionality Reduction 273A Intro Machine Learning.

Correctness of Gossip-Based Membership under Message Loss Maxim Gurevich, Idit Keidar Technion.

Diffusion Maps and Spectral Clustering

Efficient Regression in Metric Spaces via Approximate Lipschitz Extension Lee-Ad GottliebAriel University Aryeh KontorovichBen-Gurion University Robert.

Graph-based consensus clustering for class discovery from gene expression data Zhiwen Yum, Hau-San Wong and Hongqiang Wang Bioinformatics, 2007.

Presented By Wanchen Lu 2/25/2013

1 Patch Complexity, Finite Pixel Correlations and Optimal Denoising Anat Levin, Boaz Nadler, Fredo Durand and Bill Freeman Weizmann Institute, MIT CSAIL.

1 A(n) (extremely) brief/crude introduction to minimum description length principle jdu

1 Reproducing Kernel Exponential Manifold: Estimation and Geometry Kenji Fukumizu Institute of Statistical Mathematics, ROIS Graduate University of Advanced.

Cs: compressed sensing

A C B Small Model Middle Model Large Model Figure 1 Parameter Space The set of parameters of a small model is an analytic set with singularities. Rank.

Ran El-Yaniv and Dmitry Pechyony Technion – Israel Institute of Technology, Haifa, Israel Transductive Rademacher Complexity and its Applications.

ECE 8443 – Pattern Recognition Objectives: Error Bounds Complexity Theory PAC Learning PAC Bound Margin Classifiers Resources: D.M.: Simplified PAC-Bayes.

1 CS546: Machine Learning and Natural Language Discriminative vs Generative Classifiers This lecture is based on (Ng & Jordan, 02) paper and some slides.

Introduction to Evolutionary Algorithms Session 4 Jim Smith University of the West of England, UK May/June 2012.

T. Poggio, R. Rifkin, S. Mukherjee, P. Niyogi: General Conditions for Predictivity in Learning Theory Michael Pfeiffer

A Fast and Accurate Tracking Algorithm of the Left Ventricle in 3D Echocardiography A Fast and Accurate Tracking Algorithm of the Left Ventricle in 3D.

Empirical Efficiency Maximization: Locally Efficient Covariate Adjustment in Randomized Experiments Daniel B. Rubin Joint work with Mark J. van der Laan.

Clustering and Testing in High- Dimensional Data M. Radavičius, G. Jakimauskas, J. Sušinskas (Institute of Mathematics and Informatics, Vilnius, Lithuania)

ANOVA Assumptions 1.Normality (sampling distribution of the mean) 2.Homogeneity of Variance 3.Independence of Observations - reason for random assignment.

Visual Categorization With Bags of Keypoints Original Authors: G. Csurka, C.R. Dance, L. Fan, J. Willamowski, C. Bray ECCV Workshop on Statistical Learning.

The Supervised Network Self- Organizing Map for Classification of Large Data Sets Authors: Papadimitriou et al, Advisor: Dr. Hsu Graduate: Yu-Wei Su.

INTRODUCTION TO Machine Learning 3rd Edition

Vaida Bartkutė, Leonidas Sakalauskas

Structural Return Maximization for Reinforcement Learning Josh Joseph Alborz Geramifard Javier Velez Jonathan How Nicholas Roy 1.

Lecture 07: Dealing with Big Data

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Text Categorization With Support Vector Machines: Learning With Many Relevant Features By Thornsten Joachims Presented By Meghneel Gore.

Comparison of Tarry’s Algorithm and Awerbuch’s Algorithm CS 6/73201 Advanced Operating System Presentation by: Sanjitkumar Patel.

Goal of Learning Algorithms  The early learning algorithms were designed to find such an accurate fit to the data.  A classifier is said to be consistent.

Generalization Error of pac Model  Let be a set of training examples chosen i.i.d. according to  Treat the generalization error as a r.v. depending on.

Journal of Computational and Applied Mathematics Volume 253, 1 December 2013, Pages 14–25 Reporter : Zong-Dian Lee A hybrid quantum inspired harmony search.

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

1 C.A.L. Bailer-Jones. Machine Learning. Data exploration and dimensionality reduction Machine learning, pattern recognition and statistical data modelling.

Nearly optimal classification for semimetrics

Introduction to Hypothesis Test – Part 2

Intrinsic Data Geometry from a Training Set

Generalization and adaptivity in stochastic convex optimization

Regularized risk minimization

Hariharan Narayanan, University of Chicago Joint work with

Computational Learning Theory

Distributed Systems CS

Computational Learning Theory

Approximation and Generalization in Neural Networks

Presentation transcript:

Testing the Manifold Hypothesis Hari Narayanan University of Washington In collaboration with Charles Fefferman and Sanjoy Mitter Princeton MIT

Manifold learning and manifold hypothesis [Kambhatla-Leen’93, Tannenbaum et al’00, Roweis-Saul’00, Belkin-Niyogi’03, Donoho-Grimes’04]

When is the Manifold Hypothesis true?

Reach of a submanifold of R n Large reach Small reach reach

Low dimensional manifolds with bounded volume and reach

Testing the Manifold Hypothesis

Sample Complexity of testing the manifold hypothesis [

Algorithmic question

Sample complexity of testing the Manifold Hypothesis

Empirical Risk Minimization

Fitting manifolds TexPoint Display

Reduction to k-means

Proving a Uniform bound for k-means

Fat-shattering dimension

Bound on sample complexity

VC dimension

Random projection

Bound on sample complexity

Fitting manifolds

Algorithmic question

Outline

(3) Generating a smooth vector bundle

Outline

(4) Generating a putative manifold

Outline

(5) Bundle map

Outline

Concluding Remarks An algorithm for testing the manifold hypothesis. Future directions: (a)Make practical and test on real data (b)Improve precision in the reach – get rid of controlled constants depending on d. (c)Better algorithms under distributional assumptions

Thank You!