Presentation on theme: "Protein Fold Recognition with Relevance Vector Machines Patrick Fernie COMS 6772 Advanced Machine Learning 12/05/2005."— Presentation transcript:
Protein Fold Recognition with Relevance Vector Machines Patrick Fernie COMS 6772 Advanced Machine Learning 12/05/2005
Relevance Vector Machine A Bayesian treatment of a generalized linear model Yields a formulation similar to that of a Support Vector Machine Hyperparameters Instead of Margin/Costs
Relevance Vector Machine SVMRVM Hard Binary Outputs or Point Estimates Probabilistic Outputs Requires a Mercer Kernel Can Use Arbitrary Kernel Must Determine Suitable Cost and Insensitivity Values “Nuisance” Values Automatically Determined Sparse (USPS ~2500) Sparser USPS (~316!)
Relevance Vector Machine Can’t Use qp() Must solve iteratively (Sequential Minimization Optimization) As we iterate, many hyperparameters (α i ) values become arbitrarily large; allows pruning.
Relevance Vector Machine Faster Algorithm (Still not SVM fast) Minimizes Number of Active Kernel Functions to Reduce Computation Time Analytic Approach to Pruning/Adding Basis Functions
Protein Fold Recognition Protein Structure Families Many Fold Families Not Necessarily Directly Related by Protein Sequence
Protein Fold Recognition Prime Situation for Machine Learning Techniques! NN, SVM, etc. Large Number of Classes
Protein Fold Recognition 27 Fold Families Train Many 2-Class Classifiers One vs. Others – False Positives One vs. Others – False Positives Unique One vs. Others – Like One vs. Others, with Another Round of Training Unique One vs. Others – Like One vs. Others, with Another Round of Training All vs. All – Requires a Lot of Classifiers! All vs. All – Requires a Lot of Classifiers!
RVMs & Protein Folds Why RVMs? Probabilistic Outputs Probabilistic Outputs Sparsity (useful only in assessment) Sparsity (useful only in assessment) True Multiclass Prediction True Multiclass Prediction No Need to Find “Nuisance” Parameters No Need to Find “Nuisance” Parameters
Issues/Future Work Optimize RVM Classification Implement True Multiclass Reduced Greediness and Sequential Convergence Optimization Novel Kernels?
References M. Tipping, “The Relevance Vector Machine”, http://www.relevancevector.com M. Tipping, “Sparse Bayesian Learning and the Relevance Vector Machine”, JMLR, 2001 1:211- 244. M. Tipping and A. Faul, “Fast Marginal Likelihood Maximisation for Sparse Bayesian Models”, http://www.relevancevector.com C. Ding and I. Dubchak, “Multi-class Protein Fold Recognition Using Support Vector Machines”, http://www.kernel-machines.org
Your consent to our cookies if you continue to use this website.