+ Get Rich and Cure Cancer with Support Vector Machines (Your Summer Projects)

Slides:

Advertisements

Similar presentations

Statistical Machine Learning- The Basic Approach and Current Research Challenges Shai Ben-David CS497 February, 2007.

Advertisements

CSC321: Introduction to Neural Networks and Machine Learning Lecture 24: Non-linear Support Vector Machines Geoffrey Hinton.

Lecture 9 Support Vector Machines

ECG Signal processing (2)

CSCE555 Bioinformatics Lecture 15 classification for microarray data Meeting: MW 4:00PM-5:15PM SWGN2A21 Instructor: Dr. Jianjun Hu Course page:

SVM - Support Vector Machines A new classification method for both linear and nonlinear data It uses a nonlinear mapping to transform the original training.

An Introduction of Support Vector Machine

Support Vector Machines

1 Lecture 5 Support Vector Machines Large-margin linear classifier Non-separable case The Kernel trick.

SVM—Support Vector Machines

Pattern Recognition and Machine Learning: Kernel Methods.

Support vector machine

Correlation Aware Feature Selection Annalisa Barla Cesare Furlanello Giuseppe Jurman Stefano Merler Silvano Paoli Berlin – 8/10/2005.

1cs542g-term High Dimensional Data  So far we’ve considered scalar data values f i (or interpolated/approximated each component of vector values.

x – independent variable (input)

University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Support Vector Machines.

An Introduction to Kernel-Based Learning Algorithms K.-R. Muller, S. Mika, G. Ratsch, K. Tsuda and B. Scholkopf Presented by: Joanna Giforos CS8980: Topics.

Classification: Support Vector Machine 10/10/07. What hyperplane (line) can separate the two classes of data?

Support Vector Machines Kernel Machines

Support Vector Machines

The Implicit Mapping into Feature Space. In order to learn non-linear relations with a linear machine, we need to select a set of nonlinear features.

SVM (Support Vector Machines) Base on statistical learning theory choose the kernel before the learning process.

Model Assessment and Selection Florian Markowetz & Rainer Spang Courses in Practical DNA Microarray Analysis.

Outline Separating Hyperplanes – Separable Case

Whole Genome Expression Analysis

Multiple Examples of tumor tissue (public data from Whitehead/MIT) SVM Classification of Multiple Tumor Types DNA Microarray Data Oracle Data Mining 78.25%

Integration II Prediction. Kernel-based data integration SVMs and the kernel “trick” Multiple-kernel learning Applications – Protein function prediction.

计算机学院计算感知 Support Vector Machines. 2 University of Texas at Austin Machine Learning Group 计算感知计算机学院 Perceptron Revisited: Linear Separators Binary classification.

Kernel Methods A B M Shawkat Ali 1 2 Data Mining ¤ DM or KDD (Knowledge Discovery in Databases) Extracting previously unknown, valid, and actionable.

Usman Roshan Machine Learning, CS 698

SVM Support Vector Machines Presented by: Anas Assiri Supervisor Prof. Dr. Mohamed Batouche.

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

Class Prediction and Discovery Using Gene Expression Data Donna K. Slonim, Pablo Tamayo, Jill P. Mesirov, Todd R. Golub, Eric S. Lander 발표자 : 이인희.

Predicting protein degradation rates Karen Page. The central dogma DNA RNA protein Transcription Translation The expression of genetic information stored.

Evolutionary Algorithms for Finding Optimal Gene Sets in Micro array Prediction. J. M. Deutsch Presented by: Shruti Sharma.

CS 478 – Tools for Machine Learning and Data Mining SVM.

Class 23, 2001 CBCl/AI MIT Bioinformatics Applications and Feature Selection for SVMs S. Mukherjee.

DNA Microarray Data Analysis using Artificial Neural Network Models. by Venkatanand Venkatachalapathy (‘Venkat’) ECE/ CS/ ME 539 Course Project.

Classification (slides adapted from Rob Schapire) Eran Segal Weizmann Institute.

University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Support Vector Machines.

Computational Approaches for Biomarker Discovery SubbaLakshmiswetha Patchamatla.

Guest lecture: Feature Selection Alan Qi Dec 2, 2004.

Support Vector Machine Debapriyo Majumdar Data Mining – Fall 2014 Indian Statistical Institute Kolkata November 3, 2014.

CZ5225: Modeling and Simulation in Biology Lecture 7, Microarray Class Classification by Machine learning Methods Prof. Chen Yu Zong Tel:

Support Vector Machines. Notation Assume a binary classification problem. –Instances are represented by vector x   n. –Training examples: x = (x 1,

Final Exam Review CS479/679 Pattern Recognition Dr. George Bebis 1.

Text Classification using Support Vector Machine Debapriyo Majumdar Information Retrieval – Spring 2015 Indian Statistical Institute Kolkata.

Feature Selction for SVMs J. Weston et al., NIPS 2000 오장민 (2000/01/04) Second reference : Mark A. Holl, Correlation-based Feature Selection for Machine.

Support Vector Machines Reading: Ben-Hur and Weston, “A User’s Guide to Support Vector Machines” (linked from class web page)

Kernel Regression Prof. Bennett Math Model of Learning and Discovery 1/28/05 Based on Chapter 2 of Shawe-Taylor and Cristianini.

Support Vector Machines Part 2. Recap of SVM algorithm Given training set S = {(x 1, y 1 ), (x 2, y 2 ),..., (x m, y m ) | (x i, y i )   n  {+1, -1}

1 CISC 841 Bioinformatics (Fall 2008) Review Session.

Day 17: Duality and Nonlinear SVM Kristin P. Bennett Mathematical Sciences Department Rensselaer Polytechnic Institute.

Support Vector Machines Reading: Textbook, Chapter 5 Ben-Hur and Weston, A User’s Guide to Support Vector Machines (linked from class web page)

Predictive Automatic Relevance Determination by Expectation Propagation Y. Qi T.P. Minka R.W. Picard Z. Ghahramani.

Support Vector Machines

PREDICT 422: Practical Machine Learning

Support Vector Machine

Mammogram Analysis – Tumor classification

Usman Roshan Machine Learning

Support Vector Machines

Pawan Lingras and Cory Butz

Feature selection Usman Roshan.

Pattern Recognition CS479/679 Pattern Recognition Dr. George Bebis

Usman Roshan Machine Learning

Support Vector Machines

Microarray Data Set The microarray data set we are dealing with is represented as a 2d numerical array.

SVMs for Document Ranking

MIRA, SVM, k-NN Lirong Xia. MIRA, SVM, k-NN Lirong Xia.

Presentation transcript:

+ Get Rich and Cure Cancer with Support Vector Machines (Your Summer Projects)

+ Kernel Trick

+ This is achieved with a polynomial kernel Feature map: Kernel:

+ Optimization of transformed problem: Only kernel matters Dual Lagrangian for transformed problem: Optimal weight vector: Thus, optimal hyperplane:

+ Kernel Trick We can choose the kernel without first defining a feature map. How to get a feature map from a kernel? Define i.e. map vectors in the original feature space to functions. Inner product on transformed space:

+ Get rich off of support vectors

+ Making 5-day forecasts of financial futures Given data on the returns for 5 days Predict the return on the next day To achieve this, we need to figure out which 5-day stretches tend to predict good returns on the 6 th day, and which predict not-so-good returns A training data set is used for this purpose

+ Making 5-day forecasts of financial futures Day 1Day 2Day 3Day 4Day 5 x 11 x 12 x 13 x 14 x 15 x 21 x 22 x 23 x 24 x 25 x 31 x 32 x 33 x 34 x 35 x 41 x 42 x 43 x 44 x 45 …………… Day 6 y1y1 y2y2 y3y3 y4y4 y5y5 5-dimensional feature spaceReturn on 6 th day is classifier for data Routine learns how to classify 5-day-return data points by working with a training data set for 500 days. Constructs a dividing hypersurface and uses it to decide what the 6 th -day return should be for new data points.

+ Good results – you can try it yourself! Complete with R code: with-support-vector-machines-svm/ with-support-vector-machines-svm/

+ Another example: gene expression in normal and cancerous tissue Gene = unit of heredity Human genome contains about 21,000 genes Public domain image from Wikipedia

+ Another example: gene expression in normal and cancerous tissue DNA transcribes to RNA which translates to proteins This is the process whereby the “genetic code” is made manifest as biological characteristics (genotype gives rise to phenotype) Wikimedia Commons image by Madeleine Price Ball

+ Big question: Which genes are responsible for which outcomes? In various tissues (e.g. tumor versus normal), which genes are active, hyperactive, and silent?  Can use DNA microarrays to measure gene expression levels.

+ DNA Microarray Source: National Human Genome Research Institute

+ Using support vector machines to determine which genes are important for cancer classification

+ Data Data points: Patients Features: Gene expression coefficients (activity level of a given gene) Feature space will have a huge number of dimensions! Need a way to reduce. Could examine all possible subspaces of feature space, but note that if dimension (N) of feature space represents thousands of genes, will mean that number of n-dimensional subspaces is  Too large for practical examination of each subspace

+ Generate ranking of features A ranking of features allows us to make a nested sequence of subspaces of feature space F and then determine the optimum subspace to work with One possibility for ranking: Work with each gene individually, get its correlation coefficient with the classifier (i.e. find correlation of gene expression level with classification of tissue into tumor v. normal or into two different types of cancer Note: ranking by correlation coefficient assumes all the features are independent of one another.

+ Generate ranking of features Another possible way to generate a ranking of features: sensitivity analysis. Have training data set, already classified into two classes (cancerous v. non, or cancer type 1 v. cancer type 2) Construct a cost function to estimate error in classification Sensitivity of cost function to removal of a feature measures the importance of that feature and allows the construction of a ranking.

+ Ranking by Support Vector Machines Recursive Feature Elimination Idea of how to use SVM to identify important features: Consider a cartoon scenario. x1x1 x2x2  Indicates that the x 1 direction is completely superfluous for classification.

+ Ranking by Support Vector Machines This suggests the following recursive algorithm for ranking features: Find weight vector, using all features Identify the least important feature to be the one with the smallest (in absolute value) component of the weight vector List that feature as least important and eliminate itfrom the data Iterate the procedure, with the least important feature thrown out. End result: Ranked list of features!

+ Try this at home! Data is available online! Classify two types of leukemia.