Applications of Supervised Learning in Bioinformatics Yen-Jen Oyang Dept. of Computer Science and Information Engineering.

Slides:

Advertisements

Similar presentations

The Software Infrastructure for Electronic Commerce Databases and Data Mining Lecture 4: An Introduction To Data Mining (II) Johannes Gehrke

Advertisements

Introduction to Non Parametric Statistics Kernel Density Estimation.

Original Figures for "Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring"

Instance-based Classification Examine the training samples each time a new query instance is given. The relationship between the new query instance and.

Machine learning continued Image source:

Lecture 3 Nonparametric density estimation and classification

Visual Recognition Tutorial

A Bayesian Approach to Joint Feature Selection and Classifier Design Balaji Krishnapuram, Alexander J. Hartemink, Lawrence Carin, Fellow, IEEE, and Mario.

Supervised learning Given training examples of inputs and corresponding outputs, produce the “correct” outputs for new inputs Two main scenarios: –Classification:

Pattern Recognition and Machine Learning

Lecture 20 Object recognition I

RBF Neural Networks x x1 Examples inside circles 1 and 2 are of class +, examples outside both circles are of class – What NN does.

Predictive Automatic Relevance Determination by Expectation Propagation Yuan (Alan) Qi Thomas P. Minka Rosalind W. Picard Zoubin Ghahramani.

Evaluating Hypotheses

Data Mining with Decision Trees Lutz Hamel Dept. of Computer Science and Statistics University of Rhode Island.

Kernel Methods Part 2 Bing Han June 26, Local Likelihood Logistic Regression.

What is Learning All about ?  Get knowledge of by study, experience, or being taught  Become aware by information or from observation  Commit to memory.

ML ALGORITHMS. Algorithm Types Classification (supervised) Given -> A set of classified examples “instances” Produce -> A way of classifying new examples.

Applications of Data Mining in Microarray Data Analysis Yen-Jen Oyang Dept. of Computer Science and Information Engineering.

Radial-Basis Function Networks

1 Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data Presented by: Tun-Hsiang Yang.

EE513 Audio Signals and Systems Statistical Pattern Classification Kevin D. Donohue Electrical and Computer Engineering University of Kentucky.

Classification of multiple cancer types by multicategory support vector machines using gene expression data.

Machine Learning1 Machine Learning: Summary Greg Grudic CSCI-4830.

Chapter 8 Discriminant Analysis. 8.1 Introduction  Classification is an important issue in multivariate analysis and data mining.  Classification: classifies.

Data Mining: Classification & Predication Hosam Al-Samarraie, PhD. Centre for Instructional Technology & Multimedia Universiti Sains Malaysia.

The Broad Institute of MIT and Harvard Classification / Prediction.

PATTERN RECOGNITION AND MACHINE LEARNING CHAPTER 3: LINEAR MODELS FOR REGRESSION.

Scenario 6 Distinguishing different types of leukemia to target treatment.

Overview of Supervised Learning Overview of Supervised Learning2 Outline Linear Regression and Nearest Neighbors method Statistical Decision.

Mixture Models, Monte Carlo, Bayesian Updating and Dynamic Models Mike West Computing Science and Statistics, Vol. 24, pp , 1993.

Computational Intelligence: Methods and Applications Lecture 12 Bayesian decisions: foundation of learning Włodzisław Duch Dept. of Informatics, UMK Google:

CS 782 – Machine Learning Lecture 4 Linear Models for Classification  Probabilistic generative models  Probabilistic discriminative models.

Jun-Won Suh Intelligent Electronic Systems Human and Systems Engineering Department of Electrical and Computer Engineering Speaker Verification System.

Data Classification with the Radial Basis Function Network Based on a Novel Kernel Density Estimation Algorithm Yen-Jen Oyang Department of Computer Science.

Meng-Han Yang September 9, 2009 A sequence-based hybrid predictor for identifying conformationally ambivalent regions in proteins.

Evolutionary Algorithms for Finding Optimal Gene Sets in Micro array Prediction. J. M. Deutsch Presented by: Shruti Sharma.

Bayesian Generalized Kernel Mixed Models Zhihua Zhang, Guang Dai and Michael I. Jordan JMLR 2011.

Consistency An estimator is a consistent estimator of θ, if , i.e., if

CS558 Project Local SVM Classification based on triangulation (on the plane) Glenn Fung.

Guest lecture: Feature Selection Alan Qi Dec 2, 2004.

Chapter1: Introduction Chapter2: Overview of Supervised Learning

Chapter 20 Classification and Estimation Classification – Feature selection Good feature have four characteristics: –Discrimination. Features.

Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring T.R. Golub et al., Science 286, 531 (1999)

Learning Kernel Classifiers Chap. 3.3 Relevance Vector Machine Chap. 3.4 Bayes Point Machines Summarized by Sang Kyun Lee 13 th May, 2005.

Machine Learning ICS 178 Instructor: Max Welling Supervised Learning.

Week 31 The Likelihood Function - Introduction Recall: a statistical model for some data is a set of distributions, one of which corresponds to the true.

METU Informatics Institute Min720 Pattern Classification with Bio-Medical Applications Part 9: Review.

CSE182 L14 Mass Spec Quantitation MS applications Microarray analysis.

Linear Classifiers Dept. Computer Science & Engineering, Shanghai Jiao Tong University.

Computational Intelligence: Methods and Applications Lecture 26 Density estimation, Expectation Maximization. Włodzisław Duch Dept. of Informatics, UMK.

Classification of tissues and samples 指導老師：藍清隆演講者：張許恩、王人禾.

Bias-Variance Analysis in Regression  True function is y = f(x) +  where  is normally distributed with zero mean and standard deviation .  Given a.

Classifiers!!! BCH364C/391L Systems Biology / Bioinformatics – Spring 2015 Edward Marcotte, Univ of Texas at Austin.

Predictive Automatic Relevance Determination by Expectation Propagation Y. Qi T.P. Minka R.W. Picard Z. Ghahramani.

Classifiers!!! BCH339N Systems Biology / Bioinformatics – Spring 2016

Ch3: Model Building through Regression

Alan Qi Thomas P. Minka Rosalind W. Picard Zoubin Ghahramani

Overview of Supervised Learning

REMOTE SENSING Multispectral Image Classification

Neuro-Computing Lecture 4 Radial Basis Function Network

Pattern Classification All materials in these slides were taken from Pattern Classification (2nd ed) by R. O. Duda, P. E. Hart and D. G. Stork, John.

Classification and Prediction

EE513 Audio Signals and Systems

Model generalization Brief summary of methods

Parametric Methods Berlin Chen, 2005 References:

Multivariate Methods Berlin Chen

Mathematical Foundations of BME

Linear Discrimination

Using Clustering to Make Prediction Intervals For Neural Networks

Presentation transcript:

Applications of Supervised Learning in Bioinformatics Yen-Jen Oyang Dept. of Computer Science and Information Engineering

Problem Definition of Supervised Learning (or Data Classification)   In a supervised learning problem, each sample is described by a set of feature values and each sample belongs to one of the predefined classes.   The goal is to derive a set of rules that predicts which class an incoming query sample should belong to, based on a given set of training samples. Supervised learning is also called data classification.

The Vector Space Model feature 1 feature 2 ‧‧‧‧‧ feature m sample 1 sample 2 sample n Class 2 Class 1 Class C

 In microarray data analysis, supervised learning algorithms have been employed to predict the class of an incoming query sample based on the existing samples with known classes. Application of Supervised Learning in Microarray Data Analysis

 For example, in the Leukemia data set, there are 72 samples and 7129 genes. 25 Acute Myeloid Leukemia(AML) samples. 25 Acute Myeloid Leukemia(AML) samples. 38 B-cell Acute Lymphoblastic Leukemia (B-cell ALL) samples. 38 B-cell Acute Lymphoblastic Leukemia (B-cell ALL) samples. 9 T-cell Acute Lymphoblastic Leukemia (T- cell ALL) samples. 9 T-cell Acute Lymphoblastic Leukemia (T- cell ALL) samples. Application of Supervised Learning in Microarray Data Analysis

Model of the Leukemia Dataset gene 1 gene 2 ‧‧‧‧‧‧‧‧ gene 7129 sample 1 sample 2 sample 72 Class 2 Class 1 Class 3

Training Process   From the mathematical point of view, the task of the supervised learning algorithm in the training stage is to identify curves that separate samples with different classes.   Prediction of the class of an incoming query sample is carried out by referring to the separating curves identified during the training stage.

query

The Basis of Kernel Regression

 Given a set of samples randomly taken from a probability distribution. We want to find a set of Gaussian functions and the corresponding weights to obtain an approximate probability density function, i.e. Problem Definition of Kernel Density Estimation (KDE) with Gaussian Kernels

 The KDE based learning algorithm constructs one approximate probability density function for each class of samples.  Prediction is conducted based on the following likelihood function: The KDE Based Predictor

The Decision Function of the RVKDE Based Predictor

  With the KDE based predictor, each training sample is associated with a kernel function, typically with a varying width.

An Example of Supervised Learning (Data Classification)   Given the data set shown on next slide, can we figure out a set of rules that predict the classes of samples?

Data Set DataClassDataClassDataClass （ 15,33 ） O （ 18,28 ） × （ 16,31 ） O （ 9,23 ） × （ 15,35 ） O （ 9,32 ） × （ 8,15 ） × （ 17,34 ） O （ 11,38 ） × （ 11,31 ） O （ 18,39 ） × （ 13,34 ） O （ 13,37 ） × （ 14,32 ） O （ 19,36 ） × （ 18,32 ） O （ 25,18 ） × （ 10,34 ） × （ 16,38 ） × （ 23,33 ） × （ 15,30 ） O （ 12,33 ） O （ 21,28 ） × （ 13,22 ） ×

Distribution of the Data Set 。。。。。。。。。。 × × × × × × × × × × × × × ×

Rule Based on Observation

Rule Generated by a Kernel Density Estimation Based Algorithm Let and If then prediction=“O”. Otherwise prediction=“X”.

(15,33)(11,31)(18,32)(12,33)(15,35)(17,34)(14,32)(16,31)(13,34)(15,30) (9,23)(8,15)(13,37)(16,38)(18,28)(18,39)(25,18)(23,33)(21,28)(9,32)(11,38)(19,36)(10,34)(13,22)