1 Comparison of Discrimination Methods for the Classification of Tumors Using Gene Expression Data Presented by: Tun-Hsiang Yang

2 purpose of this paper Compare the performance of different discrimination methods Nearest Neighbor classifier Linear discriminant analysis Classification tree Machine learning approaches: bagging, boosting Investigate the use of prediction votes to assess the confidence of each prediction

3 Statistical problems: The identification of new/unknown tumor classes using gene expression profiles  Clustering analysis/unsupervised learning The classification of malignancies into known classes  Discriminant analysis/supervised learning The identification of marker genes that identified different tumor classes  Variable (Gene) selection

4 Datasets Gene expression data on p genes for n mRNA samples: n x p matrix X={x ij }, where x ij denotes the expression level of gene (variable) j in ith mRNA sample(observation) Response: k-dimensional vector Y={y i }, where y i denotes the class of observation i Lymphoma dataset (p=4682, n=81,k=3) Leukemia dataset (p=3571, n=72, k=3 or 2) NCI 60 dataset (p=5244, n=61, k=8)

5 Data preprocessing Imputation of missing data (KNN) Standardization of data (Euclidean distance) preliminary gene selection Lymphoma dataset (p=4682  p=50, n=81,k=3) Leukemia dataset (p=3571  p=40, n=72, k=3) NCI 60 dataset (p=5244  p=30, n=61, k=8)

6 Visual presentation of Leukemia dataset Correlation matrix (72x72) ordered by class Black: 0 correlation / Red: positive correlation / Green: negative correlation P=3571 p=40

7 Prediction Methods Supervised Learning Methods Machine learning approaches

8 Supervised Learning Methods Nearest Neighbor classifier(NN) Fisher Linear Discriminant Analysis (LDA) Weighted Gene Voting Classification trees (CART)

9 Nearest Neighbor The k-NN rule Find the k closest observations in the learning set Predict the class for each element in the test dataset by majority vote K is chosen by minimizing cross-validation error rate

10 Linear Discirminantion Analysis FLDA consists of finding linear functions a ’ x of the gene expression levels x=(x 1, …,x p ) with large ratio of between groups to within groups sum of squares Predicting the class of an observation by the class whose mean vector is closest to the discrimination variables

11 Maximum likelihood discriminant rules Predicts the class of an observation x as C(x)=argmaxkpr(x|y=k)

12 Weighted Gene Voting An observation x=(x 1,…x p ) is classified as 1 iff Prediction strength as the margin of victory(p9)

13 Classification tree Constructed by repeated splits of subsets (nodes) Each terminal subset is assigned a class label The size of the tree is determined by minimizing the cross validation error rate Three aspects to tree construction  the selection of the splits  the stopping criteria  the assignment of each terminal node to a class

14 Aggregated Predictors There are several ways to generate perturbed learning set: Bagging Boosting Convex Pseudo data (CPD)

15 Bagging Predictors are built for each sub-sample and aggregated by Majority voting with equal w b =1 Non-parametric bootstrap: drawing at random with replacement to form a perturbed learning sets of the same size as the original learning set By product: out of bag observations can be used to estimate misclassification rates of bagged predictors A prediction for each observation (xi, yi) is obtained by aggregating the classifiers in which (xi,yi) is out-of-bag

16 Bagging (cont.) Parametric bootstrap: Perturbed learning sets are generated according to a mixture of MVN distributions For each class k, the class sample mean and covariance matrix were taken as the estimates of distribution parameters Make sure at least one observation sampled from each class

17 Boosting The b th step of the boosting algorithm Get another learning set L b of the same size n L Build a classifier based on L b Run the learning set L let di=1 if the ith case is classified incorrectly di=0 otherwise Define  b =  P i d i and B b di =(1-  b )/  b Update by p i =p i B b di /  p i B b di Re-sampling probabilities are reset to equal if  b >=1/2 or  b =0

18 Prediction votes For aggregated classifiers, prediction votes assessing the strength of a prediction may be defined for each observation The prediction vote (PV) for an observation x

19 Study Design Randomly divide the dataset into a learning and test set (2:1 scheme) For each of N=150 runs: Select a subset of p genes from the learning set with the largest BSS/WSS Build the different predictors using the learning sets with p genes Apply the predictors to the observations in the test set to obtain test set error rates

20 Results Test set error rates: apply classifier build based on learning set to test set. Summarized by box-plot over runs Observation-wise error rates: for each observation, record the proportion of times it was classified incorrectly. Summarized by means of survival plots Variable selection: compare the effect of increasing or decreasing number of genes (variables)

21 Leukemia data, two classes

22 Leukemia data, three classes

23 Lymphoma data

24 Conclusions In the main comparison, NN and DLDA had the smallest error rates, while FLDA had the highest error rates Aggregation improved the performance of CART classifiers, the largest gains being with boosting and bagging with CPD For the lymphoma and leukemia datasets, increasing the number of variables to p=200 did not affect much the performance of the various classifiers. There was an improvement for the NCI 60 dataset. A more carefully selection of a small number of genes (p=10) improved the performance of FLDA dramatically