Unsupervised medical image classification by combining case-based classifiers Thien Anh Dinh1, Tomi Silander1, Bolan Su1, Tianxia Gong Boon Chuan Pang2, Tchoyoson Lim2, Cheng Kiang Lee2 Chew Lim Tan1,Tze-Yun Leong1 1National University of Singapore 2National Neuroscience Institute 3Bioinformatics Institute, Singapore Good afternoon everyone! My talk today is about building an unsupervised medical image classification by combining case-based classifiers This is the joint work of the National University of Singapore and the local hospital in Singapore

Automated medical image annotation Huge amount of valuable data available in medical image databases Not fully utilized for medical treatment, research and education Medical image annotation: To extract knowledge from images to facilitate text-based retrieval of relevant images To provide a second source of opinions for clinicians on abnormality detection and pathology classification There is huge amount of valuable data available in medical image databases. However, they have not been fully utilized for medical treatment, research and education. Automated medical Image annotation technology attempts to improve this situation. It provides a way to extract knowledge from images to facilitate text-based retrieval of relevant images and In addition, it provides a second source of opinions for clinicians on abnormality detection and pathology classification

Selecting discriminative features Problem Flowchart of current methods Challenges in current methods Highly sensitive and accurate segmentation Extracting domain knowledge Automatic feature selection Time-consuming manual adjustment process  reduces usages of medical image annotation systems Extracting features Selecting discriminative features Building classifiers Labeling Many image annotation systems have been proposed to address this problem. They usually have four main components. First is the feature extraction process for detecting areas of interest, and these features are subsequently selected for building a classifiers, and finally use the classifier for labeling a new image It is commonly acknowledged that the success of the annotation system depends significantly on the feature extraction and feature selection. However, there exists many challenges in these components. For instance, automated segmentation is considered as the most challenging task in medical imaging. Developing a highly sensitive and accurate segmentation requires a lot of time and effort. Secondly, the domain knowledge is not always available or easily extracted from experts. Therefore, it is hard to manually create a particular set of features that are useful for analyzing images. On the other hand, automatic feature selection is also a non-trivial task. It is hard to even establish a space of relevant feature candidates. Even more, the set of relevant features may well vary from case to case depending on context and the particular image, so that no single set of selected features is optimal for all images As a result, constructing these annotation systems is often a difficult, time-consuming and labor-intensive process. These problems have reduced usage of medical image annotation systems in practice

Objective An automated pathology classification system for volumetric brain image slices Main highlights Eliminates the need for segmentation and semantic or annotation-based feature selection Reduces the amount of manual work for constructing an annotation system Extracts automatically and efficiently knowledge from images Improves the utilization of medical image databases In this work, We propose an automated pathology classification system for volumetric brain image slices. The main highlights our work are: first, we eliminate the need for segmentation nor semantic- or annotation-based feature selection. Consequently, the knowledge in image databases can be extracted automatically and efficiently. And finally, we hope to improve the utilization of medical image databases.

System overview Case-based classifier Gabor filters Non domain specific features Localized low-level features Ensemble learning Set of classifiers Each classifier with a random subset of features Final classification: an aggregated result In this slide, we will talk about the architecture of our proposed system. The system consists of two main components: case-based classifier and ensemble learning. Case-based classification is performed directly on previously seen data without an intermediate modeling phase. We deploy domain independent Gabor-filters that extract localized low level features from the image. However, with such a low level feature space, selection of a small set of significant features appears counterintuitive. We therefore use ensemble learning and form a collection of weak classifiers. Each classifier specialize on a different random subset of features. The final classification is then a product of these classifiers.

Sparse representation-based classifier Sparse representation-based classifier (SRC) proposed by Wright et al. for face recognition task Non-parametric sparse representation classifier SRC consists of two stages Reconstructing: a test image as a linear combination of a small number of training images Classifying: evaluating how the images belonging to different classes contribute to the reconstruction of the test image First, we will look at the case-base classifier. In our work, we deploy a sparse representation-based classifier or SRC for short. SRC was proposed by Wright et al. [12] and has achieved high performance and success for face recognition task. This is a nonparametric sparse representation classifier. The SRC consists of two stages: reconstruction and classification In reconstruction: the SRC aims at reconstructing the test image as a linear combination of a small number of training images. Classification is then done by evaluating how the images be- longing to the different classes contribute to the reconstruction of the test image.

Sparse representation-based classifier (Wright et al.) This is the pseudo-code for this classification algorithm which I just described briefly before. The inputs for this algorithm is training samples and a test sample which are represented as a matrix The final output of the algorithm is the class specific reconstruction error r_k(y) or we call It residual. In the original algorithm, from these residuals, we can decide which class a test sample belongs to. However, in our framework, we stop at this residual and we use it in the ensemble learning framework.

Image databases x1, x2,…, x1000 Sparse reconstruction New data item y ≈ a7x7 + a23x23 + a172x172 + a134x134 + a903x903 y ≈ a7x7 + a23x23 + a172x172 + a134x134 + a903x903 Class residuals Here is the concrete example of the SRC algorithm. Suppose that we have a training databases consisting of 1000 images. Given a new image, it will be sparsely reconstructed from the training cases. This is done through the optimization of the trade-off between the approximation quality and the number of old cases allowed in the reconstruction. Notice that cases in reconstruction belong to different classes (red and blue) From this, we can calculate the class specific residuals as r1 and r2 r1 = || y – (a7x7 + a172x172 + a132x134)||2 r2 = || y – (a23x23 + a903x903)||2

Ensemble of weak classifiers Combine multiple weak classifiers Take class specific residuals as confidence measures The smaller the residual for the class, the better we construct the test by just using the samples from that class To classify image y, compute average class-specific residuals of all W weak classifiers Then, multiple SRC classifiers are combined to form an ensemble. As we observe that the smaller the residual for the class, the better we construct the test by just using the samples from that class. Therefore, we take the class specific residuals as confidence measurers. Finally, .... The test sample y is assigned to the class with minimum average residual.

Domain Automatically annotate CT brain images for traumatic brain injury (TBI) TBI: major cause of death and disability Several types of hemorrhages: Extradural hematoma (EDH) Subdural hematoma (SDH) Intracerebral hemorrage (ICH) Subarachnoid hemorrhage (SAH) Intraventricular hematoma (IVH) Subdural hematoma Extradural hematoma In principle, our discussed approach is applicable to any volumetric brain images. In this work, we focus on CT brain images for traumatic brain injury (or TBI). TBI is a major cause of death and disability in the world. TBI consists several types of hemorrhages. These classification scheme is based on different location, different shape and different size of hemorrhage region.

Data Volumetric stack of 18-30 images (slices) CT brain scans of 103 patients Each scan: Volumetric stack of 18-30 images (slices) Image resolution: 512 x 512 pixels Manually assigned a hematoma type extracted from its medical text report Our data set consists of images featuring three types of TBI: EDH (24 patients), ICH (21 patients) and SDH (58 patients). Each case is in the form of a volumetric stack consisting of 18-30 slices. The dataset is manually labeled with TBI subtype and verified with findings from medical text reports. In this work, we only focus on detection of abnormal slices, thus slices without abnormali- ties are removed from the dataset. We assume that each slice only exhibits a single type of hematoma.

A volumetric CT brain scan with 19 slices Here is an example of a volumetric brain scan. It is a stack of 19 slices. A volumetric CT brain scan with 19 slices

Experimental setup Compared performances of SRC vs. SVM vs. SVM + feature selection With/without ensemble learning Run stratified ten-fold cross-validation 50 times with different random foldings Measured the average precisions and recalls Separated training and testing dataset at the case level In our experiments, we compare performances of proposed method with plain SRC, baseline classifier SVM and SVM with feature selection. And we also study the performance of ensemble learning on both SRC and SVM. We run stratified ten-fold cross-validation 50 times with different random foldings to evaluate the performance of our system. Although the system works at the slice level, training and testing dataset are separated at the case level to avoid slices from the same case being both in training and test data.

Experimental results As discussed in the previous slide, we constructed five settings: a baseline classifier SVM, SVM with automatic feature selection technique, Ensemble with SVM, and plain SRC Our Ensemble+SRC method yields promising results in classifying abnormal TBI slices (Table 1). The performance in the majority class SDH is better than in EDH and ICH. It is also interesting to see that SRC appears to perform better than SVM in this domain. Even without feature selection. ---- We use the standard support vector machine (SVM) classifier as a baseline classifier. To study the aspect of feature selection, we construct the baseline classifier with automatic feature selection method. Features are ranked according to their variance in the dataset and the top 1000 features with largest variance serve as an input space for the SVM To study the role of ensemble, we have constructed an Ensemble+SVM method that uses an ensemble architecture and SVM as a weak classifier We have also included a plain SRC method that directly uses all the features.

Experimental results when varying the ensemble size There are two parameters that might influence on the overall performance of our system. They are the size of ensemble and the number of features per classifier. This table illustrates the performance of the system when changing the size of the ensemble. All cases are given 1000 randomly selected features. The number of classifiers appears to affect both the precision and the recall. Average precision and recall of classifiers when varying the ensemble size (number of features = 1000)

Experimental results when varying the number of features per classifier But when we vary the number of feature per classifier, they appear to have a smaller effect. In this table, we use 50 weak classifiers, when increasing the number of sampled features from 500 to 2000, it does not appear to improve the results. We also observed that when the number of features is increased, the ensemble size should be increased as well to avoid overfitting. Average precisions and recalls of classifiers when varying number of features (ensemble size = 50)

Conclusion Ensemble classification framework with sparse Gabor-feature based classifier Eliminates the requirement for segmentation and supervised feature selection Reduces the need for manual adjustment Achieves reasonable results compared to segmentation dependent techniques (Gong et al.) Limitation Longer classification time when dealing with large training data Manual weighting needed for imbalanced data We have introduced an ensemble classification framework with sparse Gabor-feature based classifiers The system does not require segmentation or supervised feature selection. This reduces the need for manual work in extracting useful information from medical images. Our experiments show that in the domain of classifying TBI images, we achieve reasonable results to segmentation-dependent techniques that rely on manually selected handcrafted features Although the proposed approach does not make any modality dependent assumptions, testing it on other modalities is an obvious topic for future work. The proposed method is non-parametric, thus when more training data is used, the classification will take a longer time. Finally, manual weighting is still needed when dealing imbalanced data

Thank you

Gabor features Localize low level features from an input image Resemble the primitive features extracted by human visual cortex Extract edge like features in different scales and orientations at different locations of the image Create a Gabor filter bank with 5 frequencies and 8 orientations A 128 x 128 grayscale image: 655360 features Randomly select 4000 Gabor features to form a feature subspace No domain specific feature Automatic Generic low level features