1. Color-based Diagnosis: Clinical Images
Research Project Funded In Part by NIH
Yue (Iris) Cheng, Dr. Scott E Umbaugh @
Computer Vision and Image Processing Research Lab
Electrical and Computer Engineering Department
Southern Illinois University Edwardsville

2. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Overview
Skin tumors can be either malignant or benign
Clinically difficult to differentiate the early stage of malignant melanoma and benign tumors due to the similarity in appearance
Proper identification and classification of malignant melanoma is considered as the top priority because of cost function
Classification of skin tumors using computer imaging and pattern recognition
Previous texture feature algorithm successfully differentiate the deadly melanoma and benign tumor seborrhea kurtosis
Relative color feature algorithm is explored in this research for differentiate melanoma and benign tumors, dysplastic nevi and nevus
Successfully classify 86% of malignant melanoma using relative color features, compared to the clinical accuracy by dermatologists in detection of melanoma of approximately 75%
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

3. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Materials and Tools
Image database
Original tumor images
512x bit color images digitized from 35mm color photographic slides and photographs
160 melanoma, 42 dysplastic, and 80 nevus skin tumor images
Border images
Binary images drawn manually and reviewed by the dermatologist for accuracy
Software
CVIPtools
Computer vision and image processing tools developed at our research lab
Partek
Statistical analysis tools
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

4. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
CVIPtools
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

5. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Method Design
Creation of relative color images
Segmentation and morphological filtering
Relative color feature extraction
Design of tumor feature space and object feature space
Establishing statistical models from relative color features
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

6. Create Relative Color Skin Tumor Images
Purpose
to equalize any variations caused by lighting, photography/printing or digitization process
to equalize variations in normal skin color between individuals
the human visual system works on a relative color system
Algorithm
Mask out non-skin part in the image to calculate the normal skin color
Separate tumor from the image
Remove the skin color from the tumor to get a relative color skin tumor image
CVIPtools functions were used to create relative color skin tumor images
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

7. Calculate Skin Color Original Noisy Skin Tumor Image
Non-skin Algorithm
Calculate
Mask out tumor
Skin Tumor Image W/O Noise
Average
R, G, B Value of Skin
Skin-Only
Image
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

8. Tumor Image Original Noisy Skin Tumor Image Tumor Border Image
AND
Original Noisy Skin Tumor Image
Border
Image
Tumor
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

9. Relative Color Tumor Image
SUBTRACT
Tumor
Image
Average
R, G, B Value of Skin
Relative Color Image of the Tumor
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

10. Segmentation and Morphological Filtering
Image segmentation was used to find regions that represent objects or meaningful parts of objects
Morphological filtering was used to reduce the number of objects in the segmented image
Easy to use CVIPtools for experimenting and analysis
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

11. Relative Color Feature Extraction
Necessary to simplify the raw image data into higher level, meaningful information
Feature vectors are a standard technique for classifying objects, where each object is defined by a set of attributes in a feature space.
Totally 17 color features and binary features were extracted using CVIPtools
The three largest objects, based on the binary feature ‘area’, were used in feature extraction
Histogram features, that is, color features, were extracted in each color band from relative color image objects
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

12. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
17 Features
Binary features
Area
Thinness
Histogram features in R, G, B bands
Mean
Standard deviation
Skewness
Energy
Entropy
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

13. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
17 Features (Cont.)
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

14. Design Two Feature Spaces
Tumor feature space
consists of 277 feature vectors correspond to 277 skin tumor images.
each feature vector has 51 feature elements, which are the total of 17 features of each three largest objects within the same tumor.
Object feature space
had 842 feature vectors corresponding to 842 image objects
each feature vector has 17 feature elements, which were the binary features and color features stated as above
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

15. Establishing Statistical Models
Two feature spaces serve as two data models in order to maximize the possibility of success
Two classification models, Discriminant Analysis and Multi-layer Perceptron, were developed for both data models
The training and test paradigm is used in statistical analysis to report unbiased results of a particular algorithm
due to small size of data set, 282 images, we used the leave x out method, with both one and ten for x
Partek software was used
to analyze the data representing the features
to develop a model or rules for classifying the tumors
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

16. Quadratic Discriminant Analysis
A statistical pattern recognition technique based on Bayesian theory, which classifies data based on the distribution of measurement data into predefined classes
Normalization the feature data as preprocessing
performed to maximize the potential of the features to separate classes and satisfy the requirement of the modeling tool such as Quadratic discriminant analysis for a Bayesian distribution of the input data
Variable selection was used to choose dominant features.
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

17. Multi-Layer Perceptron
A feed forward neural network
neural networks modeled after the nervous system in biological systems, based on the processing element the neuron
widely used for pattern classification, since they learn how to transform a given data into a desired output.
Principal Component Analysis (PCA) as preprocessing
a popular multivariate technique, is to reduce dimensionality by extracting the smallest number components that account for most of the variation in the original multivariate data and to summarize the data with little loss of information
the dispersion matrix selected for PCA in this project is correlation
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

18. Multi-Layer Perceptron (Cont.)
Creation, training and testing of neural networks
Creation a neural network involves selection of hidden and output neuron types and a random number generation.
Four output neuron types – Softmax, Gaussian, Linear and sigmoid
Three hidden neuron types – Sigmoid, Gaussian and Linear
Scaled Conjugate Gradient algorithm is used for learning in this project.
Automated and independent of user parameters
Avoids time consuming
Stopping criteria, sum-squared error, is selected to determine after how many iterations the training should be stopped
The trained data is then tested on itself first to examine how far the neural network is able to classify the objects correctly.
Leave x partition out method is used for testing the algorithm
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

19. Experiments and Analysis in Tumor Feature Space
Discriminant Analysis
24 features selected for leave ten out method
Histogram
Features
Mean
STD
Skewness
Energy
Entropy R G B
Object 1 X
Object 2
Object 3
10 features selected for leave one out method
Histogram
Features
Mean
STD
Skewness
Energy
Entropy R G B
Object1 X
Object 2
Object 3
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

20. Experiments and Analysis in Tumor Feature Space (Cont.)
Discriminant Analysis (Cont.)
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

21. Experiments and Analysis in Tumor Feature Space (Cont.)
Multi-layer Perceptron
Best features, being in the first three components of the PCA projection data, were used
Success percentages of melanoma as high as 77% and nevus is as high as 68%
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

22. Experiments and Analysis in Object Feature Space
Discriminant Analysis
8, 9, 11 and 12 significant features were selected respectively for leave one out method
Number of
Histogram
Features
Area
Mean
STD
Skewness
Energy
Entropy R G B 8 X 9 11 12
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

23. Experiments and Analysis in Object Feature Space (Cont.)
Discriminant Analysis (Cont.)
Yield consistent results in classifying melanoma from other skin tumor with above 80% success rate
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

24. Experiments and Analysis in Object Feature Space (Cont.)
Multi-layer Perceptron (MLP)
5 out of 12 hidden-output layer neuron combinations gave better classification results
Leave one out method
Yield success percentage as high as 86% for classifying melanoma.
MLP is more consistent in classifying melanoma as well as nevus
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

25. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Conclusion
Multi-Layer perceptron (MLP) with feature data preprocessed by Principal Component Analysis (PCA) gave better classification results for melonoma than Discriminant Analysis (DA)
The best overall successful rate of 78%, of which percentage correct of melanoma is 86%, nevus is 62% and dysplastic is 56%.
The best classification results are achieved with sigmoid used as the hidden and output layer neuron type for the MLP with PCA on Object Feature Space.
The three largest tumor objects are representative for the whole skin tumor.
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

26. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Conclusion (Cont.)
However the small percentage of melanoma misclassification as well as the relatively low success rate for nevus and dysplastic nevi suggests that we may not have the complete data set for the experiments.
In order to achieve better classification results, future experiments
Needs more complete skin tumor image database.
Should combine texture and color methods to get better results
Will include dermoscopy images
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

27. Computer Vision and Image Processing Research Lab @ ECE Dept., SIUE
Acknowledgement
Dr. Scott E Umbaugh, SIUE
Mr. Ragavendar Swamisai
Ms. Subhashini K. Srinivasan
Ms. Saritha Teegala
Dr. William V. Stoecker, Dermatologist, UMR
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE

28. Thank You! Yue (Iris) Cheng Graduate Student @
Computer Vision and Image Processing Research Lab
Electrical and Computer Engineering Department
Southern Illinois University Edwardsville
07/13/2005
Computer Vision and Image Processing Research ECE Dept., SIUE