1. 1 Automated Feature Abstraction of the fMRI Signal using Neural Network Clustering Techniques Stefan Niculescu and Tom Mitchell Siemens Medical Solutions, Carnegie Mellon University December 8 th, 2006

2. 2 Study: Pictures and Sentences Trial: read sentence, view picture, answer whether sentence describes picture Picture presented first in half of trials, sentence first in other half Image every 500 msec 12 normal subjects Three possible objects: star, dollar, plus Collected by Just et al.

3. 3 It is true that the star is above the plus?

4. 4

5. 5 + --- *

6. 6

7. 7 Task and Goal Task: discriminate whether a subject is looking at a picture or is reading a sentence Trained per subject classifier –Only one subject analyzed, using only CALC Goal: find useful abstractions of the data that allow accurate classification Method proposed: –Abstract at the hidden layer level of neural networks –Assign each voxel to a unique cluster (hidden unit) based on: Distance to center of cluster The weight from that voxel to the cluster

8. 8 After training, each hidden unit will summarize a useful feature extracted from a subset of voxels. Each voxel will belong to exactly one cluster Each input unit represents a voxel in the brain Learned feature abstraction / cluster Output classification The Model

9. 9 Modified Backpropagation Algorithm 1.Initialize the weights with small random numbers. Set learning rate to 0.1 2.Initialize the centers of the clusters to be all equal to the center of mass of the voxels. 3.Run stochastic backpropagation for each sample in the training set 4.For each input feature i (starting with i=1), find the hidden unit j = argmax ( || w ik || / d ik ) 5.Set w ik = 0 except for k=j, assign voxel i to cluster j and recompute the center of the cluster j 6.Compute the error on the early stopping set. If it is smaller than before, save the current clustering and weights. If it is larger, check if there was no improvement in the last fixed number of epochs (1000 by default) and, if so, GO TO 7. Otherwise GO TO 3 7.Output clustering. Report accuracy on the validation set.

10. 10 Accuracies of trained classifiers Classifier \ Dataset 40 Examples320 Examples ANN (2 clusters)1.000.94 ANN (3 clusters)1.000.93 ANN (4 clusters)1.000.90 ANN (5 clusters)1.000.90 GNB0.900.875 SVM0.8750.83 3 NN0.8750.77 *Four fold crossvalidation used.

11. 11

12. 12 An example of clustering Clustering of CALC (318 voxels) in two clusters using 320 examples Accuracy ~ 0.94

13. 13 Summary Our method yields high classification accuracy, comparable with all other classifiers we tried A way to abstract the data by mathematically summarizing the important feature in a group of voxels. The abstraction is not easy to interpret

14. 14 Directions for Future Research Apply this method for other subjects –Check if learnt clusters are similar Try automated feature abstraction for tasks where we did not get good accuracies –Discriminate between affirmative and negative sentences Try other datasets –Semantic categories – 12 way classification task