1. Project 1: Classification Using Neural Networks 2008. 9. 24 Kim, Kwonill kikim@bi.snu.ac.kr kikim@bi.snu.ac.kr Biointelligence laboratory Artificial Intelligence

2. Contents Project outline Description on the data set Description on tools for ANN Guide to Writing Reports  Style  Mandatory contents  Optional contents Submission guide / Marking scheme Demo 2 (C) 2008, SNU Biointelligence Laboratory

3. 3 Outline Goal  Understand MLP deeper  Practice researching and writing a paper Handwritten digits problem (classification)  To predict the classe labels (digits) of handwritten digit data set  Using Multi Layer Perceptron (MLP)  Estimating several statistics on the dataset Data set  Variation of the ‘Handwritten digit data set’  http://archive.ics.uci.edu/ml/datasets/Pen- Based+Recognition+of+Handwritten+Digits http://archive.ics.uci.edu/ml/datasets/Pen- Based+Recognition+of+Handwritten+Digits

4. Handwritten Digit Data Set (1/2) Description  Digit database of 250 samples from 44 writers  http://archive.ics.uci.edu/ml/datasets/Pen- Based+Recognition+of+Handwritten+Digits http://archive.ics.uci.edu/ml/datasets/Pen- Based+Recognition+of+Handwritten+Digits  16 attributes  (x t, y t ), t = 1, …, 8  0 ~ 100  Label (Class)  0, 1, 2, …, 9 4 (C) 2008, SNU Biointelligence Laboratory

5. Handwritten Digit Data Set (2/2) Constitution  Original data (./original)  Preprocessed data (*.arff, *.csv)  Use This!!  Total data(pendigits_total_set, 1099) = training data(pendigits_training, 749) + test data(pendigits_test, 350)  Data description(pendigits.names)  For WEKA(*.arff) 5 (C) 2008, SNU Biointelligence Laboratory

6. 6 Tools for Experiments with ANN Source codes - Choose anything!!  Free software  Weka (recommended)  MATLAB tool box (Toolboxes  Neural Network)  ANN libraries (C, C++, JAVA, …) Web sites  http://www.cs.waikato.ac.nz/~ml/weka/ http://www.cs.waikato.ac.nz/~ml/weka/  http://www.faqs.org/faqs/ai-faq/neural-nets/part5/ http://www.faqs.org/faqs/ai-faq/neural-nets/part5/

7. Reports Style English only!! Scientific journal-style  How to Write A Paper in Scientific Journal Style and Format  http://abacus.bates.edu/~ganderso/biology/resources/writing/HTWsections.html http://abacus.bates.edu/~ganderso/biology/resources/writing/HTWsections.html 7 (C) 2008, SNU Biointelligence Laboratory Experimental process Section of Paper What did I do in a nutshell? Abstract What is the problem?Introduction How did I solve the problem? Materials and Methods What did I find out? Results What does it mean? Discussion Who helped me out? Acknowledgments (optional) Whose work did I refer to? Literature Cited Extra InformationAppendices (optional)

8. Report Contents – Mandatory (1/2) System description  Used software and running environments Result graphs and tables Analysis & discussion (Very Important!!) 8 (C) 2008, SNU Biointelligence Laboratory

9. Report Contents – Mandatory (2/2) Basic experiments  Changing # of epochs (Draw learning curve)  Various # of Hidden Units 9 (C) 2008, SNU Biointelligence Laboratory # Hidden Units TrainTest Average  Std. Dev. BestWorstAverage  Std. Dev. BestWorst Setting 1 accuracy Setting 2 Setting 3 

10. Report Contents – Optional Various experimental settings  Normalization  Learning rates  Structure of MLP  Feature selection  Activation functions  Learning algorithm  … Evaluation techniques  ROC curve  k-fold Crossvalidation  … 10 (C) 2008, SNU Biointelligence Laboratory

11. 11 (C) 2008, SNU Biointelligence Laboratory Submission Guide Due date: Oct. 17 (Fri.) 15:00 Submit both ‘hardcopy’ and ‘email’  Hardcopy submission to the office (301-417 )  E-mail submission to kikim@bi.snu.ac.krkikim@bi.snu.ac.kr  Subject : [AI Project1 Report] Student number, Name  Length: report should be summarized within 12 pages.  If you build a program by yourself, submit the source code with comments We are NOT interested in the accuracy and your programming skill, but your creativity and research ability. If your major is not a C.S, team project with a C.S major student is possible (Use the class board to find your partner and notice the information of your team to TA(bhkim@bi.snu.ac.kr) by Oct. 1)bhkim@bi.snu.ac.kr

12. Marking Scheme 20 points for experiment & analysis  Extra 2 points for additional expriments 10 points for report 3 points for overall organization Late work  - 10% per one day  Maximum 7 days 12 (C) 2008, SNU Biointelligence Laboratory

13. References Materials about Weka  Weka GUI guide (PPT)  Explorer guide (PDF)  Experimenter guide (PDF) 13 (C) 2008, SNU Biointelligence Laboratory

14. WEKA Demo 14 (C) 2008, SNU Biointelligence Laboratory

15. Matlab 15 (C) 2008, SNU Biointelligence Laboratory

16. QnA 16 (C) 2008, SNU Biointelligence Laboratory