1. ROC Based Evaluation and Comparison of Classifiers for IVF Implantation Prediction Aslı Uyar, Ayşe Bener Boğaziçi University, Department of Computer Engineering, Istanbul Nadir Çıray, Mustafa Bahçeci IVF Unit of German Hospital, Istanbul PhD Thesis 2nd Progress Presentation 15th June 2009

2. I N- V ITRO F ERTILIZATION (IVF) IVF is a common infertility treatment method during which female germ cells (oocytes) are inseminated by sperm under laboratory conditions. Fertilized oocytes are cultured between 2-6 days in special medical equipments and embryonic growth is observed and recorded by embryologists. Finally, selected embryo(s) are transferred into the woman's womb.

3. P ROBLEM S TATEMENT In-vitro fertilization (IVF) A complex and costly process requiring automated decision support Many critical decisions affecting the success of treatment Analysis of factors affecting treatment outcome Number of embryos to be transferred Embryo selection Decision of transfer day Initial consideration Implantation prediction of individual embryos Yielding reliable elective single embryo transfer (eSET) avoiding multiple pregnancies

4. M OTIVATION AND O BJECTIVES MOTIVATION: Lack of reliable eSET criteria and public IVF datasets Limited number of machine learning based implantation prediction studies Conflicting prediction results No consensus on: input feature sets training and testing strategies performance measures OBJECTIVES Dataset construction Benchmarking machine learning based predictor models Enhancement of prediction results by methodological improvements or novel techniques Statistical validation and generalization of proposed methods

5. M ETHODOLOGY

6. D ATASET Database of German Hospital, Bahceci IVF Center Cycles performed from January 2007 through August 2008 Two classes of total 2453 embryos Each embryo was represented as an individual record with 18 input features related to clinical patient and embryo variables 1853 embryos with proven negative implantation (cycles with negative outcome) 270 embryos with proven positive implantation outcome (cycles in which number of visualized pregnancy sacs were equal to number of transferred embryos) Imbalanced class distribution with 89% negative and 11% positive cases Considering both FP and FN rates

7. D ATASET F EATURES

8. P ERFORMANCE M EASURES AND R OC A NALYSIS A single performance measure representing discriminative power of binary classification Default threshold t_0 = 0.5 Ideal case: (0,1) left corner of ROC curve Optimum threshold t_opt: threshold value that maps to nearest point to (0,1) Accuracy = TP + TN / (TP + FN + FP + TN) Sensitivity = TP / (TP + FN) ~ TPR Specificity = TN / (TN + FP) (1 – Specificity) ~ FPR

9. T RAINING & T ESTING S TRATEGY Dataset is randomly divided into 3 parts 2/3 used for training and 1/3 used for testing Parameter optimization on training set using 10 fold cross validation 2/3 – 1/3 random partitioning is repeated 10 times in order to overcome sampling bias Reported results are mean and standard deviation of these 10 repetitions Stratification principle: ensures that the proportions of implanted and not-implanted embryos were the same in both training and test sets as in the original dataset

10. B ENCHMARKING C LASSIFIERS ClassifierNBRBFMLPSVMkNNDT AUC0.739 ± 0.036 0.712 ± 0.036 0.675 ± 0.039 0.657 ± 0.020 0.612 ± 0.030 0.550 ± 0.056 Only NB and RBF provided acceptable discrimination with 0.7 ≤ AUC < 0.8. (Hosmer and Lemeshow, 2000)

11. Thanks for listening!