Machine Learning Dr. Mohamed Farouk

What’s in a Name Pattern Recognition Data Mining Machine Learning Statistical Modeling Artificial Intelligence Computational Intelligence

Outline Applications of machine learning Types of modeling tasks Machine learning algorithms

Handwritten Digits Recognition

Remote sensing image classification

Medical Diagnosis

Robot Learning

Facial Expression Recognition

Visual Object Recognition

Iris Flower Classification versicolor virginica setosa

Iris Flower Classification

Types of Modeling Tasks Regression Classification Time-Series analysis and prediction Sequence analysis Ranking Anomaly Detection

Data f1 f2 f3 fd y X1 X2 X3

Regression f1 f2 f3 fd y X1 2.37 X2 4.21 X3 -1.11 0.98 0.12 f: Rd → R f(x) = y

Classification f1 f2 f3 fd y X1 TRUE X2 FALSE X3 f: Rd → {c1,c2,c3} f(x) = y

Ranking R_1 R_2 R_3 B_1 B_2

Supervised Learning

Function Approximation

Classification Generative model Discriminative model

Labelled Training data The Big Picture Data Collection Training data Labelled Training data Annotation classifier Learning algorithm Feature Extraction How the supervised model works Collect Images Label/Annotate them Feature extraction (HOG features in our case) Apply the learning algorithm Use the model in the detector Processes the testing data in the detector Evaluate the results Test data Evaluate

Types of Features Binary Numeric (discrete or continuous) Categorical (nominal) (red, white, blue) Ordered-Categorical (small, medium, big) Order constraints (i.e. rank: 1st, 2nd, 3rd, …)

Linear Models Linear combination of features Hyper-plane in the feature space High bias / Low variance Numerous methods to train, usually very fast Categorical features need to be encoded Very fast at runtime

Artificial Neural Networks

Artificial Neural Networks 1 in out

Collection of unrelated tools Toolkits Orange Octave SQL Server WEKA Collection of unrelated tools Value Ease of Use/Practicality R Matlab Power/Comprehensiveness

What you need in your toolkit Visualization Feature Processing/Selection Modeling Algorithms Meta Algorithms/Ensemble Methods Tuning/parameter sweeps Measurement/Visualization

Weka

Weka