Week 1 - An Introduction to Machine Learning & Soft Computing -Yosi Kristian-

Soft Computing STTS – Yosi Kristian

Definition Soft Computing is a term applied to a field within computer science which is characterized by the use of inexact solutions to computationally hard tasks, for which there is no known algorithm that can compute an exact solution.  Soft computing differs from conventional (hard) computing in that, unlike hard computing, it is tolerant of imprecision, uncertainty, partial truth, and approximation. Soft Computing STTS – Yosi Kristian

Still the Definition.. In effect, the role model for soft computing is the human mind. The guiding principle of soft computing is: Exploit the tolerance for imprecision, uncertainty, partial truth, and approximation to achieve tractability, robustness and low solution cost. Soft Computing STTS – Yosi Kristian

Components of soft computing What ??? Are we going to learn them all in this subject? Neural networks (NN) Support Vector Machines (SVM) Fuzzy logics (FL) Evolutionary computation (EC), including: Evolutionary algorithms Genetic algorithms Differential evolution Meta heuristic and Swarm Intelligence Ant colony optimization Bees algorithms Bat algorithm Cuckoo search Harmony search Firefly algorithm Artificial immune systems Particle swarm optimization Soft Computing STTS – Yosi Kristian

Soft Computing in AI Soft computing may be viewed as a foundation component for the emerging field of conceptual intelligence. Machine Learning Fuzzy Systems Evolutionary Computation Probabilistic Reasoning Soft Computing is the CORE component of many Machine Learning System Soft Computing STTS – Yosi Kristian

Machine Learning Arthur Samuel (1959). Machine Learning: Field of study that gives computers the ability to learn without being explicitly programmed. Tom Mitchell (1998) Well-posed Learning Problem: A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E. Soft Computing STTS – Yosi Kristian

Machine learning usage Usage of Machine Learning is to develop applications that can’t be programed by hand. E.g., Autonomous helicopter, handwriting recognition, most of Natural Language Processing (NLP), Computer Vision etc. Or a machine learning system could be trained on email messages to learn to distinguish between spam and non-spam messages. After learning, it can then be used to classify new email messages into spam and non-spam folders. Soft Computing STTS – Yosi Kristian

Machine Learning Categorized By Data and Learning Process Soft Computing STTS – Yosi Kristian

Soft Computing In Machine Learning Soft Computing is the soul of many machine learning system. Classification and Clustering is a very common soft computing problems. Soft Computing STTS – Yosi Kristian

Intro to Supervised Learning Soft Computing STTS – Yosi Kristian

Example Housing price prediction. Supervised Learning in 1000’s Size in feet2 Supervised Learning “right answers” given Regression: Predict continuous valued output (price) Soft Computing STTS – Yosi Kristian

Example Breast cancer (malignant, benign) Classification Discrete valued output (0 or 1) 1(Y) Malignant? 0(N) Tumor Size Tumor Size Soft Computing STTS – Yosi Kristian

Another Example Clump Thickness Uniformity of Cell Size Uniformity of Cell Shape … Age Tumor Size Soft Computing STTS – Yosi Kristian

Exercise You’re running a company, and you want to develop learning algorithms to address each of two problems. Problem 1: You have a large inventory of identical items. You want to predict how many of these items will sell over the next 3 months. Problem 2: You’d like software to examine individual customer accounts, and for each account decide if it has been hacked/compromised. Should you treat these as classification or as regression problems? Soft Computing STTS – Yosi Kristian

Answer Treat both as classification problems. Treat problem 1 as a classification problem, problem 2 as a regression problem. Treat problem 1 as a regression problem, problem 2 as a classification problem. Treat both as regression problems. Soft Computing STTS – Yosi Kristian

Classification Example Another Example is for image Classification / Categorization Training Training Labels Training Images Image Features Classifier Training Trained Classifier Soft Computing STTS – Yosi Kristian

Cont… Testing Image Features Trained Classifier Test Image Prediction Outdoor Soft Computing STTS – Yosi Kristian

Learning a classifier Given some set of features with corresponding labels, learn a function to predict the labels from the features Training labels dictate that two examples are the same or different, in some sense Features and distance measures define similarity Classifiers try to learn weights or parameters for features and distance measures so that feature similarity predicts label similarity Soft Computing STTS – Yosi Kristian

Intro to Unsupervised Learning Soft Computing STTS – Yosi Kristian

Supervised Learning x2 x1 Soft Computing STTS – Yosi Kristian

Unsupervised Learning x2 x1 Soft Computing STTS – Yosi Kristian

Clustering Example Soft Computing STTS – Yosi Kristian

Contd… Soft Computing STTS – Yosi Kristian

Exercise Of the following examples, which would you address using an unsupervised learning algorithm? (Check all that apply.) Given email labeled as spam/not spam, learn a spam filter. Given a set of news articles found on the web, group them into set of articles about the same story. Given a database of customer data, automatically discover market segments and group customers into different market segments. Given a dataset of patients diagnosed as either having diabetes or not, learn to classify new patients as having diabetes or not. Soft Computing STTS – Yosi Kristian

Warming Up…. Do 10 x Push Ups. Soft Computing STTS – Yosi Kristian

Linear Regression with one variable Housing Prices (Portland, OR) Price (in 1000s of dollars) Size (feet2) Supervised Learning Given the “right answer” for each example in the data. Regression Problem Predict real-valued output Soft Computing STTS – Yosi Kristian

Linear Regression with one variable Training set of housing prices (Portland, OR) Notation: m = Number of training examples n = Number of feature x’s = “input” variable / features y’s = “output” variable / “target” variable Soft Computing STTS – Yosi Kristian

The Concept Training Set How do we represent h ? Learning Algorithm Size of house h Estimated price Linear regression with one variable. Univariate linear regression. Soft Computing STTS – Yosi Kristian

Cost Function Training Set Hypothesis: ‘s: Parameters How to choose ‘s ? Soft Computing STTS – Yosi Kristian

Contd.. Soft Computing STTS – Yosi Kristian

Cost Function.. y x Idea: Choose so that is close to for our training examples Soft Computing STTS – Yosi Kristian

Simplification: For the sake of understanding Simplified Hypothesis: Parameters: Cost Function: Goal: Soft Computing STTS – Yosi Kristian

Trial 1 y x (function of the parameter ) (for fixed , this is a function of x) y x Soft Computing STTS – Yosi Kristian

Trial 2 y x (for fixed , this is a function of x) (function of the parameter ) y x Soft Computing STTS – Yosi Kristian

Trial 3 y x (function of the parameter ) (for fixed , this is a function of x) y x Soft Computing STTS – Yosi Kristian

Done with simplification, back to real world. Hypothesis: Parameters: Cost Function: Goal: Soft Computing STTS – Yosi Kristian

The Contour Figures …. How to find minimum of J in that? Soft Computing STTS – Yosi Kristian

Gradient Descent.. Next Week… Soft Computing STTS – Yosi Kristian