1 Introduction to Machine Learning Chapter 1. cont.

Review: Learning Definition Well-posed learning problem: – Improve on task, T, with respect to performance metric, P, based on experience, E. The main issue of Learning: – Finding a general function from specific training examples

Examples T: Playing checkers P: Percentage of games won against an arbitrary opponent E: Playing practice games against itself T: Recognizing hand-written words P: Percentage of words correctly classified E: Database of human-labeled images of handwritten words T: Driving on four-lane highways using vision sensors P: Average distance traveled before a human-judged error E: A sequence of images and steering commands recorded while observing a human driver. T: Categorize messages as spam or legitimate. P: Percentage of messages correctly classified. E: Database of s, some with human-given labels

Designing a learning system 1.Choosing the training experience (data set) 2.Choosing the target function 3.Choosing a representation for the target function 4.Choosing a function approximation algorithm 5.The final design

Choosing the Training Experience – Sometimes straightforward Text classification, disease diagnosis – Sometimes not so straightforward Chess playing, checkers (indirect information is available)

Training Experience Attributes How the training experience is controlled by the learner? – Is it provided by a human process outside the learner’s control? – Does learner collect the training examples by autonomously exploring its environment? How well it represent the distribution of the examples? – Playing checkers: Playing practice games against itself

Designing a learning system 1.Choosing the training experience (data set) 2.Choosing the target function 3.Choosing a representation for the target function 4.Choosing a function approximation algorithm 5.The final design

Choosing the Target Function For checkers: – Could learn a function: ChooseMove(board, legal-moves) → best-move – Or could learn an evaluation function, V(board) → R, Where R is a real value representing how favorable the board is.

Ideal definition of V(b) If b is a final winning board, then V(b) = 100 If b is a final losing board, then V(b) = –100 If b is a final draw board, then V(b) = 0 Otherwise, then V(b) = V(b’), where b’ is the highest scoring final board position that is achieved starting from b and playing optimally until the end of the game (assuming the opponent plays optimally as well). This definition is non-operational => Approximation of the ideal function

Linear Function for Representing V(b) – bp(b): number of black pieces on board b – rp(b): number of red pieces on board b – bk(b): number of black kings on board b – rk(b): number of red kings on board b – bt(b): number of black pieces threatened (i.e. which can be immediately taken by red on its next turn) – rt(b): number of red pieces threatened

A win board example, 100> (win for black) Training Examples: { }

Designing a learning system 1.Choosing the training experience (data set) 2.Choosing the target function 3.Choosing a representation for the target function 4.Choosing a function approximation algorithm 5.The final design

Examples of Value Functions Linear Regression – Input: feature vectors – Output: Logistic Regression Input: feature vectors Output:

Examples of Classifiers Linear Classifier – Input: feature vectors – Output:

Examples of Classifiers Rule Classifier Decision tree A tree with nodes representing condition testing and leaves representing classes Decision list If condition 1 then class 1 elseif condition 2 then class 2 elseif ….

Designing a learning system 1.Choosing the training experience (data set) 2.Choosing the target function 3.Choosing a representation for the target function 4.Choosing a function approximation algorithm 5.The final design

Learning Approximating the weights using the data set

Least Mean Square, Gradient Discent MSE (mean squared error):

Gradient Descent

Designing a learning system 1.Choosing the training experience (data set) 2.Choosing the target function 3.Choosing a representation for the target function 4.Choosing a function approximation algorithm 5.The final design

Design