ELEC 303 – Random Signals Lecture 17 – Hypothesis testing 2 Dr. Farinaz Koushanfar ECE Dept., Rice University Nov 2, 2009

outline Reading: 8.2,9.3 Bayesian Hypothesis testing Likelihood Hypothesis testing

Four versions of MAP rule  discrete, X discrete  discrete, X continuous  continuous, X discrete  continuous, X continuous

Example – spam filter may be spam or legitimate Parameter , taking values 1,2, corresponding to spam/legitimate, prob p  (1), P  (2) given Let  1,…,  n be a collection of special words, whose appearance suggests a spam For each i, let X i be the Bernoulli RV that denotes the appearance of  i in the message Assume that the conditional prob are known Use the MAP rule to decide if spam or not.

Bayesian Hypothesis testing Binary hypothesis: two cases Once the value x of X is observed, Use the Bayes rule to calculate the posterior P  |X (  |x) Select the hypothesis with the larger posterior If g MAP (x) is the selected hypothesis, the correct decision’s probability isP(  = g MAP (x)|X=x) If Si is set of all x in the MAP, the overall probability of correct decision is P(  = g MAP (x))=  i P(  =  i,X  S i ) The probability of error is:  i P(  i,X  S i )

Multiple hypothesis

Example – biased coin, single toss Two biased coins, with head prob. p 1 and p 2 Randomly select a coin and infer its identity based on a single toss  =1 (Hypothesis 1),  =2 (Hypothesis 2) X=0 (tail), X=1(head) MAP compares P  (1)P X|  (x|1) ? P  (2)P X|  (x|2) Compare P X|  (x|1) and P X|  (x|2) (WHY?) E.g., p 1 =.46 and p 2 =.52, and the outcome tail

Example – biased coin, multiple tosses Assume that we toss the selected coin n times Let X be the number of heads obtained ?

Example – signal detection and matched filter A transmitter sending two messages  =1,  =2 Massages expanded: – If  =1, S=(a 1,a 2,…,a n ), if  =2, S=(b 1,b 2,…,b n ) The receiver observes the signal with corrupted noise: X i =S i +W i, i=1,…,n Assume W i  N(0,1)

Likelihood Approach to Binary Hypothesis Testing

BHT and Associated Error

Likelihood Approach to BHT (Cont’d)

Binary hypothesis testing H 0 : null hypothesis, H 1 : alternative hypothesis Observation vector X=(X 1,…,X n ) The distribution of the elements of X depend on the hypothesis P(X  A;H j ) denotes the probability that X belongs to a set A, when H j is true

Rejection/acceptance A decision rule: – A partition of the set of all possible values of the observation vector in two subsets: “rejection region” and “acceptance region” 2 possible errors for a rejection region: – Type I error (false rejection): Reject H 0, even though H 0 is true – Type II error (false acceptance): Accept H 0, even though H 0 is false

Probability of regions False rejection: – Happens with probability  (R) = P(X  R; H 0 ) False acceptance: – Happens with probability  (R) = P(X  R; H 1 )

Analogy with Bayesian Assume that we have two hypothesis  =  0 and  =  1, with priors p  (  0 ) and p  (  1 ) The overall probability of error is minimized using the MAP rule: – Given observations x of X,  =  1 is true if – p  (  0 ) p X|  (x|  0 ) < p  (  1 ) p X|  (x|  1 ) – Define:  = p  (  0 ) / p  (  1 ) – L(x) = p X|  (x|  1 ) / p X|  (x|  0 )  =  1 is true if the observed values of x satisfy the inequality: L(x)> 

More on testing Motivated by the MAP rule, the rejection region has the form R={x|L(x)>  } The likelihood ratio test – Discrete: L(x)= p X (x;H 1 ) / p X (x;H 0 ) – Continuous: L(x) = f X (x;H 1 ) / f X (x;H 0 )

Example Six sided die Two hypothesis Find the likelihood ratio test (LRT) and probability of error

Error probabilities for LRT Choosing  trade-offs between the two error types, as  increases, the rejection region becomes smaller – The false rejection probability  (R) decreases – The false acceptance probability  (R) increases

LRT Start with a target value  for the false rejection probability Choose a value  such that the false rejection probability is equal to  : P(L(X) >  ; H 0 ) =  Once the value x of X is observed, reject H 0 if L(x) >  The choices for  are 0.1, 0.05, and 0.01

Requirements for LRT Ability to compute L(x) for observations X Compare the L(x) with the critical value  Either use the closed form for L(x) (or log L(x)) or use simulations to approximate

Example A camera checking a certain area Recording the detection signal X=W, and X=1+W depending on the presence of the intruders (hypothesis H 0 and H 1 ) Assume W~N(0,  ) Find the LRT and acceptance/rejection region

Example

Example (Cont’d)

Error Probabilities

Example: Binary Channel

Example: More on BHT