CS621/CS449 Artificial Intelligence Lecture Notes Set 7: 08/10/2004, 15/10/2004 13/08/2004 CS-621/CS-449 Lecture Notes

Outline Probability, Statistics & AI The Speech Recognition problem Bayesian Decision Theory A Probabilistic Spell Checker 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Probability, Statistics & AI Importance of knowledge understood from 1950’s research on Machine Translation. Field of AI – characterized by the need for representation & processing of enormous amount of knowledge. Manual input of knowledge difficult. So, consider possibility of acquiring the knowledge or training E.g. Web data  knowledge - not easy by any means!! 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Probabilistic approaches? Fundamental question : Can probabilistic approaches be used? “Next-Word Guessing Program” – a speech recognition problem : Given w1, w2….wn , we need to guess wn+1 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

The Speech Recognition Problem Text T1: I like Artificial Intelligence and Cognitive Sciences. They excite my imagination. Text T2: I hate communal riots. They disrupt life and work. 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Sample Word Frequencies Word-list w Frequency P(w) I 2 2/20 like 1 1/20 Artificial Intelligence and Cognitive Sciences They … …. 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Guessing the next word Word-guesses or Predictions = Next-Word (Model of domain, speaker, hearer) Having seen w1 = “I” taking next word as word with the highest probability, we have: Highly unlikely sequence!! w1 w2 I I/and/they??? 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

A Better Solution Next word = word with highest conditional probability P(w2 | w1) P(like | I) = 0.5 P(hate | I) = 0.5 But, “disrupt”, “work”, “excite” are also possibilities not covered by this rule.  Language Model not adequate! Ideally, “I” is followed by not only “hate” and “like” but by verbs in general 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Bayesian Decision Theory Bayes Theorem : Given the random variables A & B, Where is the posterior probability (the conditional probability of A given that B has occurred), is the prior probability and is the likelihood of B given A 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Bayes Theorem Derivation 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Example It is known that in a population, 1 in 50000 has meningitis and 1 in 20 has stiff neck. It is also observed that 50% of the meningitis patients have stiff neck. A doctor observes that a patient has stiff neck. What is the probability that the patient has meningitis? Solution: Record the event probabilities P(s) & P(m) s – stiff neck, m - meningitis 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Probabilities P(m) = P(an individual has meningitis) = , P(s) = Prior probability = P(s | m) = 0.5 Posterior probability is l It is most likely that the person does not have meningitis since 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Some Issues Important question : Is P(m|s) greater or lesser than P(s |m)? P(m|s) could have been found as Some questions: Which is more reliable to compute, P(s|m) or P(m|s)? Which evidence is more sparse , P(s|m) or P(m|s)? Test of significance : The counts are always on a sample of population. Which probability count has sufficient statistics? 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

A Probabilistic Spell Checker 3 broad problems: Non-word deletion : Apple  Aple Isolated word correction : Apple Aple Applet Maple Detection & correction from context (very difficult and needs full NLP) : piece  peace E.g. : The piece for which he struggled long and hard was stolen at night one day. The peace for which he struggled long and hard was destroyed in a single day with the outbreak of sect rivalries. 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Types of Errors Isolated word errors: Errors which come from typing Deletion : Apple  Aple Insertion : Apple  Applet Substitution : Apple  Apqle Transposition : Apple  Aplpe Errors from OCR (Optical Character Recognizer) Blank Deletion : I watchcricket Blank Insertion : I wa tch cricket Mangling : I wdch cricket at 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes

Noisy Channel Model The problem formulation for spell checker is based on the Noisy Channel Model s t (sn, sn-1, … , s1) (tm, tm-1, … , t1) Given t, find the most probable s : Find that ŝ for which P(s|t) is maximum, where s, t and ŝ are strings Is this a Bayesian problem case?? Noisy Channel ŝ 08/10/2004 - 15/10/2004 CS-621/CS-449 Lecture Notes