Naïve Bayes for WSD w: ambiguous word S = {s 1, s 2, …, s n } senses for w V = {v 1, v 2, v 3 } words used as contextual features for disambiguation We.

Slides:

Advertisements

Similar presentations

Classification Classification Examples

Advertisements

Tweet Classification for Political Sentiment Analysis Micol Marchetti-Bowick.

1 Essential Probability & Statistics (Lecture for CS598CXZ Advanced Topics in Information Retrieval ) ChengXiang Zhai Department of Computer Science University.

Naïve Bayes Classifier

Review: Bayesian learning and inference

Assuming normally distributed data! Naïve Bayes Classifier.

Bayes Rule How is this rule derived? Using Bayes rule for probabilistic inference: –P(Cause | Evidence): diagnostic probability –P(Evidence | Cause): causal.

Optimizing Text Classification Mark Trenorden Supervisor: Geoff Webb.

Distributional Clustering of Words for Text Classification Authors: L.Douglas Baker Andrew Kachites McCallum Presenter: Yihong Ding.

Collective Word Sense Disambiguation David Vickrey Ben Taskar Daphne Koller.

I256 Applied Natural Language Processing Fall 2009 Lecture 9 Review Barbara Rosario.

Generative Models Rong Jin. Statistical Inference Training ExamplesLearning a Statistical Model  Prediction p(x;  ) Female: Gaussian distribution N(

I256 Applied Natural Language Processing Fall 2009 Lecture 6 Introduction of Graphical Models Part of speech tagging Barbara Rosario.

I256 Applied Natural Language Processing Fall 2009 Lecture 7 Practical examples of Graphical Models Language models Sparse data & smoothing Barbara Rosario.

CS 561, Sessions 28 1 Uncertainty Probability Syntax Semantics Inference rules.

Crash Course on Machine Learning

Computer vision: models, learning and inference Chapter 6 Learning and Inference in Vision.

1 Persian Part Of Speech Tagging Mostafa Keikha Database Research Group (DBRG) ECE Department, University of Tehran.

ECE 8443 – Pattern Recognition LECTURE 06: MAXIMUM LIKELIHOOD AND BAYESIAN ESTIMATION Objectives: Bias in ML Estimates Bayesian Estimation Example Resources:

MATH 1107 Elementary Statistics Lecture 8 Random Variables.

SI485i : NLP Set 5 Using Naïve Bayes.

Naïve Bayes Classifier Ke Chen Modified and extended by Longin Jan Latecki

1 Logistic Regression Adapted from: Tom Mitchell’s Machine Learning Book Evan Wei Xiang and Qiang Yang.

Crowdsourcing with Multi- Dimensional Trust Xiangyang Liu 1, He He 2, and John S. Baras 1 1 Institute for Systems Research and Department of Electrical.

1 Statistical NLP: Lecture 9 Word Sense Disambiguation.

Warm-up During the early part of the 1994 baseball season, many sports fans and baseball players noticed that the number of home runs being hit seemed.

1 CS546: Machine Learning and Natural Language Discriminative vs Generative Classifiers This lecture is based on (Ng & Jordan, 02) paper and some slides.

Finding Scientific topics August , Topic Modeling 1.A document as a probabilistic mixture of topics. 2.A topic as a probability distribution.

Topic 1Topic 2Topic 3Topic 4Topic

CSSE463: Image Recognition Day 31 Today: Bayesian classifiers Today: Bayesian classifiers Tomorrow: project day. Tomorrow: project day. Questions? Questions?

Empirical Research Methods in Computer Science Lecture 7 November 30, 2005 Noah Smith.

URL:.../publications/courses/ece_8443/lectures/current/exam/2004/ ECE 8443 – Pattern Recognition LECTURE 15: EXAM NO. 1 (CHAP. 2) Spring 2004 Solutions:

Computational Intelligence: Methods and Applications Lecture 23 Logistic discrimination and support vectors Włodzisław Duch Dept. of Informatics, UMK Google:

Type your question here. Type Answer Type your question here. Type Answer.

Political Parties and the US Political System

Mehdi Ghayoumi MSB rm 132 Ofc hr: Thur, a Machine Learning.

KNN & Naïve Bayes Hongning Wang Today’s lecture Instance-based classifiers – k nearest neighbors – Non-parametric learning algorithm Model-based.

Berendt: Advanced databases, winter term 2007/08, 1 Advanced databases – Inferring implicit/new.

Problem Set 2 Problem #6 By: Zac Snively. Question Presented  In a league that has 6 teams, every team plays every one of the other teams twice in a.

Principles of Macroeconomics Day 5 Dr. Andrew L. H. Parkes “A Macroeconomic Understanding for use in Business” 卜安吉.

MATH Section 7.4 Pt. 2. Recall: Look at the following example: The effect of exercise on the amount of lactic acid in the blood was examined.

Topic 5 Topic 1 Topic 2 Topic 3 Topic

In games of chance the expectations can be thought of as the average outcome if the game was repeated multiple times. Expectation These calculated expectations.

Introduction to Information Retrieval Introduction to Information Retrieval Lecture 15: Text Classification & Naive Bayes 1.

KNN & Naïve Bayes Hongning Wang

Text Classification and Naïve Bayes Formalizing the Naïve Bayes Classifier.

Perceptrons Lirong Xia.

Type Topic in here! Created by Educational Technology Network

Lecture 15: Text Classification & Naive Bayes

Type Topic in here! Created by Educational Technology Network

Klein and Manning on CRFs vs CMMs

Statistical NLP: Lecture 9

CSSE463: Image Recognition Day 27

CHAPTER 7 BAYESIAN NETWORK INDEPENDENCE BAYESIAN NETWORK INFERENCE MACHINE LEARNING ISSUES.

Learning to Classify Documents Edwin Zhang Computer Systems Lab

CSSE463: Image Recognition Day 31

LECTURE 23: INFORMATION THEORY REVIEW

CSSE463: Image Recognition Day 27

LECTURE 07: BAYESIAN ESTIMATION

Type Topic in here! Created by Educational Technology Network

Type Topic in here! Created by Educational Technology Network

Type Topic in here! Created by Educational Technology Network

Type Topic in here! Created by Educational Technology Network

Add Main Topic Here Created by Educational Technology Network

Type Topic in here! Created by Educational Technology Network

Logistic Regression [Many of the slides were originally created by Prof. Dan Jurafsky from Stanford.]

Type Topic in here! Created by Educational Technology Network

Type Topic in here! Created by Educational Technology Network

Perceptrons Lirong Xia.

Statistical NLP : Lecture 9 Word Sense Disambiguation

Presentation transcript:

Naïve Bayes for WSD w: ambiguous word S = {s 1, s 2, …, s n } senses for w V = {v 1, v 2, v 3 } words used as contextual features for disambiguation We use the following models for WSD

Generative vs. discriminative Write the joint probability distribution for the two GM Estimation (Training): estimation of the following probability distributions: PUT YOUR ANSWER HERE Inference (Testing): consist in computing the following Conditional probability distribution: PUT YOUR ANSWER HERE v1v1 v2v2 v3v3 sksk v1v1 v2v2 v3v3 sksk GenerativeDiscriminative Estimation (Training): estimation of the following probability distributions: PUT YOUR ANSWER HERE Inference (Testing): consist in computing the following Conditional probability distribution: PUT YOUR ANSWER HERE

Naïve Bayes for topic classification w1w1 w2w2 wnwn T Recall the general joint probability distribution: P(X 1,..X N ) =  i P(X i | Par(X i ) ) P(T, w 1..w n ) = P(T) P(w 1 | T) P(w 2 | T) … P(w n | T )= = P(T)  i P(w i | T) Inference (Testing): Compute conditional probabilities: P(T | w 1, w 2,..w n ) Estimation (Training): Given data, estimate: P(T), P(w i | T)

Topic = sport (num words = 16) D1: 2009 open season D2: against Maryland Sept D3: play six games seasons D3: schedule games weekends D4: games games games Exercise: estimation Topic = politics (num words = 20) D1: Obama hoping rally support D2: billion stimulus package D3: House Republicans tax D4: cuts spending GOP games D4: Republicans obama open D5: political season senate P(obama | T = politics) = HERE (whole formula and numerical results) P(obama | T = sport) = HERE ((whole formula and numerical results) P(season | T=politics) = HERE (only results) P(season | T= sport) = HERE (only results) Estimate: for each w i, T j

Exercise P(republicans|T=politics)= HERE P(republicans|T= sport)= HERE P(games|T=politics)= HERE P(games|T= sport)= HERE P(season|T=politics)= HERE P(season|T= sport)= HERE P(open|T=politics)= HERE P(open|T= sport)= HERE P(T=politics)= HERE P(T= sport)= HERE

Exercise: inference What is the topic of new documents: –Republicans obama season –games season open –democrats kennedy

Exercise: Bayes classification We compute P(T j | c) with Bayes rule Because of the dependencies encoded in the GM Bayes rule This GM

Exercise: Bayes classification New sentences: games season open T = politics? P(politics I c) = PUT YOU ANSWER HERE T = sport? P(sport I c) = PUT YOU ANSWER HERE That is, for each T j we calculate and see which one is higher Choose T = PUT YOU ANSWER HERE

Exercise: Bayes classification New sentences: republicans obama season T = politics? P(politics I c) = PUT YOU ANSWER HERE T = sport? P(sport I c) = PUT YOU ANSWER HERE That is, for each T j we calculate and see which one is higher Choose T = PUT YOU ANSWER HERE

Exercise: Bayes classification New sentences: democrats kennedy T = politics? P(politics I c) = PUT YOU ANSWER HERE T = sport? P(sport I c) = PUT YOU ANSWER HERE That is, for each T j we calculate and see which one is higher Choose T = PUT YOU ANSWER HERE