CS : NLP, Speech and Web-Topics-in-AI Pushpak Bhattacharyya CSE Dept., IIT Bombay Lecture 34: Precision, Recall, F- score, Map

2 Web at a glance Google indexes more than 8 billion pages Dominated by English Large part of world is deprived of this knowledge

3 Search Engines Today Keyword based Irrelevant results Meaning not taken into account Language specific No search possible across language No translation possible

IR FRAMEWORK QUERY doc 1 doc 2.. doc n

SET VIEW List of Identified Entities List of Oblique Entities I O I O I ∩ O

MEASURES Precision, P = | I ∩ O | / | O | Recall, R = | I ∩ O | / | I | F-Score = 2 (P*R) / (P + R) F β Score = (β * P * R) / (1 + β * R)

OBSERVATION F-Score is a Harmonic mean of Precision and Recall HM < AM < GM Improvement in HM leads to improvement in GM and AM If no entity is left out without assigning label to it then Precision = Recall = F-Score

RANKED LISTS Let I and O be ranked lists. Measures:- k = precision form list going down up to k MAP Value = (1/N) * k=1 ∑ k=N K

WHAT ABOUT? K = recall at K Precision at Recall R Recall at Precision P