1. Information Extraction Language Technology (A Machine Learning Approach) 24 March 2005 Antal van den Bosch and Walter Daelemans http://ilk.uvt.nl/~antalb/ltua

2. What is Information Extraction? •Input: unstructured text •Output: structured information, fills pre- existing template (find salient information) •Most often stored in database for futher processing (e.g. data mining)

3. What isn’t information extraction? •Information retrieval (we need to extract info, not only find relevant documents) •Text understanding (only specific parts of the text are interesting) –large corpora can be used –possible to score objectively

4. Applications of IE •Can make information retrieval more precise •Summarization of documents in well- defined subject areas •Automatic generation of databases from text

5. Overview •Named entity recognition –Recognizing relevant entities in text •Relation extraction –Linking recognized entities having particular relevant relations

6. De door het Amerikaanse National Hurricane Center als 'zeer gevaarlijk' omschreven orkaan Ivan nadert Cuba. Een overzicht over wat Ivan op de Kaaimaneilanden heeft aangericht, is er nog niet. Gouverneur Bruce Dinwiddy zei maandag dat duizenden mensen dakloos zijn geworden en dat ook belangrijke regeringsgebouwen zijn getroffen. Named Entity Recognition (NER) is a combination of concept chunking and labeling those chunks: we wish to identify textual information units that represent people, places, organizations, companies, bands, etc. Named Entity Recognition

7. NER has many applications Why NER? • prerequisite for information extraction • improving information retrieval  indexing  querying “ belvedere ”

8. Intuitively simple? What ’ s the problem? NER seems intuitively simple for humans. How do we determine whether or not a (string of) word(s) represents a name? • does the word start with a capital letter? (orthographic characteristics) • have we seen it before? (lists of names) • contextual clues How do we teach this to a computer?

9. Some problems … Problems: • not every word that starts with a capital letter is a name ex: “Soms is dat niet mogelijk...” • context can be misleading ex: “Er was geen land met Henk te bezeilen.” • no list can ever be complete ex: “Antbeard en zijn bemanning voeren...” ex: “Wil je wat te drinken?”

10. Feature extraction A lot of different features can be extracted for use in (inductively) learning to classify NEs. Every word can be represented with a lot of different features: “… bedrijf dat Floralux inhuurde. In ‘ 81 …” starts w/ capital letter? YES first word o/t sentence? NO contains punctuation? NO string length? 8

11. Feature extraction (2) We represent the context by sliding a ‘ window ’ over the data which is anchored in the focus word. “… het bedrijf dat Floralux inhuurde. In ‘ 81 bestond …” left context right context focus word “… het bedrijf dat Floralux inhuurde. In ‘ 81 bestond …”

12. To split or not to split? Wolff, op het moment een journalist in Argentini ë, speelde met Del Bosque bij Real Madrid in de jaren 70. determine boundaries + types Wolff, op het moment een journalist in Argentini ë, speelde met Del Bosque bij Real Madrid in de jaren 70. determine boundaries Wolff, op het moment een journalist in Argentini ë, speelde met Del Bosque bij Real Madrid in de jaren 70. determine types Wolff, op het moment een journalist in Argentini ë, speelde met Del Bosque bij Real Madrid in de jaren 70.

13. State of the art STATE OF THE ART F-score English~93% Dutch~77% German~72% Spanish~81% Human performance96-98% Lots of other different languages have been targeted as well: Chinese, French, Japanese, Portuguese, Greek, Hindi, Rumanian, Turkish, Norwegian, and so on…

14. Information extraction •Named-entity recognition •Relation extraction •Coreference resolution –PvdA-leider Wouter Bos gaat alleen voor minister-president. Vice-premier onder CDA-leider Balkenende is geen gedachte waar hij warm van wordt. "Hou het er maar op dat ik daar nee tegen zeg", aldus Bos woensdag voor RTL Nieuws.

15. Information extraction •Named-entity recognition has received a lot of attention in IE •Relation extraction is taking over as focal point of attention

16. Relation extraction Eric Schmidt is directeur van Google. N N WW N VZ N. | ----- PER ---- | | - ORG - | Example directeur

17. Why relation extraction? •Named entities can be useful to enhance information retrieval •Not enough to answer certain types of information-seeking questions •For example –Wie is de directeur van Google?

18. Why relation extraction? •Naïve strategy –Find documents in which [PER ] and [ORG Google ] are within each other’s vicinity –Can produce nice results, but does not always work –Also, user still has to find answer –It would be better if the system produced the answer 'Eric Schmidt'.

19. Examples •Some application areas are –News domain •Relations among the most typical named entities: Person, Organisation, Location, Misc •E.g. located in, parent of, part of –Biomedical domain •Relations among biomedical entities, such as DNA, proteins, diseases, etc. •Protein-protein interaction •Gene-disease relation •Every domain-specific application needs its own set of entities

20. Relation extraction •Difficult –Automatic systems still perform poorly –But a few reasonable solutions •Often only works in restricted domains –Techniques operating in the news domain are lousy in other domains, e.g. biomedical texts

21. Relations: implicit / explicit •Explicit relations are spelled out –Joe Cummings, Chairman of Sybase, spoke for four hours. •Implicit relations imply understanding a text –Sybase was scheduled to testify, and Chairman Joe Cummings spoke for four hours. •Most current research involves explicit relations

22. Difficulties •A relation can be phrased in many ways –Eric Schmidt is de directeur van Google. –Eric Schmidt, de nieuwe directeur van Google, verklaart... –Eric Schmidt zet een volgende stap in zijn carriere. Sinds kort is hij de directeur van Google. –...

23. Assumptions •Delimit the task –Relations always connect two named entities •More complex relations between >2 entities are harder –Both entities are in the same sentence •Strong simplification •See next week (Veronique Hoste guest lecture)

24. Relation extraction •Relation detection –Is there a relation between two entities? •Relation classification –Which type does the relation between two entities have?

25. MUC •Message Understanding Conference •Has organized many information extraction competitions •Since 1998, relation extraction is a MUC competition

26. ACE •Automatic Content Extraction •More recent than MUC •The ACE data is the most popular data set for relation extraction research

27. ACE •Types/Subtypes relations –ROLE •Relates a person to an organisation or geopolitical entity •member, owner, affiliate, client, citizen –PART •Generalised containment •subsidiary, physical part-of, set membership –AT •permanent and transient locations •located, based-in, residence –SOC •social relations among persons •parent, sibling, spouse, grandparent, associate

28. Automatic RE: Pipeline •Relation extraction finds relations among pairs of named entities •Assuming that named entities have already been identified •Simple case of a pipeline, a heavily used architecture in language technology

29. Pipeline Tokeniser Sentence splitter Part-of-speech tagger Syntactic parser Named-entity recogniser Relation extractor Information

30. Pipeline •Parts of a pipeline are dependent on what is done before them •A weak point of the pipeline architecture is that errors tend to propagate as snowballs

31. •Eric Schmidt is directeur van Google. ● [PER Eric Schmidt ] werkzaam-bij [ORG Google ] •Jan de Vries is vakkenvuller bij Albert Heijn. ● [PER Jan de Vries ] werkzaam-bij [ORG Albert Heijn ]

32. •PER is directeur van ORG. •PER is vakkenvuller bij ORG.

33. •PER is directeur PREP ORG. •PER is vakkenvuller PREP ORG.

34. •PER is directeur PREP ORG. •PER is vakkenvuller PREP ORG. •Jan de Vries is fan van PSV. –PER is fan PREP ORG. !

35. directeur vakkenvuller fan Semantic lexicon (e.g. WordNet) portier accountant liefhebber bewonderaar...

36. [PER Eric Schmidt ] werkzaam-bij [ORG Google ]

37. [PER Jan de Vries ] werkzaam-bij [ORG Albert Heijn ]

38. Similar?

39. PER ↑smain-su ↓smain-predc np-mod pp-obj1 ORG

40. Evaluatie •Comparable to text classification and named entity recognition –Precision •Number of correctly predicted relations / Total number of predicted relations –Recall •Number of correctly predicted relations / Total number of relations in the text –F-score •2 * precision * recall / (precision + recall)