1. Event Ordering using TERSEO system Research Group on Language Processing and Information Systems g PLSI Estela Saquete Boró, Rafael Muñoz, Patricio Martinez-Barco Departamento de Lenguajes y Sistemas Informáticos

2. NLDB 20042 Research Group on Language Processing and Information Systems g PLSI 1.Introduction 2. Previous work 3. Description of the Event Ordering System 4. Application of Event Ordering in NLP tasks 5. System evaluation 6. Conclusions Index

3. NLDB 20043 Introduction Automatic processes to extract relevant information Event ordering using dates and time –Identification of temporal expressions –Resolution of temporal expression –Chronological order Research Group on Language Processing and Information Systems g PLSI

4. NLDB 20044 Introduction Research Group on Language Processing and Information Systems g PLSI Example: “Today is July the 3rd (2003). Tomorrow is my birthday” –Anaphoric expression: “Tomorrow” –Antecedent:July the 3rd (2003) –Referent:07/04/2003

5. NLDB 20045 Research Group on Language Processing and Information Systems g PLSI 1.Introduction 2. Previous work 3. Description of the Event Ordering System 4. Application of Event Ordering in NLP tasks 5. System evaluation 6. Conclusions Index

6. NLDB 20046 Previous work Types of systems: –Based on Machine Learning: A supervised annotated corpus needed to automatically generate the system rules (percentage of appearance). –High precision results with concrete domains –Not very flexible, large annotated corpus –Based on knowledge: Previous knowledge base with rules to solve temporal expressions. –Greater flexibility Our system  based on Spanish knowledge, but this knowledge is automatically extended using automatic acquisition of rules for new languages Research Group on Language Processing and Information Systems g PLSI

7. NLDB 20047 Research Group on Language Processing and Information Systems g PLSI 1.Introduction 2. Previous work 3. Description of the Event Ordering System 4. Application of Event Ordering in NLP tasks 5. System evaluation 6. Conclusions Index

8. NLDB 20048 Research Group on Language Processing and Information Systems g PLSI Graphic representation TEMPORAL INFORMATION DETECTION EVENT ORDERING DATE ESTIMATION Dictionary TEMPORAL EXPRESSION COREFERENCE RESOLUTION T.E. TAGS ORDERED TEXT Document Temporal Expression Detection Temporal Signal Detection ORDERING KEY OBTAINING ORDERING KEYS TEMPORAL EXPRESSIONS TEMPORAL SIGNALS

9. NLDB 20049 Detection of temporal information: –Temporal Expression Detection Unit –Temporal Signal Detection Unit Temporal expressions are resolved by the Temporal Expression Coreference Resolution unit that generates the XML tags. Ordering key is obtained by the Ordering Key unit With all this information, the Event Ordering Unit orders the text. Research Group on Language Processing and Information Systems g PLSI Description of the Event Ordering system

10. NLDB 200410 Detection of temporal information: –Temporal Expression Detection Unit –Temporal Signal Detection Unit Both share a common pre-processing of texts. Text are tagged with lexical and morphological information by a PosTagger and this information is the input of a temporal parser. The temporal parser is implemented using and ascending technique and it is based on a temporal grammar. Research Group on Language Processing and Information Systems g PLSI Description of the Event Ordering system

11. NLDB 200411 One of the main tasks involved in trying to recognize and resolve temporal expressions is to classify them. A taxonomy with two different classification of the temporal expressions has been established: –Classification of the expression based on the kind of reference –Classification by the representation of the temporal value of the expression Research Group on Language Processing and Information Systems g PLSI Temporal Expression Detection

12. NLDB 200412 Classification of the expression based on the kind of reference: –Explicit Temporal Expressions: –Complete dates with or without time exp:01/01/2003 –Dates of events: Christmas –Implicit Temporal Expressions: –Exp. that refer to the Document Date: yesterday –Exp. that refer to another Date: a month later Research Group on Language Processing and Information Systems g PLSI Taxonomy of TE´s

13. NLDB 200413 Taxonomy of TE´s Classification by the representation of the temporary value of the expression: –Concrete. Give back a concrete day or/and time –Period. Give back a time interval. –Fuzzy. Give back approximate time interval. –Fuzzy concrete: a day of the last week –Fuzzy period: some months before Research Group on Language Processing and Information Systems g PLSI

14. NLDB 200414 Temporal Signal Detection Temporal signals: –Relate the different events in texts –Establish a chronological order between these events. Some examples of Temporal signals: –After –Before –During –When –Previously –While –At the time of Research Group on Language Processing and Information Systems g PLSI

15. NLDB 200415 Temporal Expression Coreference Resolution: –Anaphoric relation resolution based on a temporal model –Tagging of Temporal Expressions Research Group on Language Processing and Information Systems g PLSI Description of the Event Ordering system

16. NLDB 200416 Research Group on Language Processing and Information Systems g PLSI Looking for antecedents. –Two main candidates: Newspaper´s date (DateP), Date named before in the text (DateAnt). –Proccess: By default, the newspaper´s date is used as a base referent if it exists. If a non-anaphoric TE is found, this is stored as DateAnt. Anaphoric Relation Resolution

17. NLDB 200417 Research Group on Language Processing and Information Systems g PLSI Anaphoric Relation Resolution REFERENCEDICTIONARY ENTRY ‘ayer’ (yesterday) ‘mañana’ (tomorrow) DateP – 1 DateP +1 ‘durante el mes siguiente’ (during the following month) [DayI/Month(DateAnt) +1/Year(DateAnt) -- DayF/Month(DateAnt)+1 /Year(DateAnt)] ‘un día antes’ (a day before) DateAnt-1 ‘días después’ (some days later) >>>>>DateAnt

18. NLDB 200418 Tagging of TEs Set of XML tags (eXtensible Markup Language). Targets: –Showing the results of our system –Standarise the date-time formats of Internet texts. Research Group on Language Processing and Information Systems g PLSI

19. NLDB 200419 Tagging of TEs Set of XML tags (eXtensible Markup Language). Explicit Dates < DATE_TIME ID =”value” TYPE=”value” VALDATE1=”value” VALTIME1=”value” VALDATE2=”value” VALTIME2=”value” > Expression Research Group on Language Processing and Information Systems g PLSI

20. NLDB 200420 Tagging of TEs Implicit dates < DATE_TIME_REF ID =”value” TYPE=”value” VALDATE1=”value” VALTIME1=”value” VALDATE2=”value” VALTIME2=”value” > Expression Research Group on Language Processing and Information Systems g PLSI

21. NLDB 200421 Ordering Keys Obtaining The study of the corpus revealed a set of temporal signals. Each temporal signal denotes a relationship between the dates of the events that it is relating. Example: in EV1 S EV2, the signal S denotes a relationship between EV1 and EV2. Assuming that F1 is the date of EV1 and F2 the date of EV2, S establish an order between EV1 and EV2. Research Group on Language Processing and Information Systems g PLSI

22. NLDB 200422 Ordering Keys Obtaining Research Group on Language Processing and Information Systems g PLSI SIGNALORDERING KEY AfterF1 > F2 WhenF1 = F2 BeforeF1 < F2 DuringF2i <= F1 <= F2f PreviouslyF1 > F2 On/inF1 = F2 WhileF2i <= F1 <= F2f ForF2i <= F1 <= F2f

23. NLDB 200423 Event ordering method Building of a table with the complete information from the XML tags –This table includes the columns ID, VALDATE1, VALTIME1, VALDATE2, VALTIME2 and VALORDER. Ordering rules: –EV1 is previous to EV2, if the range associated with TE1 is prior to and not overlapping the range associated with TE2 or the ordering key is EV1<EV2 –EV1 is concurrent to EV2, if the range associated with TE1 overlaps the range associated with TE2 or the ordering key is EV1=EV2 Research Group on Language Processing and Information Systems g PLSI

24. NLDB 200424 System example Research Group on Language Processing and Information Systems g PLSI In December 1, the French bathyscaphe Nautilus arrives at the Galician coast, previously there were some cracks. Text TEMPORAL INFORMATION DETECTION TEMPORAL EXPRESSION COREFERENCE RESOLUTION T.E. TAG: in December 1 ORDERING KEY OBTAINING ORDERING KEY: event 1 > event 2 TEMPORAL EXPRESSION: In December 1 TEMPORAL SIGNAL: previously EVENT ORDERING OrderEventDate 1There were some cracks<<< 12/01/2002 2The French bathyscaphe Nautilus arrives at the Galician Coast 12/01/2002

25. NLDB 200425 Research Group on Language Processing and Information Systems g PLSI 1.Introduction 2. Previous work 3. Description of the Event Ordering System 4. Application of Event Ordering in NLP tasks 5. System evaluation 6. Conclusions Index

26. NLDB 200426 Application of Event Ordering in NLP tasks Applied in different tasks: Summarization Question Answering Etc. Temporal Question Answering can help current QA system to answer complex questions. Complex questions consist of two or more events related with a temporal signal, which establish the order between them. Research Group on Language Processing and Information Systems g PLSI

27. NLDB 200427 Application in Question Answering Possible questions: When did Iraq invade Kuwait? When is the next New Hampshire Democratic primary? Which US ship was attacked by Israeli forces during the Six Day war in the sixties? Where did Bill Clinton study before going to Oxford University? Research Group on Language Processing and Information Systems g PLSI

28. NLDB 200428 Application in Question Answering Research Group on Language Processing and Information Systems g PLSI GENERAL PURPOSE QUESTION ANSWERING SYSTEM TEMPORAL Q. A. PROCESSING SCRIPT Q. A. PROCESSING TEMPLATE Q. A. PROCESSING.. Complex Question Simple QuestionsSimple Answers Complex Answer INTERFACE Multilayered Question Answering Architecture

29. NLDB 200429 Example of Application in Question Answering Research Group on Language Processing and Information Systems g PLSI Question: “Where did Bill Clinton study before going to Oxford University? First of all, the unit recognizes the temporal signal, which in this case is “before” Secondly, the complex question is divided: Q1: Where did Bill Clinton study? Q2: When did Bill Clinton go to Oxford University?

30. NLDB 200430 Example of Application in Question Answering Research Group on Language Processing and Information Systems g PLSI Answers Q1: Georgetown University (1964-1968) Oxford University (1968-1970) Yale Law School (1970-1973) Answers Q2: 1968 Only Georgetown University fulfill the temporal constrainst, so that is the answer to the complex question.

31. NLDB 200431 Research Group on Language Processing and Information Systems g PLSI 1.Introduction 2. Previous work 3. Description of the Event Ordering System 4. Application of Event Ordering in NLP tasks 5. System evaluation 6. Conclusions Index

32. NLDB 200432 Research Group on Language Processing and Information Systems g PLSI Corpus Spanish: Training (50 articles) and Test (50 articles) Kappa factor: measures the affinity in agreement between a set of annotators when they make categories judgments k=0.953 Two measures –Precision: Num Successes / Num Treated Ref –Recall: Num Successes / Num Real Ref System evaluation

33. NLDB 200433 Research Group on Language Processing and Information Systems g PLSI The establishment of a correct order between the events implies that the resolution is correct and the events are placed on a timeline. For this reason, an evaluation of the resolution of Temporal Expressions has been made. System evaluation -EVENT 1: Jan. 1, 1967 -EVENT 2: a year later -EVENT 3: two months before EVENTS AND ITS TEMPORAL EXPRESSIONS EV1EV3EV2 01/01/1967 01/01/196810/01/1967

34. NLDB 200434 System evaluation Research Group on Language Processing and Information Systems g PLSI SPANISH TRAININGTEST Num. Art.50 Real Ref.238199 Treated Ref.201156 Successes170138 Precision84.58%88.46% Recall71.43%69.35%

35. NLDB 200435 Research Group on Language Processing and Information Systems g PLSI Expressions like “el sábado hubo cinco accidentes” (Saturday there were five accidents) need context information of the sentence where the reference is, in this case, the time of the sentence´s verb. Our system does not use this information. There is not a world knowledge database, for instance: “two days before the Iraqi war”. We don´t have this information nowadays. System evaluation

36. NLDB 200436 Research Group on Language Processing and Information Systems g PLSI 1.Introduction 2. Previous work 3. Description of the Event Ordering System 4. Application of Event Ordering in NLP tasks 5. System evaluation 6. Conclusions Index

37. NLDB 200437 Obtaining facts related to an event from a Documental Database  Chronology. System: 1.Title of the news linked to the date of the documents 2.Recognition of temporal expressions. Events  sentences with TE 3.Module for treating TE is applied 4.The ordering module tags the order of the events in the text Research Group on Language Processing and Information Systems g PLSI Conclusions

38. NLDB 200438 Application in Temporal Question Answering: Decomposition of complex temporal questions in simple ones. Future work: Cope with context information and world knowledge Multilingual evaluation of the system Research Group on Language Processing and Information Systems g PLSI Conclusions

39. Event Ordering using TERSEO system Research Group on Language Processing and Information Systems g PLSI Estela Saquete Boró, Rafael Muñoz, Patricio Martinez-Barco Departamento de Lenguajes y Sistemas Informáticos