We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byJoe Hindsley
Modified over 2 years ago
NRRC Summer ‘02 Workshop Proposal Temporal December 5, 2001 MITRE James Pustejovsky, Brandeis with input from: James Allan, UMASS Inderjeet Mani, MITRE Janet Hitzeman, MITRE John Frank, MetaCarta Erik Rauch, MetaCarta
Page 2 Copyright © 2001 The MITRE Corporation. All rights reserved. Problem Find most accurate time reference for events in news with the goal of answering temporally-based questions related to those events Sample questions: -Is Gates currently CEO of Microsoft? -Were there any meetings between the terrorist hijackers and Iraq before the WTC event? -Did the Enron merger with Dynegy take place?
Page 3 Copyright © 2001 The MITRE Corporation. All rights reserved. Answers: May be selections from Database entries May be textual fragments from articles May be abstracted and represented visually or graphically in a timeline or hyperbolic graph E2E2 E3E3 E1E1 Were there any meetings(e2,e1) between terrorist hijackers and Iraq before the WTC event (e3)?
Page 4 Copyright © 2001 The MITRE Corporation. All rights reserved. Workshop Goals: TenseML: To Define and Design a Metadata Specification for Markup of events, their temporal anchoring, and how they are related to each other in News text. Algorithm Design: Given the definition of TenseML, to design and develop algorithms for extracting such data from news texts.
Page 5 Copyright © 2001 The MITRE Corporation. All rights reserved. Preliminary Tasks: Time stamping events (identifying an event and anchoring it in time); Ordering events with respect to each other (relating more than one event in terms of precedence, overlap, and inclusion); Temporal and event anaphora Reasoning about the ramifications of an event (what is changed by virtue of an event); Reasoning about the persistence of an event (how long does an event or the outcome of an event persist).
Page 6 Copyright © 2001 The MITRE Corporation. All rights reserved. TenseML: DTD for event types (state, process, transition) - adopt a convention of how to tag events and aspect DTD for temporal types - Intervals, points, scales DTD for tense relations of precedence, overlap, inclusion, and identity DTD for persistence and ramification consequences. Basic Semantics for common English temporal expressions
Page 7 Copyright © 2001 The MITRE Corporation. All rights reserved. TIMEBANK Gold Standard Create Gold standard -Human annotated table of temporally grounded events and temporal expressions: articles tagged according to the markup language defined by workshop participants Annotation of events, temporal references, and relative event orderings: -Specification to be determined -Draft versions in Mani and Wilson (2000) -Ferro, Mani, Sundheim, and Wilson (2001) -Filatova and Hovy (2001) -Ferro (2001) -Setzer and Gaizauskas (2001)
Page 8 Copyright © 2001 The MITRE Corporation. All rights reserved. Domain and Data sets Project is likely to use the news domain Possible data sets: -Reuters Newswire source (in NewsML and CCS) -Factiva Corpus -TDT -TIDES -TIDES TDT-2 temporal corpus
Page 9 Copyright © 2001 The MITRE Corporation. All rights reserved. Algorithm Design Evaluated and Develop algorithms for extracting event and time expressions Evaluate and Develop algorithms for question answering for event and time expressions
Page 10 Copyright © 2001 The MITRE Corporation. All rights reserved. Evaluation Methods Accuracy of event/time table generation -Against gold standard Accuracy of answers for questions -NB: Assume question classification already known -Against gold standard (real answer) -Degradation from gold standard to system output Evaluation will be continuous throughout workshop with true blind final exam at the end Evaluations will be performed by the participants
Page 11 Copyright © 2001 The MITRE Corporation. All rights reserved. Work Plan and Schedule Mid-January: Kickoff Meeting: 1 week -Begin define and design of specification: annotation March: 3-day meeting -Assess annotation effort April: 1 week meeting -Test and extend existing algorithms May: 1 week meeting -Deploy algorithms June: 1 week meeting -Integration July: 1 week meeting (Close up and evaluation) -Evaluation and Wrap-up
Page 12 Copyright © 2001 The MITRE Corporation. All rights reserved. Proposed Team Membership Core Members -Robert Ingria, LingoMotors -Dragomir Radev, U. Michigan -Janet Hitzeman, MITRE -John Moynes, Hapax -Jose Castano, Brandeis -James Allen, Rochester -Rob Gaizauskas, Sheffield Leader: James Pustejovsky, Brandeis Additional Participants -Lisa Ferro, MITRE -Inderjeet Mani, MITRE -Antonio Sanfilippo, SRA -Len Schubert, Rochester
Page 13 Copyright © 2001 The MITRE Corporation. All rights reserved. Impact Better understanding of limits of current technology Preliminary capabilities for answering questions using that output Groundwork for event ordering capabilities Specification for Temporal and Event Markup Gold Standard Corpus that can be used by anyone in the community (TIMEBANK) Add a new dimension to the kinds of Q&A possible
Page 14 Copyright © 2001 The MITRE Corporation. All rights reserved. Resources and Cost Brought by the team -Expertise in Event Representation and Reasoning -Event Extraction Capabilities (LingoMotors) -TEMPEX time annotation (MITRE) -ACE specifications Costs -Participants’ salary -Corpora, likely to be quite cheap -TenseML Annotation, from LDC or local annotators -Computers and storage -Coordination help -Travel, for the separate meetings
A Database of Narrative Schemas A 2010 paper by Nathaniel Chambers and Dan Jurafsky Presentation by Julia Kelly.
1/(19) GATE Evaluation Tools GATE Training Course October 2006 Kalina Bontcheva.
NRRC Summer ‘02 Workshop Proposal: Habitability December 5, 2001 MITRE Christy Doran, MITRE Joe Marks, MERL
NRRC Summer ‘02 Workshop Proposal Re-use November 30, 2001 MITRE Marc Light and Abraham Ittycheriah.
Temporal Ordering of Events in the News Domain Preethi Raghavan.
1 End-User Programming to Support Classroom Activities on Small Devices Craig Prince University of Washington VL/HCC 2008.
1 Temporal Information Extraction Inderjeet Mani
Ad Hoc Bulk Electric System Task Force Update RPIC February 19, 2009.
Indirect Supervision Protocols for Learning in Natural Language Processing II. Learning by Inventing Binary Labels This work is supported by DARPA funding.
1 North Carolina Department of Public Instruction Fran Hoch & Hope Tesh Accountability Conference 2002 Limited English Proficiency Policy Changes North.
ACE Automatic Content Extraction A program to develop technology to extract and characterize meaning from human language.
1 Open Ontology Repository initiative - Planning Meeting - Thu Co-conveners: PeterYim, LeoObrst & MikeDean ref.:
1/(13) Using Corpora and Evaluation Tools Diana Maynard Kalina Bontcheva
Erasmus University Rotterdam Introduction With the vast amount of information available on the Web, there is an increasing need to structure Web data in.
TimeML compliant text analysis for Temporal Reasoning Branimir Boguraev and Rie Kubota Ando.
Exploring the Effectiveness of Lexical Ontologies for Modeling Temporal Relations with Markov Logic Eun Y. Ha, Alok Baikadi, Carlyle Licata, Bradford Mott,
Numerical Libraries Project Microsoft Incubation Group Mary Beth Hribar Microsoft Corporation CSCAPES Workshop June 10, 2008 Copyright Microsoft Corporation,
Splitting Complex Temporal Questions for Question Answering systems ACL 2004.
1 OOA-HR Workshop, 11 October 2006 Semantic Metadata Extraction using GATE Diana Maynard Natural Language Processing Group University of Sheffield, UK.
HathiTrust and Print Storage Building around a digital core.
Cultural Heritage in REGional NETworks REGNET. July 2002Project presentation REGNET 2 WP 7 - Management Task 7.1 – Project Management Task 7.2 – Quality.
Jing-Shin Chang National Chi Nan University, IJCNLP-2013, Nagoya 2013/10/15 ACLCLP – Activities ( ) & Text Corpora.
Assessment Workshop Title of the Project (date). Project Title Assessment Workshop October 25, 2015© Company Name All rights reserved2 Agenda Purpose.
Seminar for Teacher Assistants Liberal Studies Public Assessment.
A Probabilistic Representation of Systemic Functional Grammar Robert Munro Department of Linguistics, SOAS, University of London.
1 Establishing Performance Indicators in Support of The Illinois Commitment Presented to the Illinois Board of Higher Education December 11, 2001.
1 Copyright © 2010, Elsevier Inc. All rights Reserved Fig 3.1 Chapter 3.
10 Systems Analysis and Design in a Changing World, Fifth Edition.
A Flexible and Extensible Architecture for Linguistic Annotation Steven Bird *, David Day †, John Garofolo ‡, John Henderson †, Christophe Laprun ‡ and.
10 Chapter 10: The Traditional Approach to Design Systems Analysis and Design in a Changing World, 3 rd Edition.
© NCSR, Paris, December 5-6, 2002 WP1: Plan for the remainder (1) Ontology Ontology Enrich the lexicons for the 1 st domain based on partners remarks.
Copyright © Cengage Learning. All rights reserved. 1 Functions and Applications.
Copyright © The OWASP Foundation Permission is granted to copy, distribute and/or modify this document under the terms of the OWASP License. The OWASP.
Data Documentation Initiative (DDI) Workshop Carol Perry Ernie Boyko April 2005 Kingston Ontario.
1 Technologies for (semi-) automatic metadata creation Diana Maynard.
2011 ITRS Revision Guidelines1 JAN – MARCH – ITRS TABLES DRAFTS PREPARED –ITWGs and CrossTWG Study Group meet each month with their teams to work Difficult.
Restricted © Siemens AG All rights reserved Siemens Corporate Technology | Month 20XX Proposed topics for TDL phase 3.
Multilinguality & Semantic Search Eelco Mossel (University of Hamburg) Review Meeting, January 2008, Zürich.
HONG KONG EXAMINATIONS AND ASSESSMENT AUTHORITY 2007 HKCE English Language Examination Criteria and Standards for the SBA Component HKedCity Seminar, 10.
1 New Mexico Standards-Based Assessment New Mexico Standards-Based Assessment New Mexico Standards-Based Assessment.
1 An introduction to the TempEx Module George Wilson MITRE Corporation January 30, 2002.
1 The OneGeology project IC GS Ian Jackson, February 2007.
Role of Account Management at ERCOT 2008 ERCOT Market Participant Survey February 12, 2008.
2/12/01 Professor Richard Fikes Representing Time Computer Science Department Stanford University CS222 Winter 2001 Knowledge Systems Laboratory, Stanford.
WP 9 training2 WP 9 TRAINING Kick-off Meeting – Valkenburg 9 December 2005 dr.Marlies Bitter-Rijpkema, Open Universiteit Nederland.
Customer Relationship Management (CRM) w/ Registrar’s Office (RO) Colin Bell Director, Enterprise Architecture, IST January 9, 2014.
Copyright © 2010 American Institutes for Research All rights reserved. Washington 21st Century Community Learning Centers Program Evaluation – April Update.
THE STANDARDS DEVELOPMENT PROCESS STEP 1 PUBLIC AND COMMITTEE PROPOSAL STAGE PUBLIC AND COMMITTEE PROPOSAL CLOSING DATE FIRST TECHNICAL COMMITTEE MEETING.
Project Parkway Timeline to 2011 April 1, 2010 Town Hall Meeting Steering Committee Schools February 2009 May 18, 2009 Project Parkway Community Kick-
Washington, / Economical Damage on EM-DAT Page 1 Center for Research on the Epidemiology of Disasters (CRED) ECONOMIC DATA IN EM-DAT: OVERVIEW.
© 2017 SlidePlayer.com Inc. All rights reserved.