Presentation is loading. Please wait.

Presentation is loading. Please wait.

CMU TDT Report 12-13 November 2001 The CMU TDT Team: Jaime Carbonell, Yiming Yang, Ralf Brown, Chun Jin, Jian Zhang Language Technologies Institute, CMU.

Published byValentine McCarthy Modified over 8 years ago

Similar presentations

Presentation on theme: "CMU TDT Report 12-13 November 2001 The CMU TDT Team: Jaime Carbonell, Yiming Yang, Ralf Brown, Chun Jin, Jian Zhang Language Technologies Institute, CMU."— Presentation transcript:

1 CMU TDT Report 12-13 November 2001 The CMU TDT Team: Jaime Carbonell, Yiming Yang, Ralf Brown, Chun Jin, Jian Zhang Language Technologies Institute, CMU

2 Time Line for TDT Activities (Re)Start: Summer 2001 Baseline FSD, Link, Det: Sept 2001 Evaluation (of baseline): Oct 2001 New Techniques: Nov 2001 – Onwards Topic-conditional Novelty Situated NE’s (all tasks) Source-conditional interpolated training

3 Baseline FSD Method (Unconditional) Dissimilarity with Past Decision threshold on most-similar story (Linear) temporal decay Length-filter (for teasers) Cosine similarity with standard weights:

4 FSD Results Story weighted Topic weighted P(miss).6028 P(F/A).0207.0186 Cost.0141.0143 Norm Cost.7043.7217 Opt N. Cost.6807

5 Comparative FSD DET Curves

6 FSD Observations Cross-site comparable baselines (cost =.7) Data/labeling issues (from error analysis) “Events-vs-Topics” issue (e.g. Asia crisis) A few mislabled stories wreak havoc for FSD Eager auto-segmentation a problem (misses) Recommendations for TDT labeling FSD on true events, or events within topic(s) Change auto-segmentation optimality criterion ?? Recommendations for TDT reserachers Keep working hard on FSD – not cracked yet

7 New FSD Directions Topic-conditional models E.g. “airplane,” “investigation,” “FAA,” “FBI,” “casualties,”  topic, not event “TWA 800,” “March 12, 1997”  event First categorize into topic, then use maximally-discriminative terms within topic Rely on situated named entities E.g. “Arcan as victim,” “Sharon as peacemaker ”

8 A New Approach to First Story Detection for TDT

9 Baseline Story-Link Detection Use same term-weighting and cosine similarity as FSD and detection Decision Thresholds conditioned on language and source Lower threshold for cross-language Lower threshold cross-ASR/newswire Thresholds trained on development set 15% improvement over universal threshold

10 Primary Link

11 CMU Link

12 CMU2 Link

13 CMU Detection Auto-segmented boundaries Pre-established boundaries C det (basic).0076.0063 C det (norm).3786.3138 Incremental Retrospective Clustering Group-Average in Forward Deferral Window Same cosine similarity and terms weight as FSD

Download ppt "CMU TDT Report 12-13 November 2001 The CMU TDT Team: Jaime Carbonell, Yiming Yang, Ralf Brown, Chun Jin, Jian Zhang Language Technologies Institute, CMU."

Similar presentations

DISCOVERING EVENT EVOLUTION GRAPHS FROM NEWSWIRES Christopher C. Yang and Xiaodong Shi Event Evolution and Event Evolution Graph: We define event evolution.

DISCOVERING EVENT EVOLUTION GRAPHS FROM NEWSWIRES Christopher C. Yang and Xiaodong Shi Event Evolution and Event Evolution Graph: We define event evolution.
Supervised Learning Techniques over Twitter Data Kleisarchaki Sofia.

Supervised Learning Techniques over Twitter Data Kleisarchaki Sofia.
TASK PROCEDURE EVALUATION CONCLUSION TEACHER’S PAGE INTRODUCTION.

TASK PROCEDURE EVALUATION CONCLUSION TEACHER’S PAGE INTRODUCTION.
Proactive Learning: Cost- Sensitive Active Learning with Multiple Imperfect Oracles Pinar Donmez and Jaime Carbonell Pinar Donmez and Jaime Carbonell Language.

Proactive Learning: Cost- Sensitive Active Learning with Multiple Imperfect Oracles Pinar Donmez and Jaime Carbonell Pinar Donmez and Jaime Carbonell Language.
Probabilistic Generative Models Rong Jin. Probabilistic Generative Model Classify instance x into one of K classes Class prior Density function for class.

Probabilistic Generative Models Rong Jin. Probabilistic Generative Model Classify instance x into one of K classes Class prior Density function for class.
Systems Engineering and Engineering Management The Chinese University of Hong Kong Parameter Free Bursty Events Detection in Text Streams Gabriel Pui Cheong.

Systems Engineering and Engineering Management The Chinese University of Hong Kong Parameter Free Bursty Events Detection in Text Streams Gabriel Pui Cheong.
UMass Amherst at TDT 2003 James Allan, Alvaro Bolivar, Margie Connell, Steve Cronen-Townsend, Ao Feng, FangFang Feng, Leah Larkey, Giridhar Kumaran, Victor.

UMass Amherst at TDT 2003 James Allan, Alvaro Bolivar, Margie Connell, Steve Cronen-Townsend, Ao Feng, FangFang Feng, Leah Larkey, Giridhar Kumaran, Victor.
Techniques for Event Detection Kleisarchaki Sofia.

Techniques for Event Detection Kleisarchaki Sofia.
BuzzTrack Topic Detection and Tracking in IUI – Intelligent User Interfaces January 2007 Keno Albrecht ETH Zurich Roger Wattenhofer.

BuzzTrack Topic Detection and Tracking in IUI – Intelligent User Interfaces January 2007 Keno Albrecht ETH Zurich Roger Wattenhofer.
Alan Hamlet Dennis P. Lettenmaier JISAO Climate Impacts Group and the Department of Civil Engineering University of Washington March, 2001 CIG Water Resources.

Alan Hamlet Dennis P. Lettenmaier JISAO Climate Impacts Group and the Department of Civil Engineering University of Washington March, 2001 CIG Water Resources.
2010.03.17 - SLIDE 1IS 240 – Spring 2010 Prof. Ray Larson University of California, Berkeley School of Information Principles of Information Retrieval.

SLIDE 1IS 240 – Spring 2010 Prof. Ray Larson University of California, Berkeley School of Information Principles of Information Retrieval.
Decision Trees for Error Concealment in Video Decoding Song Cen and Pamela C. Cosman, Senior Member, IEEE IEEE TRANSACTION ON MULTIMEDIA, VOL. 5, NO. 1,

Decision Trees for Error Concealment in Video Decoding Song Cen and Pamela C. Cosman, Senior Member, IEEE IEEE TRANSACTION ON MULTIMEDIA, VOL. 5, NO. 1,
1 LM Approaches to Filtering Richard Schwartz, BBN LM/IR ARDA 2002 September 11-12, 2002 UMASS.

1 LM Approaches to Filtering Richard Schwartz, BBN LM/IR ARDA 2002 September 11-12, 2002 UMASS.
A Comparative Study on Feature Selection in Text Categorization (Proc. 14th International Conference on Machine Learning – 1997) Paper By: Yiming Yang,

A Comparative Study on Feature Selection in Text Categorization (Proc. 14th International Conference on Machine Learning – 1997) Paper By: Yiming Yang,
Machine Learning Queens College Lecture 1: Introduction.

Machine Learning Queens College Lecture 1: Introduction.
Macquarie RT05s Speaker Diarisation System Steve Cassidy Centre for Language Technology Macquarie University Sydney.

Macquarie RT05s Speaker Diarisation System Steve Cassidy Centre for Language Technology Macquarie University Sydney.
The Evolution of Shared-Task Evaluation Douglas W. Oard College of Information Studies and UMIACS University of Maryland, College Park, USA December 4,

The Evolution of Shared-Task Evaluation Douglas W. Oard College of Information Studies and UMIACS University of Maryland, College Park, USA December 4,
Topic Detection and Tracking Introduction and Overview.

Topic Detection and Tracking Introduction and Overview.
Machine Learning CUNY Graduate Center Lecture 1: Introduction.

Machine Learning CUNY Graduate Center Lecture 1: Introduction.
Data Mining Joyeeta Dutta-Moscato July 10, 2013. Wherever we have large amounts of data, we have the need for building systems capable of learning information.

Data Mining Joyeeta Dutta-Moscato July 10, Wherever we have large amounts of data, we have the need for building systems capable of learning information.

Similar presentations

Ads by Google