MUMIS User Group Workshop P. Wittenburg Max-Planck-Institut für Psycholinguistik Nijmegen

Agenda 11.00IntroductionPeter Wittenburg ASR AspectsHelmer Strik NLP/IE AspectsThierry Declerck 11.45Demo of PrototypePeter Wittenburg Lunchbreak 13.15Potential of MUMIS TechnologyDiscussion 14.00Typical QuestionsDiscussion 15.00Other Points of InterestDiscussion 15.30End

Objectives MUMIS is about Technology development to automatically index lengthy multimedia recordings Find and mark interesting scenes Technology development to exploit indexed multimedia archives Search for interesting scenes and play them via Internet Test Domain: Soccer Games / UEFA Tournament 2000

Off-line Tasks Indexing by Automatic Speech Recognition (Radio/TV Broadcasts) Transfer the speech signals into texts automatically Natural Language Processing Take all texts (newspapers, speech transcript, tickers,...) and determine interesting scenes Create a database with formal annotations Use video processing to adjust time marks

Off-line Tasks indexed video recording Radio Commenting 3 Languages Radio Commenting 3 Languages Radio Commenting 3 Languages Audio Commenting (TV, Radio) 3 Languages Newspaper Text Newspaper Text Newspaper Text Newspaper Texts 3 Languages Newspaper Text Newspaper Text Newspaper Text Close Captions 3 Languages FreekickGoalPassDefense 17 min18 min24 min28min FoulFreekickDribbling KohlerBaslerMatthäusWörns BaslerBierhoff 25 m 60 m

Event Table

On-line Tasks Searching and Displaying Search for interesting events with formal questions Give me all goals from Overmars shot with his head in 1. Half. Event=Goal; Scorer=Overmars; Cause=Head; Time<=45 Indicate hits by thumbnails & let user select scene Play scene via the Internet & allow scrolling Of course: slow motion, fast play, start/stop, etc

On-line Tasks Knowledge Guided User Interface & Search Engine München - Ajax 1998 München - Porto 1996 Deutschland - Brasilien 1998 Play Movie Fragment of that Game FreekickGoalPassDefense 17 min18 min24 min28min FoulFreekickDribbling KohlerBaslerMatthäusWörns BaslerBierhoff 25 m 60 m

On-line SW Architecture first interaction with Web-server download of applet contact between applet and distributed Java server via RMI contact via media server via RMI and media streaming annotations from DB or file server JMF serves for merging of streams on client Abstract Corpus Model this kernel is operational (distributed and local)

On-line HW Architecture efficient & reliable storage management (near-line capacity, media change, 2. Location) high storage capacity (n TB, 1 h MPEG1 = 1 GB) powerful media servers / powerful network 2nd location Mirror site RAID Tape Library 3.5 TB FC Switch Media Server GB Switch Internet MM StationsNotebook 1Gbps 100Mbps Gb-Switch Tape Library 25 TB

Gain What gets lost? Is it necessary? Potential: direct Internet Service, less dependencies Current ProcedureMUMIS Procedure Manual Video Annotation Automatic Video Annotation and DB Integration Integration Central DB Query via PC Results on PC And Select & Play Contact Video Archive Get Video Tapes Search on Tape on VCR Segment & Play

Formal Aspects Start: Duration:30 Months EC-Funds:2.4 Mio Euro Consortium CTITUniversity of Twente, Enschede, NLNLP/IE TSI University of Nijmegen, Nijmegen, NLASR DFKISaarbrücken, DNLP/IE MPI Nijmegen, NLOnline Part DCSUniversity of Sheffield, UKNLP/IE ESTEAMGothenburg, SE (location Athens, GR)Translation Software VDAHilversum, NLDissemination

Acknowledgements UEFA DFB, FA, KNVB EBU, WDR, NOS