1. MUMIS User Group Workshop P. Wittenburg Max-Planck-Institut für Psycholinguistik Nijmegen

2. Agenda 11.00IntroductionPeter Wittenburg ASR AspectsHelmer Strik NLP/IE AspectsThierry Declerck 11.45Demo of PrototypePeter Wittenburg 12.30 Lunchbreak 13.15Potential of MUMIS TechnologyDiscussion 14.00Typical QuestionsDiscussion 15.00Other Points of InterestDiscussion 15.30End

3. Objectives MUMIS is about Technology development to automatically index lengthy multimedia recordings Find and mark interesting scenes Technology development to exploit indexed multimedia archives Search for interesting scenes and play them via Internet Test Domain: Soccer Games / UEFA Tournament 2000

4. Off-line Tasks Indexing by Automatic Speech Recognition (Radio/TV Broadcasts) Transfer the speech signals into texts automatically Natural Language Processing Take all texts (newspapers, speech transcript, tickers,...) and determine interesting scenes Create a database with formal annotations Use video processing to adjust time marks

5. Off-line Tasks indexed video recording Radio Commenting 3 Languages Radio Commenting 3 Languages Radio Commenting 3 Languages Audio Commenting (TV, Radio) 3 Languages Newspaper Text Newspaper Text Newspaper Text Newspaper Texts 3 Languages Newspaper Text Newspaper Text Newspaper Text Close Captions 3 Languages FreekickGoalPassDefense 17 min18 min24 min28min FoulFreekickDribbling KohlerBaslerMatthäusWörns BaslerBierhoff 25 m 60 m

6. Event Table

7. On-line Tasks Searching and Displaying Search for interesting events with formal questions Give me all goals from Overmars shot with his head in 1. Half. Event=Goal; Scorer=Overmars; Cause=Head; Time<=45 Indicate hits by thumbnails & let user select scene Play scene via the Internet & allow scrolling Of course: slow motion, fast play, start/stop, etc

8. On-line Tasks Knowledge Guided User Interface & Search Engine München - Ajax 1998 München - Porto 1996 Deutschland - Brasilien 1998 Play Movie Fragment of that Game FreekickGoalPassDefense 17 min18 min24 min28min FoulFreekickDribbling KohlerBaslerMatthäusWörns BaslerBierhoff 25 m 60 m

9. On-line SW Architecture first interaction with Web-server download of applet contact between applet and distributed Java server via RMI contact via media server via RMI and media streaming annotations from DB or file server JMF serves for merging of streams on client Abstract Corpus Model this kernel is operational (distributed and local)

10. On-line HW Architecture efficient & reliable storage management (near-line capacity, media change, 2. Location) high storage capacity (n TB, 1 h MPEG1 = 1 GB) powerful media servers / powerful network 2nd location Mirror site RAID Tape Library 3.5 TB FC Switch Media Server GB Switch Internet MM StationsNotebook 1Gbps 100Mbps Gb-Switch Tape Library 25 TB

11. Gain What gets lost? Is it necessary? Potential: direct Internet Service, less dependencies Current ProcedureMUMIS Procedure Manual Video Annotation Automatic Video Annotation and DB Integration Integration Central DB Query via PC Results on PC And Select & Play Contact Video Archive Get Video Tapes Search on Tape on VCR Segment & Play

12. Formal Aspects Start:1.7.2000 Duration:30 Months EC-Funds:2.4 Mio Euro Consortium CTITUniversity of Twente, Enschede, NLNLP/IE TSI University of Nijmegen, Nijmegen, NLASR DFKISaarbrücken, DNLP/IE MPI Nijmegen, NLOnline Part DCSUniversity of Sheffield, UKNLP/IE ESTEAMGothenburg, SE (location Athens, GR)Translation Software VDAHilversum, NLDissemination

13. Acknowledgements UEFA DFB, FA, KNVB EBU, WDR, NOS