1. ARDA VACE Advanced Research and Development Activity (ARDA) Video Analysis and Content Extraction (VACE)

2. Video Analysis and Content Extraction (VACE) Project Description Goal: Develop revolutionary advances in automatic video extraction, understanding, and multi- modal-fusion.Goal: Develop revolutionary advances in automatic video extraction, understanding, and multi- modal-fusion. Funded in 3, 2-year phases. (we are currently in the middle of phase 2. Multi-domain: meetings, Broadcast News, UAV (Unmanned Air Vehicle), Surveillance/Ground reconMulti-domain: meetings, Broadcast News, UAV (Unmanned Air Vehicle), Surveillance/Ground recon NIST is supporting VACE meeting video needs and evaluation oversightNIST is supporting VACE meeting video needs and evaluation oversightSponsor ARDAARDAParticipants Progress/Insights Building corpus and evaluation infrastructure for video extraction community by applying knowledge from speech recognition/understanding evaluation.Building corpus and evaluation infrastructure for video extraction community by applying knowledge from speech recognition/understanding evaluation. Developing diagnostic metrics with the mind of moving toward a single- number metric.Developing diagnostic metrics with the mind of moving toward a single- number metric. Future Plans 2005 Core evaluations2005 Core evaluations –face detect/tracking –Text detect/track/recognition –person detect/tracking –hand detect/tracking 20062006 –event detection BAE BBN USC UMD TASC USF UCF Virginia Tech U Illinois Urbana Champaign Salient Stills IBM/Columbia Carnegie Mellon U Pittsburgh Pattern Recognition Advanced Interfaces

3. Issues Having “enough” data of the types being evaluated.Having “enough” data of the types being evaluated. Ground Truth (Currently we use ViPER [developed at Univ. Maryland])Ground Truth (Currently we use ViPER [developed at Univ. Maryland]) –Temporal/Spatial –bounding box or point reference –object definition MetricsMetrics FundingFunding Definition of PurposeDefinition of Purpose

4. Evaluation Domain Sub-Forums Broadcast News (in coordination with TRECVID) Meetings Surveillance UAV Tasks: –Develop evaluation recommendations for 2006+ event detection tasks These include task definitions, metrics, schedule, source data, and necessary annotations Chairs work with both research community and interested partners to satisfy Sponsor and Research needs Identify existing research sour data to be used or list realistic requirements, ensuring that corpora meet above needs produce task list and required data resources –Provide recommendations to Core Evaluation Forum –NIST will create mail lists and can set up conference calls Forums should hold biweekly teleconferences as necessary to maintain momentum –Forum chairs will give report at Fall Workshop

5. Domain Task MeetingsBroadcast NewsUAVSurveillance Text Detection (English) -Y-- Text Tracking (English) -Y-- Text Recognition (English) -Y-- Text Detection (Chinese) ---- Text Tracking (Chinese) ---- Text Detection (Arabic) -Y-- Text Tracking (Arabic) -Y-- Face Detection YY-Y Face Tracking YY-Y Hand Detection YY-Y Hand Tracking YY-Y People Detection Y-YY People Tracking Y-YY Vehicle Detection --YY Vehicle Tracking --YY Planned Core Eval Tasks/Domains (34 evaluations)

6. http://www.nist.gov Information Technology Laboratory http://www.itl.nist.govitlab@nist.gov Information Access Division http://www.itl.nist.gov/iaui/john.garofolo@nist.gov rachel.bowers@nist.gov ARDAhttp://www.ic-arda.org