Presentation is loading. Please wait.

Presentation is loading. Please wait.

3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 1 Richard Verhoeven The Candela Project.

Similar presentations


Presentation on theme: "3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 1 Richard Verhoeven The Candela Project."— Presentation transcript:

1 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 1 Richard Verhoeven The Candela Project Bosch demonstrator perspective

2 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 2 Outline Introduction Content Analysis Networked DELivery Architectures Bosch demonstrator Demo

3 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 3 Introduction ITEA project Content Analysis and Networked DELivery Architectures Partners – Philips, TU/e, Bosch (NL) – Multitel, IT-Optics, VUB, Traficon (BE) – Hantro, Solid, VTT (FI) – ESI, Amena, Ibermatica, CanalMicro (ES)

4 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 4 Introduction Focus – Home Multimedia – Surveillance – Medical systems Issues – Large multimedia collections – Distributed storage Solution – Automatic metadata extraction – Content delivery framework MPEG7 MPEG2/4

5 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 5 Introduction User Interface Cable WWW AV Content & descriptors Content Management HDD Database MPEG7 Storage

6 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 6 Content Analysis AV segmentation – Chaptering of video content Object tracking – Traffic movement, suspicious behavior Idle object detection – Airport and public transport surveillance Face detection & recognition – Photo & video content management Sports analysis – Statistics, improved video compression

7 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 7 Content Analysis Voice recognition – Video content management Optical character recognition (OCR) – License plates, news tickers, prints on trucks Classification – Traffic type based on speed and size

8 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 8 Content Analysis - Issues Light conditions – Shadow, clouds, head lights, back light Merging objects – At traffic light, near crossings Processing intensive – Different & variable frame rates

9 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 9 Network delivery Content standardization – Video content – MPEG4 – Metadata – MPEG7 Control protocol – SOAP – UPnP Distribution and load balancing Reliability and robustness

10 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 10 Network delivery Database storage – Allows persistent storage of features – Allows nonlinear features browsing and searching – Allows non real-time features processing – Enables features distribution using metadata standards – Enables new applications – Serves as a reference to evaluate new algorithms

11 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 11 Network delivery

12 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 12 Architectures Centralized & static (Bosch) – Content analysis modules known in advance – Connections between modules fixed – Central controller to manage data flows – Not easy to configure, not very robust Distributed & dynamic (Philips) – Content analysis modules can be added – Connections created on demand – Flexible and robust – a.k.a. Cassandra platform

13 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 13 Bosch Demonstrator Content analysis – Acquisition (Bosch) – Object tracking (Bosch) – Metadata analysis (Bosch) – Face recognition (TU/e, VCA group) – Idle object detection (Multitel) – Alarming (Bosch, Solid) Network delivery – Controller & Middleware (Multitel) – MPEG7 (VUB) Application / Architecture – Database (Bosch) – Retrieval client (TU/e, SAN group)

14 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 14 Bosch Demonstrator

15 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 15 Middleware Communication transparency Video & metadata router (controller) in star- structure Simple connect(), getData(), and setData() functions Decoding of MPEG-4 under API Buffering in case of computational overflow Implementation for Windows and Linux PCs

16 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 16 Middleware Controller ACQ Track MDA Idle Face DB MPEG4 MPEG7

17 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 17 VCA Modules Acquisition – Live video grabbing with real-time compression – MPEG-4 (colour, CIF res., 1 MBit, 12.5 fps) Object tracking – Algorithm provided by BOSCH Shape and bounding box of moving objects Assignment of object IDs Sensitivity difficult to control – Shape information ignored – Filter on short-term objects – Thumbnail selection

18 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 18 VCA Modules Metadata analysis – Analyse bounding boxes from object tracking – Calculate size and speed based on camera info – Classify object based on size Truck/bus, car, person/other – Extract trajectories from center of bounding boxes – Create events for certain trajectories Turn left, turn right – Future extension Color analysis in thumbnail (yellow car, blue truck)

19 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 19 VCA Modules Face recognition – Provided by Fei Zuo from VCA group – Based on neural networks – fps – Difficult to combine with other VCA modules

20 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 20 VCA Modules Idle object detection – Provided by Multitel – Changes with respect to background image Database storage – Video: MySQL database with GOVs (16 frames) – Metadata: PostgreSQL database with OpenGIS for trajectory Alarming – Receive metadata from face recognition – Inform mobile terminal of recognized person – Display video on mobile terminal

21 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 21 Integration issues Windows & Linux – Middleware library cross-platform – MPEG7 library platform specific Memory leaks in middleware libraries – Unclear who had to free memory – MPEG7 data is in XML Stability of the system – Endurance test – Fill the database with useful content

22 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 22 Integration issues Different frame rates and buffering – Face recognition at 2 fps, rest at 12.5 fps Variable frame rate once a face is recognized – Middleware doesn’t provide buffer info – Possible fix: separate thread to retrieve frames – Used fix: check delay in frame retrieval Video codecs and formats – YUV vs RGB vs RBG

23 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 23 Retrieval client Search through metadata and watch video Most visible part of the demonstrator – Only way to show the content of the database Ad-hoc application – Sketch on one A4 and a list of nice features – Integration of work-in-progress parts Trajectory search by Rob Albers – Debugging – Tight deadline ITEA review in Helsinki, all PCs shipped – Not the SEP approach

24 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 24

25 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 25 Retrieval client - Issues Testing difficult without a filled database – Data was recorded one week before shipping Metadata stored in multiple tables – Very complex SQL queries – Combined selections might not make sense Integration of existing and changing parts – GUI for video playback from a database – GUI and engine for trajectory search Synchronization of video and metadata – Date&time in MySQL, PostgreSQL, ODBC, Qt

26 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 26 Retrieval client – video playback GOV buffer 1GOV buffer 2 Bounding box metadata

27 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 27 Demo

28 3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 28 Questions?


Download ppt "3 May, 2015 Richard Verhoeven, TU/e Computer Science, System Architecture and Networking 1 Richard Verhoeven The Candela Project."

Similar presentations


Ads by Google