1. Technologies: for Enhancing Broadcast Programmes with Bridgets
Karsten Müller
Fraunhofer HHI
1st BRIDGET Workshop
Paris – 16th December 2013

2. Overview “Bridgetised” Broadcast Broadcast Media databases
Analysis, Search, Recognition
AR Authoring
Media databases
(archives, internet servers, social networks, etc)
Mobile Terminal
3D Modelling
System Architecture,
Interfaces

3. Visual Search, Recognition and Media Analysis
Reliable and fast suggestion of content bridges
Recognising objects, buildings and scenes selected by the user
Instantaneous matching with available images, video, and other content from a broadcaster or external databases
Video-to-video matching, based on the improved local descriptors and description aggregation (as employed in MPEG’s Video Signature tools)
Modifications and improvements to selected elements of the processing pipeline:
Local descriptors (targeting binary resulting in ultra-fast matching),
Hierarchical scoring strategies
Descriptor aggregation techniques
Novel methods of geometric verification

4. 3D Scene Modelling, Coding and Reconstruction
Coarse 3D scene model creation from available monoscopic, stereoscopic and 3D images, video material, and 3D Computer Graphics (CG) data in a broadcasters database
Continuous model refinement
Newly available material from other sources, like the web
Uploaded content from individuals or user groups, e.g. images taken from other viewpoints.
Addition of CG 3D models, contributed by professional artists or end users, whenever available
Newly developed VS tools and media descriptors, for extracting scene calibration information from images and videos
Device-dependent view synthesis, viewpoint selection and baseline adaptation for stereo playout
Visual compression adaptation for authoring server and user terminal

5. Augmented Reality Authoring
Integration of navigation information into temporal video structure
Association of identification descriptors to user-defined spatial regions, e.g. for possible reconstruction of multi-view videos, 3D objects and scenes.
Management of descriptor dictionaries (equivalent of a large index where (binary) descriptors will be associated with references to media from private and open libraries containing the same object)
Cloud-friendly Authoring Tool (AT)
with exploitation of the current physical infrastructure of IMT’s MyMultimediaWorld ( platform
for establishing augmentation areas by using natural markers (images, videos, etc.) that will be recognised in real time, and building the behaviour of the application by using pre-programmed scripts.
New formalism for representing the scene, the assets, the user interaction, the connection with the sensors and the access to remote resources.

6. Mobile Terminal
Creation of high quality 3D models of the scenes for optimising video-plus-depth rendering and view synthesis for content and baseline retargeting.
Lightweight and real-time methods for retargeting, that do not require content post processing to be adapted to the terminal viewing set-up for cases, where high quality models cannot be computed (e.g. where very limited views of a scene are available)
Sparse features and saliency detection for most characteristic scene objects
Hierarchical grouping approach for points with similar colour and depth for bigger “splat” rendering during the visualisation phase
3D audio rendering over small speakers for mobile terminals

7. System Architectures and Interfaces
Mechanisms for hybrid services on the main screen towards second screens in the home network (e.g., tablets and smart phones)
Usually not under direct control of broadcasters or service providers,
 Design of according signalling and content retrieval processes
Investigation of video object recognition as an alternative with less requirements and constraints in terms of device functions exposed to the applications (e.g., a UPnP SW stack),
Provision of a reference architecture to support second screen services in an open, component-based way