Presenting: Shlomo Ben-Shoshan, Nir Straze Supervisors: Dr. Ofer Hadar, Dr. Evgeny Kaminsky

 Background  Problem definition  Project goals  Proposed solution  Future work 2

 Background  FTV-Free View-Point Television  3-DTV  MVC  Problem definition  Project goals  Proposed solution  Future work 3

3DTV- 3 Dimensional television A 3D television is a television that employs techniques of 3D presentation, such as stereoscopic capture, multi-view capture, or 2D plus depth, and a 3D display—a special viewing device to project a television program into a realistic three-dimensional field. Which means there is a need for at least more than one view compressed in order to watch a 3D video. 4

5 FTV - Free View-point Television FTV is a system for viewing natural video, allowing the user to interactively control the viewpoint. With FTV, the focus of attention can be controlled by the viewers rather than a director, meaning that each viewer may be observing a unique viewpoint. In order to acquire the views necessary to allow a high quality rendering of the scene from any angle, several cameras are placed around the scene, either in a studio environment or an outdoor venue, such as a sporting arena for example.

6

MVC - Multi-view Video Coding: 3D techniques such as 3DTV, or Free View-point TV, that we described earlier, require more than one view (video sequence), in order to create their picture. The common standard developed for compressing multiple views, is MVC – Multi-View Video Coding. It is an extension of the H.264/MPEG-4 AVC video compression standard, developed to enable efficient encoding of sequences captured simultaneously from multiple cameras, and encode it into a single video stream. 7

8

MVC (multi-view Video Coding): The innovation that MVC suggests is compressing video using motion vectors not only in the time scale but also in the spatial scale (inter- view coding). The idea is to exploit the statistical dependencies from both temporal and inter-view reference pictures for motion compensated prediction. 9 Our own interpretation of video compression...

 Background  Problem definition  Project goals  Proposed solution  Future work 12

While viewing a 3D movie, or using a FTV application, on Television, computer or even a mobile phone, we consume a very large amount of data. Hence, a high number of views – a need for a large bandwidth. There for, there will always be a place for improvement of the MVC performance, in terms of Rate – Distortion. there is a need for more efficient ways to compress the video and reduce the band width, or the memory used to store the data. 13

 Background  Problem definition  Project goals  Proposed solution  Future work 14

The main goal of our project is researching and upgrading the MVC standard. We will reduce the amount of information sent by omitting one or more of the views and using the neighbor views to rebuilt it. 15

 Background  Problem definition  Project goals  Proposed solution  Future work 16

The algorithm is as described below:  Before sending the data the algorithm checks all the view-points PSNR.  The algorithm chooses the view-point with the highest PSNR, which means that reconstructing this view-point will be the easiest.  The chosen view-point data in not send, and by that we reduce the required bandwidth. 17

18 View 1 View 2 View 3

Image reconstruction: We will examine a few techniques that will accomplish the image reconstruction of the missing view, or views. The method will have to give both good PSNR results and a low computation time, since we execute it on the decoding side, and we don’t want a big delay. 19 View 3 View 1 Reconstructed frame View 2

Image reconstruction: The methods that we examine are:  Stereo Image Reconstruction – Using Algorithms that find disparity and depth map of a pair of pictures, it creates one 3D picture located between the two views.  MCTF – Motion Compensated Temporal Filtering.  Simulating motion prediction between the two views. 20

 Background  Problem definition  Project goals  Proposed solution  Future work 21

The main difficulty of our work, is to find the most suitable form of estimating and reconstructing the missing view, using one of the methods we described earlier. Currently we are examining the methods and ways of implementation.

23 Thank you very much! Any Questions?