1. Research & Development 1 High Frame Rate Television Mike Armstrong, Steve Jolly, Richard Salmon BBC R&D HPA Technical Retreat 18 February 2009

2. Research & Development 2 Standard frame/field rates TV frame/field rates were chosen over 70 years ago.

3. Research & Development 3 Standard frame/field rates TV frame/field rates were chosen over 70 years ago. They were chosen to: –exceed the threshold for apparent motion –avoid visible flicker (on small screens)‏ –avoid interaction with the mains frequency –provide a way of showing cinema film

4. Research & Development 4 Standard frame/field rates Current 50/60Hz TV was a match to –standard definition pictures –and smaller CRT displays

5. Research & Development 5 Standard frame/field rates Current 50/60Hz TV was a match to –standard definition pictures –and smaller CRT displays Problems arise with –larger displays –increased picture resolution –sample-and-hold display technology

6. Research & Development 6 Loss of detail on moving objects

7. Research & Development 7 Motion Portrayal at 50/60Hz The portrayal of motion is a trade off between –motion blur (long shutter)‏ –temporal aliasing (short shutter), leading to jerky motion and spoked wheels running backwards

8. Research & Development 8 Motion Portrayal We need the ball to remain looking like a ball as it moves, without using excessive camera shuttering

9. Research & Development 9 Dynamic Resolution at 50/60Hz When the camera pans the entire High Definition scene becomes blurred - for example when following the action during a football match

10. Research & Development 10 Dynamic Resolution at 50/60Hz When the camera pans the entire High Definition scene becomes blurred - for example when following the action during a football match As you increase the resolution so the rate of panning has to be reduced to keep the blurring under control

11. Research & Development 11 Dynamic Resolution The dynamic resolution of HDTV is no better than SD

12. Research & Development 12 Impact on the Viewer Where there is a large difference between the resolution of a static and dynamic picture, this can lead to a feeling of nausea

13. Research & Development 13 Impact on the Viewer Where there is a large difference between the resolution of a static and dynamic picture, this can lead to a feeling of nausea Therefore the higher the static resolution, the higher the dynamic resolution must be for comfortable & lifelike images

14. Research & Development 14 Up-converting Displays 100/120Hz LCD TVs are becoming common, and 180/200/240Hz models are now being exhibited –These put 50/60Hz signals on the screen interpolated to higher rates, so solving the problems of flicker and display smearing –they create intermediate pictures using motion prediction –it is done to mitigate the problems of sample- and-hold displays

15. Research & Development 15 Up-converting Displays But this is not High Frame Rate TV –Cannot reduce motion blur captured in camera –Cannot predict complex motion

16. Research & Development 16 Up-converting Displays But this is not High Frame Rate TV –Cannot reduce motion blur captured in camera –Cannot predict complex motion To make motion rendition more lifelike we need higher frame rates –in the camera –for distribution –and in the display.

17. Research & Development 17 Higher Frame Rates We would suggest that: –if SD is acceptable at 50Hz –then full HDTV needs 150Hz –as resolution increases, we probably want at least 300Hz

18. Research & Development 18 Higher Frame Rates We would suggest that: –if SD is acceptable at 50Hz –then full HDTV needs 150Hz –as resolution increases, we probably want at least 300Hz 300Hz is easy to convert to 50 or 60Hz and is compatible with mains frequencies

19. Research & Development 19 Potential HFR Issues Higher data rates means: –increased storage requirements –increased bandwidth Shorter exposure for each frame Interaction with AC lighting Loss of “film-look”?

20. Research & Development 20 HFR Data Rates HFR video should be easier to compress –smaller changes between each picture –each frame is sharper making the motion is easier to predict –less temporal aliasing –could make use of three-dimensional transforms in video compression

21. Research & Development 21 HFR and Video Noise Shorter exposure times lead to higher noise levels but –gives a clearer separation between image motion and video noise enabling more effective noise removal –and at higher display rates random noise is far less visible, c.f. DLP and plasma displays

22. Research & Development 22 HFR and lighting AC lighting will lead to changes in illumination between pictures –this may make compression more difficult –but will not be noticeable when displayed –use multiple of mains frequency to avoid beating

23. Research & Development 23 HFR and lighting AC lighting will lead to changes in illumination between pictures –this may make compression more difficult –but will not be noticeable when displayed –use multiple of mains frequency to avoid beating It is simpler to filter out temporal lighting problems and flash photography

24. Research & Development 24 HFR film-look and other possibilities With a higher frame rate shoot you can change the temporal characteristics of the video in post, for example: –add film-look later –add film-look to only part of the picture –develop a new range of motion characteristics

25. Research & Development 25 HFR Production High Frame rate production also gives the possibility of creating higher quality standard rate productions –300Hz production can be converted equally well for 50 and 60Hz TV –Better temporal down-sampling can be used to minimise aliasing and give range of motion portrayal –a greater range of motion FX can be applied

26. Research & Development 26 Demonstration of High Frame Rate TV At IBC, on the EBU Village –Video shot 1920 x 1080 at 300fps down converted to display at 1400 x 788 100fps

27. Research & Development 27 Demonstration of High Frame Rate TV 50 fps 300 fps

28. Research & Development 28 HFR - Further Work To understand how well HFR video compresses

29. Research & Development 29 HFR - Further Work To understand how well HFR video compresses To understand how visibility of noise decreases vs increase in noise with loss of sensitivity

30. Research & Development 30 HFR - Further Work To understand how well HFR video compresses To understand how visibility of noise decreases vs increase in noise with loss of sensitivity To understand what bit depth is required as frame rate increases

31. Research & Development 31 HFR - Further Work To understand how well HFR video compresses To understand how visibility of noise decreases vs increase in noise with loss of sensitivity To understand what bit depth is required as frame rate increases Compromise to choose the optimum frame for a given resolution

32. Research & Development 32 HFR Conclusion Increasing the static resolution without improving the frame rate makes the TV system less and less suitable for moving pictures

33. Research & Development 33 HFR Conclusion Increasing the static resolution without improving the frame rate makes the TV system less and less suitable for moving pictures We assert that increasing the frame rate for capture and display of television pictures produces a very significant improvement in video quality

34. Research & Development Thank you stephen.jolly@rd.bbc.co.uk

35. Research & Development 35