1. V4L2 and DVB Perspective from user-space API consumer Rémi Denis-Courmont August 28 th 2012, San Diego (CA)

2. Versioning

3. V4L2 versioning ● Kernel version ● Not available on *BSD ports ● Might break with custom distro kernels ● #ifdef V4L2_XXX : ● Cannot deal with new enumeration values ● Cannot deal with updated structure layouts ● custom copy of videodev2.h

4. V4L2 versioning (cont'd) ● follow DVB model ● Good ● Cheap ●...but not backward compatible for now ● Should be taken into use as soon as possible ? ● Was abandonned at some point(?)

5. libv4l2 versioning ● Please release every so often ! ● distros follow upstream « official » releases ● e.g. : plugin support still missing after 4 years ● Version numbers help check features ● Similar issue to kernel API

6. Controls

7. Many controls ● Class is not sufficient to evaluate relevance ● Desktop vs embedded ● Input vs output ● Video vs audio ● Burden for UI development ● Burden for localization

8. Many controls (cont'd) ● Was INTEGER_MENU really needed? ● MENU is a superset of the functionality ● Biggest source of problems with (old) versions : enumeration member ● Cannot use #ifdef ● For new menu control choices ● For new control classes

9. USB video class ● UVC controls unusable from traditional userspace ● need priviledges – webcam user is rarely root... ● persistent and shared ● important functionality obscured – e.g. H.264 encoding settings

10. Performance enhancements

11. USERBUFS ● Not supported at all ● Blocked (EINVAL) when natively supported by the driver ● Not emulated (on top of MMAP) when picture format is emulated ● Alternatively, user-space should favor READ for emulated mode ? – More complexity for apps, less complexity for library

12. Picture format conversion ● Dedicated (in-memory) API for smart apps ? ● Currently internal lbrary functions ● Conversion to custom pitch ● Could save one memory copy in some cases ● Also requires dedicated API

13. Timestamps

14. Timestamps : the problem ● Needed for A/V synchronization ● Forget about CLOCK_REALTIME ● Every half-way decent multimedia framework uses CLOCK_MONOTONIC nowadays. ● Unusable as things stand ● No way to know which clock is used, if any.

15. Timestamps : solutions ? ● Multiple potential solutions ● Always use monotonic ● Insufficient : need run-time « warranty » of sorts. ● Flag in VIODIOC_DQBUF ● Not backward compatible ● User configures its clock via dedicated ioctl() ● Backward compatible but probably overkill

16. Output

17. ● Yet Another Interface ● X11 or OpenGL, FrameBuffer ● Do (non-specialized) apps need to care ? ● Wrongly confused by users for V4L2 loopback ● Also confused by some developers ● Poor implementations matching the design and bugs of an unvetted out-of-tree driver

18. Parenthesis : V4L2 loopback ● Why (the heck) users want V4L2 loopback ? ● Make different media frameworks talk ● e.g. VLC to gstreamer of vice versa ● Certainly not the most efficient but It Just Works. ● Fake or post-processed webcam video signal ● Skype

19. Devices

20. Devices enumeration ● Devices enumeration API ● ALSA has two methods : – Old style : list (kernel) device nodes – New style : read configuration files ● Hot-plug ● ALSA has nothing, is not a good example ● Udev cannot deal with plugins

21. Audio ● mapping from audio-capable device node ● to ALSA PCM device ● to PulseAudio source ● not to mention JACK or OSSv4 ● Fundamental limitation of USB device model :-( ● Distinct interfaces in same configuration : Device path does not even match. ● Dealing with ALSA vs PulseAudio is tricky (not kernel issue, just saying)

22. DVB ● Specify mapping of frontend ↔ demux (↔ dvr) ● needed for udev integration / hotplug ● Using demux instead of dvr ● Corrupt data ?! (except budget mode)

23. DVB ● No user-space library ● Abstraction may be needed later ● Non-TS transponders ● Plugin support ? ● Multi-frontend DVBv5 version problems

24. Bugs ?

25. CROPCAP specification ● CROPCAP marked mandatory ●...but not even supported by vivi ● Should userspace assume not cropping then? ● Pixel A/R not available without CROPCAP.

26. Driver oddities ● G_PARM failing ● S_PARM failing on the result of G_PARM ● how to force "normal" mode then ? assume everything OK ? ● Frame rate enumerated, S_PARM returns 0 ● But G_PARM returns a different (lower) value ! ● QBUF fails if MMAP buffers is not mmap()ped yet ● invalid bytesused on DQBUF (pwc driver)

27. MEM2MEM ● G_INPUT & S_INPUT not supported ● Should not set CAP_VIDEO_CAPTURE ● Fixed since I wrote drafts ?