areaDetector and GStreamer Similarities and Differences Tom Cobb

What is GStreamer Streaming media framework – audio and video Many plugins available for data formats / data manipulation Higher level than just input / filters / output – Networking, audio/video mixed streams, auto data handling – Various options utilizing hardware accelerators Has been extended for use in some scientific communities Written in C with Python (and other language) bindings

What is areaDetector I hope I don’t need to say anything here...

What does GStreamer do that areaDetector does? Chains plugins for data processing Options to run any plugin in its own thread Image transformations and conversion (rotate, scale and offset, ROI, debayer, overlay, color convert) File and network I/O in various datatypes Some stats functions Rewirable chain on the fly GigE cameras Windows, Linux, MAC OS X

What does GStreamer do that areaDetector doesn’t? Some data formats (YUV is about the only one that is immediately useful) Some file formats (MJPEG, H264 multicast streaming) Autoplugging chain based on data formats A lot of plugins that we could make use of (opencv functions, Synchronization of different clock sources V4L cameras Could make use of audio processing for ADC data

What does areaDetector do that GStreamer doesn’t? Some data formats (Int32, Float32, Float64) Some file formats (HDF5, netCDF) Some stats functions Most of our detectors EPICS integration vxWorks

Who else is using GStreamer? LIGO: gravitational-wave data analysis tools for 1D data (see tutorial) LIGOtutorial Military: various 2D sources, mainly Camera Link and CoaXPress framegrabbers from Active Silicaon, NI, DALSA, Matrox, EDT (see presentation) Military presentation Software Radio: FFTs and other 1D data functions Software Radio Lots of A/V people...

What neat ideas does it have? GObject based, so written in C but get Python wrapper for free Message passing on bus Video format (Caps) negotiation Bins for groups of plugins with a bit of logic Multiple clock sources (e.g. for stream synchronisation) Connections can be run in push or pull mode (e.g. For limited rate outputs, or attribute readers)

Demo EPICS V4 monitor on NTNDArray producing data to a pipeline of GStreamer plugins