VideoTag Video Display and Recording Aaron Shepler

Motivation Enable the user to view real-time video from the camera Allow the user to annotate on top of the video image Facilitate simple video recording

Previous / Related Work Point Grey Research has written a C++ wrapper for reading/writing to an AVI Microsoft is working on an InkCanvas control (to be released in Vista) that would potentially allow for something similar – inking on multimedia

Challenges How do I use the camera in C# when the interface is in C? –The.NET framework allows for easy integration between different managed types of code: i.e. Managed C++ and Managed C# –Integration between unmanaged code and managed code is not very well documented

Challenges How do I place the image on the screen once I have data using C#? –Lower level operations, such as direct access to screen are not available

Challenges How do I manage the recording of the video –C# does not have built-in AVI recording functionality –Function calls from C would also have to be wrapped

Camera Approach / Solution Two possible approaches that allow for use of camera in C# –Link in flycapture camera methods directly Large blocks of unsafe { } code due to use of pointers –Link in a library that simplifies the interface Few lines of unsafe { } code

Camera Approach / Solution Mistake… –First tried to link in flycapture camera methods directly Unsafe { } blocks of code were everywhere due to the use of pointers Became a C-based program –Learned my lesson the hard way

Camera Approach / Solution Encapsulated lower level camera functionality in a library… –Created a small library of methods Interface was well defined Less time spent matching compatible types

Camera Approach / Solution CameraLib –Library with basic camera functionality void initialize(); –Initializes the first camera on the FireWire bus, creates context, etc void* grabImage(unsigned char*); –Retrieves an image based on the camera context void destroy(); –Destroys the camera context –Pure C library for use in managed code

Video Approach / Solution In order to place the image onto the screen, the InkPicture control was used –Allows the background image to be changed while maintaining the ink on top. –Create thread in C# continuous capturing / refreshing the image

Recording Approach Record to Avi using C# wrapper class –Same refresh thread adds the current bitmap to the next frame in the AVI –Pro: Easy to access bitmaps out of the AVI later –Con: Large file is unhandy Avi wrapper written orginally by René Nyffenegger (modified by John Corinna), located at: –

Demonstration

Discussion How can we reduce the slight flicker with the video/ink? –Double buffering? –User Drawn Control?

Discussion What methods of encoding the video data would be faster and more robust than AVI? –Various compression models? –Existing codecs?