Group 1 Final Project Demonstration 13 December 2002 Madhvi Jain Yasin Ozer Jon Shalvi Frank Patrum

The Virtual Classroom

Itinerary Project OverviewFrank Data PipelineJon Graphical User InterfaceFrank Documentation/Users ManualFrank/Yasin Signal Processing Server sideMadhvi Client sideMadhvi/Yasin General Difficulties &Frank Implementation Problems

Project Overview Basic Idea was to produce a virtual classroom with a server and client GUI –Functionality was to be primarily implemented on the client side –Server side was to collect video and “splice” video streams from two cameras into a single video stream for transmission –Client was to receive video and implement the various DSP functions

General Configuration

Data Pipeline Data acquired from cameras / files Streams are spliced Encoded into one MPEG stream / file Transmit Zoom, Pan functions are applied Jumpback and Prof. Removal Display

Data Pipeline

Dali Multimedia Run-Time library written in Tcl, ported to C Developed at Cornell – Extensive Low-Level Multimedia Support –MPEG-1, AVI, Images, Audio

Dali Low level functionality –Parsing –Header Initialization –Processing –Decoding / Encoding –File Manipulation

Dali Multimedia-Friendly Data Types –BitParser –ByteImage –ByteClip –ScImage –MpegSeqHdr, MpegPicHdr

API’s DirectX –Lots of support –Easily stream data –Hard to touch raw data Dali –Can handle data directly –Relatively simple –

Dali Benefits –Free –Fast –Simple Problems –Could use C++ wrapping –Conflicts with VC++ IDE Needed custom VC makefile

Graphical User Interface (GUI) Server GUI dialog created Client GUI basic dialog created with “child” windows created - Overhead window and Student window are activated but currently only working as Modal which means only focus on them while they are open and can’t have any other functionality Some calls work for both Server and Client GUI - File open works and calls MPEG or AVI files Issues with using various Structures and Classes of windows. - CIPAddressCtrl, Modeless “child” windows, Scrollbars

Server GUI (main)

Client GUI (main)

Client GUI “Child” Windows

Documentation/Users Manual Help files generated by VC++ –Initial file links created by VC++ –Help files (identical to users manual) added to the code base so help button works and actually calls the help files Users Manual –Installation, Hardware Setup (configuration suggestions), Software Setup, Using the Software Code generously commented to explain functionality for future development teams

Processing at Server side Splicing of two video streams Overlapping area of two Cameras –Simple averaged Difference –Cross Correlation –Edge Detection Adding the two frames Assume that the two cameras will be properly aligned and synchronized manually

Processing at Server side Edge Detection –Roberts Diagonal Operator Detecting overlapping region –Summing all rows in a column Splicing –Adding the 2 frames.

Processing at Server side

Lens Curvature –Used the inverse transformation to remove the lens curvature

Processing at Server side Professor removal – Algorithm –Edge detection –Motion Detection –Use jump back feature

Processing at Client side Processing at client side involves pan, zoom and time reversal Have developed a flowchart on how to implement “pan” Will take the position of the scroll bar as an input, relate it to the index of image array and display that part of the array Difficulty: need to know what type of input will be provided from GUI Other Difficulties Difficulty in using add-in Visual Studio feature in MATLAB which allows calling m-files directly from Visual C++

Processing at Client side Scaling is performed by taking the samples of the image array and passing them to a scaled (e.g. 2x) output array The new array is one-dimensional and interpolated by averaging the samples before and after Loops are used to perform the computations

General Difficulties/Implementation Problems Primary cause for all of our issues was lack of experience –Chose a project that had a large amount of integration required from various APIs and libraries that weren’t “friendly” to each other Dali Matlab VC++ –Entire team has minimal experience writing code and the class projects only partially translated to our design for the final project. –Design concept was simple but implementation difficul Time schedule was not sufficient to permit expertise to be gained by our team members in each of the areas we had to implement. –Needed another couple of full weeks to be able to realistically pull the whole project together.

Specific Issues Audio not implemented –Focus was for the video so used image handlers and not audio –Implementation can easily be added once video is completely integrated Most functions not actually implemented in the GUI at this point –Code is written for zoom, splicing, pan –Code for networking partially complete but still not functional –Integration of code with GUI controls is an ongoing but slow progress Pan capability is to be tied to scroll bars and not currently functional –Pan for zoom was demonstrated by hard coding the position in the zoom call Help files not being called properly in Context Sensitive Help