February 21, 2008 Center for Hybrid and Embedded Software Systems Driving Application: 4D Tele-immersion Future Work Though we have imported an important part of the 4D tele-immersion system, we anticipate making more progress in the near future. Breaking the SMR algorithm in components will allow it to take advantage of multi-core computers. Incorporating more of the overall system, we will handle real time issues related to communication over the network. To the left is Monkey in an unaltered image. Below is the result of the SMR algorithm on the image. Using Ptolemy II to Re-architect Large C and C++ Programs Hwayong Oh Teale Fristoe Above is the top level model, containing the SMR algorithm. To the right is the actual SMR actor, which has two EmbeddedCActors containing the algorithm and a simulation of capturing an image. Both to the left and right are windows with the C code in the EmbeddedCActors. The 4D tele-immersion environment, developed by Ruzena Bajcsy’s group at UCB, allows geographically separated users to collaborate in a virtual world. Independent clients use advanced image processing to capture models of users and transfer them over a network. A huge application developed by specialists that demands high performance, it makes an excellent first and guiding example for using Ptolemy II as a development environment. Single Multiscale Representation(SMR) Tele-immersion uses the single multi-scale representation algorithm to infer the third dimension from pairs of two dimensional images. An intensive algorithm that uses triangulation to calculate depth, it is the first piece of the tele-immersion system that we have successfully imported into Ptolemy II. The SMR algorithm triangulates an image to multiple levels of coarseness based on the detail of each section. Above is an image that has been broken into three levels. Background Traditionally, Ptolemy II has been a modelling tool used primarily for embedded systems. With its many models of computation, designers have been able to focus on the structures of their systems without worrying about implementation details. Its C code generator not only increases the speed of running models, but allows Ptolemy II to directly generate code for target platforms. Goals Ptolemy II will be used to develop and improve C and C++ programs. Large programs can easily be managed in components, each component in an actor. Existing Ptolemy II models of computation will automatically schedule components intelligently. Generated code will run in parallel and elegantly handle real time. Achievements Our work has focused on increasing the power of EmbeddedCActors, which allow arbitrary C or C++ code to be executed within a model. Expanding the Macro Language Originally, limitations in the C macro language greatly reduced the effectiveness of EmbeddedCActors. Programmers are now easily able to develop directly in C without relying on Java code generator helper classes. Previously, a programmer would have had to use an awkward Java code generation helper to cycle through the values at an input multiport. It can now easily be done directly using the macro language. Other code in this EmbeddedCActor can now use functions defined in the listed libraries. Extending the Type System In order to take advantage of existing features of Ptolemy II for developing C programs, new systems for handling arbitrary types are needed. The Pointer type is our first attempt at this. It allows any C or C++ pointers to be passed between EmbeddedCActors. Future goals include passing C and C++ objects by reference and allowing the user to define types. Incorporating External Libraries EmbeddedCActors can now include external libraries, allowing models to use closed source libraries or include legacy code without having to include the source directly. Both of the EmbeddedCActors here reference the same memory. The memory is being interpreted as an integer, but any class can be passed between actors in this way. Using the 4D Tele- immersion system, dancers separated by thousands of miles can interact as if they were in the same room.