1 Design and Implementation of an Efficient MPEG-4 Interactive Terminal on Embedded Devices Yi-Chin Huang, Tu-Chun Yin, Kou-Shin Yang, Yan-Jun Chang, Meng-Jyi Shieh, Wen-Chin Chen IEEE International Conference on Multimedia and Expo Volume 1, June 2004 Page(s): Reporter ：陳憲章 Date ：

2 Outline Introduction System Architecture Synchronization of Heterogeneous media Implementation and Optimization Experimental Results Conclusions

3 Introduction This paper present an efficient MPEG-4- based interactive player for PDA-like embedded devices. There are many technical issues need to be tackled:  lower computing power  less hardware acceleration  constrained memory size  lack of built-in multimedia APls

4 Introduction (con’t) To overcome these constraints, two novel mechanisms were introduced:  Adaptive Frame Rate (AFR)  Scene Cache Graph Management (SCGM) A semi-pull model was also designed for the synchronization of heterogeneous media objects.

5 System Architecture The MPEG-4 Media Process Flow

6 System Architecture (con’t)

7 Scene Cache Graph Management (SCGM)

8 Synchronization of Heterogeneous media

9 Categories of Media 1. Instant Time Media  The media data that can be computed immediately. 2. Discrete Time Media  The media whose sampling lengths are long enough so that computers can accomplish the processing. (ex: video & animation) 3. Continual Time Media  Those media whose sampling time is very short. (ex: auditory)

10 Semi-Pull Model A new decoding thread, is created for decoding. Each decoded data sample is associated with a composition timestamp. A composition timestamp is created with respect to a global reference clock generated by scene graph manager.

11 Intra-media Synchronization For instant time media, synchronization can be achieved if the Event Monitoring Procedure (EMP) execution rate is kept high enough. For other media types, semi-pull model ensures that the presented data sample is the most updated and rendered on time.

12 Inter-media Synchronization Synchronization between time-variant (discrete and continual) media  Forcing all media streams synchronized to the global clock.  Currently in MPEG-4, only audio streams have this problem. Audio streams are rendered by a sound card, which has a local clock.

13 Inter-media Synchronization (con’t) Synchronize the audio stream to the global clock by adjusting the amount of waveform data.

14 Implementation and Optimization

15 Adaptive Frame Rate Mechanism The job of visual render thread are divided into two procedures:  Event Monitoring Procedure (EMP)  Scene Render Procedure (SRP).

16 Adaptive Frame Rate Mechanism (con’t) In this mechanism, a scene runs in either active state or inert state. 1. Inert state  render thread adopts lazy-render strategy. with which the only periodic routine task is to execute EMP.

17 Adaptive Frame Rate Mechanism (con’t) 2. Active state  To continually render the active movie textures, render thread executes SRP in every specified period Pa.

18 2D/3D Graphic Engine Hardware acceleration for 3D rendering and floating-point number processing unit are not present on the target platform. To overcome these problems, the engine applies fixed-point arithmetic and performs all 3D rendering functionalities by software emulation.

19 Buffer Management This module maintains two buffers to hold decoded data. While a decoding thread writes data to one buffer, the render thread reads data from the other one. To avoid the race condition, both threads may lock the buffers.

20 Script Engine To enhance the interactivity of BlFS scene, the system supports JavaScript mechanism specified in MPEG-4. The script engine parses all the script codes in the scene and transforms them into a syntax tree for speeding up the script execution.

21 Experimental Results When the rendering frame rate is fixed at 12 fps, the average CPU load is 98.65% without SCGM. When SCGM is enabled, the CPU load is improved toward 6.37%. The 3D model placed in the middle →

22 Experimental Results (con’t) There are three objects, each of which is bound with a movie texture.  To measure the performance of AFR, we consider the lifetime of the battery with power consumption varies from 100% to 20%.  The experimental result shows that AFR gains 118% improvement of the lifetime in the first scene.

23 Conclusions This paper have successfully developed a MPEG-4 player on a resource constrained PDA. The system adopts four novel mechanisms:  Adaptive Frame Rate  Scene Cache Graph Management  Media Decoding Framework  Semi-pull Model

24 Conclusions (con’t) Resolving the problems including:  computational resource reduction  low power consumption  heterogeneous media synchronization For further, this system can be developed by new functionalities.

25 Seminar Thanks for comment !