1. SPACE SPlit Application architeCturE Enabling predictable SW integration
Bas Engel
Royal Philips / Consumer Electronics Division
CELF Embedded Linux Conference 2007
April 17, 2007

2. Introduction

3. SW in TV Software size evolution Software partitioning
Total SW size follows Moore’s law
Philips part decreasing (not just relative)
3rd party involvement increasing

4. Some rationales on 3rd party SW
Cost, however
balance NRE and royalties vs. own development cost
TTM
No time to build up competence in own team
“Off the shelf solution available”
Critical competence available at supplier side
MHEG: move to 1.06 is investment on both sides (creation vs validation) but creates healthy atmoshpere
Somebody else makes a business out of supplying common features

5. Traditional approach: integrate
process
Philips
3rd party
MIPS (Linux 2.x)

6. Integration nightmare
Integration has it’s advantages
Optimized resource usage
Better viewing experience due to integral architecture approach
Integration pitfall
High effort due to increased complexity
Longer TTM as there is no little change
System validation requires global big bang approach
No independent lifecycle, no distributed integration cycle

7. Create a Win-Win atmosphere with 3rd parties
The challenge
Create a Win-Win atmosphere with 3rd parties
We have to create an environment in which 3rd parties can integrate their software
Without deep system knowledge
We have to enable predictable integration
Preventing spaghetti SW
We must cater for sharing critical resources
AVG Platform
General purpose infrastructure
1 Wat zijn de ervaringen (positief en negatief ) met ISVs (tav integratie, effort, communicatie, roadmapping,
Market/Business aspect
1394 wrong market assesment 
MHP/OCAP don’t initated yet
Organizing for buy aspect
Gemstar
Cabot L1/L2 tipsi….
1 een soort van do's en don't's, wat kunnen ze wel en wat kunnen ze niet goed, etc) -> kan ook al in Deel 3
WIN WIN atmosphere
What when price but especially the how
Gemstar NRE discussion at last moment for extra testing
Mutual benefit
MHEG roadmap is a cabot worry

8. Design objectives
Fast and predictable integration of system extensions
Avoid an extensive (re)validation cycle
Enable convincing module test opportunities
Enable PC based testing
Orthogonal 3rd party SW integration
Leverage standard Linux infrastructure
Multi client usage of platform resources
Manage independently deployable building blocks
Limited building block correlation
Cater for extensions without the need to know all the details
Enhanced execution architecture
Independent application lifecycle
Enable true Multi window architecture
For Video and Graphics
Preserving application orthogonality

9. The mental model: SW bolt-ons
Feat1
Feat2
Feat3
Feat4
process TV
Middleware
TV Platform
MIPS (Linux 2.x)

10. SPACE

11. Introducing SPACE Have orthogonal applications Application
Feat1
FeatN
TV Middleware
Platform API
TV platform
Application Manager
Linux 2.x
Have orthogonal applications
The resources in the system are explicitly and centrally managed
The client applications are system context unaware
The lifecycle, focus and visual layout of the client applications is centrally managed

12. Application Manager Application lifecycle management
Starting, stopping applications based on remote control keys
Knows the system requirements of all applications
Focus handling
Determines the active window, reacts to user request
Manages the application requirements to the AV resources
Layout management
Determines how the applications are presented

13. Managing resources
There are implicitly managed resources by the kernel
TCP, flash, USB
Memory allocation
There are explicitly managed resources
AV platform resources
Memory and CPU resources
The explicitly managed resources
Have a statically defined execution behavior
Can by design limit the parallelism in the system
Require an explicitly resource controlled system

14. Resource Controlled System
Application Manager (amApp)
Knows the resource dependencies of any client application
Statically by design
The platform application (plfApp)
Has a static model of resource sharing
By design resources are multi/single client
Every application
Must be resource aware
Meaning the resource is or is not reserved for them

15. Multi Client Management
tvApp
otherApp
SetFrequency (tuner)
SetFrequency (tuner)  
amApp
plfApp
ResourceOwner (tvApp)
Linux 2.x

16. Resource Groups
There are different resource groups offered by the platform
Front-end, demux, decode, viewing mode, source selection
To allow multiple applications to access different parts of the system
Every resource group has a static set of interfaces
The Application Manager is the main Resource controller
It designates resource groups to clients
By informing the platform application whom to grant access

17. ? ? ? The platform API The Philips platform API
Leverages as much as possible the current APIs
Enables the resource sharing model
Is supplier independent
AV Interface definition
Analog interfaces: Philips’ Analog TV API
Digital interfaces: NXP’s Digital TV API
AV Interface instance
Simple header files and libraries
Tool to generate proxy/stub code
Graphics interface definition
DirectFB ? ? ?

18. General Connection Management Concept
Connection Management is split up in
Destination setup
Output: FullScreen, PIP
Source setup
Input: HDMI, Tuner
Connection Management is distributed
Application Manager is responsible for
Client application lifecycle
Client applications are responsible for
Client applications are destination unaware
Application Manager is source unaware

19. Example tvApp amApp plfApp Linux 2.x 3 SetSource(tuner)
6 Program Freq, PID, …
2 eventDestination(tvApp)
5 eventOnSourceSelected()
tvApp
4 SelectTuner
amApp
plfApp
Linux 2.x
1 SetDestinationFullScreen(tvApp)

20. Border Windows and Video Windows
Border windows are linked to video windows
Every application controlling video creates a border window
Application manager controls visibility of such a couple
Visible border windows are equal to the visible video windows
There can be multiple border windows for 1 video window
The border window determines the position of the video window
DirectFB support this via the concept of Input Only Windows
Geometry (positioning) is still available to allow focus management
No client buffer is required, no scaling is done

21. Graphics Connection Management
tvApp
App2
amApp
Select
App1
Draw
configure
DirectFB infrastructure
source
destination

22. Window Management responsibilities
amApp
start/kill
Layout
Size
focus
tvApp
App1
App2
tvApp
create
draw
App1
App2

23. Traditional DirectFB Window manager
Pluggable WM API
Move, Resize, Raise, Lower, Hide, Show Windows
Process Input, Handle Focus and Grabbing, Dispatch Events
Full Cursor implementation including Show, Hide, Reshape, Move
Composition of the screen or parts, e.g. triggered by
Flip() on a window surface
Move, Raise, Show etc.
2D composition
Background, Windows, Cursor
Default WM module
Just follow application requests
Basic focus handling
App
App
DirectFB API
WM Module
SHM

24. SaWMan Shared Application & Window Manager
New DirectFB window manager
On its own like default window manager implementation
Default is maintained to preserve backwards compatibility
Required was a more controlled behavior
Add/Remove windows, can modify initial configuration
Configuration requests (move, resize, opacity), modify or reject
Filter input events before being processed by SaWMan
Completely overrule window layout, enable focus borders etc.
Application Manager can hook into most of the API flow
Application Manager
App
App
App
SaWMan API
DirectFB API
SHM
RPC to AM
SaWMan Module

25. Window Management Layers Z-order Clients Message Application Border
Focus
mgt
Layers
Z-order
Clients
Child abducted San Jose
Child abducted San Jose
Message
high
Application
med
Border
low
Video HW

26. DirectFB layer mapping on HW
If we need fast rendering (>10FPS) and no HW acceleration available
We must use direct surface drawing
To achieve >25 FPS without SW scaling overhead
DirectFB must allow Layers to be mixed across HW Surfaces
Some Layers can have SW scaling, others may not
Legend
Graphics
Versatile
Layers
Z-order
Surface
Layers
Z-order
Surface
Performance
Video
Message
GFX1
Message
GFX1
high
high
Application
GFX1
Application
GFX1
med
med
Border
GFX1
Border
GFX0
low
low
Video
Video0
Video
Video0 HW HW
Option 1
Option 2

27. Window Association Windows can be associated to another
Meaning identical position and size
Focus only to associated group
Focus only to top window, not associated window
Example use-case: teletext + menu
Teletext and menu of teletext separate windows
CreateAssociative
Draw
Client
Draw
window
Flip
time

28. Pixel Alignment plfApp can crop video
Eg to remove DNM border artifacts
plfApp can scale video within border window
Eg to set the scaling to 4:3 or 16:9
Any client application can indicate ‘pixel alignment’
To assure that the graphics is pixel accurately positioned on top of the video
DirectFB caters for the alignment
plfApp configures the scale/crop factors
Other application is pan/zoom
Any application can use the SW scaling
Crop &
Scale
Crop value
VideoAligned
Crop &
Scale

29. Road to success

30. Planned extensions to DirectFB
SawMan
Shared Application Window Manager
Enhanced SW up/down scaling for windows
With dynamic layer reconfiguration and color key conversion
FusionDale & FusionIPC
Applied Fusion for Event handling and IPC
Memory usage optimization
Shared Memory Heap
Window Association
Link windows together for size and location
Window mapping on HW layers
Including layer specific configurations Pixel alignment
Extended Key handling
Sending keys to windows and return key not used
Audio Nodes
General ID for audio resources
Multi processor SPACE
Enable multiple control processors to host client applications
Published
To be published
Plan

31. Main conclusion The integration nightmare can become manageable
Using an open standard based multi application approach
DirectFB, Linux
With a clear orthogonal view
No dependencies between applications
Adding the necessary components for success
SPACE, SaWMan
SPACE
Is the architectural concept for multi application systems
Coping with the resource constraints in CE devices
SaWMan
Enables application lifecycle management, focus handling, and window management
Is fully available via the DirectFB mainline

32. Demo: SPACE