1. Schedulability-Driven Partitioning and Mapping for Multi-Cluster Real-Time Systems
Paul Pop, Petru Eles, Zebo Peng, Viaceslav Izosimov Embedded Systems Lab (ESLAB) Linköping University, Sweden
Hello everybody, and thank for your introduction.
The paper I am presenting is about the design optimization of multi-cluster systems implementing hard-real time applications.
<click>
What is a multi-cluster?

2. Heterogeneous Networks
NoCs
Distributed Heterogeneous System
...
...
...
Factory Systems
Automotive Electronics
Heterogeneous Networks Multi-Cluster Systems
...
Let’s start with something we are all familiar, which is a distributed heterogeneous system.
Such a system is composed of several different types of nodes, interconnected in a network, and they are common in most application areas today.
<click>
But we also see an increasing number of systems implemented using several such networks, interconnected with each other.
Each network has its own communication protocol, and they communicate via a gateway which is a node connected to both networks. There can be several such heterogeneous networks, and you can have many types of topologies.
We call one network here a cluster, and thus the term multi-cluster to denote such a heterogeneous network.
These multi-clusters are used in increasing numbers in several application areas.
Networks on chip are heterogeneous, we also see them in, for example, factory systems, and they are very common in vehicles.
For example, this network is responsible to implement functionality related to the engine, while this one functions related to the powertrain, like brake-by-wire, anti blocking system, etc.

3. Distributed Safety-Critical Applications
...
Applications distributed over the heterogeneous networks
Reduce costs: use resources efficiently
Requirements: close to sensors/actuators
Gateway
Applications distributed over heterogeneous networks are difficult to...
Analyze (guaranteeing timing constraints)
Design (partitioning, mapping, bus access optimization)
As we have seen, multi-clusters are already used today in several application areas, and each cluster is responsible for implementing a family of related functiontions, like is the case in this picture, where we have two distributed applications, one on each cluster.
However, what we begin to see today are applications which are distributed across <click> the heterogeneous networks, like is the case with this application, which has processes runing on both clusters.
This trend is drvien by the need to reduce costs, especially in the case you have mass production. You have to use the resources more efficiently, and there are also other requirements, like being close to sensors and actuators, that force the designers to distribute applications over several clusters.
Not only these applications are distributed over several clusters, but they also have critical communication gowing through the gateway.<click>
This makes such applications inherently difficult to analyse and design.
For example, how can one determine whether such an application is schedulable?
And, how to efficiently implement these applications in order to reduce costs?
We have addressed the analysis problem in our previous DATE paper.
In this paper we address design problems characterisitc to multi-cluster systems: partitioning and mapping of functionality,
the optimization of the access to the communication infrastructure.
[DATE’03]
This paper!

4. Outline Motivation System architecture and application model
Scheduling for multi-clusters [DATE’03]
Design optimization problems
Partitioning
Mapping
Bus access optimization
Branch and bound optimization strategy
Experimental results
Contributions and Message
We have now seen the motivation behind our research...

5. Hardware Architecture
Gateway
...
Time-triggered cluster
Static cyclic scheduling
Time-triggered protocol
Event-triggered cluster
Fixed priority preemptive scheduling
Controller area network protocol S0 S1 S2 SG
TDMA Round
Cycle of two rounds
Slot
Time Triggered Protocol (TTP)
Bus access scheme: time-division multiple-access (TDMA)
Schedule table located in each TTP controller: message descriptor list (MEDL)
Controller Area Network (CAN)
Priority bus, collision avoidance
Highest priority message wins the contention
Priorities encoded in the frame identifier
We have considerd hardware architectures composed of two very common, and at the same time very distinct types of clusters: time-triggered clusters, and event-triggered clusters.
In the time-triggered cluster activation of processes and transmission of messages is based on the progression of time. Processes are scheduled statically, and messages are sent using a time-division bus protocol.
On the other hand, in the event-triggered cluster, activation of processes and transmission of messages is done at the occurence of significant events. Processes are scheduled using fixed-priority preemptive scheduling, while messages are transmitted using a priority bus protocol.
Each node in such a cluster has a CPU, memory, I/O interface to sensors and actuators, and a communication controller, that works independent of the CPU.
<click>
The communication on the time-triggered cluster happens according to the time-triggered protocol (TTP). The bus access scheme for TTP is time-division multiple-access, this means that each process in the network can transmit during a predetermined time interval, called slot. Several such slots for each node form a round, that repeats, with different contents, in a cycle. The access scheme is imposed by the TTP communication controller using a static schedule table for messages, called the message descriptor list.
The communication on the event-triggered network is performed using the controller area network (CAN) protocol. The CAN bus is a priority bus, which employes collision avoidance, where the message having the highest priority on the network gets to be transmitted.
In the paper we have only discussed this particular cluster configuration, but the whole approach can be extended easily to any type of cluster configurations.

6. Software Architecture
...
Gateway
Time-triggered cluster N1
CPU NG
CPU N2
CAN Controller
CPU
CPU
CAN Controller P1 P4 m1 m2 m1 m3
OutCAN
OutTTP
OutN2 T T m3 P2 P3
MBI
MBI
TTP Controller
TTP Controller
Event-triggered cluster
In order to provide an analysis for hard real-time applications, we have to know exactly how message passing takes place.
<click>
Here we have P1 sending messages m1 and m2 to P2 and P3, in this way... , and finally, we see how show how m1 arrives at the destination. <click>And we have process P2 sending message m3 to process P4. The transffer process T is responsible on the gateway to copy messages from one network to the other.
I don’t want to go into the details of message passing, they are in the paper, I just want to stress some important points.
1. On the nodes of the event-triggered cluster, and on the gateway, we have output queues where messages await their transmission.
There are no queues on the time-triggered culster nodes, because the processes and messages are synchronized based on the schedule tables computed off-line.
2. These schedule tables have to be produced in such a way, that when P4 is activated, it already has in its input buffer message m3. SG S1 SG S1
Round2

7. Multi-Cluster Scheduling [DATE’03]
Application,
Partitioning, Mapping, Architecture
TT Bus Configuration
Priorities
Response Times
Response Time Analysis
Static Scheduling
Offsets
Multi-Cluster Scheduling
Schedule Tables
Response times
Here we have our multi-cluster scheduling approach introduced at the previous DATE conference.
It takes as input the application, architecture, and mapping, the time-triggered bus configuration and the event triggered prioritites. These are the given system parameters, under which we are interested if the application is schedulable or not. And it produces the schedule tables and the response times, for the two types of clusters.
The algorithm starts by determining an initial static schedule. This will determine the offsets of the event-triggered processes and messages, that is, their earliest possible start time. If we are able to obtain smaller worst case response times, this means that we can also build a tighter schedule, thus deriving new offsets. This process continues until the schedule tables and the response times do not change any more, they converge to certain values.
We now have a multi-cluster scheduling algorithm, that takes as input the application, and the system configuration.
What we are interested next, is to derive a system configuration such that the application is schedulable.
MultiClusterScheduling algorithm
Schedulability analysis: communication delays through the gateway
Scheduling: cannot be addressed separately for each cluster

8. Problem Formulation Input Output System architecture Application
Partial partitioning and mapping, based on the designer’s experience
Output
Design implementation such that the application is schedulable
Partitioning for each un-partitioned process
Mapping for each un-mapped process
Priorities for ET messages
TDMA slot sequence and sizes for the TT bus
Priorities for ET processes
Schedule table for TT messages
Partitioning and mapping
Communication infrastructure
Scheduling information
Application: set of process graphs
...
Architecture: Multi-cluster
I can now formulate our problem in more detail.
As an input to our problem we have...
By partitioning we denote the decision whether a certain process should be assigned to the TT or the ET domain (and, implicitly, to a TTC or an ETC, respectively). Mapping a process means assigning it to a particular node inside a cluster.
Very often, the partitioning decision is taken based on the experience and preferences of the designer, considering aspects like the functionality implemented by the process, the hardness of the constraints, sensitivity to jitter, legacy constraints, etc.
Many processes, however, do not exhibit certain particular features or requirements which obviously lead to their implementation as TT or ET activities. Decisions concerning the partitioning of this set of activities can lead to various trade-offs concerning, for example, the schedulability properties of the system, the amount of communication exchanged through the gateway, the size of the schedule tables, etc.
For part of the partitioned processes, the designer might have already decided their mapping. For example, certain processes, due to constraints like having to be close to sensors/actuators, have to be physically located in a particular hardware unit.
But there are many processes for which the mapping has not been decided.

9. In which cluster to place process P4?
Motivational Example #1
In which cluster to place process P4?
Deadline for P6
Deadline for P5 P4 P5 P3 P6 P2 P1 N1
Preempted
(faster) N2
(slower) N3
Met P3 P4 P6 P5 P2 P1 N1
Preempted
(faster) N2
(slower) N3
Met
Preemption
not allowed P4 P5 P3 P6 P2 P1 N1
Missed
(faster) N2
(slower) N3
Met
In this example we have an application with six processes, P1 to P6, and four nodes, N1 and N2 on the TTC, N3 on the ETC and the gateway node NG. The worst-case execution times on each node are given in this table. Note that N2 is faster than N3, and an “X” in the table means that the process is not allowed to be mapped on that node.
The mapping of P1 is fixed on N1, P3 and P6 are mapped on N2, P2 and P5 are fixed on N3, and we have to decide how to partition P4 between the TT and ET domains. Let us also assume that process P5 is the highest priority process on N3.
In addition, P5 and P6 have each a deadline, these thick vertical lines.
We can observe that although P3 and P4 do not have individual deadlines,their mapping and scheduling has a strong impact on their successors, P5 and P6, respectively, which are deadline constrained. Thus, we would like to map
P4 such that not only P3 can start on time, but P4 also starts soon enough to allow P6 to meet its deadline.
As we can see in this example, this is impossible to achieve by mapping P4 on the TTC node N2. It is interesting to observe that, if preemption would be allowed in the TT domain, as in this hypothetical case, both deadlines could be met. This, however, is impossible on the TTC where preemption is not allowed.
TT preemption: off-line partitioning is very time-consuming, NP complete problem
ET preemption: automatically on-line
Both deadlines can be met only if P4 is mapped on the slower ETC node N3. In this case, although P4 competes for the processor with P5, due to the preemption of P4 by the higher priority P5, all deadlines are satisfied. P1 P2 P1 P2 P3 P4 N1 N2 X 50 70 40 90 P5 N3 P6
CAN
TTC
ETC N1 N2 N3 P3 P4
TTP P5 P6

10. Motivational Example #2
Where to map process P2?
Deadline P1 N1 P3
TTP S2 N2 S3 SG S1 P2
Missed m1 m2 N3
CAN NG N4
Met P1 N1 P3
TTP S2 T
CAN N4 P2 NG SG S3 S1 N3 m1 m2 N2
For a multi-cluster architecture the communication infrastructure has an important impact on the design and, in particular, the mapping decisions.
Let us consider the following mapping examplke. We assume that P1 is mapped on node N1 and P3 on node N3 on the TTC, and we are interested to map process P2.
P2 is allowed to be mapped on the TTC node N2 or on the ETC node N4, and its execution times are depicted in the table.
<click>
In order to meet the deadline, one would map P2 on the node it executes fastest, N2 on the TTC. However, this will lead to a deadline miss due to the TTP slot configuration which introduces communication delays. The application will meet the deadline only if P2 is mapped on the slower node, node N4 in this case.
Not only is N4 slower than N2, but mapping P2 on N4 will place P2 on a different cluster than P1 and P3, introducing extra communication delays through the gateway node. However, due to the actual communication configuration, this mapping alternative is desirable.
CAN
TTC
ETC N1 N2 N3 N4 m1 m2 N1 N2 N3 N4 P1 20 X X X NG P1 P2 P3 P2 X 40 X 50 P3 X X 20 X
TTP

11. Motivational Example #3
What are the priorities on ETC? Which slot should come first on the TTC?
Deadline P1 N1 P4
TTP SG T
CAN N2 P2 NG S1 P3 m4 m3
Met
Round 1 m2 m1 P1 N1 P4
TTP S1 T
CAN N2 P2 NG SG m1 m2 P3
Missed m3 m4 P1 N1 P4
TTP S1 T
CAN N2 P2 NG SG m1 m2 P3 m4 m3
Round 4
Missed
CAN P1 P2 P3 P4 N1 N2 20 X 40 m1 m2 m3 m4
TTC
ETC N1 N2
TTP

12. Optimization Strategy
Multi-Cluster Configuration
Branch and Bound Partitioning and Mapping
Branching rule
Selection rule
Bounding rule: lower bound
Bus Access Optimization
Determines the slot sequence and lengths on the TTC, message priorities on the ETC
Greedy optimization heuristic
Straightforward solution
Partitioning and mapping that balances the utilization of processors and buses
Could be produced by a designer without optimzation tools
...Our strategy decides partitioning implicitely by deciding the mapping...

13. Can we increase the number of schedulable applications?
Experimental Results
Can we increase the number of schedulable applications? 0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% 50
100
150
200
250
Multi-cluster Configuration
Percentage schedulable applications [%]
Straightforward solution
graph generation
What are the 100% white bars
The non-schedulable white bar on top could contain applications which are not schedulable regardless of resources
Number of processes

14. Experimental Results, Cont.
How time-consuming is our optimization strategy? 45 50
100
150
200
250
Multi-Cluster Configuration 40 35 30 25
Average execution time [minutes] 20 15
Case study
Vehicle cruise controller
Distributed over the TT and ET clusters 10
multi-cluster takes long time because
Complete synthesis loop, contains three steps
MultiClusterScheduling is a complex analysis 5
Straightforward solution
Number of processes

15. Contributions and Message
Addressed design problems characteristic to multi-clusters
Partitioning
Mapping
Bus Access Optimization
Proposed a branch and bound approach for optimization
Analysis and optimization methods are needed for the efficient implementation of applications distributed over interconnected heterogeneous networks.