2. Real-Time Multitasking
5.1 Introduction
Task Basics
Task Control
Error Status
System Tasks
What is real-time?
Advantages of multitasking
Tasks and task states

3. What is Real-Time?
Real-time denotes the ability of a control system to keep up with the system being controlled.
A system is described as being deterministic if its response time is predictable.
The lag time between the occurrence of an event and the response to that event is called latency.
Deterministic response is the key to real-time performance.

4. Requirements of a Real-time System
Ability to control many external components:
Independent components
Asynchronous components
Synchronous components
High speed execution:
Fast response
Low overhead
Deterministic operation:
A late answer is a wrong answer.

5. Design Options
How do we design a system to control a robot arm?

6. Unitasking Approach One task controlling all the components in a loop.
arm () {
for (;;)
if (shoulder needs moving)
moveShoulder();
if (elbow needs moving)
moveElbow();
if (wrist needs moving)
moveWrist(); . }
Disadvantages of unitasking approach:
Difficult to control the components at different rates
Difficult to assign components priorities
No preemption capability
Advantage:
No context switch overhead.

7. Multitasking Approach
Create a separate task to manipulate each joint:
Each task alternates between “ready” and “waiting”.
VxWorks allows a task to wait for:
A specified time delay
An event (e.g. an interrupt)
joint () {
for (;;)
waitForMove(); /* Until joint needs moving */
moveJoint(); }
Multitasking advantages:
Using different wait commands for each task permits manipulating the components at different rates.
Using a priority-based scheduler permits running the most important task in the event that two tasks are in the “ready” state at the same time.
Using a preemptive scheduler permits an important task which enters the “ready” state to start running right away (preempting the less important task previously running).
Permits easier addition of new components.

8. Task States Non-suspended tasks may be in one of four states:
Ready Tasks eligible to execute. The actually executing task has highest priority among all ready tasks. The other tasks in the ready queue are waiting for the CPU.
Delayed Tasks waiting for an amount of time to pass.
Pended Tasks waiting for an object event (e.g., a shared resource becoming available).
Pended & Delayed Tasks waiting for a object event, but with a timeout specified.
Tasks may, additionally, be suspended:
Suspended tasks will not execute.
Tasks that are delayed or pended can be suspended. When resumed they will return to the appropriate state.

9. Multitasking Kernel
The Wind kernel is that part of VxWorks which directly manages tasks.
Allocates the CPU to tasks according to the VxWorks scheduling algorithm (to be discussed).
Uses Task Control Blocks (TCBs) to keep track of tasks.
One per task
Declared as WIND_TCB data structure in taskLib.h
O.S. control information e.g. state, task priority, delay timer, breakpoint list, error status, I/O redirections, etc.
CPU Context Information e.g. PC, SP, CPU registers, FPU registers
The TCB actually contains a pointer to the floating point register set storage location for those tasks which use floating point.
There are a few spare fields at the end of the TCB which may be used by an application to store additional information specific to a task. See the declaration in taskLib.h. The function taskTcb() may be used to obtain a pointer to a task’s TCB.

10. Kernel Operation
Kernel manages the tasks, moving them from state to state based on kernel operations invoked.
A separate pend queue exists for each “event”. For example, an event could be a particular semaphore’s being given, or a message arriving across a particular message queue.
A task in the PENDED + DELAYED state is linked into both a pend queue and the delay queue. The link nodes are part of the TCB.
The ready queue is not a simple linked list. It is a bit-mapped priority queue, so that insertion and deletion are done in constant time.
Whenever any of the above queues is manipulated, kernel state is entered. Upon exiting kernel state, code executes to check whether a reschedule (context switch) should occur.

11. Context Switch
When one task stops executing and a new task starts, a context switch or reschedule has occurred.
To schedule a new task to run, the kernel must:
1. Save context of currently executing task into its TCB.
2. Restore context of next task to execute from its TCB.
Complete context switch must be very fast.

12. Types of Context Switches
Synchronous context switches occur because the executing task
pends, delays, or suspends itself.
makes a higher priority task ready to run.
(less frequently) lowers its own priority, or exits.
Asynchronous context switches occur when an ISR:
(less frequently) suspends the current task or lowers its priority.
Synchronous context switches require saving fewer registers than asynchronous context switches, and so are faster.
A task which pends or delays or suspends itself is said to block.
ISR stands for Interrupt Service Routine, the routine which executes when a hardware interrupt occurs. Such a routine executes in preference to any task unless interrupts are locked out.

13. Priority Scheduling
Different application jobs may have different precedences, which should be observed when allocating the CPU.
Preemptive scheduling is based on task priorities chosen to reflect job precedence.
The highest priority task ready to run (not pended or delayed) is allocated the CPU.
Rescheduling can occur at any time as a result of:
Kernel calls
Interrupts (e.g., system clock tick)
The context switch is not delayed until the next system clock tick.
Reschedules do not occur merely because a task has been executing for a long time. (See, however, the discussion of round-robin scheduling on subsequent slides).
When an interrupt causes a reschedule, it is because the interrupt service routine called a kernel function. A special case of this is the processing of timeouts and delays which occurs on every system clock tick interrupt.
The precedence of a job may be more determined by timing requirements than by the job’s "intrinsic importance," which is often nebulous.

14. Priority Based Preemption
Rescheduling occurs if
The currently running task pends or delays.
A task of higher priority becomes ready.
When a task of higher priority takes the CPU away from the current task, it is preempting that running task.
If task B never blocks or pends, then task C will never run.
Equal priority tasks won’t preempt each other (by default).

15. Round-Robin Scheduling
Tasks A to D (same priority) share the CPU on an equal basis.
Round-robin mode makes all tasks of the same priority share the CPU fairly.

16. kernelTimeSlice(ticks)
Slicing Time
To allow equal priority tasks to preempt each other, time slicing must be turned on:
kernelTimeSlice(ticks)
If ticks = 0, time slicing is turned off.
Priority scheduling always takes precedence.
Round-robin only applies to tasks of the same priority.
Priority-based rescheduling can happen any time.
Round-robin rescheduling can only happen every few clock ticks.
When time-slicing is on, it is on globally for tasks of all priorities. If the tasks of priority 100 use round robin to share the CPU equally amongst themselves, then the tasks of priority 120 must also use round robin to share the CPU equally amongst themselves.
Ticks is specified in system clock ticks. To get the clock frequency (in Hz):
-> clockRate = sysClkRateGet( )

17. Performance Enhancements
All tasks reside in a common address space.
All tasks run in supervisor (privileged) mode.

18. Multitasking Facilities
Picture above represents an overview of the VxWorks multitasking facilities:
Message passing queues for intertask communication within a CPU
Network routines for intertask communication between CPU’s
Semaphores to protect data which is shared by tasks
Timers and other interrupts to trigger task execution
I/O system facilities to read/write data to hardware devices

19. How VxWorks Operating System Meets Real-time Requirements
Able to control many external components
Multitasking allows solution to mirror the problem.
Independent functions are assigned to different tasks.
Intertask communication allows tasks to cooperate.
High speed execution
Tasks are light-weight, enabling fast context switch.
No “system call” overhead.
Deterministic operation
Preemptive priority scheduling assures response for high priority tasks.

20. Real-Time Multitasking
Introduction
5.2 Task Basics
Task Control
Error Status
System Tasks
Task creation
Task names and id’s
Task priorities
Task stacks
Task options
Task deletion
Deletion safety
Resource reclamation

21. Overview
Low level routines to create and manipulate tasks are found in taskLib.
A VxWorks task consists of:
A stack (for local storage of automatic variables and parameters passed to routines).
A TCB (for OS control).
Do not confuse executable code with the task(s) which execute it:
Code is downloaded before tasks are spawned.
Several tasks can execute the same code (e.g., printf( )).
VxWorks tasks are similar to threads in process-model systems. (However, there will be differences between the threads of any particular implementation and VxWorks tasks.)

22. Creating a Task To create a task, VxWorks must:
Allocate memory for the stack and TCB from a memory pool. The taskSpawn() function allocates these in a single contiguous block.
Initialize the stack (e.g., create an initial stack frame with arguments passed to the task).
Initialize the TCB (e.g., store a pointer to the entry point function in the TCB; set the saved PC to the address of the wrapper routine vxTaskEntry which will call the entry point function; initialize the stack pointer; etc.).
Put task into ready queue.

23. Creating a Task
int taskSpawn (name, priority, options, stackSize, entryPt, arg1, …, arg10)
name Task name, if NULL gives a default name.
priority Task priority
options Task options e.g. VX_UNBREAKABLE.
stackSize Size of stack to be allocated in bytes.
entryPt Address of code to start executing (initial PC).
arg1, ..., arg10 Up to 10 arguments to entry-point routine.
Returns a task id or ERROR if unsuccessful.
Example:
newTid = taskSpawn (“tMyTask”, 150, 0, 20000, myRoutine, arg1, arg2, 0, 0, 0, 0, 0, 0, 0, 0)
The sp( ) shell built-in is equivalent to taskSpawn( ) called with default parameters.

24. Task ID’s Assigned by kernel when task is created.
Unique to each task.
Efficient 32-bit handle for task.
May be reused after task exits.
A task id of zero refers to task making call (self).
Relevant taskLib routines:
taskIdSelf( ) Get ID of calling task.
taskIdListGet( ) Fill array with ID’s of all existing tasks.
taskIdVerify( ) Verify a task ID is valid.
Note: use of taskIdListGet( ) and taskIdVerify( ) may generate race conditions.

25. Task Names Provided for human convenience.
Typically used only from the shell (during development).
Within programs, use task Ids.
By (often ignored) convention, task names start with a t.
Promotes interpretation as a task name.
Default name is a t followed by an ascending integer.
Doesn’t have to be unique (but usually is).
Relevant taskLib routines:
taskName( ) Get name from tid.
taskNameToId( ) Get tid from task name.
When the WDB agent is in task mode, tasks spawned with the WindSh built-in sp( ) are given a name sMuN, where the integer M numbers the shell instance and N increments each time sp( ) is used in that shell. Thus, even the WindSh shell does not always obey the (soft) task naming convention!

26. Task Priorities Range from 0 (highest) to 255 (lowest).
Determining how to set task priorities is a difficult subject, and beyond the scope of this course. However:
Timing requirements rather than hazy ideas about task importance should govern priorities.
A substantial body of theory exists.
One can manipulate priorities dynamically with:
taskPriorityGet (tid, &priority)
taskPrioritySet (tid, priority)
Doing so may make your application’s behavior more difficult to analyze.
For debugging, a task’s priority should be lower than the WDB agent task’s priority to ensure that the Tornado tools can control and debug the task. Further, communicating with the WDB agent via a network connection will only be possible if the network task tNetTask gets a chance to run. For more information on VxWorks system tasks, see the System Tasks section of this chapter.
A nice introduction to scheduling theory is provided in chapter 13 of Burns and Wellings, “Real-Time Systems and Programming Languages.” 2nd Ed., Addison Wesley Longman, ISBN X
A more encyclopaedic treatment is provided in Klein, et. al., “A Practitioner’s Handbook for Real-Time Analysis: Guide to Rate Monotonic Analysis for Real-Time Systems.” Kluwer Academic Publishers, 1993, ISBN This book assumes previous experience with real-time development and RMA.

27. Task Stacks Allocated from system memory pool when task is created.
Fixed size after creation.
The kernel reserves some space from the stack, making the stack space actually available slightly less than the stack space requested.
Exceeding stack size (“stack crash”) causes unpredictable system behavior.
Once the stack is overwritten, the integrity of the system is lost. The output of checkStack( ), on the next page, illustrates this, indicating that the task name of the third task has been lost.
Each byte in a task’s stack is filled with 0xee when the task is created. The checkStack( ) function searches from the end of the stack for this value in order to find the maximum stack usage.
VxSim targets add 8000 bytes to the requested stack size, to deal with simulated interrupts, which are handled on the stack of the current task.

28. Stack Overflows To check for a stack overflow use the Browser:
Press the check-stack button:
Examine the high water mark indicator in the stack display window.
High water mark indicator (Unix) Current usage shown by shaded bar and number inside.
From the shell:
-> checkStack (taskNameOrId)
If taskNameOrId is 0, summarize for all tasks.
A target-based implementation of checkStack() is available in wind/target/src/usr/usrLib.c.
High water mark shown by shaded bar. (Windows)
Current usage shown by number inside bar.

29. Task Options Can be bitwise or’ed together when the task is created:
VX_FP_TASK Add floating point support.
VX_NO_STACK_FILL Don’t fill stack with 0xee’s.
VX_UNBREAKABLE Disable breakpoints.
VX_DEALLOC_STACK Deallocate stack and TCB when task exits (automatically set for you).
Use taskOptionsGet( ) to inquire about a tasks options.
Use taskOptionsSet( ) to unset VX_DEALLOC_STACK.
Options are defined in taskLib.h.
VX_DEALLOC_STACK is set by taskSpawn() even if this bit is not present in the passed task options. It is not set by the taskInit() function, which allows you to specify the locations of a task’s stack and TCB (possibly in statically allocated memory), and creates the new task in a suspended state.
VX_SUPERVISOR_MODE is an obsolete option.
You may or may not see this option when calling taskOptionsGet( ).
All tasks run in supervisor mode, regardless of this option.
VX_STDIO is an obsolete option.
Tasks can now use buffered I/O regardless of this option.

30. Task Creation
During time critical code, task creation can be unacceptably time consuming.
To reduce creation time, a task can be spawned with the VX_NO_STACK_FILL option bit set.
Alternatively, spawn a task at system start-up which blocks immediately, and waits to be made ready when needed.
We will look at routines allowing a task to pend later in the course.

31. Task Deletion taskDelete (tid) exit(code) Deletes the specified task.
Deallocates the TCB and stack.
exit(code)
Equivalent to a taskDelete( ) of self.
code parameter is stored in the TCB field exitCode.
TCB may be examined for post-mortem debugging by
Unsetting the VX_DEALLOC_STACK option or,
Using a delete hook.
The exit( ) routine is called automatically if a task returns from its entry-point function.
In general, programmatically deleting another task is risky, because it is often impossible to know whether it is safe to do so, given that a task thus deleted could have had an exclusive lock on a resource which would thereby never be released, and left in an inconsistent state as well. This issue will be discussed further in a later chapter.

32. Resource Reclamation
Contrary to the philosophy of sharing system resources among all tasks.
Can be an expensive process, which must be the application’s responsibility.
TCB and stack are the only resources automatically reclaimed.
Tasks are responsible for cleaning up after themselves.
Deallocating memory.
Releasing locks on shared resources.
Closing files which are open.
Deleting child tasks when parent task exits.
In VxWorks all tasks are peers, with no strict parent-child relationship as exists in UNIX and other operating systems: VxWorks does not record which task spawned a given task. However, a conceptual parent-child relationship may exist, as in the case of a concurrent server application, wherein a server task spawns subordinate tasks for concurrent processing of client requests.

33. Real-Time Multitasking
Introduction
Task Basics
5.3 Task Control
Error Status
System Tasks
Restart
Suspend/Resume
Delay
Task Variables and Task Hooks
Task Information

34. Task Restart taskRestart (tid)
Task is terminated and respawned with original arguments and tid.
Usually used for error recovery.
If a task has modified the arguments passed to its entry point function, it may be restarted with either the modified values or the original values, depending on the target architecture, the number of arguments actually used, and the level of compiler optimization. If the task has changed priority or options since it was first started, its original priority and options will not be restored.
If a task tries to restart itself, the work of doing this is actually handled by a new task of priority 0 named tRestart which is spawned for just this purpose. The tRestart task terminates after restarting the original task. The task tExcTask is not involved.

35. Task Suspend/Resume taskSuspend (tid) taskResume (tid)
Makes task ineligible to execute.
Can be added to pended or delayed state.
It is safest to have a task suspend itself.
taskResume (tid)
Removes suspension.
Usually taskSuspend() and taskResume() are used for debugging and development purposes.

36. STATUS taskDelay (tics)
To delay a task for a specified number of system clock ticks:
STATUS taskDelay (tics)
To poll every 1/7 second:
FOREVER {
taskDelay (sysClkRateGet( ) / 7)
... }
Accurate only if clock rate is a multiple of seven ticks/second.
Can suffer from “drift.”
Use sysClkRateSet( ) to change the clock rate.
sysClkRateGet ( ) and sysClkRateSet ( ) are part of the BSP library sysLib. They may be implemented in either sysLib.c, or in a clock chip driver under target/src/drv/timer which might be shared among several BSPs.
Changes the task state from “ready” to “delayed.”

37. Reentrancy and Task Variables
If tasks access the same global or static variables, the resource can become corrupted (called a race condition).
Possible Solutions:
Use only stack variables in applications.
Protect the resource with a semaphore.
Use task variables to make the variable private to a task.
Task Variables cause a 32-bit value to be saved and restored on context switches, like a register.
Caveat: task variables increase context switch times.
See the taskVarLib manual pages for details.
Because task variables increase context switch times, they should be avoided when possible. They are intended for use when the interface to non-reentrant process model legacy code must be preserved.
Task variables can cause VxWorks to save and restore an arbitrary number of 32-bit, global or static variables on context switches. An application requiring many task variables should:
1. Define all of its task variables in a structure.
2. Access this structure through a global or static pointer
3. Make this pointer the task variable.
4. Have each task allocate its own structure
Now VxWorks need save and restore only a single task variable.
Some resources, such as memory shared for inter-task communication, should be protected with semaphores rather than with task variables.

38. Task Hooks
User-defined code can be executed on every context switch, at task creation, or at ask deletion:
taskSwitchHookAdd ( )
taskCreateHookAdd ( )
taskDeleteHookAdd ( )
VxWorks uses a switch hook to implement task variables.
See manual pages on taskHookLib for details.
The hook tables have space for 16 switch hooks, 16 create hooks, and 16 delete hooks. VxWorks facilities may take up some of the available hooks. For instance, fppLib makes use of a create hook for floating point support; envLib makes use of both a create hook and a delete hook; the WDB agent and the ansiStdio library make use of delete hooks; the WDB agent also uses switch hooks to manage task-specific breakpoints and stepping.
Task switch hooks and create hooks are called in the order in which they were installed. Task delete hooks are called in the reverse of the order in which they were installed.
Create hooks run in the context of the creating task. Delete hooks run in the context of the deleting task, or tExcTask if a task exits or otherwise tries to delete itself. Switch hooks run in a minimal context in which only a few VxWorks routines may be legitimately called.

39. Task Information ti (taskNameOrId) Like i( ), but also displays:
Stack information
Task options
CPU registers
FPU registers (if the VX_FP_TASK option bit is set).
Can also use show ( ):
-> show (tNetTask, 1)

40. Task Browser
To obtain information about a specific task, double-click on the task’s summary line in the main task browser display, or enter the task’s ID in the Show box.

41. What is POSIX?
Originally, an IEEE committee convened to create a standard interface to UNIX for:
Increased portability.
Convenience.
VxWorks supports almost all of the b POSIX Real- time Extensions.
The POSIX real-time extensions are based on implicit assumptions about the UNIX process model which do not always hold in VxWorks. In VxWorks,
Context switch times are very fast.
Text, data, and bss segments are stored in a common, global address space.
POSIX is an acronym for Portable Operating System Interface (The ‘X’ was added to make it UNIXish.).
POSIX standard b was formerly called

42. What does VxWorks Support?
Library Description
aioPxLib Asynchronous I/O
semPxLib POSIX Semaphores
mqPxLib POSIX Message Queues
mmanPxLib POSIX Memory Management
schedPxLib POSIX Scheduler Interface
sigLib POSIX Signals
timerLib, clockLib POSIX Timer/Clock Interface
dirLib File/Directory Information
Of the POSIX b real-time extensions, the following functions are not presently supported:
For more information about how to use the POSIX real-time extensions, consult the Programmer’s Guide, the VxWorks Reference Manual and the online man pages.

43. Real-Time Multitasking
Introduction
Task Basics
Task Control
5.4 Error Status
System Tasks

44. Error Status
Global integer errno is used to propagate error information:
Low level routine detecting an error sets errno.
Calling routine can examine errno to discover why the routine failed.
Code using errno should include errno.h.

45. Errno and Context Switches
errno is part of task’s context. A task’s error status is saved and restored on each context switch just like a register’s value:
Error value for executing task is stored in the global variable errno.
Error value for non-executing task is stored in task’s TCB.
At each context switch, the kernel saves and restores the value of errno.

46. Setting Errno
Lowest level routine to detect an error sets errno and returns ERROR:
STATUS myRoutine() {
...
if (myNumFlurbishes >= MAX_FLURBISH)
errno = S_myLib_TOO_MANY_FLURBISHES;
return (ERROR); }
pMem = malloc (sizeof(myStruct));
if (pMem == NULL)
/* malloc() sets errno - do’nt redefine it */
return (ERROR)

47. Examining Errno Examine errno to find out why a routine failed.
errno is only valid after an error occurs.
if ( reactorOk( ) == ERROR ) {
switch (errno)
case S_rctorLib_TEMP_DANGER_ZONE: startShutdown( );
break;
case S_rctorLib_TEMP_CRITICAL_ZONE: logMsg(“Run!”);
case S_rctorLib_LEAK_POSSIBLE: checkVessel( );
default: startEmergProc( ); }

48. Interpreting Errno VxWorks uses the 32-bit value errno as follows:
VxWorks module numbers are defined in vwModNum.h.
Each module defines its own error numbers in its header file.
For example, an errno of 0x would be:
Module number 0x11 (defined in vwModNum.h to be memLib) and
Error number 0x01 (defined in memLib.h to be “not enough memory”).
module
error number

49. Error Messages
VxWorks uses an error symbol table (statSymTbl) to convert error numbers to error messages.
To obtain the error string corresponding to errno:
To print the error message associated with an error number to the WindSh console:
-> printErrno (0x110001)
S_memLib_NOT_ENOUGH_MEMORY {
char errStr [NAME_MAX];
strerror_r (errno, errStr);
... }
printErrno( ) is a WindSh primitive. With no argument, it prints the errno value for the task which executed the most recent target function call from the shell. A target-based version is available in target/src/usr/usrLib.c.
NAME_MAX is defined in limits.h, and strerror_r() is declared in string.h.
The routine perror( ) may be used to print out the error string corresponding to the calling task’s current errno value on STD_ERR.

50. User-Defined Error Codes
To allow strerror_r() to support your error messages:
1. Create a user header file directory.
2. Create a file xxModNum.h in the user header directory:
#define M_myLib (501 << 16)
3. Define error macros in your header files (which must be in the user header directory):
#define S_myLib_BAD_STUFF (M_myLib|1)
4. Rebuild the system error table, statTbl.o.
5. Include the component development tool components > symbol table components > error status table in VxWorks. Add statTbl.o to the EXTRA_MODULES macro.
6. Rebuild VxWorks.
VxWorks reserves the first 500 module numbers.
Your error macro must be defined as S_xxx.
To rebuild the system error table, do the following
1. Execute makeStatTbl :
(UNIX) makeStatTbl systemHeaderDir userHeaderDir > statTbl.c
(Windows) makeStatTbl systemHeaderDir userHeaderDir
where systemHeaderDir is the path to the target/h subdirectory of the Tornado tree.
2. Compile the resulting file statTbl.c to produce statTbl.o.
The work of building the error table and linking it with VxWorks may be automated using the project facility, with a makefile extension. A detailed procedure is provided as an advanced lab.

51. Real-Time Multitasking
Introduction
Task Basics
Task Control
Error Status
5.5 System Tasks
This section provides an overview of VxWorks system tasks.

52. System Tasks
Task Name Priority Function
tUsrRoot 0 First task. Initializes included facilities, spawns user application, and exits.
tLogTask 0 Message logging.
tExcTask 0 Server which executes miscellaneous task-level functions at high priority.
tWdbTask 3 WDB run-time agent.
tNetTask 50 Task-level network functions.
tFtpdTask 55 FTP server.
These tasks are started when VxWorks boots.

53. Summary Real-time multitasking requirements:
Preemptive priority-based scheduler
Low overhead
Task properties stored in task’s TCB.
OS control information (priority, stack size, options, state, ...).
Saved CPU context (PC, SP, CPU registers, ...).
taskSpawn( ) lets you specify intrinsic properties:
Priority
Stack size
Options

54. Summary Task manipulation routines in taskLib: Task information
taskSpawn/taskDelete
taskDelay
taskSuspend/taskResume
Task information
ti/show
Task Browser
Additional task context:
errno
Task variables
Task hooks