1. VxWorks & Memory Management
Group A7
CSE8343

2. Agenda General Overview
High level overview of how VxWorks thinks about memory
Virtual Memory
Caching
Specific example of how VxWorks allocates memory on a Synergy Dual Processor

3. General Overview
In basic VxWorks, all memory can be conceived as a singular linear array of words*.
All processes (theoretically) can access all words.
*=Or what ever the appropriate addressing unit is: byte, word, long word

4. Memory Layout (VxWorks on PPC)
Interrupt Vector Table
Exception/Interrupt vectors
Shared Memory Anchor (if necessary)
Boot Line
ASCII string of boot parameters
Exception Message
ASCII string of the(prior) fatal exception message

5. Memory Layout, II Initial Stack System Image
Initial stack for usrInit( ), until usrRoot( ) allocates the real stack.
System Image
VxWorks itself (three sections: text, data, bss). The entry point for VxWorks is at the start of this region, which is BSP dependent. The entry point for each BSP is as follows:

6. Memory Layout, III Host Memory Pool Interrupt Stack
Memory allocated by host tools. The size depends on the the macro WDB_POOL_SIZE. Modify WDB_POOL_SIZE under INCLUDE_WDB.
Applications downloaded to processor are allocated space here.
Interrupt Stack
Size is defined by ISR_STACK_SIZE

7. Memory Layout, IV System Memory Pool
Size and location depend on the size of the system image. Malloc() allocates space from here.
Many of these items can be changed by modifying various macros (ISR_STACK_CHANGE, WDB_POOL_SIZE are just two) and then recompiling VxWorks.

8. Memory Access (Translations)
Untranslated: physical address is used unchanged (physical == virtual)
The processors usually boot into this mode until MMU hardware is initialised.
BAT Registers
Set of four/eight registers which define (large) blocks of memory
Each logical address is converted by BAT registers
Responsibility of user to correctly set BAT registers

9. Memory Access, II Page mode
As you would expect: logical addresses are decoded through segment registers and TLB page tables to construct physical address
Be careful on size: mapping 1GB => 16MB of page table space!
User must correctly set-up page table entries for certain (non-standard) memory locations.
Usually a combination of BAT and Page is used, depending on size of memories to map (VME / PCI / IO, etc). Page mode is preferred for use with Virtual Memory and Caching.

10. Virtual Memory VxWorks does not require a Memory Management Unit (MMU)
Not all systems have a MMU
System performance is best when a MMU is used
vmBaseLib allows 1 global Virtual Memory mapping for the system
vmBaseLib allows user to set cacheable / uncacheable memory blocks

11. VxWorks Memory, ctd
Extensive Virtual Memory support available is separate product (VxMI, aka vmLib)
Allows private virtual memory contexts
User could set up each task with a separate VM context
But: increased context switch time, user must manage contexts correctly
Not normally used in my company’s embedded products.

12. VxWorks and Caching
VxWorks is designed to the worst case scenario (greatest number of coherency issues):
Harvard architecture (separate Instruction and Data caches)
copy-back mode
DMA transfers and devices
Multiple bus masters
No Hardware support

13. More on Caching
VxWorks supplies the functions to control cache settings (write-through/ copy-back)
It is the responsibility of the user to handle the intricacies of cache / dma / virtual memory.
User must indicate whether memory buffers are cacheable / noncacheable
User must handle cache invalidation / cache flushing to maintain cache coherency
User must correctly set up DMA transfers with the use of the MMU and cache
Most of the time, the default settings are fine. Only when accessing other devices does this become some thing that must be fully analyzed.
VxWorks / BSP supply the functions, the user supplies how/when to use them.

14. How does VxWorks allocate memory?
VxWorks starts at address 0:
includes exception vector, etc.
During kernel initialisation, kernelLib will use memLib to allocate the system memory partition.
Malloc() will use the system memory partition for kernel memory needs.
User can create memory partitions by calling memPartCreate() in memPartLib.
This allows user to maintain different pools of memory, each of a different size, and allocate from those pools.
Generally these allocates are pulled from the system memory partition.
User can add more memory to the partition, and it need not be contiguous.
With appropriate number and appropriately sized partitions (a priori analysis) the effects of memory fragmentation can be reduced.

15. One problem with VxWorks
PPC’s addresses are 24 bits-- all code (VxWorks & application) must be within 32 MB.
There are several methods of dealing with this problem:
Add memory to global memory pool _AFTER_ all application code is loaded;
Compile code w/ -mlongcall option (reduced efficiency in code due to extra instructions to handle long calls)
User manages extra memory (not visible to VxWorks malloc())

16. Specifics of VxWorks
vmLib.h: handles the physical to virtual memory mapping
Data structure: sysPhysMemDesc
Partly filled in by VxWorks at system configuration (boot time)
Filled in by user prior to VxWorks compilation to add other memory (memory mapped devices)

17. Example sysPhysMemDesc[] (Synergy Dual processor board)
This is essentially the first page table in the system (it contains the first set of Page Table Entries)
It is an array of records:
Virtual Address, Physical Address, Length in Bytes; Virtual Memory Mask, Virtual Memory Enable
Note: This is a dual processor board, with all of RAM accessible to both processors. Different board configurations will yield different physical memory layouts.

18. sysPhysMemDesc[], ctd
The following are generally completed by the BSP vendor:
Element 0: Maps mailboxes, starts at address 0, and should be uncached.*/
Element 1: Maps gemini registers and ethernet data arrays. Starts at 0xF000000
Element 2: Define the space for VxWorks kernel (processor 1). Modified by kernel at run time.
Element 3: Define the space for VxWorks kernel (processor 2). Modified at run time.
Element 4: Shared Memory. Starts at 0x
Element 5: Processor 1’s RAM. Modified at run time.
Element 6: Processor 2’s RAM. Modified at run time.
Element 7: Page tables space. Modified at run time.

19. sysPhysMemDesc[], ctd
These elements are the responsibility of the user to modify prior to compiling:
PCI SPACE -- 8 entries
initially zero (so if they are not used, no memory is allocated)
my current project has a mapping here to a device on the PCI bus
VME SPACE -- last 8 entries
Describe the VME master ports for the architecture
Initially zero (like PCI maps)
We will add a mapping to a VME device here as well
The base system is untranslated, physical = virtual

20. What about the BAT registers?
BAT registers can conflict with the routines in cacheLib and vmBaseLib
ie, OS and HW may not play well together
BAT registers map memory outside of the processors (Flash, PROM, etc)
Locations where fine-grained control is not necessary

21. Conclusion
VxWorks offers much power to the user (application / system programmer) to control exactly how memory is addressed, allocated, cached.
With power comes responsibility:
user must not rely on VxWorks to perfectly implement an application’s memory needs.
User will need to consider caching (flushing, disabling), DMA accesses, etc. as part of design.

22. Definitions
Board Support Package (BSP): The extensions to the generic VxWorks that apply to a specific processor and its board.
Copy-back: Only write to memory when necessary
Direct Memory Access (DMA): Transfering data withou processor support.
Write-through: Write to cache and then into memory

23. References VxWorks Reference Manual Synergy User Guide for Dual-PPC
VxWorks on-line documentation
VxWorks source code (all the libs)