 Wind River Systems, Inc Appendix - E Shared Memory Network

E-2 Shared Memory Network Overview

E-3 Shared Memory Network Implementation Uses global (“shared”) memory accessible to all CPU’s on the backplane. Uses test-and-set atomic operation to mutually exclude access to global data structures. Destination CPU notified of packet arrival via: Mailbox interrupts (most flexible). VMEbus interrupts (limited number). Polling (if interrupts not available).

E-4 Terminology Shared Memory Master - CPU 0. Initialize the shared memory network. Once initialized, all boards are peers. Shared Memory - Memory used for passing packets between processors. Can reside on CPU0 or a memory card. Anchor - Structure containing ready value and pointer to shared memory. Must be at a known address. Heartbeat - Integer incremented once per second by CPU0, used to indicate that the shared memory network is alive. Ready Value - Indicates if the shared memory network is initialized.

E-5 Choosing a Network Number Shared memory network number options: Distinct networkIn an isolated environment, choose any unused network number. SubnetworkIf the host is already on a subnet, choose an unused subnet. Proxy networkUse the same network number as the host. Easiest to configure. Examples: RelationshipHost networkSM network Distinct Subnet :ffff :ffff0000 Proxy :ffff :ffff0000

E-6 Sequential Addressing The internet address of CPU0 on the SM network: If INCLUDE_PROXY_DEFAULT_ADDR is defined, it is one more than inet on ethernet (e). Otherwise, assigned in the boot parameters. The internet address of other boards on the SM network: If INCLUDE_SM_SEQ_ADDR is defined, will be sequentially ordered (by processor number). Otherwise, assigned in the boot parameters.

E-7 Example Configuration Overview 1. Configure the host’s network softwre: Modify the host table. On UNIX, modify.rhosts. (UNIX Host) 2. ConfigAll.h specifies as defaults: shared memory network support sequential addressing proxy client support 3. Modify config.h for CPU0 to specify: proxy server support proxy default addressing. where shared memory resides, how large it is. 4. Calculate the address of the anchor.

E-8 Example Configuration

E-9 UNIX: Host Configuration Configure host so that targets can boot: Host table Routing (if necessary -- when proxy ARP not used) Permissions Example/etc/hosts file: vxEth vxSm vxSm1 Example ~/.rhosts file (if using RSH): vxEth0 vxSm0 vxSm1

E-10 Installing the Shared Memory Network These defaults are specified in configAll.h: #define INCLUDE_PROXY_CLEINT... #define INCLUDE_SM_NET #define INCLUDE_SM_SEQ_ADDR... #define SM_TAS_TYPE SM_TAS_HARD #define SM_ANCHOR_ADDR... /* arch. specific */ Modify CPU0’s config.h file to specify: #define INCLUDE_PROXY_SERVER #define INCLUDE_PROXY_DEFAULT_ADDR #define SM_OFF_BOARD FALSE #define SM_MEM_ADRS... #define SM_MEM_SIZE...

E-11 Shared Memory Location / Size Example wind/target/config/target/config.h stub: #define SM_OFF_BOARD FALSE #if SM_OFF_BOARD #undef SM_ANCHOR_ADRS #define SM_ANCHOR_ADRS ((char *) 0xfb800000) #define SM_MEM_ADRS SM_ANCHOR_ADRS #define SM_MEM_SIZE 0X80000 /* 512k */ #else #define SM_MEM_ADRS NONE /* NONE = ALLOCATE */ #define SM_MEM_SIZE 0X10000 /* 64k */ #endif /* SM_OFF_BOARD */

E-12 CPU0 Boot Parameters boot device : in processor number : 0 host name : ohio file name : wind/target/config/frc40/vxWorks inet on ethernet (e) : inet on backplane (b): host inet (h) : user (u) : marc

E-13 CPU0 Booting Example... Attaching network interface ln0... done. Attaching network interface lo0... done. Loading Starting at 0x Attaching network interface ln0... done. Initializing backplane net with anchor at 0x done. Backplane anchor at 0x Attaching network interface sm0... done. Backplane address: Creating proxy network:

E-14 Booting Other CPU’s  Other boards must be able to access the anchor, through the VMEbus, in order to boot.  CPU’s local anchor address (0x600) is not identical to its corresponding VMEbus address.  To attach to the Shared Memory Network, other CPU’s must: 1. Determine the VMEbus address of the anchor (how CPU0 dual-ports its local RAM). 2. Determine the local address, on CPU1, which is used to access the anchor’s VMEbus address (how CPU1 maps its address space).

E-15 VMEbus Address Spaces The VMEbus provides several address spaces to accommodate varying needs. Address space is specified by an address modifier, which encodes: Address size32 bit (extended), 24 bit(standard), or 16 bit (short I/O). Privilege level User or supervisor. VxWorks always uses supervisor mode. Access typeData, instruction, or block transfer (DMA). Symbolic constants for address modifiers defined in vme.h (e.g., VME_AM_EXT_SUP_DATA).

E-16 Off Board Memory Access

E-17 Calculating the VMEbus Address of the Anchor from CPU0 -> busAdrs = 0 new symbol “busAdrs” added to symbol table. value = 0 = 0x0 -> sysLocalToBusAdrs (0xd, 0x600, &busAdrs) value = 0 = 0x0 -> busAdrs busAdrs = 0xffe7d0: value = = 0x value = 0 = 0x0

E-18 Local Address of Anchor As Seen from Other Boards -> localAdrs = 0 new symbol “localAdrs” added to symbol table. localAdrs = 0x3fe7d0: value = 0 = 0x0 -> sysBusToLocalAdrs (0xd, 0x , &localAdrs) value = 0 = 0x0 -> localAdrs localAdrs = 0x3fe7d0: value = = 0x Can’t perform these steps from CPU1, since not yet booted. Can perform from CPU0 if boards are the same. Can examine sysLib source code otherwise.

E-19 Calculating the Anchor 1. Call sysLocalToBusAdrs( ) on CPU0 to calculate the VMEbus address of the anchor. 2. Call sysBusToLocalAdrs( ) on the other CPU boards to calculate the local address which maps to the anchor’s VMEbus address. May need to examine source code for this routine.

E-20 CPU1 Boot Parameters boot device : sm=0x processor number: 1 host name: ohio file name : wind/target/config/frc40/vxWorks inet on backplane (b): host inet (h) : gateway inet (g) : user (u): marc flags (f) : 0x100

E-21 Booting CPU1... Backplane anchor at 0x Attaching network interface sm0... done. Backplane inet address: registering proxy client: done Attaching network interface lo0... done. Loading Starting at 0x

E-22 Other Configurations If not using a proxy network: Must modify the host’s routing table. Slaves must specify gateway inet (g) boot parameter. Can use BOOTP with the proxy network. See the Programmer’s Guide for details.

E-23 Avoiding Bus Errors Default bus arbitration favors boards close to first slot. Distant boards may experience bus (timeout) errors while booting. Possible solutions: Stagger the boot countdown for each board by modifying wind/target/config/all/bootConfig.c. Increase bus timeout value. Modify the bus arbitration scheme (software changes are typically made in sysProcNumSet( ) and sysHwInit( )). Software solutions require making new boot ROM’s which is discussed in the Reconfiguration chapter.

E-24 A Troubleshooting Tool -> smNetShow Anchor Local Addr: 0x Sequential addressing enabled. Master address: heartbeat = 6712, header at 0x1fb8144, free pkts = 29 cpuint typearg1 arg2 arg3queued pkts mbox-1r0x2d0x mbox-1r0x2d0x input packets = 109 output packets = 109 input errors = 0 output errors 0 collisions = 0 value = 0 = 0x0

E-25 Summary Host configuration: Modify host tables. Modify ~/.rhosts. (UNIX Host) Modify routing table if not using a proxy network. Defaults in configAll.h: INCLUDE_SM_NET and INCLUDE_SM_SEQ_ADDR SM_ANCHOR_ADRS INCLUDE_PROXY_CLIENT CPU0 (the shared memory master) specifies SM_OFF_BOARD SM_MEM_SIZE and SM_MEM_ADRS INCLUDE_PROXY_SERVER INCLUDE_PROXY_DEFAULT_ADDR

E-26 Summary Additional boot parameter for CPU0: inet on backplane (b): If using default proxy address, leave blank. Accessing the anchor from other boards: sysLocalToBusAdrs( ) sysBusToLocalAdrs( ) Additional boot parameters for CPU1: boot device: sm=localAdrsOfAnchor inet on backplane (b). If using sequential addresses, leave blank. gateway inet(g). If using proxy ARP, leave blank. register proxy client boot flag (0x100)