1. LaRC p174/ MAPLD 2004Jones Slide 1 Experiences in the Development of an FPGA Based Radiation Tolerant Design Mark Jones Mark.l.jones@nasa.gov 757-864-7878 Dr. Robert Klenke rhklenke@vcu.EDU (804) 827-7007

2. LaRC p174/ MAPLD 2004Jones Slide 2 Gifts Geosynchronous I maging Fourier Transform Spectrometer Was not funded to completion

3. LaRC p174/ MAPLD 2004Jones Slide 3 Gifts Modules Sensor Module Control Module Modulators (Downlink)

4. LaRC p174/ MAPLD 2004Jones Slide 4 Gifts Control Module 6U CPCI 33MHz, 32 Bit, 3.3 Volt 6U CPCI 33MHz, 32 Bit, 3.3 Volt IC (Instrument Controller) BAE 750 IO DL EDS MEM 2 MB SRAM

5. LaRC p174/ MAPLD 2004Jones Slide 5 Gifts Control Module Data Flow MEM DLINK IC IO SM Data I/F Serialized LVDS 21-bit data,16 MHz SM Command I/F 422 Differential Spacecraft I/F 1553 Requires Sequential “Block” Readout to Downlink X-Band 80 Mbps SMQ-11 Actel PCI Core EDS Actel PCI Core CPCI Bus

6. LaRC p174/ MAPLD 2004Jones Slide 6 MEM overall function and purpose CM must accept SM LVDS data up to 256Mbits/sec, average 160Mbits/sec and transfer that data to the communications payload at a rate of 80Mbits/sec This throughput requires the data must be compressed (DWNLNK) resulting in a non-continuous and variable throughput –Compression ratio achieved by the DWNLNK varies with the entropy of the current image. –Worst case is unable to match the bandwidth of the incoming data.

7. LaRC p174/ MAPLD 2004Jones Slide 7 MEM overall function and purpose, continued This mismatch of data rates between the SM and the DWNLNK output during periods of low image compression requires that the MEM be able to store incoming data from the LVDS interface until the DWNLNK is ready to process and send it System engineering studies, using typical values of data entropy across an image, indicate that the MEM must contain at least 1Mbytes of data storage space to avoid overflow and the resulting loss of image data, therefore to provide a 2X margin, a 2Mbyte buffer is used The MEM 2Mbytes of SRAM memory are organized as a First In First Out (FIFO) buffer.

8. LaRC p174/ MAPLD 2004Jones Slide 8 Memory Board Design Flow and Tools Actel Libro Platinum Toolset –VHDL design entry using text editor –Pre-synthesis Functional Simulation using ModelSim and highly modified Actel-supplied PCI test bench –Synthesis using Synplicity synthesis tool –Post-synthesis simulation using ModelSim Full licensed copy of ModelSim (as opposed to the reduced performance version supplied with Libro) necessary to complete post- synthesis and post-place & route simulations in reasonable time –Place & route with Actel’s Designer tools Timer static timing analysis tool used to check rough timing and apply timing constraints –Post-place & route timing simulation with ModelSim

9. LaRC p174/ MAPLD 2004Jones Slide 9 Functional Block Diagram – Clock Domains Actel PCI Core DMA FIFO LVDS FIFO cPCI Bus DMA Engine SRAM Controller BAE 238A792 SRAM (2) Actel RT54SX32S FPGA Actel RT54SX32S FPGA (2 – bit sliced) Actel RT54SX32S FPGA Deseri alizer LVDS Data LVDS Clock Domain Core Clock Domain (Delayed cPCI Clock) cPCI Clock Domain BAE 238A792 SRAM (2) 16 32 64

10. LaRC p174/ MAPLD 2004Jones Slide 10 GIFTS Memory Board Architecture –100 ohm Differential LVDS Impedance requirement –CPCI protocol includes: Bus terminating resistors within.5 inches of cPCI connector 65 ohm board impedance for CPCI signals Specific trace lengths which will affect FPGA placement relative to CPCI connector –The data received at the LVDS data interface is received in a serial bit stream. The serialization however is transparent to the MEM as the output of the Deserializer is parallel. –There are 3 clock domains, the incoming LVDS, the PCI clock domain, and because the CPCI spec only allows 1 load on the CPCI clock and a rad hard(low SEU) low skew buffer or PLL was not found, the clock for the DMA Engine and SRAM controllers use a 3 rd clock domain delayed from the PCI clock. –For a design independent of the amount of clock skew and not rely on place and route for proper function, this resulted in Chip to chip asynchronous data transfer using control signals that employ 4 state handshaking –The “initiator” asserts the “Ready” signal –The “receiver” asserts the “Acknowledge” signal –The “initiator” de-asserts the “Ready” signal when done with transfer –The “receiver” de-asserts the “Acknowledge” signal indicating it is done with transfer and ready for next

11. LaRC p174/ MAPLD 2004Jones Slide 11 GIFTS Memory Board Architecture –The cPCI bus has the capability to transmit one 32-bit double word (Dword) each clock cycle of the 33MHz cPCI clock. This capability of course, requires that the PCI core be supplied with a 32-bit Dword on its input queue each 30.3 ns clock cycle in order to maintain 100% bandwidth on the cPCI bus. However, the SRAM devices that were available for use in the MEM design originally had a fastest access time of 35 ns. This access time restriction required that, in order to achieve improved bus bandwidth, two Dwords must be retrieved from the SRAM and transmitted to the PCI core during a single memory read operation. Furthermore, because data will be arriving across the LVDS interface during the DMA operation, it is necessary to be able to perform both a read operation, to retrieve data for the current DMA operation, and a write operation, to store the just received LVDS data, on the SRAM concurrently. These two considerations required that the SRAM in the MEM be configured as two 64-bit banks that are 128Kbytes deep, thus allowing simultaneous read and write operations to be performed on opposite banks in a ping-pong fashion. The MEM design requires that all DMA data transfers must be performed on a 64-bit boundary (i.e., an even number of 32-bit Dwords).

12. LaRC p174/ MAPLD 2004Jones Slide 12 Functional Block Diagram – Data Flow SRAM Controller PCI Core LVDS Fifo Left SRAM Right SRAM 64 LVDS Data cPCI Bus 128K X 64

13. LaRC p174/ MAPLD 2004Jones Slide 13 Functional Block Diagram – Data Flow SRAM Controller PCI Core LVDS Fifo Left SRAM Right SRAM 64 LVDS Data cPCI Bus 128K X 64

14. LaRC p174/ MAPLD 2004Jones Slide 14 Functional Block Diagram – Data Flow SRAM Controller PCI Core LVDS Fifo Left SRAM Right SRAM 64 LVDS Data cPCI Bus 128K X 64

15. LaRC p174/ MAPLD 2004Jones Slide 15 SRAM Controller Functional Block Diagram – Data Flow 64 PCI Core LVDS Fifo 64 LVDS Data cPCI Bus Left SRAM 128K X 64 Right SRAM 128K X 64 LVDS Fifo w0 w1 w2 w3 w4 w5 w6 w7 … w7, w6, w5, w4, w3, w2, w1, w0 Right SRAM 128K X 64 w0 w1 w2 w3 Left SRAM 128K X 64 w4 w5 w6 w7 PCI Core [w3 w2], [w1 w0] [w7 w6], [w5 w4]; [w3 w2], [w1 w0] w8 w9 w10 w11

16. LaRC p174/ MAPLD 2004Jones Slide 16 GIFTS Memory Board Architecture –Since Actel PCI core IP was designed for a standard memory module, all of the data that is to be DMAed must be available (otherwise the Core is starved for data). This is not the case with this design since at any given time the IC does not know exactly how much data is available for DMA transfer. Therefore, it was necessary to design a “DMA engine” into the MEM that can keep track of how much data has been received by the MEM and stored in the SRAM, and configure and initiate the individual DMA operations in the cPCI core until the requested total amount of data has been transferred to the destination across the cPCI bus.

17. LaRC p174/ MAPLD 2004Jones Slide 17 GIFTS Memory Board Architecture Bit-slicing SRAM Controller Functionality –64 bit data paths as a result of memory speed and throughput –However cannot be implemented in one RT54SXS due to I/O limitations –Results in SRAM Controller Core bit-sliced across 32-bit boundaries

18. LaRC p174/ MAPLD 2004Jones Slide 18 Bit-Sliced FPGA Functional Block Diagram Left SRAM Controller PCI Core LVDS Fifo Left SRAM Right SRAM 64 Right SRAM Controller 32 128K X 64

19. LaRC p174/ MAPLD 2004Jones Slide 19 GIFTS Memory Board Architecture Requires 4 Actel RT54SX32S FPGAs –SRAM Controller Core bit-sliced into 2 FPGAs –Based on total cell utilizations for –PCI CORE FGPA was 69% –Left Sram Controller was 67% –Right Sram Controller was 53% –LVDS FIFO was 72% –PCI Core and LVDS FIFO each implemented on a separate FPGA resulting in 4 FPGAs

20. LaRC p174/ MAPLD 2004Jones Slide 20 Block Diagram Actel PCI Core DMA FIFO LVDS FIFO cPCI Bus BAE 238A792 SRAM (2) Actel RT54SX32S FPGA Actel RT54SX32S FPGA Deseri alizer LVDS Data LVDS Clock Domain Core Clock Domain (Delayed cPCI Clock) cPCI Clock Domain BAE 238A792 SRAM (2) 16 32 Left SRAM/ DMA Controller 64 Right SRAM Controller 32 Actel RT54SX32S

21. LaRC p174/ MAPLD 2004Jones Slide 21 MEM Design Status Initial prototype design completed –Limitations imposed by available FPGA size, speed, and I/O pins overcome –Limitations imposed by Actel-supplied PCI Core overcome –Limitations imposed by available SRAM speed overcome –Complexity of design resulting from 3-asynchronous clock domains overcome Initial brassboard completed and tested –Demonstrated fundamental design concept –Successfully completed reliable DMA of data from LVDS stimulator to PCI-resident memory –Uncovered several significant bugs and needed additional functionality –Successfully redesigned and simulated at the behavioral and post- synthesis level –Revised version of the design will work with existing PCB without redesign

22. LaRC p174/ MAPLD 2004Jones Slide 22 MEM Rev. 2 Design to-be-completed Complete timing layout and test in post layout simulation with back annotated timing Test on the brassboard Complete documentation of the design

23. LaRC p174/ MAPLD 2004Jones Slide 23 Backup

24. LaRC p174/ MAPLD 2004Jones Slide 24 Brassboard Layout Layout details –CPCI requirements met for terminating resistor placement and trace lengths to the PCI FPGA. –CPCI 65 ohm Impedance requirement met on all signal layers facilitated by stackup design and trace width and separation relationship –30mils separation between asynchronous CPCI signals and others –100 ohm Differential LVDS Impedance requirement met on all signal layers facilitated by stackup design and trace width and separation relationship, and custom thru-hole arrangement for Micro_Twinax cable connection –Differential LVDS signals surrounded by ground guard plane spaced 20mils (2 x distance between pair) away from pairs –Clocks are routed internal to the board, between planes, and have ground guard traces on both sides. –Edges of Board will be milled to allow 90mil board to slide into 62mil guides –Layout approximately 99% complete

25. LaRC p174/ MAPLD 2004Jones Slide 25 Brassboard Layout

26. LaRC p174/ MAPLD 2004Jones Slide 26 Layer 1,9 65 Ohms: 5 mil wide/10 mil sep. 100 Ohms Diff. 7 mil wide/10 mil sep. Layer 3, 6 65Ohms: 5 mil wide/10 mil sep. 100 Ohms Diff. 7 mil wide/12 mil sep. Brassboard Layout – Stackup