CRKIT R5 Architecture rev 0.1 WINLAB – Rutgers University April 25, 2013 Khanh Le

R5 Architecture Summary Objective : migration of R4 to Zynq platform + support of new RF module (400MHz-4GHz tuning range) Zynq : Integrated dual-core ARM processor + peripherals e.g. GigE, USB, i2c, SPI, UART… What are the Hardware implications ? 1.Access to GigE port through AMBA AXI bus -> new Ethernet port module (ethernet framing by PS) 2.New RF ports e.g. DA/AD interface modules 3.New PCORE wrapper containing ARM processor and dual AXI busses 4.New AXI ipif modules for AXI bus interfacing : one for RMAP Processor, one for Ethernet port 5.Replace SPI with I2C interface. Current SPI is no longer needed (remove). I2C is provided by PS. 6.Potentially replace all R4 FIFOs and Block RAMs 7.New clock architecture What are the Software implications ? 1.Add DMA support for PS GigE -> PL Ethernet Port 2.Add DMA support for PL Ethernet Port -> PS GigE 3.Some changes to overall memory map due to dual AXI bus architecture 4.Changes to RF control due to I2C interface and new radio (refer to ADI reference design) APP design remains the same, only Xilinx IPs may need to be replaced due to tool revision updates Overall software remain the same, no Linux support for the initial phase. Once HW is stable, port Linux including VITA support. Target design methodology : mixed-language, C/SystemC, UVM Goal for DARPA Challenge : functional spectrum sensing APP (mid-august time frame)

Implications to R4 Architecture Modify access thru AXI bus Serial IO + ref. clock modifications SPI no longer applicable. Replace with I2C interface Replace with ARM core, contains dual AMBA AXI busses : AXI0 -> GbE, AXI1 -> Framework Replace PLB IPIF with AXI IPIF

Zynq System-on-Chip Overview Programmable Peripherals : UART, I2C, GigE CPU Cores AXI Master/Slave Ports Programmable Logic (PL) Processor subsystem (PS) Refer to Xilinx user guide ug585-Zynq-7000-TRM.pdf for further details Fixed external IO pins

CRKIT Framework Integration PCORE AXI1 GbE RMAP Processor AXI0 0x4000_0000 0x7FFF_FFFF 0x8000_0000 0xBFFF_FFFF CRKIT AXI IPIF APP + PKT RF AXI0 – 32-bit, Master Port AXI1 – 32-bit, Master Port

PS IPIF data paths PS M PL SM SM system_processing_system7_0_wrapper (processor) system_axi_interconnect_0_wrapper (axi bus 0) system_axi_ext_slave_conn_0_wrapper (external connector 0) PL SM axi_ipif_slv (gbe) BUS2IP IP2BUS PCORE Bring internal AXI signals to top level M PL SM SM system_axi_interconnect_1_wrapper (axi bus 1) system_axi_ext_slave_conn_1_wrapper (external connector 1) PL SM axi_ipif_slv ( rmap processor) BUS2IP IP2BUS 0x4000_0000 0x7FFF_FFFF 0x8000_0000 0xBFFF_FFFF M_AXI_GP0 M_AXI_GP1

R4 - host2app PCORE CMD FORMAT If (V==1) then VITA context packet Else non-VITA packet use ethertype field for further parsing Endif; Forward ethernet payload if :  incoming MAC = dMAC  incoming MAC = Broadcast Append Ethertype field (16-bit) to ethernet payload if (ethertype == IPv4 & Incoming IP == dIP & UDP = 1000) then forward UDP payload to VITA Receiver else forward packet to PCORE Ethertype = 0x IPv4 0x ARP Use CMD_CNT as ACK to MEM_CTL to indicate completion of PCORE data removal from MEM. REPLACE

GbE GbE -> IP Data Path (Option 1) DP RAM DMA AXI SLAVE IPIF PKT CTL CMD FIFO Elastic Buffer AXI BUS 0 GbE Rx (R5) Skip ETH header, Prepend Ethertype field to payload Packet Processor (R4) Complete ETH packet storage (circular buffer) Ethertype field + IP packet ptr[15:0]size[13:0]- Processor Subsystem CRKIT Framework 3. Generate CMD ARM Cortex A9 2. DMA done 1. DMA xfer Fixed length buffer size allocation e.g. min bytes -> 400 words (may depend of DP RAM size -> end on a full buffer boundary) -MAC filtering -Jumbo frames not supported

GbE GbE -> IP Data Path (Option 2, no processor assistance) DP RAM DMA AXI SLAVE IPIF PKT CTL CMD FIFO Elastic Buffer AXI BUS 0 GbE Rx (R5) Skip ETH header, Prepend Ethertype field to payload Packet Processor (R4) Complete ETH packet storage (circular buffer) Ethertype field + IP packet ptr[15:0]size[13:0]- Processor Subsystem CRKIT Framework ARM Cortex A9 2. DMA done 1. DMA xfer Fixed length buffer size allocation e.g. min bytes -> 400 words (may depend of DP RAM size -> end on a full buffer boundary) -MAC filtering -Jumbo frames not supported CMD GEN Snoop on bus Interface to generate CMDs (write only).

GbE -> IP Data & Control Flows On-chip Memory 256KB GbE DMA ARM Cortex A9 BRAM 1. Init Rx Buffer Descriptors (RBD) (refer to ug585, pg. 418) 2. +Init DMA controller +init ptr to RBD entries +enable GbE receiver (refer to programming guide, ug585, pg. 436) GbE OCM BRAM 8-byte descriptor1600-byte buffer 3. Fetch descriptor (ptr to BRAM) 4. DMA xfer 5. Update xfer control stat 6. DMA done intr 16 RBD entries e.g. 32 words 7. +Search for used descriptor e.g. ownership flag == 1 +clear ownership flag (allows descriptor to be reused by GbE DMA ctl.) Assuming available rx packet Enable Receiver : gem.net_ctrl[rx_en] == 1 Refer to ug585, Programming guide, page 436 for additional details.

Rx Buffer Descriptor Entry Pointer to Rx DPRAM buffer Ownership flag Reference : ug585, page 418

R4 - app2host VRT Receiver Lookup using PortID dMAC/Ethertype from IP Processor if (IP == 1) then Enable IP processing (append dIP, sIP & UDP) Forward dMAC/Ethertype (Note, sMAC provided in RMAP) else Disable IP Processing Forward dMAC/Ethertype (Note, sMAC provided in RMAP) endif Lookup using PortID if (V == 1) then Enable VITA formatting else Disable VITA formatting endif dMAC/Ethertype appended to IP/VITA data REPLACE

GbE IP -> GbE Data Path DP RAM DMA AXI SLAVE IPIF ETH HDR GEN CMD FIFO Elastic Buffer AXI BUS 0 GbE Tx (R5)Packet Processor (R4) 2. Fetch CMD Fixed length buffer allocation dMAC + Ethertype + IP packet +Jumbo frames not supported + append FCS field (enabled at Tx Buffer Descriptor) Processor Subsystem CRKIT Framework ARM Cortex A9 CMD GEN INT REG 1. Intr 4. Setup DMA 5. DMA xfer ptr[15:0]size[13:0]- Fixed length buffer size allocation e.g. min bytes -> 400 words (may depend of DP RAM size -> end on a full buffer boundary) Generate complete Ethernet header e.g. dMAC, sMAC, ethertype/size. FCS added by PS GbE MAC sMAC CTL REG

IP -> GbE Data & Control Flows On-chip Memory 256KB GbE DMA ARM Cortex A9 CRKIT GbE Tx 3. +setup Tx Buffer Descriptors (TBD) +clear used flag (refer to ug585, pg. 423) 4. Set Transmit_Start bit e.g. bit-9 of Network Control register) (ug585, pg. 436) 5. Fetch descriptor (ptr to ETH packet) 6. DMA xfer 7. Update xfer control stat (set used flag to ‘1’) 1 TBD entry e.g. 2 words Assuming available tx packet 1. intr2. Fetch CMD size + ptr Init. + enable Tx DMA engine ( gem.net_ctrl[tx_en] == 1) Refer to ug585, Programming guide, page 436 for additional details. 8. DMA done intr (optional) 9. clear all status bits, including used flag (optional)

Tx Buffer Descriptor Entry Pointer to Tx Buffer used flag Reference : ug585, page 423 wrap flag Set to 1 for single buffer

RF Interface dac_data_out[15:0] dac_clk_out dac_frame_out (unused) Not used for word-level, only for Byte- or Nibble-level AD9122 DAC AD9548 Clock Sync AD9523 Clock Gen AD9643 ADC dac_clk_in ref_clk_out(~30MHz) Jitter clean up adc_data_in[13:0] adc_clk_in adc_or_in I Q 1 0 LVDS DCM ODDR clock feedback DAC Interface RF Reference Clock ADC Interface I2C Interface

DA Port

AD Port

Clock architecture

Memory Map

R4 - PCORE PCORE – RMAP RD/WR IP Host -> PCORE (UDP-1001) IP PCORE -> Host (UDP-1001) Address Decoding RTYPE: 0x2000 – RMAP READ 0x2001 – RMAP WRITE RADDR: Register address RDATA: Register data

R4 - APP

R4 – Memory Map Upper 4 MSBs : 0x0-0x1 : PCORE 0x2 : CRKIT Others : Unused 0x0 : CMN 0x1 : ETH 0x2 : PKT 0x4-0xB : APP 0xC : DAC IF 0xD : ADC IF INT SPI, LED DCM/CLOCK CE

R4 – Interrupt Architecture INTCuPIPIF CMN INT ETH INT PKT INT APP INT SYS INT PCORECR