1. UniBoard Meeting, October 12-13th 2010 Jonathan Hargreaves, JIVE EVN Correlator Design UniBoard Meeting, October 12-13 th 2010 Contract no. 227290

2. UniBoard Meeting, October 12-13th 2010 Basic Configuration FN 0 Station 0-7 2 Pol, 8 Bit, 64 MHz BW FN 1 Station 8-15 2 Pol, 8 Bit, 64 MHz BW FN 2 Station 16-23 2 Pol, 8 Bit, 64 MHz BW FN 3 Station 24-31 2 Pol, 8 Bit, 64 MHz BW BN 0 2112 corr products 16 MHz BW BN 1 2112 corr products 16 MHz BW BN 2 2112 corr products 16 MHz BW BN 3 2112 corr products 16 MHz BW Station Data in to 10GbE Ports Correlation Products out to backend computer Works with 800MT/s DDR3

3. UniBoard Meeting, October 12-13th 2010 Expanded Configuration FN 0 Station 0-7 2 Pol, 8 Bit, 128 MHz BW FN 1 Station 8-15 2 Pol, 8 Bit, 128 MHz BW FN 2 Station 16-23 2 Pol, 8 Bit, 128 MHz BW FN 3 Station 24-31 2 Pol, 8 Bit, 128 MHz BW BN 0 2112 corr products 32 MHz BW BN 1 2112 corr products 32 MHz BW BN 2 2112 corr products 32 MHz BW BN 3 2112 corr products 32 MHz BW Station Data in to 10GbE Ports Correlation Products out to backend computer Needs 4GB DDR for FN

4. UniBoard Meeting, October 12-13th 2010 Spectral Line FN 0 Station 0-7 2 Pol, 8 Bit, 64 MHz BW FN 1 Station 8-15 2 Pol, 8 Bit, 64 MHz BW FN 2 Station 16-23 2 Pol, 8 Bit, 64 MHz BW FN 3 Station 24-31 2 Pol, 8 Bit, 64 MHz BW BN 0 2112 corr products 16 MHz BW BN 1 2112 corr products 16 MHz BW BN 2 2112 corr products 16 MHz BW BN 3 2112 corr products 16 MHz BW Station Data in to 10GbE Ports Correlation Products out to backend computer Subset of frequency points from each FN routed via spare 10GbE ports to another UniBoard

5. UniBoard Meeting, October 12-13th 2010 Proof of Concept First show we can get data flowing through by doing autocorrelations: 4 x 16MHz bands per station 2 polarizations in separate frames 2 bits per sample Use validity bits to tell the correlator which data to process Simplifications Delay model not needed for autocorrelations Only need one FN Can read out correlation products through 1Gb ports on BNs

6. UniBoard Meeting, October 12-13th 2010 FN - VDIF Frame Buffering

7. UniBoard Meeting, October 12-13th 2010 FN - VDIF Frame Buffering Assume 4 stations per port. If they are not used data gets marked invalid in the buffer VDIF header is checked for frame length, station, and band. If any are outside the range set by the controller, or the frame is marked invalid, the frame is ignored FIFOs can store two 8192 byte frames. A frame must be stored in the buffer before another one arrives Use validity bits to tell the correlator which data to process. One validity bit per frame which is set when good data arrives and reset when the data is readout Readout starts on an FFT period tick The scheduler tops up the read side FIFOs to maintain a continuous flow of data for the complete FFT period. Data can be read in an order which minimises overhead by jumping between DDR3 banks

8. UniBoard Meeting, October 12-13th 2010 FN – Polyphase Filter Bank PFB Sin/Cos Gen 0 X8 Phase Rotator PFB DM Control Sync and Validity FB Control Sync and validity Re-Quantize to 9 bits RQ Control Re-Quantize to 9 bits Validity Phase Rotator DS0 DS1 DS14 DS15 PFB Sync Validit y 0-15 Sin/Cos Gen 7 Phase Counter Sync Valid 0-15 X8

9. UniBoard Meeting, October 12-13th 2010 FN – Polyphase Filter Bank Each Filter bank produces 4096 frequency bins Can clock at over 300MHz but choose 266.5 as this is the DDR3 local bus speed This is over 4 times real time for a 64MHz band (complex) but...... probably need to time share them due to memory constraints Still 128MHz per FN is realistic We are comparing 4 taps vs. 6 taps per frequency bin We can easily try different window functions... Kaiser vs. Blackman Harris etc. FFT should not re-order data – this can be done in the corner turner

10. UniBoard Meeting, October 12-13th 2010 FN to BN mesh Val, sync Framer for frequency bins 0-1023 Mesh Re-orderMesh Re-order To BN0 Data x16 Val, sync Data x16 Val, sync Data x16 Val, sync Data x16 Framer for frequency bins 1024-2047 Framer for frequency bins 2048-3071 Framer for frequency bins 3072-4095 ALTGX 6.25Gbps serialize 8b/10b encoder ALTGX 6.25Gbps serialize 8b/10b encoder ALTGX 6.25Gbps serialize 8b/10b encoder ALTGX 6.25Gbps serialize 8b/10b encoder To BN1 To BN2 To BN3 FN0 ALTGX 6.25Gbps deserialize 8b/10b decoder From FN0 Alignment Logic Mesh Re-order Mesh Re-order FIFO FIFO BN0 ALTGX 6.25Gbps deserialize 8b/10b decoder ALTGX 6.25Gbps deserialize 8b/10b decoder ALTGX 6.25Gbps deserialize 8b/10b decoder From FN1 From FN2 From FN3 FIFO FIFO FIFO FIFO FIFO FIFO

11. UniBoard Meeting, October 12-13th 2010 BN Overview Frame no. within integration period DDR3 Module I (1GB) Scheduler Deframer FIFOFIFO DDR3 Controller IP DDR3 Module II (1GB) Data, validity DDR3 Controller IP Deframer FIFOFIFO Corner Turner Station code Start of integration Scheduler Correlator 132 Complex MACs Correlator 132 Complex MACs Data Buffer A Data Buffer B Validity Buffer A Validity Buffer B Validity bit accumulat or Copy output to Avalon MM registers

12. UniBoard Meeting, October 12-13th 2010 BN Correlation Engine 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 0 1 2 3 4 5 6 7 24 25 26 27 28 29 30 31 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 22 21 20 19 18 17 15 14 13 12 11 10 9 8 17 18 19 20 21 22 21 20 19 18 17 1623 24 25 26 27 28 29 30 31 31 30 29 28 27 26 25 24 a) b) c) 132 MAC cells clocked > 260MHz One half-DSP block plus 72 registers per MAC cell for 36 bit accumulation Sixteen passes: L-L, R-R, L-R, R-L

13. UniBoard Meeting, October 12-13th 2010 Development Path Delay models to be calculated in the FNs using coefficients downloaded from the control computer The four FNs need to be synchronised Measure average power in each frequency bin Increase BW throughput in the FN by increasing the duty cycle of the filter banks Increase BW correlated in the BN by inplementing two correlation engines Implement 10Gbps ports on the BN to export correlation products