1. Aperture Array Station Processing
Andrew Faulkner
SKADS Project Engineer
September 2009
Station Processing – Cape Town

2. Station Processing – Cape Town
SKA Overall Structure
Beam Data
Mass Storage
GHz
Wide FoV
Tile &
Station
Processing
Central Processing Facility - CPF
Dense AA
To 250 AA Stations
...
Correlator – AA & Dish
16 Tb/s
Data
... ..
MHz
Wide FoV ..
Time
Control
Post Processor
Sparse AA
...
GHz
Single
Pixel or
Focal plane
array
DSP
Control Processors
& User interface
12-15m
Dishes
80 Gb/s
DSP
Time
Standard
...
To 2400 Dishes
User interface
via Internet
September 2009
Station Processing – Cape Town

3. Mid-Frequency Aperture Array
Tile
Support
Bunker
Freq range 0.3  1.2 GHz
Dense array (Nyquist sampled)
~75,000 Receiver chains
September 2009
Station Processing – Cape Town

4. EMBRACE - SKADS
August 2009
Station Processor

5. Station Processing – Cape Town
The AA Station
Next Proc. Bunker
n x Optical
fibres per 2nd
stage processor
>1016 MACs
Station
Processor 1 }
1st Stage Processors
Mid P1
Station
Processor 2
Mid P2
AA-hi
To Correlator .
Mid Py …
GHz Analog links
Internal
Digital links
Station
Processor X . .
Box
Low P1
O-E?
Phase
Standard &
Distribution ..
...
Phase transfer
over fibre . .
Low Pz
O-E?
Control
processors
500MHz Analog links
To Central
Control
AA-lo
10Gb Digital
fibre links
Prev. Proc. Bunker
September 2009
Station Processing – Cape Town

6. Station Processing – Cape Town
1-D Beamforming
Incoming signal
Geometric
Delay, t
Elements
Electronic Delay
Beam + + + + + + + + + + + + + + + + + + + + + + + + t
Delay
Element #
September 2009
Station Processing – Cape Town

7. 1-D Beamforming Incoming signal Geometric Delay, t
Elements
Spectral Separation
+ Electronic Delay
Beam + + + + + + + + + + + + + + + + + + + + + + + +
C0I0+ C1I1+ C2I2+ C3I3+ C4I4+ C5I5 + C6I6+ C7I7+ C8I8+ C9I9+ C10I10 + C11I11+ C12I12+ C13I13+ C14I14+ C15I15 + C16I16+ C17I17+ C18I18+ C19I19+ C20I20+ C21I21+ C22I22+ C23I23
Using Phase delay approximation in narrow frequency bands
C0I0+ C1I1+ C2I2+ C3I3+ C4I4+ C5I
September 2009
Station Processing – Cape Town

8. Hierarchical structure
Incoming signal
Geometric
Delay, t
Elements
Electronic Delay + + + + + + + + + + + + + + + + + + + + + + +
Tiles
Station
processor
Beam + + + + +
September 2009
Station Processing – Cape Town

9. Station Processing – Cape Town
AA Station Implementation
Tile beam data
Station
Processor
TH_n
…..
…..
TH_1 Ae
Tile
Processor
- hi
TH_0
…….
…..
To Correlator
….. Ae Ae
Station beam data
September 2009
Station Processing – Cape Town

10. Survey Speed – AA’s forte!
At 1 GHz with 250 deg2 FoV:
AA station is big: say 60m dia.  small beams  ~0.28º width
So, many, many station beams: >3000 !
But, Tile beams are larger, ~5º width
So, # of Tile beams required: 10
Easy....?
August 2009
Station Processor

11. Station Processing – Cape Town
Visually..
Station beams
Tile beam
Great!
Constrains data rate of
Tile  Station processor
September 2009
Station Processing – Cape Town

12. Ah, but... Only perfect for the central beam on Tile beam
Station beams
Tile beam
Only perfect for the central beam on Tile beam
This is what we’ve done.....
September 2009
Station Processing – Cape Town

13. Dynamic range badly affected
Tile Beamforming
Incoming signal
Elements
Electronic Delay + + + + + + + + + + + + + + + + + + + + + + +
Tiles
Station
processor
Beam + + + + + t
Delay
Discontinuities will give high sidelobes and variable forward gain:
Dynamic range badly affected
Tile Beam
Element #
September 2009
Station Processing – Cape Town

14. Hierarchical Beamforming looks best option
Some Consequences
Need ‘extra’ Tile beams over minimum calculated.
Probably also interpolate between Tile beams for more precision
Performance can still be arbitrarily good: dynamic range etc. Maybe on limited FoV.
Bandwidth from Tiles to Station processor determined on quality of beams as well as FoV: still programmable.
Concentrate beamforming centrally as much as possible
Calculating “allowable error”, hence Tile  Station Proc. Communications requirment is “work-in-progress”
Hierarchical Beamforming looks best option
September 2009
Station Processing – Cape Town

15. AA Station Processor Reqts.
Beamform the output of all the Tile processors into Station beams
Send “correlator ready” data over fibre to the central processor
Part of the station calibration scheme
Flexibly handle the data from the AA-hi and AA-lo arrays
Maybe provide the ‘F’ part of the FX correlator
Maybe provide station data for local processing
Possibly hold ‘observation history’ data to post process
Be readily scaleable for:
Number of tiles c. Internal station data rates
Data bandwidth to correlator d. Data length to correlator
August 2009
Station Processor

16. Outline spec. of Processing Chip
Inputs
Outputs
Processing
Device:
PChip
1013 MAC
or 10 TMAC 1 1
Each Stream:
12 x 10Gb/s 2 2 3 3 4 4 5 5
September 2009
Station Processing – Cape Town

17. PChip B’forming requirements
Assuming: All 6 outputs at full speed using all 6 inputs reals per complex sample and 4 real MACs per complex MAC.
6 inputs, I0 – I5 and 6 outputs O0 – O5.
Beamforming for one output stream:
On = C0I0+ C1I1+ C2I2+ C3I3+ C4I4+ C5I5
The input data rate per stream is 120Gb/s raw: 96Gb/s actual
Each complex sample is 2 x 4 bit reals so: ~12.5GS/s per stream.
Processing per output stream is: 4*6*12.5 = 300 GMACs
Total processing for 6 output streams = 1,800 GMACs.
This is well within the PChip capability of 10TMACs
Maybe some pre-processing required on each sample.
September 2009
Station Processing – Cape Town

18. Station Comms 120 Gb/s total 50m range Power: 12-channel fibre
Tx: 2.4 watts
Rx: 2.0 watts
12-channel fibre
10Gb/s channels
VCSEL technology
Pluggable
19mm pitch
August 2009
Station Processor

19. Station Processing – Cape Town
AA Station Data rates
TH_n
…..
…..
12 fibre each
12 fibre each
Long distance drivers
e/o
TH_1
e/o Ae
e/o
Tile
Processor
- hi
TH_0
Station
Processor
o/e
e/o
o/e
…..
o/e
o/e
…..
e/o
o/e Ae
e/o
o/e
e/o
…..
e/o
…….
e/o Ae
e/o
Long distance drivers
e/o
o/e
e/o
o/e
o/e
o/e
GHz
analogue
o/e
…..
…..
Inputs #: 1296
Channel rate: 120Gb/s
(raw)
Total i/p rate: 1.5 Pb/s
Typical:
AA-hi tiles: 300
AA-lo tiles: 45
Total: 345
I/p data rate: 42Tb/s
o/e
………...
o/e
o/e
…..
To Correlator
…….
…..
Tile
Processor
- lo
e/o
e/o
10Gb/s fibre
…..
e/o
…..
…..
....
e/o
Max 4 Station
Processors
Station Processor n
TL_0
Long distance drivers
TL_1
1.0 GHz
analogue
…..
…..
Notes:
1. No control network shown
2. Up to 4 station processor systems can be implemented in parallel
3. Data shown are raw, typ. get 80% data
TL_m
Local Processing
e.g. Cal; pulsars
September 2009
Station Processing – Cape Town

20. Station Processor Board
Each link is 12 diff. copper
o/e
PChip
PChip
e/o 1
o/e
e/o 2
o/e
e/o 3
o/e
e/o 4
o/e
e/o 5
o/e
e/o
o/e
PChip
PChip
e/o
o/e
e/o
o/e
e/o
o/e
e/o
o/e
e/o
12-channel Rx module. e.g Avago AFBR-820BXXZ
o/e
e/o
12-channel Tx module. e.g Avago AFBR-810BXXZ
o/e
PChip
PChip
e/o
o/e
e/o
o/e
e/o
o/e
e/o
To 1st stage
Processors
or Primary Station Processors
To Secondary Station
Processors or long distance fibre drivers
o/e
e/o
o/e
e/o
o/e
PChip
PChip
e/o
o/e
e/o
o/e
e/o
o/e
e/o
o/e
e/o
o/e
e/o
Each link is 12 fibre
Each link is 12 fibre
o/e
PChip
PChip
e/o
o/e
e/o
o/e
e/o
o/e
e/o
o/e
e/o
o/e
e/o 30
o/e
PChip
PChip
e/o 31
Total Raw input
data rate: 4.32Tb/s
o/e
“All to All”
Connections
e/o
Total Raw output data
rate: 4.32Tb/s max 32
o/e
e/o 33
o/e
e/o 34
o/e
e/o 35
o/e
e/o
Control
Processor
Tx/Rx
Line
Station
Control
Station Processor Board
August 2009
August 2009
Station Processor
Station Processor

21. Station Processing – Cape Town
Station Processor system (120Gb/s per Tile)
Station Processor
Primary
Station
Processor
Board
Secondary
Station
Processor
Board
Long distance drivers
Max no. of Tiles
AA-hi+AA-lo is
1296 1 2 1 2
…..
…..
…..
…..
….. 35
AA-hi
Primary
Station
Processor
Board 1
…..
Secondary
Station
Processor
Board 1
Long distance drivers
To each
1st stage
Processor
…..
…..
…..
….. To
Correlator
…..
…..
Primary
Station
Processor
Board
(max 35)
…..
Secondary
Station
Processor
Board
(max 35)
Long distance drivers
AA-lo
“All to All”
Connections
…..
…..
…..
…..
Each correlator channel is10Gb/s (maybe colour multiplexed together)
Each link is 12 fibre
Each link is 12 fibre
September 2009
Station Processing – Cape Town

22. Long distance drives ….. Block 0 To Secondary Station To Correlator
PChip
e/o
e/o
e/o
e/o
e/o
e/o
Block 0
e/o
e/o
e/o
e/o
e/o
12-channel Rx module. e.g Avago AFBR-820BXXZ
e/o
Each link is fibre10Gb/s raw
o/e
To Secondary Station
Processors 1
….. To
Correlator
o/e 2
o/e 3
o/e 4
o/e
Long distance 10km Tx module. 5
o/e
e/o
Each link is 12 fibre
e/o
e/o
e/o
e/o
e/o
Block 5
e/o
Total Raw output data
rate: 720Gb/s max
e/o
e/o
Total Raw input
data rate: 720Gb/s
e/o
e/o
e/o
Control
Processor
Tx/Rx
Line
Station
Control
August 2009
Station Processor

23. Estimated Costs (120Gb/s per Tile)
Total: €480k
August 2009
Station Processor

24. AA Station Beamforming can be done!
Conclusions
Beamform to 1 chip depth at the Tile
Station Processor data rate is much higher than output
Processing performance is unlikely to be an issue
Comms costs and performance critical
Focus on making wide-area comms cheaper!
AA Station Beamforming can be done!
August 2009
Station Processor