- 1 - RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 LOFAR RFI Mitigation spatial filtering at station level Albert-Jan Boonstra Mark Bentum Mathheijs Eikelboom

- 2 - RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Contents LOFAR overview Spectrum environment RFI mitigation in LOFAR Data model, spatial filtering algorithm Spatial filtering in LOFAR, considerations Spatial filter results Conclusion

- 3 - RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 LOFAR signal processing, overview LOFAR: fsky ~ 30 – 240 MHz BlueGene central Processor CEP (correlator) LBA HBA RSP receiver antenna beam station beams synth. beams 1 x 32 MHz High Band Antenna Low Band Antenna

- 4 - RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 LBA antenna layout Operational Oktober 1, 2006 with “final” prototype hardware at Exloo 96 dual-dipole LBA antennas distributed over ~500m: one cluster with 48 dipoles three clusters of 16 dipoles Total 24 microstation, 4 dipoles each Goal: emulate LOFAR with 24 micro-stations at reduced bandwidth or act as a single station at full BW Exloo R.Nijboer 2006 LOFAR CS1 configuration – Exloo LBA CS10

- 5 - RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 LOFAR status Stations 18 core stations + 18 remote stations + 8 int. Validated: 14 CR, 6 RS In progress: 6 CS, 1 RS, 3 German, 1 French Next: 9 RS, 1 UK, 1 Germany, 1 Sweden

- 6 - RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 LOFAR numbers Number of sensors on the various fields: Core station fields (18) 96 Low Band Antennas, 2 x 24 High Band Antenna Tiles (HBA field is split) Remote station fields (18) 96 Low Band Antennas, 48 High Band Antenna Tiles Microbaromater (infrasound) Geo-Remote station fields (10) Geophones & Microbarometers International station fields (8) 96 Low Band Antennas, 96 High Band Antenna Tiles Numbers for the LOFAR telescope performance Frequency range: MHz and MHz Polarisations 2 Bandwidth 32 MHz (currently 48 MHz investigated) Stations: 18 core, 18 remote, 8 international Baseline length: 100 m to 1500 km Simult. dig. beams: 8 Sample bit depth: 12 Spectral resolution: 0.76 kHz

- 7 - RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Some LOFAR imaging results High-resolution LOFAR 3C61.1 image Credit: Reinout van Weeren (Sterrewacht Leiden) 8 feb 2010 Cas A, Sarod Yatawatta 23 Dec LOFAR HBA tile all-sky image Michiel Brentjens, 22 nov LOFAR all-sky image Stefan Wijnholds 19 Nov LOFAR LBA alll-sky image Sarod Yatawatta & Jan Noordam 20 April 2007 Deep LOFAR HBA Image Sarod Yatawatta, 21 Feb LOFAR all-sky image Stefan WIjnholds 25 June

- 8 - RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Spectrum environment Spectrograms 2009 FM band II AM TV band I Lopik weather sat. DAB TV#6,7,... pager ambu- lance, taxi mariphone geostat. mil. satellite TV band III / DAB (DVB) FM band II aviation RAS land mobile land mobile mobile land mobile frequency (MHz)

- 9 - RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 RR LOFAR overview spectral estimation: multiply one arm per interferometer with: e i  t R clean = R-R  spectral estimation: derive  for AM from R  Spatial filtering: w new = P w LOFAR station on-line correlator One covariance matrix R per second 512 subbands correlated in ~8.5 min.

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 The radio spectrum: occurrence of weak RFI 1 minute: ~ 0.02 dB relatively few weak RFI sources: “horizon effect” LOFAR High Band Antenna var(R 11 ) =  4 / N Tsky = 333 K Nf = 256 Nt = 300  = 60 s  t = 5 h  f = 0.76 kHz

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 The radio spectrum: effect on data transport rate Purpose: - increase the number of beams from 8 to 24 (both for 24 MHz bw) - Without increase of station output data rate - Solution: reduce data rate to the LOFAR central processor from 16 to 4 bits (complex) for each beam Loss when using 4 bit could be solved by spatial filters at stations, but only for fixed transmitters (“fast moving nulls” would hamper calibration) Experiments in cleanest part of the spectrum indicated that < 10% of the data would be lost (no spatial filtering applied). e.g. L HBA bands: In 3 of 23 bands: 16 bits: 4 bits ~ 50% In 20 of 23 bands: no loss Average loss: 6.5%

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Requirement: “smooth” station beamshape changes Credit: Sarod Yatawatta, ASTRON LOFAR CS1 calibration/imaging Observation: 16 single-dipole stations, 48 h 20 subbands, each 0.14 MHz “Calibrated”: removing phase drift (uv) “Residual”: peeling CasA and CygA LOFAR ITS 2004 observations 60 antennas, MHz, basel.<200 m with transmitter (left) after subtraction filtering (right) after projection filtering (middle) LOFAR spatial filtering At stations, filters for fixed directions (at subband level, ~ 200 kHz) Post correlation: offline spatial filtering (in ~1 kHz channels)

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Data model – signal model Consider an array of p antennas with baselines : b ij = r i -r j Array output signals x i (t) and the noise signals n i (t) (from LNAs, spillover etc) are stacked in a vector: x(t) = [x 1 (t), …, x p (t)] t,n(t) = [n 1 (t), …, n p (t)] t Suppose there is one source (astronomical or RFI) with signal s(t) from direction s, and with spatial signature vector a: The signal vector is defined by: x(t) = a s(t) + n(t) t t

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Data model – covariance model Define the signal covariance sample estimate (observational data): R =  x n x n H with x n = x (nT s ) Given i.i.d. noise vector n(t), E{n(t)n(t) H } =  n 2 I, and E{s(t)} 2 =  2 : R = E{R} =  2 a a H +  n 2 I Data model easily exended to multiple sky sources and multiple RFI sources Complication: low frequency sky contains strong extended structures Solution: extend model, use baseline restrictions, factor analysis algorithms However: this is not always a problem, e.g. in estimating DOA of strong RFI n=1 N ^ ^

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Data model with sky sources having covariance R v, and interference with power  r 2 and signature vector a r : R = R v +  n 2 I +  r a r a r H Spatial filtering using projections Projection matrix: P = I – a r (a r H a r ) -1 a r H note: Pa r = 0, Pa ≠ 0 Applying projection: R = P R P Spatial filtering using subtraction: R = R –  r 2 a r a r H Note: the subtraction filter can be rewritten as a projection filter by adding a scaling factor , dependent on the noise and on the RFI power: P = I –  a r (a r H a r ) -1 a r H cf. A. Leshem, A.J. van der Veen, and A.J. Boonstra. Multichannel interference mitigation techniques in radio astronomy. The Astrophysical Journal Supplement Series, 131(1):355–373, November Spatial filtering after correlation ~ ^ ~ ^

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Beamforming and spatial filtering narrow band beamforming Source power: I B = R yy = E{yy H } = w H E{xx H }w = w H Rw => station sky map I b (s) Recall data model: R =  s 2 aa H +  n 2 I Maximum if w = a: w H Rw =  s 2 w H a a H w +  n 2 w H w beamfomer output y=w H x to central processor (BlueGene) local processing: station correlator, one second integrated R every 512 seconds- used for station calibration and RFI mitigation Spatial filtering (beamformer impl.), with P a spatial filter: w’ = P w And: I B = w H PRP w … y = w H x x1x1 x2x2 xpxp w1w1 w2w2 wpwp

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 How to find DOA and time-occupancy DOA: From a subspace analysis, R =U  U: Finding maxima in sky maps Transmitter locations may be known Using factor analysis, efficient rank-one methods ITS data Credit: M.Tanigawa& M.Moren How to assess the time-occupancy of transmitters: sorting eigenvalue spectra and make daily percentile plots of number of eigenvalues above threshold

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Spatial filtering, LBA station at 50.2 MHz One-hour LOFAR LBA spectr (left) and one-day duration Frobenius norm spectrogram (right). Data: 1 second integrated LOFAR station subbands, every subband is updated once per 512 seconds

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Spatial filtering, LBA station at 50.2 MHz LBA station 50.2 MHz (upper) LBA station spectrogram (upper left) and same data after spatially filtering, based on first time slot at 50.2 MHz (lower left) After one hour a second transmitter at a different direction emerges

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Integration time = 60 s Filter: one dimension is projected out Spatial filter suppession: 20 dB (right figure) 2nd obs: second 60 filter of previous time slot, result: no suppression Note: HBA station data is correlated by CEP, forming ~1kHz channels Spatial filtering, HBA station at MHz

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Spatial filtering, HBA station at MHz Fixed spatial projection filter estimated from and applied to first 60 s integration time (right) 16 dB suppression, one subspace dimension removed 38 dB suppression after two dim. Removed (not shown) 4 dB supp using spatial filet of first time 60 s slot (not shown) Air traffic band: moving transmitter or strong changes in propagation

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Spatial filtering, HBA station at MHz Fixed spatial projection filter estimated from and applied to first 60 s integration time (right) 10 dB suppression, one subspace dimension removed 6 dB supp using spatial filter of first time 60 s slot (not shown) 1 dB supp using spatial filter after one hour (not shown) Somewhat erratic suppression numbers over time

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 fixed spatial projection filter (one dim) estimated from and applied to first 60 s integration time (upper right), and applied after 8 hours (right) Spatial filtering, HBA station at MHz

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Channel without RFI (phase fluctuations partly due tot sky) Channel with RFI Spatial filtering, HBA station at MHz

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 LOFAR Core Station, LBA antennas subspace analysis: eigenvalues Frobenius norm spectrogram October 2009

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Core station: off-line spatial filtering spectra before filteringspectra after filtering

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Core station: off-line spatial filtering before filteringafter filtering, filter update every snapshot

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Core station: off-line spatial filtering after filtering, only one filter settingafter filtering, filter update every snapshot

RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 Conclusions and next steps No accumulation observed of weak -240 dBWm -2 Hz -1 levels (horizon effect) Experiments indicated that station output signal data rate reduction (16 to 4 bit) would lead to < 10% data loss in cleanest parts of the spectrum Fixed spatial filters can be applied at station level to suppress fixed transmitters –LOFAR systems are stable enough, performance will be improved by applying station calibration Coming year: RFI direction inventory At some later stage: reconsider station spatial filter updates every second using filters with constraints at the direction of the strongest “peeling sources”