lofar rfi mitigation spatial filtering at station level
DESCRIPTION
Albert-Jan Boonstra Mark Bentum Mathheijs Eikelboom. LOFAR RFI Mitigation spatial filtering at station level. Contents. LOFAR overview Spectrum environment RFI mitigation in LOFAR Data model, spatial filtering algorithm Spatial filtering in LOFAR, considerations Spatial filter results - PowerPoint PPT PresentationTRANSCRIPT
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 1 -
LOFAR RFI Mitigationspatial filtering at station level
Albert-Jan BoonstraMark Bentum
Mathheijs Eikelboom
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 2 -
Contents
• LOFAR overview• Spectrum environment• RFI mitigation in LOFAR• Data model, spatial filtering algorithm• Spatial filtering in LOFAR, considerations• Spatial filter results• Conclusion
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 3 -
LOFAR signal processing, overview
LOFAR: fsky ~ 30 – 240 MHz
BlueGenecentral
ProcessorCEP
(correlator)
LBA
HBA
RSP
receiver
antennabeam
stationbeams
synth.beams
1 x 32 MHz
High Band Antenna
Low Band Antenna
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 4 -
LBA antenna layout
Operational Oktober 1, 2006 with “final” prototype hardware at Exloo96 dual-dipole LBA antennas distributed over ~500m:• one cluster with 48 dipoles • three clusters of 16 dipoles
Total 24 microstation, 4 dipoles each
Goal: emulate LOFAR with 24 micro-stations at reduced bandwidth or act as a single station at full BW
Exloo
R.Nijboer 2006
LOFAR CS1 configuration 2006-2008 – Exloo
LBA CS10
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 5 -
LOFAR status
Stations 18 core stations + 18 remote stations + 8 int.
Validated: 14 CR, 6 RS
In progress: 6 CS, 1 RS, 3 German, 1 French
Next: 9 RS, 1 UK, 1 Germany, 1 Sweden
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 6 -
LOFAR numbers
Number of sensors on the various fields:Core station fields (18)
96 Low Band Antennas, 2 x 24 High Band Antenna Tiles (HBA field is split)Remote station fields (18)
96 Low Band Antennas, 48 High Band Antenna TilesMicrobaromater (infrasound)
Geo-Remote station fields (10)Geophones & Microbarometers International station fields (8)
96 Low Band Antennas, 96 High Band Antenna Tiles
Numbers for the LOFAR telescope performanceFrequency range: 30- 80 MHz and 120 - 240 MHzPolarisations 2Bandwidth 32 MHz (currently 48 MHz investigated)Stations: 18 core, 18 remote, 8 internationalBaseline length: 100 m to 1500 kmSimult. dig. beams: 8Sample bit depth: 12Spectral resolution: 0.76 kHz
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 7 -
Some LOFAR imaging results
High-resolution LOFAR 3C61.1 imageCredit: Reinout van Weeren (Sterrewacht Leiden) 8 feb 2010
Cas A, Sarod Yatawatta 23 Dec. 2009
LOFAR HBA tile all-sky imageMichiel Brentjens, 22 nov. 2007
20102009
2008
2007
LOFAR all-sky imageStefan Wijnholds19 Nov. 2008
LOFAR LBA alll-sky image Sarod Yatawatta & Jan Noordam 20 April 2007
Deep LOFAR HBA Image Sarod Yatawatta, 21 Feb. 2008
LOFAR all-sky imageStefan WIjnholds25 June 2006
20062007
2008
2004
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 8 -
Spectrum environmentSpectrograms 2009 FM band IIAM
TV band I
Lopik
weathersat.
DABTV#6,7,...
pagerambu-lance,taxi
mariphone
geostat.mil.
satelliteTV band III / DAB (DVB)
FM band II
aviationRAS
RASland
mobileland mobilemobileland
mobile
frequency (MHz)
frequency (MHz)
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 9 -
R
LOFAR overview
spectral estimation:multiply one arm per interferometerwith:
eit
Rclean = R-R
spectral estimation:
derive for AM from R
Spatial filtering:wnew = P w
LOFAR stationon-line correlator
One covariance matrix R per second
512 subbands correlated in ~8.5 min.
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 10 -
The radio spectrum: occurrence of weak RFI
1 minute: ~ 0.02 dB
relatively few weak RFI sources:“horizon effect”
LOFAR High Band Antennavar(R11) = 4 / N
Tsky = 333 KNf = 256Nt = 300 = 60 st = 5 hf = 0.76 kHz
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 11 -
The radio spectrum: effect on data transport rate
Purpose: - increase the number of beams from 8 to 24 (both for 24 MHz bw)- Without increase of station output data rate- Solution: reduce data rate to the LOFAR central processor
from 16 to 4 bits (complex) for each beamLoss when using 4 bit could be solved by spatial filters at stations, but only for fixed transmitters (“fast moving nulls” would hamper calibration)
Experiments in cleanest part of the spectrumindicated that < 10% of the data would be lost(no spatial filtering applied).
e.g. L2007-0189525 HBA bands:In 3 of 23 bands:loss @ 16 bits: 0% @ 4 bits ~ 50%In 20 of 23 bands:no lossAverage loss: 6.5%
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 12 -
Requirement: “smooth” station beamshape changes
Credit: Sarod Yatawatta, ASTRON
LOFAR CS1 calibration/imagingObservation:
• 16 single-dipole stations, 48 h• 20 subbands, each 0.14 MHz
“Calibrated”: removing phase drift (uv)“Residual”: peeling CasA and CygA
LOFAR ITS 2004 observations60 antennas, 26.75 MHz, basel.<200 m• with transmitter (left)• after subtraction filtering (right)• after projection filtering (middle)
LOFAR spatial filtering• At stations, filters for fixed directions (at subband level, ~ 200 kHz)• Post correlation: offline spatial filtering (in ~1 kHz channels)
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 13 -
Data model – signal model
Consider an array of p antennas with baselines :
bij = ri-rj
Array output signals xi(t) and the noise signals ni(t) (from LNAs, spillover etc) are stacked in a vector:
x(t) = [x1(t), … , xp(t)]t, n(t) = [n1(t), … , np(t)]t
Suppose there is one source (astronomical or RFI) with signal s(t) from direction s, and with spatial signature vector a:
The signal vector is defined by:
x(t) = a s(t) + n(t)
t
t
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 14 -
Data model – covariance model
Define the signal covariance sample estimate (observational data):
R = xn xnH with xn = x (nTs)
Given i.i.d. noise vector n(t), E{n(t)n(t)H} = n2 I, and E{s(t)}2 = 2:
R = E{R} = 2 a aH + n2 I
Data model easily exended to multiple sky sources and multiple RFI sources
Complication: low frequency sky contains strong extended structures
Solution: extend model, use baseline restrictions, factor analysis algorithmsHowever: this is not always a problem, e.g. in estimating DOA of strong RFI
n=1
N
^ ^
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 15 -
Data model with sky sources having covariance Rv, and interference with power r2
and signature vector ar: R = Rv + n
2 I + r ar arH
Spatial filtering using projections
Projection matrix: P = I – ar (arHar) -1 ar
H note: Par = 0, Pa ≠ 0
Applying projection: R = P R P
Spatial filtering using subtraction:
R = R – r2 ar ar
H
Note: the subtraction filter can be rewritten as a projection filter by adding a scaling factor , dependent on the noise and on the RFI power:
P = I – ar (arHar)-1 ar
H
cf. A. Leshem, A.J. van der Veen, and A.J. Boonstra. Multichannel interference mitigation techniques in radio astronomy. The Astrophysical Journal Supplement Series, 131(1):355–373, November 2000.
Spatial filtering after correlation
~ ^
~ ^
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 16 -
Beamforming and spatial filtering
narrow band beamforming
Source power: IB = Ryy = E{yyH} = wHE{xxH}w = wHRw=> station sky map Ib(s)
Recall data model: R = s2 aaH + n
2 I
Maximum if w = a: wHRw = s2 wHa aHw + n
2 wHw
beamfomer output y=wHxto central processor (BlueGene)
local processing: station correlator, one second integrated R every 512 seconds- used for station calibration and RFI mitigation
Spatial filtering (beamformer impl.), with Pa spatial filter: w’ = P wAnd: IB = wH PRP w
… y = wHx
x1
x2
xp
w1
w2
wp
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 17 -
How to find DOA and time-occupancy
DOA:• From a subspace analysis, R =U U:
• Finding maxima in sky maps• Transmitter locations may be known• Using factor analysis, efficient rank-one methods
ITS data
Credit: M.Tanigawa& M.Moren
How to assess the time-occupancy oftransmitters: sorting eigenvalue spectra and make daily percentile plots of number of eigenvalues above threshold
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 18 -
Spatial filtering, LBA station at 50.2 MHz
One-hour LOFAR LBA spectr (left) and one-day duration Frobenius norm spectrogram (right).
Data: 1 second integrated LOFAR station subbands, every subband is updated once per 512 seconds
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 19 -
Spatial filtering, LBA station at 50.2 MHz
LBA station eignevalues @ 50.2 MHz (upper)
LBA station spectrogram (upper left) and same data after spatially filtering, based on first time slot at 50.2 MHz (lower left)
After one hour a second transmitter at a different direction emerges
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 20 -
Integration time = 60 s
Filter: one dimension is projected out
Spatial filter suppession: 20 dB (right figure)
2nd obs: second 60 sample @ filter of previous time slot, result: no suppression
Note: HBA station data is correlated by CEP, forming ~1kHz channels
Spatial filtering, HBA station at 143.75 MHz
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 21 -
Spatial filtering, HBA station at 131.25 MHz
Fixed spatial projection filter estimated from and applied to first 60 s integration time (right)• 16 dB suppression, one subspace dimension removed• 38 dB suppression after two dim. Removed (not shown)• 4 dB supp using spatial filet of first time 60 s slot (not shown)Air traffic band: moving transmitter or strong changes in propagation
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 22 -
Spatial filtering, HBA station at 185.02 MHz
Fixed spatial projection filter estimated from and applied to first 60 s integration time (right)• 10 dB suppression, one subspace dimension removed• 6 dB supp using spatial filter of first time 60 s slot (not shown)• 1 dB supp using spatial filter after one hour (not shown)Somewhat erratic suppression numbers over time
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 23 -
fixed spatial projection filter (one dim) estimated from and applied to first 60 s integration time (upper right), and applied after 8 hours (right)
Spatial filtering, HBA station at 225.04 MHz
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 24 -
Channel without RFI(phase fluctuations partly due tot sky) Channel with RFI
Spatial filtering, HBA station at 225.04 MHz
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 25 -
LOFAR Core Station, LBA antennas
subspace analysis:eigenvalues
Frobenius norm spectrogram
October 2009
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 26 -
Core station: off-line spatial filtering
spectra before filtering spectra after filtering
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 27 -
Core station: off-line spatial filtering
before filtering after filtering, filter update every snapshot
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 28 -
Core station: off-line spatial filtering
after filtering, only one filter settingafter filtering, filter update every snapshot
RFI2010 Workshop, Groningen, Nl, March 29-31, 2010 - 29 -
Conclusions and next steps
• No accumulation observed of weak RFI @ -240 dBWm-2Hz-1 levels (horizon effect)
• Experiments indicated that station output signal data rate reduction (16 to 4 bit) would lead to < 10% data loss in cleanest parts of the spectrum
• Fixed spatial filters can be applied at station level to suppress fixed transmitters– LOFAR systems are stable enough, performance will be
improved by applying station calibration
• Coming year: RFI direction inventory
• At some later stage: reconsider station spatial filter updates every second using filters with constraints at the direction of the strongest “peeling sources”