1. Radial-by-Radial Noise Power Estimation Igor Ivić and Christopher Curtis CIMMS/University of Oklahoma and NSSL/National Oceanic and Atmospheric Administration NEXRAD TAC Norman, OK August 29, 2012

2. 2 Incorrect noise power measurements can lead to: –Reduction of coverage when noise power is overestimated Case in most radar sites Radar product images cluttered by noise speckles if the noise power is underestimated –Usually occurs in cases of strong interference Biased meteorological variables at low to moderate SNR Blue-sky noise used to produce system noise power –Adjusted for lower elevations using correction factors –For each elevation, the same value is used at all azimuths However, noise drifts with time and varies with antenna position in azimuth and elevation Motivation + the system gain can change within minutes …

3. 3 Solution Estimate receiver noise power at each antenna positionEstimate receiver noise power at each antenna position –Noise power needs to be computed in real-time (i.e., from data containing mixed signals and noise) BUT HOW DO YOU DO THAT?BUT HOW DO YOU DO THAT? Radial-by-radial noise power estimation technique provides the solution

4. 4 Radial-by-Radial Noise Estimation History –First Version presented at ERAD in September 2010 required rough initial noise value –Second version presented at AMS in January 2011 no rough initial noise value required –Algorithm description of the latest version delivered to the ROC in May 2012 includes high gradient signal removal simplified so it operates only on the measured powers The technique has been in use on the NWRT since June 2011

5. 5 Technique Description Expert noise power N e determined visually Used to assess the technique accuracy

6. 6 Power Profile of Test Data Received power as a function of range at the elevation angle of 0.5 deg. The expert noise power (N e ) is indicated with a grey line.

7. 7 Step 1: Strong Point Clutter Rejection Gate at location k is considered to contain point clutter if its power is much larger than the power at surrounding gates BEFORE POINT CLUTTER REJECTIONAFTER POINT CLUTTER REJECTION

8. 8 Step 2: Detect Flat Sections The flat sections of the power profile are identified as this is an indication of the potential signal-free regions –This is done by estimating local variance along range –(local range variance at k range gate at k considered potentially signal-free Notice that the local range variance is smaller in noise regions

9. 9 Step 2: Flat Sections Detected The mean power is computed for each group of contiguous range gates classified as signal-free Out of those, the smallest one is taken to be the intermediate noise power estimate (N int )

10. 10 Step 3: SNR Censoring Using N int Discard all samples at range locations for which the power estimate is larger than the threshold (THR(M)xN int )

11. 11 Step 3: Output after N int SNR Censoring SNR profile of data after discarding samples at range gates where power estimate is larger than the threshold. Mean power/expert noise = -0.216 dBMean power/expert noise = 0.243 dB Potential signal regions

12. 12 Step 4: Apply "range persistence" Filter Detects larger sample powers that exhibit continuity in range – Finds 10 or more consecutive power values larger than the median power

13. 13 Steps 5&6:Update Active Range Gates and Censor using SNR Threshold Mean power/expert noise = -0.2376 dB Mean power/expert noise = 0.1586 dB Discard the samples marked by the range persistence filter Compute the mean power of the remaining samples Perform SNR censoring

14. 14 Step 7: Apply Running Sum Filter A running sum is performed over the array of remaining powers –makes regions with weak signals visible After 1 iteration mean power/expert noise = -0.0043 dB After 1 iteration mean power/expert noise = -0.3 dB

15. 15 Accuracy Assessment Algorithm failed to produce noise estimate 43 times out of 155164 (0.03%) BIAS = -4×10 -3 dB or -0.086% of the true noise STD = 5.3×10 -2 dB or 1.22% around the mean To assess the accuracy estimated noise is subtracted from all powers simulated noise is artificially added to the real data

16. 16 SUMMARY Radial-by-radial noise power estimation technique accurately produces noise power measurements in real-time! GREAT SUCCESS!

17. 17 Benefits of Radial-by-Radial Noise Estimation Comparison between the measured noise power and legacy noise power –Can see effects from man-made (KVNX) and mountain (KPDT) clutter –Will use data from these cases to illustrate benefits KVNX: Vance AFB, OK KPDT: Portland, OR 0.9 dB 1 dB 4 dB 1 dB

18. 18 Reflectivity Coverage KPDT Data with Legacy Noise (0.9°, VCP 12, Cut 3)

19. 19 Coverage increased by 18% Reflectivity Coverage KPDT Data with Measured Noise (0.9°, VCP 12, Cut 3)

20. 20 Invalid Spectrum Width Values 69.6% of v estimates with legacy noise are valid (i.e., v > 0) 89.0% of v estimates with measured noise are valid KPDT Data with Legacy vs. Measured Noise (0.9 °, VCP 12, Cuts 1&2) 49.4% IMPROVEMENT IN THE NUMBER OF VALID ESTIMATES

21. 21 78% of | hv (0)| estimates with legacy noise are less than one 84.48% of hv (0) estimates with measured noise are less than one 14.9% IMPROVEMENT IN THE NUMBER OF VALID ESTIMATES Invalid Correlation Coefficient Values KVNX Data with Legacy vs. Measured Noise (0.5°, VCP 11, Cut 1)

22. 22 Effects on Z DR Differential reflectivity is estimated as where N h and N v are errors in noise floors Using perturbation analysis, we get –If errors in H and V are of different sign they add up, otherwise they may cancel

23. 23 Contribution of Noise Error to Z DR Bias KPDT N h = -80.6 dBm & N v = -80.3 dBm N h = -4 dB & N v = -1 dB KVNX N h = -80.7 dBm & N v = -81.2 dBm N h = -0.9 dB & N v = -1 dB In the current system, noise errors are significant contributors to the Z DR bias for low to moderate SNR Real-time noise estimates eliminate this contribution because of accurate measurements (i.e., N h 0, N v 0)

24. 24 Summary Improved data quality with measured noise: –Increased coverage for all moments and dual- polarization variables –Reflectivity – decreased bias –Spectrum Width ( v ) – decreased bias and increased number of valid estimates –Correlation coefficient (| hv (0)|) – decreased bias and increased number of valid estimates –Differential reflectivity (Z DR ) – decreased bias Accurate noise measurement is crucial for keeping measurement biases within acceptable levels

25. 25 Conclusions Accurate noise estimation accounts for noise variations due to: –Changes in system gain (Melnikov et al.) –Radiation caused by clutter, storms and man made objects –These noise variations can lead to: Reduction of coverage when noise power is overestimated Radar product images cluttered by noise speckles if the noise power is underestimated Biased meteorological variables at low to moderate SNR (e.g., differential reflectivity and correlation coefficient) The current system calibration does NOT address these issues! Radial-by-Radial Noise Estimation eliminates or greatly mitigates all of these issues stemming from incorrect noise power measurements.

26. 26 Backup Slides

27. 27 KVNX Data with Legacy Noise (0.5°, VCP 11, Cut 1) Reflectivity Coverage

28. 28 Coverage increased by 6.18% Reflectivity Coverage KVNX Data with Measured Noise (0.5°, VCP 11, Cut 1)

29. 29 KPDT Data with Legacy Noise (0.5°, VCP 12, Cuts 1&2) Invalid Spectrum Width Values

30. 30 Invalid Spectrum Width Values KPDT Data with Measured Noise (0.9°, VCP 12, Cuts 1&2)