L2OS RFI status Nicolas Lamquin, Jean-Luc Vergely Jacqueline Boutin Paul Spurgeon ICM, Barcelona, 16/17 May 2013

RFI flags in v600 processing Version of products: v600 source: sources-L2OS-core-v6.00.tar.gz job orbit: descending SM_TEST_MIR_SCSF1C_ T040402_ T045804_600_001_0 L1c flag description : SO-TN-IDR-GS-0005 (24/10/2012) CNFOSF file: SM_TEST_AUX_CNFOSF_ T000000_ T000000_001_023_8 RFI region: arctic

L1c RFI flags (measurement) Fm_rfi_mitigation_tot (bit 7)  (v600) rfi_xx – analysis of NIR BT in X polarization Fm_rfi_strong_tot (bit 15)  (v600) rfi_yy – analysis of NIR BT in Y polarization Fm_rfi_tails_tot (bit 12)  never raised – affected by tails identified in the AUX_RFI list Fm_rfi_point_tot (bit 16) – affected by point source identified in the AUX_RFI list

L1c  L2 (DAP: measurements) L2 flag « L1c_rfi »: True if any of L1c RFI flag True snapshot 34 (id: ): Fm_rfi_point_tot flag in L1c  L2 « L1c_rfi » is True No RFI detection from NIR Tails in Fm_rfi_point_tot ? Y

green = RFI_XX True red = RFI_XX True and RFI_YY True snap. 632 id: snap. 633 id: snap. 634 id: ? Y X

L2 RFI flags (measurements) Fm_rfi_L1 Fm_l2_rfi Fm_l2_rfi_outlier Fm_l2_rfi_snapshot_out_of_range Fm_l2_rfi_high_snapshot_std Fm_l2_rfi_high_snapshot_std_st3 Fm_l2_rfi_high_snapshot_std_st4

L2 RFI flags (measurements) Fm_l1c_rfi Fm_l2_rfi Fm_l2_rfi_outlier Fm_l2_rfi_snapshot_out_of_range Fm_l2_rfi_high_snapshot_std Fm_l2_rfi_high_snapshot_std_st3 Fm_l2_rfi_high_snapshot_std_st4 gridpoint analysis (risk of RFI + RFI_nsig test)  local information per measurement snapshot analysis  raised for complete snapshot  from L1c  True if Fm_l2_rfi_outlier or Fm_out_of_range, nominally includes snapshot analysis below but not in current configuration

Fm_l2_rfi = Fm_l2_rfi_outlier OR Fm_out_of_range Fm_out_of_range: threshold test on residual per measurement (Tm_out_of_range=50 K) Fm_l2_rfi_outlier: analysis of all residuals per gridpoint (« risk of RFI ») + threshold test on residuals (comparison to std) snapshot 34 (id: )

extra points in Fm_l2_rfi = Fm_out_of_range Fm_l2_rfi_outlier is not raised although Fm_out_of_range is raised  bRiskOfRFI seems not raised sensitivity to examine (threshold 50 K) snapshot 649 (id: ) Sun tails ?

flags all snapshot if any Fm_out_of_range does not proceed if some coast/ice nearby: set to True only if %(coast/ice) < Ts_snapshot_out_of_range Ts_snapshot_out_of_range = 20%  test threshold Fm_l2_rfi_snapshot_out_of_range

example 1 snapshot 34 (id: ): example 2 snapshot 649 (id: ): Land contamination : Snapshot flag not raised

Fm_l2_rfi_high_snapshot_std Fm_l2_rfi_high_snapshot_std_st3 and Fm_l2_rfi_high_snapshot_std_st4 snapshot standard deviation tests per measurement in snapshot (« Fs_high_std... ») propagates through complete snapshot test 1: amount of data for statistics must be > Ts_meas_min (default: 35%) test 2: sqrt(std / nMeas) > Ts_std (default: 2.5) same threshold for « std », « std_st3 », and « std_st4 »  refinement of threshold(s) ?

Test definition for each configuration : process 1 month of data ? NO ! (too much thresholds to be tested) tests on one orbits and do relative comparison : relative SSS, number of valid TBs …etc. tests on several orbits in a (lat,lon) window in order to check the salinity estimation (comparison with ISAS maps). after a choice of configuration : monthly test

Remarks would be useful to have snapshot_ID for all measurements in L2 DAP need « Fs_high_std... » in DAP ? DPM not coherent in 8_2-1 and 8_3-1 (OK in the code) Some variables not defined in the DPM: RFI_c1, RFI_c2 Fm_RFI_L1_tot (type ‘s’) in DPM ? (OK in the code : all L1c flags are used)

L2OS RFI detection Decision tree

Introduction Difference between outlier and RFI ? RFIs : unexpected physical source + Gibbs and tails Outliers : reconstruction problem ? - we expect snapshot signature for RFI and outlier - long time period contamination for RFI : seen by all the orbits which overpass the source : AUX_DGG_RFI information

Flags in the decision tree if Fm_Resol(im,igp) or (Fm_outlier(im,igp) and not Fg_ctrl_suspect_RFI) or (Fm_L2_RFI(im,igp) and Fg_ctrl_suspect_RFI) or Fm_out_of_range(im,igp) or Fm_border_fov(im,igp) or Fm_Moon_SpecDir(im,igp) or Fm_gal_noise_error(im,igp) or Fm_sun_limit(im,igp) or Fm_L1c_sun(im,igp) then Fm_valid = false endif Fg_ctrl_suspect_RFI is raised if (number of Fm_L2_RFI==1 )/Na > Tg_num_RFI_max Fm_outlier : comes from comparison between TB_smos and TB_model corrected from a global shift applied at the dwell line level. nsigma comparison. Fm_L2_RFI : Fm_L2_RFI_outlier or Fm_L2_RFI_snapshot_out_of_range or Fm_L2_RFI_high_snapshot_std Fm_L2_RFI_outlier : comes from comparison between TB_smos and TB_model. RFI_nsigma comparison. Fm_outlier and Fm_L2_RFI_outlier very close: NO REASON TO USE DIFFERENT nsigma criteria (?). Proposition : suppress Fm_L2_RFI_outlier and keep only snapshot information for Fm_L2_RFI.

INSTEAD OF : if Fm_Resol(im,igp) or (Fm_outlier(im,igp) and not Fg_ctrl_suspect_RFI) or (Fm_L2_RFI(im,igp) and Fg_ctrl_suspect_RFI) or Fm_out_of_range(im,igp) or Fm_border_fov(im,igp) or Fm_Moon_SpecDir(im,igp) or Fm_gal_noise_error(im,igp) or Fm_sun_limit(im,igp) or Fm_L1c_sun(im,igp) then Fm_valid = false Endif PUT : if Fm_Resol(im,igp) or Fm_outlier(im,igp) or (Fm_L2_RFI(im,igp) and Fg_ctrl_suspect_RFI) or Fm_out_of_range(im,igp) or Fm_border_fov(im,igp) or Fm_Moon_SpecDir(im,igp) or Fm_gal_noise_error(im,igp) or Fm_sun_limit(im,igp) or Fm_L1c_sun(im,igp) then Fm_valid = false Endif WITH Fm_L2_RFI(im,igp) : Fm_L2_RFI_snapshot_out_of_range or Fm_L2_RFI_high_snapshot_std

Remarks Fm_l2_RFI and Fg_ctrl_suspect_RFI do not integrate L1c RFI information Fm_l2_RFI and Fg_ctrl_suspect_RFI do not integrate AUX_DGG_RFI information -> AUX_DGG_RFI and L1c information not used in the decision tree

Extra slides

Snapshot out-of-range TB detection (PRP_8_2) -> Fs_out_of_range -> Fm_l2_rfi_snapshot_out_of_range Snapshot standard deviation tests (PRP_8_3) -> Fs_high_std -> Fm_l2_rfi_high_snapshot_std Measurement outlier detection (PRP_8_4) -> Fm_l2_RFI_outlier -> Fm_outlier OTT correction RFI AUX detection (PRP_8_5) -> Fm_l2_RFI (from Fm_l2_RFI_outlier, Fm_l2_rfi_snapshot_out_of_range, Fm_l2_rfi_high_snapshot_std) -> Fg_ctrl_suspect_RFI Snapshot level Dwell line level Grid point level