1. Frequency Lines Identification: cmt package for On-line (by Fd ) and Off-line Virgo data quality Sabrina D’Antonio Roma2 Tor Vergata Roma 1 Pulsar group: Pia Astone, Sergio Frasca, Cristiano Palomba, Federica Antonucci

2. Data quality tool: frequency domain lines identification by the PSS library (Rome 1 group) SUMMARY: Time domain disturbances: event’s identification and removal (to obtain clean data) Estimation of the average AR spectrum Frequency domain lines identification The log file and the spectra files

3. We want to identify and remove “high frequency shorts events”, which will increase the level of the noise. Events identification: after high-pass bilateral filtering (not phase shifts) with f hp =100 Hz. Once high frequency events have been found and their parameters registered, we “subtract” them from the original time series. Hence we produce the “cleaned data sets”. Time domain disturbances: event’s identification and cleaning

4. Time domain disturbances: event’s identification We evaluate AR mean and standard deviation; We evaluate AR mean and standard deviation; The threshold is set on the critical ratio CR defined as The threshold is set on the critical ratio CR defined as The memory time  depends on the apparatus. We set it to 600 s and the CR thr to 6. The memory time  depends on the apparatus. We set it to 600 s and the CR thr to 6. The “dead time”, minimum time between two events, depends on the noise and the expected signal. We are using 1s. The “dead time”, minimum time between two events, depends on the noise and the expected signal. We are using 1s.

5. The cleaning requires the set up of another parameter, “the edge width”: it indicates how many seconds before and after the event are used in the cleaning of the data. We have used 0.15 s The cleaning requires the set up of another parameter, “the edge width”: it indicates how many seconds before and after the event are used in the cleaning of the data. We have used 0.15 s From the “beginning time” up to the “beginning time + duration” we subtract the high frequency component to the data. From the “beginning time” up to the “beginning time + duration” we subtract the high frequency component to the data. Data from the “beginning time – edge width“ to “beginning time” and data up to “beginning time+duraton+edge width” are linearly interpolated. Data from the “beginning time – edge width“ to “beginning time” and data up to “beginning time+duraton+edge width” are linearly interpolated. Time domain disturbances: cleaning

6. The procedure to estimate the average spectrum A good estimator should have the following properties: If peaks in the frequency domain are present, the estimator should not be affected by peaks. This should be as much as possible independent on the SNR of the peak; If the noise level varies, either slowly or rapidly, the estimator should be able to follow the noise variations. We refined the procedure, with the use of an autoregressive estimation (AR) of the average of the spectrum, with the basic idea of a “clean estimator”.

7. AR estimation. Already described applied from higher frequencies toward lower frequencies. To deal with the increasing of the noise level toward lower frequencies. FFT length: 4194314 ->T=1048.6s FFT mode: overlapped (or not) by the half, flat top- cosine window The procedure to estimate the average spectrum

8. The frequency lines search starts with the ratio R of the spectrum to its AR estimation. On this function, we set a threshold at the level of SNR thr =(2.5) 0.5. GEN_FAC* In the log files we record frequency lines with SNR≥GEN_FAC*SNR thr (GEN_FACT used =2). All the data which cross the threshold are local maxima are then registered into the log file. Lines frequency identification: peak map

9. Output Files Log file Date_of_creation.log: Log file Date_of_creation.log: * information about time domain and frequency domain events *one file for all the processed time period (?) *24 Mb (10 days). Spectra files : Spectra files : PS with high frequency resolution df=9.5367e-004 Hz A new file every 100 FFT (?) Dimension ~ 850 Mb PS with lower frequency resolution df*128 Hz Dimension ~ 6.3 Mb

10. PSS crea_sfdb job log file PSS crea_sfdb job log file started at Wed Jan 23 09:51:32 2008 started at Wed Jan 23 09:51:32 2008 INPUT : VIR_h_4000Hz_869983200.GWF First data time in the first file of the run INPUT : VIR_h_4000Hz_869983200.GWF First data time in the first file of the run OUTPUT : VIR_h_4000Hz_869983200.SBL The first SBL file opened OUTPUT : VIR_h_4000Hz_869983200.SBL The first SBL file opened ! even NEW: a new FFT has started ! even NEW: a new FFT has started ! PAR1: Beginning time of the new FFT ! PAR1: Beginning time of the new FFT ! PAR2: FFT number in the run ! PAR2: FFT number in the run ! even EVT: time domain events ! even EVT: time domain events ! PAR1: Beginning time, in mjd ! PAR1: Beginning time, in mjd ! PAR2: Duration [s] ! PAR2: Duration [s] ! PAR3: Max hp data amplitude*EINSTEIN ! PAR3: Max hp data amplitude*EINSTEIN ! PAR4: Max CR ! PAR4: Max CR ! PAR5: Energy (sum of squared amp) ! PAR5: Energy (sum of squared amp) Log file info

11. ! even EVF: frequency domain events, with high threshold ! even EVF: frequency domain events, with high threshold ! PAR1: Beginning frequency of EVF ! PAR1: Beginning frequency of EVF ! PAR2: Duration [Hz] ! PAR2: Duration [Hz] ! PAR3: Ratio, in amplitude, max/average ! PAR3: Ratio, in amplitude, max/average ! PAR4: Power*EINSTEIN**2 or average*EINSTEIN (average if duration=0, when age>maxage) ! PAR4: Power*EINSTEIN**2 or average*EINSTEIN (average if duration=0, when age>maxage) ! par GEN: parameters of the AR spectrum estimation ! par GEN: parameters of the AR spectrum estimation (PAR) GEN_THR = 2.500000000 (PAR) GEN_THR = 2.500000000 (PAR) GEN_TAU = 0.020000000 (PAR) GEN_TAU = 0.020000000 (PAR) GEN_MAXAGE = 0.020000000 (PAR) GEN_MAXAGE = 0.020000000 (PAR) GEN_FAC = 2.000000000 (PAR) GEN_FAC = 2.000000000 ! GEN_THR is the threshold in amplitude ! GEN_THR is the threshold in amplitude ! GEN_TAU the memory frequency of the AR estimation ! GEN_TAU the memory frequency of the AR estimation ! GEN_MAXAGE [Hz] the max age of the process. If age>maxage the AR is re-evaluated ! GEN_MAXAGE [Hz] the max age of the process. If age>maxage the AR is re-evaluated ! GEN_FAC is the factor for which the threshold is multiplied, to write less EVF in the log file ! GEN_FAC is the factor for which the threshold is multiplied, to write less EVF in the log file

12. Log file info ! par GEN: general parameters of the run (PAR) GEN_BEG = 54313.249837963 (PAR) GEN_NSAM = 2097152.000 (PAR) GEN_DELTANU = 0.000953674 (PAR) GEN_FRINIT = 0 ! GEN_BEG is the beginning time (mjd) ! GEN_NSAM the number of samples in 1/2 FFT ! GEN_DELTANU the frequency resolution ! GEN_FRINIT the beginning frequency of the FFT (PAR) EVT_CR = 6 (PAR) EVT_TAU = 600 (PAR) EVT_DEADT = 1 (PAR) EVT_EDGE = 0.15 ! EVT_CR is the threshold ! EVT_TAU the memory time of the AR estimation ! EVT_DEADT the dead time [s] ! EVT_EDGE seconds purged around the event

13. (PAR) EVF_THR = 2.5 (PAR) EVF_THR = 2.5 (PAR) EVF_TAU = 0.02 (PAR) EVF_TAU = 0.02 (PAR) EVF_MAXAGE = 0.02 (PAR) EVF_MAXAGE = 0.02 (PAR) EVF_FAC = 2 (PAR) EVF_FAC = 2 ! EVF_THR is the threshold in amplitude ! EVF_THR is the threshold in amplitude ! EVF_TAU the memory frequency of the AR estimation ! EVF_TAU the memory frequency of the AR estimation ! EVF_MAXAGE [Hz] the max age of the process. If age>maxage the AR is re-evaluated ! EVF_MAXAGE [Hz] the max age of the process. If age>maxage the AR is re-evaluated ! EVF_FAC is the factor for which the threshold is multiplied, to write less EVF in the ! EVF_FAC is the factor for which the threshold is multiplied, to write less EVF in the Log file info

14. --> NEW > 54313.249837963 1 --> NEW > 54313.249837963 1 --> EVT > 54313.249837963 0.0055 3629.99 181.981 5.03206e+07 --> EVT > 54313.249837963 0.0055 3629.99 181.981 5.03206e+07 --> EVT > 54313.250697575 1.0575 241.162 11.2496 3.04115e+07 --> EVT > 54313.250697575 1.0575 241.162 11.2496 3.04115e+07 ……. ……. --> EVF > 1999.696731567 0.0038147 6.29069 292307 --> EVF > 1999.696731567 0.0038147 6.29069 292307 --> EVF > 1999.495506287 0.0038147 9.73132 714481 --> EVF > 1999.495506287 0.0038147 9.73132 714481 --> EVF > 1998.849868774 0.00286102 5.82867 452195 --> EVF > 1998.849868774 0.00286102 5.82867 452195 >>> TOT > 22629 >>> TOT > 22629 --> NEW > 54313.255906111 2 --> NEW > 54313.255906111 2 --> EVT > 54313.255906111 0.006 11499.6 248.579 4.91038e+08 --> EVT > 54313.255906111 0.006 11499.6 248.579 4.91038e+08 Log file info

15. Virgo data from T0=869983200 (2007-08-01-05:59:45) up to T=870537636 818 FFT (4.9637 days) 818 FFT (4.9637 days) EVT veto=21303/2 EVT veto=21303/2 (total time vetoed 2919.2/2 s) (total time vetoed 2919.2/2 s) EVF=245996 EVF=245996

16. From the log files: Time-events veto =8.895 10 -17  =5.95 10 -15

17. From the log files: Time-events veto one with double duration respect to the other. 4 events family

18. From the log files: Time-events veto

19. From the log files: Time-frequency plot

23. Frequency lines detected from: ( from Gabriele LineMonitor) (mag) 874444952 to 874589542 (green) 863680494 to 863831687

24. From the log files: Time-frequency plot Frequency lines detected from: ( from Gabriele LineMonitor) (mag) 874444952 to 874589542 (green) 863680494 to 863831687

25. From the log files: Time-frequency plot

26. From the log files: frequency lines hist.

28. From the log files: Amplitude-frequency.

29. From the log files: CR-frequency.

30. From the log files: CR

32. From the log files: Duration vs frequency red dot : EVF_MAXAGE = 0.02 Duration> EVF_MAXAGE = 0.02 Hz

33. From the short PS Time-frequency plot

35. TO BE DONE: To write the documentation To define with interested people : files dimension (open a new file after N FFT) Writing of the output files optional Lower the SNR thr …. Suggestions…