1. Network analysis and statistical issues Lucio Baggio An introductive seminar to ICRR’s GW group

2. Topics of this presentation Setting confidence intervals False discovery probability Gravitational wave bursts networks From the single detector to a worldwide network IGEC (International GW Collaboration) Long-term search with four detectors; directional search and statistical issues From raw data to probability statements; likelihood/Byesian vs frequentist methods Multiple tests and large surveys change the overall confidence of the first detection Miscellaneous topics The LIGO-AURIGA white paper on joint data analysis;problems with non-aligned or different detectors; coherent data analysis.

3. Network analysis is unavodable, as far as background estimation is concerned

4. Gravitational wave burst events For fast (~1 ÷ 10ms) gw signals the impulse response of the optimal filter for the signal amplitude is an exponentially damped oscillation Even at a very low amplitude the signals from astrophysical sources are expected to be rare. A candidate event in the gravitational wave channel is any single extreme value in a more or less constant time window. Background events come from the extreme distribution for an (almost) Gaussian stochastic process

5. Amplitude distribution of events AURIGA, Jun 12-21 1997 The background in practice (1) vetoed (  2 test) simulation (gaussian) L. Baggio et al.  2 testing of optimal filters for gravitational wave signals: an experimental implementation. Phys. Rev. D, 61:102001–9, 2000

6. Amplitude distribution of events AURIGA Nov. 13-14, 2004 Remaining events after vetoing vetoed glitches epoch vetoes (50% of time) cumulative event rate above threshold false alarm rate [hour -1 ] after vetoing The background in practice (2)

7. Cumulative power distribution of events TAMA Nov. 13-14, 2004 from the presentation at The 9th Gravitational Wave Data Analysis Workshop (December 15-18, 2004, Annecy, France) The background in practice (3) DT9 DT8 DT6 DT9 (before veto)

8. 8 The background in practice (4) Environmental Monitoring Try to eliminate locally all possible false signals Detectors for many possible sources (seismic, acoustic, electromagnetic, muon) Also trend (slowly-varying) information (tilts, temperature, weather) Matched filter techniques for `known' signals  this can only decrease background (no confidece for not matched signal) but not increase the (unknown) confidence for remaining signals. Non-coeherent methods coincidences among detectors (also non-GW: e.g., optical, g-ray, X-ray, neutrino) Coeherent methods Correlations Maximum likelihood (e.g.: weighted average) Two good reasons for multiple detector analysis 1.the rate of background candidates can be estimated reliably 2.the background rate of the network can be less than that of the single detector

9. M-fold coincidence search A coincidence is defined as a multiple detection on many detectors of triggers with estimated time of arrival so close that there is a common overlap between their time windows t w. The latter are defined by the estimated timing error. The ideal “off-source” measure of the background cannot be truly performed (no way to shield the detector). The surrogate solution consists in computing coincidence search after proper delays dt k (greater than the timing errors) have been applied to event series. Then, the coincidences due to real signals disappear, and only background coincidences are left.

10. M-fold coincidence search (2) The expected coincidence rate is given by: C(t) depends on the choice of the the time error boxes: equal and constant vary with detector vary with event Monte Carlo (by shifted times resampled statistics) From IGEC 1997-2000: example of predicted mean false alarm rates. Notice the dramatic improvement when adding a third detector: the occurrence of a 3-fold coincidence would be interpreted inevitably as a gravitational wave signal. In practice, when no signal is detected in coincidence, the upper limit is determined by the total observation time

11. International networks of GW detectors Interferometers Operative: GEO600 – (Germany/UK) LIGO Hanford 2km – (USA) LIGO Hanford 4km – (USA) LIGO Livingstone 4km – (USA) TAMA300 – (Japan) Upcoming: VIRGO – (Italy/France) CLIO – (Japan) Resonant bars ALLEGRO – (USA) AURIGA – (Italy) EXPLORER – (CERN, Geneva) NAUTILUS – (Italy) LIGO GEO600Virgo, AURIGA, NAUTLUS TAMA300 CLIO100 EXPLORER

12. International networks of GW detectors 15 years of worldwide networks 1989 – 2 bars, 3 months E. Amaldi et al., Astron. Astrophys. 216, 325 (1989). 1991 – 2 bars, 120 days P. Astone et al., Phys. Rev. D 59, 122001 (1999). 1995-1996 – 2 detectors, 6 months P. Astone et al., Astropart. Phys. 10, 83 (1999). 1989 – 2 interferometers, 2 days D. Nicholson et al., Phys. Lett. A 218, 175 (1996). 1997-2000 – 2, 3, 4 resonant detectors, resp. 2 years, 6 months, 1 month P. Astone et al., Phys. Rev. D 68, 022001 (2003). 2001 – 2 detectors, 11 days TAMA300-LISM collaboration (2004) Phys. Rev. D 70, 042003 (2004) 2001 – 2 detectors, 90 days P. Astone et al., Class. Quant. Grav 19, 5449 (2002). 2002 – 3 detectors, 17 days LIGO collaboration B. Abbott et al., Phys. Rev. D 69, 102001 (2004) 1969 -- Argonne National Laboratory and at the University of Maryland J. Weber, Phys. Rev. Lett. 22, 1320–1324 (1969) 1973-1974 – Phys. Rev. D 14, 893-906 (1976) GW detected? If NOT, why?

13. The International Gravitational Event Collaboration

14. http:// igec.lnl.infn.it LSU group:ALLEGRO (LSU) http://gravity.phys.lsu.edu Louisiana State University, Baton Rouge - Louisiana AURIGA group:AURIGA (INFN-LNL) http://www.auriga.lnl.infn.it INFN of Padova, Trento, Ferrara, Firenze, LNL Universities of Padova, Trento, Ferrara, Firenze IFN- CNR, Trento – Italia NIOBE group:NIOBE (UWA) http://www.gravity.pd.uwa.edu.au University of Western Australia, Perth, Australia ROG group:EXPLORER (CERN) http://www.roma1.infn.it/rog/rogmain.html NAUTILUS (INFN-LNF) INFN of Roma and LNF Universities of Roma, L’Aquila CNR IFSI and IESS, Roma - Italia The International Gravitational Event Collaboration

15. The IGEC protocol The source of IGEC data are different data analysis applied to individual detector outputs. The IGEC members are only asked to follow a few general guidelines in order to characterize in a consistent way the parameters of the candidate events and the detector status at any time. Further data conditioning and background estimation are performed in a coordinated way

16. Exchanged periods of observation 1997-2000 fraction of time in monthly bins exchange threshold ALLEGRO AURIGA NAUTILUS EXPLORER NIOBE Fourier amplitude of burst gw arrival time

17. The exchanged data time (hours) gaps minimum detectable amplitude (aka exchange threshold) events amplitude and time of arrival amplitude (Hz -1 ·10 -21 )

18. M-fold coincidence search (revised) A coincidence is defined when for all 0<i,j<M  t i – t j  <  t ij ~0.1 sec Coincidence windows  t ij depend on timing error, which is  non-gaussian at low SNR !  < 5% false dismissal for k =4.5 (Tchebyceff inequality)  strongly dependent on SNR ! To make things even worse, we would like the sequence of event times to be described by a (possibly non-homogeneous) Poisson point series, which means rare and independent triggers, but this was not the case.

19. Timing error uncertainty (AURIGA, for  -like bursts )

20. Auto- and cross-correlation of time series (clustering)  Auto-correlation of time of arrival on timescales ~100s No cross-correlation AL = ALLEGRO AU = AURIGA EX = EXPLORER NA = NAUTILUS NI = NIOBE x-axis: seconds y-axis: counts

21. Amplitude distributions of exchanged events normalized to each detector threshold for trigger search  typical trigger search thresholds: SNR  3 ALLEGRO, NIOBE SNR  5 AURIGA, EXPLORER, NAUTILUS The amplitude range is much wider than expected extreme distribution: non modeled outliers dominate at high SNR

22. False alarm reduction by amplitude selection Corollary: Selected events have naturally consistent amplitudes With a small increase of minimum amplitude, the false alarm rate drops dramatically.

23. amplitude directional sensitivity time (hours) amplitude (Hz -1 ·10 -21 ) time (hours) Sensitivity modulation for directional search amplitude (Hz -1 ·10 -21 )

24. A small digression: different antenna patterns and the relevance of signal polarization

25. In order to reconstruct the wave amplitude h, any amplitude has to be divided by Introduction At any given time, the antenna pattern is:  it is a sinusoidal function of polarization , i.e. any gravitational wave detector is a linear polarizer  it depends on declination and right ascension  through the magnitude A and the phase  We will characterize the directional sensitivity of a detector pair by the product of their antenna patterns F 1 and F 2  F 1 F 2 is inversely proportional to the square of wave amplitude h 2 in a cross- correlation search  F 1 F 2 is an “extension” of the “AND” logic of IGEC 2-fold coincidence  This has been extensively used by IGEC: first step is a data selection obtained by putting a threshold  F -1 on each detector

26. For linearly polarized signal,  does not vary with time. The product of antenna pattern as a function of  is given by:               The relative phase  1 -  2 between detectors affects the sensitivity of the pair. Linearly polarized signals

27. AURIGA -TAMA sky coverage: (1) linearly polarized signal AURIGA 2 TAMA 2 AURIGA x TAMA

28. If:  the signal is circularly polarized:  Amplitude h(t) is varying on timescales longer than 1/f 0 Then:  The measured amplitude is simply h(t), therefore it depends only on the magnitude of the antenna patterns. In case of two detectors:  The effect of relative phase  1 -  2 is limited to a spurious time shift  t which adds to the light-speed delay of propagation: (Gursel and Tinto, Phys Rev D 40, 12 (1989) ) Circularly polarized signals 

29. AURIGA 2 TAMA 2 AURIGA -TAMA sky coverage: (2) circularly polarized signal AURIGA x TAMA

30. AURIGA -TAMA sky coverage Linearly polarized signal Circularly polarized signal

31. IGEC (continued)

32. time (hours) Data selection at work Duty time is shortened at each detector in order to have efficiency at least 50% A major false alarm reduction is achieved by excluding low amplitude events. amplitude (Hz -1 ·10 -21 )

33. amplitude of burst gw Duty cycle cut: single detectors total time when exchange threshold has been lower than gw amplitude

34. Duty cycle cut: network (1) Galactic Center coverage

35. Duty cycle cut: network (2) search threshold 6  10 -21 /Hz search threshold 3  10 -21 /Hz

36. False dismissal probability data conditioning. The common search threshold H t guarantees that no gw signal in the selected data are lost because of poor network setup. …however the efficiency of detection is still undetermined (depends on distribution of signal amplitude, direction, polarization) Best choice for 1997-2000 data: false dismissal in time coincidence less than 5%  30% no amplitude consistency test time coincidence constraint The Tchebyscheff inequality provides a robust (with respect to timing error statistics) and general method to limit conservatively the false dismissal false alarms  k amplitude consistency check: gw generates events with correlated amplitudes testing (same as above) A coincidence can be missed because of… fraction of found gw coincidences fluctuations of accidental background  When optimizing the (partial) efficiency of detection versus false alarms, we are lead to maximize the ratio

37. time (hours) Resampling statistics by time shifts amplitude (Hz -1 ·10 -21 ) We can approximately resample the stochastic process by time shift. The in the resampled data the gw sources are off, along with any correlated noise Ergodicity holds at least up to timescales of the order of one hour. The samples are independent as long as the shift is longer than the maximum time window for coincidence search (few seconds)

38. Poisson statistics For each couple of detectors and amplitude selection, the resampled statistics allows to test Poisson hypothesis for accidental coincidences.   Example: EX-NA background (one-tail  2 p-level 0.71) As for all two-fold combinations a fairly big number of tests are performed, the overall agreement of the histogram of p-levels with uniform distribution says the last word on the goodness-of-the-fit. verified

39. Setting (frequentist) confidence intervals

40. Unified vs flip-flop approach (1) experimental data physical results hypothesis testing (CL) x upper limit estimation (with error bars)  (x)  k CL    up (CL) Flip-flop method null claim

41. Unified vs flip-flop approach (2) experimental data physical results confidence belt x estimation (with confidence interval) Unified approach  min (CL)  max (CL)

42. Setting confidence intervals IGEC approach is Frequentist in that it computes the confidence level or coverage as the probability that the confidence interval contains the true value Unified in that it prescribes how to set a confidence interval automatically leading to a gw detection claim or an upper limit however, different from F&C References G.J.Feldman and R.D.Cousins, Phys. Rev. D 57 (1998) 3873 B. Roe and M. Woodroofe, Phys. Rev. D 63 (2001) 013009 F. Porter, Nucl. Inst. Meth. A368 (1986), http://www.cithep.caltech.edu/~fcp/statistics/ Particle Data Group: http://pdg.lbl.gov/2002/statrpp.pdf

43. A few basics: confidence belts and coverage experimental data physical unknown

44. A few basics (2) experimental data physical unknown confidence interval coverage For each outcome x one should be able to determine a confidence interval I x For each possible , the measures which lead to a confidence interval consistent with the true value have probability C(  ), i.e. 1-C(  ) is the false dismissal probability

45. I  can be chosen arbitrarily within this “horizontal” constraint Feldman & Cousins (1998) and variations (Giunti 1999, Roe & Woodroofe 1999,...) Freedom of choice of confidence belt Fixed frequentistic coverage Maximization of “likelyhood” I x can be chosen arbitrarily within this “vertical” constraint Roe & Woodroofe [2000]: a Bayesian inspired frequentistic approach Fine tune of the false discovery probability Non-unified approaches Other requirements...

46. Confidence level, likelyhood, maybe probability? The term “CL” is often found associated with equations like In general the bounds obtained as a solution to these equations have a coverage (or confidence level) different from “CL” likelihood integral likelihood ratio relative to the maximum likelihood ratio (hipothesis testing)

47. Confidence intervals from likelihood integral I fixed, solve for : Compute the coverage Let Poisson pdf: Likelihood:

48. Example: Poisson background N b = 7.0 99% 95% 85% N Likelihood integral

49. Dependence of the coverage from the background N b =0.01-0.1-1.0-3.0-7.0-10 likelihood integral = 0.90

50. From likelihood integral to coverage Plot of the likelihood integral vs. minimum (conservative) coverage min  C(  ), for sample values of the background counts N b, spanning the range N b =0.01-10

51. IGEC results (and what we learned from experience)

52. Setting confidence intervals on IGEC results Example: confidence interval with coverage  95% HtHt “upper limit”: true value outside with coverage  95% GOAL: estimate the number (rate) of gw detected with amplitude  H t

53. Uninterpreted upper limits …on RATE of BURST GW from the GALACTIC CENTER DIRECTION whose measured amplitude is greater than the search threshold no model is assumed for the sources, apart from being a random time series ensured minimum coverage true rate value is under the curves with a probability = coverage search threshold (Hz -1 ) rate (year –1 )

54. Upper limits after amplitude selection systematic search on thresholds many trials ! all upper limits but one: overall false alarm probability 33% at least one detection in case NO GW are in the data NULL HYPOTHESIS WELL IN AGREEMENT WITH THE OBSERVATIONS

55. Multiple configurations/selection/grouping within IGEC analysis

56. Resampling statistics of accidental claims event time series coverage“claims” 0.900.866 (0.555) [1] 0.950.404 (0.326) [1] expected found Resampling  blind analysis!

57. False discovery rate: setting the probability of false claim of detection

58. Why FDR? When should I care of multiple test procedures?. All sky surveys: many source directions and polarizations are tried Template banks Wide-open-eyes searches: many analysis pipelines are tried altogether, with different amplitude thresholds, signal durations, and so on Periodic updates of results: every new science run is a chance for a “discovery”. “Maybe next one is the good one”. Many graphical representations or aggregations of the data: “If I change the binning, maybe the signal shows up better…

59. Preliminary (1) : hypothesis testing False discoveries (false positives) Detected signals (true positives) Reported signal candidates inefficiency Null Retained (can’t reject) Reject= Reject Null = Accept Accept Alternative Total Null (H o ) True Background (noise) U B Type I Error α = ε b momo Alternative True signal Type II Error β = 1- ε s T Sm1m1 m-R R = S+B m

60. Preliminary (2): p-level Assume you have a model for the noise that affects the measure x. However, for our purposes it is sufficient assuming that the signal can be distinguished from the noise, i.e. dP/dp  1. Typically, the measured values of p are biased toward 0. signal You derive a test statistics t(x) from x. F(t) is the distribution of t when x is sampled from noise only (off-signal). The p-level associated with t(x) is the value of the distribution of t in t(x): p = F(t) = P(t>t(x)) Example:  2 test  p is the “one-tail”  2 probability associated with n counts (assuming d degrees of freedom) Usually, the alternative hypothesis is not known. p-level 1 background pdf The distribution of p is always linearly raising in case of agreement of the noise with the model P(p)=p  dP/dp = 1

61. Usual multiple testing procedures For each hypothesis test, the condition {p<   reject null} leads to false positives with a probability  In case of multiple tests (need not to be the same test statistics, nor the same tested null hypothesis), let p={p 1, p 2, … p m } be the set of p-levels. m is the trial factor. We select “discoveries” using a threshold T(p): {p j <T(p)  reject null}. Uncorrected testing: T(p)=  –The probability that at least one rejection is wrong is P(B>0) = 1 – (1-  ) m ~ m  hence false discovery is guaranteed for m large enough Fixed total 1 st type errors (Bonferroni): T(p)=  /m –Controls familywise error rate in the most stringent manner: P(B>0) =  –This makes mistakes rare… –… but in the end efficiency (2 nd type errors) becomes negligible!!

62. p S pdf m0m0 Let us make a simple case when signals are easily separable (e.g. high SNR) Controlling false discovery fraction We desire to control (=bound) the ratio of false discoveries over the total number of claims: B/R = B/(B+S)  q. The level T(p) is then chosen accordingly. B m q p B S cumulative counts R

63. Benjamini & Hochberg FDR control procedure Among the procedures that accomplish this task, one simple recipe was proposed by Benjamini & Hochberg (JRSS-B (1995) 57:289-300 ) choose your desired FDR q (don’t ask too much!); define c(m)=1 if p-values are independent or positively correlated; otherwise c(m)=Sum j (1/j) compute p-values {p 1, p 2, … p m } for a set of tests, and sort them in creasing order; p m determine the threshold T(p)= p k by finding the index k such that p j k; reject H 0 q/c(m)

64. LIGO – AURIGA: coincidence vs correlation

65. LIGO-AURIGA MoU A working group for the joint burst search in LIGO and AURIGA has been formed, with the purpose to: » develop methodologies for bar/interferometer searches, to be tested on real data » time coincidence, triggered based search on a 2-week coincidence period (Dec 24, 2003 – Jan 9, 2004) » explore coherent methods ‘best’ single-sided PSD Simulations and methodological studies are in progress.

66. White paper on joint analysis Two methods will be explored in parallel: Method 1: IGEC style, but with a new definition of consistent amplitude estimator in order to face the radically different spectral densities of the two kind of detectors (interferometers and bars). To fully exploit IGEC philosophy, as the detectors are not parallel, polarization effects should be taken into account (multiple trials on polarization grid). Method 2: No assumptions are made on direction or waveform. A CorrPower search (see poster) is applied to the LIGO interferometers around the time of the AURIGA triggers. Efficiency for classes of waveforms and source population is performed through Monte Carlo simulation, LIGO-style (see talks by Zweizig, Yakushin, Klimenko). The accidental rate (background) is obtained with unphysical time-shifts between data streams.

67. Summary of non-directional “IGEC style” coincidence search detector 1 detector 2 AND detector 3  Detectors: PARALLEL, BARS  S hh : SIMILAR FREQUENCY RANGE  Search: NON DIRECTIONAL  Template: BURST =  (t) The search coincidence is performed in a subset of the data such that:  the efficiency is at least 50% above the threshold (H S )  significant false alarm reduction is accomplished The number of detectors in coincidence considered is self-adapting This strategy can be made directional HSHS

68. Cross-correlation search (naïve)  Detectors: PARALLEL  S hh : SAME FREQUENCY RANGE NEEDED  Search: NON DIRECTIONAL  Template: NO Selection based on data quality can be implemented before cross-correlating. The efficiency is to be determined a posteriori using Monte Carlo. The information which is usually included in cross-correlation takes into account statistical properties of the data streams but not geometrical ones, as those related to antenna patterns. detector 1 detector 2 detector 1 * detector 2 Threshold crossing after correlation T

69. Comparison between “IGEC style” and cross-correlation IGEC style search was designed for template searches. The template guarantees that it is possible to have consistent estimators of signal amplitude and arrival time. A bank of templates may be required to cover different class of signals. Anyway in burst search we don’t know how well the template fits the signal A template-less IGEC style search can be easily implemented in case of detectors with equal detector bandwidth. In fact it is possible to define a consistent amplitude estimator. (Karhunen-Loeve, power…) Cross-correlation among identical detectors is the most used method to cope with lack of templates. Cross-correlation in general is not efficient with non-overlapping frequency bandwidths, even for wide band signals. Some more work is needed to extend IGEC in case of template-less search among (spectrally) different detectors. Hint: the amplitude estimators should have spectral weights common to all detectors, to be consistent without a template. The trade-off will be between between efficiency loss and network gain (sky coverage and false alarm rate) Template search Template-less search IGEC cross- corr IGEC cross- corr IGEC