1. Introduction to Power Spectrum Estimation Lloyd Knox (UC Davis) CCAPP, 23 June 2010

2. Goal of Talk Take someone who is starting from zero in power spectrum estimation to where they have some intuition for what the issues are, and they know where to go in the literature to begin estimating power spectra in practice.

3. Outline Motivating the use of the power spectrum Estimation Under Ideal Conditions Impact of various non-idealities Estimation under non-ideal conditions

4. Power Spectra: Useful for studying statistical properties of statistically homogeneous random fields Statistical homogeneity: statistical properties of the field are independent of location. Examples: CMB temperature maps, cosmic shear maps, galaxy number count maps*, … *cosmological evolution actually breaks homogeneity in radial direction, but one can study 2-D slices, or try to correct 3-D map for evolution

5. Power Spectra Examples Map at 150 GHz + Map at 220 GHz Random field(s) Power spectrum/spectra Data plus modeled contributions from four different statistically isotropic components (Hall et al. 2010) Nolta et al. (2009)

6. Power Spectrum Example Random field Power spectrum  T( ,  ) =  lm a lm Y lm ( ,  ) C l  ll’  mm’ = Power spectrum } Consequence of statistical homogeneity (isotropy in this case) = ensemble average Nolta et al. (2009)

7. Power Spectrum Interpretation  T( ,  ) =  lm a lm Y lm ( ,  ) C(  ) = =  l (l+1/2)/(2  ) C l P l (cos(  ))  2 = C(0) =  l (l+1/2)/(2  ) C l = s d(lnl) l(l+1/2)C l /(2  ) } Contribution to variance from a logarithmic interval in l C l = Nolta et al. (2009)

8. Why is the power spectrum useful? For Gaussian homogeneous random fields, it captures all the information not in the mean. Even for non-Gaussian fields, it can be a highly informative statistic. There will be additional information in other statistics, but the power spectrum is usually a sensible place to start.

9. Why C l Instead of the Correlation Function, C(  )? They are linear transformations of each other, carrying the same information. For Gaussian fields, the covariance structure of power spectrum estimates is much simpler. For linear perturbation theory, time evolution of a single Fourier mode is simple and decoupled from other modes ==> simple physical interpretation of the power spectrum. Nonlinearity of evolution, and/or non-Gaussianity, weakens these two advantages. C(  ) =  l (l+1/2)/(2  ) C l P l (cos(  ))

10. PS Estimation: Simplest Case of Uniform Full-sky Coverage with no noise a lm = s d   T Y lm a lm = a lm s signal C l =  m |a lm | 2 /(2l+1) Each a lm provides an unbiased estimate of C l. For each l there are 2l+1 values of m so we can average them all together to get ^ This is both the minimum-variance and maximum-likelihood estmator. Note that despite no noise, there is uncertainty in the true value of C l = 2/(2l+1)(C l ) 2 ^

11. PS Estimation: Uniform Full-sky Coverage With Noise a lm = s d   T Y lm a lm = a lm s + a lm n signal noise If noise is uncorrelated from pixel to pixel and homogeneous, then = w -1 where w is the statistical weight per solid angle, w = (1/  2 pix )/  pix, and this “noise bias” needs to be subtracted from our estmate: C l =  m |a lm | 2 /(2l+1) - w -1 = 2/(2l+1)(C s l +w -1 ) 2 ^ ^

12. PS Estimation: Uniform Full-sky Coverage With Noise and Finite Resolution a lm = s d   T Y lm a lm = a lm s + a lm n signal noise Convolution of the sky signal with the response function of the telescope, B( ,  ), is a multiplication in the spherical harmonic domain by B l = s d  Y l0 B( ,  ). We need to compensate by dividing the map a lm by B l so that C l =  m |a lm | 2 /(2l+1)B l -2 -B l -2 w -1 = 2/(2l+1)(C s l +B l -2 w -1 ) 2 ^ ^

13. WMAP Power Spectrum Errors 1/2 = [2/(2l+1)] 1/2 (C s l +B l -2 w -1 ) Few samples per l value; i.e., [2/(2l+1)] factor large Beam-deconvolved noise large

14. PS Estimation with Partial Sky Coverage, Finite Resolution and Inhomogeneous Correlated Noise One approach: Optimal methods (ssuming Gaussian random field) P(  T | C l ) \propto M -1/2 exp(-  T i M -1 ij  T j /2) with M ij = S ij (C l ) + N ij By Bayes’ Theorem P(C l |  T) \propto P(  T | C l ) \propto M -1/2 exp(-  T i M -1 ij  T j /2) But calculation is computationally intractable for maps greater than tens to hundreds of thousands of pixels Quadratic estimator, likelihood approximations, Gibbs sampling + Blackwell-Rao Estimator (see references at end)

15. PS Estimation with Partial Sky Coverage, Finite Resolution and Inhomogeneous Correlated Noise Another approach: Pseudo-C l methods Sub-optimal, but good enough and fast Basic idea is to use the simple estimator, and then a combination of analytic and Monte Carlo methods to estimate the offset and gain relating the simple estimator (the pseudo-C l ) and the real C l.

16. Pseudo-C l Random field Power spectrum  T( ,  ) =  lm a lm Y lm ( ,  ) a lm = s d  Y lm ( ,  ) [W( ,  )  T( ,  )] W = mask that’s zero in galactic plane, and smoothly goes to one outside of it Multiplication in real space is convolution in Fourier space ~ a lm will have contributions from a l’m’ for l’ near l

17. Pseudo-C l That convolution has an analytically calculable effect on the ensemble average of the pseudo-C l =  l’ M ll’ B l’ 2 C l’ + ~~ Effect of mask Beam Noise bias Noise bias can be calculated via noise-only Monte-Carlo simulations Estimate C l by subtracting noise-bias and then deconvolving. Estimate C l errors by noise + signal Monte-Carlo simulation

18. Eliminating Noise Bias =  l’ M ll’ B l’ 2 C l’ + ~ ~ Form a lm from two different maps, each with noise, but noise that is not correlated from one map to the next. Reduces sensitivity to knowing noise level imperfectly. ~ 0

19. Zoom in on 2 mm map ~ 4 deg 2 of actual SPT data In addition to large-scale masks (due to partial sky coverage, or the galaxy) need to mask point sources too!

20. Lots of bright emissive sources ~15-sigma SZ cluster detection All these “large-scale” fluctuations are primary CMB. Zoom in on 2 mm map ~ 4 deg 2 of actual data

21. Point-source Masking  T( ,  ) =  lm a lm Y lm ( ,  ) a lm = s d  Y lm ( ,  ) [W( ,  )  T( ,  )] W = mask that’s zero near a point source and smoothly goes to 1 away from point source Multiplication in real space is convolution in Fourier space ~ If mask is over very small area, a lm will have contributions from a l’m’ for l’ far from l The resulting transfer of power over large  l can cause problems. Use fat masks (very simple) or prewhiten your data

22. References Quadratic estimator: Bond, Jaffe & Knox (1998) Approximate likelihoods: Bond, Jaffe & Knox (2000), Verde et al. (2003) Gibbs sampling: Wandelt et al. (2004), Eriksen et al. (2004), Chu et al. (2005) Pseudo-C l method: Hivon et al. (2002) Point Source Masking and Pre-whitening: Das, Hajian & Spergel (2009)

23. Summary The power spectrum is a very useful summary statistic for comparing data with theory. Optimal estimation, assuming Gaussianity, is difficult and for most applications (not all) it is also pointless. Approximate and fast schemes exist that handle a variety of non-idealities -- in principle, via Monte Carlo, can handle them all.

24. Power Spectra Examples Nine shear maps: 8 from galaxies in eight photometric redshift bins and one reconstructed from the CMB Song & Knox 2003 Some of the 9*(9+1)/2 = 45 power spectra Random fields 0.2-0.2 3.0-3.0 1100-1100