Practical Statistics for Particle Physicists Lecture 3 Harrison B. Prosper Florida State University European School of High-Energy Physics Anjou, France 6 – 19 June, ESHEP2012 Practical Statistics Harrison B. Prosper

Outline  Lecture 1  Descriptive Statistics  Probability  Likelihood  The Frequentist Approach – 1  Lecture 2  The Frequentist Approach – 2  The Bayesian Approach  Lecture 3 – Analysis Example 2

Practicum Toy data and code at topdiscovery.tar contactinteractions.tar just download and unpack 3

Recap: The Bayesian Approach – 1 Definition: A method is Bayesian if 1.it is based on the subjective interpretation of probability and 2.it uses Bayes’ theorem for all inferences. Dobserved data θparameter of interest ωnuisance parameters πprior density ESHEP2012 Practical Statistics Harrison B. Prosper4

Recap: The Bayesian Approach – 2 Nuisance parameters are removed by marginalization: in contrast to profiling, which can be thought of as marginalization with a δ-function prior ESHEP2012 Practical Statistics Harrison B. Prosper5

Recap: The Bayesian Approach – 3 Bayes theorem can be used to compute the probability of a model Dobserved data θ M parameters of model M Mmodel ωnuisance parameters πprior density ESHEP2012 Practical Statistics Harrison B. Prosper6

Bayesian Methods: Theory & Practice. Harrison B. Prosper7 Recap: The Bayesian Approach – 5 1.Factorize the priors:  (  , ω, M) =  (θ M, ω|M)  (M) 2.Then, for each model, M, compute the function 3.Then, compute the probability of each model, M

8 Recap: The Bayesian Approach – 6 In order to compute p(M|D), however, two things are needed: 1.Proper priors over the parameter spaces 2.The priors  (M). Usually, we compute the Bayes factor: which is the ratio in the first bracket, B 10.

An Analysis Example Search for Contact Interactions

In our current theories, all interactions proceed via the exchange of particles: But,… Contact Interactions – 1

Contact Interactions – 2 … when the experimentally available energies are << the mass of the exchanged particles, the interactions can be approximated as contact interactions (CI), for example:

Contact Interactions – 3 Consider the model* L ~ λ Ψγ μ Ψ Ψγ μ Ψ. with λ = ξ / Λ 2, where ξ can be positive or negative. At leading order, the possible reactions are: The amplitude is linear in λ a = a SM + λ a CI Eichten, Hinchliffe, Lane, Quigg, Rev. Mod. Phys. 56, 579 (1984)

Contact Interactions – 4 Contact interactions calculated at leading order with Pythia 6. QCD calculated at next-to-leading order with FastNLO. Expect cross section in each bin to be of the form

Bayesian Analysis

Simulated Data – 1 Data M = 25 bins (362 ≤ pT ≤ 2000) D = 575,999 to 0 (large dynamic range!) Parameters λ =parameter of interest nuisance parameters c = QCD cross section per p T bin b, a = signal parameters for destructive interference, b < 0 ESHEP2012 Practical Statistics Harrison B. Prosper15

Simulated Data – 2 Choose b < 0 Assume integrated luminosity of 5 fb α = 5000 pb -1

Analysis – 1 Step 1. Assume the following probability model for the observations where ESHEP2012 Practical Statistics Harrison B. Prosper17

Analysis Issues 1.Sensitive to jet energy scale (JES) 2.Sensitive to the parton distribution functions (PDF) 3.Large dynamic range causes the limits on Λ to be very sensitive to the value of α. For example, changing α from 5000 to 5030 decreases the limit by 25%! Solution: 1., 2. Integrate likelihood over JES and PDF parameters 3. Integrate likelihood over the scale factor α ESHEP2012 Practical Statistics Harrison B. Prosper18

Analysis – 2 Step 2: We can re-write as where ESHEP2012 Practical Statistics Harrison B. Prosper19 Exercise 11: Show this

Analysis – 3 We now eliminate α by integrating with respect to α. But to do so, we need a prior density for α. In the absence of reliable information about this parameter, we shall use which is an example of a reference prior. ESHEP2012 Practical Statistics Harrison B. Prosper20

Analysis – 4 Step 3: The integration with respect to α yields Step 4: Randomly sample from: 1.the jet energy scale, 2.jet energy resolution, 3.the PDF parameter sets, 4.the factorization an renormalization scales 5.and any other nuisance parameters of the problem This generates an ensemble of points {ω i } ESHEP2012 Practical Statistics Harrison B. Prosper21

Analysis – 5 Step 5: We approximate the posterior density using where, again, we use a reference prior for π (λ|ω). It turns out that this prior can be calculated exactly. ESHEP2012 Practical Statistics Harrison B. Prosper22

Analysis – 6 Step 6: Finally, we can compute a 95% Bayesian interval by solving for λ UP, from which we compute Λ = 1/√λ UP. For the simulated data (and ignoring Step 4., i.e., systematic uncertainties), we obtain Λ > % CL ESHEP2012 Practical Statistics Harrison B. Prosper23 Exercise 12: Write a program to implement this analysis

24 Summary Probability Two main interpretations: 1.Degree of belief 2.Relative frequency Likelihood Function Main ingredient in any non-trivial statistical analysis. Frequentist Principle Construct statements such that a given (minimum) fraction of them will be true over a given ensemble of statements.

Summary Frequentist Approach 1.Use likelihood function only 2.Eliminate nuisance parameters by profiling 3.Fisher: Reject null if p-value is judged to be too small 4.Neyman: Decide on a fixed threshold for rejection and reject null if threshold has been breached, but only if the probability of the alternative is high enough Bayesian Approach 1)Model all uncertainty using probabilities and use Bayes’ theorem to make inferences. 2)Eliminate nuisance parameters through marginalization. ESHEP2012 Practical Statistics Harrison B. Prosper25