1. Why do Wouter (and ATLAS) put asymmetric errors on data points ? What is involved in the CLs exclusion method and what do the colours/lines mean ? ATLAS J/ Ψ peak (muons) Excluding SM Higgs masses LEP exclusion Tevatron exclusion

2. Why do you put an error on a data-point anyway ? ATLAS J/ Ψ peak (muons) Estimate of underlying truth (model value)

3. Poisson distribution Probability to observe n events when λ are expected λ=4.90 Number of observed events #observed Lambda hypothesis fixed varying

4. Poisson distribution: properties Poisson distribution properties  the famous √N (1) Mean: (2) Variance: (3) Most likely value: first integer ≤ λ http://www.nikhef.nl/~ivov/Statistics/Poisson.pdf

5. Lambda known  expected # events λ=0.00λ=1.00 λ=4.90 λ=5.00

6. Large number of events λ=40.0 Unfortunately this is not what you wanted to know … What you have: What you want:

7. From data to theory Likelihood: Poisson distribution “what can I say about the measurement (Number of observed events) given an expectation from an underlying theory ?” This is what you want to know: “what can I say about the underlying theory given my observation of a given number of events ?”

8. λ (hypothesis) N obs known (4)  information on lambda “Given a number of observed events (4):  what is the most likely / average / mean underlying true vanue of λ ?” #observed Lambda hypothesis fixed P(N obs =4|λ) varying Normally you plot -2log(Likelihood) Likelihood:

9. Properties of P(λ|N) for flat P(λ) properties (1) Mean: (2) Variance: (3) Most likely value: λ most likely = x http://www.nikhef.nl/~ivov/Statistics/Poisson.pdf Assuming P(λ) is flat

10. This is normally presented as likelihood curve λ (hypothesis) P(N obs =4|λ) -2Log(P(N obs =4|λ)) Likelihood -2Log(Prob) 4.00 2.32 -1.68 6.35 +2.35 sigma: ΔL=+1 ΔL=+1 68.4% Pdf for λ

11. ATLAS J/ Ψ peak (muons) So, if you have observed 4 events your best estimate for λ is … :

12. CL S method http://www.nikhef.nl/~ivov/Statistics/thesis_I_v_Vulpen.pdf Chapter 7.4

13. Your Higgs analysis Discriminant variable Higgs SM Hebben we nou de Higgs gezien of niet ? Higgs SM SM+Higgs Scaled to correct cross-sections and 100 pb-1 Can also be an invariant mass plot

14. Approach 1: counting Discriminant variable tellen Experiment 1 Experiment 2 Origin# events SM12.2 Higgs5.1 MC total17.3 Data11 Origin# events SM12.2 Higgs5.1 MC total17.3 Data17

15. Expectations If the Higgs is NOT there: On average 12.2 events If the Higgs is there: On average 17.2 events Experiment 2: 17 events observed Experiment 1: 11 events observed SMSM + Higgs

16. Discovery - Only look at what you expect from Standard Model background - Given the SM expectation: if probability to observe as many events you have observed (or more) is smaller than 5.7 10 -7  SM hypothesis is very unlikely  reject SM  discovery ! - Only look at what you expect from Standard Model background - Given the SM expectation: if probability to observe as many events you have observed (or more) is smaller than 5.7 10 -7  SM hypothesis is very unlikely  reject SM  discovery !

17. Test hypotheses: rules for discovery In the hypothesis that there is NO Higgs (SM hypothesis): What is the probability to observe as many events as I have observed …OR EVEN MORE If P < 5.7 10 -7  reject SM P(N≥33|12.2) = 6.35 10 -7 P(N≥34|12.2) = 2.24 10 -7 P(N≥33|12.2) = 6.35 10 -7 P(N≥34|12.2) = 2.24 10 -7 Integrate this plot SM + HiggsSM

18. Question 1: did you make a discovery ? See previous slide: Yes DiscoveryNo discovery No

19. Question 2: did you expect to make a discovery: If the Higgs is NOT there: On average 12.2 events If the Higgs is there: On average 17.2 events If you observe exactly the number of events you expect (assuming the Higgs is there), it is not unlikely enough to be explained by the SM  NO discovery expected SMSM + Higgs

20. Question 3: At what luminosity do you expect to make a discovery ? Lumi x 1 N SM = 12.2 N Higgs = 5.1 N SM = 122.0 N Higgs = 51.0 Lumi x 10 Lumi x 12.5 N SM = 152.5 N Higgs = 63.75 no yes SM + Higgs SM

21. Discovery or not It is not likely you get exactly the number of events you expect.  You can be lucky … or unlucky.

22. From simple counting to the real thing in 3 steps 1) Introduce X (Likelihood ratio) test statistic 2) From simple counting to weighted counting (a real analysis) 3) Toy Monte-Carlo (fake experiments)

23. From simple counting to the real thing in 3 steps 1) Introduce X (Likelihood ratio) test statistic 2) From simple counting to weighted counting (a real analysis) 3) Toy Monte-Carlo (fake experiments)

24. Hypothesis testing: likelihood ratio  frequently used: X=-2ln(Q) Hypothesis 1: the Standard Model without the Higgs boson Hypothesis 2: the Standard Model with the Higgs boson Definieer een statistic (= variabele) die onderscheid maakt tussen de 2 hypotheses. Note: kan vanalles zijn: # events of Neural net output. Ex: counting experiment Likelihood ratio

25. Likelihood ratio: counting Counting experiment N events left after some a selection of cut on discriminant Note: X = 0 means hypoteses equally likely Used in plots: More SM+Higgs likeMore SM like 100.000 SM experiments 100.000 SM + Higgs experiments 14 events observed Variabele transformatie

26. Likelihood ratio: counting Counting experiment N events left after some a selection of cut on discriminant Note: X = 0 means hypoteses equally likely Used in plots: More SM+Higgs likeMore SM like 100.000 SM experiments 100.000 SM + Higgs experiments 14 events observed 15 events observed

27. From simple counting to the real thing in 3 steps 1) Introduce X (Likelihood ratio) test statistic 2) From simple counting to weighted counting (a real analysis) 3) Toy Monte-Carlo (fake experiments)

28. Likelihood ratio Counting experimentWeighted counting experiment Eveny event has a weight according to a NN output or discriminant called p i : Signal: S(p i ) and Background B(p i ) B(p i ) S(p i )+B(p i ) N events left after some a selection of cut on discriminant tellen

29. From simple counting to the real thing in 3 steps 1) Introduce X (Likelihood ratio) test statistic 2) From simple counting to weighted counting (a real analysis) 3) Toy Monte-Carlo (fake experiments)

30. Many possible experiments Discriminant variable tellen Experiment 1 Experiment 2 1) Experiment condensed in 1 variable Note: Each experiment (read ATLAS) yields only ONE value of Q see 2 slides ago for counting example 2) Do Toy-MC experiments to study distribution of Q Note: Two distributions: for SM and SM+Higgs hypothesis

31. Toy Monte Carlo experiment SM toy experiment: Draw for each bin i a random number from Poisson with μ= λSM (i) SM+Higgs toy experiment: Draw for each bin i a random number from Poisson with μ= λSM(i)+ λSM+Higgs(i) λSM(i)+ λSM+Higgs(i) λSM(i)

32. The Higgs does not exist: 100,000 toy-experiments (SM) The Higgs exists: 100,000 toy-experiments (SM+Higgs)

33. With 1 and 2 sigma bands for SM hypothesis Note (again): each experiment will produce 1 (one) number in this plot

34. Different masses … different cross-sections Small Higgs cross-section Large Higgs cross-section Two hypotheses are more apart if: 1) cross-section of Higgs is larger 2) Higgs is more different from SM

35. LEP plots LEP paper Fig 1 Cross-section drops as function of mass dummy

36. Expectation for Q or -2ln (Q): toy experiments Probability that background results in the numer observed or (even) more If 1-CL b < 5.7 10 -7 we can say we reject the SM hypothesis  discovery ! The famous 5 sigma Probability that background results in the numer observed or less Cl b = confidence level in the background SM SM+Higgs

37. Discovery

38. Do you expect to discover Higgs with at this mass ? Average SM+Higgs experiment: 1-CLb = 2 10^-7 So yes, you expect to make a discovery IF 10xSM

39. The one 2-sigma is not the other 2-sigma 2.X sigma discrepancy at mh ~ 97 GeV Far away form what you expect from Higgs 1.X sigma away at mh = 114 GeV Exactly what you expect from Higgs No 5 sigma discovery  what Higgs hypotheses can we reject

40. No discovery No 5 sigma deviation found … what now ? Trying to say something on the hypothesis that the Higgs exists  exclusion

41. Exclusion - Look at what you expect from Standard Model +Higgs - Given the SM + Higgs expectation: if probability to observe as many events you have observed (or less) is smaller than 5%  SM+Higgs hypothesis is not very likely  reject SM+Higgs - Look at what you expect from Standard Model +Higgs - Given the SM + Higgs expectation: if probability to observe as many events you have observed (or less) is smaller than 5%  SM+Higgs hypothesis is not very likely  reject SM+Higgs

42. Expectation for Q or -2ln (Q): toy experiments If CL s < 0.05 we are allowed to reject the SM+Higgs at 95% confidence level The famous 95% confidence level Probability that signal hypothesis results in the numer observed or less Cls = confidence level in the signal SM SM+Higgs Extra Normalisation: This is why it is called modified frequentist

43. CLs mean SM-only expeciment is 0.13  > 0.05 so NO ! Question 2: did you expect to be able to exclude ?

44. Question 3: At what luminosity do you expect to make a discovery ? Lumi = 1x normal lumi CLs = 0.13  no exclusion for average SM-only experiment Lumi = 2x normal lumi CLs = 0.034  exclusion for average SM-only experiment #SM = 100 #H = 10 #SM = 200 #H = 20

45. A scan: Luminosity / nominal luminosity CLs CLs = 0.05 CLs = 0.13 CLs = 0.66 CLs = 0.046 2 sigma up 1 sigma down Si: If you would have a 1 sigma downward fluctuation, i.e. you see less events than you expect there is less room for a SM+Higgs hypothesis. In this case you would have been able to exclude it. You expect to be able to exclude at Lumi / Lumi nominal = 1.70

46. Question 4: At what Higgs xs do you expect to make a discovery ? Higgs XS = 1x normal Higgs XS CLs = 0.13  no exclusion for average SM-only experiment Higgs XS = 2x normal Higgs XS CLs = 0.006  exclusion for average SM-only experiment #SM = 100 #H = 10 #SM = 100 #H = 20

47. A scan: Higgs XS / nominal Higgs XS CLs CLs = 0.05 CLs = 0.13 CLs = 0.66 CLs = 0.046 2 sigma up 1 sigma down You expect to be able to exclude at Higgs XS / Higgs XS nominal = 1.40

48. A projection along the CLs = 0.05 line Higgs XS / nominal Higgs XS Nominal luminosity SM only (mean) At what Higgs XS scale factordo you expect to be able to exclude the Higgs hypothesis ? SM only (1 sigma up) SM only (2 sigma up) SM only (2 sigma down) SM only (1 sigma down) 1.4

49. Higgs XS / nominal Higgs XS 1.4 You can now scan over Higgs masses The important thing is of course what you actually measured

50. Finito!