1. Tests Jean-Yves Le Boudec

2. Contents 1.The Neyman Pearson framework 2.Likelihood Ratio Tests 3.ANOVA 4.Asymptotic Results 5.Other Tests 1

3. Tests Tests are used to give a binary answer to hypotheses of a statistical nature Ex: is A better than B? Ex: does this data come from a normal distribution ? Ex: does factor n influence the result ? 2

4. Example: Non Paired Data Is red better than blue ? For data set (a) answer is clear (by inspection of confidence interval) no test required 3

5. Is this data normal ? 4

6. 5.1 The Neyman-Pearson Framework 5

7. Example: Non Paired Data; Is Red better than Blue ? 6

8. Example: Non Paired Data; Is Red better than Blue ? ANOVA Model 7

9. Critical Region, Size and Power 8

10. Example : Paired Data Is A better than B ? 9

11. 10

12. Power 11

13. 12 Grey Zone power

14. p-value of a test 13

15. p-value of a test 14

16. 15

17. Tests are just tests 16

18. 17 Grey Zone power

19. 18

20. 2. Likelihood Ratio Test A special case of Neyman-Pearson A Systematic Method to define tests, of general applicability 19

21. 20

22. Example : Paired Data Is A better than B ? 21

23. 22

24. A Classical Test: Student Test The model : The hypotheses : 23

25. 24

26. Example : Paired Data Is A better than B ? 25

27. Here it is the same as a Conf. Interval 26

28. Test versus Confidence Intervals If you can have a confidence interval, use it instead of a test 27

29. The “Simple Goodness of Fit” Test Model Hypotheses 28

30. 1. compute likelihood ratio statistic 29

31. 2. compute p-value 30

32. Mendel’s Peas P= 0.92 ± 0.05 => Accept H 0 31

33. 3 ANOVA Often used as “Magic Tool” Important to understand the underlying assumptions Model Data comes from iid normal sample with unknown means and same variance Hypotheses 32

34. 33

35. 34

36. The ANOVA Theorem We build a likelihood ratio statistic test The assumption that data is normal and variance is the same allows an explicit computation it becomes a least square problem = a geometrical problem we need to compute orthogonal projections on M and M 0 35

37. The ANOVA Theorem 36

38. Geometrical Interpretation Accept H 0 if SS2 is small The theorem tells us what “small” means in a statistical sense 37

39. 38

40. ANOVA Output: Network Monitoring 39

41. The Fisher-F distribution 40

42. 41

43. Compare Test to Confidence Intervals For non paired data, we cannot simply compute the difference However CI is sufficient for parameter set 1 Tests disambiguate parameter sets 2 and 3 42

44. Test the assumptions of the test… Need to test the assumptions Normal In each group: qqplot… Same variance 43

45. 44

46. 4 Asymptotic Results 45 2 x Likelihood ratio statistic

47. 46

48. The chi-square distribution 47

49. Asymptotic Result Applicable when central limit theorem holds If applicable, radically simple Compute likelihood ratio statistic Inspect and find the order p (nb of dimensions that H1 adds to H0) This is equivalent to 2 optimization subproblems lrs = = max likelihood under H1 - max likelihood under H0 The p-value is 48

50. Composite Goodness of Fit Test We want to test the hypothesis that an iid sample has a distribution that comes from a given parametric family 49

51. Apply the Generic Method Compute likelihood ratio statistic Compute p-value Either use MC or the large n asymptotic 50

52. 51

53. Is it normal ? 52

54. 53

55. 54

56. Mendel’s Peas P= 0.92 ± 0.05 => Accept H 0 55

57. Test of Independence Model Hypotheses 56

58. Apply the generic method 57

59. 58

60. 5 Other Tests Simple Goodness of Fit Model: iid data Hypotheses: H 0 common distrib has cdf F() H 1 common distrib is anything Kolmogorov-Smirnov: under H 0, the distribution of is independent of F() 59

61. 60

62. Anderson-Darling An alternative to K-S, less sensitive to “outliers” 61

63. 62

64. 63

65. Jarque Bera test of normality (Chapter 4) Based on Kurtosis and Skewness Should be 0 for normal distribution 64

66. 65

67. Robust Tests Median Test Model : iid sample Hypotheses 66

68. Median Test 67

69. Wilcoxon Signed Rank Test 68

70. Wilcoxon Rank Sum Test Model: X i and Y j independent samples, each is iid Hypotheses: H 0 both have same distribution H 1 the distributions differ by a location shift 69

71. Wilcoxon Rank Sum Test 70

72. Turning Point 71 the indices of X (1), X (2), X (3) form a permutation, uniform among 6 values