1. Estimating the tail index from incomplete data
Yongcheng Qi
University of Minnesota Duluth
UCR, Feb 19, 2008

2. Outline Introduction Estimators of the tail index Edgeworth expansion
Empirical likelihood method
Simulation

3. Outline Introduction Estimators of the tail index Edgeworth expansion
Empirical likelihood method
Simulation

4. Introduction

5. Introduction
Some examples

6. Heavy-tailed distribution (1) characterized by a tail index
Introduction
Heavy-tailed distribution (1)
characterized by a tail index
applications in fields such as meteorology, hydrology, climatology, environmental science, telecommunications, insurance and finance
See, e.g. Embrechts, Kluppelberg and Mikosch (1997).
Modelling Extremal Events for Insurance and Finance. Berlin: Springer

7. Estimators of the tail index in the literature
Introduction
Estimators of the tail index in the literature
(Based on a complete sample, but only a few of upper order statistics used in the estimation)
- Hill (1975), Ann. Statist. 3,
- Pickands (1975), Ann. Statist. 3,
- Dekkers, Einmahl and de Haan (1989), Ann. Statist. 17,

8. Introduction: a full sample

9. Introduction: for incomplete data
Data are grouped, and only a few largest observations are observed within groups
Potential observations are i.i.d. with a heavy tail (1)
-- Previous estimation methods don’t apply in this case

10. Introduction: examples
1. For some financial data, only the information on a few yearly largest losses or claims is reported to public.
2. In Olympic games, only a few best players are allowed to participate, and thus only the scores for those players are observed within each game

11. Introduction: setting-up

12. Introduction: setting-up

13. Outline Estimators of the tail index Introduction Edgeworth expansion
Empirical likelihood method
Simulation

14. Estimators of the tail index

15. Estimators: second-order condition

16. Estimators: limiting distribution

17. Estimators: limiting distribution

18. Estimators: further extension
It is possible to consider the situation when the numbers of observations within the groups are different.
The numbers of the largest observations within groups can be different and at least 2.

19. Estimators: further extension

20. Estimators: further extension

21. Outline Edgeworth expansion Introduction Estimators of the tail index
Empirical likelihood method
Simulation

22. Edgeworth expansion
For the confidence intervals based on the asymptotic normality of our estimator, how does the selection of kn and mn impact the convergence rate for the coverage probability for our estimator?

23. Edgeworth expansion

24. Edgeworth expansion

25. Edgeworth expansion

26. Edgeworth expansion
The coverage probability of IN:

27. Outline Empirical likelihood method Introduction
Estimators of the tail index
Edgeworth expansion
Empirical likelihood method
Simulation

28. Empirical likelihood method
Owen (1988) Biometrika 75, ,
Owen (1990) Ann. Statist. 18,
for the mean vector of iid observations;
Owen (2001) Empirical Likelihood. Chapman and Hall
a wide range of applications
-- It allows the use of likelihood methods, without having to pick a parametric family for the data.
-- It produces confidence regions whose shape and orientation are determined entirely by the data.

29. Empirical likelihood method
For heavy-tailed distribution,
Lu and Peng (2002) Extremes 5(4),
Confidence intervals for the tail index
Peng and Qi (2006) Ann Statist. 34 (4),
Confidence intervals for high quantiles

30. Empirical likelihood method

31. Empirical likelihood method

32. Empirical likelihood method

33. Empirical likelihood method

34. Empirical likelihood method

35. Empirical likelihood method

36. Outline Simulation Introduction Estimators of the tail index
Edgeworth expansion
Empirical likelihood method
Simulation

37. Simulation 1. Burr (, ) distribution, given by
2. Frechet () distribution, given by

38. Simulation (r=1)
We generated 10,000 pseudorandom samples of size n = 1000 from one of the following distributions
Burr(1, 0.5), Burr(0.5, 1), Frechet(1)
Confidence level =95%
mn=[n/kn], the integer part of n/kn
Empirical coverage probabilities are plotted against different values of k = 10, 15, 20, …, 100 (Table 1)

39. Simulation
We generated 10,000 pseudorandom samples of size n = 1000 from one of the following distributions
Burr(1, 0.5), Burr(0.5, 1), Frechet(1)
Confidence level =95%
mn=[n/kn], the integral part of n/kn
Averaged lengths of the confidence intervals are plotted against different values of k = 10, 15, 20, …, 100 (Table 2)

40. Simulation

41. Simulation

42. Simulation: conclusion
Empirical likelihood method is better:
It generates
shorter confidence intervals, with
more accurate coverage probabilities

43. Comment: Why the normal approximation doesn’t work very well?
The coverage probability of IN:
For large kn, the leading term is

44. Thank you!