1. Likelihood intervals vs. coverage intervals
Günter Zech
Likelihood intervals vs. coverage intervals
(What is better: exclude parameter with high likelihood by coverage interval
or have bad coverage in a likelihood ratio interval?)
Likelihood ratio
Coverage
Parameters are included if
observation has relatively high p.d.f. (competition)
add log-likelihoods
 single interval
observation has high
g.o.f.-probability (no competition)
About 68 % of intervals contain true value. They cannot be combined.

2. Example: H1: N(7.5, 2.5) H2: N(50, 100) t=10.1  LR=26 in favor of H1
Observaten is off by >1 st.dev of H1 prediction
but only half a st.dev off the H2 prediction
measurement
H1: N(7.5, 2.5)
H2: N(50, 100)
t=10.1  LR=26
in favor of H1
Likelihood ratio favors hypothesis 1
Frequentist view excludes hypothesis 1 at 1 st. dev. but supports hypothesis 2

3. Continuous version: Assume following theory:
(includes discrete hypotheses H1 and H2 for specific values of m) H1
The likelihood ratio limit does not cover.
The coverage inteval excludes hig likelihood values.
What is preferable?

4. Another illustration:
Asume earthquake theory following a similar formula as shown in the previous slide with different constants, and specific parameter values:
H1: predicts eathquake at 2008, Feb day
H2: predicts eathquake at 2010, Feb years
Assume earthquake happens at 2008, Feb. 15
Frequentist would exclude H1 by 2 st.dev.!
It is clear that H1 should not be excluded but why exclude H2 even though the observation is only half a standard deviation off the prediction?
Answer: Only one parameter is correct. If the parameter corresponding to H1 is very likely, H2 must be unlikely.

5. They ignore the precision of a prediction.
Conclusion
Problems with coverage intervals:
They do not take into account that parameter values are exclusive. If Parameter 1 applies parameter 2 cannot.
They ignore the precision of a prediction.
The contain only part of the information provided by the data.
All this is due to the violation of the Likelihood Principle
We should not require likelihood ratio intervals to cover.
Coverage intervals excluding regions of high likelihood are very problematic.

6. Some complementary remarks
Overcoverage
There is nothing wrong with conservative upper limits.
(It is better to present a conservative limit than a doubtfull one)
Including systematic errors
More specifically: calibration uncertainties, theoretical uncertainties, uncertainties which are not improving with 1/sqrt(N)
The only reasonable way is to invent a Bayesian p.d.f. and to integrate out the corresponding parameter. For upper limits one should be conservative. (It is better to overestimate the errors than to underestimate them.
Priors
Priors should incorporate prior knowledge and not be selected such that they produce the output you like.