GEODETIC INSTITUTE LEIBNIZ UNIVERSITY OF HANNOVER GERMANY Ingo Neumann and Hansjörg Kutterer The probability of type I and type II errors in imprecise hypothesis testing REC 2008 Reliable Engineering Computing February 20, 2008

Motivation 2 February 20, 2008 The probability of type I and type II errors “Sometimes the uncertainty budget in geodetic applications is too optimistic” Examples:  A geometric leveling around a big lake in Germany, Switzerland and Austria  Control networks that are observed with two different techniques (i) terrestrial measurements (ii) satellite measurements Why?  Ignorance of non-stochastic errors in the measurements and in the preprocessing steps of the measurements!?

Motivation 3 February 20, 2008 The probability of type I and type II errors

Motivation 4 February 20, 2008 The probability of type I and type II errors  The model constants are only partially representative for the given situation (e. g., the model constants for the refraction index for distance measurements).  The number of additional information (measurements) may be too small to estimate reliable distributions.  Displayed measurement results are affected by rounding errors.  Other non-stochastic errors of the reduced observations occur due to neglected correction and reduction steps and for effects that cannot be modeled. Why non-stochastic errors?

Motivation Uncertainty modeling in geodetic data analysis Statistical hypothesis tests in case of imprecise data  General form of a linear hypothesis  Probability of type I/II errors Geodetic applications  One and multidimensional case (weak imprecision)  Congruence test (strong imprecision) Conclusions and future work 5 February 20, 2008 The probability of type I and type II errors Agenda

Uncertainty Modeling 6 February 20, 2008 The probability of type I and type II errors Systematic effects Measurement process: -Stochasticity -Observation imprecision -(Outliers) Object fuzziness, etc... Stochastics (Bayesian approach) Interval mathematics Fuzzy theory Occurring uncertainties in this presentation :

Requirements:  Adequate description of Stochastics  Adequate description of Imprecision Solution: Describing the influence factors for the preprocessing step of the originary observation with fuzzy sets e. g., LR-fuzzy-number  7 February 20, 2008 The probability of type I and type II errors Uncertainty Modeling

Sensitivity analysis for the calculation of the imprecision of some parameters x of interest: - Instrumental error sources - Uncertainties in reduction and corrections - Rounding errors Influence factors (p) Linearization Partial derivatives for all influence factors Imprecision of the influence factors 8 February 20, 2008 The probability of type I and type II errors Uncertainty Modeling

9 February 20, 2008 The probability of type I and type II errors Uncertainty Modeling    -discitization Imprecise analysis ( ): Observation Imprecision Stochastic (Bayesian approach)        

Parameter Estimation (Linear case) Global test Outlier detection Congruence tests Model selection Sensitivity analysis These tests are based on a linear hypothesis Statistical hypothesis tests Tasks in (linear) parameter estimation: 10 February 20, 2008 The probability of type I and type II errors

General form of a linear hypothesis: Statistical hypothesis tests n:= number of observations u:= number of parameters precise case Introduction of a linear hypothesis: 11 February 20, 2008 The probability of type I and type II errors

Introduction of a quadratic form / test value: Test decision: And with imprecision? 12 February 20, 2008 The probability of type I and type II errors General form of a linear hypothesis: Statistical hypothesis tests

13 February 20, 2008 The probability of type I and type II errors General form of a linear hypothesis: Statistical hypothesis tests With imprecise influence factors p Introduction of a quadratic form / test value:

14 February 20, 2008 The probability of type I and type II errors General form of a linear hypothesis: Statistical hypothesis tests Resulting test scenario  1D comparison Final decision based on the comparison of the tests value with the regions of acceptance and rejection (card criterion)

Degree of agreement Degree of disagreement Basic idea 15 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests With:

Degree of rejectability Test decision: Degree of agreement Degree of disagreement 16 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests

The card criterion: with: ~~~ Overlap region 17 February 20, 2008 The probability of type I and type II errors with: Statistical hypothesis tests

With denoting the inverse function Probability of a type I error in the imprecise case: Statistical hypothesis tests.. 18 February 20, 2008 The probability of type I and type II errors

(3) Find in such a way that the following Equation is fulfilled within a negligible threshold: Probability of a type I error in the imprecise case: Statistical hypothesis tests.. (1) Choose an adequate value for : (2) Compute February 20, 2008 The probability of type I and type II errors

Probability of a type II error in the imprecise case: Statistical hypothesis tests.. 20 February 20, 2008 The probability of type I and type II errors Choose the non-centrality parameter or the probability of a type II error in the imprecise case

(3) Find in such a way that the following Equation is fulfilled within a negligible threshold: Probability of a type II error in the imprecise case: Statistical hypothesis tests.. (2) Choose an adequate value for : 21 February 20, 2008 The probability of type I and type II errors (1) Compute the probability of a type I error in the im- precise case

(3) Find in such a way that the following Equation is fulfilled within a negligible threshold: Non-centrality parameter in the imprecise case: Statistical hypothesis tests.. (2) Choose an adequate value for : 22 February 20, 2008 The probability of type I and type II errors (1) Compute the probability of a type I error in the im- precise case

Geodetic applications.. 23 February 20, 2008 The probability of type I and type II errors A geodetic monitoring network of a lock: The lock Uelzen I Monitoring network monitoring the actual movements of the lock:

Geodetic applications.. 24 February 20, 2008 The probability of type I and type II errors A geodetic monitoring network of a lock: Measurements: - horizontal directions (a) - zenith angles (b) - distances (c)

Geodetic applications.. 25 February 20, 2008 The probability of type I and type II errors Probability of a type I error in the imprecise case: A single outlier test (weak imprecision):

Geodetic applications.. 26 February 20, 2008 The probability of type I and type II errors A single outlier test (weak imprecision): Probability of a type I error in the imprecise case:

Geodetic applications.. 27 February 20, 2008 The probability of type I and type II errors Probability of a type I error in the imprecise case for a multiple outlier test: Non-centrality parameter for a multiple outlier test:

Geodetic applications.. 28 February 20, 2008 The probability of type I and type II errors epoch 1999epoch 2004 observations parameters identical points Congruence Test (strong imprecision):

Geodetic applications.. 29 February 20, 2008 The probability of type I and type II errors Congruence test (strong imprecision): Test situation: Probability of a type I error in the imprecise case for the congruence test:

Statistical hypothesis tests in linear parameter estimation (impecise case) Type I and Type II error probabilities The non-centrality parameter in the imprecise case 1D case is straightforward, mD case needs  -cut optimization The difference between the precise and the imprecise case depends on the task of the test: (i) outlier tests or (ii) safety-relevant test and on the order of magnitude of imprecision 30 February 20, 2008 The probability of type I and type II errors Conclusions and future work In progress: Assessment and validation using real data Reduce the computational complexity Take object fuzziness into account

The presented results are developed within the research project KU 1250/4 ”Geodätische Deformationsanalysen unter Verwendung von Beobachtungsimpräzision und Objektun- schärfe”, which is funded by the German Research Foundation (DFG). This is gratefully acknowledged Thank you for your attention! 31 February 20, 2008 The probability of type I and type II errors Acknowledgements

Contact information Ingo Neumann and Hansjörg Kutterer Geodetic Institute Leibniz University of Hannover Nienburger Straße 1, D Hannover, Germany Tel.: +49/+511/ {neumann, The probability of type I and type II errors in imprecise hypothesis testing 32 February 20, 2008 The probability of type I and type II errors

33

Tasks and methods  Determination of relevant quantities / parameters  Calculation of observation imprecision  Propagation of observation imprecision to the est. parameters  Assessment of accuracy (imprecise case)  Regression and least squares adjustments  Statistical hypothesis tests - General form of a linear hypothesis - Probability of Type I/II errors  Optimization of configuration 34 February 20, 2008 The probability of type I and type II errors Uncertainty Modeling

 Schnitt Optimierung Methoden zur Analyse der Impräzision

 cut optimization

1 x Precise case (1D) Example: Two-sided comparison of a mean value with a given value Null hypothesis H 0, alternative hypothesis H A, error probability  → Definition of regions of acceptance A and rejection R Clear and unique decisions ! 37 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests

38 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests 1 x Imprecise case Consideration of imprecision Precise case x 1 Imprecision of test statistics due to the imprecision of the observations

1 xx 1 Fuzzy-interval Imprecise casePrecise case Imprecision of the region of acceptance due to the linguistic fuzziness or modeled regions of transition 39 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests Consideration of imprecision

40 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests 11 xx Imprecise casePrecise case Imprecision of the region of rejection as complement of the region of acceptance Consideration of imprecision

11 xx Imprecise casePrecise case Conclusion: Transition regions prevent a clear and unique test decision ! 41 February 20, 2008 The probability of type I and type II errors Statistical hypothesis tests Consideration of imprecision

Conditions for an adequate test strategy Quantitative comparison of the imprecise test statistics and the regions of acceptance and rejection Precise criterion pro or con acceptance Probabilistic interpretation of the results Statistical hypothesis tests