REC Savannah, Febr. 22, 2006 Title Outlier Detection in Geodetic Applications with respect to Observation Imprecision Ingo Neumann and Hansjörg Kutterer Geodetic Institute University Hannover Germany Steffen Schön Engineering Geodesy and Measurement Systems Graz University of Technology Austria REC 2006

REC Savannah, Febr. 22, 2006 Contents Motivation Uncertainty modeling in geodetic data analysis Statistical hypothesis tests in case of imprecise data - One-dimensional case - Multi-dimensional case Geodetic applications - Global test in least squares adjustment - Outlier detection (e.g., GPS-baseline) Conclusions and future work

REC Savannah, Febr. 22, 2006 Tasks and methods  Determination of relevant quantities / parameters  Calculation of observation imprecision  Propagation of observation imprecision to the est. parameters  Assessment of accuracy (imprecise case)  Regression and least squares adjustments  Statistical hypothesis tests  Optimization of configuration Motivation

REC Savannah, Febr. 22, 2006 Systematic effects Measurement process: -Stochasticity -Observation imprecision -(Outliers) Model uncertainty, object fuzziness, etc. Stochastics (Bayesian approach) Interval mathematics Fuzzy theory Motivation Focus in this presentation :

REC Savannah, Febr. 22, 2006 Requirements:  Adequate description of Stochastics  Adequate description of Imprecision Solution: Describing the influence factors for the preprocessing step of the originary observation with fuzzy sets e. g., LR-fuzzy-number Uncertainty modeling in geodetic data analysis 

REC Savannah, Febr. 22, 2006 Sensitivity analysis for the calculation of observation imprecision: Uncertainty modeling in geodetic data analysis - Instrumental error sources - Uncertainties in reduction and corrections - Rounding errors Influence factors (p) Linearization Partial derivatives for all influence factors Imprecision of the influence factors

REC Savannah, Febr. 22, 2006 Sensitivity analysis for the calculation of observation imprecision: The sensitivity of the observations as a result of the preprocessing steps The fuzzy sets of the observation are splitted in a centre ( ) and radius ( ) part for a sufficient number of  cuts. Uncertainty modeling in geodetic data analysis with

REC Savannah, Febr. 22, 2006 Tasks and methods (Special case of Random-Fuzzy)  Determination of relevant quantities / parameters  Calculation of observation imprecision  Propagation of observation imprecision to the estimated parameters Uncertainty modeling in geodetic data analysis Observation imprecision Stochastics (Bayesian approach) 1

REC Savannah, Febr. 22, x Precise case (1D) Statistical hypothesis tests in case of imprecise data Example: Two-sided comparison of a mean value with a given value Null hypothesis H 0, alternative hypothesis H A, error probability  → Definition of regions of acceptance A and rejection R Clear and unique decisions !

REC Savannah, Febr. 22, x Imprecise case Consideration of imprecision Precise case x 1 Imprecision of test statistics due to the imprecision of the observations Statistical hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, xx 1 Fuzzy-interval Imprecise case Consideration of imprecision Precise case Imprecision of the region of acceptance due to the linguistic fuzziness or modeled regions of transition Statistical hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, xx Imprecise case Consideration of imprecision Precise case Imprecision of the region of rejection as complement of the region of acceptance Statistical hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, xx Imprecise case Consideration of imprecision Precise case Conclusion: Transition regions prevent a clear and unique test decision ! Statistical hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, 2006 Statistical hypothesis tests in case of imprecise data Conditions for an adequate test strategy Quantitative comparison of the imprecise test statistics and the regions of acceptance and rejection Precise criterion pro or con acceptance Probabilistic interpretation of the results

REC Savannah, Febr. 22, 2006 Degree of agreement Degree of disagreement Statistical hypothesis tests in case of imprecise data Considered alternatives Basic idea height criterion card criterion

REC Savannah, Febr. 22, 2006 Degree of rejectability Statistical hypothesis tests in case of imprecise data Test decision: Degree of agreement Degree of disagreement

REC Savannah, Febr. 22, 2006 The height criterion: ~ ~~ with: Statistical hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, 2006 The card criterion: with: ~~~ Overlap region Statistical hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, 2006 Test situation with tight bounds (weak imprecision): ~~ ~~ Statistical hypothesis tests in case of imprecise data degree of rejectability for the card criterion degree of rejectability for the height criterion

REC Savannah, Febr. 22, 2006 degree of rejectability for the card criterion degree of rejectability for the height criterion ~~ ~ ~ Statistical hypothesis tests in case of imprecise data Test situation with wide bounds (strong imprecision):

REC Savannah, Febr. 22, 2006 Multidimensional hypothesis tests in Geodesy n:= number of observations u:= number of parameters d:= rank deficiency of the normal equations matrix precise case Test situation and test value without imprecision:

REC Savannah, Febr. 22, 2006 with and precise case f = n - u + d := degrees of freedom n:= number of observations u:= number of parameters d:= rank deficiency of the normal equations matrix Multidimensional hypothesis tests in Geodesy

REC Savannah, Febr. 22, 2006 Hypotheses: Test decision: (  error probability) Multidimensional hypothesis tests in Geodesy 1 x precise case

REC Savannah, Febr. 22, 2006 Multidimensional hypothesis tests in case of imprecise data Search the smallest and largest element s for for a sufficient number of  -cuts Strict realization of Zadeh‘s extension principle! Optimization algorithm

REC Savannah, Febr. 22, 2006  cut optimization for a 2-dimensional point test: Multidimensional hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, 2006  cut optimization for a 2-dimensional point test: Multidimensional hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, 2006 Resulting test scenario  1D comparison Final decision based on height or card criterion Multidimensional hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, 2006 A geodetic monitoring network of a lock: Applications The lock Uelzen IMonitoring network monitoring the actual movements of the lock:

REC Savannah, Febr. 22, 2006 A geodetic monitoring network of a lock: Applications n = 313 observations u = 45 parameters d = 3 datum defects OUTLIERS in the collected measurements! Remove the OUTLIERS from the collected measurements, because they may falsify point coordinates! Statistical hypothesis tests in case of imprecise data

REC Savannah, Febr. 22, 2006 Applications Global test in least squares adjustment

REC Savannah, Febr. 22, 2006 Applications GPS-baseline test ( )

REC Savannah, Febr. 22, 2006 Applications GPS-baseline test ( )

REC Savannah, Febr. 22, 2006 Conclusions and future work Statistical hypothesis tests can be extended for imprecise data Degrees of agreement and disagreement Degree of rejectability  comparison of fuzzy sets 1D case is straightforward, mD case needs  -cut optimization card criterion more adequate than (easier-to-apply) height crit. Not shown but computable: Type I and Type II error probs. Not shown but available: Extended regression and optimization In progress: Assessment and validation using real data

REC Savannah, Febr. 22, 2006 Acknowledgements The presented results are developed within the research project KU 1250/4-1 ”Geodätische Deformationsanalysen unter Verwendung von Beobachtungs- impräzision und Objektunschärfe”, which is funded by the German Research Foundation (DFG). This is gratefully acknowledged. The third author stays as a Feodor-Lynen-Fellow with F. K. Brunner at TU Graz, Austria. He thanks his host for giving the possibility to contribute to this study and the Alexander von Humboldt Foundation for the financial support. Thank you for your attention!