1. Wolfgang Niemeier Hamburg, Sept. 15, 2010 Tutorial: Error Theory and Adjustment of Networks Institut für Geodäsie und Photogrammetrie

2. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 2 Lecture I : Error Theory and Basics of Adjustment 1.Introduction 2.Quality Estimates for Observations 3.GUM 4.Variance and Covariance Propagation 5.Concept of Parametric Adjustment 6.Further Adjustment Concepts

3. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 3 Typical Measuring Instruments Levelling: Lasertracker Total Station

4. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 4 3D Cartesian Coordinate System Principle: 3D-Cartesian Space Often separated (Due to orientation to normal gravity vector) - Horizontal xy-plane - Vertical direction: z-component Measurements: - Horizontal directions / bearings r 1’ r 2’ … Angles are differences between directions:  = r 2 – r 1 - Slope distances d i - Zenithal angles z i Elevation angles 100gon - z i

5. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 5 Validity of 3D-Cartesian Coordinate System A local cartesian coordinate system neglects the curvature of the earth. This effect can be computed for deviation in heights and for distances: Distance [ m ] Deviation in Height [mm] Deviation for Distance in horizontal plane [mm] 50-0.190.000 100-0.780.000 250-4.900.000 500-19.620.001 1000-78.480.008 2000-313.920.066 5000-1962.051.026

6. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 6 Quality Estimates for Observations

7. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 7 Basic statistics:  Random variable  Measurements Example: Distance measurements Processing of repeated observations: Arithmetic mean Empirical residuals: Empirical variance (standard deviation * 2 )

8. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 8 Classical Concept: Normal Distribution Reasoning  Gauß: „Guess“ (1801)  Cramer: Central limit theorem (1947)  Hagen: Sum of elementary errors (1837) Normal distribution 2 parameters: μ mean σ 2 variance Density function:

9. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 9 Normal distribution is still valid !?! But other concepts exist:  modulated normal distribution mixture of 2 or more normal distributions  random noise, white, colored spectral decomposition of error budget  robust estimates estimates, if several gross (systematic) errors are in the data  extended uncertainty estimates GUM

10. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 10 Confidence region for μ mostly at 95% confidence level Property of observation: Expection of mean: Sum of residuals Classical statistical treatment Quantile of normal distribution

11. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 11 Correlation/Covariance between observations Two variables with series of n observations: Mean value and (true) errors Covariance matrix:

12. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 12 Quality Estimates for Observations Complete Covariance Matrix Meaning: Variance of variable X i Correlation coefficient: n Variables

13. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 13 GUM

14. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 14 GUM Guide to the Expression of Uncertainty in Measurement (GUM) ISO/BIPM Publication,1995 Idea: „New concept, to attach realistic estimate for variability (variance) to a measuring quantity. Two Catagories: Type A : Classical statistical components, e.g. variance estimates Type B : Further influence factors, to account for centering errors, systematic effects, atmospheric effects, etc Why ?  „Push the button“ does not allow to give realistic estimate!  Repeated observations are not meaningful  Has to include further informations

15. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 15 Uncertainty according to GUM Estimates of Type A and B are combined according to the rules of variance propagation: Resulting Uncertainty is u c (similar to standard deviation ?) Extended uncertainty Problem (and main criticism): - Need estimates for each influence factor of Type B - Type B effects have to be modelled as stochastic variables

16. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 16 Variance and Covariance Propagation

17. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 17 Variance and Covariance Propagation For derived variables: General function: For uncorrelated observations:

18. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 18 Variance-Covariance Progagation (Linear Function) (n,1) random vector X with covariance matrix Task: Determine covariance matrix for (m,1) vector Y A and b are non-stochastic quantities, i.e. constants. Introducing the given function Final formula:  YY

19. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 19 Concept of Parametric Adjustment or: Adjustment of Observations Gauss - Markov - Model

20. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 20 Treatment of redundant observations Linear Regression: Processing strategies:  Use line between each 2 points  Use all points to determine the parameter of the line Transformation:  Using just 2 identical points  Using all available identical points: Overdetermined transformation (adjustment approach)

21. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 21 Treatment of redundant observations Coordinate determination of a new point in horizontal plane:  Observations: - 2 distances D1 and D2 - 2 angles a 2 and a 2  Parameter: - Coordinate X N - Coordinate Y N  Type of processing: - Use two angles - Use two distances - Use one distance and one angle - Use all information : Adjustment

22. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 22 Concept of Adjustment of Observations Functional model: The observations are a function of the coordinates: Stochastic model: variance factor, should be equal to 1 a priori knowledge of the cofactor matrix of the observations Taylor series expansion with approximate coodinates X 0 : Introducing a residual v for the observations:

23. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 23 Solution Algorithm for Gauss-Markov Model Least-squares-method optimizes the weighted sum of quadratic residuals: With the functional model The quadratic sum of residuals is a function of the parameters x To find the minimum of the sum we set the gradient to zero:

24. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 24 Solution Algorithm for Gauss-Markov Model  Estimated parameters:  Cofactor matrix for parameters:  Estimation for variance factor: „a posteriori“ Final results, to be used in practise:  Final coordinates:  Final covariance matrix:

25. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 25 Model Quality ( Global Test for Adjustment Model ) Statistical Test: Reasons for test failure  Gross and systematic errors in observations  Functional relationship are not correct  Calibration parameters not considered  Weighting between groups of observation not correct: need: variance-component estimation Relations between L and X are modeled in proper way, if after the adjustment coincides with the a priori value

26. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 26 If is not valid, especially if several gross errors are expected in data, e.g. 2 – 10 % (break even point ?) Robust statistics (Hampel and Huber (since 1965), Rouseeuw (1987): Basic Idea: Restrict influence of outlying observations on results : Various theoretical concepts, algorithms,... Easy applicable in practise: Reweighting: Observations with large normalised residual: Increase variance, i.e. reduce weight of observation in adjustment model Common strategy: - Best result is least-squares-solution - If errors are indicated, use reweighting Robust statistics

27. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 27 Different Concepts for Adjustment Conditional Adjustment Only observations are adjusted, example: levelling loops Coordinates are computed with adjusted quantities in separate step Parametric Adjustment (Gauss-Markov-Model) The observations are considered as function of the parameters L=f(X) For each observation its relation to coordinates has to be expressed. Examples: Normal network adjustments Combined Adjustment of Observations and Parameters Only implict formulation is possible: f(L,X)=0 Basis are functional relations between various observations and various parameters in each equation.

28. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 28 Lecture II : Adjustment of Networks 1.Typical observations and their pre-processing 2.Adjustment Process 3.Quality Measures for Adjustment Results 4.Datum Problem 5.Simulation + Optimization 6.Use of Lasertrackers in Adjustment Process 7.Software Package PANDA

29. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 29 Typical observations and their pre-processing

30. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 30 Typical Measuring Instruments Levelling: Lasertracker Total Station

31. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 31 Typical observations and their pre-processing Typical observations in local 3D networks  Leveling -Height differences  Total Station -Distances -Directions/Bearings -Zenith angels  Laser tracker (horizontation ?, see last chapter) -Distances -Directions/Bearings -Zenith angels

32. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 32 Functional relation between observations and coordinates Observations between point P i and P j. X, Y, Z are point coordinates. Parameter o: orientation unknown

33. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 33 Additional Parameters:  Calibration is not perfect, add: -Additional constant for distances -Scale factor for distances  Refraction unknown for zenith angles to account for atmospheric effects, e.g. in tunnels  Deviations of the vertical affects directions and zenith angles; important to account for gravity field variations

34. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 34 Adjustment Process

35. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 35 Functional Model : Relations between Observations and Coordinates Linearise functional relations: => Observation Equations with:

36. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 36 Stochastic model: Summarize knowledge on variability of observations Normal assumption: For each type of observation just one variance estimate exists: => Derived a priori covariance function (often as diagonal matrix)

37. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 37 Adjustment computations Solution, according to derivation in Lecture I: Estimated parameters : Covariance matrix of parameters: Variance after the adjustment: But: Is this result final or are there outliers, systematic effects, etc ? Is the achieved quality sufficient ? Is the datum of the computation chosen properly ?

38. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 38 Quality Measures

39. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 39 Precision of coordinates: Derived out of Absolute measures: - variances for coordinates - confidence ellipses for points General probability relation for confidence ellipses: For a single point P j :As relative measures between point P i and P j : Relative measures: - for distances (derived quantity) - relative confidence ellipses between points

40. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 40 Precision of coordinates: Derived out of = Q xx Cofactor matrix Q xx as basic information for precision values:

41. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 41 Reliability Concept Only distances measured: no control 2 distances and 1 angle measured Controlability ! 2 distances and 2 angle measured: Separability ! (Errors are detectable) Example: Determination of point N, Points P1 and P2 are known.

42. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 42 Reliability measures The residuals are important to assess the results, because the true errors are unknown. The residuals are a linear function of the observations: For uncorrelated observations the diagonal of Q vv is dominant and determines the fraction of the observation error reflected in the residual. This fraction is called the redundancy number r i : A redundancy number of zero indicate an observation without control! r < 0.2 bad control; 0.2 < r < 0.8 ok; 0.8 < r well controlled Derived out of Q vv

43. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 43 Check of gross errors The residual v follows the normal distribution: The standardized residual w is defined as: with Residuals with w > y are detected as gross errors! Quantile of normal distribution

44. Datum Problem Observations Datum definition

45. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 45 2D-network with angle-observations, only - Can be positioned and oriented arbitrarily in coordinate frame - Scale is not defined, i.e. absolute size is not fixed

46. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 46 Datum Parameters in Geodetic Networks Network Dimension Maximum Datum- defect Datum Parameter Observations with information on datum parameters 1 vertical network 1Translation in zHeight differences: No Absolute heights: Yes 2 horizontal network 4Translation: x, y Orientation: z Scale Distances => scale Azimuth => orientation around z 3 3D-spatial network 7Translation: x, y, z Orientation: x, y, z Scale Distances => scale Azimuth => orientation around z Zenith angles => orientation around x and y

47. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 47 Different models for datum fixation  Zero Variance computational base. Select number of coordinates according to free datum parameters and set them fix (no variance)  Minimum constraint solution Position network geometry on set of all approximate parameters Gives best (smallest) variances for points  Weak datum Introduce „given coordinates“, i.e. use coordinate values for selected points as observations and attach variance information to them the network. Network geometry will be effected.  Hierachical adjustment Fix coordinates of several (higher order ?) points. If number of free datum parameters is exceeded, the network geometry will be effected / detroyed.

48. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 48 Zero Variance Computational Base Plane network, datum parameters 4: Approach: Select given coordinates of two points to define the datum

49. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 49 Minimum Constraint Solution Plane network, datum parameters 4: Approach: Position (inner) network geometry on all approximate coordinate (Similar to a Helmert-transformation on all given points)

50. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 50 Simulated Adjustment and Optimization Objective: Evaluate and optimize network design.  Quality check, before observations are taken: Is expected precision and reliability sufficient ?  Type of instrument and number of observations: Is the selected instrumentation adequate ?  Configuration of network points: Is the strenght of the geometry of the network sufficient ? Precision estimates: Out of and Reliability estimates: Out of => Computation of precision and reliability measures is possible, before observations are made !!

51. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 51 Simulated network, Example from M. Schlösser Minimum constrain solution, based on 8 outer points N1 to N8 2D-network, 3 points in the middle are observation stands Used standard deviations: directions: 0.3 mgon, distances: 0.2 mm Minimum constraint solution: Datum is based on points N1 to N8

52. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 52 Simulated network, Example from M. Schlösser Partial Minimum constraint solution: Datum is based on points N1 to N4 2D-network, 3 points in the middle are observation stands Used standard deviations: directions: 0.3 mgon, distances: 0.2 mm

53. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 53 Simulated Example: Check of the Redundancy (Reliability) Colour for each observation indicates the redundancy number r i : r i 0.2 green Remark: Redundancy is depending on datum fixation Here: Minimum constraint solution

54. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 54 Combination of Lasertracker with other instruments

55. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 55 Characteristics of a Lasertracker  Absolute and relative distances, horizontal and vertical angles/directions  Not automatically oriented to earth gravity field (only by external levels)  Calibration is given by manufacturer, can be controlled by user  Precision of Distance Measurements - Interferometric: Continuous with nm precision : 10 -9 - Absolute: 0.025 mm + 0.001 ppm  Precision of Angles Observations (automatic readings) - Horizontal : 0,6 mgon - Vertical : 0,6 mgon i.e. less precise than total stations with e.g. σ H = σ V = 0.15 mgon

56. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 56 Use of Lasertracker with arbitrary orientation : Rotation up to 180°

57. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 57 Include for each set-up (position) of lasertracker data a complete set of three orientation angles φ ω κ i.e. use full 3D-rotation matrix for observations To be introduced for each instrumental set up Requires : Approximate values for these rotation angles. (Their determination is without the scope of this tutorial) Treatment of extremly rotated Lasertracker :

58. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 58 Rigorous hybrid adjustment is possible: Lasertracker, Total Station and levelling Use lasertracker measurements, i.e. horizontal directions, vertical angles and distances with their adequate covariance-matrix as additional group of observations in adjustment model. L hz v h L vz v z L dist v dist x x L lev v lev x y L az + v az = A * x z L hz, LT v hz, LT x add L v, LT v v, LT x trans L dist, LT v dist, LT..

59. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 59 Software Package PANDA

60. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 60 All computations are performed with software package PANDA Package for Adjustment of Networks and Deformation Analysis

61. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 61 Software Package PANDA Package for Adjustment of Networks and Deformation- Analysis Interactive processing of 1D-, 2D- and 3D-geodetic networks out of all areas of surveying and geodesy Preprocessing of raw total station and leveling data, including plausibility control and computation of approximate coordinates Combined rigorous adjustment of lasertracker. total station, leveling and azimuth observations together with GPS-coordinate sets and existing/given coordinates Deformation analysis for 1D-, 2D- and 3D-networks (e.g. rigorous congruency tests) Interactive simulation and network optimization

62. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 62 Independant development for PC under WINDOWS Local cartesian, local spherical, ellipsoidal and global cartesian coordinate systems; different mapping systems Different transformation concepts are included All common datum definitions: Fixed datum, minimum constraint and weak datum; hierarchical concept All common quality measures for precision and reliability Can process large networks (e.g. 20.000 stations) in acceptable time Software Package PANDA Package for Adjustment of Networks and Deformation- Analysis

63. Niemeier, Wolfgang | Tutorial on Adjustment | IWAA | September 15, 2010 63 Thank you for your attention!