VieVS User Workshop 7 – 9 September, 2010 Vienna Creating OPT-files Detecting and solving problems Tobias Nilsson.

Slides:



Advertisements
Similar presentations
Assumptions underlying regression analysis
Advertisements

VieVS User Workshop 7 – 9 September, 2010 Vienna Creating special files: Process lists, parameter files, TRF and EOP files Tobias Nilsson.
Statistical Techniques I EXST7005 Multiple Regression.
1 Chapter 4 Experiments with Blocking Factors The Randomized Complete Block Design Nuisance factor: a design factor that probably has an effect.
Chapter 4 Randomized Blocks, Latin Squares, and Related Designs
VieVS User Workshop 7 – 9 September, 2010 Vienna Vie_LSM Kamil Teke and Johannes Böhm.
Model Adequacy Checking in the ANOVA Text reference, Section 3-4, pg
AP Statistics Chapters 3 & 4 Measuring Relationships Between 2 Variables.
Graduate School Gwilym Pryce
1 Module II Lecture 3: Misspecification: Non-linearities Graduate School Quantitative Research Methods Gwilym Pryce.
VieVS User Workshop 14 – 16 September, 2011 Vienna Processing Intensive Sessions with VieVS Johannes Böhm.
VieVS User Workshop 14 – 16 September, 2011 Vienna VIE_MOD Lucia Plank.
VieVS User Workshop 14 – 21 September, 2011 Vienna Reprocessing the complete history of VLBI observations with VieVS Sigrid Böhm.
VieVS User Workshop 14 – 16 September, 2011 Vienna VIE_GLOB Hana Spicakova.
VieVS User Workshop 7 – 9 September, 2010 Vienna Introduction Johannes Böhm.
VieVS User Workshop 7 – 9 September, 2010 Vienna VIE_INIT Tobias Nilsson.
VieVS User Workshop 7 – 9 September, 2010 Vienna High frequency EOP estimated from the CONT campaigns Tobias Nilsson.
VieVS User Workshop 7 – 9 September, 2010 Vienna Automatic download and processing of VLBI data using VieVS Tobias Nilsson.
VieVS User Workshop 7 – 9 September, 2010 Vienna VIE_SETUP Tobias Nilsson.
© John M. Abowd 2005, all rights reserved Statistical Tools for Data Integration John M. Abowd April 2005.
VieVS User Workshop 7 – 9 September, 2010 Vienna VIE_GLOB Hana Spicakova.
VieVS User Workshop 14 – 16 September, 2011 Vienna VIE_SIM Jing SUN, Andrea PANY.
VieVS User Workshop 14 – 16 September, 2011 Vienna VIE_SCHED Jing SUN.
VieVS User Workshop 14 – 16 September, 2011 Vienna VIE_LSM Kamil Teke and Johannes Böhm.
Linear and generalised linear models Purpose of linear models Least-squares solution for linear models Analysis of diagnostics Exponential family and generalised.
VieVS User Workshop 7 – 9 September, 2010 Vienna Overwiev of VieVS: Installation, files, structure of the program Tobias Nilsson.
Slide 1 Detecting Outliers Outliers are cases that have an atypical score either for a single variable (univariate outliers) or for a combination of variables.
Regression Analysis Regression analysis is a statistical technique that is very useful for exploring the relationships between two or more variables (one.
Copyright © 2010, 2007, 2004 Pearson Education, Inc. All Rights Reserved Section 10-3 Regression.
1 SVY207: Lecture 18 Network Solutions Given many GPS solutions for vectors between pairs of observed stations Compute a unique network solution (for many.
(Station Dependent) Correlation in VLBI Observations John M. Gipson NVI, Inc./NASA GSFC 4 th IVS General Meeting 9-11 January 2006 Concepcion, Chile.
The Examination of Residuals. The residuals are defined as the n differences : where is an observation and is the corresponding fitted value obtained.
CJT 765: Structural Equation Modeling Class 7: fitting a model, fit indices, comparingmodels, statistical power.
1 1 Slide © 2014 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole.
Intervention models Something’s happened around t = 200.
Modern Navigation Thomas Herring
The Examination of Residuals. Examination of Residuals The fitting of models to data is done using an iterative approach. The first step is to fit a simple.
WS5-1 ADM730, Workshop 5, September 2005 Copyright  2005 MSC.Software Corporation WORKSHOP 5 RESULTS INTERPRETATION Response = 3 + 7X 1 + X 2 + 4X 1 X.
CJT 765: Structural Equation Modeling Class 8: Confirmatory Factory Analysis.
Sullivan – Fundamentals of Statistics – 2 nd Edition – Chapter 4 Section 2 – Slide 1 of 20 Chapter 4 Section 2 Least-Squares Regression.
Multiple Regression Petter Mostad Review: Simple linear regression We define a model where are independent (normally distributed) with equal.
Univariate Linear Regression Problem Model: Y=  0 +  1 X+  Test: H 0 : β 1 =0. Alternative: H 1 : β 1 >0. The distribution of Y is normal under both.
Copyright ©2011 Brooks/Cole, Cengage Learning Inference about Simple Regression Chapter 14 1.
Modern Navigation Thomas Herring MW 11:00-12:30 Room
Diagnostics – Part II Using statistical tests to check to see if the assumptions we made about the model are realistic.
Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved. McGraw-Hill/Irwin Model Building and Model Diagnostics Chapter 15.
Creating a Residual Plot and Investigating the Correlation Coefficient.
SEM Basics 2 Byrne Chapter 2 Kline pg 7-15, 50-51, ,
Copyright ©2006 Brooks/Cole, a division of Thomson Learning, Inc. More About Regression Chapter 14.
I271B QUANTITATIVE METHODS Regression and Diagnostics.
Statistical Data Analysis 2010/2011 M. de Gunst Lecture 10.
8-1 MGMG 522 : Session #8 Heteroskedasticity (Ch. 10)
Geology 5670/6670 Inverse Theory 16 Mar 2015 © A.R. Lowry 2015 Last time: Review of Inverse Assignment 1 Expect Assignment 2 on Wed or Fri of this week!
Rodent Complaints in Boston Number of rodent complaints in Boston per 2010 US census tract, Question: Is the spatial pattern of rodent complaints.
Scenario use cases Szymon Mueller PSNC. Agenda 1.General description of experiment use case. 2.Detailed description of use cases: 1.Preparation for observation.
MODEL DIAGNOSTICS By Eni Sumarminingsih, Ssi, MM.
Basic Estimation Techniques
CH 5: Multivariate Methods
GLOBK Velocity Solutions
CJT 765: Structural Equation Modeling
Lecture Slides Elementary Statistics Thirteenth Edition
Kamil Teke and Johannes Böhm
Troposphere and Clock Parameterization During Continuous VLBI Campaigns Kamil Teke1, 2, Johannes Boehm1, Hana Spicakova1, Andrea Pany1, Harald Schuh1 1.
3.5 Solving Nonlinear Systems
VIE_INIT_V30 Daniel Landskron.
Discrete Least Squares Approximation
Regression Forecasting and Model Building
Solving Equations 3x+7 –7 13 –7 =.
Correlation coefficients
Model Adequacy Checking
Presentation transcript:

VieVS User Workshop 7 – 9 September, 2010 Vienna Creating OPT-files Detecting and solving problems Tobias Nilsson

VieVS User Workshop Sometimes there are problems with a session... Clock breaks Noisy stations Bad sources...

VieVS User Workshop How do you know if there is a problem with a session? Large residuals after processing, e.g. chi- squared of main solution is large (larger than ) Strange results Stations or correlator report a problem

VieVS User Workshop Residuals of a good session Session 10JAN25XA (R1415) Chi-squared of main solution: 0.86 Looks good, no OPT-file is needed for this session (except for special investigations)

VieVS User Workshop Residuals of a bad session Session 09NOV30XA (R1407) Chi-squared of main solution: 2.32 Probably a clock breaks

VieVS User Workshop The OPT-file Contains specific information for making the problematic sessions work: Reference clock Clock breaks Stations to be excluded Sources to be excluded Baselines to be excluded Directory: DATA/OPT/SUBDIR/YYYY/

VieVS User Workshop Example of an OPT-file CLOCK REFERENCE: GOODSTAT CLOCK BREAKS: 3 JUMPY JUMPY BADCLOCK STATIONS TO BE EXCLUDED: 2 BADSTAT1 BADSTAT2 SOURCES TO BE EXCLUDED: BASELINES TO BE EXCLUDED: 1 BAD_BAS1 BAD_BAS2 NO CABLE CAL: 1 BADCABLE # This is a comment GOODSTAT is chosen as reference clock 3 clock breaks (2 in JUMPY, 1 in BADCLOCK) 2 station excluded 1 source excluded 1 baseline excluded Cable cal. of station BADCABLE not used.

VieVS User Workshop Reference clock The clock reference option specifies the default reference clock for the experiment. A station with a good clock should be used (i.e. it should not have any clock breaks!). If not specified, first station in the antenna structure array will be used as reference. CLOCK REFERENCE: GOODSTAT

VieVS User Workshop Clock breaks Sometimes there is a jump in a stations clock. The number after “CLOCK BREAKS:” specifies the number of clock breaks in the session. The following lines specifies the clock breaks: station name (8 characters) followed by the modified Julian date of the break. How to detect clock breaks: Find it manually using VieVS From correlator report or from e.g. the Goddard analysis. Divine inspiration. CLOCK BREAKS: 3 JUMPY JUMPY BADCLOCK

VieVS User Workshop Finding clock breaks Unselected the “apply main solution” in the first Vie_lsm GUI. Now only the first solution will be performed (only estimating quadratic clock functions and one ZWD per station). Afterwards, station-wise residual plots will be shown (except for the station with the reference clock).

VieVS User Workshop Finding clock breaks (II) Station-wise residuals plots after first solution:

VieVS User Workshop Finding Clock breaks (III) Residuals for WETTZELL, session 09NOV24XA_N004 Large residuals for observations with station 4 (BADARY). Sudden change between negative and positive residuals. Indicates a clock break at BADARY. If BADARY would be the reference clock, you would have to change the reference clock to another station and re-run the solution to get the residual plot for BADARY.

VieVS User Workshop Finding Clock breaks (IV) Residuals for BADARY, session 09NOV24XA_N004 Clear clock break in the middle of the session. To find the exact time of the break, zoom in.

VieVS User Workshop Finding Clock breaks (V) Zoomed in around the time of the clock break. Clock break seems to be between Modified Julian dates and Any time between the last observation before the break and the first after the break will do (e.g ). The clock break then needs to be entered into the OPT-file.

VieVS User Workshop Finding Clock breaks (VI) Create/edit the OPT-file and enter the clock break. When you now re-run the solution (using the OPT- file), the clock breaks should be corrected for in the first solution. CLOCK BREAKS: 1 BADARY NOV30XA.OPT

VieVS User Workshop Finding Clock breaks (VII) After entering the clock break into the OPT-file, the solution gets better. Before correcting for clock break Chi-squared: 2.32 After correcting for clock break Chi-squared: 0.69

VieVS User Workshop Excluding stations Reasons for excluding a station: Problematic station (e.g. very noisy observations for the station). Few observations for the station. Special investigations (test the impact of excluding the station). STATIONS TO BE EXCLUDED: 2 BADSTAT1 BADSTAT2

VieVS User Workshop Detecting bad stations Example of a noisy station: BADARY, session 09JUN04XE (R4381) Residuals after first solution Large residuals, no obvious reason (e.g. clock breaks) Removing BADARY from this session decreases chi-squared from 1.7 to 0.8

VieVS User Workshop Excluding sources Reasons for excluding a source: Bad source Bad coordinates for the source (and source coordinates fixed) Very few observations of a source (when you estimate all source coordinates). Special investigations (test the impact) SOURCES TO BE EXCLUDED:

VieVS User Workshop Detecting bad sources Normally quite difficult to detect bad sources Example: 09DEC02XA (RDV78) Some large residuals (chi- squared of main sol.: 21.9) Hint that there could be problems with the sources: many non-CRF sources observed (e.g. these may have bad coordinates)

VieVS User Workshop Detecting bad sources (II) Zoom in a bit around some large residuals Large residuals seems to be grouped. Could mean that certain scans are bad. Checking which scans that gives large residuals, it turns out that most are for scans observing sources and

VieVS User Workshop Detecting bad sources (II) Residuals of main solution after removal of sources and Large residuals disappeared (chi-squared now 0.79)

VieVS User Workshop Excluding baselines Excludes all observations from a specific baseline Reasons to exclude a baseline: Some error with the observations for that baseline Special investigations BASELINES TO BE EXCLUDED: 1 BAD_BAS1 BAD_BAS2

VieVS User Workshop Detecting bad baselines Example: 04DEC14XA (EURO74) Residuals after first solution: EffelsbergNy Ålesund (other stations similar)Wettzell Effelsberg large residuals with Wettzell Wettzell large residuals with Effelsberg All other stations: large residuals to Effelsberg and Wettzell Indicate problem with observations between Effelsberg and Wettzell

VieVS User Workshop Remove cable measurements Sometimes the cable measurements of a station is of bad quality. It is better not to use them at all (i.e. assume cable corrections zero) The “NO CABLE CAL” option specifies that the calbel calibration measurements of a station should not be used. How to know if a cable is bad: Correlation report, Goddard analysis report, station logs etc. Visual inspection of plots of the cable cal. measurements NO CABLE CAL: 1 BADCABLE

VieVS User Workshop Example of a bad cable FORTLEZA, session 09JAN13XA (RD0901) From:

VieVS User Workshop Other problems that can occur A few observations are bad Solution: add observations manually to Outlier file (if not done by outlier test) A clock has a large diurnal variability Solution, use very loose constraints for the piece- wise linear offsets of that clock. The normal equation matrix in the Least squares adjustment is (close to) singular Solution: reduce the number of parameters to be estimated

VieVS User Workshop Hopefully, you are now able to process VLBI sessions with VieVS without any problems!