1. Lineage February 13, 2006 Geog 458: Map Sources and Errors

2. Outlines What is lineage? –Role of lineage in quality assessment –Contents of lineage –Detecting errors in lineage What do standards say about lineage? –SDTS –CSDGM Exercise

3. What is lineage? Definition –Lineage means the history of a data set –It also means the recounting of the life cycle of a data set: {collection, compilation, interpretation} Relation to data quality –Part of the data quality statement –Basis of quality assessment –Most primitive and most important element for fitness- for-use assessment –It is critical to all other components of data quality {accuracy, consistency, completeness} because they are affected by the contents of the lineage

4. Lineage from two different perspectives From data producers’ point of view –Ways to ensure that standards are being maintained –Provide an estimate of the errors at each stage of data production –Can be thought of a direct assessment of quality From data users’ point of view –Can determine whether or not the data set is fit for use given their particular applications –Some users are interested in creating data sets; they should know combined effects of integrating different data sets

5. Contents of lineage Divided into (1) source and (2) life cycle of data Source –Origin Who creates/collects the data? When is the data collected/created? Where does the data cover? Unique identifier –Reference fields Theoretical field which the data was referenced against (e.g. theoretical gravity field, geomagnetic fields, reference datum such as geoid and ellipsoid Introduces systematic error (bias) This information is necessary to make adjustment (correction or calibration) or transformation between distinct data set

6. Contents of lineage –Spatial data characteristics Scale Resolution Accuracy –Coordinate system Map projection if any Reference points (e.g. where’s origin in UTM?) –Corrections and calibrations Atmospheric and radiometric corrections (satellite image) Instrument calibrations Digitizing corrections

7. Contents of lineage Life cycle of a data set –Acquisition What assumptions are made when data is acquired? (e.g. measurements are made relative to datum) –Compilation Data being processed into sensor units (e.g. brightness in satellite image) Environmental variables are derived (e.g. vegetation index) –Derivation Data mapped on uniform space time grid scales with some completeness and consistency properties applied (e.g. transformation from aerial photo to orthophoto quadrangles; generation of gridded or contoured data sets; interpolation) Model output or results of analysis of lower level data

8. Life cycle of a data set A data set has different characteristics along each life cycle in terms of errors, irreversibility and usability AcquisitionCompilationDerivation Errors Irreversibility Usability depending on applications

9. Some typical spatial transformation that has a profound effect on a data set Coordinate transformation –Affine, rotation, polynomial –Complex (e.g. rubber-sheeting) –Georeferencing (e.g. image registration) Interpolation –Deterministic (e.g. nearest neighbor, IDW) –Statistical (e.g. Kriging) Cartographic abstraction –Displacement, generalization, exaggeration Reclassification –Map algebra (e.g. NDVI)

10. Errors introduced during spatial transformation Selection of different parameters could result in a different data set –Different versions of DEM due to different parameters Selected criteria and/or assumptions could introduce a bias into the resultant data –Systematic shift in location by coordinate transformation –Which tic marks are used in image registration? –What was the tolerance level for generalization algorithm? Such information should be stated in the metadata so that users can –reverse data set into the original data –take into consideration bias in integrating data sets into other data sets –determine whether or not the data is suitable

11. Lineage as a gateway to identifying potential errors introduced into the data set Source –Uncertainty is inherent in source data set due to the gap between reality and data set Three components of uncertainty –Interpretation: ambiguity, vagueness –Measurement: error Measurement issues related to errors –Accuracy, precision, resolution, scale –How do you tell these from metadata? –What kind of map projection: geometric properties (equal area, equidistant) Processing steps –From single data source Collection/acquisition: digitizing error, equipment error Sampling: is the sampling point as representative as possible? Registration: location of control point Interpretation: image classification (e.g. image to vegetation index) –From distinct data sources Coordinate system: different datum? Coordinate transformation: which algorithm? The effect of combining data of different resolution & currentness condition

12. How to document lineage Difficulty in documenting lineage information lies in finding concise qualitative ways in which to express it effectively Should be flexible given that data sets are created in very different settings Should be clear and understandable Some suggests presenting the information at different levels (e.g. UK case: feature-level to database-level) Hardly standardized, but some standards (e.g. SDTS, CSDGM) provide some common terms and definitions

13. Standardization of lineage information SDTS Description of the source material from which the data were derived The methods of derivation, including all transformations involved in producing the final digital files Dates of the source material and the dates of ancillary information used for update Currentness reference {ground or publication date}

14. Standardization of lineage information CSDGM Information about the events, parameters, and source data which constructed the data set, and information about the responsible parties Read production rules for Data Quality – Lineage from the course packet (p. 10-11) Read the definition of terms used in data elements of lineage

15. Exercises Go to USGS data catalog service website at http://geo-nsdi.er.usgs.govhttp://geo-nsdi.er.usgs.gov Choose any metadata from the website Answer questions provided in handouts 5 minutes group speech on Wednesday Do the reading for positional accuracy –Minnesota handbook –DCW error analysis