We think you have liked this presentation. If you wish to download it, please recommend it to your friends in any social system. Share buttons are a little bit lower. Thank you!
Presentation is loading. Please wait.
Published byAlejandra Titman
Modified over 2 years ago
Fundamentals of GIS Materials by Austin Troy & Brian Voigt © 2011 Lecture 9: More Input Methods and Data Quality and Documentation By Austin Troy & Brian Voigt University of Vermont
Fundamentals of GIS Part 1: Data input methods: Geocoding ------Using GIS-- Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS What is Geocoding? Convert a list / spreadsheet to spatial data / geographical features Needs a mechanism to calculate the geographic coordinate –Address matching: uses street address database, created from a streets layer –XY coordinates Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Address Matching Geocoding Two required inputs: –1) a table with the address records, and –2) a geographic reference layer (e.g. streets, E911) Output: a point file, where each point represents an address record Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS How are addresses matched? Common method: matching address to street ranges. Urban areas: usually each street segment (arc) corresponds to a block. Each segment has attributes for the left from and to and right from and to addresses. Computer knows topological left and right for each street segment Step 1:Computer looks for segment with correct name and address range Step 2: Computer interpolates the position of the address point on segment Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Geocoding Example: 1060 Main Street 1060 Main St Locate point on even (upper) side of street Position of 1060 is interpolated Main St 1000 1100 10011101 L-F-ADDR L-T-ADDR R-F-ADDR R-T-ADDR Look for Main street, then for the 1000-1100 block direction Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Create address locator in ArcCatalog Defines reference layer Also where you specify information about your reference layer that ArcGIS might not know, allowing for more efficient geocoding Many styles to choose from in for address locators Address Matching in ArcGIS Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Geocoding Service Geocoding styles are necessary because –Reference layers come in many forms and formats. For instance, a reference layer may have the from right address attribute as fr_rt_add or add_rt_frm –There are other types of geocoding, besides address geocoding, like geocoding points to the center of zip codes, and there are other types of address geocoding besides street address geocoding, like using a property parcel layer as reference. Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Geocoding in ArcGIS In geocoding style interface: choose your reference file and then specify which attributes in the reference layer correspond with the inputs that ArcGIS needs to do geocoding. It also asks for some information about what to expect in your geocoding table (what the required attribute headings are called) and how sensitive to be to things like spelling differences Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Materials by Austin Troy © 2008 Specify reference file Specify address range attributes Specify rules for address list Specify zone
Fundamentals of GIS Geocoding in ArcGIS ArcMap >>> Tools >>> Geocode Addresses –Select the geocoding service you want to use This brings up the geocoding interface where we specify which field holds the address and which holds the zone Also specify an output shapefile or geodatabase and geocoding sensitivity Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Geocoding Results How many records were successfully matched How many were unmatchable How many were potentially matchable –Interactively match the potential ones Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Geocoding and Error Result is only as good as reference data –If the streets layer is only accurate to 200 meters, the geocoded points are assumed to be within 200 meters –If the streets data is consistently 100 meters to the north, then the geocoded points will be the same too Some roads layers may have better attributes than other too Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Geocoding and Error Heres an example where the same address list was geocoded with two different street layers. Note here how the same house is 100 m off between the two geocoding attempts 100 m Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Materials by Austin Troy © 2008 Geocoding and Error Here we see that many points were coded for Napa1 that were not coded for Napa2 possibly because Napa1s street reference layer is newer, and has more streets
Fundamentals of GIS Geocoding and Error This error is due to an attribute error in one of the layers which puts that address in the wrong street segment 300 m Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Geocoding and Error Rural road segments: long road segments, less precise Urban road segments: smaller, more precise Rural street segments are also more subject to more error because street segments longer, so relies more on interpolation Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Geocoding in Action Mapping hazard zone properties in L.A. to see effects on property values Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS XY Geocoding We can also create points from a table by their latitude and longitude Do this by clicking: CA haz. waste sites Then we specify the X,Y coordinate fields and the spatial reference system Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Part 2: Data input methods: Digitizing ------Using GIS-- Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Digitizing This is generally the process of converting data from analog to digital with a device, such as a digitizing tablet or mouse, to create new vector features User defines features by pointing and clicking. Table digitizing involves use of a digitizing tablet or table A digitizing table is a big table with an electronic mesh that can sense the position of a digitizing cursor Transmits X,Y coordinates of each mouse / cursor click to the computer and usually joins those with lines Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Digitizing Notice how it is attached with tape Source: http://ndis.nrel.colostate.edu/ndis/riparian/Tablet.jpg If it moves, the map will be inaccurate, because its recording position relative to the tablet, not the map
Fundamentals of GIS Digitizing Materials by Austin Troy & Brian Voigt © 2011 Snapping: Arc will also snap closed any unsnapped lines or polygons and will crop dangling lines, based on user-defined tolerances Snap tolerance: wont snap together Snap tolerance: will snap together Dangling arc Snapped to other arc
Fundamentals of GIS Digitizing Digitizing on a tablet requires defining control points which allow the conversion of the digitized map to real world coordinates. Usually, a corner point on the map of known geographic location is digitized first and its coordinates are assigned in some sort of header file Heads up digitizing involves scanning a paper map to a digital file, or otherwise obtaining a digital raster map/ image and digitizing on top of it on computer Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Materials by Austin Troy & Brian Voigt © 2011 Part 3: Spatial Data Quality
Fundamentals of GIS Data Quality Accuracy + Precision = Quality Error = f(accuracy, precision) Cost vs. quality tradeoff Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Materials by Austin Troy © 2008 Accuracy the degree to which information on a map or in a digital database matches true or accepted values. From Kenneth E. Foote and Donald J. Huebner http://www.colorado.edu/geography/gcraft/notes/error/error_f.html Reflection of how close a measurement represent the actual quantity measured and of the number and severity of errors in a dataset or map. Image source: http://oopslist.com/
Fundamentals of GIS Precision Intensity or level of preciseness, or exactitude in measurements. The more precise a measurement is, the smaller the unit which you intend to measure Hence, a measurement down to a fraction of a cm is more precise than a measurement to a cm However, data with a high level of precision can still be inaccuratethis is due to errors Each application requires a different level of precision Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Random and Systematic Error Error can be systematic or random Systematic error can be rectified if discovered, because its source is understood Image source: http://oopslist.com/ Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Random and Systematic Error Systematic errors affect accuracy, but are usually independent of precision; data can use highly precise methods but still be inaccurate due to systematic error Accurate and precise: no systematic, little random error inaccurate and precise: little random error but significant systematic error Accurate and imprecise: no systematic, but considerable random error inaccurate and imprecise: both types of error Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Measurement of Accuracy Positional accuracy is often stated as a confidence interval: e.g. 104.2 cm +/-.01 means true value lies between 104.21 and 104.19 One of the key measurements of positional accuracy is root mean squared error (RMSE); equals squared difference between observed and expected value for observation i divided by total number of observations, summed across each observation i This is just a standardized measure of errorhow close the predicted measure is to observed Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Positional Accuracy Positional accuracy standards specify that acceptable positional error varies with scale Data can have high level of precision but still be positionally inaccurate Positional error is inversely related to precision and to amount of processing Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Materials by Austin Troy © 2008 Accuracy is tied to scale
Fundamentals of GIS Positional Error Standards Different agencies have different standards for positional error Example: USGS horizontal positional requirements state that 90% of all points must be within 1/30th of an inch for maps at a scale of 1:20,000 or larger, and 1/50th of an inch for maps at scales smaller than 1:20,000 Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Positional Error Standards USGS Accuracy standards on the ground: 1:4,800 ± 13.33 feet 1:10,000 ± 27.78 feet 1:12,000 ± 33.33 feet 1:24,000 ± 40.00 feet 1:63,360 ± 105.60 feet 1:100,000 ± 166.67 feet See image from U. Colorado showing accuracy standards visually Hence, a point on a map represents the center of a spatial probability distribution of its possible locations probability distribution Thanks to Kenneth E. Foote and Donald J. Huebner, The Geographer's Craft Project, Department of Geography, The University of Colorado at Boulder for links Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Attribute Accuracy Attribute accuracy and precision refer to quality of non-spatial, attribute data Precision for numeric data means lots of digits Example: recording income down to cents, rather than just dollars Quantitative measurement errors: e.g. truncation A common error is to measure a phenomenon in only one phase of a temporal cycle: bird counts, river flows, average weather metrics, soil moisture Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Categorical Attributes Accuracy refers to amount of misclassification of categorical data The chance for misclassification grows as number of possible classes increases; accuracy is a function of precision, or number of classes If just classifying as land and water, that is not very precise, and not likely to result in an error Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Other Measures of Data Quality Logical consistency Completeness Data currency/timeliness Accessibility These apply to both attribute and positional data Image source: http://oopslist.com/ Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Examples of Currency and Timeliness Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Shelburne Road: 1937Shelburne Road: 2003
Fundamentals of GIS New North End and Colchester: 2003
Fundamentals of GIS Common sources of error Numerical processing (math operations, data type, rounding, etc) Geocoding (e.g. rural address matching and street interpolation) Topological errors from digitizing (overshoots, dangling nodes, slivers, etc) Automated classification steps, like unsupervised or supervised land cover classification in remote sensing, can result in processing errors Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Error Propagation and Cascading Materials by Austin Troy & Brian Voigt © 2011 Propagation: one error leads to another Cascading: errors are allowed to propagate unchecked from one layer to the next and on to the final set of products or recommendations Cascading error can be managed to a certain extent by conducting sensitivity analysis Image source: http://oopslist.com/
Fundamentals of GIS Conflation When one layer is better in one way and another is better in another and you wish to get the best of both Way of reconciling best geometric and attribute features from two layers into a new one Very commonly used for case where one layer has better attribute accuracy or completeness and another has better geometric accuracy or resolution Also used where newer layer is produced for some theme but it has lower resolution than older one Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Conflation Attribute conflation: transferring attributes from an attribute rich layer to features in an attribute poor layer Feature conflation: improvement of features in one layer based on coordinates and shapes in another, often called rubber sheeting. User either transforms all features or specifies certain features to be kept fixed Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Conflation Example Source: Stanley Dalal, GIS cafe Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata To avoid many of these errors, good documentation of source data is needed Metadata is data documentation, or data about data Ideally, the metadata describes the data according to federally recognized standards of accuracystandards of accuracy Almost all state, local and federal agencies are required to provide metadata with geodata they make Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata 1.Information retrieval, cataloguing, querying and searching for data electronically. 2.Describing fitness for use and documenting the usability and quality of data. 3.Describing how to transfer, access or process data 4.Documenting all relevant characteristics of data needed to use it Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata The federal geographic data committee (FGDC) is a federal entity that developed a Content Standard for Digital Geospatial Metadata in 1998, which is a model for all spatial data users to followFGDC Purpose is: to provide a common set of terminology and definitions for the documentation of digital geospatial data. All federal agencies are required to use these standards Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata Materials by Austin Troy & Brian Voigt © 2011 Metadata usually include sections similar to these
Fundamentals of GIS Metadata Critical components usually break down into: Dataset identification, overview Data quality Spatial reference information Data definition Administrative information Meta metadata Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata Data identification, overview and administrative info: General info: name and brief ID of dataset and owner organization, geographic domain, general description/ summary of content, data model used to represent spatial features, intent of production, language used, reference to more detailed documents, if applicable Constraints on access and use This is usually where info on currency is found Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata Data quality should address: Positional accuracy Attribute accuracy Logical consistency Completeness Lineage Processing steps Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata Spatial reference should include: horizontal coordinate system (e.g. State Plane) Includes projection used, scale factors, longitude of central meridian, latitude of projection origin, distance units Geodetic model (e.g. NAD 83), ellipsoid, semi- major axis Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata Data definition, also known as Entity and Attribute Information, should include: Entity types (e.g. polygon, raster) Information about each attribute, including label, definition, domain of values Sometimes will include a data dictionary, or description of attribute codes, while sometimes it will reference a documents with those codes if they are too long and complex Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata Data distribution info usually includes: Name, address, phone, email of contact person and organization Liability information Ordering information, including online and ordering by other media; usually includes fees Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata Metadata reference, or meta-metadata This is data about the metadata Contains information on When metadata updated Who made it What standard was used What constraints apply to the metadata Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata in ArcGIS ArcGIS allows you to display, import and export metadata in and to a variety of Metadata formats: It defaults to FGDC ESRI which looks like: Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata in ArcGIS XML is the most flexible form because its tag structure allows it to be used in programming; tags can be called as variables or can be created through form interfaces; allows for compatibility across platforms and programs Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata in ArcGIS In the past, complete metadata was only available as text; you had to create most embedded metadata tags yourself. Today many state and nationwide datasets come with complete embedded metadata including full attribute codes E.g. NEDs, NLCD, all VCGI data Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Metadata in ArcGIS Can edit, import, edit and export metadata in multiple formats allowing helping with proper sharing of data. Can also make templates to save time in repeat documentation of big data sets See NPS metadata extension for cool utilities Materials by Austin Troy & Brian Voigt © 2011
Fundamentals of GIS Materials by Austin Troy © 2008 Lecture 23: Data Quality and Documentation By Austin Troy University of Vermont NR 143.
Lecture 23: Data quality and documentation By Austin Troy Using GIS-- Introduction to GIS.
Fundamentals of GIS Materials by Austin Troy © 2008 Lecture 18: Data Input: Geocoding and Digitizing By Austin Troy University of Vermont NR 143.
Lecture 16: Data input 1: Digitizing and Geocoding By Austin Troy University of Vermont Using GIS-- Introduction to GIS.
Fundamentals of GIS Materials by Austin Troy © 2008 Lecture 18: Data Input: Geocoding and Digitizing By Austin Troy University of Vermont.
Lecture 24: More on Data Quality and Metadata By Austin Troy Using GIS-- Introduction to GIS.
Fundamentals of GIS Materials by Austin Troy © 2008 Lecture 9: More Input Methods and Data Quality and Documentation By Austin Troy University of Vermont.
Lecture 23: Brief Introduction to Data quality By Austin Troy Using GIS-- Introduction to GIS.
Fundamentals of GIS Materials by Austin Troy © 2006 Lecture 9: Data Quality and Input Methods By Austin Troy University of Vermont.
Geocoding Chapter 16 GISV431 &GEN405 Dr W Britz. Georeferencing, Transformations and Geocoding Georeferencing is the aligning of geographic data to a.
GIS Data Quality. Lecture Outline Accuracy Precision Error –Error Sources Using Multiple Data Sets Together –Completeness –Compatibility –Consistency.
Maps as Numbers GIS requires that both data and maps be represented as numbers. The GIS places data into the computer’s memory in a physical data structure.
GIS Tutorial 1 Lecture 6 Digitizing. Outline Digitizing overview GIS features Digitizing features Advanced digitizing tools Spatial adjustments.
Data Input How do I transfer the paper map data and attribute data to a format that is usable by the GIS software? Data input involves both locational.
Chapter 3 Sections 3.5 – 3.7. Vector Data Representation object-based “discrete objects”
ESRI Education User Conference – July 6-8, 2001 ESRI Education User Conference – July 6-8, 2001 Introducing ArcCatalog: Tools for Metadata and Data Management.
Introduction to ArcGIS for Environmental Scientists Module 2 – GIS Fundamentals Lecture 5 – Coordinate Systems and Map Projections.
Digitizing There are three primary methods for digitizing spatial information: Manual Methods include: Tablet Digitizing Heads-up Digitizing An Automated.
URBDP 422 Urban and Regional Geo-Spatial Analysis Lecture 2: Spatial Data Models and Structures Lab Exercise 2: Topology January 9, 2014.
Data Acquisition Lecture 8. Data Sources Data Transfer Getting data from the internet and importing Data Collection One of the most expensive.
GTECH 361 Lecture 13a Address Matching. Address Event Tables Any supported tabular format One field must specify an address The name of that field is.
URBDP 422 URBAN AND REGIONAL GEO-SPATIAL ANALYSIS Lecture 3: Building a GeoDatabase; Projections Lab Session: Exercise 3: vector analysis Jan 14, 2014.
Data Creation and Editing Based in part on notes by Prof. Joseph Ferreira and Michael Flaxman Lulu Xue | Nov. 3, :A Workshop on Geographical.
@ 2007 Austin Troy. Geoprocessing Introduction to GIS Geoprocessing is the processing of geographic information. Perform spatial analysis and modeling.
GIS Data Structures How do we represent the world in a GIS database?
Rebecca Boger Earth and Environmental Sciences Brooklyn College.
Geog 458: Map Sources and Errors January 20, 2006 Data Storage and Editing.
©2005 by Austin Troy. All rights reserved Lecture 5: Introduction to GIS Legend Visualization Lecture by Austin Troy, University of Vermont.
@2007 Austin Troy Lecture 4: An Introduction to the Vector Data Model and Map Layout Techniques Introduction to GIS By Brian Voigt University of Vermont.
Dr. David Liu Objectives Understand what a GIS is Understand how a GIS functions Spatial data representation GIS application.
CFR 250/590 Introduction to GIS, Autumn 1999 Data Search & Import © Phil Hurvitz, find_data 1 Overview Web search engines NSDI GeoSpatial Data.
NR 143 Study Overview: part 1 By Austin Troy University of Vermont Using GIS-- Introduction to GIS.
GIS Project1 Physical Structure of GDB Geodatabase Feature datasets Object classes, subtypes Features classes, subtypes Relationship classes Geometric.
Copyright, © Qiming Zhou GEOG1150. Cartography Data Models for Computer Cartography.
Vector-Based GIS Data Processing Chapter 6. Vector Data Model Feature Classes points lines polygons Layers limited to one class of data Figure p. 186.
INTRODUCTION TO GIS Used to describe computer facilities which are used to handle data referenced to the spatial domain. Has the ability to inter-
Data Quality Data quality –a measure of how well the GIS data represents the target domain Related terms: –Data uncertainty –Data error –Data accuracy.
Starting from Text Files. User Interface Graphic user interface –Mouse, buttons, windows, menus Commands-driven interface –Commands, parameters.
1 Overview Finding and importing data sets –Searching for data –Importing data_.
Introduction to ArcGIS for Environmental Sciences Day 2 – Fundamentals Module 8 Creating & Editing Data Creating Metadata.
Introduction to GIS Lecture 2: Part 1. Understanding Spatial Data Structures Part 2. Legend editing, choropleth mapping and layouts Part 3. Map layouts.
Managing Error, Accuracy, and Precision In GIS. Importance of Understanding Error *Until recently, most people involved with GIS paid little attention.
When you begin a project, a reference data layer is placed on the map first. This initial layer(s) is called the base map. There are different types of.
Geographic Information Systems. What is a Geographic Information System (GIS)? A GIS is a particular form of Information System applied to geographical.
Applied Cartography and Introduction to GIS GEOG 2017 EL Lecture-3 Chapters 5 and 6.
Geographic Information Systems : Data Types, Sources and the ArcView Program.
School of Geography FACULTY OF ENVIRONMENT Working with Tables 1.
© 2017 SlidePlayer.com Inc. All rights reserved.