Geoprocessing and georeferencing raster data

Raster conversion tools Geoprocessing tools ArcCatalog tools Raster data conversion tools: Populating a database is one of the most critical processes involved in building a GIS system. ArcGIS provides a multitude of tools to aid this process. •Raster to ASCII Converts rasters into an ASCII file. •Raster to Float Converts rasters into a floating point file. •Raster to Point Converts rasters into a point feature class. •Raster to Polygon Converts rasters into a polygon feature class. •Raster to Polyline Converts rasters into a polyline feature class. When you need to convert your feature data into raster data, several To Raster conversion tools are available. When converting your data, make sure you use the appropriate conversion tool for the feature type at hand. You will be able to write out to four possible raster formats: ESRI GRID, ERDAS IMAGINE, TIFF, or geodatabase. ASCII to Raster Converts ASCII files to a raster format. •DEM to Raster Converts DEM files to a raster format.. •Feature to Raster Converts feature classes to a raster format. Float to Raster Converts floating point files to a raster format.. Raster to Other Format Converts raster datasets to ESRI Grid, ERDAS IMAGINE, or TIFF format. ArcGIS cannot directly read and perform analysis on a DEM, so they must be converted into a supported raster format. When the DEM is converted to a raster format, the data type can be float or integer. The To Coverage toolset contains several conversion tools that allow you to convert various file formats into coverages. The SDTS tools found here will converts from SDTS Topological Vector (TVP), Raster, or Point to Grid or coverage.

ArcMap - raster projection options Best situation - all inputs have same spatial reference Simple or robust algorithm choice Applies projection on the fly Projection of data frame or first layer Export option - data frame or original projection Various export formats Projecting raster datasets in ArcMap ArcMap performs what is commonly known as on-the-fly projection. This means ArcMap can display data stored in one projection as if it were in another projection. The new projection is used for display and query purposes only. The actual data is not altered. Projecting on the fly Data is projected on-the-fly any time a data frame contains a layer whose coordinate system is defined as something different from the coordinate system definition of the data frame. ArcMap will not project data on the fly if the coordinate system for the dataset has not been defined. A dataset with an undefined coordinate system will simply be displayed in its native coordinate system. On-the-fly projection uses a single polynomial transformation rather than a cell-to-cell projection. Inaccuracy with on-the-fly projection can be more of a concern for raster layers located at a latitude greater than 70 degrees north or south or for dataset blocks greater than one degree in extent. ArcGIS provides two internal methods of displaying raster data projected on-the-fly: a simple algorithm and a more robust algorithm. The simple algorithm is based on a single polynomial transformation and it is fast, but it may not meet certain accuracy needs if the raster data covers a large area, the raster is located at high latitudes, or the raster is being transformed between certain complicated projections. The more robust algorithm is a piecewise polynomial transformation, where the data is transformed block-by-block, therefore guaranteeing high accuracy. By default, ArcGIS will automatically use the simple raster transformation method because highly accurate raster transformation methods require more processing time. You can choose simple or robust,by picking ‘Favor rendering speed over accuracy’ in the Options dialog box on the Raster tab.

Geoprocessing - raster projection Data Management Tools -> Projections Best raster projection tool Use resample method Nearest neighbor Bilinear interpolation Cubic convolution Option to specify registration points origin point for anchoring output cells Geoprocessing environment General settings for geoprocessing Applied to all output rasters The Project raster tool found in the geoprocessing tools for ArcGIS, projects a raster dataset into a new spatial reference using a single polynomial fit to compute the adjustment between coordinate systems. You are able to choose a pre-existing spatial reference, import it from another dataset, or create a new one. The output has several parameters that can be set, such as the Resampling technique and Output cell size. The resampling options available are: •The NEAREST option, which performs a nearest neighbor assignment •The BILINEAR option, bilinear interpolation, determines the new value of a cell based upon weighted distance average of surrounding cells. •The CUBIC option, cubic convolution, determines the new cell value by fitting a smooth curve through the surrounding points. It is not recommended that BILINEAR or CUBIC be used with categorical data, because the cell values may be altered. Geoprocessing environment settings When geoprocessing tools are run, default environment settings set for the application are applied to all tools. The General Settings section of the Environment Settings dialog box contains settings that are applicable to most output data types. Included in this section are settings that you can change: for the current workspace, the scratch workspace, the output coordinate system.

Conversion on the fly Many functions accept feature or raster data as input Feature data automatically converted when necessary Non-grid rasters converted to grid for analysis Consider converting compressed data before processing Feature or raster dataset as inputs? Many ArcGIS Spatial Analyst tools will accept either feature or raster data as input. Vector inputs are either used as-is or are automatically converted to raster before the operation executes. If only rasters are allowed as input, the browser will only display raster data. Feature data used directly Certain operations require feature data as input. The surface interpolation functions (like IDW and Kriging) are examples; they perform their interpolations directly against sample point features.

Geoprocessing - raster tools Tools to: Flip raster along horizontal axis. Flip raster along vertical axis. Converts between two coordinate systems. Scale by the specified x and y Rotate around a specified point by a specified angle Shift by specified x and y shift Transform using links The raster geoprocessing tools Flip - flips the grid from top to bottom along the horizontal axis through the center of the region Mirror - flips the raster from left to right along the vertical axis through the center of the region Rescale - Scales a raster by the specified x and y scale factors. The output size is multiplied by the scale factor for both the X and Y directions. The number of columns and rows stay the same in this process, but the cell size is multiplied by the scale factor. Rotate - Rotates a raster around the specified pivot point by an angle specified in degrees. Use rotate when the raster dataset is in the wrong orientation. Keep in mind that the default pivot point is in the lower left corner of the raster dataset, however a different location can be specified. Shift - Shifts a raster by the specified x and y shift values. This tool is helpful if your raster dataset needs to be shifted, to align it with another data file. Warps - warps a raster based upon the input control points using a polynomial transformation. This is very similar to georeferencing using a text file. The user must specify source and target coordinates. Also, the transformation type (polynomial order) to choose from is dependent on the number of control points entered.

Conversion: polygon to raster Convert using string or numeric field Unique attributes assigned value in the output raster Conversion field added to VAT May results in: Loss of detail Smaller cell size — better representation Larger cell size — more generalization Loss of topological relationships 100m Converting Polygon features to raster Converting vector data into a raster data structure is conceptually straightforward, although practically difficult. Polygon features from any type of source file can be converted to a raster using both string and numeric attribute fields. If you are using a string field, each unique string in the field is assigned a unique value in the output raster. A field will be added to the table of the output raster to hold the original string value from the features. The fixed grid of the raster data structure invariably leads to a jagged, or stair-stepped, representation of polygon boundaries which are oriented at an angle to the grid. By forcing real world features into a fixed raster grid, feature boundaries will shift by as much as half the dimension of the grid cells. The typical conversion rule of assigning values is to use the polygon which occupies the greatest proportion of the grid cell. This may however result in the deletion of features which are smaller than a grid cell in either the X or Y dimension. Not only might small patches be deleted, but extensive narrow features such as roads or streams may drop out as well. When converting polygons, cells are given the value of the polygon that contains the centroid of the cell. Therefore serious consideration should be given to choosing an appropriate cell size. If too large, the data will be excessively coarse; if too small, the size of the dataset will slow processing and consume disk space. 400m

Conversion: line to raster Identifies raster cell crossed by the line Codes cells with the attribute value associated with line if more than one value for a cell longest arc used Cell size should be Average width of the linear features Lines Converting line features to raster Line features from any type of source file can be converted to a raster using both string and numeric attribute fields. If you are using a string field, each unique string in the field is assigned a unique value in the output raster. A field will be added to the table of the output raster to hold the original string value from the features. When converting lines, cells are given the value of the line that intersects each cell. Cells not intersected by a line are given the value of NoData. If more than one line is found in a cell, the cell is given the value of the first line it encounters when processing. The fixed grid of the raster data structure invariably leads to a jagged, or stair-stepped, representation of line features which are oriented at an angle to the grid. By forcing real world features into a fixed raster grid, line feature may shift by as much as half the dimension of the grid cells. This may also result in the deletion of line features which are smaller than a grid cell in either the X or Y dimension. Further consideration should be given to choosing an appropriate cell size. If too large, the data will be excessively coarse; if too small, the size of the dataset will slow processing and consume disk space.

Conversion: Point to raster Method: Cell with center closest to point xy - coded with attribute of point NoData assigned if no point available. Cell size — overriding factor Note: Most often interpolate z values for points,not convert Seldom can original points be retrieved from converted raster without loss Converting Point features to raster Point features from any type of source file can be converted to a raster using both string and numeric attribute fields. If you are using a string field, each unique string in the field is assigned a unique value in the output raster. A field will be added to the table of the output raster to hold the original string value from the features. The typical conversion rule of assigning values is to use the point with its location closest to the center of the grid cell. This may however result in input points that are not assigned to a cell in the output. Therefore, when converting points and cells are given the value of the point closest to the centroid of the cell, serious consideration should be given to choosing an appropriate cell size. If too large input points will be lost; if too small, the size of the dataset will slow processing and consume disk space, but more input points will be converted. Keep in mind that after conversion of points to raster, original data point features can seldom be retrieved from the raster data without loss of some data.

Conversion: Raster to feature Raster to polygon Regions vector polygons Cell size controls “blockiness” Deploy raster generalization to reduce “stair-step” effect Raster to lines Stream to feature tool Raster to Point center of cell defines point feature XV Raster to polygon conversion Vector and raster formats store similar GIS data in very different ways. Most often when converting from vector to raster the results are visually satisfactory, but the conversion techniques can produce results that are not satisfactory to the attributes each grid cell represents. It is particularly true along the edges of areas, where the user seldom knows the decision rules concerning how the partial cells are handled. Alternatively, by converting from raster to vector, you may preserve the vast majority of the attribute data, but the visual results will often reflect the blocky, step-like form. The size of the grid cells from which conversion proceeds is an important factor controlling the “blockiness” of the resulting vector. Different mathematical smoothing algorithms can minimize this effect. Methods for mitigating problems in data conversion • Filter raster data prior to performing a conversion to vector data, so that there are no single pixel polygons or ambiguous connections between proximate features. •Use line generalization functions in the ArcGIS after converting a raster file to a vector data structure. This will reduce unnecessary points, but may also distort shapes in undesirable ways if not used with caution.

Georeferencing a raster Image: Not georeferenced Georeferencing creates a relationship between a raster’s coordinate space and another desired coordinate space. For example, the image of a scanned map may have arbitrary coordinates (based on the scanner) that correspond to actual map projection coordinates. Once registered, the raster may be drawn with other geographic data based on the common projection and combined with them in analysis. The process You must be able to provide the correct coordinates for locations on the raster. By comparing the correct coordinates to the current coordinates, the software can develop the transformation coefficients needed to scale and shift the raster. Typically, the “old” and “new” coordinates arc provided as a set of links. You identi’ common locations, like street intersections, that are visible in both the raster and a control layer that is in the desired coordinate space. Using the tools in the Georeferencing toolbar, you construct links between the common points. Once a sufficient number of links have been created, you use the tools to develop the transformation and apply it to the raster. Applying the transformation You can choose to either transform the raster or store the transformation coefficients with it. If you transform the raster, the coefficients are applied, a new raster is created in the new coordinate space, and the coefficients are not stored because they are no longer needed. If you store the coefficients with the raster, ArcMap and other ESRI software will use them to transform the raster on the fly when you load it. Some raster formats, like Arclnfo grids and ERDAS IMAGINE images, store the coefficients internally. Other formats, like JPEG and BMP, store them externally in the AUX file. Vector Data

Georeferencing steps 1. Add Georeferencing toolbar 2. Add Layers 3. Establish Links The Georeferencing toolbar The tools you need to georeference a raster are found on the Georeferencing toolbar (View> Toolbars> Georeferencing). This is part of ArcMap; you do not need the ArcGIS Spatial Analyst extension to georeference raster datasets. Add the control layer You typically create links between common locations visible in the input raster and another control layer (usually a vector layer). Add the control layer, zoom to the extent of the raster, and from the Georeferencing menu on the Georeferencing toolbar, click Fit To Display, which will superimpose the control layer over the input raster. Add links Links identify locations on your input raster that will be shifted to locations on the control layer. You need at least three links to do any kind of adjustment. You digitize the links with the tools on the toolbar. Test the transformation error After setting the transformation order for your raster, you can evaluate the difference between the locations of the links and how close the transformation could get to those locations. With this information, you can delete and add links accordingly until your error is acceptable. Apply or save the transformation You can permanently transform your raster after georeferencing it when you use the Rectjfy command on the Georeferencing toolbar, or you may simply save the transformation coefficients to the raster. ArcMap and other ESRI software will use them to transform the raster on the fly. 4. Assess Accuracy Save Transformation Update georeferencing Rectify

Georeferencing toolbar Component of ArcGIS deployed in ArcMap. Does not require ArcGlS Spatial Analyst Rotate and Shift The Georeferencing toolbar has several tools to perform geometric transformations of an input raster dataset. In addition to georeferencing, you may rotate, shift, and flip a raster. Geometric transformation The geometric transformation functions either change the location of each cell in the raster dataset or alter the geometric distribution of the cells within a dataset to correct a distortion. These functions are generally performed before adding links. Some of the geometric transformation functions are available in the Georeferencing toolbar in ArcMap, and all are available as Map Algebra functions. The geometric transformation functions that alter the geometric distribution within a raster dataset change the count of cells in some areas to correct geometric distortion. Geometric distortion occurs when features in a raster dataset are not located where they should be in the real world. From a known set of real-world coordinates that match known locations in a raster dataset, the raster cell locations can be adjusted to more closely represent reality. There are two groups of geometric transformation functions, translation (Shift) and rotation, which change the location represented by the cells. Translation shifts the coordinates of the raster dataset by a specified x,y offset, and rotation rotates a raster dataset by a specified amount. Shift translates the raster to a new position without changing its size and lets you specie’ the distance to shift the raster in the x and y directions. Rotate lets you rotate the raster by the angle input in degrees around the input pivot point, if specified. If no pivot is specified, the raster is rotated around its center point Flip Horizontal or Flip Vertical and Rotate Left or Rotate Right are special cases of rotation functions. Using Flip, a raster dataset can be flipped in a specified y direction. With Rotate, a raster dataset can be rotated in the x direction.

Establishing links Links used to tie unreferenced raster to geo-referenced source data Requires: At least three links Evenly distributed over the entire raster Choose Link features that will not change position with time Links Links are fundamental to the rectification process. They tie points on your input raster to points that are defined in real-world coordinates on the control layer. The number of links you need depends on the order of transformation you intend to use. You need a minimum of three links for a first-order transformation, six for a second order, and ten for a third order. However, you generally need more than the bare minimum for an accurate transformation. Distribution of links Links should be evenly distributed throughout the entire extent of the input raster. An even distribution of a few links will produce a better alignment than many links concentrated in one area of the raster. Choosing link locations You generally add links between features that are visible in both the input raster and the control layer. The actual features you choose for links are important. Often, you might be trying to align a new raster to an older map. In such cases, it is important that you always try to use fixed features, such as road intersections or corners of buildings, which should not change with time. Features such as field boundaries, stream intersections, and coastal features will all change through time and are not recommended. Reference data 0,0

Assessing accuracy of links The Link Table Shows accuracy of transformation Reports residual error of each link and RMS error for whole image RMS error depends on Raster cell size Accuracy in adding links Assessing accuracy Assessing the accuracy of your alignment is an essential and important step in using the Georeferencing tool. This can be done visually by checking to see if features in both layers align well or not; however, the software provides better mathematical measures of the transformation accuracy. Residual error Each link is individually tested for error. The “from” coordinates are run through the transformation and compared to the “to” coordinates that you digitized. The distance between the computed and actual “to” coordinates is reported in output map units (feet or meters). Links with high residual errors should be deleted. The RMS error The RMS (root mean square) error is the average error of all the links. There will always be some error, but your goal is to reduce the error to a small multiple of the raster’s cell size. For example, if the raster has 10-meter cells, an RMS error of 100 would be considered high. The lower the error, the greater the accuracy of the transformation. RMS= E12 + E22 +…en2 n

Initiating transformation Final stage of alignment Two choices Update Georeferencing Transformation information stored with raster and used when displayed or analyzed No resampling of original data Rectify Raster is resampled New output created Update georeferencing vs. Rectify As the final step in the georeferencing process, you have to decide whether to permanently transform your raster or just create and update the stored georeferencing information. Transform the raster To make the transformation permanent, you must use Rec4fy. Recty creates a new dataset using the current transformation. The new dataset will be rectangular in extent, and therefore will be resampled. New datasets can be created in GRID, TIFF, and Imagine format. Resampling can degrade your data and is not always recommended. Update the georeferencing information Update Georeferencing writes the transformation coefficients into the auxiliary file associated with the dataset. The presence of this transformation information in the AUX file means the data can be transformed on the fly to its correct position for display and analysis. However, because the transformation is stored in the AUX file, only ArcGIS, ArcObjects, and ERDAS Imagine will be able to use this information.

Transformation process Applies polynomial equation to unreferenced raster Source coordinates converted to rectified coordinates Transformation complexity determined by: polynomial order number of links and distortion of source raster 1st order: Linear Transformation Change of scale Change of skew Rotation Transforming the image The coordinates from the input raster are converted to real-world coordinates in the control dataset using a transformation matrix computed from the links. The matrix is a set of coefficients used by a polynomial equation. You may speci’ the order of the transformation. Polynomial equations Polynomial equations are used to transform the input coordinates into output coordinates. The order of the polynomial is simply the highest exponent it uses. The order you speci’ depends on the nature of the distortion in the input raster, the number of links you can provide, and the locations of the links relative to one another. First-order transformation A first-order transformation is linear. It applies differential scale factors in x and y, skews in x and y, and a rotation factor. A first-order transformation with about 12 links is often your best choice unless you understand the distribution of error in the input raster. Second-and higher-order transformations Second- or higher-order transformations are nonlinear and can correct nonlinear distortions. This process is also known as rubber sheeting. They can be used to convert Lat/Long data to a planar projection (not recommended), or for data covering a large area (to account for the earth’s curvature) and with distorted data. Third-order transformations are often used with distorted aerial photographs, scans of warped maps, and certain types of imagery (Radar). In X In Y Original Image 2nd order or higher: Polynomial Transformation

The Rectification process Creates output raster from link positions Resamples source raster Fits source raster to output raster Rectify If you choose to save the aligned image to a new file using the Rectify function, you will be resampling the image. This process involves placing values from the source image pixels into the correct location in a new, aligned grid. The steps used in resampling can be illustrated as follows: The identified control points in the source (unrectified) image (1) are located in an output grid. This output grid is the georectified grid for the image area. The control point locations can also be found in this output grid (2). Next, the grids are overlaid so that the control point locations are aligned (3). Finally, the output grid is populated with values from the source grid. This is achieved using a resampling method. Resampling You can choose from three resampling methods: • Nearest neighbor assignment: This is the resampling technique of choice for categorical data since it does not alter the value of the input cells. Once the location of the cell’s center on the output raster dataset is located on the input raster, nearest neighbor assignment will determine the location of the closest cell center on the input raster, and assign the value of that cell to the cell on the output raster. • Bilinear interpolation: Uses the value of the four nearest input cell centers to determine the value on the output raster. The new value for the output cell is a weighted average of these four values, adjusted to account for their distance from the center of the output cell in the input raster. • Cubic convolution: Similar to bilinear interpolation, except the weighted average is calculated from the 16 nearest input cell centers and their values.