1. INFORMATION TECHNOLOGY FOR MINNESOTA GOVERNMENT Geoprocessing in ArcGIS Fall 2015 GIS Support Team MN.IT Services @ Department of Natural Resources

2. Welcome and Introductions What is geoprocessing?  An overview of ArcToolbox geoprocessing tools and environments  An overview of ArcMap geoprocessing tools and environment What are the common geoprocessing tools?  ArcGIS Geoprocessing menu  MN DNR Toolbox Geoprocessing with multiple tools  Using ModelBuilder

3. What is geoprocessing? Extracting Data Overlay Layers Proximity Analysis Statistics How do we geoprocess? Tools and toolboxes Models ( ModelBuilder) Python (scripts)

4. The Geoprocessing Menu Access common tools some proximity, extract, overlay Access to search window, system tools, environment settings, and results Access to ModelBuilder and Python too!

5. ArcToolbox  System or Custom Toolbox  Tools and Toolsets  Scripts  Models

6.  Search

7.  Search Keyword  Item Description (Right Click)  Use Smart Search Dropdown 7

8.  Great documentation for tools  right click on tool  “show help” or “tool help” buttons

9. Workspace management  Data storage (inputs & outputs for tool)  Geodatabase is best choice for ArcGIS Geoprocessing … Environment settings

10. Common Options: Workspace Processing Extent Output Coordinates Others …

11. Set for ArcToolbox ArcToolbox Context Menu Set in Tool Properties over-rides ArcToolbox settings

12.  Useful to track output names and errors/messages

13. Analysis Tools Geoprocessing Menu Common Tools

14. Analysis Tools” Toolbox  Extract  Overlay  Proximity  Statistics

15. Clip : Clip one feature class by the extent of another feature class (need two) Select : Similar to Select by Attribute, but creates a new feature for you Split : Split one feature class into new feature classes based on input feature Table Select : Create a subset table that you selected using SQL Extract Toolset

16.  Extracts input features that overlay the clip features.  Area attributes are not updated in shapefiles, they are in geodatabases

17. Split the Input Feature into multiple output features. Example: split watersheds by district boundaries to identify area (acres) of watershed in each individual district.

18.  Creates a new feature of the selection results  Use Expression to “extract” from input

19.  Similar to Select Tool  Creates new table, not new feature

20. Open C:\Student\DNRTraning\Geoprocessing\Geoprocessing.mxd Set ArcToolbox environment settings –Set Output to our class folder & the ExerciseData.gdb –Set Extent to the same as the HUC10 layer Use Select Tool to select Mille Lacs County –SQL statement required! Use Clip Tool to clip HUC10 layer using Mille Lacs County –Call output MilleLacsHUC10 Bonus: –Using the layers available, try the Split Tool and review the output feature classes (e.g. split HUC10s by Counties) –Our output features are in a geodatabase so the Shape_Area and Shape_Perimeter attributes are updated. –We don’t have ACRES. Can you create a new field and Calculate the Geometry of that new field to create acres? (this was covered in Introduction to ArcGIS class) ask for help? –How can you use the Select Table tool with the available data layers to create a new table of a subset? (e.g. select just those split watersheds by a specific county) Extraction Exercise

21. All overlay tools require two feature classes Create output feature class as a result of overlay operation The tools are described below Intersect and Union are most common, therefore added to Geoprocessing menu! Overlay Toolset (Vector)

22.  Creates a feature class by overlaying the Input Features with the polygons of the Erase Features.  Opposite of Clip. Input Feature geometries coincident with Erase Feature geometries will be removed.  Input Features can be point, multipoint, line, or polygon. 22

23.  Computes a geometric intersection of the input features and identity features.  The input features or portions thereof that overlap identity features will get the attributes of those identity features.  Input Features can be point, multipoint, line, or polygon.  Identity Features must be polygons or have the same geometry type as the Input Features. 23

24.  Features or portions of features which overlap in all layers and/or feature classes will be written to the output feature class.  Input Features can be point, multipoint, line, or polygon.  Attribute values from the input feature classes will be copied to the output feature class.

25.  Join the attributes from Join Feature to the Target feature *Add watershed attributes to well points in that watershed  A new feature class is created with the joined attributes Copies the geometry of the Target but adds the join feature’s attributes  Join Operation & Fields controlled here

26.  Features or portions of features in the input and update features that do not overlap will be written to the output feature class.  The input and update feature class or feature layer must be of the same geometry type.  Attribute values from the input feature classes will be copied to the output feature class.

27.  Computes a geometric union of the input features.  All features and their attributes will be written to the output feature class.  Only polygon as input features!  Output will contain areas of overlap and not overlap.

28.  Computes a geometric intersection of the Input Features and Update Features.  The attributes and geometry of the input features are updated by the update features in the output feature class.  The Input and Update Features must be of type polygon.  The input feature class and update feature class field names must match.

29. Open C:\Student\DNRTraning\Geoprocessing\OverlayMXD.mxd Set ArcToolbox environment settings Use Intersect Tool – Where do the flood zones and parcels intersect? Use Union Tool – What happens when you union parcels and flood zones? Bonus Challenge: – Using the layers available, use the Spatial Join tool Vector Overlay Exercise

30. 15 Minutes BREAK TIME

31. Common questions for GIS: What is nearby my feature? How far away is that other feature? What is the closest feature? Location Matters! Proximity Toolset

32.  Creates buffer polygons around input features to a specified distance.  Can be used with points, lines, or polygons.  Specify the dissolve type:

33.  Creates multiple buffers at specified distances around the input features.

34.  Calculates distance and additional proximity information between the closest features (X, Y values, near angle value)  The Near Features can include one or more feature classes of different shape types.  “ID” attributes added to the Input feature

35.  Measure between point feature classes  Calculate the distance of inputs to closest source  Output is a table *What is the closest DNR Region Headquarters to a verified county well?

36.  Recall the Join by Spatial Location Tool …  Recall the Select by Location Tool …  These are Overlay AND Proximity Tools!  Discussion: How is the Spatial Join similar to near and point distance, yet different? How is Select by Location similar to Buffer, yet different?

37. Open map ProximityMXD.mxd Use Buffer Tool – Buffer Nuclear Plants by 50 miles (the response zone) Use Select by Location Tool – Select Wildlife Management Areas within one of the plant buffers Bonus: – Use the Near Tool to determine how far each plant is to the different Region Headquarters Proximity Exercise

38. Extract statistics from given input attributes Must have attributes to count and summarize by Some statistics are generated by input values (e.g. percent) Statistic Toolset

39.  Example: How many wildfires are caused by arson?

40.  Example: How many total acres?

41.  Recall Field Context Menu

42.  Identify where two feature classes intersect and cross- tabulate the area, length, or count of the intersecting features  Percent of original input are provided in output table

43. Open map document, set tool environments. – How many total acres of wildfire are caused by arson? Use Summary Statistics Tool – Which HUC8 has the most HUC10s? Use Frequency Tool. Bonus: – Use the Tabulate Intersect Tool to determine what percent of each watershed is in each DNR Region. Statistic Tools Exercise

44. Hundreds of tools! We’ll explore General Data management tools toolbox

45.  Combines multiple input datasets of the same data type into a single, new output dataset.  This tool can combine point, line, or polygon feature classes or tables.  Unless specified, all the fields from the input feature datasets will be brought into the new dataset.

46.  Append will combine multiple input datasets into an existing target dataset, while Merge will create a new output dataset.  When using Append, set the Schema-Type to NO_TEST.  Why? If the input features don’t have the same fields and if the Schema Type is set to TEST, the Append operation will fail.

47.  Reorders, in ascending or descending order, records in a feature class or table based on one or multiple fields.  Unlike sorting records in the attribute table, the reordered results of the Sort tool are preserved and written to a new dataset.

48.  Example – Major Cities Sorted by the Name field

49.  Aggregates features based on specified attributes.  Creates new output dataset of dissolved features.

50. What if the Division of Water administrative units were organized by major watershed and not counties?  Open WatersDivisionMXD.mxd  Set ArcToolbox environment settings  Spatial Join the district name to major watershed  This adds the waters district name to the watershed  Dissolve the watersheds by district name  This creates a new polygon feature class by district name  Create map of new Water District Administrative boundaries (add labels) – compare to old boundaries!

51. 60 minutes LUNCH BREAK

52. Saving favorites (creating your own toolboxes) Spatial Analyst Tools … And Some Raster Analysis Tools MNDNR Tools V10 Custom Geoprocessing Tools

53.  Save your favorites in a personal toolbox  Access in the ArcCatalog Window  Can add your toolbox to ArcToolbox  Right click on ArcToolbox name, and add Toolbox

54. Spatial Analyst Extension Raster Geoprocessing Toolsets In ArcMap: Turn Extension On, Add Toolbar Spatial Analyst Tools

55.  A raster is composed of equally sized square cells arranged in rows and columns  Each cell contains a value representing information  The size of the cell is the resolution.

56. Factors to consider when specifying cell size: Resolution of input data Application and analysis to be performed Size of the result compared to disk capacity Response time (processing time)

57. Cell Size of Raster Data  Smaller cell size  Higher resolution  Higher feature spatial accuracy  Slower display  Slower processing  Larger file size  Larger cell size  Lower resolution  Lower feature spatial accuracy  Faster display  Faster processing  Smaller file size

58.  GDRS Elevation Data – access with Quick Layers!

59. Clip  Creates a spatial subset of raster: Data Management Tools > Raster > Raster Processing > Clip Extract by Mask  Extracts cells of the area defined by the mask: Spatial Analyst Tools > Extraction > Extract by Mask

60. Contour Creates a line feature class of contours (isolines) from a raster surface.

61. Hillshade Creates a shaded relief from a surface raster by considering the illumination source angle and shadows

62. Create Contour  Create individual contours lines at specific locations on a single-band raster, TIN, LAS, or Terrain dataset. Create Histogram  Enables you to explore the distribution of values in a raster layer.

63. 63

64. Created by MNDNR for our users Some of them are division-specific Tip Sheet on intranet! We’ll look at a few… MNDNR Tools v10

65. Clip DEM and Create Contours  Clips a DEM based on an input polygon and then creates contours for the area.  Convert the Z-units of the grid from meters to feet (option) can also use:  Project the DEM from to any coordinate system supported by ArcGIS (option)

66. Calculate Surface Area (acres)  This tool calculates the slope of each cell then calculates the true surface area in that cell.  The output raster has cell values that represent the true surface area in acres, while the output table sums those areas.

67. PLS to XY Coordinates tool  Calculate X and Y coordinate (in UTM and or Lat/Long) for a table with a list of PLS location  Gives the X,Y coordinate for the center of the PLS location.

68.  This tool calculates spatial attributes for a polygon, line, or point feature class.  Shape attributes include, perimeter, area in coordinate units, area in acres, centroid, x,y, length

69.  Cross Tab Report  Creates a summarized report based on specified fields  Output is a text file  File added to the data frame

70.  List the data source for all layers in the current map document.  Press OK button to run

71. Sort and Calculate Unique IDs (model)  Sorts the table by the field that you specify,  Creates a new ID field, and  Assigns sequential number for the new ID field by the interval that you specify.

72. Open MNDNRToolbox.mxd Set ArcToolbox environments Clip raster by study area polygon and create 2 foot contours (MNDNR Tool) Generate points from a PLS table –BLM_Islands.dbf table stored in Geoprocessing ExerciseData.gdb Sort and calculate the attributes for the new table – e.g. sort by county name then give each island a unique ID Try another tool of your choice! Explore. MNDNR Toolbox Exercise

73. 15 Minutes BREAK TIME

74. Application you use to create, edit, and manage models. Models are workflows that string together sequences of geoprocessing tools, feeding the output of one tool into another tool as input. It is visual programming! Automate the geoprocessing routines that you keep using again and again! What is ModelBuilder?

75.  Create new model in a toolbox (My Toolboxes)  Add model elements (data and tools)  Connect elements (output of process one is input of process two)  Rename the elements and adjust the layout (optional, but highly recommended)  Validate the model  Run the model  Revisit errors and settings  SAVE OFTEN  Share model (with tool input parameters, or as a graphic)

76.  Open ModelBuilder from the Standard Toolbar or the Geoprocessing menu

77.  Model elements are the basic building blocks of models.  The diagram shows how model elements are symbolized in ModelBuilder

78.  A model process consists of a tool and all variables connected to it.  There will often be several processes within a model. These processes can be connected together so that the output from one process becomes the input for another process.

79.  Each process in a model has four process states Not Ready-to-run Ready-to-run Running Has been run

80. There are two ways to add data or tools to the model  Go to Insert > Add Data or Tool. Browse to and choose the data or tool and click Add.  Drag and Drop data or tools from the Catalog window, ArcToolbox, Search window, Table of Contents…

81.  Inside the model, double-click a tool to open the tool dialog box. Use the drop down to add layers in your current map. Or folder icon to browse to add data.

82. There are two ways to connect the data with the tools 1. Use the Connect tool by clicking on the Connect tool on the ModelBuilder toolbar.

83. 2. The second way to connect the data is to use the tool dialog box.

84.  If you fill in all the required information in the tool dialog window, connections are automatically created.  Double-click the tool to open  OR Right Click > Open

85.  To change to appearance of your model open Model menu > Diagram Properties  The easiest way to cleanup the layout is to use Auto Layout button.

86.  You can change the name of an element to make it more user-friendly and reusable by right-clicking and selecting Rename

87.  Adding Labels can give users more helpful information  Labels are not really part of the model  Add labels from Insert > Create Label

88.  Once the model is ready-to-run, you should validate the model  Model validation means verifying that all data elements and parameter values are acceptable.  You can use the Validate tool button or click Model > Validate Entire Model  If no error message, you are ready to run the model!

89. There are three ways to run a model from within ModelBuilder  Run Single tool – Right-click on a tool and select Run.  Run Ready-to-run tools – Click Run from the Model menu or the run tool on ModelBuilder toolbar. All tools that are in a ready-to-run state will run.  Run entire model – Click Run Entire Model from the Model menu. This runs all tools that are in a ready-to-run state. The model can also be run as a tool in the Catalog window

90.  When the tool is red, the model is that particular tool running  Once the model is run, the model element (tool and output feature) will have drop-shadow  If the run is successful, you get the “Completed” message

91.  Click the Save button on the ModelBuilder Standard toolbar. Or select Save from the Model menu.  Models must be saved in a Toolbox. Navigate to a current Toolbox or click New Toolbox on the navigation window.  Name the model and click Save.  No spaces are allowed in the model name. However, the label is displayed in the Catalog window and can contain spaces.  To open the saved model for editing, right-click the model tool in the Catalog window and click Edit.  Always remember to save and save often!

92. Create a new model in your “My Toolboxes” Try at least two different geoprocessing tools that we’ve covered in class and exercise today. Your choice! Practice setting the parameters for the tools in the model, as well as running the model ModelBuilder Exercise

93. Models can be used as tools in a toolbox Set parameters for the model – Parameters request the user to provide an input file, output file and or other tolerances to perform the tool, such as buffer distance. Models can be easily shared (Toolboxname.tbx) Export model as graphic! Document your analysis… Sharing Models

94.  Wouldn't it be nice if you could run the model using its tool dialog box? Just supply the name of the input datasets, like you would with a system tool?  But when you open the tool dialog box, it displays This tool has no parameters, so there is no way for you to enter the datasets you want to process.

95.  To add variables in the tool dialog box, you must create Model Parameters  Model parameters are things that can be changed. Once variables are converted to model parameters, you can see them when you open a model tool.  Any model variable can be made into a model parameter.  To create a model parameter, right-click a model variable and choose Model Parameter.

96.  In the model, a letter “P” appears beside the variable, indicating it has been made a model parameter.  When a variable is made into a model parameter, its value can be specified in the model tool dialog box.

97.  Values can also be exposed as model parameters.  First, the value must be set as a variable.  Right-click on the tool and select Make Variable > From Parameter > desired value  Then, it can be set as a model parameter.  Right-click on the variable in a model, select Model Parameter

98.  The output result from the ModelBuilder doesn’t get added to the TOC automatically  Right-click on the output and select Add to Display

99.  Since models must be saved inside a toolbox, they can be e- mailed to your co-workers!

100.  You can also share your model as a picture.  Insert into a presentation or report to document your analysis process.

101.  You can also share your model as a python script.

102. Export your model as a graphic and insert it into a Microsoft Word document (or power point?) Export your model as a python script – Browse to the location saved – Right click and choose Edit with IDLE ModelBuilder Sharing Exercise

103. 103 Additional Resources Courses –Introduction to ArcGIS –Geoprocessing in ArcGIS –Editing in ArcGIS Webinars –See posted schedule Support requests

104. For assistance: –Email mnitservices.dnr@state.mn.us and include the keyword GIS in subject line mnitservices.dnr@state.mn.us –DNR Intranet pages http://intranet.dnr.state.mn.us/gis/i ndex.html Thanks for attending!

105. MN.IT Services @ Department of Natural Resources