How to use a Spatial ETL Model to Transform an Updated CAD Drawing

How to use a Spatial ETL Model to Transform an Updated CAD Drawing
American Sentinel University GIS515 Final Project Christin S. Huot July 13, 2014

In order to show someone else how to use a spatial ETL model to transform an updated CAD Drawing it is first necessary that one understands what the main purpose of a spatial ETL (extract, transform, and load) capability is and what it is used for.

Spatial ETL tools are designed to allow a user to overcome the challenges of transferring and transforming data form one format into another format; which is often referred to as data interoperability. Spatial ETL tools therefore make it possible for users such as planners, executives, and the general public, to work with spatial data so that it can be used in analysis, visualization, and planning purposes.

Therefore a spatial ETL is designed to accomplish three purposes: 1. Extract the data from its source 2. Transform the data as required to make it usable 3. Load the data into the destination view or dataset

First, extraction is the process where the spatial ETL tool “gathers the source data for processing, copying it from its original location. This ensures that when transformation is performed, it can be done without impacting existing infrastructure…Transformation is the core functionality of spatial ETL, as it is the process that converts the data into the format, structure and coordinate system that end users require…Once data has been extracted from its source, converted into the required format, data model, and coordinate system, and even integrated with data from other sources, it is time to the load the data. The data can be loaded into either a destination dataset or a data view” (Safe Software). Therefore, in our example we will be extracting and transforming a CAD (computer-aided drafting) file into a file geo-database (FGDB) format so that one can load it and use it in our GIS.

In this regards, there are many different spatial ETL software programs available in which a user might want to use. It just depends on one’s level of comfort with that program and whether or not they are looking for a commercial or open source solution in order to meet their needs. For this particular demonstration, we will be using ESRI's Data Interoperability Extension.

ESRI’S Data Interoperability Extension “enables ArcGIS for Desktop users to easily use data in many formats…adds additional format support for more than 110 GIS, CAD, BIM/3D, and database formats for reading and writing… Quick Import and Quick Export tools allow automated data translation between formats… Use the Workbench application, an integrated semantic translation engine, to create custom format definitions and data transformation tools that easily support your complex data models… Enjoy full integration with the ArcGIS geo-processing environment including the Model Builder framework.” (ArcGIS Data Interoperability, 2014)

TOOLS NEEDED: 1. ESRI ArcGIS Desktop 10 or 10.1 and Data Interoperability Extension. 2. Bentley View software package. One must sign up for a free Bentley account and download the free Bentley View V8i software from their website. 3. The source CAD drawing file site_healthcenter.DGN. 4. A computer with Memory/RAM of a minimum of 2 GB (Giga-bytes).

INSTRUCTIONS

First, we will need to setup the Initial Transformation Workspace. a. Open Arc catalog and set up your workspace. b. Create a new toolbox. c. Right Click and choose select new Spatial ETL Tool. d. This will open up the create translation workspace wizard this is where you will define the initial configuration options for your transformation model.

Select the format of the source data. a. In our case, is a Micro station Design File (or .DGN). b. Click on the ellipses button (“…”) and find the right source data format. c. Choose “Bentley Micro station Design (V8)” and click O.K. d. Click on next to go to next screen

Setting format specific parameters for the source data file. (extracting our data): a. Click on the ellipses button (“…”) choose the source CAD .DGN file b. Click on Parameters c. Choose “Level Names” d. Click the “OK” button to go back to the “Locate Source Data” window. e. Click next

The next step is to define the destination file format for the transformation, which for us is the ESRI Geo-database format. a. Click on the ellipses b. Type “ESRI” in the search Field. c. Highlight the “ESRI Geo-database (File Geo-database Arc Objects)” format. d. Click OK and then click next to go to Specify Writer Parameters window e. Click next f. Choose static schema g. Click finished. h. The Select Feature Types dialog window will open then select Ok (here we are able to see all the individual Levels in the site_healthcenter.dgn CAD file)

Now our initial workspace is setup. Therefore we have extracted the site_healthcenter.DGN file and have defined the file format in which we intend to use. As a result, this “kicks off a process that combines the source CAD drawing file and your choice of destination format into an ESRI Data Interoperability Workspace within the Data Interoperability Extension. Arc Catalog manages all of the software processes to configure and manage this workspace, as well as integrate it with other ESRI tools such as Model Builder and ArcGIS Server. From this point on, the transformation process is entirely one of configuring the workspace to provide a desired output structure.” ( /intro/M2%20Lab.pdf)

The workspace now provides us with the ability to view the source data format (reader) and destination data format (writer); as well as shows us the individual parameters. In addition to this, we are presented with the coordinate system in which we can configure according to our preference. Also we can view both the published parameters and other configurable parameters for the source and destination data formats in our tree structure. Furthermore, this is where one can select and choose transformers and create bookmarks to use in the model.

During this part of the instruction; it is not my intention to repeat the whole process that I have already created, but rather discuss how one can transform and maintain the workspace that they will be using.

In this regards, it is important during the spatial ETL process, to clean up the workspace and make it more organized. In doing so a user should group their data into bookmarks because as the data transformation model increases in complexity so will the ability to understand the individual transformations in the data model.

Another way one can clean up the workspace is by also running the transformation to the visualizer. By doing this “you will find reporting on the features read, the features written, whether the translation was successful or not, how many errors or warnings were encountered, the session duration (time the transformation was running), CPU usage, and memory usage” ( n/intro/M4%20Lab.pdf). By performing this operation it allows a user to further cut down on the number of transformation processes which are taking place in the workspace.

In addition to this, another way in which one can further clean up their workspace is by getting rid of any unused readers, writers or transformers. Keeping unused items such as these will result in one’s workspace writing out blank feature classes into the output file geo- database.

Therefore since this is something we would rather prevent because it may cause problems to our transformation one should group these unused objects together and create another bookmark and label it disabled and place all of the unused items into it. “The goal is to use only those input readers, transformers, and output writers that are absolutely necessary to transform the data the way we want” ( n/intro/M5%20Lab.pdf). Doing this will allow us to still have access to these items if we should need to use them in future processes.

Another important step in an effort to further cleanup our workspace would be to explore the CAD data with the Visualizer. Exploring our data in the Visualizer will let us determine other information about our transformation and the source CAD data that we are using for this project. This is a very important step because this will allow us to further reduce the amount of work that will be required when we try to convert our CAD data into our file geo-database

An important note to make here is that CAD data is represented differently than GIS data formats. ” CAD, or “computer-aided drafting”, is a data type specially designed for engineers, architects, and anyone else focused on drawing with clarity and precision… CAD formats involve concepts like styles, symbology, dimensions, labels, and specific geometries like splines and blocks/cells. Example formats include AutoCAD DWG and Micro Station DGN.” Whereas, “GIS formats involve concepts like data models, attributes, simple geometries, and domains. Example formats include Esri Shapefile, Esri Geo-database, and GML” ( cad-to-gis-starts-preschool/).

Therefore with this in mind, in order for us to transform our CAD data into a GIS data format we must transform the CAD data into a different structure so that it will be able to support spatial information and attributes. In order to accomplish this we must do several different things.

First, we need to validate our CAD data; this means fixing items which may affect the quality of the data. Second, we need to manipulate the CAD data by using any of the transformers that are available in our workspace so that we can get it to fit into the GIS format we are using. Third, we will need to take our CAD information such as the labels, text, blocks, dimensions, styles and symbols and interpret them into GIS attributes. Fourth, we will need to geo-reference our CAD data by identifying its coordinate system and reprojecting it from its local grid. With this in mind, it may be necessary to spend some time looking over the other areas in the visualizer to see where else one could clean up their data.

Note, one should keep in mind how you want to display the features in the output FGDB. Afterwards you will be ready to create your first-pass FGDB.

“In most CAD software there are things called Tags (DGN) or Attributes (DWG) that can be set on a drawing element in the CAD file. Fortunately, this has already been done with our health center DGN file, so we can go back into Bentley View and see what those tags look like. We can see that our manholes have two Tags that duplicate the label elements held in the C- STRM-IDEN level – ID and ELEV. What we have to do now is figure out how to access this Tag information in our spatial ETL transformation model” M8%20Lab.pdf. ” Therefore open up Bentley View and open the health center DGN file, and review the two tags that are listed for the manholes in the C-STRM-IDEN level. They should be ID and ELEV.

Then go back to the workbench and zoom into the “Storm Water Drains and Manholes” bookmark group. Make room for another transformer. Then run the transformation to the visualizer.

Disable all the layers except the C_STRM_MHOL polygon layer. Now, activate the identify tool and draw a bounding box around one of the manhole features. The feature information about it will now be viewable.

Next, review the feature attribute panel and view both the ELEV tag and the ID tag from the source CAD file. Then return back to the workbench window. Locate the AttributeExposer transformer in the transformer gallery and drag an instance of it on to the workbench window.

Next, connect the transformer to the process path which will connect the C-STRM-MHOL Reader output port to the GeometryFilter INPUT port.

Click on the red exclamation button in order to open the AttributeExposer Properties window. Type in the ID and ELEV attributes and add them to the list under the title “Attributes to Expose” in the parameters window. Then click OK to accept.

Open the AttributeExposer attributes list.

Next, open up the feature type properties window and go to the user attributes window and enter the two manhole attributes.

Then, open the area and ellipse attribute lists and connect both the ID and ELEV to the MH_ID and MH_ELEV input ports on the C_STRM_MHOL_polygon Writer.

Then repeat these steps for C_STRM_MHOL_line Writer. As a result the MH_ID and MH_ELEV will should show up as valid attributes in the output data.

In this manner, if one follows the instructions that I have developed throughout this course in how to use a spatial ETL model to transform an updated CAD drawing of the health center campus then many users should be able to use and manage the solution on their own with minimal assistance from me or my company.

References: American Sentinel University, (2014). Retrieved from: o/M1%20Lab.pdf American Sentinel University, (2014). Retrieved from: o/M2%20Lab.pdf American Sentinel University, (2014). Retrieved from: o/M3%20Lab.pdf American Sentinel University, (2014). Retrieved from: o/M4%20Lab.pdf American Sentinel University, (2014). Retrieved from: o/M5%20Lab.pdf American Sentinel University, (2014). Retrieved from: o/M6%20Lab.pdf

References: American Sentinel University, (2014). Retrieved from: o/M7%20Lab.pdf American Sentinel University, (2014). Retrieved from: o/M8%20Lab.pdf ArcGIS Data Interoperability, (2014.) Retrieved from features Safe Software, (2014). Spatial ETL: Making Data Accessible. Retrieved from: Safe Software, (2014). Spatial ETL, SDI’s and INSPIRE. Retrieved from: Warner, T., (2013. How Converting CAD to GIS Starts in Preschool. Retrieved from:

How to use a Spatial ETL Model to Transform an Updated CAD Drawing

Similar presentations

Presentation on theme: "How to use a Spatial ETL Model to Transform an Updated CAD Drawing"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

How to use a Spatial ETL Model to Transform an Updated CAD Drawing

Similar presentations

Presentation on theme: "How to use a Spatial ETL Model to Transform an Updated CAD Drawing"— Presentation transcript:

Similar presentations

About project

Feedback