Confident Data Integration and QC with FME Shideh Habibi Senior GIS Analyst 1

Agenda Who we are Abstract Project Introduction Project Overview Project Workflow Quality Control Project Summary Place your logo here

Who We Are Utility Solutions Corp. (USC) founded in 1998 Currently employ approximately 20 staff Office in the City of Toronto - provide services primarily within Ontario Clients are principally mid to larger size utility and government organizations Focus is to provide spatial data management services Over 13 years experience with major GIS and CADD systems, including FME, ESRI, Intergraph, AutoCAD and MicroStation Place your logo here

Abstract USC purchased FME licence in 2009 We have used FME to perform asset type data transfers between different GIS data formats such as CADD and GIS The existing tools in FME are very useful for data analysis and quality control Our primary usage of FME is for automated QC, data conversion and data extraction at a low cost with a high level of confidence Place your logo here

Cranbrook Project Introduction Municipal Project for the city of Cranbrook, BC. Population is about 20,000. The city is about 30 km2. Focused on data conversion and creating shape files from the city’s existing data. Field work was performed to obtain road condition data. Place your logo here 5 5

Cranbrook Project Overview Input data Shape files AutoCAD files Access files Workflow Data conversion Georeferencing Data overlay Quality Control Output Shape file Excel file for MDW (Municipal Data Works) Place your logo here

Cranbrook Project Overview (Cont.) Texts Points Blocks Lines Polygons others Shape file AutoCAD file Excel file FME Projected shape files Place your logo here

Cranbrook Project Overview (Cont.) Data Integration (Input Data) Road data was received in shape file format and projected in the “NAD83/BC Albers” projection. AutoCAD engineering files were provided without proper coordinates for water main, sewer, sanitary and city boundaries Road condition assessment information was delivered in Access database. Place your logo here

Workflow (Cranbrook Project Boundary) Created vector file Used AffineWarper tool in FME to overlay the AutoCAD file on the road centre line Clipped the road centre line shape file inside the city boundary. Place your logo here

Cranbrook Project Workflow ( Transfer Tabular Data) Extracted GPS coordinates of file images Linked GPS coordinates to the road condition Access file 2D points replaced Neighborhood finder tool used to join the attributes Segmented the road data

Georeferenced AutoCAD Data To georeference the AutoCAD files, vector files were created by point to point method to define the correct projection for the AutoCAD data.

Data Extraction From AutoCAD Files Water mains Manholes Sanitary lines Storm lines Year of installation Diameter Pipe length Pipe material

Project Workflow (Cont.) FME workbench for data extraction and conversion

Project Workflow (Cont.) AutoCAD data type: Text strings Blocks Lines Circles Polygons

FME tools

Project Workflow (Cont.)

Project Workflow (Cont.)

Quality Control Data is the foundation of every successful GIS. To ensure a reliable foundation for their GIS, organizations should have a well-designed quality assurance (QA) plan and quality control (QC) procedures integrated with the production and maintenance of GIS data. Quality control is a process employed to ensure existence of a certain level of quality in a product. It may include whatever actions a business deems necessary to provide for the control and verification of certain characteristics of a product.

Quality Control (Cont.) Major items QC’d for Cranbrook project: Geometry Missing features Gaps in the attribute table Attributes (Diameter, Location, Material) transferred correctly Check to ensure spatial locations are correct

Quality Control (Cont.) The QC of the files showed that if the FME tools are selected properly the manual work will be reduced by a factor of 70 - 99% Human eye cannot detect all the features that FME can easily capture through one process Visualizer tool used during the process helped us to build up the workbench step by step FME Tools such as geometry filter, attribute filter and expression evaluator were used to QC the data

Quality Control (Cont.) Complications in the workflow: Gaps in the attributes Limitation in the buffer Structure of AutoCAD files Wrong layer name Unknown coordinate system Mismatched data values in different files

Project Summary One database for all the assets All the assets can be easily analyzed and reviewed Original information was spread out amongst disparate digital and paper formats. The data created in GIS helped the municipality manage: Road Assessment Condition Asset excavation reports Budgeting and road maintenance planning. Using FME tools is an efficient and cost effective manner. 22

Thank You! Questions? For more information: Shideh Habibi Utility Solutions shabibi@utilitysolutions.ca www.utilitysolutions.ca