Data Exchange Needs Using ETL Tools

Overview Client Project Issue Methodology Schedule & Budget Results & Recommendations Challenges Acknowledgments Questions

Ministry Of Transportation

Project Issue & Primary Goal Current data conversion process o Each time data is updated it needs to be redone manually o Need to streamline the process o Extract Transform Load (ETL) Tools Safe Software Feature Manipulation Engine (FME) Esri ArcGIS Geoprocessing Tools (Data Interoperability Extension)

Methodology Standard Methodological Approach: Identify objectives, issues & goals Research topic and best practices Identify scheduling, budget, risks Communicate & Conduct Meetings Gather information for evaluation, reporting and decision making Report findings

Streamlined data conversion process User know-how ETL tools capabilities and limitations ETL tool decision making Final Report and recommendations Goals & Deliverables

Extract Transform Load ETL - Traditional ETL is used to migrate data from one database to another, to form data marts and data warehouses and also to convert databases from one format or type to another. Spatial ETL – Safe Software Feature Manipulation Engine - FME Provides the data processing functionality of traditional ETL, but with a primary focus on the ability to manage spatial data (the geometry, attributes, and coordinate systems of spatial datasets)

Oracle SQL Bridge Date xls ArcSDE ArcGIS Server ArcGIS Desktop Current Process

MTO Proof of Concept

FME Output

FME Server FME Desktop is the authoring tool for FME Server Any spatial ETL workflow created in FME Desktop can be published to FME Server FME Server is a scalable processing environment that allows multiple users via the web Workflows can be shared and automated in this environment

“Safe Software sets the standard for spatial data interoperability, so it was an obvious choice to use their FME technology for Esri’s ArcGIS Data Interoperability Extension.” – Jack Dangermond, Esri President ArcGIS Desktop Data Interoperability Spatial ETL

ArcGIS Output

Arc Server

Evaluation Architecture ETL Functionality Ease-of-Use Reusability Debugging Real-time Connectivity General ETL tool characteristics

Challenges GIS Expertise, knowledge and training Assumed licenses acquired and ready to use Data availability and ability to work on data Client Representation

Recommendations Streamlined data conversion process User know-how ETL tools capabilities and limitations ETL tool decision making

Acknowledgements Client o Laura Kingston Project Thesis Adviser o Mike Wallace Technical Advising o Colin Bissell o Jonathan Lepage o Allison Hughes FME Software Acquisition o Gerry McCoubrey o Craig Vernon

Questions