ENOVIA SmarTeam Data Loading Strategies Marc Young, CEO xLM Solutions, LLC

Strategies to ease the migration pain… Loading, or migrating, data into ENOVIA SmarTeam can be a daunting task… With diligent planning and thorough testing, the process can go smoothly

Data Clean-up and Process Review Data Formats Data Loading Rules Migration Cost Estimation 1. Pre-Migration Data Analysis Bulk Loader Import Tools Custom Scripts 2. Data Loading Options Testing the Migration Process Data Verification Pre-Data Load Planning Production Data Load 3. Execution

Analyze the Data to be Loaded Determine legacy data nature and location File-/Folder-based Legacy systems (databases, vaults, etc.) Hard copy (e.g., aperture cards) Type of data: CAD (with internal links), office, etc. Anticipate data access issues All sources available? Third-party systems accesses? Additional experts? Additional resources? Is CAD data conversion needed? e.g., ProE → CATIA V4 → V5 Data volume and estimated time to load Will/Can downtime be scheduled for data loading? Data Analysis

Analysis: Data Clean-Up Opportunity for Data Clean-Up Typically involves human intelligence May be time consuming May not be worthwhile

Analysis: Data Clean-Up What types of data clean-up are required? Duplicated part numbers per file Duplicated file names Missing or incorrect file attributes Missing or incorrect file associations or links File revision not current Non-current parts in current assemblies Non-compliance or old revision/versioning scheme Validate data accuracy

Analysis: Process Reviews and Updates An opportunity to review business processes Opportunity to evaluate and update user experiences Important: new changes to the underlying data model must be verified with the migration processes

Analysis: Data Formats CAD Data File-based Typically has relationships among files Attributes and properties contained within the file Meta Data to be Loaded or Added Text description of data external to the associated (CAD) file Users Import from existing Active Directory Lookup Lists Projects File location pointers Combined Data CAD data with embedded relationships; but… Attributes and properties reside outside of the CAD file e.g., in an Excel spreadsheet Construct a link between the CAD file and attribute data CAD Data Meta Data to be Loaded or Added Combined Data

Analysis: Determine the Data Loading Rules What are the rules to determine where the data goes? How will data fields be mapped from legacy system to SmarTeam? What state(s) will the data be loaded to? Will the data be linked to other SmarTeam data? How to handle the cases with missing data? Which data revision to load: latest or full history? Should the data be located in a folder? Should a folder be automatically created for the data? Text, Lookup, Reference To Class, URL If so, what is the ruleset? If a full data history is desired, determine how the revision history is maintained in the legacy system

Analysis: Effort and Cost Estimations Data Load Project Cost Quantification Time (data loading tools development and testing) Price Resources involved Impact of downtime for users

Data Clean-up and Process Review Data Formats Data Loading Rules Migration Cost Estimation 1. Pre-Migration Data Analysis Manual Processing SmarTeam Import and Export Tools Custom Scripts (executables, add-ins, etc) Combined Manual Processing and Custom Scripting Pre-Developed Data Migration Programs 2. Data Loading Options Testing the Migration Process Data Verification Pre-Data Load Planning Production Data Load 3. Execution 2. Data Loading Options 3. Execution

Overview SmarTeam Bulk Loading Tool Load CAD File data that SmarTeam has integration for CATIA, SolidWorks, SolidEdge, Inventor, AutoCAD Highlight a set of files that should be loaded into SmarTeam The bulk loader will open each file in the CAD application one at time and perform a SmarTeam Save Recommended way to bulk load standard parts

Setup for SmarTeam Bulk Loading Tool CAD Integration should be configured Associated CAD File Type to SmarTeam Class Map CAD Attributes to SmarTeam field Sequence needs to be assigned to the primary Id Batch Mode Save Option should be turned on Otherwise, you will have to hit okay for each file to be loaded. Default Class to be specified

Issues with SmarTeam Bulk Loading Tool Speed – an open and save performed on all files being loaded CAD data integrity – If links are bad/missing in the CAD data: the files will not be saved may require user intervention per occurrence Attribute data needs to exist in CAD properties* No log created for data load* Difficult if CAD data is to be associated with more than one class* An Engineering Part, Manufacturing Part, Standard Part All data is linked to only one project* May require copying CAD to data from various directories to one directory Does not handle loading of multiple revisions *Avoidable with scripting

Overview of SmarTeam Import Tool Out of the box tool provided by SmarTeam to load meta data The ability to pick a class to load data Leaf Classes Link Classes i.e. Document Project Links The ability to map data from: Database – SmarTeam DB or non SmarTeam DB Text File – Delimited or Fixed Width Default Value – Defined in SmarTeam Default Value – Specific for the data load Populate field by a Sequence Log created for data that is imported Import Tool can run in an Add or Update mode

Issues with SmarTeam Import Tool Limited to 32,000 records per load Greater than 32,000, data needs to be split and multiple loads conducted Tool can be “buggy” Crashing from time to time Data Mappings may get lost Tool is manual For each class to be imported mappings should be configured Can be tedious to do for many classes Tool does not handle certain condition (with out Scripting) Loading multiple revisions and specifying the latest revision Loading data to internal SmarTeam fields or Reference to Class fields

Overview Custom Scripts / Programs Best suited for larger data migrations Almost any type of data loading rule can be created Determine what class to load the data Modify data to be loaded based on data in the system Re-formatted before going into SmarTeam Need to create or build link relationships in the system Various complex Data load scenarios will be conducted Need custom logging an error handling

Custom Script Data Loading - Simple Example Data exists in a legacy Folder File Name contains meta data attributes such as ID_Descritpion_Rev .doc files will go into an Engineering folder .xls files will go into an Manufacturing folder Project will need to be specified Maintain latest revison

SmarTeam Custom Script Tool Example

Combination Bulk Load and Custom script / Import Utility Bulk Load CAD data into SmarTeam Hook scripts to the SmarTeam “Before Add” Event to retrieve information i.e. from the file name to update fields on the Profile Cards Write custom code that will query the database based on the file name Update information from another source where the file name is used to link the file data /object already in SmarTeam and the other external source for data

Data Clean-up and Process Review Data Formats Data Loading Rules Migration Cost Estimation 1. Pre-Migration Data Analysis Bulk Loader SmarTeam Import and Export Tools Custom Scripts (executables, add-ins, etc) Combined Manual Processing and Custom Scripting Pre-Developed Data Migration Programs 2. Data Loading Options Testing the Migration Process Data Verification Pre-Data Load Planning Production Data Load 3. Execution 2. Data Loading Options 3. Execution

Execution: Testing the Migration Process Test the migration process!!! Mitigate all discovered issues Legacy data issues Missing data files Orphan links and/or blank fields Inconsistent legacy data such as revision schema Bad formats E.g., Attempts to load characters into integer fields Class determination rules fail Repairing the legacy data Manual clean-up Change data load rules (i.e., change the code) Modify the SmarTeam data model to better accommodate the legacy data Bug(s) in the migration tool code Data loading procedural issues e.g., failure to delete all files with a creation date before 12/31/1999 Document the migration procedure or update the tool to automate the migration process Test again!!!

Crucial to implement a data verification strategy Analyze migration process log files Spot-check data Randomly verify the data objects loaded Check at least 10% of data User validation (done through testing with the pilot system) – an important part of the verification process User validations >> IT/CAD administrators sign-offs Develop a tool to compare and analyze the data in SmarTeam and legacy system(s) Verify the number of records between systems Ensure pertinent fields and data are preserved in both systems

Execution: Pre-Data Load Planning When will the legacy system be turned off? How long will it take to load the data? Will the data loading occur during non- business hours? Specify the data load process: What are the required pre-data loading activities/events? e.g., legacy data clean-up Document the data load sequence Populate the lookup tables Import users before documents Import projects and documents, followed by the associated links Create references to class data Identify personnel responsible for the various parts of the data load process

Execution: Final Data Load With proper planning and testing, the final data load process “should” go smoothly Advices: Follow the documented procedures Allocate buffer time for unexpected issues Prepare to support users in the new system

Thank You! Questions? Script example shown is available at: http://www.xlmsolutions.com/COE2009/example.zip Marc Young xLM Solutions, LLC. 248-926-5932 myoung@xlmsolutions.com