DAT 332 SQL Server 2000 Data Transformation Services (DTS) Best Practices Euan Garden Product Unit Manager SQL Server Development Microsoft Corporation

Technologies Covered Data Transformation Services Analysis Services (Decision Support Objects) Transact SQL ADO ActiveX Scripting

“Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

Package Design Practices Drive Packages through Metadata Dynamic Properties Task Access to Property of any object Sources values from many locales Global Variables Building blocks for communicating metadata Populated by the Dynamic Properties Task UDL Files Configurable File for storing Connection Information

Package Design Practices Designing Modular Parent/Child Packages Execute Package Task Allows joining of Transaction Supports passing of Globals Execution through COM Automation Programmatic access to Package objects Eliminates Version Affinity

Parent Child Packages Child Package is Invoked by ActiveX Script Can also be done with Execute Package Task DTS Parent DTS Child – Error Handler Example

Demo Demo Parent/Child Package(s) And UDL Files Design Practices

“Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

Data Loading Practices Leverage Fast Load Interface(s) Bulk Insert (Not always possible) Data Pump (For More Complex Logic) Characteristics of this Approach Bulk Insert Task Execute SQL Task Rowset Parameter ADO Disconnected Recordsets DTS Multiphase Data Pump

Data Loading Practices Benefits to this Approach Cache Rowset on Client (fast lookups) Hold updates on Client Conditional Inserts/Updates without Data Driven Query

Demo Demo Conditional Inserts And Updates With The Multiphase Data Pump Design Loading

“Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

Advanced Practices Multi-Threaded Data Loading Requires Multiple Connections Data is Loaded in Parallel Avoid Contention by Loading into Separate Entities

Advanced Practices Task Looping Requires at least two tasks (Executive and Trigger) Facilitated by Manipulating Workflow.ExecutionStatus = DTSStepExecStat_Waiting Maintain State in Global Variables

Advanced Practices Communication through Global Variables Only Means for Communicating among Tasks Examples Pointer to Position within Global Array Status of a Stored Procedure Call Flag to Shutdown Parallel Load Processes

Advanced Practices – Benefits Higher Utilization during Data Load Compressed Data Loading Window Easy Orchestration within DTS

Multi-Threaded Loading Task Marshall Thread 1 Loading Task Initialization Task Thread n Loading Task Thread 1 Restart Task Thread n Restart Task

Demo Demo Multi-Threaded Data File Loading Advanced Practices

“Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

Auditing And Error Handling Logging Options Logging to MSDB Logging to Text File Logging to Event Log Enabled through Package Properties Capture Step/Package Errors Enabled through Data Pump Properties Capture Erroneous Data

Auditing And Error Handling DTSPackageLog Object WriteStringToLog() WriteTaskRecord() DTS Lineage Global Variables DTSLineage_Full DTSLineage_Short DTSErrorRecords Object Add() Clear()

Auditing And Error Handling Monitoring Execution – Simple as Writing SQL select package.name, package.description, step.stepname, step.errorcode, step.errordescription from msdb..sysdtspackagelog package, msdb..sysdtssteplog step where package.lineagefull = step.lineagefull and step.errorcode <> 0

Demo Demo Custom Logging Using Intrinsic DTS Options Error Handling

“Best Practices” Agenda Package Design Data Loading Advanced Techniques Auditing and Error Handling Analysis Services Management

Analysis Services And DTS Analysis Services Processing Task Supports Processing of all OLAP Objects Customizable Processing through TreeKey and other properties Prominent Properties TreeKey ServerName\DBName\CubeFolder\CubeName ItemType 1=OLAP DB, 4=Cube, 7=Partition and 9=Dimension ProcessingOption 0=Full Process, 1=Refresh Data and 2=Incremental Update

Analysis Services And DTS Decision Support Objects Provide access to the AS Management Interfaces Automated via AxScript Supports Partition Management Creating Cloning Dropping Merging of partitions Should be an OLAP Administrator to Use

DSO Hierarchy

Analysis Services And DTS Benefits Opportunity to Dynamically Manage Partitions and other Objects Support for Dynamic Processing Inclusion in the BI Solution Load

Demo Demo Dynamic Partition Creation And Processing Analysis Services

Community Resources DTS Whitepaper – Search on MSDN for “Using DTS for Business Intelligence” Microsoft Site SQLDTS Site SQL Server Developers Network SQLServer Central

Ask The Experts Get Your Questions Answered I’ll be in the Experts booth right after this session

Questions? Questions?

Community Resources Most Valuable Professional (MVP) Newsgroups Converse online with Microsoft Newsgroups, including Worldwide User Groups Meet and learn with your peers

evaluations evaluations

© 2003 Microsoft Corporation. All rights reserved. This presentation is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS SUMMARY.