Jemini Joseph

About me Working in Microsoft BI field since Mostly consulting in SSIS Worked as programmer in Visual Basic before moving to BI When I’m not working…

What’s it Read SSIS packages programmatically to report details using C# script Create new SSIS 2012/2014 package and deploy using new project deployment model Convert 2008 packages into 2012/2014 and deploy

Why Need to find the package that affects a table Need to convert 100’s of packages to 2012/2014 (end of server life) Need to create package for unknown format files

What do we need SQL Server 2008 installed including SDK SQL Server 2012/2014 installed including SDK SQL Server Data tools Knowledge of C# (or VB.net) scripting. Don’t have to be an expert

Controls in a package (Control flow) TaskHost (Executable) Containers Precedence Constraints

Controls in Data Flow Source Transformations Destination Paths

Deployment difference 2005/2008 used package deployment Configure each package 2012/2014 use project deployment model Can share configuration (Environment) Uses new SSISDB database and Integration Services Catalogs

Libraries References C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\Microsoft.CSharp.dll C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SqlServer.ConnectionInfo.dll C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SqlServer.DTSPipelineWrap.dll C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SQLServer.DTSRuntimeWrap.dll C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SqlServer.ManagedDTS.dll C:\Windows\assembly\GAC_MSIL\Microsoft.SqlServer.Management.IntegrationServices\ __89845dcd80 80cc91\Microsoft.SqlServer.Management.IntegrationServices.dll C:\Windows\assembly\GAC_MSIL\Microsoft.SqlServer.Management.Sdk.Sfc\ __89845dcd8080cc91\Micr osoft.SqlServer.Management.Sdk.Sfc.dll C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Tasks\Microsoft.SqlServer.ScriptTask.dll C:\Windows\assembly\GAC_MSIL\Microsoft.SqlServer.Smo\ __89845dcd8080cc91\Microsoft.SqlServer.S mo.dll C:\Program Files (x86)\Microsoft SQL Server\120\DTS\Tasks\Microsoft.SqlServer.SQLTask.dll 100 = SQL 2008, 110=2012, 120 = 2014 GAC – 11= 2012 and 12=2014

Namespaces using System; using System.Data; using Microsoft.SqlServer.Dts.Runtime; using System.Windows.Forms; using System.Collections.Generic; using System.Text; using System.IO; using Microsoft.SqlServer.Server; using Microsoft.SqlServer.Dts.Pipeline.Wrapper; using Microsoft.SqlServer.Dts.Runtime; using Microsoft.SqlServer.Management.IntegrationServices; using Microsoft.SqlServer.Management.Smo; using System.Collections; using System.Data.SqlClient; using Microsoft.SqlServer.Management.Common;

Demo Read 2008 Package Create new 2012 package and project Copy 2008 to 2012 and deploy

The program flow create new Set the SSISDB and folder on 2014 box Create a 2014 project and package Create Parameters for the project Deploy project to SSISDB Create Environment (Get data from each package) Configure the project using new environment

Program Flow copy/read 2008 Copy connections to project connection managers Read through control flow If the control is TaskHost (Executable) copy it If the control is container Read through controls in container If the control is executable copy it Copy Precedence constraints to link controls If the control is DFT Copy controls Copy paths between objects Refresh metadata to get columns Copy the mapping for destination controls

Some code samples Opening and reading a package object obj ssisApp = new Microsoft.SqlServer.Dts.Runtime.Application(); Microsoft.SqlServer.Dts.Runtime.Package Srcpkg = ssisApp.LoadPackage(SrcPkgFile, null);; for (int i = 0; i < Srcpkg.Executables.Count; i++) { obj = SrcPackage.Executables[i]; //ExecuteSQL and Data Flow Task if (obj.ToString() == "Microsoft.SqlServer.Dts.Runtime.TaskHost") (Data flow task and execute SQL) if (obj.ToString() == "Microsoft.SqlServer.Dts.Runtime.Sequence")

Reading TaskHost TaskHost SourcetaskHost = (TaskHost)th; if (th.InnerObject is MainPipe) Data flow if (th.InnerObject.ToString() == "Microsoft.SqlServer.Dts.Tasks.ExecuteSQLTask.Execu teSQLTask") Execute SQL

Creating controls Executable ExecSQL ExecSQL = DstPackage.Executables.Add("STOCK:SQLTask"); Executable dataFlowTask = null; dataFlowTask = DstPackage.Executables.Add("STOCK:PipelineTask");

Creating controls in DFT MainPipe DstPipeline = dataFlowTask.InnerObject as MainPipe; IDTSComponentMetaData100 Targetcomponent = DstPipeline.ComponentMetaDataCollection.New(); Targetcomponent.ComponentClassID = "DTSAdapter.OleDbSource"; Targetcomponent.RuntimeConnectionCollection[0].ConnectionManager = DtsConvert.GetExtendedInterface(SrcCM); Targetcomponent.RuntimeConnectionCollection[0].ConnectionManagerID = SrcCM.ID; CManagedComponentWrapper srcDesignTime = Targetcomponent.Instantiate(); srcDesignTime.ProvideComponentProperties(); srcDesignTime.SetComponentProperty("AccessMode", 0); srcDesignTime.SetComponentProperty("OpenRowset", TableName); srcDesignTime.AcquireConnections(null); srcDesignTime.ReinitializeMetaData(); srcDesignTime.ReleaseConnections();

Some properties of OLE source and destination

What’s not done Doesn’t work for multiple inputs or outputs (Union all merge join, multicast) Reading packages from MSDB ([dbo].[sysssispackages]) Events

Some points Consider Sequence as a package The order of the object is unknown. So when you create path, need to find the control without any input (OLE Source) and start from there. The order of executables in package is unknown. Need to creating precedenceconstraints in right order when you copy existing package Need to add input columns to Derived column May need to fix the LineageID between Derived column and OLE Destination BIML can create new packages. Can it work with flat files or excel?

Some important classes IDTSComponentMetaData100 – Data flow controls IDTSComponentMetaDataCollection100 IDTSOutput100 – Output of a control (the pipe) IDTSOutputCollection100 IDTSOutputColumn100 IDTSOutputColumnCollection100 – Generally one, multicast will have multiple IDTSInput100 IDTSInputCollection100 IDTSInputColumn100 IDTSInputColumnCollection100 Generally one. Union all will have multiple IDTSCustomProperty100 – Custom property of a control like expression IDTSCustomPropertyCollection100 IDTSPath100 IDTSPathCollection100 – Collection of path. My example have two paths. PrecedenceConstraint PrecedenceConstraints

Some links (MSDN Developers guide) (SSIS Object Model) andplusDeployplusSSI (Creating and deploying 2012 package) andplusDeployplusSSI (Search for SSIS 2012 automation) package-implementation-programmatically.aspx (Creating 2008 package) package-implementation-programmatically.aspx us/library/microsoft.sqlserver.dts.pipeline.wrapper.idtscomponentmetadata10 0.aspx (This is Data Flow control) us/library/microsoft.sqlserver.dts.pipeline.wrapper.idtscomponentmetadata10 0.aspx

Questions? Can send questions to