1. SSIS 2012: A Deep Dive into the SSIS Catalog
Phil Brammer
@PhilBrammer
ssistalk.com

2. Phil Brammer
Over 12 years’ experience in database technologies, data warehousing, ETL, on-call…
Started on Red Brick Data Warehouse. Ralph Kimball’s product.
Worked with Teradata at PayPal
Operationally manages multi-terabyte instances
Dabbles a bit in SSIS – ssistalk.com
Microsoft MVP, SQL Server – 6th year

3. SSIS Pre-SQL 2012 Born in 2005 as a replacement for DTS
Groans or applause?
Client focused

4. SSIS in SQL Server 2012 Server Focused!
Minimal, but juicy changes in the client
Cluster aware. Sort of.
DTS is no longer supported.
Visual Studio 2010 integration

5. SSIS Project Overview Project-based deployments Parameters
Environments
Connections
Legacy, but not DTS

6. SSIS Projects Project versioning
Limited by server configuration option
Can restore to a previous version
Can only export current version with the GUI
The number of available project versions depends on the server configuration setting
See for an unsupported method of exporting an older version of a project.

7. SSIS Catalog Execution
ISServerExec.exe
Program Files\Microsoft SQL Server\110\DTS\Binn
External host for SSIS package operations such as deployment, validation, and execution
Opens its own Named Pipe
ISSERVER Assembly (UNSAFE)
Microsoft.SqlServer.IntegrationServices.Server.dll
SQLCLR stored procedures/functions
Call ISServerExec
Use IPC to exchange information to ISServerExec
An assembly can be of three permission types – safe, external access, and unsafe. ISSERVER needs to be UNSAFE as it writes to the local filesystem and to give it the flexibility it needs to perform operations outside of SQL Server.

8. SSIS Catalog Execution
SQLCLR Stored Procedures/Functions communicate via IPC over named pipes to/from ISServerExec
ISServerExec uses a SqlConnection to write logging events back to the SSIS catalog
Examples:
Start Execution
catalog.start_execution
Create Execution Dump
catalog.create_execution_dump
Return performance counter data
catalog.dm_execution_performance_counters

9. ISServerExec Execution Flow
Stopping (8)
Canceled (3)
catalog.create_execution
catalog.stop_operation
ISServerExec
Unexpected Crash
Created (1)
Pending (5)
Running (2)
Unexpected Termination / Crash (6)
catalog.start_execution
Success (7)
Completed (9)
Failed (4)

10. SSIS Catalog Objects 1 Assembly 1 SQL Server Agent Job 2 Logins
16 User-Defined Functions
17 User-Defined Types
30 Tables
33 Views
94 Stored Procedures
“internal” and “catalog” schemas

11. SSIS Catalog Security Objects are encrypted with database master key
Back it up, save the password!
All projects and sensitive parameter values are stored encrypted. (AES_256; modifiable)
Encrypted with symmetric key and certificate
Certificates and keys are created for each project/execution/environment object
Sensitive parameters are not set after deployment. You must configure the project/package accordingly once deployed.
Demo
SELECT * FROM sys.certificates
SELECT * FROM sys.keys
Sensitive parameters can only be used for objects marked sensitive – e.g. the password of a connection manager object
The database master key is encrypted by the service master key, which is specific to a machine.

12. SSIS Catalog Security
Row-level security (folders, projects, environments, operations/executions)
ssis_admin & sysadmin roles can access all securables
catalog.explicit_object_permissions shows object permissions that have been set for a user
catalog.effective_object_permissions shows object permissions that are in effect recursively
Show the difference between the two by giving a demo.
Create user, add to public role
Grant user access to a folder with ‘read objects’ permission
Query the two views
Note that there is a bug (as far as I am concerned) with explicit_object_permissions in that it shows other users’ permissions if they have access to the same object as another user.

13. SSIS Catalog Logging Can set a server-wide logging level
Four levels: None, Basic, Performance, Verbose
Basic logging captures most log events- OnError, OnWarning, OnInformation, OnPre/PostValidate, OnPre/PostExecute
Performance logging captures OnError, OnWarning, and great performance data – data flow component execution timings
Verbose logging captures everything, including row counts transferred.

14. SSIS Catalog Logging
Logging can be extended with 3rd party products or by using the existing framework
Custom log events are only captured in the verbose logging level
Script Tasks can log via the FireXXXXXX method (FireInformation, FireError, etc…)
New DiagnosticEx event under verbose logging captures parameter information passed down to child packages
Show example of existing logging using FireInformation
SSIS_Logging.sql
Package.dtsx logs an OnInformation event with message code 9999 (can query catalog.event_messages filtering on this message_code)

15. SSIS Data Taps Two procedures: A SQL-based data viewer
catalog.add_data_tap
catalog.add_data_tap_by_guid
A SQL-based data viewer
With verbose logging, no need to open up package
Insert data tap to investigate issues
Can only store files in <SQLInstallDir>\110\DTS\DataDumps
CSV format
Use SSIS_DataTaps.sql for an example on setting a data tap while executing a package.

16. SSIS Performance Monitoring
One DMV
catalog.dm_execution_performance_counters
DMV asks ISServerExec for an execution’s statistics (or all executions if NULL is used)
ISServerExec passes the information back via named pipe
Run PerfCounters.dtsx and while it is running (it has a one minute delay), query dm_execution_performance_counters

17. SSIS Dump Files One procedure:
catalog.create_execution_dump
Create an execution dump on the fly for use with CSS or other support activities
Stored in <SQLInstallDir>\110\Shared\ErrorDumps
Creates .mdmp and .tmp debug files
Can also set parameters to cause dumps
DUMP_ON_EVENT
DUMP_EVENT_CODE
DUMP_ON_ERROR
Use SSIS_DumpFiles.sql to execute CreateDump.dtsx and create dump files when an error is encountered.

18. SSIS Dump Files
To see prior event codes from an execution, select from catalog.event_messages, casting message_code to BINARY(4)
Take converted message_code and set DUMP_EVENT_CODE to trigger dump file creation
Example:
SELECT *, CAST(message_code AS BINARY(4)) AS message_code_hex
FROM catalog.event_messages
Use message_code_hex values in the DUMP_EVENT_CODE parameter for those events you want to capture

19. SSIS Catalog Maintenance
One SQL Agent Job - SSIS Server Maintenance Job
Two stored procedures
internal.cleanup_server_project_version
internal.cleanup_server_retention_window
One database user - ##MS_SSISServerCleanupJobUser##
Above user is allowed to execute the above two procedures and is the job owner

20. SSIS Catalog Maintenance
The Server Maintenance Job purges all of the catalog tables based on retention windows – properties of the SSIS Catalog
Relies on cascade deletes. Job simply deletes from parent tables in batches of 10 records.
Causes extreme blocking on busy systems and can result in canceled package executions
SQL 2012 SP1 should alleviate the canceled execution problem by better handling blocking scenarios

21. SSIS Catalog and AlwaysOn
SSISDB is supported in an AlwaysOn setup
Remove SSISDB from the availability group before patching
Remember that SSISDB’s master key is encrypted with the service master key, which is tied to the machine hosting SSISDB.
On failover you will get an encryption error
Can correct manually, or use a SQL Agent job to open the master key

22. Questions
Questions and comments

23. Resources http://msdn.microsoft.com/en-us/library/hh479588.aspx

24. End Thank you for attending! Phil Brammer @PhilBrammer