1. Data Platform and Analytics Foundational Training
Microsoft C+E Technology Training
Data Platform and Analytics
Foundational Training
Solution Area
Data Analytics
Solution
Business Intelligence
Technology
SQL Server EIM (SSIS, MDS, DQS)
[Speaker Name]

2. Enterprise Information Management
“Enterprise Information Management (EIM) is a best practice for creating, managing, sharing, and leveraging information in an enterprise, holistic manner that’s aligned with strategic, data-driven business objectives”
TDWI – Philip Russom

3. Enterprise Information Management
Through team collaboration and tool interoperability, EIM seeks to improve the “four C’s” of structured and unstructured data:
Completeness
Cleanliness
Consistency, and
Currency

4. SQL Server EIM
SQL Server has evolved to deliver EIM with a rich set of services designed to improve the overall relevance and value of organizational information

5. SQL Server EIM Empowers people to:
Discover and resolve problems with data
Streamline existing data management processes using familiar tools like Excel

6. SQL Server EIM Provisions and manages enterprise information:
Manage more effective data integration and management across the organization
Deliver trusted information for Business Intelligence, data warehouse, and transaction processing workloads

7. SQL Server EIM SQL Server 2016 delivers EIM with three services:
Integration Services
Master Data Services
Data Quality Services
These services can be installed on-premises, or hosted inside a virtual machine (IaaS)

8. Complete, Clean, Consistent and Current Data
SQL Server EIM
Integration Services
Master Data Services
Data Quality Services
Primarily designed to implement ETL processes
Provides a robust, flexible, fast, scalable and extensible architecture
Master Data Management
Manages reliable, centralized data
Broadens its reach with an Excel Add-in that can also use Data Quality Services for data matching
Data Quality Management
Knowledge-driven data cleansing
Corrects and de-duplicates data
Can correct data errors in the Integration Services data flow
Complete, Clean, Consistent and Current Data

9. Integration Services

10. Integration Services
SQL Server Integration Services (SSIS) is a service for building enterprise-level data integration and data transformation solutions
It includes:
A rich set of built-in tasks and transformations
Tools for constructing packages, and
The SSIS service for running and managing packages

11. Integration Services
SSIS was primarily designed to implement Extract, Transform and Load (ETL) processes
Provides a robust, flexible, fast, scalable and extensible architecture
Its capabilities are useful in many other scenarios:
Assessing data quality
Cleansing and standardizing data
Merging data from heterogeneous data stores
Implementing repetitive and ad hoc data transfers
Automating administrative tasks

12. Integration Services Architecture
SQL Server Integration Services (SSIS) is a service
The services has two different runtime engines:
Control flow
Data flow

13. Integration Services Architecture: Control Flow
Control Flow is the process-oriented workflow engine
A package consists of a single control flow
Control flow elements:
Variables
Containers
Tasks
Precedence constraints, and
Event handlers

14. Integration Services Architecture: Data Flow
The Data Flow task encapsulates the Data Flow engine
It is designed by connecting:
Sources
Destinations, and
Transformations

15. Integration Services Architecture: Data Flow
Extract
Transform
Load

16. Integration Services SQL Server 2016 New Features
SQL Server 2016 SSIS includes many new features:
Improved deployment
Improved debugging
Improved package management
Control Flow package parts
Expanded on-premises connectivity
Expanded cloud connectivity
And many more…

17. Integration Services SQL Server 2016 New Features: Azure Feature Pack
Download and install the Azure Feature Pack
New features include:
Connection managers
Enumerator
Tasks, and
Data Flow components

18. Integration Services SQL Server 2016 New Features: Azure Feature Pack

19. Integration Services SQL Server 2016 Support by Edition
SSIS 2016 is supported by Enterprise, Standard and Developer editions
Import/export Wizard, Built-in and Azure data source connectors and tasks are also supported by Web and Express editions
Hadoop/HDFS connectors and tasks are also supported by Web edition
Supported only by Enterprise edition:
CDC Service and Designer for Oracle by Attunity
Advanced adapters
Advanced transforms

20. Integration Services Resources
MSDN: SQL Server Integration Services
MSDN: What’s New in Integration Services
MSDN: Features Supported by the Editions of SQL Server 2016
SQL Server Integration Services (SSIS) Official Blog

21. Integration Services Resources
TechNet Virtual Lab:
Exploring What's New in SQL Server 2016 Integration Services
Use the search pane to locate the lab

22. Master Data Services

23. Master Data Services
SQL Server Master Data Services (MDS) is a service designed to implement and manage Master Data Management (MDM) solutions
v1: First released with SQL Server 2008 R2
v2: The SQL Server 2012 release included many new features and enhancements
v3: The SQL Server 2014 release included no new capabilities or features
v4: The SQL Server 2016 release included many new capabilities and features
Supported by Enterprise edition only
For non-production use, it is also supported by Developer and Evaluation editions

24. Master Data Services MDM
MDM is about taking control of data, and making data work to the organization’s advantage
It enables an organization to discover and define non-transactional lists of data, and to compile maintainable, reliable master lists
A successful MDM solution delivers a master data hub that provides access to authoritative, standardized, and validated versions of data

25. Master Data Services MDM
MDM can address several problems:
No authoritative source of data ► Reports may not be trusted ► Poor decisions
Systems with different versions of the same data
Data spread across multiple systems can lack integrity and consistency
Mergers and acquisitions that involve integrating data contribute to different versions
No formal dimension maintenance capability
Multiple stakeholders for the same record
Inability to enable, monitor and enforce data stewardship processes

26. Master Data Services MDM
MDM can be implemented as:
Operational MDM
Analytical MDM

27. Master Data Services Operational MDM
Operational MDM delivers a Master Data hub as the System of Record (Master Data is created), and with a focus on integration achieved with centralized governance

28. Master Data Services Analytical MDM
Analytical MDM delivers a single and consistent view of Master Data in the downstream data warehouse
Having implemented an Analytical MDM solution, a master list of records can then become the dimension source

29. Master Data Services Architecture
MDS is implemented as a WCF Data Service
Internet Information Services (IIS) is required for the Master Data Services web application and web service
MDS can be programmatically manipulated
The repository for metadata and data is a SQL Server database
The MDS database can be created on either the local or a remote computer
Microsoft provides two user interfaces:
Master Data Manager web application
Excel add-in

30. Master Data Services Sample Implementation
Source System(s)
MDS
Administration
Data Steward via Master Data Manager or Excel
WCF Data Service
Data Changes
SSIS
SSIS
Notification
Outlook
Data Warehouse
SSIS

31. Master Data Services EIM Integration
MDS can integrate with SSIS
Importing master data is achieved by loading data into SQL Server tables
Exporting master data is achieved by retrieving data from SQL Server views
MDS can also integrate with DQS

32. Master Data Services SQL Server 2016 New Features
SQL Server 2016 MDS includes many new features:
Improved user interfaces
Improved permission management
Improved performance and scale
Improved business rules, including custom T-SQL logic
Merge conflict detection and resolution
Changesets to enable approval workflows
Improved hierarchies
Entity Sync to share common entities between models
And many more…

33. Master Data Services Resources
MSDN: SQL Server Master Data Services
MSDN: What’s New in Master Data Services
MSDN: Features Supported by the Editions of SQL Server 2016
Master Data Services Blog

34. Master Data Services Resources
TechNet Virtual Labs:
1: Developing a SQL Server 2016 Master Data Services Solution
2: Exploring What's New in SQL Server 2016 Master Data Services
Use the search pane to locate the lab

35. Data Quality Services

36. Data Quality Services
SQL Server Data Quality Services (DQS) is a knowledge-driven data cleansing product
It is a product dedicated to data correction, standardization, enrichment and matching
v1: First released with SQL Server 2012
v2: The SQL Server 2014 release included no new capabilities or features
v3: The SQL Server 2016 release included no new capabilities or features
Supported by Enterprise edition only
For non-production use, it is also supported by Developer and Evaluation editions

37. Data Quality Services Architecture
Source: MSDN

38. Data Quality Services Benefits
The benefits of using DQS include:
Accumulating cleansing and matching knowledge in a knowledge base
Updating cleansing and matching knowledge, as required
Leveraging built-in functionality and algorithms to cleanse and match data
Applying consistent cleansing logic
Automating cleansing activities—reducing time, cost and the potential for “human” error

39. Data Quality Services EIM Integration
DQS can integrate with SSIS
The DQS Cleansing data flow transform is available with SSIS 2012, and later

40. Data Quality Services EIM Integration

41. Data Quality Services EIM Integration
DQS can integrate with MDS
MDS Add-in for Excel can be used to help identify duplicate members before they are published to an entity

42. Data Quality Services SQL Server 2016 New Features
SQL Server 2016 DQS includes no new features

43. Data Quality Services Resources
MSDN: Introduction to Data Quality Services
Data Quality Services Blog
eBook: Data Quality Services
Do a web search on “Data Quality Services.pdf”

44. Summary

45. Summary
SQL Server 2016 continues to deliver a comprehensive, EIM platform with three services:
Integration Services
Master Data Services
Data Quality Services
They function independently, and also together in integrated designs

46. Complete, Clean, Consistent and Current Data
SQL Server 2016 EIM
Integration Services
Master Data Services
Data Quality Services
Primarily designed to implement ETL processes
Provides a robust, flexible, fast, scalable and extensible architecture
Master Data Management
Manages reliable, centralized data
Broadens its reach with an Excel Add-in that can also use Data Quality Services for data matching
Data Quality Management
Knowledge-driven data cleansing
Corrects and de-duplicates data
Can correct data errors in the Integration Services data flow
Complete, Clean, Consistent and Current Data

47. © 2016 Microsoft Corporation. All rights reserved
© 2016 Microsoft Corporation. All rights reserved. Microsoft, Windows, Microsoft Azure, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION