Meinrad Weiss Principal Consultant Trivadis AG Master Data Services (MDS) und Data Quality Services (DQS) im Überblick

IT SOLUTIONS, SERVICES, & PRODUCTS TECHNOLOGIES Microsoft, Oracle, IBM, Open Source Integration, Application Performance Management, Security TrainingManaged Services Infrastructure Engineering Application Development Enterprise Content Management BUSINESS INTEGRATION SERVICES IT departments Business departments CUSTOMER Business Intelligence Trivadis solutions portfolio and competences

Hamburg Düsseldorf Frankfurt Stuttgart Munich Freiburg Vienna Basel Bern Zurich Lausanne ~370 employees ~170 employees ~20 employees Trivadis facts & figures 11 Trivadis locations with more than 550 employees Financially independent and sustainably profitable Key figures 2010 Revenue CHF 101 / EUR 73 mio. Services for more than 700 clients in over 1‘800 projects Over 170 Service Level Agreements More than 5'000 training participants Research and development budget: CHF 5.0 / EUR 3.6 mio.

Korrekte Daten halten das Interesse am Leben Meinrad Weiss An sich sind die meisten Menschen sehr interessiert. Michael Richter, Widersprüche

Overview Central place with clean and valid reference data Multiple parallel versions Audit trail Security “Toolbox” that helps increasing the data quality Data cleansing Data mapping Reference data New, clean data CRM SAP XY CRM SAP XY

Overview Central place with clean and valid reference data Multiple parallel versions Audit trail Security “Toolbox” that helps increasing the data quality Data cleansing Data mapping Reference data New, clean data CRM SAP XY CRM SAP XY

Where is it useful Master Data are non transactional data that are used in one or more systems within a company Non transactional, a.k.a. “Stammdaten” Usually not a huge amount of rows Often initialized in a BI project But not necessary. Actual Trivadis statistic is 1:1 Additional attributes e.g. Segment New (drill down) hierarchies Additional attributes e.g. Segment New (drill down) hierarchies “Parallel” versions Central place for reference data “Parallel” versions Central place for reference data

How does it work A little bit like Access … just better Version Management TX-Annotations Security Business Rules Workflow Get CRUD UI Define System and Data Model Define System and Data Model

Architecture WCF Endpoint Stewardship UI Excel SQL CLR Service Broker Sprocs UDFs Business Logic Master data Model metadata Import Export SQL Mail Notifications SQL Mail Notifications Staging Tables Subscription Views Other Application Security data Audit history Deploy Create Deployment Module

Entities, Attributes, Attribute Groups and Relationships Attribute Groupe Entities act as container Attributes define the structure of the entities Domain-based attributes represent relationships between entities Attribute Groupe

Batch Import via Staging Table Data can be imported via staging tables Each entity get its own staging table (new in Denali) Best practices for naming _ Ready to load Insert Record

Derived Hierarchies Structure based on relationships between domains No additional maintenance work

Explicit Hierarchies Consolidation Member Structure based on consolidation members Must be maintained

Business Rules Business rules can be defined and applied to verify MDS data All defined business rules must be passed to commit a version Rules are created via drag & drop interface drag & drop

“Batch” Exporting Data can be exported via Subscription Views Best practices for practices for naming _

Version Management Integrated Version Management A version contains meta data and data Meta data change impacts all versions E.g. deleting an attribute deletes it in all versions! Multiple parallel versions can be accessed Labels can be used to identify specific versions

Status & Flag Management

Transactions & Annotations Transaction details are provided for the specified Model and Version Details include prior vs. new values, when the change was made, and who made the change

Workflow SharePoint workflow integration “Real workflows” notifikation Human workflow Both are activated via Business Rules

Overview Central place with clean and valid reference data Multiple parallel versions Audit trail Security “Toolbox” that helps increasing the data quality Data cleansing Data mapping Reference data New, clean data CRM SAP XY CRM SAP XY

DQS in a Nutshell Data Quality Services (DQS) is a Knowledge-Driven data quality solution, enabling IT Pros and data stewards to easily improve the quality of their data Data Quality Services (DQS) is a Knowledge-Driven data quality solution, enabling IT Pros and data stewards to easily improve the quality of their data

DQS Process Build Use DQ Projects Knowledge Management Match & De-dupe Correct & standardize Manage Knowledge Base Discover / Explore Data / Connect Cloud Services Knowledge Base Enterprise Data Reference Data Excel Add-In SSIS

Architecture DQ Server Knowledge Base Composite Domain Domain Matching Policy Cloud Service On Premise DQ Service On Premise DQ Service DQ Client Mange DQ Server Matching Cleansing SSIS Cleansing MDS Excel Add-In Matching

DQS Domain Mgmt Known reference data Relationships Known reference data Relationships

DQS Cleansing Project DQS Knowledge Base “New data to be cleansed”

DQS Cleansing Project (2) View and Export data (SQL Server or CSV File)

Data Cleansing using SSIS  SSIS can be used to clean data via DQS  Conditional Split will be used to further distribute data

Overview Central place with clean and valid reference data Multiple parallel versions Audit trail Security “Toolbox” that helps increasing the data quality Data cleansing Data mapping Reference data New, clean data CRM SAP XY CRM SAP XY

DQS Matching Policy

DQS Support in MDS

DQS Support in MDS (Data Matching)

What is missing? Master Data Services Graphical visualization of Data Model Work around: Generate SQL Server DB based on MDS-Metadata and then use DB diagrams Simple command line interface Solution: Trivadis-XMLMD Data Quality Services A lot, but it is to early in the release cycle to comment it Product Version_1 Product Product Version_1 Product

Conclusion  MasterDataServices (MDS) –> Release 2 Highlights Multiple parallel versions Transaction logging & annotations Attribute groups and security Simplicity  DataQualityServices (DQS)–> Release < 1.0 Stability Performance Documentation Installation  Each one can be used at its own  They fit together and address real user requirements

Meinrad Weiss Principal Consultant Trivadis AG Closing word Die stillstehende Uhr, die täglich zweimal die richtige Zeit anzeigt, blickt nach Jahren auf eine lange Reihe von Erfolgen zurück. Marie von Ebner-Eschenbach, Aphorismen

Please help us make TechDays even better by Evaluating this Session. Thank you! Give us your feedback!

© 2011 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.