Where I am at: Swagatika Sarangi MDM Lead PASS Summit SQL Saturdays MDM/MDS/DW Consultant Where I am at: PASS Summit SQL Saturdays SQL Server User groups www.Jazzintelligence.com Swagatika Sarangi MDM Lead /sarangiswagatika @gatorjazz12 swagatika.sarangi

Why Change Data Capture

Use Case 1: Option #1: Dumping Data Probably the easiest solution (and many companies do so) is to export the entire database from the master database, and import it to your BI database, say, every 24 hours. This works fine while your data size is small. However, it starts hitting limits pretty quickly as your user base grows. You’ll reach a point where you can’t export and import the entire database quickly enough for your business needs. In fact, a lot of companies regard 24 hours to be too long to wait in this fast- moving “Big Data” age.

Use Case 2: Option #2: Utilizing CDC This is where CDC comes to the rescue. Change Data Capture (CDC), as it’s name suggests, is a design pattern that captures individual data changes instead of dealing with the entire data. Instead of dumping your entire database, using CDC, you would capture just the data changes made to the master database and apply them to the BI databases to keep both of your databases in sync. This is much more scalable because it only deals with data changes. Also, the replication can be done much faster, often in near real-time.

Let’s demo..

Change Tracking at SQL Server: Create Database, Table Make sure that the SQL Server Agent is running for the database instance Enable CDC for that new database Check results select name, is_cdc_enabled from sys.databases Enable CDC for that new table with attribute values Check the system Change Tracking table for the table you are tracking on Perform CRUD operation Query the System Change Tracking table for all historical Changes

Checking CDC enabling on database level

Enabling CDC on Table level

Enabling CDC on Table Level

Find the respective CDC system table cdc.dbo_AddressEntity_CT is the new table for tracking our changes.

Perform any CRUD operation:

Perform any CRUD operation:

Tracking at System Change Tracking Tables:

Change Tracking at MDS Web UI: Create the Model, Entity, Attributes, Members Enable “Change Tracking” at the “attribute level” by going to “System Administration”. Perform changes at the member level in UI by navigating to “Explorer” and selecting the right “Entity”. Select the member row and navigate to “View History”

What is Master Data Services Keeps single version of truth for the dimension members Provides two ways of interacting via Excel Sheet add-in and MDS web UI for business users. Provides Subscription views as a consumption for downstream system using SQL server as their in-house database. For ETL, MDS uses SSIS packages to cleanse, transfer, match and merge data from Pre-staging area to staging area, and finally pushing data to the UI.

RELATIONAL DB EQUIVALENT TERMS MDS Terms MDS TERMS RELATIONAL DB EQUIVALENT TERMS Model Database Entity Table Attribute Column Member Record

CDC at MDS Web UI

Changing AddressLine1 of CostCenterID with Code = 1517

Select “View History” for the updated Cost Center ID:

Summary:

Change Tracking at SQL Server: Create Database Make sure that the SQL Server Agent is running for the database instance Enable CDC for that new database Check results select name, is_cdc_enabled from sys.databases Enable CDC for that new table with attribute values Check the system Change Tracking table for the table you are tracking on Perform CRUD operation Query the System Change Tracking table for all historical Changes

Change Tracking at MDS Web UI: Create the Model, Entity, Attribute, Member Enable “Change Tracking” at the “attribute level” by going to “System Administration”. Perform changes at the member level in UI by navigating to “Explorer” and selecting the right “Entity”. Select the member row and navigate to “View History”

Thank You