Bus Matrix… the foundation of your Data Warehouse The Bus Matrix is the cornerstone of a successful Dimensional Data Warehouse strategy. It serves many purposes: from communicating requirements, capabilities, and expectations with the business users down to the prioritization and delegation of tasks across the development team. Join me in this session and learn what a Bus Matrix is, why it is the single most important document in your Data Warehouse project, and what can go wrong without it. We'll also cover several approaches for creating and maintaining the Bus Matrix document. Bill Anton is an independent consultant whose primary focus is designing and developing Data Warehouses and Business Intelligence solutions using the Microsoft BI stack. When he's not working with clients to solve their data-related challenges, he can usually be found answering questions on the MSDN forums, attending PASS meetings, or writing blog entries over at http://byobi.com. Bill Anton Prime Data Intelligence

About Me I Love Data! …also, Microsoft DW/BI (MCTS/MCITP, MCSA/MCSE) Independent Consultant @ Prime Data Intelligence, LLC Atlanta BI SQL Server Users Group Twitter: @SQLbyoBI Blog: http://byoBI.com Email: william.anton@gmail.com

What we will cover today  Dimensional Modeling 101 What, Why, How Common Challenges Bus Matrix What is it? How does it help? Examples Links on resource page of blog

What is Dimensional Modeling? Facts additive amounts E.g. Sales amount, inventory quantity SUM, AVERAGE, MAX, MIN, COUNT Dimensions descriptive attributes E.g. Date, Product, Location, Customer GROUP BY <attribute>, <attribute>, etc

What is Dimensional Modeling? Each fact forms the center of a star Ex. this customer, bought this product on this date…. “Star Schema”

What is Dimensional Modeling? Denormalization “Repeating Values” Opposite of “normalized” (e.g. 3rd Normal Form) Optimized for reads (not writes)

Dimensional Modeling 101 Question: What are the most common types of Data Warehouse methodologies/architectures? Kimball Inmon Data Vault Kimball: star-schema, conformed dimensions Inmon: 3NF data warehouse, Coporate Information Factory, Hub-n-Spoke Data Vault: highly controversial…I love it. Is Anyone Familiar with Data Vault?

All of them  Dimensional Modeling 101 Question: For which of these DW methodologies should you include a dimensional model? Kimball, Inmon, Data Vault All of them  With Kimball it is built in. With Inmon/DataVault you need to add a dimensional layer.

Kimball Dimensional DW

Kimball Dimensional DW Sales Production Finance Supply Chain bus architecture

Inmon 3NF EDW + Data Mart(s) Does anyone know what “EDW” stands for? How is that different from “Kimball” method?

Data Vault + Data Mart(s) time-invariant system of record (S-O-R) Copies of source systems Business rules applied downstream Hubs/links/satellites dan linstedt Data Vault Method: Since no business rule is applied to the data on the way into the EDW, the EDW becomes a ‘statement of fact’.  This single version of facts now becomes a time-invariant system of record (S-O-R) where the facts as they were known to the business at any point in history are stored. In our example above, the EDW stores both the addresses, that borrower has. http://makingdatameaningful.com/2012/01/05/data-vault-the-preferred-flavor-for-dw-architecture-in-bi-part-iv/

Why Dimensional Modeling Intuitive to Business Users Simpler than OLTP/3NF Rise of Self-Service (E.g. Power Pivot, Power View) Iterative Development “Agile” Performance Optimized for analytical queries e.g. sales amount by product in 2013 for top 10 all-time customers And many more… See Teo Lachev’s “WHY SEMANTIC LAYER” newsletter: http://www.prologika.com/Newsroom/Newsletter2013Fall.aspx With Kimball it is built in With Inmon/DataVault you need to add a dimensional layer Self-Service BI = implies more and more users will be looking for access to data (think: PowerPivot) “Semantic Layer” http://www.prologika.com/Newsroom/Newsletter2013Fall.aspx

Intuitive to Business Users Business users think in terms of Dimensions and Fact whether they know it or not. Teach a Business User how to use a pivot table…

How many bikes did we sell last year?

Do we sell more bikes to single or married females?

What was our most/least profitable product this year?

What was the Average Monthly Gross Margin Return on Inventory Investment (GMROII) by Product Category for the trailing 6 months? It’s Complicated We’ll come back to this…

Star-Schema Each fact forms the center of a star Ex. this customer, bought this product on this date….

1 “Star” per Fact table Sales Process Inventory Process Each fact forms the center of a star Sales Process Inventory Process

Facts are related through dimensions… But there are usually dimensions in common GMROII (Gross Margin Return on Inventory Investment) Sales Process Inventory Process

Facts are related through dimensions… “Conformed Dimensions” A conformed dimension is a set of data attributes that have been physically referenced by multiple fact tables using the same key value to refer to the same structure, attributes, domain values, definitions and concepts. Dimensions are conformed when they are either exactly the same (including keys) or one is a perfect subset of the other. Dimension tables are NOT conformed if the attributes are labeled differently or contain different values. MUST BE IDENTICAL By linking facts through conformed dimensions, cross-process analysis is possible Wikipedia (http://en.wikipedia.org/wiki/Dimension_(data_warehouse)#Conformed_dimension)

Dimensions: Conformed vs Unconformed

Revisiting Average Monthly Gross Margin Return on Inventory Investment (GMROII) By linking facts through conformed dimensions, cross-process analysis is possible Sales + Inventory  GMROII Average Monthly GMROII Profit for total time period Sum of each month ending inventory cost

What was the Average Monthly Gross Margin Return on Inventory Investment (GMROII) by Product Category for the trailing 6 months? This becomes easy… * As long as users understand the dimensions in common

Where things start to get complex… 1 Star per Fact table Multiple Fact tables per business process Multiple business processes in an enterprise Modeling business processes

Dimensional Model becomes a “Galaxy of Stars” Finance Production Sales Distribution Modeling business processes HR

ER Diagram: Adventure Works Sample DW Common Method for Visually Documenting a Database Not so bad… Just a sample – not realistic Business Users don’t understand Simpler than Normalized data model

For bigger Data Warehouses… Multiple fact tables per business process Specialty fact tables for certain metrics Inventory Control Process Work Orders Purchasing/Procurement Model to the metrics SQL Bits: Data Modeling for Analysis Services Cubes Alex Whittles http://sqlbits.com/Sessions/Event11/Data_Modeling_for_Analysis_Services_Cubes This ^^ Turns into this ^^

Variety of Problems to Overcome with Dimensional Modeling Communication & Strategy What’s the short term plan of attack? What’s the long term plan of attack? Documentation What’s in our Data Warehouse? Business Users can’t read ER diagrams Business Users are typically only familiar with a 1 or 2 business processes E.g. Sales User vs Inventory User; Warehouse Supervisor vs CEO Conforming Dimensions is hard…REALLY hard So are changes (E.g. Impact Analysis) POA Boil the ocean vs Iterative Conforming Dimensions political battle takes time (e.g. MDM) business needs to understand the importance of it

What’s the Solution? What about a Bus Matrix? Train business users to read ER Diagrams? Simplify Data Model? Ignore certain business processes? Don’t use Conformed Dimensions? Force business users to manually map data between processes? What about a Bus Matrix? How do events in one business process impact/correlate with other Business Processes Not a single answer – but a Bus Matrix can address most of the problems

What is a Bus Matrix? 2-dimensional visualization showing the intersection of facts and dimensions In the most basic form Understandable by business users

Variety of Use-Cases for a Bus Matrix Documentation, Communication, Training Facilitate User Adoption of BI tools Communicate Expectations w/ Business New users unfamiliar with new business process Team Development Agile Prioritization of Tasks Divide & Conquer Road-Mapping Prioritization of Business Processes in a Business Intelligence “Program” Documentation: Facilitate User Adoption of BI tools Communicate Expectations w/ Business New users unfamiliar with new business process Team Development: Agile, Prioritization of Tasks Roadmapping Tool: Prioritization of Business Processes in Business Intelligence “Program” How many people understand the difference between a BI project and BI program?

Documentation For Business Easier than an ER diagram In Business Terms Users can now understand what dimensions to slice/dice measure groups from a specific business process Shows how difference business processes are related

Documentation for IT Adding context to relationships Differentiating between role playing dimension and base dimension Indicating type of Dimension, type of Fact, and grain

Master Bus Matrix Impact Analysis

Team Development Sprint 1 Internet Sales Sprint 2 Reseller Sales Break dimensions/facts into separate tasks (database, ETL, cube) Prioritize Dimensions before Fact Tables Next sprint, only create the missing dimensions

Road-Mapping Quadrant: Business Value vs Implementation Complexity Bus Matrix Version Context for delivery dates Source Systems contain data for multiple business processes

When To Create a Bus Matrix During Requirements Gathering Before You Start Development! Updated Over Time Changes to Business Processes New Source Systems (E.g. mergers/acquisitions)

How To Create a Bus Matrix Manual via Excel Automated via SSRS

Manual Only option when starting out ;-) Updates can be made quickly made as requirements come in Adds development overhead, but the ROI is well worth it

Automated Reporting pack with drill-through to data dictionary information Can be based on Cube or Relational Database (*FK required) Incorporate query statistics to visualize common usage patterns Use MDS to allow SME’s to manage business definitions Based on example from Alex Whittles http://www.purplefrogsystems.com/blog/2010/09/olap-cube-documentation-in-ssrs-part-1/ Based on example from Alex Whittles http://www.purplefrogsystems.com/blog/2010/09/olap-cube-documentation-in-ssrs-part-1/

QUESTIONS

http://byobi.com/blog/bus-matrix/ References Twitter: @SQLbyoBI Blog: http://byoBI.com Email: william.anton@gmail.com http://byobi.com/blog/bus-matrix/