1. Aggregating Knowledge in a Data Warehouse and Multidimensional Analysis
Rafal Lukawiecki Strategic Consultant, Project Botticelli Ltd

2. Objectives Explain the basics of:
Data Warehousing
ETL
OLAP/Multidimensional Data
Relate the theory to SQL Server 2008 SSAS and SSIS
This seminar is based on a number of sources including a few dozen of Microsoft-owned presentations, used with permission. Thank you to Marin Bezic, Kathy Sabourin, Aydin Gencler, Bryan Bredehoeft, and Chris Dial for all the support. Thank you to Maciej Pilecki for assistance with demos.
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material presented is not certain and may vary based on several factors. Microsoft makes no warranties, express, implied or statutory, as to the information in this presentation.
Portions © 2009 Project Botticelli Ltd & entire material © 2009 Microsoft Corp. Some slides contain quotations from copyrighted materials by other authors, as individually attributed or as already covered by Microsoft Copyright ownerships. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Project Botticelli Ltd as of the date of this presentation.  Because Project Botticelli & Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft and Project Botticelli cannot guarantee the accuracy of any information provided after the date of this presentation. Project Botticelli makes no warranties, express, implied or statutory, as to the information in this presentation. E&OE.

3. 1. Data Warehouse

4. Rich Connectivity Data Providers
ODBC
SQL Server
SAP NetWeaver BI
SQL Server Report Server Models
SQL Server Integration Services
Teradata
XML
OLE DB
DB2
MySAP
SQL Server Data Mining Models
Oracle
SQL Server Analysis Services
Hyperion Essbase

5. Let’s Store the Intelligence: DW
SQL Server Analysis Services server is a logical endpoint for data being aggregated with SSIS
But do not store actual data in it
Data physically rests in another database called a Data Warehouse
You can manipulate it directly, or build in parallel with OLTP processing
Modelling of data stored in DW and analysed using SSAS is at the heart of good Data Warehouse design

6. Microsoft BI Voyage
Star Schema

7. Star Schema Benefits Transforms normalized data into a simpler model
Microsoft BI Voyage
Star Schema Benefits
Transforms normalized data into a simpler model
Delivers high-performance queries
Delivers higher performing queries using Star Join Query Optimization
Uses mature modeling techniques that are widely supported by many BI tools
Requires low maintenance as the data warehouse design evolves

8. Snowflake Dimension Tables
Microsoft BI Voyage
Snowflake Dimension Tables
Define hierarchies using multiple dimension tables
Support fact tables with varying granularity
Simplify consolidation of data from multiple sources
Potential for slower query performance in relational reporting
No difference in performance in Analysis Services database

9. Hierarchies Benefits Implementation
Microsoft BI Voyage
Hierarchies
Benefits
View of data at different levels of summarization
Path to drill down or drill up
Implementation
Denormalized star schema dimension
Normalized snowflake dimension
Self-referencing relationship

10. Fact Table Fundamentals
Microsoft BI Voyage
Fact Table Fundamentals
Collection of measurements associated with a specific business process
Specific column types
Foreign keys to dimensions
Measures – numeric and additive
Metadata and lineage
Consistent granularity – the most atomic level by which the facts can be defined

11. Fact Table Examples Quarter Grain Day Grain Reseller sales data by:
Microsoft BI Voyage
Fact Table Examples
Reseller sales data by:
Product
Order Date
Reseller
Employee
Sales Territory
Sales quota data by:
Employee
Time
Quarter Grain
Day Grain

12. Microsoft BI Voyage
Date Dimension Table
Most common dimension used in analysis (aka Time dimension)
Used consistently with all facts for efficient and flexible analysis
Useful common attributes – Year, Quarter, Month, Day
Time series analysis support
Navigation and summarization enabled with hierarchies, such as calendar or fiscal
Single table design (typically not snowflake design)
Tip: Format the key of the dimension as yyyymmdd (e.g ) to make it readily understandable

13. Parent-Child Hierarchy
Microsoft BI Voyage
Parent-Child Hierarchy
A dimension that contains a parent attribute
A parent attribute describes a self-referencing relationship, or a self-join, within a dimension table
Common examples
Organizational charts
General Ledger structures
Bill of Materials

14. Parent-Child Hierarchy Example
Microsoft BI Voyage
Parent-Child Hierarchy Example
Brian
Amy
Stacia
Stephen
Shu
Michael
Peter
José
Syed

15. Slowly Changing Dimensions
Microsoft BI Voyage
Slowly Changing Dimensions
Support primary role of data warehouse to describe the past accurately
Maintain historical context as new or changed data is loaded into dimension tables
Implement changes by Slowly Changing Dimension (SCD) type
Type 1: Overwrite the existing dimension record
Type 2: Insert a new ‘versioned’ dimension record
Type 3: Track limited history with attributes

16. SCD Type 1 Existing record is updated History is not preserved
Microsoft BI Voyage
SCD Type 1
Existing record is updated
History is not preserved

17. SCD Type 2 Existing record is ‘expired’ and new record inserted
Microsoft BI Voyage
SCD Type 2
Existing record is ‘expired’ and new record inserted
History is preserved
Most common form of SCD

18. SCD Type 3 Existing record is updated Limited history is preserved
Microsoft BI Voyage
SCD Type 3
Existing record is updated
Limited history is preserved
Implementation is rare
SalesTerritoryKey update to 10

19. Let’s Get the Data
We would like to populate facts and dimensions in our Data Warehouse from OLTP data...

20. 2. Integration and ETL

21. Let’s do ETL with SSIS SQL Server Integration Services (SSIS) service
Microsoft BI Voyage
Let’s do ETL with SSIS
SQL Server Integration Services (SSIS) service
SSIS object model
Two distinct runtime engines:
Control flow
Data flow
32-bit and 64-bit editions

22. The Package The basic unit of work, deployment, and execution
Microsoft BI Voyage
The Package
The basic unit of work, deployment, and execution
An organized collection of:
Connection managers
Control flow components
Data flow components
Variables
Event handlers
Configurations
Can be designed graphically or built programmatically
Saved in XML format to the file system or SQL Server

23. Control Flow Control flow is a process-oriented workflow engine
Microsoft BI Voyage
Control Flow
Control flow is a process-oriented workflow engine
A package contains a single control flow
Control flow elements
Containers
Tasks
Precedence constraints
Variables

24. Data Flow The Data Flow Task Encapsulates the data flow engine
Microsoft BI Voyage
Data Flow
The Data Flow Task
Encapsulates the data flow engine
Exists in the context of an overall control flow
Performs traditional ETL in addition to other extended scenarios
Is fast and scalable
Data Flow Components
Extract data from Sources
Load data into Destinations
Modify data with Transformations
Service Paths
Connect data flow components
Create the pipeline

25. Data Flow Sources Sources extract data from
Microsoft BI Voyage
Data Flow Sources
Sources extract data from
Relational tables and views
Files
Analysis Services databases

26. Data Flow Destinations
Microsoft BI Voyage
Data Flow Destinations
Destinations load data to
Relational tables and views
Files
Analysis Services databases and objects
DataReaders and Recordsets
Enterprise Edition only

27. Populating Fact Tables
Microsoft BI Voyage
Populating Fact Tables
Fact source
Transform
Lookup dimension key
Repeat for each dimension key
Lookup failed? Y
Insert new dimension record N
Insert new record

28. Populating Dimension Tables
Microsoft BI Voyage
Populating Dimension Tables
Dimension source
Transform
Correlate records
New record? Y N
Type 1 change? Y
Update changed column(s) N
Type 2 change? Y
Expire existing record
Insert new record

29. Data Flow Transformations
Microsoft BI Voyage
Data Flow Transformations
Aggregate, merge, distribute, or modify data
Include error outputs in some cases
Transformation Categories
Row
Rowset
Split and Join
Business Intelligence (BI)
Script
Other

30. Row Transformations Update column values or create new columns
Microsoft BI Voyage
Row Transformations
Update column values or create new columns
Transform each row in the pipeline input

31. Rowset Transformations
Microsoft BI Voyage
Rowset Transformations
Create new rowsets that can include
Aggregated values
Sorted values
Sample rowsets
Pivoted or unpivoted rowsets
This is a heavy-weight performer of SSIS
Are also called asynchronous components

32. Split and Join Transformations
Microsoft BI Voyage
Split and Join Transformations
Distribute rows to different outputs
Create copies of the transformation inputs
Join multiple inputs into one output
Perform lookup operations

33. 8/7/ :46 PM
Demo
Using SQL Server Integration Services for Aggregating and Deriving Data
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

34. 3. OLAP/Multidimensional Data

35. SQL Server 2008 Analysis Services
Microsoft BI Voyage
SQL Server 2008 Analysis Services
OLAP component
Aggregates and organizes data from business data sources
Performs calculations difficult to perform using relational queries
Supports advanced business intelligence, such as Key Performance Indicators
Data mining component
Discovers patterns in both relational and OLAP data
Enhances the OLAP component with discovered results

36. Cube = Unified Dimensional Model
Microsoft BI Voyage
Cube = Unified Dimensional Model
Multidimensional data
Combination of measures and dimensions as one conceptual model
Measures are sourced from fact tables
Dimensions are sourced from dimension tables

37. Microsoft BI Voyage
Dimensions
Members from tables/views in a data source view (based on a Data Warehouse)
Contain attributes matching dimension columns
Organize attributes as hierarchies
One All level and one leaf level
User hierarchies are multi-level combinations of attributes
Can be placed in display folders
Used for slicing and dicing by attribute

38. Hierarchy Defined in Analysis Services
8/7/ :46 PM
Hierarchy
Defined in Analysis Services
Ordered collection of attributes into levels
Navigation path through dimensional space
Very important to get right!
Customers by Geography
Customers by Demographics
Country
Marital
State
Gender
City
Customer
Customer

39. Measure Group Group of measures with same dimensionality
Analogous to a fact table
Cube can contain more than one measure group
E.g. Sales, Inventory, Finance
Defined by dimension relationships

40. Measure Group Measure Group Dimension Sales Inventory Finance
Customers X
Products
Time
Promotions
Warehouse
Department
Account
Scenario
Dimension

41. Dimension Relationships
Microsoft BI Voyage
Dimension Relationships
Define interaction between dimensions and measure groups
Relationship types
Regular
Reference
Fact (Degenerate)
Many-to-many
Data mining

42. Dimension Model Attributes Hierarchies 8/7/2018 12:46 PM Country
State
Marital
City
Gender
Gender
State
Customer
Customer
Customer
City
Gender
Marital
Age
Customer
State
Gender
City
Country
Marital
Customer
Attributes
Hierarchies

43. Microsoft BI Voyage
Calculations
Expressions evaluated at query time for values that cannot be stored in fact table
Types of calculations
Calculated members
Named sets
Scoped assignments
Calculations are defined using MDX
MDX = MultiDimensional EXpressions

44. 8/7/ :46 PM
Demo
1. Using BIDS to Review Dimension Design 2. Cube Design and Functionality
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

45. Summary
As a platform for enterprise Business Intelligence you should consider three things:
A Data Warehouse
Process of Data Integration (incl. ETL)
Multidimensional Analysis (OLAP)
= SQL Server 2008 Engine, SSIS, and SSAS
Now you can support decision making and performance management through:
Reports, dashboards, Excel integration, data mining, and better business software

46. © 2009 Microsoft Corporation & Project Botticelli Ltd
© 2009 Microsoft Corporation & Project Botticelli Ltd. All rights reserved.
The information herein is for informational purposes only and represents the opinions and views of Project Botticelli and/or Rafal Lukawiecki. The material presented is not certain and may vary based on several factors. Microsoft makes no warranties, express, implied or statutory, as to the information in this presentation.
Portions © 2009 Project Botticelli Ltd & entire material © 2009 Microsoft Corp. Some slides contain quotations from copyrighted materials by other authors, as individually attributed or as already covered by Microsoft Copyright ownerships. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Project Botticelli Ltd as of the date of this presentation.  Because Project Botticelli & Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft and Project Botticelli cannot guarantee the accuracy of any information provided after the date of this presentation. Project Botticelli makes no warranties, express, implied or statutory, as to the information in this presentation. E&OE.