3. SQL Server 2008 BI-sovelluskehitys uudet ominaisuudet Pekka Korhonen Seniorikonsultti FC Sovelto Oyj

4. Business Intelligence BI is used to… –Understand the health of the organization –Collaborate on a shared view of business drivers –Reduce the time to decision Its goal is often to… –Impact the bottom line by measuring specific operations –Enhance competitive advantage BI is no longer a luxury afforded by a few large companies—it is now considered an essential part of the IT portfolio

5. Source Systems Process real-time transactions Contain data structures optimized for modifications –Normalized schema –Minimal indexing strategy Usually provide limited decision support Are commonly referred to as: –Online transaction processing (OLTP) systems –Operational systems HR Finance Inventory

6. Silos of Data Data Warehouse Call Center Web Apps Inventory ERPHR Finance CRM 5

7. Data Warehouse Characteristics Data warehouse systems… –Present data for business analysis processes –Commonly store data in subject-specific stores called data marts –Contain structures optimized for rapid ad hoc information retrieval –Combine valid source data –Integrate data from heterogeneous source systems –Provide a consistent historical data store

8. Extract, Transform, and Load 1.Extract data from the source systems 2.Transform the data to convert it to a desired state 3.Load the data into the data warehouse ETL

9. Analytical Systems Multidimensional databases are also called online analytical processing (OLAP) databases and… –Contain structures optimized for rapid ad hoc information retrieval –Pre-calculate and store aggregated values –Include calculation engines for fast, flexible transformation of base data –Are designed to reveal business trends and statistics not directly visible in the data retrieved from a data warehouse Data mining models discover patterns in data, typically for prediction analysis Sales Finance Product Association

10. Client Access Client access and distribution mechanisms can include: –Static report viewers and browsers –Ad hoc query tools –Report writers –Modeling applications –Scorecard applications –Portals and dashboards Delivering data is a process of continuous business improvement: –Monitor –Analyze –Plan What happened? What is happening? Why? What will happen? What do I want to happen?

11. Integrated Reporting and Analytics Data Sources Staging Area Manual Cleansing Data Marts Data Warehouse Client Access 1: Clients need access to data 2: Clients may access data sources directly 3: Data sources can be mirrored/replicated to reduce contention 4: The data warehouse manages data for analyzing and reporting 5: Data warehouse is periodically populated from data sources 6: Staging areas may simplify the data warehouse population 7: Manual cleansing may be required to cleanse dirty data 8: Clients use various tools to query the data warehouse 9: Delivering BI enables a process of continuous business improvement 10

12. Data Platform Information Worker Platform Microsoft BI Platform Enterprise Grade Pervasive Integrated Flexible Full Featured Interoperable Extensible Powerful Cost Effective Fast Time-to-Market Choice of Integration Points Performance Management Integrated BI Solution

13. SQL Server 2008 BI Platform Data acquisition from source systems and integration Data transformation and synthesis Data enrichment, with business logic, hierarchical views Data discovery via data mining Data presentation and distribution Data access for the masses Integrate Analyze Report

14. SSIS - Script Task Editor Choose script language when adding task or component Use Edit Script button on first page for easy access Select ReadOnlyVariables and ReadWriteVariables from list

15. Web Service Script Design Step 1 Add a Web Reference to the project Provide the URL to the ASMX or WSDL file of the service

16. Web Service Script Design Step 2 v Include the full name of the object in the Using directive

17. Web Service Script Design Step 3 Instantiate an object for the Web service and its methods Compile code

18. Improving Package Performance Persistent Lookups –Benefits of Persistent Lookups –Lookup Cache Types –Cache Connection Manager –Cache Transform –Lookup Configuration Pipeline Scalability –Benefits of Pipeline Scalability –Thread Scheduler

19. Lookup Cache Types No Cache Reference dataset uses OLE DB connection Lookup executes one query for each row in the pipeline No match handler: 1.Ignore 2.Redirect to error output 3.Fail component 4.Redirect to no match output N Match? Lookup Reference Dataset Pipeline Y

20. Lookup Cache Types Partial Cache Reference dataset uses OLE DB connection Lookup searches cache first, then executes non-cache query if index columns not found in cache No match handler: 1.Ignore 2.Redirect to error output 3.Fail component 4.Redirect to no match output N Match? Lookup Reference Dataset Pipeline Y Hit/Miss Cache

21. Lookup Cache Types Full Cache In-Memory Reference dataset uses OLE DB connection Cache loads into memory during PreExecute phase and remains static throughout package execution 20 No match handler: 1.Ignore 2.Redirect to error output 3.Fail component 4.Redirect to no match output N Match? Lookup Reference Dataset Pipeline Y Cache PreExecute

22. Data Profiling Task Profiles tables for exploring or preserving data quality –Run as a task in SSIS –Produces XML file output –Has a nice visual tool for working with profiles Analyzes a set of columns / tables –Candidate keys –Functional dependencies –Value inclusion Analyzes a single column –Column length distribution –Null Ratio –Pattern detection –Statistics –Value distributions

23. Set-Based Profile Types Functional Dependency Determine whether the dependent column depends on the values in the determinant column(s) Find invalid values, such as incorrect combinations of US zip codes and US states 22

24. Column-Based Profile Types Column Length Distribution Calculate distinct lengths of string values and percentage of rows each length represents Find invalid values, such as values shorter or longer than expected

25. Column-Based Profile Types Column Null Ratio Calculate percentage of null values Find unexpectedly high ratio of null values, such as a column with high percentage of missing codes

26. Column-Based Profile Types Column Value Distribution Calculate distinct values and percentage of rows for each value Find anomalous distinct values

27. Drilldown to details Browse profiles by column

28. Table-level implementation to track changes in a relational structure Change data stored in tables –Details about inserts, updates, and deletes –Log sequence number (LSN) for the commit transaction –Begin and end time of each LSN Stored procedures and functions available to query for configuration or change data details Alternative approach to managing slowly changing dimension scenarios Change Data Capture

29. Change Data Capture Configuration Enabling a database for CDC –EXECUTE sys.sp_cdc_enable_db_change_data_capture; Enabling a table for CDC –EXECUTE sys.sp_cdc_enable_table_change_data_capture @source_schema = 'Person', @source_name = 'Contact', @role_name = 'cdc_admin', @filegroup_name = ‘CDC'

30. Report Designer Layout Globals Data Pane Paramete rs Data Source & Dataset Office- style Ribbon Propertie s Pane Group Task Pane Expression Placeholder s

31. 20012002Total 20,23574,28194,517 10,085 8,369 4,545,3379,190,83813,736,175 2,850,01210,765,17713,615,189 7,415,58420,048,75027,464,334 AccessoriesHelmets Locks Pumps BikesMountain Bikes Road Bikes Grand Total Product Accessories Helmets Locks Pumps Bikes Mountain Bikes Road Bikes Grand Total Avg Sale 18.19 19.56 14.92 11.94 964.54 1,445.61 722.10 794.52 ix table + Matr 20,23592,735112,971 7,395,34919,956,01527,351,363 Introducing Tablix

32. Flexible grid layout –Fixed columns and dynamic rows like a Table –Dynamic rows and columns like a Matrix –Any combination of Table and Matrix layouts Flexible grouping –Nested groups –Adjacent groups –Recursive groups

33. Hierarchical rows with dynamic headers 20012002 Accessories20,23592,735 Helmets20,23574,281 Locks10,085 Pumps8,369 Bikes7,395,34919,956,015 Mountain Bikes4,545,3379,190,838 Road Bikes2,850,01210,765,177 20012002 AccessoriesTotal20,23592,735 Helmets20,23574,281 Locks10,085 Pumps8,369 BikesTotal7,395,34919,956,015 Mountain Bikes4,545,3379,190,838 Road Bikes2,850,01210,765,177 Current Desired Reviewing Tablix Examples

34. Mixing dynamic and static columns Avg Sale AccessoriesHelmets19.56 Locks14.92 Pumps11.94 BikesMountain Bikes1,445.61 Road Bikes722.10 20012002 AccessoriesHelmets20,23574,281 Locks10,085 Pumps8,369 BikesMountain Bikes4,545,3379,190,838 Road Bikes2,850,01210,765,177 Current Desired 20012002Avg Sale AccessoriesHelmets20,23574,28119.56 Locks10,08514.92 Pumps8,36911.94 BikesMountain Bikes4,545,3379,190,8381,445.61 Road Bikes2,850,01210,765,177722.10 Reviewing Tablix Examples

35. Parallel Dynamic Groups EuropeNorth America AccessoriesHelmets6,96387,554 Locks1,0509,035 Pumps9637,406 BikesMountain Bikes 569,24413,166,930 Road Bikes 731,68212,883,507 20012002 AccessoriesHelmets20,23574,281 Locks10,085 Pumps8,369 BikesMountain Bikes 4,545,3379,190,838 Road Bikes 2,850,01210,765,177 Current Desired 20012002EuropeNorth America AccessoriesHelmets20,23574,2816,96387,554 Locks10,0851,0509,035 Pumps8,3699637,406 BikesMountain Bikes4,545,3379,190,838569,24413,166,930 Road Bikes2,850,01210,765,177731,68212,883,507 Reviewing Tablix Examples

36. Working with the Chart Layout Smart Tags Edit and format chart title here Fly-out menu Calculate d Series

37. Using New Chart Features Secondary Axes Scale BreakMultiple chart areas with optional alignment

38. Introducing Gauges and Indicators Display and monitor real-time data Use as dashboard or scorecard components for visualizing KPIs

39. Design Improvements in Analysis Services 2008 Improved ease of use Decreased time to develop solutions Embedded best practices and performance tuning tips into object model and user interface Redesigned interface to ensure the natural outcome is optimal design

40. Cube and Dimension Wizard Improvements Cube Wizard –Supports building cube from one table –Produces simpler output –Provides safer error configuration settings Dimension Wizard –Creates parent-child attributes automatically –Enables assignment of attribute type –Supports classification of member properties –Provides safer error configuration settings

41. Best Practices Warnings Visual indicators to highlight best practice violations Ability to dismiss warnings by instance or globally with optional comment

42. Attribute Relationship Designer Graphical editor for attribute relationships Visualization of rigid and flexible relationships

43. Aggregation Design Wizard Improvements Ability to review and modify aggregation usage settings Name the aggregation design

44. Aggregation Designer View aggregation designs and aggregations Add, change, or delete aggregations manually Assign an aggregation design to another partition