7 Few questions to start with ... how many of you worked in PowerPivot?how many used PowerPivot on SharePoint?how many of you has experience in building cubes?how many used cubes for reporting?[to get a feel of the audience and their experience, in order to be able to adjust the speed and the depth of the information presented in the presentation slides]=> I estimate there were ~50 attendees, only ~15% had previous experience with OLAP/PowerPivot
9 Analytical systems turn data into information goals of their architectureoptimize access to source dataprepare the data for analysisIn order to explain what BISM is, we’ll first start with the analytical systems in general.There are two main goals in their architecture, something that every analytical system inherintly must do: taking care of data access and enriching the data for analysis.
10 I – Optimize access to source data store the data in structures suitable for analysis and ad-hoc querying or leave them „as is“ on the sourcerequirementsgood performanceflexibilityscalability...which technology to choose/use?Data can be left in the source or aquired and stored in appropriate format in another place (moved). Thereby certain requirements must be met: performance flexibility, scalability and so on. Which technology is the right one?
11 MOLAP, ROLAP, HOLAP row-based databases MOLAP = multidimensionalROLAP = relationalHOLAP = hybrid (only for measure groups = facts)performance based on aggregationspartitioningcompression (MOLAP)not limited by available memorySQL Server Analysis Services is architectured to store the data in one of the three storage modes. In case of ROLAP and HOLAP, data is left in the source system; in case of MOLAP, it is aquired and compressed on a separate location. In other words, existing SSAS supports both scenarios for data access.Other characteristics are that the data can be partitioned, aggregations can be built, and that the storage is designed not to be dependent on available memory.
12 II – Prepare the data for analysis enable the data analysis like there’s only one source of dataactually, there can be many various data sourcesrelational databasesExcel filestext filesRSS feeds, ...where to perform the join of the distributed systems?provide the interpretation (business meaning) to datahow to make it comprehendable to the end users?From the data preparation perspective, data from various sources can be gathered and represented to end-users like it comes from a single source. One of the challenges is where to perform this virtual integration?The other is how to make it understandable to end-users, how to enrich the data with meaning?
13 Semantic data model a metadata layer (on top of the data) describes concepts (entities) and connections (relations) between themuser-orientedshows what the tables and relations represent in the real worldsemantics = the study of meaningSemantic data model serves that purpose. It’s a user-oriented metadata layer that comes on top of the data and enriches it.BI developers model the data in such a way that it represents something meaningful to the end-users.The quality of the analytical solution depends on two things: the richness or the capabilities of the underlying semantic model and the knowledge BI developer posseses regarding that model.
14 UDM existing semantic model for BI came with SQL Server 2005 enables connecting data from various sourcesadd relations between tablesadd calculationsin tables – calculated columns in DSV (T-SQL)over the model – MDX script (MDX)calculations cannot be combined (different languages!) or it’s very hard to do soSQL Server Analysis Services uses UDM. UDM stands for Unified Dimensional Model. It exists from SQL Server 2005 onwards and it features two main designing capabilities: building relations and calculations.
15 UDM – data modeling 1/3 Here’s the example of UDM in action A DSV (data source view) and its objects is a place where the relations are made and various data sources are virtually linked. The model supports multiple key relations, parent-child relations, many-to-many relations, custom views and calculations on top of the existing objects (tables or views) and other friendly features.
16 UDM – data modeling 2/3The Dimension designer is a place where we design dimensions, establish attribute relationships, build custom hierarchical structures and many other things. The designer allows for a very fine and detailed configuration of dimensions and its properties.
17 UDM – data modeling 3/3The Cube designer is a place where we design cubes, establish relation between fact tables and dimensions, provide custom calculations, actions, design partitions and aggregations and many other things. User interface and exposed properties allow for a very fine tuning of the cube and its minor objects.
18 UDM – data analysisFinally, the model can be analyzed in the developers environment or in a separate tool.Here, we’re seeing the data through the prism of the semantic model made by a BI developer for us. The richness of the model enables fast navigation and flexible combination of objects in the model.
19 SQL Server 2005 Analysis Services UDMExcel add-inExcel 2007&Reporting Services 2005Report Builder 1SharePoint 2003 & SharePoint 2007Reporting Services3rd party SSAS clientsPerformancePoint 2007MDXMDXSQL Server Analysis Services supports the UDM as its semantic model from the version This diagram shows which tools were available to connect and query the UDM cubes and the query language they used to get the data from UDM cubes.MDXMDXMDX
20 SQL Server 2008 Analysis Services UDMExcel 2007Reporting Services 2008Report Builder 2&SharePoint 2007Reporting ServicesPerformancePoint ServicesExcel ServicesVisio Services3rd party SSAS clientsMDXMDXThe support for UDM continues in SSAS Excel add-in becomes obsolete and Performance Point becomes a service inside the SharePoint. Excel Services and Visio Services emerge there too.MDXMDX
21 SQL Server 2008 R2 Analysis Services UDMExcel 2010Reporting Services 2008 R2Report Builder 3&SharePoint 2010Excel ServicesPerformancePoint ServicesVisio Services3rd party SSAS clientsMDXMDXFinally, the current version of the SQL Server Analysis Services supports UDM as well. No big changes, only new versions of existing applications.MDXMDX
22 PowerPivot add-in for Excel 2010 PPExcel 2010But, at the same time, something else appears. Something completely different. That’s PowerPivot and its data model!
24 PowerPivotSQL Server 2008 R2 Analysis Services as an integrated versioncovers personal and team BI segmentsIt’s SSAS undercover, a managed version of it.
25 Integration Excel 2010 SharePoint 2010 free add-in for Excel 2010 (www.powerpivot.com)comes as 32-bit and 64-bit applicationcomes with SQL Server 2008 R2 x64during the installation, you choose SQL Server PowerPivot for SharePointIt’s an analytical system that comes in two shapes: one, for Excel and the other, for SharePoint.
26 VertiPaq in-memory column-based database very high data compression scales linearly with the number of coresdoesn’t require the process of designing and building aggregations and other tunningenvisioned to support partitioning and paging on large data sizes (although it doesn’t do that in PowerPivot)Like any other analytical system, PowerPivot must also take care about storing the data and optimizing it for analysis.Instead of MOLAP, ROLAP and HOLAP, PowerPivot uses its own storage mode called VertiPaq, a vertically organized database.
27 BISM v1 the new semantic model for BI came with SQL Server 2008 R2 (for PowerPivot!)enables connecting data from various data sourcesadd relations between tablesadd calculations, two places:in tables – calculated columns (DAX)over the whole model – calculated measures (DAX)calculations can be combined (same language)As mentioned earlier, the other characteristic of analytical systems is their semantic model. Again, PowerPivot has its own semantic model which currently doesn’t have a name, or it’s simply being called “the PowerPivot model”. And although it’s never been called BISM before, based on information about the next version of SQL Server called “Denali” it’s very reasonable to call it the version one of BISM.It came with SQL Server 2008 R2 and brought another way of defining relations and calculations. This time the calculation laguage was DAX. The good things about this model is that calculations defined on two separate places can be combined.
28 BISM v1the result of merging todays analytical (UDM = SSAS) and reporting (SDML = SSRS) semantic data model with the goal that it integrates (in due time)reporting, analysis, dashboards and scorecardspersonal, team and corporate BI segmentsworks in cached (VertiPaq) modeThe initial version of this semantic model is, just like the UDM, based on the cached storage.
29 BISM v1 – data modelingThe designer’s interface is simplified in terms of configuration options, yet on the other hand, it is enriched in a sense that the designer sees the data during the modeling process. This, and the relatively easy principles of relating tables and expanding them with additional columns, is very attractive and effective way of modeling the data for analysis. And the simplicity and interactivity it offers makes it appealing for much wider audience than UDM.
30 BISM v1 – data analysisData analysis can be performed on spot, right in the model, right during modeling. In fact, the process of modifying the model and analyzing the data becomes a new way of getting insights about the data. Although not nearly as rich as UDM in various features, this flexible semantic model wins in terms of interactivity that one can experience while working with data.
31 DEMO BISM v1 in SQL Server 2008 R2 Now comes the time for a little demo. The problem is - there is no publicly available version of BISM yet.What to show in the demo?As you’re about to see, there are lots of things hidden in the current release of SQL Server, the 2008 R2 version of it, that prove that BISM is already here although we’re not aware of it at the moment. Here we go!
32 Short contents of the demo PowerPivot is an arhived SSAS databaseDAX functions are hidden among MDX functionsSharePoint controls with the BISM instance of SSASthe existance of new DMVs for BISM instance on SharePointBISM models support MDXSQL Profiler VertiPaq SE eventsFor those of you that missed the demo, here’s a short overview of the things shown in the demo:Open Excel, start PowerPivot, create a model, save the file, close it, rename the extension of the file (xlsx to zip), find the “item1” file inside that zip, change its extension to “abf” and restore it on the SharePoint PowerPivot instance (PowerPivot model uses the existing SSAS UDM database container)Start a new MDX query in SSMS, go to the functions list, expand several groups of functions, notice empty functions (no names), drag and drop few of those empty functions in the editor and see what they turn into - those are DAX functions (EARIER, ...) hidden among regular MDX functions; the same thing can be shown using DMVs (the Origin field represents: 1 = UDM/MDX, 3 = VBA/EXCEL, 4= BISM/DAX)Connect SSMS to SSAS running in integrated mode on SharePoint, navigate the databases and notice their names - guids are a sign the SharePoint takes care of them completelyThe last 3 DMVs show tables for SSAS running on SharePoint, their memory footprints, etc.MDX query can be run against SSAS cube running in SharePoint integrated modeMDX queries targeting SSAS on SharePoint show VertiPaq SE events in SQL Server Profiler
34 BISM v2 one model for all reporting, analysis, dashboards, scorecards personal, team, corporate BIhas a relational and multidimensional APIit was announced that it will eventually support both cached (VertiPaq) and the pass-through* (realtime) mode* only SQL Server data sources for nowBISM is a semantic model envisioned to encompass reporting and analysis in a broader sense.
35 Pass-through no additional database data stays as is in the original structuresideal for the realtime analysis“Denali” will enable the other type of data access, the so-called “pass-through” mode. As mentioned in the introduction, one of the option is to leave the data in the source system. “Pass-through” is that option; VertiPaq means the data is aquired and stored separately.
36 BI semantic model and its layers Data modelrelational modelsupports both relational (DAX) and multidimensional(MDX) interfaceBusiness logicDAX calculations (columns and measures)after„Denali” maybe even MDXData accesscached (VertiPaq)in „Denali” also the realtime (pass-through mode)The slide is based on the TechNet source:It shows the key BISM features across three important layers. The lower one is what we called “optimizing data access” in the introduction of this presentation, the upper two are the semantic layer on top of the data: establishing relations, that is, data modeling, and providing calculations, that is, enriching the data model.
37 Analysis Services „Denali” - UDM Excel 2010Reporting Services „Denali”SharePoint 2010Excel ServicesReporting ServicesPerformancePoint ServicesVisio Services3rd party SSAS clientsProject CrescentMDXMDXAs before, UDM in will continue to enjoy the support of various applications in SQL Server “Denali”. In addition to that, new applications may appear as long as they support MDX as the query language.MDXMDX?MDX
38 Analysis Services „Denali” - BISM Excel 2010Reporting Services „Denali”SharePoint 2010Excel ServicesReporting ServicesPerformancePoint ServicesVisio Services3rd party SSAS clientsProject CrescentDAX?MDXMDXBISM, on the other hand will have its standalone incarnation in “Denali”. Because of its support for both multidimensional and relational interface, existing applications will be able to use it by issueing MDX queries. However, new applications will emerge, the ones that will support DAX query language. Will some of the existing application or services that we know support DAX too? We are about to see that soon ...MDXDAXMDXDAX
39 PowerPivot „Denali” add-in for Excel 2010 BISMExcel 2010Excel and its PowerPivot stay together as an integrated environment for personal BI.
40 Announced improvements of BISM* hierarchies, KPIs, parent-child, drillthrough, perspectivesadditional DAX functions (RankX, DistinctCount, GroupBy, Lookup, ...)security (role-based with Active Directory, column/row based)* doesn’t mean that „Denali“ will actually have these exact featuresThe following features were mentioned during the PASS Summit 2010 conference
41 What BISM in „Denali” won’t have? some of the UDM featuresscripts, actions, translations, role-playing dimensionsobject modelwrite-backotherrealtime for non-SQL Server data sourcesMDX query support for realtimeAlso, based on the PASS Summit 2010 announcements
42 UDM improvements in „Denali” mostly outside of GUIno more 4GB string store limitbetter support for PowerShellbetter monitoring (XEvents)various engine optimizationsthe goal is to make the SSAS (UDM instance) more scalableUDM will have its own improvements, although many will not be visible.
43 The Significance of BISM 4Which brings us to the significants of BISM. How important is BISM, what it really means to all of us?The Significance of BISM
44 Advantages of BISMrelatively simple model (no cubes, dimensions, measures, ...)fast response no matter the granularity of queryflexible (every column is also a measure and an attribute)DAX calculations are similar to Excel formulasrelational model and result format is more understandable and user-friendly to majority of peopleBISM brings simplicity in analytics. It lowers the entering bar and hence allows more people to do BI with their data, whatever that may be to them.
45 The same semantic model PowerPivot for ExcelPowerPivot for SharePointSSAS „Denali”Another important aspect is that the same semantic model will be available in all usage scenarios: personal BI, team BI and corporate BI.
46 Scalability Excel SharePoint „Denali” Having the same model that can scale from a personal BI all the way up to the corporate BI is a great proposition. That means the model is simple enough to be used in the small environment by non-professionals, yet it is powerful enough to support, in the beginning definitely some and later other corporate scenarios.Here, it is appropriate to mention that the model started with Excel and SharePoint, and that “Denali” will be the first stand-alone version of the model, an instance managed by IT professionals.
47 Other advantages of BISM faster development than in UDMprototyping by end-usersthey know the best what they want from the data and what the data means (semantic!)easier changes of modelreduction of cost in developing the full BI solutionAmong other benefits BISM promises to bring it’s worth emphasizing that BISM, in personal and team BI, allows end-users to model the data the way they want which is good because they are the ones who understand the meaning of that data the best
48 New possibilities in data modeling Highlights from various blogsMarco Russo – ad-hoc joins, ETL?Thomas Ivarsson – master-detail designAlberto Ferrari – point-in-time, SCD-2, rankLast but not the least, BISM bring new possibilities in data modeling. Here are some of the ideas or experiments.
49 5So what are the guidelines? How does the BISM positions in relation to, let’s say, UDM or other technologies?Guidelines
50 Positioning the BISM Scalability* ROLAP MOLAP BISM PowerPivot source: Thomas Kejser, SQLCATROLAPColumnStore50 TBScalability=- good support for concurrent queries- not dependent on available memory- good maintainability5 TBMOLAPMOLAP is much more complex than PowerPivot, but it offers greater scalability. Although it is everything but user-friendly, we can make great solutions with it, solutions which can scale up to 5TB.ROLAP is even more limited, but it scales above 50TB space.PowerPivot models can grow up to 2GB which is the limit set by SharePoint if they want to be shared among others. Otherwise, only the memory is the limit.BISM comes in the middle and fills the space between MOLAP and PowerPivot. It adds extra features (and hence becomes less usable by everyone), but on the other hand, it comes as a stand-alone SSAS instance which means it can scale with the memory that is available on the server. Let’s say that today the practical limit is around the 100 GB limit (it’s not just about plugging more chips, data needs to be loaded in memory and that takes time).For the space way above the 50TB there are new ColumnStore indexes (coming in relational engine).SQLCAT (http://sqlcat.com)100 GBBISM2 GBPowerPivotUsability
51 Conclusion BISM is not a replacement for UDM DAX is not a replacement for MDXevery model has its advantagesBI architects must decide when to apply which modelTo conclude ... Both models have a place dedicated for themselves, situations where they excel and situations where they are definitely not the best choice. This is how we should understand them.
52 Advantages and disadvantages BISMUDMscopeleaf levelm2mBoth BISM and UDM have their advantages and disadvantagesad-hoc modelscomplex models
53 BI developers most of the existing solutions should stay on UDM new solutionsuse UDM for complex systems (complex calculations, write-back, budgeting, forecasting, ...)try BISM as a potential solution for the rest, mostly simple systemsUDM and BISM can be installed on the same server (SSAS instances)BI developers should be familiar with both models (UDM and BISM) and know that for the majority of complex solutions, UDM is still the only way to go. The good thing is that UDM and BISM should be able to co-exist on the same server?
54 Users, here’s your DIY kit! try and/or migrate to Office 2010install the free PowerPivot for Excel 2010 add-inload and connect various data sourcestry and learn DAXanalyze your datavalidate the completness of your modelUsers and power-users, what about them?They finally got their Do-It-Yourself kit! They can play with it, experiment, the learning curve is not that hard. They should be able to build at least the simple models.
55 Model is ok? use it for your personal BI share the model with others Excel 2010 and PowerPivotshare the model with othersSharePoint 2010 and soon SQL Server „Denali“If the model they’ve built is ok, they can use it or share it with others
56 Model is not (yet) ok? improve it more experienced colleges give the PowerPivot model to your company guruexternal consultantimprovement is possible?PowerPivotSQL Server „Denali“ (BISM instance)or else ...build the UDM model based on the BISM templateimplement it on SQL Server „Denali“ (UDM instance)If the model is to simple and needs enhancements, there are two options. Either there’s a guru inside the company who can help or the help should be asked from the external consultant. In either way, the potential solution exist.But what’s more important, the model can, as said before, end up being a PowerPivot application in Excel or SharePoint or it can appear as a new database on SQL Server. “Denali”. All options are open!
57 source: TechNetThat’s what BISM promises to bring!