1. Presentation held by Tomislav Piasevoli at the local WinDays 11 conference, Rovinj, Croatia. Monday, 16:10-17:00, Room 6.

2. Business Intelligence Semantic Model
Tomislav Piasevoli
SoftPro Tetral d.o.o.

3. As always, a big „Thank you” goes to sponsors ...

4. ... and you for keeping your phones muted.

5. Contents What is BISM? PowerPivot SQL Server „Denali”
The significance of BISM
Guidelines
This is what we’re going to talk about today

6. Prerequisites
interest in BI
curiosity 

7. Few questions to start with ...
how many of you worked in PowerPivot?
how many used PowerPivot on SharePoint?
how many of you has experience in building cubes?
how many used cubes for reporting?
[to get a feel of the audience and their experience, in order to be able to adjust the speed and the depth of the information presented in the presentation slides]
=> I estimate there were ~50 attendees, only ~15% had previous experience with OLAP/PowerPivot

8. 1
What is BISM?

9. Analytical systems turn data into information
goals of their architecture
optimize access to source data
prepare the data for analysis
In order to explain what BISM is, we’ll first start with the analytical systems in general.
There are two main goals in their architecture, something that every analytical system inherintly must do: taking care of data access and enriching the data for analysis.

10. I – Optimize access to source data
store the data in structures suitable for analysis and ad-hoc querying or leave them „as is“ on the source
requirements
good performance
flexibility
scalability
...
which technology to choose/use?
Data can be left in the source or aquired and stored in appropriate format in another place (moved). Thereby certain requirements must be met: performance flexibility, scalability and so on. Which technology is the right one?

11. MOLAP, ROLAP, HOLAP row-based databases
MOLAP = multidimensional
ROLAP = relational
HOLAP = hybrid (only for measure groups = facts)
performance based on aggregations
partitioning
compression (MOLAP)
not limited by available memory
SQL Server Analysis Services is architectured to store the data in one of the three storage modes. In case of ROLAP and HOLAP, data is left in the source system; in case of MOLAP, it is aquired and compressed on a separate location. In other words, existing SSAS supports both scenarios for data access.
Other characteristics are that the data can be partitioned, aggregations can be built, and that the storage is designed not to be dependent on available memory.

12. II – Prepare the data for analysis
enable the data analysis like there’s only one source of data
actually, there can be many various data sources
relational databases
Excel files
text files
RSS feeds, ...
where to perform the join of the distributed systems?
provide the interpretation (business meaning) to data
how to make it comprehendable to the end users?
From the data preparation perspective, data from various sources can be gathered and represented to end-users like it comes from a single source. One of the challenges is where to perform this virtual integration?
The other is how to make it understandable to end-users, how to enrich the data with meaning?

13. Semantic data model a metadata layer (on top of the data)
describes concepts (entities) and connections (relations) between them
user-oriented
shows what the tables and relations represent in the real world
semantics = the study of meaning
Semantic data model serves that purpose. It’s a user-oriented metadata layer that comes on top of the data and enriches it.
BI developers model the data in such a way that it represents something meaningful to the end-users.
The quality of the analytical solution depends on two things: the richness or the capabilities of the underlying semantic model and the knowledge BI developer posseses regarding that model.

14. UDM existing semantic model for BI came with SQL Server 2005
enables connecting data from various sources
add relations between tables
add calculations
in tables – calculated columns in DSV (T-SQL)
over the model – MDX script (MDX)
calculations cannot be combined (different languages!) or it’s very hard to do so
SQL Server Analysis Services uses UDM. UDM stands for Unified Dimensional Model. It exists from SQL Server 2005 onwards and it features two main designing capabilities: building relations and calculations.

15. UDM – data modeling 1/3 Here’s the example of UDM in action
A DSV (data source view) and its objects is a place where the relations are made and various data sources are virtually linked. The model supports multiple key relations, parent-child relations, many-to-many relations, custom views and calculations on top of the existing objects (tables or views) and other friendly features.

16. UDM – data modeling 2/3
The Dimension designer is a place where we design dimensions, establish attribute relationships, build custom hierarchical structures and many other things. The designer allows for a very fine and detailed configuration of dimensions and its properties.

17. UDM – data modeling 3/3
The Cube designer is a place where we design cubes, establish relation between fact tables and dimensions, provide custom calculations, actions, design partitions and aggregations and many other things. User interface and exposed properties allow for a very fine tuning of the cube and its minor objects.

18. UDM – data analysis
Finally, the model can be analyzed in the developers environment or in a separate tool.
Here, we’re seeing the data through the prism of the semantic model made by a BI developer for us. The richness of the model enables fast navigation and flexible combination of objects in the model.

19. SQL Server 2005 Analysis Services
UDM
Excel add-in
Excel 2007
&
Reporting Services 2005
Report Builder 1
SharePoint 2003 & SharePoint 2007
Reporting Services
3rd party SSAS clients
PerformancePoint 2007
MDX
MDX
SQL Server Analysis Services supports the UDM as its semantic model from the version This diagram shows which tools were available to connect and query the UDM cubes and the query language they used to get the data from UDM cubes.
MDX
MDX
MDX

20. SQL Server 2008 Analysis Services
UDM
Excel 2007
Reporting Services 2008
Report Builder 2
&
SharePoint 2007
Reporting Services
PerformancePoint Services
Excel Services
Visio Services
3rd party SSAS clients
MDX
MDX
The support for UDM continues in SSAS Excel add-in becomes obsolete and Performance Point becomes a service inside the SharePoint. Excel Services and Visio Services emerge there too.
MDX
MDX

21. SQL Server 2008 R2 Analysis Services
UDM
Excel 2010
Reporting Services 2008 R2
Report Builder 3
&
SharePoint 2010
Excel Services
PerformancePoint Services
Visio Services
3rd party SSAS clients
MDX
MDX
Finally, the current version of the SQL Server Analysis Services supports UDM as well. No big changes, only new versions of existing applications.
MDX
MDX

22. PowerPivot add-in for Excel 2010PP
Excel 2010
But, at the same time, something else appears. Something completely different. That’s PowerPivot and its data model!

23. 2
The story about PowerPivot ...
PowerPivot

24. PowerPivot
SQL Server 2008 R2 Analysis Services as an integrated version
covers personal and team BI segments
It’s SSAS undercover, a managed version of it.

25. Integration Excel 2010 SharePoint 2010
free add-in for Excel 2010 (
comes as 32-bit and 64-bit application
comes with SQL Server 2008 R2 x64
during the installation, you choose SQL Server PowerPivot for SharePoint
It’s an analytical system that comes in two shapes: one, for Excel and the other, for SharePoint.

26. VertiPaq in-memory column-based database very high data compression
scales linearly with the number of cores
doesn’t require the process of designing and building aggregations and other tunning
envisioned to support partitioning and paging on large data sizes (although it doesn’t do that in PowerPivot)
Like any other analytical system, PowerPivot must also take care about storing the data and optimizing it for analysis.
Instead of MOLAP, ROLAP and HOLAP, PowerPivot uses its own storage mode called VertiPaq, a vertically organized database.

27. BISM v1 the new semantic model for BI
came with SQL Server 2008 R2 (for PowerPivot!)
enables connecting data from various data sources
add relations between tables
add calculations, two places:
in tables – calculated columns (DAX)
over the whole model – calculated measures (DAX)
calculations can be combined (same language)
As mentioned earlier, the other characteristic of analytical systems is their semantic model. Again, PowerPivot has its own semantic model which currently doesn’t have a name, or it’s simply being called “the PowerPivot model”. And although it’s never been called BISM before, based on information about the next version of SQL Server called “Denali” it’s very reasonable to call it the version one of BISM.
It came with SQL Server 2008 R2 and brought another way of defining relations and calculations. This time the calculation laguage was DAX. The good things about this model is that calculations defined on two separate places can be combined.

28. BISM v1
the result of merging todays analytical (UDM = SSAS) and reporting (SDML = SSRS) semantic data model with the goal that it integrates (in due time)
reporting, analysis, dashboards and scorecards
personal, team and corporate BI segments
works in cached (VertiPaq) mode
The initial version of this semantic model is, just like the UDM, based on the cached storage.

29. BISM v1 – data modeling
The designer’s interface is simplified in terms of configuration options, yet on the other hand, it is enriched in a sense that the designer sees the data during the modeling process. This, and the relatively easy principles of relating tables and expanding them with additional columns, is very attractive and effective way of modeling the data for analysis. And the simplicity and interactivity it offers makes it appealing for much wider audience than UDM.

30. BISM v1 – data analysis
Data analysis can be performed on spot, right in the model, right during modeling. In fact, the process of modifying the model and analyzing the data becomes a new way of getting insights about the data. Although not nearly as rich as UDM in various features, this flexible semantic model wins in terms of interactivity that one can experience while working with data.

31. DEMO BISM v1 in SQL Server 2008 R2
Now comes the time for a little demo. The problem is - there is no publicly available version of BISM yet.
What to show in the demo?
As you’re about to see, there are lots of things hidden in the current release of SQL Server, the 2008 R2 version of it, that prove that BISM is already here although we’re not aware of it at the moment. Here we go!

32. Short contents of the demo
PowerPivot is an arhived SSAS database
DAX functions are hidden among MDX functions
SharePoint controls with the BISM instance of SSAS
the existance of new DMVs for BISM instance on SharePoint
BISM models support MDX
SQL Profiler VertiPaq SE events
For those of you that missed the demo, here’s a short overview of the things shown in the demo:
Open Excel, start PowerPivot, create a model, save the file, close it, rename the extension of the file (xlsx to zip), find the “item1” file inside that zip, change its extension to “abf” and restore it on the SharePoint PowerPivot instance (PowerPivot model uses the existing SSAS UDM database container)
Start a new MDX query in SSMS, go to the functions list, expand several groups of functions, notice empty functions (no names), drag and drop few of those empty functions in the editor and see what they turn into - those are DAX functions (EARIER, ...) hidden among regular MDX functions; the same thing can be shown using DMVs (the Origin field represents: 1 = UDM/MDX, 3 = VBA/EXCEL, 4= BISM/DAX)
Connect SSMS to SSAS running in integrated mode on SharePoint, navigate the databases and notice their names - guids are a sign the SharePoint takes care of them completely
The last 3 DMVs show tables for SSAS running on SharePoint, their memory footprints, etc.
MDX query can be run against SSAS cube running in SharePoint integrated mode
MDX queries targeting SSAS on SharePoint show VertiPaq SE events in SQL Server Profiler

33. 3
What comes next?
SQL Server „DENALI”

34. BISM v2 one model for all reporting, analysis, dashboards, scorecards
personal, team, corporate BI
has a relational and multidimensional API
it was announced that it will eventually support both cached (VertiPaq) and the pass-through* (realtime) mode
* only SQL Server data sources for now
BISM is a semantic model envisioned to encompass reporting and analysis in a broader sense.

35. Pass-through no additional database
data stays as is in the original structures
ideal for the realtime analysis
“Denali” will enable the other type of data access, the so-called “pass-through” mode. As mentioned in the introduction, one of the option is to leave the data in the source system. “Pass-through” is that option; VertiPaq means the data is aquired and stored separately.

36. BI semantic model and its layers
Data model
relational model
supports both relational (DAX) and multidimensional(MDX) interface
Business logic
DAX calculations (columns and measures)
after„Denali” maybe even MDX
Data access
cached (VertiPaq)
in „Denali” also the realtime (pass-through mode)
The slide is based on the TechNet source:
It shows the key BISM features across three important layers. The lower one is what we called “optimizing data access” in the introduction of this presentation, the upper two are the semantic layer on top of the data: establishing relations, that is, data modeling, and providing calculations, that is, enriching the data model.

37. Analysis Services „Denali” - UDM
Excel 2010
Reporting Services „Denali”
SharePoint 2010
Excel Services
Reporting Services
PerformancePoint Services
Visio Services
3rd party SSAS clients
Project Crescent
MDX
MDX
As before, UDM in will continue to enjoy the support of various applications in SQL Server “Denali”. In addition to that, new applications may appear as long as they support MDX as the query language.
MDX
MDX ?
MDX

38. Analysis Services „Denali” - BISM
Excel 2010
Reporting Services „Denali”
SharePoint 2010
Excel Services
Reporting Services
PerformancePoint Services
Visio Services
3rd party SSAS clients
Project Crescent
DAX ?
MDX
MDX
BISM, on the other hand will have its standalone incarnation in “Denali”. Because of its support for both multidimensional and relational interface, existing applications will be able to use it by issueing MDX queries. However, new applications will emerge, the ones that will support DAX query language. Will some of the existing application or services that we know support DAX too? We are about to see that soon ...
MDX
DAX
MDX
DAX

39. PowerPivot „Denali” add-in for Excel 2010
BISM
Excel 2010
Excel and its PowerPivot stay together as an integrated environment for personal BI.

40. Announced improvements of BISM*
hierarchies, KPIs, parent-child, drillthrough, perspectives
additional DAX functions (RankX, DistinctCount, GroupBy, Lookup, ...)
security (role-based with Active Directory, column/row based)
* doesn’t mean that „Denali“ will actually have these exact features
The following features were mentioned during the PASS Summit 2010 conference

41. What BISM in „Denali” won’t have?
some of the UDM features
scripts, actions, translations, role-playing dimensions
object model
write-back
other
realtime for non-SQL Server data sources
MDX query support for realtime
Also, based on the PASS Summit 2010 announcements

42. UDM improvements in „Denali”
mostly outside of GUI
no more 4GB string store limit
better support for PowerShell
better monitoring (XEvents)
various engine optimizations
the goal is to make the SSAS (UDM instance) more scalable
UDM will have its own improvements, although many will not be visible.

43. The Significance of BISM4
Which brings us to the significants of BISM. How important is BISM, what it really means to all of us?
The Significance of BISM

44. Advantages of BISM
relatively simple model (no cubes, dimensions, measures, ...)
fast response no matter the granularity of query
flexible (every column is also a measure and an attribute)
DAX calculations are similar to Excel formulas
relational model and result format is more understandable and user-friendly to majority of people
BISM brings simplicity in analytics. It lowers the entering bar and hence allows more people to do BI with their data, whatever that may be to them.

45. The same semantic model
PowerPivot for Excel
PowerPivot for SharePoint
SSAS „Denali”
Another important aspect is that the same semantic model will be available in all usage scenarios: personal BI, team BI and corporate BI.

46. Scalability Excel SharePoint „Denali”
Having the same model that can scale from a personal BI all the way up to the corporate BI is a great proposition. That means the model is simple enough to be used in the small environment by non-professionals, yet it is powerful enough to support, in the beginning definitely some and later other corporate scenarios.
Here, it is appropriate to mention that the model started with Excel and SharePoint, and that “Denali” will be the first stand-alone version of the model, an instance managed by IT professionals.

47. Other advantages of BISM
faster development than in UDM
prototyping by end-users
they know the best what they want from the data and what the data means (semantic!)
easier changes of model
reduction of cost in developing the full BI solution
Among other benefits BISM promises to bring it’s worth emphasizing that BISM, in personal and team BI, allows end-users to model the data the way they want which is good because they are the ones who understand the meaning of that data the best

48. New possibilities in data modeling
Highlights from various blogs
Marco Russo – ad-hoc joins, ETL?
Thomas Ivarsson – master-detail design
Alberto Ferrari – point-in-time, SCD-2, rank
Last but not the least, BISM bring new possibilities in data modeling. Here are some of the ideas or experiments.

49. 5
So what are the guidelines? How does the BISM positions in relation to, let’s say, UDM or other technologies?
Guidelines

50. Positioning the BISM Scalability* ROLAP MOLAP BISM PowerPivot
source: Thomas Kejser, SQLCAT
ROLAP
ColumnStore
50 TB
Scalability=
- good support for concurrent queries
- not dependent on available memory
- good maintainability
5 TB
MOLAP
MOLAP is much more complex than PowerPivot, but it offers greater scalability. Although it is everything but user-friendly, we can make great solutions with it, solutions which can scale up to 5TB.
ROLAP is even more limited, but it scales above 50TB space.
PowerPivot models can grow up to 2GB which is the limit set by SharePoint if they want to be shared among others. Otherwise, only the memory is the limit.
BISM comes in the middle and fills the space between MOLAP and PowerPivot. It adds extra features (and hence becomes less usable by everyone), but on the other hand, it comes as a stand-alone SSAS instance which means it can scale with the memory that is available on the server. Let’s say that today the practical limit is around the 100 GB limit (it’s not just about plugging more chips, data needs to be loaded in memory and that takes time).
For the space way above the 50TB there are new ColumnStore indexes (coming in relational engine).
SQLCAT (
100 GB
BISM
2 GB
PowerPivot
Usability

51. Conclusion BISM is not a replacement for UDM
DAX is not a replacement for MDX
every model has its advantages
BI architects must decide when to apply which model
To conclude ... Both models have a place dedicated for themselves, situations where they excel and situations where they are definitely not the best choice. This is how we should understand them.

52. Advantages and disadvantages
BISM
UDM
scope
leaf level
m2m
Both BISM and UDM have their advantages and disadvantages
ad-hoc models
complex models

53. BI developers most of the existing solutions should stay on UDM
new solutions
use UDM for complex systems (complex calculations, write-back, budgeting, forecasting, ...)
try BISM as a potential solution for the rest, mostly simple systems
UDM and BISM can be installed on the same server (SSAS instances)
BI developers should be familiar with both models (UDM and BISM) and know that for the majority of complex solutions, UDM is still the only way to go. The good thing is that UDM and BISM should be able to co-exist on the same server?

54. Users, here’s your DIY kit!
try and/or migrate to Office 2010
install the free PowerPivot for Excel 2010 add-in
load and connect various data sources
try and learn DAX
analyze your data
validate the completness of your model
Users and power-users, what about them?
They finally got their Do-It-Yourself kit! They can play with it, experiment, the learning curve is not that hard. They should be able to build at least the simple models.

55. Model is ok? use it for your personal BI share the model with others
Excel 2010 and PowerPivot
share the model with others
SharePoint 2010 and soon SQL Server „Denali“
If the model they’ve built is ok, they can use it or share it with others

56. Model is not (yet) ok? improve it more experienced colleges
give the PowerPivot model to your company guru
external consultant
improvement is possible?
PowerPivot
SQL Server „Denali“ (BISM instance)
or else ...
build the UDM model based on the BISM template
implement it on SQL Server „Denali“ (UDM instance)
If the model is to simple and needs enhancements, there are two options. Either there’s a guru inside the company who can help or the help should be asked from the external consultant. In either way, the potential solution exist.
But what’s more important, the model can, as said before, end up being a PowerPivot application in Excel or SharePoint or it can appear as a new database on SQL Server. “Denali”. All options are open!

57. source: TechNet
That’s what BISM promises to bring!

58. Questions?

59. Related content Presentations Workshops
MS BI Roadmap (SQL Server Denali)
Workshops
Build Your First Microsoft BI Solution with SQL Server 2008 R2

61. Thank you!