1. Best practice using Power pivot. Henk Vlootman, Excel MVP

2. Thanks to our platinum sponsors :
PASS SQL Saturday – Holland

3. Thanks to our silver and gold sponsors :
PASS SQL Saturday – Holland

4. Agenda Introduction Need to know about Power pivot Easy tips
Tips for the Pro’s.
11/20/2018 |
Agenda

5. Flavors and differences
Excel Power pivot
Way of work
Differences in building Excel and SQL PP models.
Not to do: Linked tables
SQLPower pivot
11/20/2018 |
Need to know about Power pivot

6. The flavors of Power pivot
Excel Power pivot = desktop app = single user = private
SQL Power pivot = server app = real shared = public
Combo:
Excel Power pivot = prototyping
SQL Power pivot = production
11/20/2018 |
Need to know about Power pivot

7. The Structure of SQL Power pivot
11/20/2018 |
Need to know about Power pivot

8. Different kind of tables in Power pivot
Power pivot works with tables. Be aware there that there are different kind of tables. I find the next classification handy:
Dim(ension) tables
Fact tables
Measure tables
Variable tables
Fact tables – contains data from the database. Used to make your calcultions.
Dim Tables – contains the data to make relationships. Used by presentation as filter items
Measure tables – contains de measures of a Fact table. Just container of enriched calcultations. Based for the weakness of connections.
Variable tables – contains data to be connected to fact tables by unique key columns by calculation (LOOKUPVALUE).
11/20/2018 |
Need to know about Power pivot

9. Use empty tables to create Measure tables
How to create empty tables?
In Excel:
Select in a sheet any random blank cell and press Copy
Switch to Power pivot. Beware that the calculation section should not be active.
Choose for Paste and follow the instructions.
In SQL:
If you have a SQL database, choose Existing connections | Open
Use a dummy SQL script, give a (unique) friendly name.
The script could be something like:
Select “” as Column1
Beware of hiding columns
Use the second column for documentation.
Or:
Make a text file with only the header and import this one.
11/20/2018 |
Need to know about Power pivot

10. Relations: that is all about Power pivot
Relations seen through the eyes of the Excel user
Relations seen through the eyes of the BI specialist
What is a relation?
11/20/2018 |
Need to know about Power pivot

11. A little more on compression…
Power pivot uses two sorts of compression:
Hash Encoding:
Used for tables of unique value (read: Dim tables)
Value Encoding:
Used for numeric value
Mathematical patterns and CPU operations (read: Fact tables)
Fact tables only have columns with the datatype Integer.
Dim tables can have columns with datatype Text for slicing.
Key columns in Dim and Fact tables should always be of the datatype Integer.
Hash encoding: uses RAM -> this is slower.
Numeric value: read integer Can be placed in a register in the CPU
Use Power Query (get and transform) in Excel to replace text to integers
11/20/2018 |
Need to know about Power pivot

12. Narrow your tables.
Excel users are addicted to two things. One of them is data. They look at data as “I can get it, so it is mine…”.
Bi specialist tends to be better in importing data, but...
Here big gains can be found.
Be aware that extra columns can also drastically affect compression.
Columns can always be added later on when needed.
If possible don’t import columns with unique values in Fact tables.
11/20/2018 |
Easy tips

13. Shorten your tables.
Compression algorithms also works on rows in Columns. The more rows…
Filter out the rows you don’t need by importing the data. You can always add later on, if necessary.
11/20/2018 |
Easy tips

14. Narrow your calculations
As said: Excel users are addicted to two things. The first one is grabbing data when ever possible, the second one is calculations. They look at calculations as “I don’t need it, but I can calculate this, so lets do it…”.
Here also big gains can be experienced here.
Affected measures are always recalculated if you use slicers and timelines.
Be aware that filter actions (by a slicer for instance) can affect multiple tables and therefore calculations on more then one table..
Excel is calculation-driven
Power pivot is filter-driven
Do not think in calculations, think in filters!
11/20/2018 |
Easy tips

15. Calculated Columns or Measures?
Calculated Columns are recalculate by opening the data connection once. That means more Calculated Columns takes time at the initial opening of the Power pivot model. Usually the user of the model can live with that (he / she gets a cup of coffee).
Measures are recalculate when using filters (read: Slicers and timelines). That means: more measure more time required to reflect in the dashboard. Usually the user of the dashboard gets quite annoyed if he has to wait to see the results of his action.
Be sure to optimize your Measures towards filters. That will fasten your response time.
Although Calculated Columns are better in performance then Measures, they still take valuable memory. Can you avoid them, don’t hesitate.
11/20/2018 |
Easy tips

16. Import of Dim tables
Fact tables are based on databases, which can come from different sources. Dim tables for connecting the two together are therefore often not available. Excel can be a get tool to produce such tables. Create all dimension tables in one Excel model, for it is easier to manage them.
Files on a windows OS uses lock files to indicate that a file is in use. That means if your Excel file is locked, refresh will result in an error. You can prevent this by exporting the results as text files and then import those.
Text files don’t have an data type marking in it. So Power pivot decides on import the data type for every imported column. Unfortunately this data type can be changed, but sometimes the results are catastrophic. Use Schema.ini file at the first import to keep control of the data types within a text file.
11/20/2018 |
Easy tips

17. Slicers. Fancy but killing the speed
As said slicers increases refresh (and thereby waiting) time.
But slicer settings are also not optimized for performance.
As said slicers increases refresh (and thereby waiting) time.
But slicer settings are not optimized for performance.
Slicer setting
Refresh time
Hide items with no data
7,3
sec
Visually indicate items with no data
4,8
Show items with no data last
5,2
Visually indicate and show items last
* Based on a model with 5 tables, 70k rows of data and 47 slicers.
* Many thanks for Ken Puls for this table.
11/20/2018 |
Tips for the pro’s

18. Precision is priceless
Precision comes with a cost, namely memory. Decimals (which includes the time section of dates) will cost more memory. If you can split them in integers, you’ll be better off.
Memory stats based om ~1M rows of random datetimes spanning 17 years and holding precision to the second.
Many thanks for Ken Puls for this table.
Split for instance columns in Euro’s and cents of the data type Integer and re-aggregate the columns by Measures:
TotalEuro:=SUM([Euro])+DIVIDE(SUM([EuroCents]),100)
11/20/2018 |
Tips for the pro’s

19. References
Checklist for Memory Optimizations in PowerPivot and Tabular Models
Memory Considerations about PowerPivot for Excel
Creating a Memory Efficient Data Model Using Excel 2013 and Power Pivot
Pivot-add-in-951c73a9-21c4-46ab-9f5e-14a2833b6a70?CorrelationId=3569f6ee-c419-4b59-95ec-
2c726900e016&ui=en-US&rs=en-US&ad=US
How Does Power Pivot Store and Compress Data?
What is Eating Up My Memory in Power Pivot?
11/20/2018 |
Tips for the pro’s

20. Please review the event and sessions Y
You -> awesome
Me -> grateful
Henk Vlootman
Excel MVP
PASS SQL Saturday – Holland