1. Modern Business Intelligence Platforms using Azure in PaaS
Florian

2. 50 Consultants 10 MVP 7 MCT > 30 > 100 blog posts per year
Cellenza
156, bd Haussmann
75008 Paris, France
50 Consultants
Cloud & Integration
Collaboration & Content
Data & Analytics
Application Development
ALM & Agile Dev.
10 MVP
7 MCT
4 P-Sellers
Over 12d Training
Per year
> 30
Press articles
per year
> 100 blog posts per year

3. Modern
Business Intelligence
Platforms
Azure
PaaS : Platform as a Service

4. Why even bother?
Life is short
Do Once
Deploy
Improve

5. In Business Intelligence?
SCRUM
Project Methodology, at the scale of a project, not a platform
Agile development: iterative on business value adding perimeters
BI Markup Language
For the back office (ETL and Database)
Limited, but in active development by very smart people

6. Dependence on other systems Inertia due to data volume
Why so little?
We are constrained
Dependence on other systems
Inertia due to data volume

7. What if we could do anything?
R&D
Prod – Customer 2
API
API
API
Prod – Customer 1

8. That’s the state of the art in ALM…
Softwares containers and
Application Platforms +
Micro Services
(« SOA done right »)

9. Where to put all that? In the Cloud!
Easy access from everywhere, for the data, for users
Multi-tenancy : multiple customers (ROI yo!!!)
Scalability : to handle the multi-tenancy

10. Why in PaaS?
Source : Technet
Kevin Remde

11. To what end? An example Dashboard Analytics (Answers) Search
288 million monthly active users
500 million tweets per day
Analytics (Answers)
Search

12. A first approach : Lambda Architecture
Nathan -
1 : All data entering the system is dispatched to both the batch layer and the speed layer for processing.
2 : The batch layer has two functions: (i) managing the master dataset (an immutable, append-only set of raw data), and (ii) to pre-compute the batch views.
3 : The serving layer indexes the batch views so that they can be queried in low-latency, ad-hoc way.
4 : The speed layer compensates for the high latency of updates to the serving layer and deals with recent data only.
5 : Any incoming query can be answered by merging results from batch views and real-time views.

13. Immutable append-only raw data?
Martin – Turning the dB inside out (youtube)

14. Views as layers ? Master Dataset (Logs) Use case A
3 intermediate steps
Master Dataset
(Logs)
Use case B
2 intermediate steps

15. Imagine you do that in a BI context
Supply Fact
Procurement
Logs
...
at date:time, product P was bought
from supplyer S in quantity Q2 at price P2
with attributes {...} by Employee E;
Dimension
Product
Sales Fact
PoS Logs
...
at date:time, product P was sold
to customer C in quantity Q1 at price P1;
CRM Logs
...
at date:time, employee E changed
Customer C attributes to {...};
Dimension
Customer

16. Can we go further? Lambda (implemented)
Jay – Questioning the Lambda architecture (oreilly)
Lambda (implemented)
Dual implementation (Storm and Hadoop)
No consideration for ALM
> Why can’t the stream processing system be improved
to handle the full problem set in its target domain?

17. Kappa Architecture Kappa (implemented)
Jay – Questioning the Lambda architecture (oreilly)
Kappa (implemented)
Everything done in the Streaming (Speed) Layer
Take into accounts the versioning of the application

18. Kappa Architecture, in Azure, Managed
Stream
Analytics
Azure SQL
Database
...
Customer C attributes to {...};
at date:time, employee E changed
Event Hub
...
at date:time, product P was bought
from supplyer S in quantity Q2 at price P2
with attributes {...} by Employee E;
...
at date:time, product P was sold
to customer C in quantity Q1 at price P1;
...
at date:time, employee E changed
Customer C attributes to {...};
Immutable
Append-only
Raw data
Materialized Views
Stream Processing System

19. Kappa Architecture, in Azure, Managed
Pseudo
T-SQL
Immutable Logs
API Call
Stream Analytics
Power BI
Pull mode
Azure SQL Database
Event Hub

20. Demo : Business case, by Microsoft
Dipanjan Banik – Stream Analytics Hands on Lab : TollApp (download)
Our toll station has multiple toll booths, where a sensor placed on top of the booth scans an RFID card affixed to the windshield of the vehicles as they pass the toll booth.
The passage of vehicles through these toll stations can be modelled as event streams over which interesting operations can be performed.
Entry data stream
Commercial vehicle registration database
Toll Id
EntryTime
LicensePlate
State
Make
Model
Vehicle Type
Vehicle Weight
Toll
Tag 1
:01:00.000
JNB 7001 NY
Honda
CRV
1535 7 2
:02:00.000
YXZ 1001
Toyota
Camry
1399 4 …
LicensePlate
RegistrationId
Expired
SVT 6023 1
XLZ 3463
BAC 1005 1  …

21. Demo

22. Kappa Architecture, in Azure, Managed
Power BI API
Push mode
Pseudo
T-SQL
Immutable Logs
API Call
Stream Analytics
Power BI
Pull mode
Azure SQL Database
Event Hub
Blob Storage (csv)
Document DB (json)

23. A Conforming Dimension Service?
Inline API call
(a SK for a BK)
Manages SCDs
Interface to the MDM
Pseudo
T-SQL
Exposes
Dim Tables
Immutable Logs
API Call
Stream Analytics
Power BI
Azure SQL Database
Event Hub

24. Points of friction
Managed approach : you are dependent on the product team for features…
Limited retaining capacity in the stream (7 days) will force you to reload everything when required (blob storage)
That’s just theory at the moment (in the Azure world at least) ;)

25. A theory being created right now!
Nathan -
Ed – Lambda architecture implementation at Anwsers (Twitter)
Jay – Questioning the Lambda architecture (oreilly)
Martin – Turning the dB inside out (youtube)
Chen (Gwen) – Agile Data Pipelines (slideshare)
Sheetal – Design patterns for RT streaming data analytics (hortonworks)
Michal – Spark, whistling past the data platform graveyard (localytics)
Curt – Data Models (dbms2)
Tyler Akidau - Further dispeling the myths of the lambda architecture (google)
Martin – Micro Services and SOA (blog)
Do it yourself ;)
Dipanjan Banik – Stream Analytics Hands on Lab : TollApp (download)
Visual Studio 2013 Community Edition (download)
Azure 1 month Free Trial : 150€ (activate)

26. Explore Everything PASS Has to Offer
Free SQL Server and BI Web Events
Free 1-day Training Events
Regional Event
This is Community
Business Analytics Training
Local User Groups Around the World
Session Recordings
PASS Newsletter
Free Online Technical Training