Presentation is loading. Please wait.

Presentation is loading. Please wait.

Real-Time Analytics at Salesforce.com Donovan Schneider Principal Architect SDForum May, 2010.

Similar presentations


Presentation on theme: "Real-Time Analytics at Salesforce.com Donovan Schneider Principal Architect SDForum May, 2010."— Presentation transcript:

1 Real-Time Analytics at Salesforce.com Donovan Schneider Principal Architect SDForum May, 2010

2 Agenda Motivation Our approach Making it work Conclusions and future directions

3 Evolution of Business Intelligence: Canned reports Ad-hoc query DW Real-Time Cloud Analytics More than 50 percent of data warehouse projects will have limited acceptance, or be an outright failure

4 Real-time. Always. Accessible By Mere Mortals Flexible. In Sync. Reportable. Our Vision for CRM Analytics Deliver Insight That is Accessible, Real-time, and Trustworthy

5 What Drives Actionable Insight? Responsive Relevant Easy to Use Actionable Reliable Reporting and Analytics that are… Increased User Adoption Business user friendly Powerful capabilities to answer real-world business questions Fast performance, timely insight when needed Integrated into the CRM to enable actions from insight Accurate & consistent results Real- Time Visibility User Adoption Actionable Insight

6 And Our Customers Use It. A Lot! 12M+ reports 2.5M+ run per day 750K dashboards 700K views per day

7 Agenda Motivation Our approach Making it work Conclusions and future directions

8 We take a fundamentally different approach than most Painfully slow unless against DW Data is never fresh against DW Changes to CRM propagate slowly ETL process is complicated and expensive Usable by all, from rep to SVP Real-time, all the time Flexible & customizable Powerful w/o complicated DW One sharing model Easy, Real-Time and FlexibleComplicated, Out of Date, & Rigid Single tenant DW CRM Other Systems ETL Processes Real-Time Reporting DW Reporting 1x / day

9 Is a Data Warehouse Needed for CRM Analytics? Performance Pre-aggregation was the only way to get decent reporting/analytics performance out of OLTP/CRM Why people think they need a DW Requirement to combine multiple data sources in 1 report/dashboard -CRM systems were hard to integrate with external data sources -And then, they were not built at all for BI Business View of the Data -Corporate wide ontology -Single view of the customer -Historical data capture Why we dont need one Force.com API 200M+ API calls/day 10M records/hour Consumes External Web Services Entire System is Business Driven -Business people configure the system with their business terms, there is no IT translation reqd -Sales, Service, Analytics Clouds are all on same platform -History Tables and Analytical snapshots Cloud Computing Scale & Multi- tenant Optimization Engine And DW based architecture makes the system out of date, rigid and expensive

10 Analytics Dashboards Reports List Views Search

11 Agenda Motivation Our approach Making it work Conclusions and future directions

12 Building a Multi-tenant Cloud Platform is Hard! Lots of Pieces to Assemble! Relational / Text / Non-relational Application Services / Lifecycles Caching and Performance Scalability and Reliability Infrastructure and Backups Release Processes Development Lifecycle

13 Brief Review of Force.com Multi-tenancy Real-time App Composition Massive Shared Database Shared General Purpose Kernel

14 True Multi-tenancy: Why Share Everything? ~15 Databases~100 Servers 2 Mirrors 100,000s of Unique Applications 1 Code Base

15 Force.com Data Architecture Shared Metadata Cache Bulk Processing Engine Multi-Tenant-Aware Query Optimizer Runtime Application Generator Full-Text Search Engine Real-time App Composition

16 Sharing Relational Data Structures is Hard Your Definitions Your Data Your Optimizations Indexes Pivot table for non- unique indexes UniqueFields Pivot table for unique indexes Relationships Pivot table for foreign keys MRUIndex Pivot table for most-recently-used FallBackIndex Pivot table for Name field index …others… Harrahs Data Dells Products Your Reps Data

17 Flex Schema on Steroids: Everyones Data Flex Column: Multiple Data Types IDTenantData 2 1000001Harrahs$190 1000002Harrahs$250 1000003Harrahs$680 1000004HarrahsPoker 1000005HarrahsBlack Jack 1000006HarrahsCraps 1000007DellDisplay 1000008DellLaptop 1000009DellServer

18 IDData 1Data 2 10002unus erattoto naturae 10003vultusin orbe 10004quem dixerechaeos 10005rudisindigestaque 10006meisperpetuum 10007deducitetemopra 10008carmenante 10009mareet terras 10010tegitet quod 10011omniacaelum 10012unus erattoto naturae 10013vultusin orbe 10014quem dixerechaeos 10015rudisindigestaque 10016meisperpetuum 10017deducitetemopra 10018carmenante 10019mareet terras 10020tegitet quod 10021omniacaelum 10022unus erattoto naturae 10023vultusin orbe 10024quem dixerechaeos 10025rudisindigestaque 10026meisperpetuum 10027deducitetemopra 10028carmenante 10029mareet terras 10030tegitet quod 10031omniacaelum 10032unus erattoto naturae 10033vultusin orbe Flex Schema: Everyones Optimizations Multi-tenant Index Muti-Tenant Table IDTenantData 2 1000001Harrahs$190 1000002Harrahs$250 1000003Harrahs$680 1000004HarrahsPoker 1000005HarrahsBlack Jack 1000006HarrahsCraps 1000007DellDisplay 1000008DellLaptop 1000009DellServer TenantTextNumber Harrahs$190 Harrahs$250 Harrahs$680 HarrahsPoker HarrahsBlack Jack HarrahsCraps DellDisplay DellLaptop DellServer Sync Copy

19 Reporting Index Optimization Reporting Index Muti-Tenant Table IDTenantData 2 1000001Harrahs$190 1000002Harrahs$250 1000003Harrahs$680 1000004HarrahsPoker 1000005HarrahsBlack Jack 1000006HarrahsCraps 1000007DellDisplay 1000008DellLaptop 1000009DellServer TenantData 2Data 7…Data k DellDisplay DellLaptop DellServer Sync Copy

20 But How Do You Make the Queries Fast? Real-time App Composition Shared Metadata Cache Bulk Processing Engine Multi-Tenant-Aware Query Optimizer Runtime Application Generator Full-Text Search Engine

21 A Real World Question Michael Dell wants to know if Servers are selling well in the West. How will Force.com answer this question quickly?

22 IDData 1Data 2 10002unus erattoto naturae 10003vultusin orbe 10004quem dixerechaeos 10005rudisindigestaque 10006meisperpetuum 10007deducitetemopra 10008carmenante 10009mareet terras 10010tegitet quod 10011omniacaelum 10012unus erattotonaturae 10013vultusin orbe 10014quem dixerechaeos 10015rudisindigestaque 10016meisperpetuum 10017deducitetemopra 10018carmenante 10019mareet terras 10020tegitet quod 10021omniacaelum 10022unus erattoto naturae 10023vultusin orbe 10024quem dixerechaeos 10025rudisindigestaque 10026meisperpetuum 10027deducitetemopra 10028carmenante 10029mareet terras 10030tegitet quod 10031omniacaelum 10032unus erattoto naturae 10033vultusin orbe Visibility Indexes Millions of Sales Line Items The fastest path to the answer IDData 1Data 2 10002unus erattoto naturae 10003vultusin orbe 10004quem dixerechaeos 10005rudisindigestaque 10006meisperpetuum 10007deducitetemopra 10008carmenante 10009mareet terras 10010tegitet quod 10011omniacaelum 10012unus erattotonaturae 10013vultusin orbe 10014quem dixerechaeos 10015rudisindigestaque 10016meisperpetuum 10017deducitetemopra 10018carmenante 10019mareet terras 10020tegitet quod 10021omniacaelum 10022unus erattoto naturae 10023vultusin orbe 10024quem dixerechaeos 10025rudisindigestaque 10026meisperpetuum 10027deducitetemopra 10028carmenante 10029mareet terras 10030tegitet quod 10031omniacaelum 10032unus erattoto naturae 10033vultusin orbe M. Dell Servers West Multi-tenant Query Optimizer

23 Run pre-queries Check user Visibility Check filter selectivity Build query based on results of pre-queries Execute query User Visibility # of rows that the user can access = Filter Selectivity How specific is this filter? = Multi-tenant Query Optimizer Shared Visibility Shared Indexes IDData 1Data 2 10002unus erattoto naturae 10003vultusin orbe 10004quem dixerechaeos 10005rudisindigestaque 10006meisperpetuum 10007deducitetemopra 10008carmenante 10009mareet terras 10010tegitet quod 10011omniacaelum 10012unus erattotonaturae 10013vultusin orbe 10014quem dixerechaeos 10015rudisindigestaque 10016meisperpetuum 10017deducitetemopra 10018carmenante 10019mareet terras 10020tegitet quod 10021omniacaelum 10022unus erattoto naturae 10023vultusin orbe 10024quem dixerechaeos 10025rudisindigestaque 10026meisperpetuum 10027deducitetemopra 10028carmenante 10029mareet terras 10030tegitet quod 10031omniacaelum 10032unus erattoto naturae 10033vultusin orbe IDData 1Data 2 10002unus erattoto naturae 10003vultusin orbe 10004quem dixerechaeos 10005rudisindigestaque 10006meisperpetuum 10007deducitetemopra 10008carmenante 10009mareet terras 10010tegitet quod 10011omniacaelum 10012unus erattotonaturae 10013vultusin orbe 10014quem dixerechaeos 10015rudisindigestaque 10016meisperpetuum 10017deducitetemopra 10018carmenante 10019mareet terras 10020tegitet quod 10021omniacaelum 10022unus erattoto naturae 10023vultusin orbe 10024quem dixerechaeos 10025rudisindigestaque 10026meisperpetuum 10027deducitetemopra 10028carmenante 10029mareet terras 10030tegitet quod 10031omniacaelum 10032unus erattoto naturae 10033vultusin orbe Stop Go Multi-tenant Optimizer Statistics

24 Acting on pre-queries Pre-Query Selectivity measurements Construct final database query, forcing… UserFilter Low …nested loops join; drive using view of rows that the user can see LowHigh…use of index related to filter HighLow…ordered hash join; driving using data table High … use of index related to filter

25 Report Execution Joins Filters Hints Aggregations Joins Filters Hints Aggregations Sorts Aggregations Filters Sorts Aggregations Filters Application Server rows cache Pre-queries SQL

26 Agenda Motivation Our approach Making it work Conclusions and future directions

27 Conclusions Real-time BI is possible A data warehouse is not required Interesting technical challenges –Cannot rely on databases cost-based optimizer –Sophisticated sharing models great for customers but technically challenging –Real-time data limits caching applicability –Protect tenants from each other We need help

28 Technical Direction Increased focus on usability Expanded analytical capabilities Collaboration Scalability


Download ppt "Real-Time Analytics at Salesforce.com Donovan Schneider Principal Architect SDForum May, 2010."

Similar presentations


Ads by Google