Presentation is loading. Please wait.

Presentation is loading. Please wait.

Dr. Vishal Sikka Chief Software Architect SAP AG Data Management in Enterprise Apps: Some Perspectives.

Similar presentations


Presentation on theme: "Dr. Vishal Sikka Chief Software Architect SAP AG Data Management in Enterprise Apps: Some Perspectives."— Presentation transcript:

1 Dr. Vishal Sikka Chief Software Architect SAP AG Data Management in Enterprise Apps: Some Perspectives

2 Our Approach to Two of These Divides The Lessons Learned and Some Open Problems A Brief Introduction to SAP and Data Management in Our Applications The Current Situation: Some Existing and Emerging “Divides”

3  SAP AG 2006, SAP Tech Ed Shai Agassi / 3 SAP at a Glance Who we are Founded in revenues: €8.5 Billion 34,600+ customers 37,500+ employees 12+ Million users in 120 countries 1,600+ partners What we do Largest enterprise applications company in the world Serve most back-end and front- end business processes  Leader in ERP, CRM, SCM, …  Leading platform to build and run apps on 25+ industry solutions SAP NetWeaver SAP Legacy Home Grown / ISV Embedded Duet Mobile Forms Widgets RSS Portal Dashboards Project Muse Voice RFID mySAP ERP Industry Standards Biz partner SAP GUI Data Infrastructure CRMSCMSRMPLM Biz partner SAP Composites Other Composites Infrastructure

4  SAP AG 2006, SAP Tech Ed Shai Agassi / 4 SAP for Engineering & Construction Customer with 5,000 concurrent active users mySAP SCM Customer with 4.5 million characteristic combinations & 512 GB memory in live cache SAP for Consumer Products Customer with 1.4 million sales order line items per day SAP for Utilities 25 million business partners – 85 million service and sales orders per year SAP NetWeaver BI Customer with 40 TB database live Average DB size of top 10 live BI customers: 5.5TB mySAP ERP HCM Customer with payroll calculations for 500,000 employees in 3 hours mySAP ERP A customer with 5 users on a laptop SAP NetWeaver Portal Customer with 300,000 users (20,000 concurrent) Our data management requirements are massive mySAP Business Suite

5  SAP AG 2006, SAP Tech Ed Shai Agassi / 5 Data Management from SAP’s Perspective There is >10 PB of transactional and analytical data processed by SAP apps worldwide We are the largest applications consumer and reseller of data worldwide Our data is of many different types, shape and sizes: Transactional, Analytical, Text/Unstructured, Master, Events, … Data has different requirements & different optimizations Significant need for deriving value from this data SAP Applications Master Transactional Analytical Unstructured Event

6  SAP AG 2006, SAP Tech Ed Shai Agassi / 6 Transactional Data Analytical Data Master Data Event Data Textual and Unstructured Data Progression Over Time Data through the SAP Lens – “Not All Data Is Alike” Order ~ 100G Write > read Many changes Accurate Consistent Performance All back-end apps Order > Tb Read only Slow changes Many queries Flexibility Performance Order ~ 1G Mostly read Mid change Many queries Distributed Order < Tb Many writes Few queries Distributed Filtering Correlation Order > Tb Mostly read Slow change Many queries Unstructured Contextual

7  SAP AG 2006, SAP Tech Ed Shai Agassi / 7 3-tier C/S Architecture of Basis: Our Application Server

8  SAP AG 2006, SAP Tech Ed Shai Agassi / 8 Memory Management in Basis outside the DBMS Buffers in the application server help significantly improve performance. In a classical 3-tier system, network round trips mitigated benefits of the DBMS cache, while TCO optimization required one DB for >10+ app servers. Application level locking (Enqueue and Application LUW) mitigates the absence of fine granularity of locking in DBMS and transaction support needed by Application Servers (multiple users accessing the same DB, complex screen processing with workflow on front-end). Numerous other optimizations and DB abstractions.

9  SAP AG 2006, SAP Tech Ed Shai Agassi / 9 Bringing Data Closer to Applications: SAP LiveCache LiveCache is a main-memory DB component used in SAP SCM’s APO Rapid Planning Matrix in the Automotive Industry Common ERP system: Plan the mfg of 20,000 Cars / Day Needed volumes are much higher liveCache enables planning 500,000 Cars / Hour Demand Planning (DP): Interactive planning: 10x performance gain compared to DB based solution Consistent storage of data (no need for aggregation/disaggregation batch jobs) Production Planning (PP/DS): Performance gain of 15x in rescheduling production runs and DS heuristics Data volume 5x higher in planning board compared to common ERP system Consolidation of data structures via generic liveCache data types: E.g. 1 order data type 1 order type with multiple attributes instead of a few dozen different specific order types in ERP Bringing development teams closer together LiveCache applications team bridges technology knowledge with business process knowledge by working together with the application team on the usage of the liveCache, as well as in optimization of business logic. Common team working together for several years  happy deployments.

10 Our Approach to Some of These Divides The Lessons Learned and Some Open Problems A Brief Introduction to SAP and Data Management in Our Applications The Current Situation: Some Existing and Emerging “Divides”

11  SAP AG 2006, SAP Tech Ed Shai Agassi / 11 Once my system is up and running, you, SAP, can touch my core processes once every 5 years... and it needs to be a Saturday … and my CEO wants me to innovate every quarter” “ New needs: Innovate, Be flexible, Stay high-performant CIO, Fortune 1000 Manufacturing Company

12  SAP AG 2006, SAP Tech Ed Shai Agassi / 12 New requirements, New “divides” SAP NetWeaver SAP Legacy Home Grown / ISV Embedded Duet Mobi le Form s Widget s RS S Portal Dashboa rds Project Muse Voice RFID mySAP ERP Industry Standar ds Biz partner SAP GUI Data Infrastruct ure CR M SCMSRMPLM Biz partner SAP Composites Other Composites Infrastructure More decoupled business processes More visible Physical-Digital divide Infrastructure subjected to much higher volumes (events, sensors, …) Greater need for in-context usage Multiple UIs More visible work-personal divide Users are a lot more used to search, lack of structure is academic to them Different requirements on front-end than on back-end e.g. easier front-end application composition Many more deployment options Greater flexibility  easy integration, better components semantics New application architectures are necessary: SOA is the biggest component, but there are others

13  SAP AG 2006, SAP Tech Ed Shai Agassi / 13 Addressable Memory CPU Technology Shifts Disk Speed Improvement Memory 15 Kilo RPM 3x3x Disk based data storage Simple consumption of applications (Fat client UI, EDI) General- purpose, application- agnostic database Architectural ShiftTechnology Drivers x2 x 64 Bits 16 Bits 250 x 5 MB/$ 0.02 MB/$ 143 x 7.15 MIPS/$ 0.05 MIPS/$ 48 5 Kilo RPM Network Speed 100 Mbps 10 Gbps 100 x In-memory data stores Multi-channel UI, high event volume, cross industry value chains Application- aware and intelligent data management

14 Our Approach to Two of These Divides The Lessons Learned and Some Open Problems A Brief Introduction to SAP and Data Management in Our Applications The Current Situation: Some Existing and Emerging “Divides”

15  SAP AG 2006, SAP Tech Ed Shai Agassi / 15 Addressing DB Architecture Gap: SAP BI Accelerator Performance1 Billion records analyzed in 3 seconds DeliveryOff the shelf hardware, appliance setup PredictabilityConsistent response, no tuning, fast load Integration Built for & closely integrated with SAP NW BI legacy Any source, any tool

16  SAP AG 2006, SAP Tech Ed Shai Agassi / 16 Addressing DB Architecture Gap: SAP BI Accelerator Performance1 Billion records analyzed in 3 seconds AffordabilityOff the shelf hardware, appliance setup AgilityConsistent response, no tuning, fast load Integration Closely integrated with SAP BI

17  SAP AG 2006, SAP Tech Ed Shai Agassi / 17 BI Accelerator Key Technology Main memory technology Inspired by text search On the fly aggregation L2 cache miss optimization Column based data structures  Highly compressed, dictionary based, golomb, sparse,... Fast updates with write-optimized delta mechanism Compressed data structures for read access Parallel and distributed execution engine Distributed joins, horizontal table split Intelligent partitioning (along join paths) Data distribution optimizer Model based data layer Exploit data model for performance optimization and data distribution BI Application Server Database Server SAP BI AppServer Scalability by adding blades SAP BI Accelerator Storage subsystem

18  SAP AG 2006, SAP Tech Ed Shai Agassi / 18 Key Benefits Predictable (near constant) query response time Query execution shifted from DB to BI Accelerator Fast in memory full table scans guarantee stable response times Column based data structures support fast joins Intelligent partitioning and data distribution allows massive parallelization Reduced maintenance costs Simplified cube modeling (normalization for semantic reasons only) No more aggregates (or aggregate administration) Less need for DB optimization Reduced hardware costs Commodity hardware (blades) with standard equipment Linear scalability with number of processors / cores Use of blade infrastructure instead of big SMP box Packaged as an appliance

19  SAP AG 2006, SAP Tech Ed Shai Agassi / 19 BI Accelerator Future Complete OLAP layer as part of the BI Accelerator Integration of text search and BIA technology Master data support Enterprise search on BI data Reporting on text data ( information extraction) Support of flat cubes Use of commodity coprocessors Network processors Graphic card processors Application data model integration

20  SAP AG 2006, SAP Tech Ed Shai Agassi / 20 SAP Enterprise Search Search in the enterprise Business objects Business context awareness  Role  Authorizations, Compliance  Current work context Graceful degradation with decreasing structure Multiple clients Stand alone and embedded into applications Integration into non-SAP sources SAP Enterprise Search is a stand alone business search xApp and a framework for search as a service Portal Office Devices Desktop SAP NetWeaver Business Process Platform Search Indexing SAP Enterprise Search Desktop Search Service Internet Search Service 3 rd party my SAP Bus. Suite Docu- ments R/3 via BAPI’s

21  SAP AG 2006, SAP Tech Ed Shai Agassi / 21 SAP Enterprise Search Access more information from any place Get the right answer to enterprise questions anywhere, anytime Access data from your workplace or mobile device. Simple to use: Open to everyone Pre-build common queries Smart context Better Answers: Leverage context information and meta data Support targeted search for object types Enhance search and displays by contextual meta data: related queries, object scoping Go Deep: Find the right information – Across all your sources Penetrate entire corporate data sources including Search for documents and business objects simultaneously Ensure service-oriented, multi-device scalable operation Reach Out: Embed search into everyday tools Design simple search front ends that are compliant to the respective devices, including Portal, Desktop, SMS, , mobile

22  SAP AG 2006, SAP Tech Ed Shai Agassi / 22 The Argo Widget

23  SAP AG 2006, SAP Tech Ed Shai Agassi / 23 Enterprise Search Example

24  SAP AG 2006, SAP Tech Ed Shai Agassi / 24 Enterprise Search Example (Cont’d)

25 Our Approach to Some of These Divides The Lessons Learned and Some Open Problems A Brief Introduction to SAP and Data Management in Our Applications The Current Situation: Some Existing and Emerging “Divides”

26  SAP AG 2006, SAP Tech Ed Shai Agassi / 26 Master Data Management Master Data Management Architecture Characterized By Business Entities with  Multiple data models  Multiple application sources Reference Models  Single logical model  Multiple physical models Source of Truth  No single source of truth Access Characteristics  Serves as reference data  Few systems write  Many systems read 360 ° view of data  Full analytics view  Full operational view MDM Application Services QualityVisibilityGovernanceValidationAnalytics Connectivity Fabric Multiple Data Source Management Unified Data Management Layer Data Mappings Legacy Data Unstructured Data Structured Data Distributed QueryData Federation EventsServices Meta-data Master

27  SAP AG 2006, SAP Tech Ed Shai Agassi / 27 Event Processing Characterized By Continuous Streams of near real-time data High data flow rate and large volume needs parallel processing Significant main memory processing Continuous evaluation of rules Edge Devices as data producers — (RFID, sensor data) generate significant number of events — orders of magnitude scale data e.g., shop floor sensor devices Large volume of event data dictates pre-processing for consumption Events externalized non-invasively for several forms of consumption Automatic correlation and context determination of business events Input Streams Event Streams Data (IN) Business Events Actions Query Results BI/Reports Alerts Output Streams Correlation Rules Event Management Filters Response Correlation Engine Event Memory/Storage

28  SAP AG 2006, SAP Tech Ed Shai Agassi / 28 Lessons Learned It’s not the technology, stupid. Application perspectives provide grounding for data management.  So learn what the apps needs are One size does not fit all. Applications’ data mgmt needs are changing and this requires a rethink in data mgmt architecture.  So let’s go rethink data mgmt for the enterprise


Download ppt "Dr. Vishal Sikka Chief Software Architect SAP AG Data Management in Enterprise Apps: Some Perspectives."

Similar presentations


Ads by Google