Presentation is loading. Please wait.

Presentation is loading. Please wait.

BW Basic Architecture Klaus Majenz SAP – Product Line BI.

Similar presentations


Presentation on theme: "BW Basic Architecture Klaus Majenz SAP – Product Line BI."— Presentation transcript:

1 BW Basic Architecture Klaus Majenz SAP – Product Line BI

2  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 2 Overview  complete DW & BI product, comprising... ETL tools (extractors, transformation, monitoring, scheduling,...) OLAP engine data mining engine repository analytical front-end (web- or Excel-based, agents, GIS,...) prepacked models, built by SAP application departments  client-server architecture SAP web application servers database server: 7 commercial RDBMS platforms supported (Oracle, MS, 4  IBM, SAP)  part of SAP Netweaver™ SAP's open integration and application platform more details:

3  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 3 Overview

4  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 4 CharacteristicsKey Figures Infoobjects Scenario (1)

5  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 5 Scenario (2)  Dimension Time Day Month Year  Dimension Region City Region Country  Dimension Sales Org Sales Person Division Distribution Channel Sales Organization Dimension Product Product Product Group Key Figures Quantity (in pieces) Profit (in US$)

6  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 6 An adequate BW Infocube IUSALES Dimension IUSALEST 0CALDAY 0CALMONTH 0CALYEAR Dimension IUSALES1 IUCITY IUREGION IUCOUNTRY Dimension IUSALES2 IUSALPER IUDIV IUDCHAN IUSALORG Dimension IUSALES3 IUPROD IUPRODGRP Key Figures IUQUAN IUPROFIT

7  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 7 Data Flow in BW Source System (e.g. R/3, other DB, File,...) Persistent Staging Area (PSA) Operational Data Store (ODS) Infocube: F fact table Infocube: E fact table Extraction Compression Infocube Upload (from PSA) Infocube Upload (from ODS) ODS Upload Aggregate Initial Fill, Roll-Up ODS Activate BW Query Cube Query ODS Query V.P. Query

8  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 8 Data Flow in BW – what we will look at Source System (e.g. R/3, other DB, File,...) Persistent Staging Area (PSA) Operational Data Store (ODS) Infocube: F fact table Infocube: E fact table Extraction Compression Infocube Upload (from PSA) ODS Upload Aggregate Initial Fill, Roll-Up ODS Activate BW Query Cube Query Infocube ODS PSA

9  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 9 PSA

10  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 10 PSA table huge number of individual INSERTs no UPDATE SELECT * FROM … WHERE "REQUEST" = … mass deleteion: DELETE … WHERE "PARTNO" = … / DROP PARTITION … request package (within request) partition no. record no. (within package)

11  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 11 ODS

12  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 12 ODS object = 3 tables active data: /BIC/AOIUSALES00 modified data ("activation queue"): /BIC/AOIUSALES40 delta data ("change log"): /BIC/B (PSA) 1.ODS upload: INSERT INTO "/BIC/AOIUSALES40" 2.ODS data activation: UPSERT "/BIC/AOIUSALES00" delta records: INSERT INTO "/BIC/B " (mass) DELETE FROM "/BIC/AOIUSALES40" 3.infocube delta upload from ODS: SELECT * FROM "/BIC/B "

13  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 13 ODS tables: /BIC/AOIUSALES00, /BIC/AOIUSALES40 active data modified data same as in PSA table

14  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 14 ODS Object (BW 3.0) Active data Staging Engine Req2 Req3 Req1 Activation queue Req.ID I Pack.ID I Rec.No Change log Doc.No I Value ODSRx I P 1 I Rec.1I4711I I 10 Activation  During activation the data is sorted by the key fields of active data plus key fields of Activation queue.  This guaranties the correct sequence of the records and allows inserts instead of table locks. REQU1 I P 1 I Rec.1I4711I 10 REQU2 I P 1 I Rec.1I4711I 30 Upload to Activation queue  Data from different requests are uploaded in parallel to the activation queue ODSRy I P 1 I Rec.1I4711I-10 ODSRy I P 1 I Rec.2I4711I I 30  Before- and After Image  Request ID in activation queue and change log differ from each other.  After update, data in the activation queue is deleted.

15  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 15 Infocube

16  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 16 InfoCube: Star Schema F, E D S Y X (1) Fact Table (2) Dimension (3) time-independent-SID time-dependent-SID master SID Char (4) SID Attr S

17  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 17 Infocube IUSALES X (City) S (Population) Facttable Dimension 1

18  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 18 Infocube Indexing (1) – Oracle X (City) S (Population) Facttable Dimension 1 Bitmap Index B-Tree (unique) B-Tree (non-unique) line item dimension

19  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 19 Infocube Indexing (2) – MS SQL Server X (City) S (Population) Facttable Dimension 1 B-tree Index (nonunique, nonclustered) B-Tree (unique, clustered) line item dimension

20  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 20 Infocube Indexing (3) – Oracle Bitmap Index B-Tree (unique) B-Tree (non-unique) F Facttable single column indexes support queries P-index: compress additional bitmap index on part. column E Facttable "P-index" partitioning column (for E facttable)

21  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 21 Infocube Indexing (4) – MS SQL Server F Facttable single column indexes support queries P-index: compress E Facttable "P-index" B-tree Index (nonunique, nonclustered) B-Tree (unique, nonclustered) Does not exist on MS-SQL

22  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 22 Infocube Operations (1)  INSERT: only F facttable array INSERT if array INSERT fails: UPSERT logic  DELETE request (mass deletion): only F facttable DELETE FROM "/BIC/FIUSALES" WHERE KEY_IUSALESP = … alternatively: DROP PARTITION  DELETE specified data DELETE FROM … WHERE …  UPSERT: only E facttable infocube compression (separate slide)  SELECT separate slide

23  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 23 Infocube Compression (ex.: request 3) before after UPDATE INSERT

24  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 24 Infocube Compression (2)  Oracle (via stored procedure; on DB server) loop over rows for request REQ in F facttable  attempt UPDATE of E facttable  if UPDATE fails then INSERT rowid into temporary table INS do mass INSERT INTO E facttable using INS DROP PARTITION corresponding to REQ in F facttable  MS SQL-Server (via ABAP; via application server) loop over rows for request REQ in F facttable  attempt UPDATE of E facttable  if UPDATE fails then attempt INSERT DELETE FROM F facttable WHERE requestid = REQ

25  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 25 Aggregate Fill  INSERT INTO [/BIC/E100010]  SELECT [D1].[SID_IUCITY] AS [KEY_ ],  [D2].[SID_IUSALPER] AS [KEY_ ],  0 AS [KEY_100010P],  SUM ([F].[/BIC/IUPROFIT]),  SUM ([F].[/BIC/IUQUAN]),  COUNT(*) AS [FACTCOUNT]  FROM [/BIC/FIUSALES] [F],  [/BIC/DIUSALES1] [D1],  [/BIC/DIUSALES2] [D2],  [/BIC/DIUSALESP] [DP]  WHERE [F].[KEY_IUSALES1] = [D1].[DIMID] AND  [F].[KEY_IUSALES2] = [D2].[DIMID] AND  [F].[KEY_IUSALESP] = [DP].[DIMID] AND  [DP].[SID_0CHNGID] = 0 AND  ( [F].[KEY_IUSALESP] = 0 OR [F].[KEY_IUSALESP] = 2 ) AND  [DP].[SID_0REQUID] BETWEEN 0 AND 40  GROUP BY [D1].[SID_IUCITY],  [D2].[SID_IUSALPER]

26  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 26 Aggregate Roll-Up  INSERT INTO [/BIC/F100011]  SELECT [D1].[SID_IUCITY] AS [KEY_ ],  [D3].[SID_IUPROD] AS [KEY_ ],  7 AS [KEY_100011P],  SUM ([F].[/BIC/IUPROFIT]),  SUM ([F].[/BIC/IUQUAN]),  COUNT(*) AS [FACTCOUNT]  FROM [/BIC/FIUSALES] [F],  [/BIC/DIUSALES1] [D1],  [/BIC/DIUSALES3] [D3],  [/BIC/DIUSALESP] [DP]  WHERE [F].[KEY_IUSALES1] = [D1].[DIMID] AND  [F].[KEY_IUSALES3] = [D3].[DIMID] AND  [F].[KEY_IUSALESP] = [DP].[DIMID] AND  [DP].[SID_0CHNGID] = 0 AND  [F].[KEY_IUSALESP] = 5 AND  [DP].[SID_0REQUID] = 498  GROUP BY [D1].[SID_IUCITY],  [D3].[SID_IUPROD]

27  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 27 Infocube Query Example: Infocube IUSALES (1) Fact Table (2) Dimensions (3) Characteristics (simplified) month year day cityregioncountry product group sales person division distribution channel sales organization

28  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 28 Query Example & Processing (under Oracle) month year = [98-99] regioncountry = 'US' product group (1) Fact Table (2) Dimensions (3) Characteristics (simplified)

29  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 29 Step 1: Restrictions Master Data  Dimensions month year = [98-99] regioncountry = 'US' product group (1) Fact Table (2) Dimensions (3) Characteristics (simplified) Typical Query Processing

30  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 30 Step 2: Restrictions Dimensions  Fact Table product group bitmap index (1) Fact Table (2) Dimensions (3) Characteristics (simplified) Typical Query Processing

31  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 31 Step 3: Assemble Result product group (1) Fact Table (2) Dimensions (3) Characteristics (simplified) small subset of facttable Typical Query Processing month year = [98-99] regioncountry = 'US'

32  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 32 Query Example (1) – simple  SELECT "DT"."SID_0CALMONTH" AS "S____081" ,"DT"."SID_0CALYEAR" AS "S____083" ,"D1"."SID_IUCOUNTRY" AS "S____520" ,"D3"."SID_IUPRODGRP" AS "S____524" , COUNT( * ) AS "1ROWCOUNT" , SUM ( "F"."/BIC/IUPROFIT" ) AS "IUPROFIT" , SUM ( "F"."/BIC/IUQUAN" ) AS "IUQUAN"  FROM "/BIC/FIUSALES" "F" , "/BIC/DIUSALEST" "DT" , "/BIC/DIUSALES1" "D1" , "/BIC/DIUSALES3" "D3" , "/BIC/DIUSALESP" "DP"  WHERE "F"."KEY_IUSALEST" = "DT"."DIMID" AND  "F"."KEY_IUSALES1" = "D1"."DIMID" AND  "F"."KEY_IUSALES3" = "D3"."DIMID" AND  "F"."KEY_IUSALESP" = "DP"."DIMID" AND  ( "DT"."SID_0CALMONTH" = AND  "DT"."SID_0CALYEAR" = 2000 AND  "DP"."SID_0REQUID" <= 745 )  GROUP BY "DT"."SID_0CALMONTH", "DT"."SID_0CALYEAR",  "D1"."SID_IUCOUNTRY", "D3"."SID_IUPRODGRP"

33  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 33 Query Example (2) – navigational attribute SELECT "DT"."SID_0CALMONTH" AS "S____081","DT"."SID_0CALYEAR" AS "S____083","D1"."SID_IUCOUNTRY" AS "S____520","X1"."S__IUCOLOR" AS "S____530", COUNT( * ) AS "1ROWCOUNT", SUM ( "F"."/BIC/IUPROFIT" ) AS "IUPROFIT", SUM ( "F"."/BIC/IUQUAN" ) AS "IUQUAN" FROM "/BIC/FIUSALES" "F", "/BIC/DIUSALEST" "DT", "/BIC/DIUSALES1" "D1", "/BIC/DIUSALES3" "D3", "/BIC/XIUPROD" "X1", "/BIC/DIUSALESP" "DP" WHERE "F"."KEY_IUSALEST" = "DT"."DIMID" AND "F"."KEY_IUSALES1" = "D1"."DIMID" AND "F"."KEY_IUSALES3" = "D3"."DIMID" AND "D3"."SID_IUPROD" = "X1"."SID" AND "F"."KEY_IUSALESP" = "DP"."DIMID" AND ( "DT"."SID_0CALMONTH" = AND "DT"."SID_0CALYEAR" = 2000 AND "DP"."SID_0REQUID" <= 745 AND "X1"."OBJVERS" = 'A' ) GROUP BY "DT"."SID_0CALMONTH", "DT"."SID_0CALYEAR", "D1"."SID_IUCOUNTRY", "X1"."S__IUCOLOR"

34  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 34 Query Example (3) – external hierarchy SELECT "DT"."SID_0CALYEAR" AS "S____083","DT"."SID_0CALMONTH" AS "S____081","D1"."SID_IUCOUNTRY" AS "S____520","H1"."PRED" AS "S____524", COUNT( * ) AS "1ROWCOUNT", SUM ( "F"."/BIC/IUPROFIT" ) AS "IUPROFIT", SUM ( "F"."/BIC/IUQUAN" ) AS "IUQUAN" FROM "/BIC/FIUSALES" "F", "/BIC/DIUSALES3" "D3", "/BIC/DIUSALEST" "DT", "/BIC/DIUSALES1" "D1", "/BIC/DIUSALESP" "DP", "/BI0/ " "H1" /* This is a (UNION) view! */ WHERE "F"."KEY_IUSALES3" = "D3"."DIMID" AND "F"."KEY_IUSALEST" = "DT"."DIMID" AND "F"."KEY_IUSALES1" = "D1"."DIMID" AND "F"."KEY_IUSALESP" = "DP"."DIMID" AND "D3"."SID_IUPRODGRP" = "H1"."SUCC" AND ( "DT"."SID_0CALYEAR" = 2000 AND "DP"."SID_0REQUID" <= 745 AND "H1"."SUCC" <> ) GROUP BY "H1"."PRED", "DT"."SID_0CALYEAR", "DT"."SID_0CALMONTH", "D1"."SID_IUCOUNTRY"

35  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 35 Examples of Conceptual Modeling in SAP BW

36  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 36 Examples  Reveal why pure RDBMS technology...  sometimes requires an additional conceptual layer on top,  is not sufficient is some cases,  has no chance in some situations because it has to be more general than necessary.  Examples  example 1: infoproviders in SAP BW  uniform view on differing physical layouts  example 2: non-cumulative key figures in SAP BW  semantic relationship between table columns  example 3: aggregates in SAP BW  could be implemented by using materialized views (or equivalent)  but they have proved to be inferior

37  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 37 Example 1: Infoprovider (1)  An infoprovider in SAP BW...  comprises a reporting scenario,  is the entity on which a query is defined,  combines (aggregated or non-aggregated) operational data with master data (e.g. product, customer,... data),  or constitutes a master data entity

38  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 38 Example 1: Infoprovider (2) – Examples  Example A:  a cube is an infoprovider  fact table holds operational data on certain granularity  dimensions hold master data  Example B:  customer master data can be an infoprovider  same UI as for other infoproviders  selections, projections, summaries using attributes (e.g. address, customer category, region,...)

39  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 39 Example 1: Infoprovider (3) -- Overview (SAP BW 3.x) Infoprovider Multi- provider UNION Infoset JOIN Infocube multi-dim. analytical sales cube ODS-Object flat operational POS data Characteristic (master data) flat master data product data Virtual Infoprovider API e.g. remote access real time

40  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 40 Example 2: Non-Cumulative Key Figures (1)  also: "semi-additive measures"  example: account balance  conceptually:   physically: accountdaybalancedelta A29-Sep100 €10 € B29-Sep500 € A30-Sep110 € B30-Sep500 €- 100 € A1-Oct110 € B1-Oct400 € A2-Oct110 € B2-Oct400 € A3-Oct110 €- 60 € B3-Oct400 € A4-Oct50 € B4-Oct400 € accountdayref pointdelta A29-Sepno 10 € B30-Sepno € A3-Octno - 60 € A4-Octyes50 € B4-Octyes400 €

41  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 41 Example 2: Non-Cumulative Key Figures (2)  non-cumulative key figures / semi-additive measures balance can be reconstructed for any moment in the past  that information has not to be physically stored advantages  significantly reduced data volumes  better performance  more flexibility however: algorithms are required for  reconstruction  read  insertion  load

42  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 42 Example 3: Aggregates in SAP BW  SAP BW constraints:  only SUM, MIN, MAX aggregations are materialized  uploaded data (in an infocube) can still be identified   delta roll-ups are simple  Materialized or Indexed Views / Automatic Summary Tables  could be used in theory  however: maintenance is considerably slower ... due to expensive tracking and logging mechanisms that are necessary if the general case has to be covered

43  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 43 Summary

44  SAP-AG 2005,BW Basicarchitecture, Klaus Majenz 44 Summary  brief introduction to SAP BW  three examples: 1.an additional conceptual layer on top of the relational one 2.a semantical pattern that is frequently used in business 3.an object that might suffer from the generic approach  Do the examples reveal shortcomings of RDBMS or are they application domain specific ?


Download ppt "BW Basic Architecture Klaus Majenz SAP – Product Line BI."

Similar presentations


Ads by Google