1. Building OLAP Applications with Mondrian and MySQL Julian Hyde, lead developer April 26, 2006

2. Agenda Introduction to OLAP Key features and architecture Getting started Schema Design MDX Query language Tuning Business Intelligence suite Q & A

3. What is OLAP? View data “dimensionally” i.e. Sales by region, by channel, by time period Navigate and explore Ad Hoc analysis “Drill-down” from year to quarter Pivot Select specific members for analysis Interact with high performance Technology optimized for rapid interactive response

4. A brief history of OLAP 1970 1980 1990 2000 2010 APL Express Mondrian Pentaho Essbase Business Objects Cognos PowerPlay Mainframe OLAP Comshare MicroStrategy JPivot Desktop OLAP Open-source OLAP Open-source BI suites OpenI BEE RDBMS support for DW Sybase IQ Informix MetaCube Oracle partitions, bitmap indexes JRubik Palo Microsoft OLAP Services Codd’s “12 Rules of OLAP” Spreadsheets

5. Mondrian features and architecture

6. Key Features On-Line Analytical Processing (OLAP) cubes Automated aggregation Speed-of-thought response times Open Architecture 100% Java J2EE Supports any JDBC data source No processing required Analysis Viewers Enables ad-hoc, interactive data exploration Ability to slice-and-dice, drill-down, and pivot Provides insights into problems or successes

7. How Mondrian Extends MySQL for OLAP Applications MySQL Provides Data storage SQL query execution Heavy-duty sorting, correlation, aggregation Integration point for all BI tools Mondrian Provides Dimensional view of data MDX parsing SQL generation Caching Higher-level calculations Aggregate awareness

8. Cube Schema XML Cube Schema XML Cube Schema XML Open Architecture Open Standards (Java, XML, MDX, JOLAP, XML/A, SQL) Cross Platform (Windows, Unix/Linux, …) J2EE Architecture Server Clustering Fault Tolerance Data Sources JDBC JNDI >10 SQL dialects J2EE Application Server Mondrian Web Server JDBC RDBMS cube File or RDBMS Repository RDBMS JDBC JPivot servlet Viewers JPivot servletXML/A servlet

9. Getting started

10. Getting started with Mondrian 1. Download mondrian from http://mondrian.sourceforge.net and unziphttp://mondrian.sourceforge.net 2. Copy mondrian.war to TOMCAT_HOME /webapps. 3. Create a database: $ mysqladmin create foodmart $ mysql mysql> grant all privileges on *.* to 'foodmart'@'localhost' identified by 'foodmart'; Query OK, 0 rows affected (0.00 sec) mysql> quit Bye 4. Install demo dataset (300,000 rows): $ java -cp "/mondrian/lib/mondrian.jar:/mondrian/lib/log4j- 1.2.9.jar:/mondrian/lib/eigenbase-xom.jar:/mondrian/lib/eigenbase- resgen.jar:/mondrian/lib/eigenbase- properties.jar:/usr/local/mysql/mysql-connector-java-3.1.12-bin.jar" mondrian.test.loader.MondrianFoodMartLoader -verbose -tables -data -indexes -jdbcDrivers=com.mysql.jdbc.Driver -inputFile=/mondrian/demo/FoodMartCreateData.sql -outputJdbcURL= "jdbc:mysql://localhost/foodmart?user=foodmart&password=foodmart“ 5. Start tomcat, and hit http://localhost:8080/mondrianhttp://localhost:8080/mondrian

11. demonstration

12. Designing a Mondrian schema

13. A Mondrian schema consists of… A dimensional model (logical) Cubes & virtual cubes Shared & private dimensions Calculated measures in cube and in query language Parent-child hierarchies … mapped onto a star/snowflake schema (physical) Fact table Dimension tables Joined by foreign key relationships

14. Writing a Mondrian Schema Regular cubes, dimensions, hierarchies Shared dimensions Virtual cubes Parent-child hierarchies Custom readers Access-control (Refer to http://mondrian.sourceforge.net/schema.html.)http://mondrian.sourceforge.net/schema.html

15. Mapping Each catalog object (cube, hierarchy, measure) maps onto a part of a snowflake schema links a shared dimension into a particular cube defines a member using a formula ([Measures].[Variance] / [Measures].[Budget]) * 100

16. Security Role-based access to data Container authenticates and chooses role (Mondrian does not authenticate) Examples: Alan can only see sales for Electronics products Beatrice can see aggregated sales, but cannot drill down to individual customers Charles cannot slice using Gender hierarchy <HierarchyGrant hierarchy="[Store]" access="custom" topLevel="[Store].[Store Country]"> <HierarchyGrant hierarchy="[Customers]“ access="custom" topLevel="[Customers].[State Province]“ bottomLevel="[Customers].[City]">

17. MDX query language

18. Who needs a query language? Your end-users don’t need a query language A query language helps you build more complex reports MDX features: Calculated members and sets Localized format strings Parameters Member properties Large set of built-in functions

19. MDX – Multi-Dimensional Expressions A language for multidimensional queries Plays the same role in Mondrian’s API as SQL does in JDBC SQL-like syntax … but un-SQL-like semantics A query consists of: Axes (usually ROWS and COLUMNS ) Slicer expression ( WHERE ) Members and measures SELECT {[Measures].[Unit Sales]} ON COLUMNS, {[Store].[USA], [Store].[USA].[CA]} ON ROWS FROM [Sales] WHERE [Time].[1997].[Q1]

20. MDX evaluation 1. Every query is based on a cube 2. Slicer sets the context 3. Columns and rows expressions are evaluated 4. Each cell in the resulting grid is evaluated Calculated members cause recursive evaluation WITH MEMBER [Measures].[Margin] AS ‘ ([Measures].[Store Sales] – [Measures].[Store Cost]) / [Measures].[Unit Sales] ‘ SELECT {[Measures].[Unit Sales], [Measures].[Margin]} ON COLUMNS, Filter({[Store].[USA], [Store].[USA].Children}, [Measures].[Unit Sales] > 1000) ON ROWS FROM [Sales] WHERE ([Time].[1997].[Q1], [Product].[Beer])

21. demonstration

22. Advanced MDX Calculated members Calculated sets Parameters Time-series functions Format strings For additional details, see MDX resources on MSDN: http://msdn.microsoft.com/library/en-us/olapdmad/agmdxbasics_1bzs.asp

23. Tuning

24. Tuning Mondrian Some techniques: Test performance on a realistic data set! Use tracing to identify long-running SQL statements Tune SQL queries Run the same query several times, to examine cache effectiveness When Mondrian starts, prime the cache by running queries Get to know Mondrian’s system properties

25. Tuning MySQL for Mondrian Fact table: Index foreign keys Try indexing combinations of foreign keys Use MyISAM or MERGE (also known as MRG_MyISAM ) Partitioned tables (MySQL 5.1) Dimension tables: Index primary and foreign keys Use MyISAM for large dimension tables, MEMORY for smaller dimension tables Create aggregate tables: Contain pre-computed summaries Prevent full table scan followed by massive GROUP BY Parent-child hierarchies Create closure tables

26. The whole enchilada

27. Business Intelligence Suite Mondrian OLAP server Analysis tools: Pivot table Charting Dashboards ETL (extract/transform/load) Integration with operational reporting Integration with data mining Actions on operational data Design/tuning tools

28. Pentaho Open Source BI Offerings All available in a Free Open Source license

30. Q & A

31. Next Steps and Resources More information http://www.pentaho.org and http://mondrian.sourceforge.nethttp://www.pentaho.org http://mondrian.sourceforge.net Pentaho Community Forum http://www.pentaho.orghttp://www.pentaho.org Go to Developer Zone Discussions Pentaho BI Platform including Pentaho Analysis http://sourceforge.net/project/showfiles.php?group_id=140317 http://sourceforge.net/project/showfiles.php?group_id=140317 Pentaho Analysis (pentaho_demo-1.1.4.239.zip) Pentaho Report Design Wizard (pentaho-report-design-wizard-1.1.4.167.zip) Mondrian OLAP Library only http://sourceforge.net/project/showfiles.php?group_id=35302

32. Thank you for attending!

33. Extra material

34. Architecture Storage LayerStar LayerPresentation LayerDimensional Layer

35. Extending Mondrian User-defined functions User-defined cell, member and property formatters Member readers Schema processors Localized resources

36. End User UI JPivot (sister SourceForge project) is a JSP UI Analytical grids Charting Tag library

37. Administration UI Workbench (written in Java/SWT)