1. Oracle Database In-Memory Performance Benefits with MicroStrategy Analytics Platform

2. A Little About Us
Who are we?
Founded in 1989 by data mining experts from MIT, today MicroStrategy is the top independent analytic software vendor in the world with a comprehensive suite designed for team, departmental, and enterprise analytical use cases or even larger. We are a publicly traded company with $600 million in revenue in 2012, no debt, and over $200 million in cash in the bank, so we are a sound and stable partner for our customers. We have global reach with presence in over 23 countries, and from a technology standpoint, MicroStrategy has been a long time innovator with a unique, unitary architecture known throughout the analytics industry for high performance and scalability. We are also known for our leading edge mobile technology and unique Cloud-based deployment options. More recently, we’ve added new innovations in mobile commerce and identity management.
MicroStrategy has achieved very positive analyst recognition and we are consistently regarded as a leader in major analyst reports.
Today, our technology powers thousands of mission critical applications and supports millions of business users. We have nearly 4000 customers worldwide, including some of the largest and most cutting edge companies out there.
A global footprint. A proven innovator. A leader in enterprise analytics for 25 years.
A top analytics software vendor by revenue.
Operates in 44 cities in 26 countries worldwide.
Over 4,000 customers across 20 industries.
$576M in revenue in 2013.
$100M+ dedicated annually to R&D.
Innovator in web, mobile, and cloud-based analytics.

3. MicroStrategy and Oracle
A longstanding partnership
MicroStrategy is an Oracle Gold certified partner
Significant number of MicroStrategy customers use Oracle as their main database platform
Partner on multiple levels
Participate in all Oracle beta programs
MicroStrategy was invited to participate in the Oracle Database In-Memory Launch
Why does MicroStrategy speak here at OOW? Why would you want to listen to us?

4. Personal / Departmental
From Big Data to Business Value
MicroStrategy provides the easiest way to deliver advanced analytics on Big Data to business users
Stunning
Ease of Use
Comprehensive Analytics
Benchmarking
Projections
Trend Analysis
Data Summarization
Relationship
Analysis
Optimization
What is MicroStrategy known for?
So fundamentally, what we’re doing from an analytical perspective is we’re delivering data from across the enterprise, from discreet data sources like Excel files ranging all the way up to the largest, most massive data warehouse and Hadoop environments. We’re applying a comprehensive set of powerful analytics to that data and then we’re pushing it through the world’s most productive interfaces in a very visual and interactive sort of a way. Together, these layers of technology combine in a very powerful way that our traditional platform competitors and the agile analytics vendors just can’t match.
Any and All
Data
Structured
Semi-Structured
Unstructured
Databases
Big Data
Personal / Departmental
Cloud Data

5. Are You Ready for the Age of Analytics?
Data Scale
Requirements
Driven by new data sources, data volumes are growing exponentially.
Leading companies are using advanced analytics to compete and win. Laggards are struggling.
Mobile technology is making analytics ubiquitous and effortless. But it also demands unprecedented sub-second performance and massive user scale.
IT can’t keep up as outdated
analytics approaches fail.
Business users are taking
matters into their own hands
and disintermediating IT.
Real-time
Analytics
Performance
What are the challenges to our customers?
Agile analytics is very popular because, again as I mentioned before, it changes the way that business intelligence traditionally works. It’s so productive so that every user who has tied into data discovery functionality like you see here has created five to ten times more content. At a recent MicroStrategy conference, our customer Adobe cited a 20x improvement. You also can have more people using it than ever before because it is so easy and intuitive. Finally, MicroStategy extends data discovery by making it easier than ever to share and collaborate. So you have more content, more users, and more sharing and as a result you have net-net greater than 100 times more content creation and consumption -- which means that your data, the key facts and metrics that drive your business, are being socialized and communicated in a much broader, more effective, and more impactful way than ever before.

6. Oracle Database 12c In-Memory Option Goals
100x Faster Queries: Real-Time Analytics
Instantaneous Queries on OLTP Database or Data Warehouse
Faster Mixed Workload OLTP
Transparent: no application changes
Simple to Implement
Why is Oracle In-Memory interesting to MicroStrategy customers (and to anybody with Analytical workloads)?
With the introduction of the Oracle Database In-Memory Option it is now possible
to run real-time, ad-hoc, analytic queries on your business data as it exists
right at this moment and receive the results in sub-seconds.
True real real-time analytics.
Imagine being able to know the total sales you have made in the state of California as of right now. Not last week, or even last night but right now and have that query
return in sub-second time.

7. MicroStrategy Analytics + Oracle Database In-Memory
A Perfect Match?
Operational and analytic workloads against the same tables
Eliminates data movement into EDW
Enables Real-time BI
Columnar data store in memory enables improved analytical query performance
Improved query performance
No application changes required
Optimizer decides when to use columnar or row-based data structures
Easy to adopt
Less data movement
Less performance optimizations required
Reduced cost
… this is why.
But don’t believe me on account of these slides.
… we’re going to find out!

8. Hosted in Oracle Solaris Labs under supervision of development teams
Joint Benchmarking
Hosted in Oracle Solaris Labs under supervision of development teams
MicroStrategy SuperApp
Simulates typical workload
2TB sized dataset
14B row fact table
No aggregation
Snowflake schema
4 dimensions,
37 attributes
79 reports and dashboards
120 different queries
1 to 149 SQL passes per query (17.5 avg)
Oracle Database In-Memory
SPARC M6 Server
8 x 3.6GHz 12 core SPARC M6 Processors
96 Cores, 8 HW threads per core for a total of 768 threads
4 TB Memory
Solaris 11.1
Sun ZFS Storage Appliance
68 x 600GB 15K SAS-2 Drives
4 x 68GB Write Optimized SSD Drives (Write Cache)
4 x 477GB Read Optimized SSD Drives (Read Cache)
2 x 8GBit FC Connection to M6
To find out what’s what Oracle and MSTR teamed up to conduct a joint benchmarking effort.
The goals were obviously to verify the design goals for Oracle In-Memory and to identify any new best practices for joint customers
And in the one corner the first contestant brings to the table:
SuperApp simulates mixed reporting patterns (reports and dashboards) reflecting typical BI usage
SuperApp database schema reflects common modeling techniques and is available in 600GB/1.2TB/2.5TB flat file data volumes
Tests were run in single user and with varying concurrency levels
Tests did not utilize MicroStrategy report caching or In-Memory Cubes
80 parameterized reports show typical hockey-stick skew perf. curve
In the other corner weighing in considerably is the Oracle 12c In-memory server:
- Sparc M6, …
seconds

9. Interesting Results
Implementing Oracle Database In-Memory requires following best practices
Tests reveal great performance improvements comparing Oracle Database In-Memory vs. traditional buffer cache
Test scenario
In-Memory
Buffer Cache
Improvement
Single user, final
0:37:07
1:03:53
73%
BUT: this pales to the performance improvements compared to the first test run which naïvely had not implemented common best practices for analytical workloads
Test scenario
In-Memory
Buffer Cache
Improvement
Single user, first run
18:16:52
11:32:56
- 35%

10. Oracle In-Memory Best Practices
How to achieve optimal performance for Analytical workloads
Use existing hardware efficiently
Help the Oracle optimizer to make the right decisions
Minimize necessary IO
Give Oracle (In-Memory) sufficient resources
MicroStrategy best practices for Oracle In-Memory

11. Parallel Execution Uses Available HW Resources
Analytical queries commonly access large ranges of data
Default configuration assumes relatively small data volumes and does not parallelize queries effectively
Analytic queries commonly process large data volumes and benefit significantly from parallel processing
Tips:
Enable AutoDoP (*.parallel_degree_policy='AUTO')
Disable Old-style multi-user management (*.parallel_adaptive_multi_user=FALSE)
Avoid starting parallel execution servers dynamically (*.parallel_min_servers=n)

12. Verify Parallel Execution
Explain plan reports steps executed in parallel (PX for parallel execution)

13. Oracle In-Memory Best Practices
How to achieve optimal performance for Analytical workloads
Enable parallel execution
Help the Oracle optimizer to make the right decisions
Minimize necessary IO
Give Oracle (In-Memory) sufficient resources
MicroStrategy best practices for Oracle In-Memory

14. Breakthrough: Dual Format Database
Memory
SALES
Row
Format
Column
BOTH row and column formats for same table
Simultaneously active and transactionally consistent
Analytics & reporting use new in-memory Column format
OLTP uses proven row format

15. Column Store Replaces Analytic Indexes
1 – 3
OLTP
Indexes
Table
In-Memory
Column Store
Fast analytics on any columns
Better for unpredictable analytics
Less tuning & administration
Column Store not persistent so update cost is much lower
OLTP & batch run faster
The In-Memory Column store can now remove the need for additional analytic indexes if tables fit in memory.
Makes DML faster and reduces the overall storage space required for system.
And unlike a pure in-memory database, if the system should crash and need to restart your business can still operate fully.
OLTP queries and updates (the heart of any transaction based system) will perform just as they always do against the indexed row store.
Analytical queries will execute slowly until the In-Memory Column Store is populated, but they will still run.
You don’t have to wait for all of the data to be populated in memory before resuming your business.
Removing the need for analytic indexes greatly simplifies tuning and reduces ongoing administration.
Queries on large tables no longer need to be indexed to perform well.

16. Optimizer Decides If Columnar Store is Used
Several best practices help the optimizer understand the query
The DBA needs to help the Oracle optimizer to make the right decisions
Use Columnar Store vs. Row based store
Use Bloomfilter, Starschema, Vector optimizations
The good news: existing best practices are sufficient
Simplify: DROP ANALYTIC INDEXES
Use of NOT NULL in DDL
Use PK, FK contrains (“RELY DISABLE NOVALIDATE” is sufficient for this)
Enable the (new in ) Autostats feature which will capture statistics based on actual queries (will auto tune Oracle over time)
Enable star transformation (*.star_transformation_enabled='TRUE')
References:
Optimizer with Oracle Database 12c (June 2013)
Understanding Optimizer Statistics with Oracle Database 12c (June 2013)
Help the Oracle optimizer to make the right decisions (use Bloomfilter Join, Starschema optimizations, Vector optimizations)
Use of NOT NULL, PK, FK contrains (“RELY DISABLE NOVALIDATE” is sufficient for this)
Enable the (new in ) Autostats feature which will capture statistics based on actual queries (will auto tune Oracle over time)
Star transformation enabled (*.star_transformation_enabled='TRUE')

17. Verify Optimizer Decisions
Explain shows method of data access and use of advanced features
“TABLE ACCESS INMEMORY FULL“ in explain plan can be used to determine whether the inmemory tables are used. A good sign is when heavy tables (fact tables) are accessed from inmemory. In this eg, Order_detail
Proper design of tables, e.g.
Good partitioning scheme, table Constraints like PK, FK, nulls, etc, can help the usage of optimal “bloom filter operation” to filter data from big fact tables. Utilization of bloom filter can be identified from “filter(SYS_OP_BLOOM_FILTER(:BF0001”, clause in explain plan.
Note:- This was key to multifold performance gains in our test

18. Oracle In-Memory Best Practices
How to achieve optimal performance for Analytical workloads
Enable parallel execution
Help the Oracle optimizer to make the right decisions
Minimize necessary IO
Give Oracle (In-Memory) sufficient resources
MicroStrategy best practices for Oracle In-Memory

19. Orders of Magnitude Faster Analytic Data Scans
Memory
Each CPU core scans local in-memory columns
Scans use super fast SIMD vector instructions
Originally designed for graphics & science
Billions of rows/sec scan rate per CPU core
Row format is millions/sec
Example:
Find all sales
in region of CA
REGION
CPU
Vector Register
Load
multiple
region
values CA
Vector
Compare
all values
an 1 cycle CA CA
The Oracle in-memory column format is designed to enable very fast SIMD processing (single instruction processing multiple data values). You can imagine SIMD or vector processing as array processing.
SIMD was originally designed for accelerating computer generated animation and High Performance Scientific computing.
Lets assume we are looking for the total number of sales we have had in the region of California this year.
The sales table is stored in the In-Memory Column Store so we simply have to scan the state column and count the number of occurrence of the state of California. With SIMD processing we can check 16 values or entries in the state column in a single CPU cycle.
Columnar speed comes from
Scanning only needed columns
SIMD optimized format
Column specific compression algorithms CA
> 100x Faster

20. Joining and Combining Data Also Dramatically Faster
Example: Find total sales in outlet stores
Stores
Sales
Converts joins of data in multiple tables into fast column scans
Joins tables 10x faster
StoreID in
15, 38, 64
Type
Store ID
Store ID
Amount
Type=Outlet
We have shown that we can scan and filter in-memory data extremely quickly.
But with any new data format we need to be able to join and aggregate the data as well as scan it.
With In-memory vector joins, the join between the stores and sales table can be converted into a scan of the sales
table with the ‘where’ clause predicate on the store table being converted into a filter on the sales table.
This allows us to take full advantage of what In-memory is best for, fast scans and filters.
Sum

21. Analytical Queries Access Both Row And Column Stores
Data still needs to be organized on disk according to best practices
Tips
Use optimal table partitioning strategy (range vs. hash partition)
Use single BIGFILE tablespace for performance tables and indexes
Use single BIGFILE for temporary tablespace
Enable tablespace compression following best practices (COMPRESS BASIC)
References
Best Practices for Implementing a Data Warehouse on the Oracle Exadata Database Machine (March 2014)
Tablespace compression

22. Verify Partitioning is Effective
Optimizer sees an opportunity to parallelize IO operations
Optimizer sees an opportunity to parallelize operations on In-memory Columnar and Disk Row tables.

23. Oracle In-Memory Best Practices
How to achieve optimal performance for Analytical workloads
Enable parallel execution
Help the Oracle optimizer to make the right decisions
Minimize necessary IO
Give Oracle (In-Memory) sufficient resources
MicroStrategy best practices for Oracle In-Memory

24. Column Store Is Part of SGA
Optimal configuration needs to balance memory resources
Tips
Set appropriate In-memory area of SGA (*.inmemory_size= ???G)
Leave room for buffer cache
References:
Best Practices for a Data Warehouse on Oracle Database 11g (Nov 2010)

25. Oracle In-Memory Best Practices
How to achieve optimal performance for Analytical workloads
Enable parallel execution
Help the Oracle optimizer to make the right decisions
Minimize necessary IO
Give Oracle (In-Memory) sufficient resources
MicroStrategy best practices for Oracle In-Memory

26. MicroStrategy Best Practices
SQL Generation is very flexible
Enable use of temporary tables for intermediate result sets
Create single BIGFILE scratch tablespace for MSTR intermediate tables
Use Oracle join syntax rather than ANSI syntax
Allow Oracle to identify queries coming from MicroStrategy

27. Personal / Departmental
MicroStrategy Provides Two Data Access Options
Choose how to access and analyze data
Report
Modeled
Visual Insight
Dashboard
Direct
Report
Dashboard
Visual Insight
Unified MicroStrategy Metadata
Reusable Data
Reusable Objects
Reusable Design
MicroStrategy provides options to either gather direct analysis of the data or utilize the modeled format to analyze the data. The modeled creation process allows for greater customization and building the report or analysis from the ground the way the user wants. This approach offers opportunities for quicker and streamlined reports, dashboards, and visual insights to look at the data. The direct approach means that the user accesses previously created reports, dashboards, and visual insights directly from the cloud.
To achieve greater analytical richness Modeled access contains a multi-pass SQL capabilities. This will create intermediate results to be used in successive passes.
Databases
Big Data
Personal / Departmental
Cloud Data

28. Let Optimizer Manage Intermediate Results
Derived Table and Query Factoring syntax generate single query
Tends to result in very large queries!

29. MicroStrategy To Generate Script Using Temp Tables
Intermediate results managed explicitly in script
Default option for Intermediate Table Type (no change necessary)
Temp tables create lots of activity on temporary tablespace
Create single BIGFILE scratch tablespace for MSTR intermediate tables
Use fastest storage
Oracle join syntax is currently better understood by Optimizer than ANSI syntax
VLDB: Joins -> Join Type: Join 89

30. MicroStrategy Best Practices
SQL Generation is very flexible
Enable use of temporary tables for intermediate result sets
Create single BIGFILE scratch tablespace for MSTR intermediate tables
Use Oracle join syntax rather than ANSI syntax
Allow Oracle to identify queries coming from MicroStrategy

31. Comment Adds MicroStrategy User And Report Details
Allows identifying origin of SQL syntax in Oracle stats
VLDB Category
VLDB Property Setting
Value
Select/Insert
SQL Hint
/* MSTRUser=!u;Report=!r;*/

32. Oracle In-Memory Best Practices
How to achieve optimal performance for analytical workloads
Use existing hardware efficiently
Help the Oracle optimizer to make the right decisions
Minimize necessary IO
Give Oracle (In-Memory) sufficient resources
MicroStrategy best practices for Oracle In-Memory

33. Review other relevant technical content at this conference
Next Steps
Review other relevant technical content at this conference
Session ID: CON8221 Session Title: Real-World Performance of Star and Snowflake Schemas, Part 1: The Theory Venue / Room: Moscone South Date and Time: 9/30/14, 12: :45 Session ID: CON8222 Session Title: Real-World Performance of Star and Snowflake Schemas, Part 2: The Reality Venue / Room: Moscone South Date and Time: 10/1/14, 11: :15

34. Conclusion
Oracle In-Memory offers significant improvements for analytical workloads
Benefits
Enables Real-time Analytics
Improved query performance
Easy to adopt
Reduced cost
Implement Best Practices to achieve the benefits offered by Oracle In-Memory
Use existing hardware efficiently
Help the Oracle optimizer to make the right decisions
Minimize necessary IO
Give Oracle (In-Memory) sufficient resources
Implement MicroStrategy best practices for Oracle In-Memory