1. InfiniDB Overview

2. Copyright © 2014 InfiniDB. All Rights Reserved.
What is InfiniDB?
Massively Parallel MySQL Storage Engine for Fast Analytics
Linear scale to handle exponential growth
Open-Source
Runs on premise, on AWS cloud or Hadoop HDFS cluster
Standard ANSI SQL compliance
First MySQL storage engine to support ANSI SQL11-compliant windowing functions
100 % Columnar architecture
Scalable, high-performance analytics platform
Massive parallel processing (MPP) technology
Linearly scale with storage hardware and data volumes
Standard ANSI SQL compliance as a storage engine of MySQL
First MySQL storage engine to support ANSI SQL11-compliant windowing functions
Open-Source
Runs on premise, on AWS cloud or Hadoop HDFS cluster
Copyright © 2014 InfiniDB. All Rights Reserved.

3. Custom Handler Class InfiniDB Server User Connections MySQL Functions
MySQL Client
MySQL Connectivity (JDBC, ODBC)
MySQL Security
Initial SQL Statement Parsing
Initial SQL Optimization
< Custom Handler Class >
Execute final sort and final limit
Display final results
InfiniDB ExeMgr Functions
SQL Optimization
Distribute work for scan, filter, join, functions, expressions, group by, aggregation, etc. to the all available Performance Modules to be run in parallel.
Collect the results returned by the Performance Modules
Return the final results to MySQL for display
InfiniDB Server
MySQL
InfiniDB ExeMgr
User
Module
Performance Module(s)
Storage

4. InfiniDB Design Principles
Fast
Scalable
Simple

5. Copyright © 2014 InfiniDB. All Rights Reserved.
InfiniDB Parallelism
User Module – Processes SQL Requests
Performance Module – Executes the Queries
Single Server
MPP or
Copyright © 2014 InfiniDB. All Rights Reserved.

6. Tiered MPP Building Blocks
Module
Process
Functionality
Value
MySQL
Hosts MySQL
Connection management
SQL parsing & optimization
Familiar DBMS interface
Leverages existing partner integrations
Delivers full SQL syntax support
Extent Map
Abstracts physical and logical storage
Metadata store
Enables shared nothing and shared everything storage
Enables partition elimination
Built-in failover
ExeMgr
Work distribution
Final results management and aggregation
Independent scalability and tunable concurrency
Multi-threaded to take advantage of multi-core HW platforms
SQL

7. Tiered MPP Building Blocks
Module
Process
Functionality
Value
PrimProc
Scale-out cache management
Distributed scan, filter, join and aggregation operations
Resource management
Independent scalability and tunable performance
Multi-threaded to take advantage of multi-core HW platforms
Data
High Speed Bulk Load
Transactional DML and DDL
Online schema extensions
Enables concurrent reads and writes, non-blocking read enabled
Data Blocks

8. InfiniDB Foundation - Parallelism
Purpose-built C++ engine
Parallelism is at the thread level
Example: 12 PM Servers with 8 cores each yields 96 parallel processing engines.
SQL is translated into thousands or tens of thousands of discrete jobs or “primitives”.
The UM sends primitives to the processing engines.

9. InfiniDB Parallelism – Fixed Thread Pool
User Module – Processes SQL Requests
Performance Module – Executes the Queries
Single Server
MPP
Primitives are issued into a thread queue within each performance module.
Local disk / EBS GlusterFS / HDFS
Copyright © 2014 InfiniDB. All Rights Reserved.

10. Architectural Differentiation
Greenplum, Netezza, etc
Parent
Process
Parent
Process
Database Layer 1
- Executing SQL
Worker
Process
Worker
Process
Worker
Process
Database Layer 2
- Executing SQL
Database Layer
- Executing SQL
Block Processing Layer
Custom DoW

11. Architectural Differentiation
Greenplum, Netezza, etc
Parent
Process
Parent
Process
Worker
Process
Worker
Process
Worker
Process
Threads dedicated for the duration of a query.
Threads operate from queue, dedicated for a fraction of a second.

12. InfiniDB Design Principles
Fast
Scalable
Simple

13. Row-Oriented vs. Column-Oriented
Row-oriented: rows stored sequentially
Key
Fname
Lname
State
Zip
Phone
Age
Sex 1
Bugs
Bunny NY
11217
(718) 34 M 2
Yosemite
Sam CA
95389
(209) 52 3
Daffy
Duck
10013
(212) 35 4
Elmer
Fudd ME
04578
(207) 43 5
Witch
Hazel MA
01970
(978) 57 F
Column-oriented: each column is stored in a separate file
Key 1 2 3 4 5
Fname
Bugs
Yosemite
Daffy
Elmer
Witch
Lname
Bunny
Sam
Duck
Fudd
Hazel
State NY CA ME MA
Zip
11217
95389
10013
04578
01970
Phone
(718)
(209)
(212)
(207)
(978)
Age 34 52 35 43 57
Sex M F
Each column for a given row is at the same offset.
Copyright © 2014 InfiniDB. All Rights Reserved.

14. 2-Dimensional Data Partitioning
Vertical Partitioning by Column
Not Column-Family (no relation to HBase)
Only do I/O for columns requested
Horizontal Partitioning by range of rows
Meta-data stored within in-memory structure
10 TB of data maps to ~150k-300k discrete files.
Copyright © 2014 InfiniDB. All Rights Reserved.

15. Column Restriction and Projection
| Column # Four |
| Column # Six |
| Column # Seventeen |
Extent # 5
Filter 1
Filter 2
Filter 3
Projection
Projection
Extent # 27
Automatic Vertical Partitioning + Horizontal Partitioning
Just-In-Time Materialization 15 15

16. InfiniDB Design Principles
Fast
Scalable
Simple

17. Simplicity – Automated Everything
Column storage
Compression /compression type
No index build or maintenance required
Extent Map partitioning – Vertical/ Horizontal
Distribution of data across server/disk resources
Distribution of work
Ad-hoc performance 17

18. Windowing Analytic Functions
InfiniDB What’s New
Open Source – GPL v2
New Company Name
Funding
InfiniDB for Hadoop
Windowing Analytic Functions
Fast
Scalable
Simple

19. What is InfiniDB for Hadoop?
Fast SQL for Hadoop offering for real-time and ad-hoc reporting and analytics
Non-map/reduce engine for real-time SQL
40x to 100x faster than Hive
SQL in Hadoop
Reads and writes directly to HDFS/GPFS
Best of breed SQL in Hadoop
Superior ad-hoc usage, syntax vs. Impala/Presto
MySQL Compatibility
InfiniDB presents Hadoop as MySQL data source
Don’t talk about every bullet, pick out key ones and relate comments back to highlights from the ask.com portion

20. InfiniDB Background – InfiniDB for Hadoop
InfiniDB is a non-map/reduce engine
Reads and writes natively to HDFS
HBase
Pig/Hive
InfiniDB
for
Hadoop
Map Reduce
Hadoop Distributed File System

21. Value Proposition For InfiniDB for Hadoop
Enables access to Hadoop data via familiar interface
Response to competitive challenge from Cloudera Impala
Complete the Hadoop Checklist
Cost-effective storage
Robust transforms via map/reduce
Real-time SQL for analytics with InfiniDB for Hadoop
Don’t talk about every bullet, pick out key ones and relate comments back to highlights from the ask.com portion

22. Benchmark Hive, Presto, Impala, InfiniDB
Copyright © 2014 InfiniDB. All Rights Reserved.

23. PARTITION and FRAME
For each row, calculation for an aggregation is done over a FRAME of rows
The PARTITION of a row is the group of rows that have a value for a specific column same as the current row
FRAME for each row is a subset of a PARTITION for the row
SELECT x,y,sum(x) OVER (PARTITION BY y RANGE BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING) FROM a
Row Number X Y
PARTITION
FRAME 1
Partition for rows 1 to 4
Frame for row 1
sum(x) = 22
Frame for row 2
sum(x) = 21
Frame for row 3
sum(x) = 17
Frame for row 4
sum(x) = 10 2 4 3 7 10 5
Partition for rows 5 to 7
Frame for row 5 15
Frame for row 6 13
Frame for row 7
sum(x) = 8 6 8
Partition for rows 8 to 10
Frame for row 8 18
Frame for row 9
Frame for row 10
sum(x) = 9 9

24. InfiniDB Use Cases Who is using it? When to use it? ® Scalable Fast
Simple

25. Copyright © 2014 InfiniDB. All Rights Reserved.
InfiniDB Customers
Copyright © 2014 InfiniDB. All Rights Reserved.

26. InfiniDB’s place in the Big Data world
Designed for high performance analytics
Provides flexibility for ad hoc queries
Not suited for OLTP, NoSQL, KeyValue
Copyright © 2014 Calpont. All Rights Reserved.

27. Workload – Query Vision/Scope
,000 1,000, ,000,000 10,000,000,000
Query Vision/Scope
OLTP/NoSQL Workloads
Analytic Workloads
General DBMS missed the target (dated database technology generally suboptimal)
Copyright © 2014 Calpont. All Rights Reserved.

28. What is your typical query?
Analytic Workloads
,000 1,000, ,000,000 10,000,000,000
OLTP/NoSQL Workloads
Query Vision/Scope
There is no “average” query.
The challenges are at the extremes:
The challenge of high concurrency levels with OLTP/NoSQL.
The challenge of latency for very large queries.
Most use cases imply multiple data technologies.

29. Columnar Appropriate Workloads
ROLAP/Analytic/Reporting Workloads
,000 1,000, ,000,000 10,000,000,000
OLTP/NoSQL Workloads
Query Vision/Scope
Pure Columnar about 10x worse I/O for single record lookups
Pure Columnar about 10x better I/O for large data access patterns

30. Benefits of InfiniDB Products Real-time, Consistent Query Performance
Linear Scale for Massive Data
Removes Limits to Dimensions and Granularity
Easy to Deploy and Maintain
Scale-out MPP analytic database
MySQL Columnar + Map Reduction
Commercial Open Core model
Products
InfiniDB Enterprise
Forthcoming 4th major release
InfiniDB Community
Modified Open Source license

31. Core Features of InfiniDB
Scalable MPP architecture
Performant ad hoc analysis
Consistent query response time
Simplified data administration
Analytic window functions
Native MySQL® driver support
Open source license
Deployable on premise, in the cloud, & on Apache Hadoop™
Optional Enterprise support subscription
Copyright © 2014 Calpont. All Rights Reserved.