High Performance Analytical Appliance MPP Database Server Platform for high performance Prebuilt appliance with HW & SW included and optimally configured.

Slides:



Advertisements
Similar presentations
Extreme Performance with Oracle Data Warehousing
Advertisements

2012 © Trivadis BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN TechTalk Beste Skalierbarkeit dank massiv.
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1.
Big Data Working with Terabytes in SQL Server Andrew Novick
Microsoft Data Warehouse Vision Massive Scalability at Low Cost Improved Business Agility and Alignment Democratized Business Intelligence Hardware.
Doug Lanman Data Warehousing SSP North Central, Midwest and Heartland Districts SQL Server Data Warehousing.
1. Aim High with Oracle Real World Performance Andrew Holdsworth Director Real World Performance Group Server Technologies.
Scale-out Central Store. Conventional Storage Verses Scale Out Clustered Storage Conventional Storage Scale Out Clustered Storage Faster……………………………………………….
Danny Tambs Solution Architect. VOLUME (Size) VARIETY (Structure) VELOCITY (Speed)
Meanwhile RAM cost continues to drop Moore’s Law on total CPU processing power holds but in parallel processing… CPU clock rate stalled… Because.
Microsoft Ignite /16/2017 5:47 PM
Introduction to the new mainframe: Large-Scale Commercial Computing © Copyright IBM Corp., All rights reserved. Chapter 1: The new mainframe.
5 Creating the Physical Model. Designing the Physical Model Phase IV: Defining the physical model.
Database System Architectures  Client-server Database System  Parallel Database System  Distributed Database System Wei Jiang.
10-fold increase in data volume every 5 years “DW has shifted almost entirely towards the appliance model due to speed of the balanced appliance and.
Fast Track, Microsoft SQL Server 2008 Parallel Data Warehouse and Traditional Data Warehouse Design BI Best Practices and Tuning for Scaling SQL Server.
© Hitachi Data Systems Corporation All rights reserved. 1 1 Det går pænt stærkt! Tony Franck Senior Solution Manager.
April 10-12, Chicago, IL PDW Architecture Gets Real: Customer Implementations Brian Walker | Microsoft Corporation PDW Center of Excellence Murshed Zaman.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. 1 Preview of Oracle Database 12 c In-Memory Option Thomas Kyte
Analyzing the Energy Efficiency of a Database Server Hanskamal Patel SE 521.
An Introduction to Infrastructure Ch 11. Issues Performance drain on the operating environment Technical skills of the data warehouse implementers Operational.
PMIT-6102 Advanced Database Systems
SQL Server Warehousing (Fast Track 4.0 & PDW)
NCR CORPORATION Presented by: Dave Raspberry Cheng Murray-Khoo Eric Braun A Data Warehousing Solutions Provider A Data Warehousing Solutions Provider.
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information herein is subject to change without notice. HP Restricted. HP AppSystem for.
Microsoft ® SQL Server ® 2008 and SQL Server 2008 R2 Infrastructure Planning and Design Published: February 2009 Updated: January 2012.
1 Advanced Storage Technologies for High Performance Computing Sorin, Faibish EMC NAS Senior Technologist IDC HPC User Forum, April 14-16, Norfolk, VA.
Keith Bodell, Account Executive - Federal Matt Campbell, Systems Engineer - Federal NetezzaTwinFin.
SQL Server Data Warehousing Overview
DBI332 ilikesql brianwmitchelll UNSTRUCTURED UNBALANCED UNPREDICTABLE.
Introduction to Hadoop and HDFS
The Red Storm High Performance Computer March 19, 2008 Sue Kelly Sandia National Laboratories Abstract: Sandia National.
MediaGrid Processing Framework 2009 February 19 Jason Danielson.
SESSION CODE: BIE07-INT Eric Kraemer Senior Program Manager Microsoft Corporation.
2012 © Trivadis BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN SQL Server 2012 Parallel Data Warehouse.
MIS2502: Data Analytics The Information Architecture of an Organization.
Building BI Solutions with SQL Server PDW AU3 Ruwen Hess Senior Program Manager Microsoft Corporation DBI321.
 2009 Calpont Corporation 1 Calpont Open Source Columnar Storage Engine for Scalable MySQL Data Warehousing April 22, 2009 MySQL User Conference Santa.
Srik Raghavan Principal Lead Program Manager Kevin Cox Principal Program Manager SESSION CODE: DAT206.
CS338Parallel and Distributed Databases11-1 Parallel and Distributed Databases Lecture Topics Multi-CPU and distributed systems Monolithic system Client–server.
2012 © Trivadis BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN Welcome November 2012 Vorstellung Parallel.
Solution to help customers and partners accelerate their data.
© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. Big Data Directions Greg.
Introduction to the new mainframe © Copyright IBM Corp., All rights reserved. 1 Main Frame Computing Objectives Explain why data resides on mainframe.
Infrastructure for Data Warehouses. Basics Of Data Access Data Store Machine Memory Buffer Memory Cache Data Store Buffer Bus Structure.
CERN - IT Department CH-1211 Genève 23 Switzerland t High Availability Databases based on Oracle 10g RAC on Linux WLCG Tier2 Tutorials, CERN,
Last Updated : 27 th April 2004 Center of Excellence Data Warehousing Group Teradata Performance Optimization.
Rushabh Mehta Managing Director (India) | Solid Quality Mentors
Database CNAF Barbara Martelli Rome, April 4 st 2006.
SMP MPP with PDW ** Workload requirements usually drive the architecture decision.
Azure SQL DW – Elastic Data Analytics in the cloud Josh Sivey | Microsoft TSP #492 | Phoenix.
Microsoft Analytics Platform System Stefan Cronjaeger, Microsoft.
Modern Data Warehousing Symmetric Multi-Processing SQL (SMP) vs Massive Parallel Processing SQL (MPP) Alain Dormehl P-Cubed Session Level : Intermediary.
SQL Server 2008 R2 Parallel Data Warehouse: Under the Hood Brian Mitchell Senior Premier Field Engineer.
Clint Kunz Data Platform Technology Specialist
…the secret sauce! Diagrams and video from Microsoft white papers and slide decks.
Integration of Oracle and Hadoop: hybrid databases affordable at scale
Data Platform and Analytics Foundational Training
Integration of Oracle and Hadoop: hybrid databases affordable at scale
Data Warehouse in the Cloud – Marketing or Reality?
Data Warehousing: SQL Server Parallel Data Warehouse AU3 update
What is the Azure SQL Datawarehouse?
Azure SQL Data Warehouse for SQL Server DBAS
Microsoft Analytics Platform System
Ch 4. The Evolution of Analytic Scalability
Managing batch processing Transient Azure SQL Warehouse Resource
Microsoft Analytics Platform System 03 – Distribution Theory & Design
Dell EMC SQL Server Solutions Doug Bernhardt
Moving your on-prem data warehouse to cloud. What are your options?
Presentation transcript:

High Performance Analytical Appliance MPP Database Server Platform for high performance Prebuilt appliance with HW & SW included and optimally configured Shared nothing architecture; In-Memory Columnstore engine Value Lowest price per terabyte of high end DW appliances on the market Up to 100x faster than legacy SMP Database queries Up to 15x data compression Scale & Support Scales from Terabytes to Petabytes; Built with Big Data in mind Architected with redundancy throughout Integrated, single call Support

MPP Massively Parallel Processing SMP Symmetric Processing Multiple CPUs used to complete individual processes simultaneously All CPUs share the same memory, disks, and network controllers Mostly, the solution is housed on a shared SAN Increase compute capacity via scale-up design Uses separate CPUs running in parallel to execute a single query Each CPU has its own allocated memory High-speed communications between nodes Increase compute capacity via scale-out design

2nd Scale Unit (additional 3 nodes optional ) Base Unit (3 nodes) Infiniband Switch Ethernet Switch Management Control Node Management Failover Node Ethernet Switch Infiniband Switch 3 rd Scale Unit (additional 3 nodes optional) JBOD Compute Server JBOD Compute Server JBOD Compute Server JBOD

3 – 54 Compute Nodes 1 – 6 Racks 1TB, 2TB or 3TB Drives TB – 1,223TB Raw 113 TB – 6PB User Data Linear ScaleMixed Workloads No DowntimeStart Small & Grow 113TB6 PB

Time Dim Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Store Dim Store Dim ID Store Name Store Mgr Store Size Store Dim ID Store Name Store Mgr Store Size Product Dim Prod Dim ID Prod Category Prod Sub Cat Prod Desc Prod Dim ID Prod Category Prod Sub Cat Prod Desc Mktg Campaign Dim Mktg Campaign Dim Mktg Camp ID Camp Name Camp Mgr Camp Start Camp End Sales Facts Date Dim ID Store Dim ID Prod Dim ID Mktg Camp Id Qty Sold Dollars Sold

SF -1 Sales Facts Date Dim ID Store Dim ID Prod Dim ID Mktg Camp Id Qty Sold Dollars Sold SF -1 SF -2 SF -3 Time Dim Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Store Dim Store Dim ID Store Name Store Mgr Store Size Store Dim ID Store Name Store Mgr Store Size Product Dim Prod Dim ID Prod Category Prod Sub Cat Prod Desc Prod Dim ID Prod Category Prod Sub Cat Prod Desc Mktg Campaign Dim Mktg Campaign Dim Mktg Camp ID Camp Name Camp Mgr Camp Start Camp End

Time Dim Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Date Dim ID Calendar Year Calendar Qtr Calendar Mo Calendar Day Store Dim Store Dim ID Store Name Store Mgr Store Size Store Dim ID Store Name Store Mgr Store Size Product Dim Prod Dim ID Prod Category Prod Sub Cat Prod Desc Prod Dim ID Prod Category Prod Sub Cat Prod Desc Mktg Campaign Dim Mktg Campaign Dim Mktg Camp ID Camp Name Camp Mgr Camp Start Camp End TDTD PDPD SDSD MDMD TDTD PDPD SDSD MDMD Smaller Dimension Tables are Replicated on Every Compute Node TDTD PDPD SDSD MDMD Sales Facts Date Dim ID Store Dim ID Prod Dim ID Mktg Camp Id Qty Sold Dollars Sold SF -1 SF -2 SF -3 Result: Fact -Dimension Joins can be performed locally

Infiniband Switch Ethernet Switch Management Control Node Management Failover Node Ethernet Switch Infiniband Switch JBOD Compute Server JBOD Compute Server JBOD Compute Server JBOD

From 2.5 Hours to 5.5 Minutes From 3.3 TB To 486 GB 3.3TB to 90 GBFrom 43 min to 90 sec Minimal concurrency impact

SQL Server Integration Services (SSIS) DWLoader Achieve data load speeds of up to 1.7 TB per hour Accommodate multiple and concurrent incremental loads Provides transactional protection and configurable batch size (10,000) Supports direct load of compressed files SQL Server Parallel Data Warehouse Connection Manager SQL Server Parallel Data Warehouse Destination