Presentation is loading. Please wait.

Presentation is loading. Please wait.

© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3 Big Data: High Volumes.

Similar presentations


Presentation on theme: "© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3 Big Data: High Volumes."— Presentation transcript:

1

2

3 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3 Big Data: High Volumes of Structured + Semi-Structured+ Unstructured Data What is Big Data? Big Data are high-volume, high-velocity, and/or high- variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization Big Data encompasses not only the content itself, but how it’s consumed 85% of enterprises data- typically structured- goes unharnessed Actionable intelligence require context; unstructured data is most relevant when combined with structured data Variety Velocity Volume

4 TODAY, A NEW SET OF QUESTIONS ARE BEING ASKED OF THE BUSINESS

5 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 5 Types of data * May require word-to-word and/or semantic analysis, metadata sometimes helps put into context. Information CRM Order ERP Finance Structured XML Spreadsheets Flat files in record format RSS feeds Web logs Semi-structured 90% of today’s data volume eMail text SMS BLOGSVideo Audio Social networking Images Unstructured *

6 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 6 What is unstructured data? Unlike structured data, which is organized, has clearly defined relationships unstructured data is free-flowing, disorganized, and needs new technologies like Hadoop to run analytics on it Structured Data: Example:* Unstructured data Example:* Characteristics:Charakteritics:  Organized in tables  Stored as records in a database  Well-defined relationships between data field  Typically used for data warehousing and analytics  Widespread in usage today  Free-flowing, unorganized, and unstructured  Stored as file and blobs  Little or no defined relationship  Need unstructured data analytics platforms like Hadoop  Emerging and fast growing space

7 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 7 The Challenge: The Current Solutions 10,000 20052015 2010 5,000 0 Current Database Solutions are designed for structured data.  Optimized to answer known questions quickly  Schemas dictate form/context  Difficult to adapt to new data types and new questions  Expensive at Petabyte scale STRUCTURED DATAUNSTRUCTURED DATA GIGABYTES OF DATA CREATED (IN BILLIONS) 10%

8 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 8 Big Data is exploding

9 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 9 YOTTABYTE A yottabyte is a data storage benchmark that's equal to 1,000 zettabytes. The total amount of data stored worldwide in 2012 reached 2.8 zettabytes, according to an IDC calculation. So we're a long way from reaching the yottabyte threshold -- although with the rate of big data growth, it might come sooner than we think.

10 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 10 You need a Next Gen Information Platform to tackle tomorrow’s information challenges A Day in the Life of Information Workloads, how fast? Transactional OperationalStrategic How is information turned into intelligent insight? Volume Variety Velocity Executive Dashboards Enterprise Search Customer Interaction Predictive Analytics Web Engagement How is the intelligent information insight consumed to drive intelligent business decisions? Value Where is the data created? (In which LoB?) CRM (Sales) Web ERP (Procurement) Supply Chain (Ops) Smart MetersHR Social Media VideoAudio Email Texts Transactional Data Word, Excel Logs Clickstream Data Images In what format? HP SOLUTIONS FOR BI AND BIG DATA

11 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 11 Disruption in data management solutions create new leaders Evolution of Data Hierarchical Data Models Relational Data Models 19601970198019902000 2010 Document Data Models Information Mgmt System (IMS) Structured Data Unstructured Data

12 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 12 Acquire “Big Data” solution Acquire Business Intelligence Buy new tier one hardware appliance Keep legacy investment Limited scalability High costs Significant training Solution complexity But current “big data” options are limiting

13 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 13 Fundamental Breakthrough in Data Processing Polybase Single Query; Structured and Unstructured Query and join Hadoop tables with Relational Tables Use Standard SQL language Select, From Where Existing SQL Skillset No IT Intervention Save Time and Costs Databas e HDFS (Hadoop) SQL Server 2012 PDW Powered by PolyBase SQL Analyze All Data Types

14 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 14 HP | Microsoft Frontline Innovation 30 years of joint commitments and innovation to help you succeed » Global reach, local services, & hosting increases your productivity » Complete lifecycle service & support 24x365 lowers your risk via mission critical support » Simpler procurement, deployment and servicing » Joint customer outreach to build the most relevant solutions that advance your business » End-to-end alignment on strategy, sales, and partner development to ensure customer satisfaction » Global Windows and solutions testing network improves performance and reliability » Proof-of-concept labs worldwide helps ease installation and implementation at a lower cost with lower risk » Over 500M invested across both companies dedicated to R&D » New product and technology innovations help you meet changing business requirements » Shared engineering resources and joint development provide improved TCO, reliability, availability and performance Joint Services and Support Joint Customer Outreach Joint Solutions Testing and Development Joint R&D and Shared Engineering “Our joint solutions deliver exceptional innovation and value, helping customers solve problems, capitalize on opportunities and drive success.” Meg Whitman, CEO, Hewlett-Packard Co “Microsoft and HP are betting on each other so our customers don’t have to gamble on IT.” Steve Ballmer, CEO, Microsoft Corp

15 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 15 Infrastructure-to-Applications 3-year agreement just renewed through 2015 Private Cloud and Mission Critical SQL – two contracts $7M joint 50/50 investment in Mission Critical SQL Global Initiative Program Office – Executive Sponsorship – Regional Exec Sponsorship WW & Regional Initiative Leads, extended ww and regional support team Governance and Joint Rhythm of the Business in place to manage execution

16 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 16 HP enterprise data warehouse and Microsoft PDW HP Enterprise Data Warehouse Integrated appliance based on HP Converged Infrastructure Microsoft Parallel Data Warehouse: Robust business analytics software platform Unmatched synergy offering true value to enterprise data warehouse market

17 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 17 Our Collective Joint Solutions Portfolio Build Your Own » Repeatable guideline » Performance tuned » Proven best practices » Sizing guidance » Reduced customer effort » Factory-built and pre- validated » Rapid Service Delivery » Customer optimized » Minimal disruption » Lowest customer effort » Infrastructure (servers, storage, networking) » Virtualization & Management » Cloud OS, Apps, Services » Greatest customer effort Reference Architectures Complete Integrated Solutions HP AppSystem for Microsoft Parallel Data Warehouse

18 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 18 Racked, stacked, wired, and ready for deployment HP builds the HP PDW Solution for you © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. HP Restricted. For HP and SI/O Partner use only. 18 Rather than deliver pallets of individual components... …your HP AppSystem for Microsoft PDW 2.0 arrives preassembled and ready to run

19 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 19 Site Preparation Survey – What we need to build the Appliance

20 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 20 Delivery time for HP AppSystem for PDW CountryOrder DateShip Date#weeks Sweden April 22 Jun 3 6,0 Netherlands May 15Jun 4 2,9 Belgium April 15May 16 4,4 ItalyApril 16May 10 3,4 Bahrain April 10May 3 3,3 Before release date ~3 weeks to ship. Upon arrival is ready for production “Jointly with my 2 TS Support colleagues of the Netherlands, we delivered the PDW to VX Company up and running yesterday, ready for use !” - Philippe Blondeaux – TS Consulting

21 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 21 New architecture, new name HP Enterprise Data Warehouse (EDW) v1.4 Appliance HP AppSystem for Microsoft SQL Server 2012 Parallel Data Warehouse

22 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 22 50% Lower cost per TB 3 100x Faster query speed 1 Part of a complete portfolio of solutions for Microsoft SQL Server 2012 HP AppSystem for Parallel Data Warehouse 30% Better scan rate 2 Key differentiators: Next generation data warehouse for mission-critical environments Factory built appliance based on HP Converged Infrastructure Pre-loaded with Microsoft software integrated, tested, and tuned Architecture chosen for best data warehouse performance Single view of information across the enterprise 1,2 Than previous generations 3 Than competitive offerings valueprism.com/resources/resources/Resources/PDW%20Compete%20Pricing%20FINAL.pdf

23 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 23 Beating Netezza POC results PDW is beating IBM on perform in most queries One user query which took 190 seconds on Netezza was now running in 40 seconds on PDW after some tuning for a specific query we are down to 3 seconds on PDW for a 10 sec query on Netezza We also showed much faster perf when running against DB2 on AS400, and the costomer has the largest midframe system in Europe User testing showed that they could run their existing developments with minimal changes on the PDW, something that IBM is unable to match. It is fair to say that “PDW is 3x faster than Netezza”

24 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 24 Customers Feedback French POC (Advertising) Saw a 7 minute Hive query reduced to 1 minute (120 node Hadoop cluster vs. 10 node PDW). After another query reduced from 5 minutes to under 1 second, they said, “C’est impossible!” and had to run it again to believe it. Russia POC (Telecom Software) Saw aggregate queries for 6 months data improving from 90 minutes to 49 seconds. They stated they thought it ‘impossible that ultimately a SQL Server solution’ can provide these query results. (Versus SQL Server 2008).

25 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 25 Customers Feedback UK POC (Public Transit) saw 10x better load speeds; data transformation from 4 hours to 136 seconds; and concurrency testing on 7 years of data with 200 users going from 2 hours to 12 minutes. (Versus Oracle and SQL Server). German POC (Health Care) saw complex query (for 80M rows) improve 1.7 min to 5 sec, ETL performance improve 10x, SSIS performance improved 6x, and compression improve 7x. (Versus SQL Server 2008). They also tested versus PDW V1 (full rack vs. full rack), and saw SSIS load tests improve 2x and complex query processing time improve from 1.5 minutes to 5-20 seconds.

26 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 26 HP and partner assessment, design, and planning services Deep integration for rapid time-to-value HP AppSystem for Parallel Data Warehouse Collaborative support services from HP and Microsoft 7 On-site deployment services Microsoft Windows 2012 and Microsoft SQL Server 2012 Parallel Data Warehouse software 6 Hadoop integration HP management tools and utilities HP Converged Infrastructure: servers, storage, and networking 6 Priced separately 7 HP Proactive Care. Microsoft support priced and purchased separately

27 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 27 Start Small With A Few TB and Linearly Scale OUT Seamlessly Add Capacity Smallest (15TB) To Largest (6PB) Start small with a few Terabyte warehouse Add capacity up to 6 Petabytes 15TB 6 PB Add Capacity Add Capacity Largest Warehouse PB Start Small And Grow No Downtime

28 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 28 Hardware Details – Full Rack 1 – HP 36-port FDR Infiniband Switches (qty 2) 2 – HP 5120 Ethernet Switches (qty 2) 3 – Passive nodes 1 and 2 (HST01 and HST02) HST01 hosts the management and control node functions (VMs) HST02 is the primary failover target for any of the other nodes  Optional HST03 third passive node (second failover target) would be installed under HST02 4 – Base Active Scale Unit, consisting of Active nodes 1 and 2 (HSA01 and HSA02) D6000 storage with 70 LFF MDL SAS spindles (1, 2 or 3 TB) 5 – Future add-capacity Active Scale Unit HSA09 and HSA10, plus D6000 6 – 3 rd Active Scale Expansion Unit HSA07 and HSA08, plus D6000 7 – 2 nd Active Scale Expansion Unit HSA05 and HSA06, plus D6000 8 – 1 st Active Scale Expansion Unit HSA03 and HSA04, plus D6000  Power for full rack: 10.3kW (49.5A at 208V)  Cooling for full rack: 35000 BTU/hr  Power connections: both 3-phase and single phase options

29 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 29 Active scale unit Two server nodes HP ProLiant DL360 Gen8 Server – 1U each – 2 x E5-2690 “Sandy Bridge” processors (2.90 GHz, 20 M cache, 8 cores, 8 GT/s QPI speed) – 2 internal 600 GB disks (all TempDB and logs in the storage node) – 256 GB RAM Runs compute node VM (and iSCSI VM for quorum) One storage node D6000 JBOD, connected to each server through an H221 SAS HBA – 5U – 70 Spindles (35 per server) – 6 Gb/s, LFF, 7200 RPM MDL SAS – Disk capacities: 1, 2, and 3 TB Cables (from each server) 2 Ethernet 2 InfiniBand 2 SAS 2 Power HP AppSystem for Parallel Data Warehouse hardware

30 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 30 HP AppSystem for Parallel Data Warehouse hardware Passive scale unit One server node HP ProLiant DL360 Gen8 Server – 1U – 2 x E5-2690 “Sandy Bridge” processors (2.90 GHz, 20 M cache, 8 cores, 8 GT/s QPI speed) – 2 internal 600 GB disks – 256 GB RAM Runs management and control VMs (and iSCSI VM for quorum)

31 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 31 HP AppSystem for Parallel Data Warehouse – HA and failover Passive server (#1) Passive server (#2) Compute nodes D6000 Storage 1 Storage 2 Node 1 Node 2 Private Ethernet network Dual InfiniBand SAS Base scale unit Control and management functionality Compute nodes access their local D6000 in their respective scale unit

32 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 32 HP AppSystem for Parallel Data Warehouse – HA and failover Passive server (#1) Passive server (#2) Compute nodes D6000 Storage 1 Storage 2 Node 1 Node 2 Private Ethernet network Dual InfiniBand SAS Base scale unit Control and management functionality

33 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 33 HP AppSystem for Parallel Data Warehouse – HA and failover Passive server (#1) Passive server (#2) Compute nodes D6000 Storage 1 Storage 2 Node 1 Node 2 Private Ethernet network Dual InfiniBand SAS Base scale unit Compute node functionality (SQL Server) for failed node kicks in during failover Passive server #1 takes over Passive server runs SQL Server and accesses the failed server’s D6000 data

34 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 34 HP AppSystem for Parallel Data Warehouse – HA and failover Passive server (#1) Passive server (#2) Compute nodes D6000 Storage 1 Storage 2 Node 1 Node 2 Private Ethernet network Dual InfiniBand Compute node takes over (SQL Server) SAS Base scale unit SQL Server runs in new compute node Data is accessed via InfiniBand remote direct memory access (RDMA)  Minimal performance impact on remote server  Most CPU utilization is in the new compute node Control and management functionality

35 © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. PDW v2 Capacity

36 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 36 Assumes a 5:1 compression ratio, based on ColumnStore Index HP AppSystem for PDW v2 capacities: 1TB disks # racksCompute nodes User data (TB) Total Raw (TB) Raw PRI (TB)Raw TempDB (TB) Raw Log (TB) 12762915103.8 41515830207.6 622787453111.3 8302116604115.1 2 (future: 1)10378145765118.9 212453174916122.7 166052331218130.2 32490734918112245.3 432120946524216360.5 540151158130220375.6 648181469836324490.7 7562116814423285105.8 2 to 56 nodes 15 TB to 1.3 PB raw Up to 6.3 PB user data 2 node increments for small topologies

37 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 37 RAW Data Calculation Example  1TB * 32 (Number of disk drives per enclosure) * 2 (Number of enclosures or JBODs) / 2 (RAID1 mirrored) = 1TB * 32 * 2 /2 = 32TB of raw space  Formatting: 32*0,93= 29,76TB of total raw data Note: We should Not exceed 85% of the total disk space and 40% of Disk Space is designed for replicated space (LOG – TempDB, etc..)  25,1 TB*0,6 (40%)=15.1TB of non-compressed primary raw data And with 5:1 compression:  15,1TB*5(compression)= 75,5TB of user data HP AppSystem for PDW v2 capacities: 1TB disks Assumes a 5:1 compression ratio, based on ColumnStore Index

38 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 38 Assumes a 5:1 compression ratio, based on ColumnStore Index # racksCompute nodes User data (TB) Total Raw (TB) Raw PRI (TB)Raw TempDB (TB) Raw Log (TB) 121515830207.6 4302116604115.1 6453174916122.7 86052331218130.2 2 (future: 1)1075629115110237.8 21290734918112245.3 16120946524216360.5 324181469836324490.7 4322418930484326120.9 54030231163605407151.1 64836271395725488181.4 75642321628846570211.6 2 to 56 nodes 15 TB to 1.3 PB raw Up to 6.3 PB user data 2 node increments for small topologies HP AppSystem for PDW v2 capacities: 2TB disks

39 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 39 Assumes a 5:1 compression ratio, based on ColumnStore Index HP AppSystem for PDW v2 capacities: 3TB disks # racksCompute nodes User data (TB) Total Raw (TB) Raw PRI (TB)Raw TempDB (TB) Raw Log (TB) 1222787453111.3 4453174916122.7 66802621369234 890734918112245.3 2 (future: 1)10113343622715356.7 212136052327218368 16181469836324490.7 32427201046544366136.0 43236271395725488181.4 54045341744907610226.7 648544120931088732272 756634724411269854317.4 2 to 56 nodes 15 TB to 1.3 PB raw Up to 6.3 PB user data 2 node increments for small topologies

40 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 40 Reduce operational costs and ease maintenance HP Appsystem for PDW Appliance Support Pack Utility Suite Unique set of tools to simplify the installation, operation and maintenance keeping the appliance running at optimal levels –One-button update – 1 click, 2,000 components (drivers and firmware) –Utility to set the password for all devices (password changing script) –Utility to collect serial numbers from all devices and generate a report of all of the serial numbers and rack locations of all devices Utility to run diagnostics on the appliance (beyond PAV) – Quickly identifies hardware issues from shipment or component failure – Checks that all cables are plugged in to the right ports and run at the correct speed, checks that all devices are in a healthy state and properly configured from a hardware perspective HP only - value add to PDW software

41 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 41 HP Proactive Care for HP AppSystem for Parallel Data Warehouse Complete proactive and reactive support reduces risk Semi-annual firmware, patch, and software update recommendations Semi-annual proactive scan and recommendation Quarterly incident and trend reporting Direct connection to level two expertise Documented, jointly agreed HP/Microsoft collaborative support processes Single point of contact for end-to-end case ownership and call management Enhanced response time and choice of HW on-site support levels Secure 24x7 monitoring, diagnostics, and notifications Required (installation assistance provided) Advisory support Hardware and software support Insight remote support Reduce implementation time and cost Expand/consolidate data warehouse capability Improve price/performance ratio Reduced risk in implementation Integrate smoothly with BI architecture Key Benefits Insight remote support

42 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 42 HP AppSystem for Parallel Data Warehouse Summary High performance Seamless scalabilityLowest cost/TB Fast time-to-value Ready for any data Extensive integration for BI and analytics

43 © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 43 Thank you

44


Download ppt "© Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. 3 Big Data: High Volumes."

Similar presentations


Ads by Google