Presentation on theme: "Huawei Big Data Storage N9000"— Presentation transcript:
1 Huawei Big Data Storage N9000 Jan StiborStorage & Server Product Manager
2 1 2 3 Definition of Big Data HUAWEI Big Data Storage Solution Application Scenarios of HUAWEI Solution
3 Era of Big Data Is Coming 250 million pictures per dayVideos of 28,800 hours per day2.9 million s per second globally24 PB data processed per day200 PB per safe city6.3 million orders per dayMore than 1 PB per 3D movie50 million messages per day250 million pictures are uploaded to Facebook per day on average.There are 200 million video playbacks on Youku per day.The capacity of a 3D movie is more than 1 PB.The storage capacity of a safe city is 200 PB.Starting from 2012, data amount doubles every two years. By 2020, data amount is 44 times that of 2009.In the first quarter of 2012 alone, the website and mobile client of gained over 0.8 billion views monthly.During the period when Euro CUP is held, the daily average number of users that visit v.qq.com/ is 1.76 million.If you learn about the big data market composed of servers, storage devices, networks, software, and services defined by IDC, you will discover a market that has a value of 3.2 billion dollars in 2010 and is expected to grow with the compound annual growth rate of 39.4% for five years, reaching 16.9 billion dollars in IDC said that such a growth rate is seven times that of the entire IT market within the same five years.By 2015, data amount will reach 8 ZB with the valued of $16.9 billion.
4 What Is Big Data? Mass data Volume Velocity Exponential growth of unstructured dataQuick data flow and dynamic data systemReal-time analysis, not post analysisVeracityVarietyVarious data typesUnstructured data such as files, s, and videos accounting for 90% of the data amount generated in the next 10 years Mass data (volume), quick data flow and dynamic data system (velocity), various data types (variety), and great data value (value)Veracity of Great dataPrediction and analysis of the future and the behavior modeIDC: four characteristics of big data
5 Driving Force of Big Data 1Storage — open architecture, gradual scale out, and integration of near-line systems and offline systemsBig DataCollection2Analysis — compound data, unified storage of multiple data sources, unified query, and sharing of storage resourcesStorage3Archiving — lifecycle management of mass data at a low cost and high energy efficiencyBig data analysis is a technology-driven strategy that aims to obtain comprehensive, accurate, and in-depth insights into customers, partners, and business, gaining competitive strength. By processing stable real-time data flows, an enterprise is able to promptly make time-sensitive decisions, monitor the latest trend, and adjust its strategy to seize new business opportunities.Big data is a also a big challenge to traditional data storage in terms of data collection > storage > management > cleanup > analysis.Big data requires the following:Big storage: distributed storage and flexible management of mass dataHigh performance: quick access and searchEfficient data analysis: real-time analysis, predictive and prescriptive analysisBig analýza dat je technologie-řízený strategie, která si klade za cíl získat komplexní, přesné a do hloubky pohled na zákazníky, partnery a podniky získávají konkurenční sílu. Zpracováním stabilní real-time datové toky, je podnik schopen okamžitě provést časově citlivé rozhodnutí, sleduje nejnovější trendy a přizpůsobit svou strategii využívat nové obchodní příležitosti.Big data jsou také velkou výzvou pro tradiční ukládání dat z hlediska sběru dat> Úložiště> Správa> čištění> analýza.Big dat vyžaduje následující: Big skladování: distribuované ukládání a flexibilní řízení hromadných dat Vysoký výkon: rychlý přístup a vyhledávání Efektivní analýza dat: analýza v reálném čase, prediktivní a normativní analýzaManagement4Maintenance — easy to useAnalysis
6 Contents 1 2 3 Definition of Big Data HUAWEI Big Data Storage Solution Application Scenarios of HUAWEI Solution
7 HUAWEI Big Data Storage Solution CSS/CSEHUAWEI Big Data Storage SolutionAnalysisStorageArchivingLarge-capacity solutionHigh-performance solutionBigDataMass data query and analysis solutionHigh-performance solutionLarge-capacity solutionMass data query solution
8 HUAWEI OceanStor N9000—X in 1 Quick data importSupport for JDBC and ODBCSQL 92/2003Data compressionLinear expansion from 3 to 288 nodesMore than 3,000,000 OPSCapacity up to 100 PBNFS/CIFS interfacesHDFS interfaceGlobal online deduplicationSupport for LTFSScale-out NASScale-out DBScale-out Backup
9 HUAWEI OceanStor N9000 Architecture ApplicationlayerMedia assetInternetHPC…Behavior prediction, real-time analysis, …Apple……Unstructured data, object data, structured dataBig data sharingBIBig data analysisNAS interfaceObject interfaceArchiving interfaceDatabase interfaceMap-reduceHDFSUnified management and namespaceLifecycle policyIndex of quick file searchRemote replication and snapshotGlobal deduplicationPeriodic data inspectionHPC: High-Performance ComputingStorageAnalysisArchivingStoragelayer10GE/InfinibandNodeNodeNodeNodeNodeNode
10 N9000 Hardware 1 U high High-performance node： 25 disk slots2-channel hexa-core CPU48 GB memory at leastSSDs for storing metadataHigh-performance node：The scene of frequent read and write small filesFront viewRear viewFront view4 U high36 disk slots2-channel hexa-core CPU48 GB memory at leastHigh-bandwidth node：The scene of large file sequential read and writeRear view4 U high36 disk slots2-channel quad-core CPU16 GB memory at leastFront viewLarge-capacity node：Near-line storageRear viewAcceleration node1 U high
11 N9000 Software — Wise Series WushanFSSelf-developed distributed file systemMandatory, one-off purchaseWise Data Protection (WiseDP)1WiseLinkCluster load balancing softwareSupport for multiple load balancing policiesOptional, allowing for separate purchase2WushanFSWiseQuotaQuota management software that sets space usage limits on directories or users.Optional, allowing for separate purchase3WushshanFS: a suite of enterprise software program used to provide a single volume, single file system, and a single namespace for forming a base for an enterprise DC.WushanFS: WushanFile SystemWiseTierDST softwareSupport for policy-based automatic migrationOptional, allowing for separate purchase4
12 Extensive Flexibility: WiseDP Number of NodesODCRDCRDNMax. No. of Allowed Faulty DisksMax. No. of Allowed Faulty NodesUtilization342167%675%5880%10141286%71%20Switch+1ECC+2ECCOriginal data counts (ODC)Redundant data counts (RDC)Redundant data nodes (RDN)+3ECC+4ECCN+1 to N+4 data protectionFile-level data protection1 hr/TB data recovery
13 Outstanding Performance: WiseLink Automatic load balancing among existent and newly added nodes...Single node expansion within 60sWithout human intervention and manual modifications
14 Outstanding Performance: WiseTier Enables DST to work among diverse types of nodes.Adopts tiering policies based on I/O access popularity.Adopts hotspot anti-shake policy to prevent unnecessary data migration.Seamlessly adapts to service changes without interrupting ongoing applications.Consolidates multiple storage systems into a single storage system, eliminating the possibilities of frequent data migration and simplifying management.High-OPS nodesLarge-capacity nodesHot dataCold data
15 Simplified Management: Refined Space Management WiseQuota: sets quotas for subdirectories to facilitate space management.100 TB10 TB20 TB40 TB30 TBFast automatic deploymentComprehensive performance monitoringUserUserUser groupUser groupUnified topology display20 TB10 TBCentralized management of a single file system of 100 PBUserUser
16 HUAWEI 9000 — Adapting to Changes and Converging for Easy Management Extensive FlexibilityOutstanding PerformanceOptimal IntegrationStreamlined ManagementScaling from 3 to 288 nodesSupport up to 100 PB of capacityLinear scaling of performance and capacityWorld's leading file system access performance of more than 3 million OPSDistributed scale-out architectureIntegration of scale-out NAS, scale-out DB, and scale-out backupUnified management of all nodes on a single interfaceLCMAdapting to changes and converging for easy management is the core design concept of the N9000. The N9000 provides a powerful scalability from 3 to 288 nodes capable of delivering up 10 PB of storage capacity. The file system access performance is expected to exceed 3 million OPS. The linear scaling of capacity and performance contributes to the unprecedented ultra-large capacity and ultra-high performance. The N9000 innovatively incorporates scale-out NAS, DB, and backup for bearing multiple types of applications and centrally manages all nodes on a single interface for providing complete LCM.
17 Contents 1 2 3 Definition of Big Data HUAWEI Big Data Storage Solution Application Scenarios of HUAWEI Solution
18 HUAWEI OceanStor N9000 Scenario— Media Asset Management Focusing on five aspectsAnimation rendering…Non-linear editing systemHD editingSD editingCatalogingTV station non-linear editing system solutionStable bit steams and low single-stream bandwidth, with a strong need for data sharingAnimation rendering solutionFrame image file storage, with a strong need for high performance in sequential read of small-sized I/OsVOD solutionConcurrent reads by multiple users, with a strong need for high bandwidthMedia asset management solutionLarge capacity, high bandwidth, and data sharingMovie post-production solutionStable bid streams and high-bandwidth throughput, providing data sharing...HPCN9000 high-performance nodesN9000 high-bandwidth nodes and large-capacity nodesVOD server clusterTV set + set-top boxPCSmart phoneVOD systemLANThe solution provides over 2.2 GB/s access throughput to meet the need for design and production of 10 channels.The storage system supports high-speed access to hotspot files.The storage system provides the capability to manage the space and rights of production projects, allowing easy project-level and directory- level space allocation and management of project team members' data access rights.3.6 GB/s production and rendering performance is provided. Three-level performance acceleration improves hotspot data access performance, allowing 120 rendering computing nodes to access the storage system concurrently.Highly reliable storage is provided, with the support for 7x14 service continuity.The scale-out architecture supports online smooth expansion.Comprehensive capability to manage user rights and space quotas.Snapshot, mirror, and backup provide multi-level continuous protection of critical data.EditingAudioEffect…Query/CatalogingPost-production systemMedia asset management system
Your consent to our cookies if you continue to use this website.