Presentation is loading. Please wait.

Presentation is loading. Please wait.

Scality RING Organic Storage 4 Jan. 2013

Similar presentations

Presentation on theme: "Scality RING Organic Storage 4 Jan. 2013"— Presentation transcript:


2 Scality RING Organic Storage 4 Jan. 2013

3 Agenda Corporate Profile Industry Trends and Business Challenges
Scality RING 4 Scality Solutions Ecosystem Roadmap and Vision Conclusion

4 Scality – Quick Facts Founded 2009 Experienced management team
HQ in the San Francisco, Global reach <50 Employees, 20 engineers in Paris 24 x 7 support team US patents $13M invested in Scality to-date 120% annual growth Industry Associations “Aggressive use of a scale-out architecture like that enabled by Scality's RING architecture will become more prevalent, as IT organizations develop best practices that boost storage asset use, reduce operational overhead, and meet high data availability expectations.”

5 A Rolling and Unstoppable Ball…

6 Industry Trends Rapid unstructured data growth
“Enterprises will see a staggering 52% growth in data over the next years — much of it an increase in unstructured data”, IDC x8-10 in 5 years, x100 in 10 years !! 100TB today will be 1PB then 10PB Rapid decline in storage costs “Cost of disk storage systems is declining >30% annually”, IDC Skyrocketing storage budget & complexity More storage devices need to be managed Increasing cost of commercial real estate Increasing energy costs (power, cooling) Cost of managing storage between 5 and 10 times the acq. cost (Gartner, IDC) Storage Budget TB Growth $/TB Cost

7 Data and Storage Challenges
100s Millions of Users, 10s-100s PB of Data and Billions of files to store and serve What all these companies have in common ? Internet/Cloud business impossible to sustain and develop with traditional IT approaches

8 Control and Efficiency at Scale
Scality’s Mission Their DC Their App. YOUR Data Their DC YOUR App. YOUR Data YOUR DC YOUR App. YOUR Data Control and Efficiency at Scale Unified Scale-Out Storage Software from primary to long term storage with Cloud’s advantages, investment protection and ready for future

9 Scality RING Organic Storage 4
File Storage StaaS Digital Media Big Data* Enterprise & Cloud System Scale Out File System S3 & CDMI API Data Processing with Hadoop Origin Server Scality RING Organic Storage 4 Ring Topology P2P End-to-End Parallelism DATA MD Object Storage Replication ARC Erasure Coding Geo Redundancy Tiering S3 CDMI Standard Management MESA NewSQL DB x86 * Available Q2/2013

10 Scale-Out Storage Software based on Shared Nothing Model
Hardware Agnostic Racks (10…40U) x86 server (1U, 2U…4U) with CPU, RAM, Ethernet and DAS running Linux Clusters Scale-Out Storage Software based on Shared Nothing Model

11 Distributed Architecture
P2P Scality RING 1 node manages 1/36th of the key space servers (6) storage nodes (ex: 6/server, total=36) 36 storage nodes projected on a ring From Servers to Storage Nodes RING Topology, P2P Architecture Limitless Scale-Out Storage based on Shared Nothing model Fully Distributed Storage (Data and Meta-data)

12 Distributed Architecture
P2P DHT & Consistent Hashing 360° key space, each node manages a key range Key generation and projection (no overhead at running time) MIT CHORD Inter-routing Algo. + 3 Scality methods (Proxy, Balance & Rebuild) 1/6 key space to rebalance when a storage node is lost No Single Point of Failure, No central Catalog or DB Stateless architecture Easy to grow, high resistance and seamless to failures Self Healing and Organic Upgrade Elastic Clustering Inherent Load balancing & Symmetry Every node can serve a request Maximum # of hops 100 nodes 3 1 000 nodes 5 nodes 7 Topology is cashed at connector level.

13 End-to-End Parallelism
APPLICATIONS / CONNECTORS APPLICATIONS / CONNECTORS APPLICATIONS / CONNECTORS APPLICATIONS / CONNECTORS APPLICATIONS / CONNECTORS Parallel Connectors access to Storage Nodes Performance aggregation Redundant Data Path Multiple Storage Nodes per server Minimum 6 to increase parallelism and data independence Fast and easy rebuild Multiple IO Daemons per Server Control Physical and Boost IO STORAGE NODES STORAGE NODES STORAGE NODES STORAGE NODES I/O DAEMONS I/O DAEMONS I/O DAEMONS I/O DAEMONS I/O DAEMONS I/O DAEMONS SSD SATA TIERED STORAGE Scality Parallelism Factor #Storage Nodes x #IO Daemons vs Simple server node with only 1 IO Engine Independent Performance and Capacity Scalability

14 Object and Key/Values Stores
DATA MD Object as a opaque entity with Metadata and Data No limit for object size and number Random block R/W access on large objects Object Chunking, Splitting, Striping and Streaming 160 bit Key Space with 128 bit Object key Store node acting as Simple and High Performance Key/Value stores Fully Autonomous Replica Class 128 bits 160 bits 24 bits Dispersion Payload Class of Service 8 bits

15 Internal Distributed DB
Scality MESA NewSQL Philosophy Distributed Tables across Storage Nodes Multiple Indexes per table Schemas updates/Flexible schemas 100% ACID 100% Elastic Linear Transactional Performance Automatic Fault Tolerance SQL front-end (not exposed) Main DEFS Properties/Name Mapping Table Key Page -> Page Page # Value

16 Data Replication No data transformation Class of Storage Rack-aware
00 20 40 60 80 100 11 120 140 160 180 200 220 91 171 No data transformation Clear/Native data format Very fast access Simple projection Class of Storage Up to 5 replicas (6 copies) Rack-aware Guarantee of full independent object location Self-healing Balance misplaced objects Transparently proxy misplaced objects Rebuild missing replicas Permanent CRC of all contents (no silent data corruption)

17 ARC: Scality’s Erasure Coding
RAID 5 or 6 can’t scale without risk With Bit Error Rate vendors information (1 bit error among 1014 bits) 55% of probability to receive an error for a volume of 10TB 99.9% for 100TB !! (worst beyond) Replication is a great method but too costly for large storage environment Solution: Scality Advanced Resilience Configuration (ARC) Available as a standard RING Core feature (no extra cost) Configuration option, running within the RING Leveraging Reed Solomon technology

18 Better Durability & Reduced Storage overhead than Replication
Scality ARC ARC Fully configurable, by default ARC(14,4) meaning… 14 data fragments + 4 checksums fragments so max. loss of 4 elements in a transaction (18 total fragments committed on disk) Read() is satisfied with minimum of 14 fragments: (14,0), (13,1), …, (10,4) Data fragments = Native Data (no transformation) Direct and fast read access Scality ARC(14,4) Data Check sums Data inputs Data source Output to be stored Scality RING 4 14 HW overhead ratio = 1.3 (14+4/14) (n,k) configurable to accept different tolerance profile for drive, node and site failure Arguments against: key difference vs Cleversafe and Amplidata (only equations stored, need to decode at every read – very CPU intensive and latency added) Better Durability & Reduced Storage overhead than Replication

19 Scality ARC Comparison
Replication A B Storage space x3 Multiple copies Direct access Reduced storage space Redundant information Direct access Scenario #1: loss of A read(A+B) - read(B) = A Scenario #2: loss of B read(A+B) - read(A) = B Scality ARC A B A+B Dispersed A-B Reduced storage space Scrambled data Latency with additional computation for each read (decoding phase) A+B A*B

20 Geo Redundancy Business Continuity with “true %” availability including maintenance Multi-site topology with Scality RING (up to 6 sites) Replication or Geo Erasure Coding implementation – Synchronous Or Multi-RINGs on Multi-site (independent topology) – Asynchronous Asynchronous multiple independent RINGs Synchronous stretched RING across 2 sites

21 20% 80% Auto Tiering Tiering like HSM 20/80 approach Criteria
Primary Site Site A Tiering like HSM Fully automated and transparent for application 20/80 approach Perfect for Hot data on SSD/Flash and Cold data on SATA Criteria Age, Access Time & Object size Potentially different data protection mechanisms Replication with 3 copies on Tier 1 Erasure Code – Scality ARC – on Tier 2 Data Migration across RINGs within same site or across sites 80% Tier 2 Secondary Site Site B

22 Access Methods and Standards
S3 CDMI Open Cloud Access Strategy Scality RING as the storage backend File and Object Access Methods Local and Remote Global Namespace Scality RS2 API (HTTP/REST) Compatible with Amazon S3 Full CDMI server implementation By object ID and hierarchical path- based namespace Cloud Files as the OpenStack Swift protocol S3 CDMI Internet NFS NFS NFS SOFS SOFS Scality RING

23 Management Supervisor RingSH Central management platform
Monitor application connectors & storage nodes down to individual disk drives Passive component Detailed Activity Log RingSH Command line interface Easy to integrate or script with Manage platform, store/retrieve/delete objects List all keys

24 « Exceptional Performance »
“ESG Lab verified exceptional performance for an object-based storage solution, which rivals block-based solutions. Aggregate throughput scaled linearly as nodes were added to a RING. Response times improved as the RING grew in size, allowing for predictability when deploying a RING.” Content Delivery Object Type Simultaneously Sustained Objects Delivered 5 servers with Intel SSD Internet Audio (MP3) 211,424 Internet Image (JPG) 135,311 Internet Video (MPEG) 90,208 CD Audio (ISO) 18,877 Broadcast TV (HD) 2,298

25 Scality RING – The BIG picture

26 Scality Solutions Email File Storage StaaS Digital Media Big Data2
Scality RING Core P2P, Replication, Erasure Code, Tiering, API/Interface (HTTP/REST with Scality SRWS and RS2 Light, CDMI by Obj. ID), Management Full platform integration Certified with Openwave Messaging, Critical Path, Cyrus, Dovecot, VMware Zimbra, Open-Xchange File Storage Scale-Out File System for Linux NFS/CIFS CDMI by Path Global Namespace GeoSync StaaS Amazon S3 API Authentication, Metering Option Multi-Geo Sync-n-Share1 Digital Media Optimized SOFS for CDN Origin server Big Data2 Hadoop integration with in-place data processing Scality SOFS instead of HDFS Open Cloud Access S3 & CDMI API 1: Product available separately / 2: Available Q2/2013

27 Scality Ecosystem and Partnership
File Sync & Share Backup & Archive Gateway Hardware vendor strategic partnerships

28 Scality Product availability
Software only Hardware agnostic RING Core + Solutions Pack Multi-Geo option Price per usable storage capacity “Appliance” based on MIS (Duo or Solo server) RING Core + StaaS Pack Multi-Geo option Price per enclosure (4U) Highest storage density on the market (72 x 4TB/4U) Scality RING Organic Storage 4.0

29 Release Calendar & Roadmap
H1/2014 TBC RING 5.0 “Universal Data Platform” - Block interface - Policy Engine & Data Placement June 2012 RING 4.0 - ARC (Erasure Code) SOFS 1.0 (FUSE) Supervisor (Keys, Agent, SNMP) 2012 | 2013 | 2014 Q4 2013 RING 4.3 OCA Phase 3 (Metering/Multi-Tenant + Swift + S3) Act./Act. Geo RS2 Q2 2013 RING 4.2 SOFS 2.0 (NFS, CIFS) + GeoSync Hadoop (HDFS replacement, computing on nodes) Multipart Upload Q1 2013 RING 4.1 Digital Media Open Cloud Access (CDMI & SOFS) SOFS GeoSync Supervisor (usage) OpenStack integration (EBS with Cinder) Feb. 2013

30 Scality Technology Milestones
wth Replication and Tiering for primary storage (HTTP, S3) Blob Store Authentication, Metering, Security for S3 Policy data placement, data protection Erasure Code data protection CIFS based on Samba 4 Hadoop for Big Data Cloud Files Swift Protocol 2010 | 2011 | 2012 | Q2/2013 | Q4/2013 CDMI NFS Distributed Database File Sync-and-Share OEM Nomadesk Scale Out File System for Linux SQL front-end for Distributed DB Block Storage (to add storage to VMs) with OpenStack integration via Cinder Multi-Tenancy, QoS Mgmt (IOPS, Throughput, Capacity) & Snapshot Mature Ready Beta Alpha Prototype Design

31 Scality Technical Vision
Compute Media HPC StaaS Note: Underlined items are roadmap Tenant A Tenant B Tenant C Tenant D Unified Storage Interface (File, Block, Object and HDFS Local and Remote) Scale Out Access Layer Metering Auth. Real-time Policy & QoS Big Data Analytics Content Indexing Security Storage Back-end Local, Distant or Multi-Geo RINGs High Perf Transactional Standard Storage 1 year Long-Term Storage 100 years Statistics SSD SATA Erasure Code SATA Discovery Customer IT Infrastructure

32 Unified Scale-Out Storage Software
Conclusion Technology Leader Fully Autonomous Software only solution with oem agreements High Performance and strong Security Proven deployed solution Cost efficiency and rapid ROI 100% Data Availability and Durability Unified Scale-Out Storage Software


Download ppt "Scality RING Organic Storage 4 Jan. 2013"

Similar presentations

Ads by Google