1. File & Object Data Management
Optimising file and object storage in a software defined environment
Ian Hancock
IBM UK Technical Sales

2. Abstract
This presentation focuses on how files and objects can be stored and managed in an optimised fashion within a software defined environment. The latest technologies and the integration points which bring new architectural possibilities for hybrid cloud deployments will be explored.

3. Topics Growth Issues Data Management at scale Data types
File vs Object
New solutions for growth
File Storage Technology
Object Storage Technology
The Software Defined Storage Environment
Summary

4. Today, data is everything
© 2016 IBM Corporation
Page 4

5. With Growth Comes Complexity of Management
332%
90%
growth in mobile data traffic between 2015 and 20181
of total mobile data traffic will be cloud apps by 20192
10x
80%
growth of the amount of data on the planet by 20203
of all data is unstructured (web, social, video, audio, pictures, scans, )4
1. Extrapolated from Gartner press release, Cisco Visual Networking Index: Global Mobile Data Traffic Forecast Update, 
3. IDC annual Digital Universe study,
4. IBM data.

6. Web Scale Data Growth Issues
Objects 1000x larger
HD videos/images/audio
Genomic/seismic data
Internet of Things/social media
Large and growing storage requirement
Up to exabytes with always-on availability and zero-touch, carrier-grade security
Legacy storage won’t scale and is less cost-effective
Traditional storage requires copies, replication, mirroring and disaster recovery (DR) to protect data—Cleversafe can help eliminate those requirements, and can help lower both cost and complexity
Can also reduce requirements for power, cooling and management for lower TCO
Cleversafe provides web-scale performance/capacity at any time with zero downtime operations
Multiple Interface: customer deployment flexibility
Industry standard object API compatibility

7. Compute Systems & Workloads
Benefit from simplified infrastructure
Require cost efficiency through improved virtualization and automation
Drive controlled data growth
Systems of Record
Traditional Workloads
Transactional Systems
, Supply Chain, HR
Virtual Servers and Desktops
Integrated Approach
New Workloads
Social and Media
Mobile Applications
Big Data & Analytics
Systems of Engagement
Require massive scale and rapid pace
Accelerate business insights
Rely on data elasticity, supporting diverse hardware

8. Data Growth 60–80% per year Unstructured Data Structured Data
Problem - Traditional and Legacy Storage Designed for Transactional, Not Unstructured Data
Unstructured Data
Structured Data
* Exabytes
Unstructured data growth of
60–80% per year
creates Web-scale storage needs
*1 exabyte = 1,000 petabytes =1 million terabytes = 1 billion gigabytes
Source: IDC

9. Different Storage Technology is Required
Storage Virtualization
Scalable Storage V1 V2 V3 V4 V5
... …. Vn C
New Generation Applications
Traditional Applications
Systems of Engagement
Systems of Record
Insights and engagement
Rapid pace and massive data scale
Global Elasticity
Transactions and processes
Controlled data growth
Efficiency through virtualization
Integrated Storage Management and Data Protection

10. Storage Types

11. Workload Suitability Object File Block Transaction units Protocols
Objects: files with custom metadata
Files
Blocks
Protocols
REST and SOAP over HTTP
CIFS and NFS
Fibre Channel, iSCSI, SATA
Metadata support
Support of custom metadata
Fixed File-system attributes
Fixed system attributes
Best suited for
Active archive and content repository
Shared file data
Transactional data and frequently changing data
Biggest strength
Scalability and distributed access
Simplified access and management of shares files
High performance for transactional data
Supported updates
No in-place update support; updates create new object version
Supports in-place updates
Limitations
Not designed for frequently changing transactional data
Limited metadata and scale issues beyond billions of inodes
Difficult to extend beyond the data center

12. File Storage At Scale

13. Private, Public or Hybrid Cloud
FlashSystem
Any Storage
Private, Public or Hybrid Cloud
Control
Virtualize
Accelerate
Scale
Family of Storage Management and Optimization Software
Protect
Archive
Family of Software-Defined-Storage
Common management and user experience
Deployment flexibility as software, appliance or cloud service
High-performance, enterprise storage
Securely manage the explosive growth of data
Unified analytics-driven management for any storage
Advanced data placement to maximize performance, availability, and cost efficiency
Built upon proven, flexible and scalable solutions from IBM

14. IBM Spectrum Scale for common workloads
Big Data Analytics
Archive and analyze in place
Hadoop Transparency
Content Repository
Seamless growth
Unified file and object
Private Cloud
Data management at scale
Integrated with OpenStack
Compute Clusters
Scalable performance & throughput
Advanced routing and caching
Footnote goes here

15. Reduce Complexity. Redefining Unified Storage
SSD
Fast Disk
Slow Disk
Tape
Spectrum Scale
NFS
SMB
POSIX
Swift/S3
HDFS
Challenge
Managing data growth
Lowering data costs
Managing data retrieval & app support
Protecting business data
Unified Scale-out Data Lake
File In/Out, Object In/Out; Analytics on demand.
High-performance native protocols
Single Management Plane
Cluster replication & global namespace
Enterprise storage features across file, object & HDFS
Footnote goes here

16. Spectrum Scale: Redefining Unified Storage
Users and applications
Client workstations
Traditional applications
Compute farm
Powered by
Single name space
Spectrum Scale
SMB
NFS
POSIX
Transparent HDFS
Disk
Tape
Shared Nothing Cluster
Flash
Off Premise
OpenStack
Object
Swift S3
Cinder
Glance
Manilla

17. IBM Spectrum Scale: Store Everywhere. Run Anywhere.
Ideal for high performance applications and transparent tiered storage
IBM Spectrum Scale enables compute clusters, global storage cloud and big data analytics
Low-latency and high-performance using parallel data access that excels on massive data sets
Global shared file system on scalable, distributed storage infrastructure to multi-PB scale
Supports mixed workloads, i.e. any combination of traditional workloads like DB, , and SAP as well as new generation workloads like cloud apps, Hadoop and Spark
Any combination of storage devices - Flash, Disk or Tape as well as 3rd party cloud storage pools to create tiering
Automatic movement of data based on policy to optimize for performance or cost (Cognitive ILM)
Unified storage support including, file, object, HDFS and OpenStack
Global name space
CIO
Finance
Engineering
IBM Spectrum Scale
IBM Spectrum Archive
Flash
Gold Pool
Disk
Silver Pool
Tape
LTFS or
Tier 1
Tier 2
Tier 3 1 2

18. Ideal for Cold Archives and Large Files
IBM Spectrum Scale + IBM Spectrum Archive Keeping data online at the lowest cost
Ideal for Cold Archives and Large Files
IBM Spectrum Archive lowers storage cost by leveraging low cost filesystem based storage on tape
Easy: Data still listed in directories (self describing storage)
Cognitive: Once data is accessed it is moved to disk
Transparent: Other than longer access times, users have no idea data is stored on LTFS
Did you know?
Modern tape can deliver data rates of 360 MB/second and 10TB in capacity
LTO-7 has a bit error rate of 10x191 which is better than SATA hard drives at 10x142
LTO-7 acquisition cost is around 1¢ per GB
Tape only uses power when data is being accessed, making it the most energy efficient media on the market
Global name space
CIO
Finance
Engineering
IBM Spectrum Scale
IBM Spectrum Archive
Flash
Gold Pool
Disk
Silver Pool
Tape
LTFS
Tier 1
Tier 2
Tier 3 1 2

19. IBM Elastic Storage Server (ESS) Integrated scale out data management for file and object data
Optimal building block for high-performance, scalable, reliable enterprise storage
Faster data access with choice to scale-up or out
Easy to deploy clusters with unified system GUI
Simplified storage administration with IBM Spectrum Control integration
One solution for all your data needs
Single repository of data with unified file and object support
Anywhere access with multi-protocol support: NFS 4.0, SMB, OpenStack Swift, Cinder, and Manila
Ideal for Big Data Analytics with full Hadoop transparency with 4.2
Ready for business critical data
Disaster recovery with synchronous or asynchronous replication
Ensure reliability and fast rebuild times using Spectrum Scale RAID’s dispersed data and erasure code

20. Advantages of Spectrum Scale RAID
Use of standard and inexpensive disk drives
Erasure Code software implemented in Spectrum Scale
Faster rebuild times
More disks are involved during rebuild
Approx. 3.5 times faster than RAID-5
Minimal impact of rebuild on system performance
Rebuild is done by many disks
Rebuilds can be deferred with sufficient protection
Better fault tolerance
End to end checksum
Much higher mean-time-to-data-loss (MTTDL)
8+2P: ~ 200 Years
8+3P: ~ 200 Million Years
JBODs
Elastic Storage Server
Spectrum Scale RAID

21. Spectrum Scale De-Clustered RAID
Conventional RAID: Narrow data+parity arrays
Rebuild can only use the IO capacity of 4 (surviving) disks
20 disks (5 disks per 4 conventional RAID arrays)
Striping across all arrays, all file accesses are throttled by array 2’s rebuild overhead.
4x4 RAID stripes (data plus parity)
Failed Disk
Declustered RAID: Data+parity distributed over all disks
Rebuild can use the IO capacity of all 19 (surviving) disks
20 disks in 1 Declustered RAID array
Load on files accesses are reduced by 4.8x (=19/4) during array rebuild.
16 RAID stripes (data plus parity)
Failed Disk

22. IBM Spectrum Protect with IBM Spectrum Scale
“Win the backup race“ every night using the high- performance parallel file system
Scale capacity seamlessly and transparently to apps and users under the shared file system global namespace
Automated fail-over for high availability
Storage compression & replication options
Deploy using IBM Spectrum Protect Blueprints with ESS for ease and performance sizing.
Backup clients
Consolidate
w/ performance
Spectrum Protect instances
IBM Spectrum Scale
shared file system
Storage

23. Object Storage At Scale

24. Cleversafe: Easy to deploy, Petabyte to Exabyte Scale Cloud Storage
Ideal for Active Archives and Cloud storage
The leader in Scale-out object storage
No downtime for software updates, hardware service, hardware refresh, or expansion
Easy to manage/ easy to grow: No RAID or replication
Rapid deployment with minimal configuration required
Rich management capability via UI and API
Site fault-tolerant with geographic dispersal: No loss of functionality when deployed across 3+ sites.
System fault tolerant in 1 & 2 site deployments
Lower TCO with fewer physical storage needs than RAID/replication based approaches
Less HW to purchase
Less space, power, and cooling
Secure data with built in encryption of data at rest and TLS/SSL support to protect data in motion
567 TB Raw
RAID 6
+ Replication
Cleversafe®
1 PB
3.6 PB
900
3.6x
3 FTE
Replication/backup
Usable Storage
Raw Storage
4TB Disks
Racks Required
Floor Space
Ops Staffing
Extra Software
1.7 PB
432
1.7x
.5 FTE
None

25. IBM Cleversafe Efficiency
How to build a highly reliable storage system for 1 Petabyte of usable data?
RAID 6 + Replication
Cleversafe®
Onsite mirror 1.20 PB Raw
Original 1.20 PB Raw
1 PB
1.7 PB
432
1.7x
.5 FTE
None
567 TB Raw
Remote copy
1.20 PB Raw
1 PB
3.6 PB
900
3.6x
3 FTE
Replication/backup
Usable Storage
Raw Storage
4TB Disks
Racks Required
Floor Space
Ops Staffing
Extra Software $
70% +
TCO Savings

26. IBM Cleversafe Multi-site Options for Object Storage
On-Premise
Single tenant (Regulatory)
Design specific to needs
Total control of system
Local to on-premise compute
Dedicated
No datacenter space required
Single tenant (Regulatory)
Flexible configuration options
OPEX vs CAPEX
Public
Usage-based pricing
Data local to in-cloud compute
Elastic capacity
No datacenter space required
Immediate worldwide footprint
Fully managed
OPEX vs CAPEX
IBM managed options provide full management, monthly billing
Hybrid
Same as on-premise plus the following:
Economic benefits of more dispersed sites (i.e., 3 rather than 2)
On-premise storage replicated to the cloud
Ability to add capacity to an on-premise deployment when there is no more data center space available

27. Workload Comparison IBM Spectrum Scale - File & Object
Ideal Workloads
Big Data Analytics
HPC (Engineering Applications)
Performance optimized Backup and Restore
Multi-Site file collaboration
File Synch and Share (multi tier)
Cold data archive (scale + tape)
Differentiation
Designed for high performance
Unified Storage Infrastructure: Native File, Object & Hadoop
Robust Tiering with policy based data placement and data movement
Multi site collaboration with advanced routing and caching
Enterprise Features, e.g. Encryption, compression, QoS, & Disaster Recovery
IBM Cleversafe – Object only
Ideal Workloads
Active Archive (warm data, mostly static)
Cost optimized Cloud backup target
Web app content
Remote office storage consolidation
Storage as a service
Differentiation
Designed for easy deployment and management at scale
Always-on architecture
Geo-dispersed erasure coding for site fault tolerance and DR
Simple keyless native encryption and multi-tenant security
Reduced cost and complexity

28. How IBM Cleversafe Works
CONTENT TRANSFORMATION
Cleversafe software encrypts, slices and applies Information Dispersal Algorithms otherwise known as erasure coding policies to the data.
Data Ingest
Accesser Software
Storage Nodes
Site 1
Site 2
Site 3
Physical Distribution
Slices are distributed to separate disks and industry standard x86 hardware across geographic locations.
Data Retrieval
Reliable Retrieval
An operator defined subset of slices is needed to retrieve data bit perfectly in real time.
BENEFITS
The level of resiliency is fully customizable resulting in a massively reliable and efficient way to store data at scale as opposed to RAID and replication techniques.
Slicestor Software

29. Cleversafe Customer Examples
Major League Baseball Organization
Business requirement:
Update the team’s system for storing, protecting and accessing all of the video data and other information their coaches and other employees use to make critical decisions every day
Active Archive
Results:
Reduced administration time
Up-to-date data, backed up and replicated across all sites
Improved access to the data through a simple log-in
Scalability to meet future needs
Data available to coaches, players, marketing, press and minor league affiliates
IT staff free to focus on innovation
Major Japanese telecoms operator
A highly reliable and secure solution for protecting their customers’ mobile data plus a flexible, multi-tenant storage as a service offering for their enterprise customers
Backup
Solutions supports millions of mobile subscribers
Highly reliable and available solution that tolerates site outages without expensive copies of data
Savings over original storage
Flexibility to offer multiple services to their enterprise customers
Leading European Home Entertainment and Telecoms
A secure solution for their massive storage and immediate access needs with growing storage needs
Enterprise Storage-as-a-Service
Zero-touch security for all content
Lower data center costs with improved service levels
Always on availability; capacity provisioning in minutes

30. IBM Cleversafe Active Archive Example
Photo and video objects are sent to Cleversafe via REST based protocols
Users upload photo and video content via web based application
Metadata is captured and stored
Scale – 130 petabytes and growing: more than 50 Billion images stored
Security – 50,000+ uploads per minute with zero touch security
Always-on availability – SLA of 100% download on demand – even during CA to Nevada datacenter move
Manageability – 3 Administrators manage entire environment
Economics – Operating costs reduced by more than 70%
Key decision makers – Technical team backed by financial cost cutting mandates

31. IBM Spectrum Scale + Cleversafe with transparent cloud tiering: 2+2=5
IBM Spectrum Scale enables migrating files or objects from IBM Spectrum Scale to/from Cleversafe storage pools; on-premise or in the cloud
Coming Soon!
Tier 1
Global name space
IBM Spectrum Scale
CIO
Finance
Engineering
Transparent to end-users of IBM Spectrum Scale
Secure, reliable & policy-driven
& other swift/S3 supported clouds
Private Cloud & multi-site reliability
Public Cloud
On-premise

32. The Software Defined Environment

33. IBM Spectrum Storage family and Cleversafe Available as software, appliance or as a service in the cloud
Storage Management and Applications
Backup/Archive IBM Spectrum Protect
Manage & Report IBM Spectrum Control
Cloud Option Storage Insights
Storage Infrastructure optimized for the data underlying a workload –file, block or object
Scale-out File IBM Spectrum Scale
Scale-out Block IBM Spectrum Accelerate
Scale-out Object Cleversafe
Virtualized Block IBM Spectrum Virtualize
Non-IBM storage
Traditional IBM storage
High IOPS All Flash
Servers
Disk
Cloud
Tape Economics IBM Spectrum Archive
All of IBM SDS can adapt to high performance or high capacity needs by leveraging appropriate underlying storage media – Flash, NL-SAS or Tape

34. Key Values for File vs Object and Disk vs Tape
IBM Spectrum Scale
File based storage with object & HDFS support
Super High performance
Information Lifecycle Management (ILM)
Micro-seconds access time
CleverSafe
Object Based Storage
Site Fault Tolerant
Easy to Deploy
Milli-seconds access time
IBM Spectrum Archive
Tape ILM target
Lowest cost TCO
Long term retention
Minutes access time
Our portfolio works together in a variety of combinations to solve your customers most challenging storage problems.

35. Summary Applications are driving more and more unstructured data
Webscale applications are driving huge data growth
Both File and Object data storage are increasingly necessary
Old data protection technologies such as RAID are no longer adequate
Data is controlled through software and stored on commodity hardware

36. Thank You for Listening!

37. Legal notices
Copyright © 2015 by International Business Machines Corporation. All rights reserved.
No part of this document may be reproduced or transmitted in any form without written permission from IBM Corporation.
Product data has been reviewed for accuracy as of the date of initial publication. Product data is subject to change without notice. This document could include technical inaccuracies or typographical errors. IBM may make improvements and/or changes in the product(s) and/or program(s) described herein at any time without notice. Any statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only. References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business. Any reference to an IBM Program Product in this document is not intended to state or imply that only that program product may be used. Any functionally equivalent program, that does not infringe IBM's intellectually property rights, may be used instead.
THE INFORMATION PROVIDED IN THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER OR IMPLIED. IBM LY DISCLAIMS ANY WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT. IBM shall have no responsibility to update this information. IBM products are warranted, if at all, according to the terms and conditions of the agreements (e.g., IBM Customer Agreement, Statement of Limited Warranty, International Program License Agreement, etc.) under which they are provided. Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. IBM makes no representations or warranties, ed or implied, regarding non-IBM products and services.
The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents or copyrights. Inquiries regarding patent or copyright licenses should be made, in writing, to:
IBM Director of Licensing
IBM Corporation
North Castle Drive
Armonk, NY
U.S.A. 37

38. Information and trademarks
IBM, the IBM logo, ibm.com, IBM System Storage, IBM Spectrum Storage, IBM Spectrum Control, IBM Spectrum Protect, IBM Spectrum Archive, IBM Spectrum Virtualize, IBM Spectrum Scale, IBM Spectrum Accelerate, Softlayer, and XIV are trademarks of International Business Machines Corp., registered in many jurisdictions worldwide. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at
The following are trademarks or registered trademarks of other companies.
Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.
IT Infrastructure Library is a Registered Trade Mark of AXELOS Limited.
Linear Tape-Open, LTO, the LTO Logo, Ultrium, and the Ultrium logo are trademarks of HP, IBM Corp. and Quantum in the U.S. and other countries.
Intel, Intel logo, Intel Inside, Intel Inside logo, Intel Centrino, Intel Centrino logo, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries.
Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both.
Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.
Java and all Java-based trademarks and logos are trademarks or registered trademarks of Oracle and/or its affiliates.
Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States, other countries, or both and is used under license therefrom.
ITIL is a Registered Trade Mark of AXELOS Limited.
UNIX is a registered trademark of The Open Group in the United States and other countries.
* All other products may be trademarks or registered trademarks of their respective companies.
Notes:
Performance is in Internal Throughput Rate (ITR) ratio based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput that any user will experience will vary depending upon considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve throughput improvements equivalent to the performance ratios stated here.
All customer examples cited or described in this presentation are presented as illustrations of the manner in which some customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics will vary depending on individual customer configurations and conditions.
This publication was produced in the United States. IBM may not offer the products, services or features discussed in this document in other countries, and the information may be subject to change without notice. Consult your local IBM business contact for information on the product or services available in your area.
All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
Information about non-IBM products is obtained from the manufacturers of those products or their published announcements. IBM has not tested those products and cannot confirm the performance, compatibility, or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
Prices subject to change without notice. Contact your IBM representative or Business Partner for the most current pricing in your geography.
This presentation and the claims outlined in it were reviewed for compliance with US law. Adaptations of these claims for use in other geographies must be reviewed by the local country counsel for compliance with local laws. 38

39. Special notices DCP03271USEN-00
This document was developed for IBM offerings in the United States as of the date of publication. IBM may not make these offerings available in other countries, and the information is subject to change without notice. Consult your local IBM business contact for information on the IBM offerings available in your area.
Information in this document concerning non-IBM products was obtained from the suppliers of these products or other public sources. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products.
IBM may have patents or pending patent applications covering subject matter in this document. The furnishing of this document does not give you any license to these patents. Send license inquires, in writing, to IBM Director of Licensing, IBM Corporation, New Castle Drive, Armonk, NY USA.
All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
The information contained in this document has not been submitted to any formal IBM test and is provided "AS IS" with no warranties or guarantees either expressed or implied.
All examples cited or described in this document are presented as illustrations of the manner in which some IBM products can be used and the results that may be achieved. Actual environmental costs and performance characteristics will vary depending on individual client configurations and conditions.
IBM Global Financing offerings are provided through IBM Credit Corporation in the United States and other IBM subsidiaries and divisions worldwide to qualified commercial and government clients. Rates are based on a client's credit rating, financing terms, offering type, equipment type and options, and may vary by country. Other restrictions may apply. Rates and offerings are subject to change, extension or withdrawal without notice.
IBM is not responsible for printing errors in this document that result in pricing or information inaccuracies.
All prices shown are IBM's United States suggested list prices and are subject to change without notice; reseller prices may vary.
IBM hardware products are manufactured from new parts, or new and serviceable used parts. Regardless, our warranty terms apply.
Any performance data contained in this document was determined in a controlled environment. Actual results may vary significantly and are dependent on many factors including system hardware configuration and software design and configuration. Some measurements quoted in this document may have been made on development-level systems. There is no guarantee these measurements will be the same on generally-available systems. Some measurements quoted in this document may have been estimated through extrapolation. Users of this document should verify the applicable data for their specific environment.
DCP03271USEN-00