Evolution of Flash-based storage in virtualized data centers

Evolution of Flash-based storage in virtualized data centers
Murali Vilayannur Senior Technical Director March 11th 2015 © PernixData. All rights reserved.

© PernixData. All rights reserved.
Agenda Storage stack of virtualized data centers Overview Scale and performance problems Impedance mismatch Potential solutions Flash-based storage: A panacea for all storage related problems? Challenges PernixData FVP platform © PernixData. All rights reserved.

Overview of vSphere storage stack
© PernixData. All rights reserved.

Typical vSphere storage architecture

VMFS VMware VMFS (Virtual Machine File System) Cluster file system Multiple ESXi servers can read and write to the same file-system simultaneously, while individual VMs are locked Block based VMFS 5 current implementation Shared up to 64 ESXi Servers Max volume size 64 TB Powered on VMs per VMFS 2048 1 MB blocksize, uses block sub allocation for small files Limit of ~130K files Per-file locking VMFS3: SCSI-2 VMFS5: Atomic Test and Set, part of VAAI (vSphere Storage API for Array Integration) © PernixData. All rights reserved.

NFS NFS (Network File System) Support introduced in ESX version 3. Support NFS v3 Support NFS v4.1 announced in vSphere 6 Parallel NFS is not supported ESXi Max configurations NFS mounts per host 256 No “datastore limit” 62 TB file size limit VM uses SCSI adapter VMkernel translate SCSI cmds into NFS cmds ESXi host runs NFS client in Hypervisor © PernixData. All rights reserved.

Storage stack problems

Scale issues Cluster size 32 ESXi hosts per cluster ( hosts) 4000 virtual machines per cluster Max 512 VMs per host Shared infrastructure Oversubscribed data paths. Finite compute power in storage processors to manage storage and data services. Finite cache on storage processor. Cache pollution on storage array. © PernixData. All rights reserved.

Non-deterministic performance
ESXi Hypervisor is I/O blender Context aware inside hypervisor RM: Fairness policy Exit Hypervisor = loss of application identity Non-Integrated QOS Various RM & priority algorithms throughout the stack © PernixData. All rights reserved.

Impedance mismatch Lacking application awareness Disparate stack of components No application knowledge No integration VAAI & VVOL initiatives Application needs to consume NFS/Block and needs to make it “work” Application/virtualization development cycle much faster than storage infrastructure refresh cycle. Speed mismatch between data and compute tiers. Oracle used to recommend 200MB/sec worth of data to keep a core busy. This was for their data warehousing environment. This was a decade ago. NOw the cores are even faster and multiple cores are present now that this speed mismatch is going to cause cores to idle quite a bit. © PernixData. All rights reserved.

Dinosaurs in space Our search for the Next Big Thing… Hollywood works in an interesting way, there are two classes of movies. Movies that are based on a tried and tested formula The Space themes work very well Dinosaur themes work very well Robot movies work very well The romeo and juliet theme work very well. You know the audience loves it, If you’re in action movies you are going to love the next robot movie that comes out. The producers love it, less risk for all the money putting in and it goes on and on and on. But the interesting ones, the really interesting ones are the ones that are based off an absolutely radical mindblowingly different concept Those are the ones that people talk about many many years to come. And so it turns out that the way technology works is very similar as well © PernixData. All rights reserved.

Is this the next big thing?
Flash happened! All Flash Cheaper, faster storage Hybrid Flash in legacy And its going on for a few years and it’s a great advancement in the storage technology and which we haven’t seen for decades. One fine morning you wake up and you storage substrate is ready to go order of magnitude faster then your older storage substrate. $ per IOPs may be cheaper with flash. Well what do you do with it and it turns there’s many folks thinking about and and it turns out that every thought was to make something that’s already existed, which was a storage array, faster. And by itself it’s a great thing, that is needed. But the REAL question is, is that the next big idea? Is that what Flash Tops at? Is it the Next Big Thing? Or can we do something with flash, enable something that was not previously not possible. Is this the next big thing? © PernixData. All rights reserved.

The irony of flash What happened when the processor became faster?
Did we stop at a cheaper, faster workstation? Let me paint you the same picture in a different way. And I do this because I can from VMware and spend there some of my best years of my life. The founders of VMware witnessed a very interesting change. Microprocessors went from 200 Mhz to 2 GHz, so the Microprocessor became 10x faster and the observation was that well that although the CPU became 10x faster the application cannot consume all of it all the time. Well what do you do? Consolidate a bunch of physical machines into one physical machine and that was a great start, but that’s not where it stopped. It hasn’t stopped we are currently here. Look at the change, did the change of 10x change fo the speed of the microprocessor lead to faster laptops, faster desktops? No, it lead us to make a cloud. And that is the change we are looking for! And the irony of flash. If the speed increase of 10 x on CPU leads us to virtualization, why is that the magnitude of order change in storage speeds can only do faster and cheaper storage arrays? There has to be something better! Faster CPU: Virtualization = Flash: Faster storage: ? © PernixData. All rights reserved.

Flash – Savior of our Universe When we started PernixData three years ago, we tried to see what it is what we can attack with this new found medium and what we narrow it down to was these set of points. Although flash was there, and was awesome, there was no good way to use it in a way that is applicable to all the virtualized datacenter of the world. Because if you put Flash in a storage system, it becomes an either-or problem, you have to buy that storage system to operationalize that flash technology So the question was how can we operationalize flash in every datacenter and maybe even other datacenters going forward. In a way that is not tied to the capacity problem, it’s all about speed. Not about capacity. And so it was about making this technology much more widely adoptable. There is also this thing when you operationalize this component in this thing called an array with finite amount of compute power, finite amount of network power and so. Then it turns out that the resource doesn’t quite scale because its limited by other finite parts in the box. And then of course, the fact of the matter is of course is that flash is incredibly fast, so that any amounts of moving parts that you put between the application and the resource are now going to show up and that is even worse because we live in a world where flash is not the end but the beginning. © PernixData. All rights reserved.

Distance But actually when you look at flash and where do I place the flash resource, the placement becomes context sensitive. If I put flash in an array, primarily the performance of the data services improve as the array can process metadata updates faster. Snapshots become faster, replication can become faster. Question yourself, Will this improve application performance? © PernixData. All rights reserved.

Storage systems are just not disks And then of course, the fact of the matter is of course is that flash is incredibly fast, so that any amounts of moving parts that you put between the application and the resource are now going to show up. You can view them as speed bumps along the highway...Death by a thousand cuts. Look for an image with a highway info graphic with speed bumps and a rig to explain the slowdown effect it has. © PernixData. All rights reserved.

The shared abyss Insert Flash Here Image courtesy of Chad Sakac

Flash development And then of course, the fact of the matter is of course is that flash is incredibly fast, so that any amounts of moving parts that you put between the application and the resource are now going to show up and that is even worse because we live in a world where flash is not the end but the beginning. © PernixData. All rights reserved.

Industry trends Flash-RAM convergence Flash behind HBA Flash on PCIe
Stage 1 Stage 2 Now Flash-RAM convergence Flash behind HBA Flash on PCIe Flash on DIMMs (MCS) Memory (PCM, ReRAM, MRAM) We went from a few years ago when people were making ssd devices to now where people are making pci flash devices to tomorrow where people making MCS devices, putting flash that go into memory slots and what not. Any slot in the computer seems to be fair game nowadays. So when you see this kind of evolution, when the evolution of hardware for a change almost breaks Moore law. It is now getting to a mature phase that gets you a tremendous order of magnitude of performance. That’s what we are witnessing, so when getting all the orders of magnitude performance almost every year, the real question is how do you harvest this performance in a way that is path breaking, just not in the plain old way. © PernixData. All rights reserved.

Hard problems remain unsolved Maximize flash’s reach to every data center Remove moving parts to let fast media really shine Storage performance that scales like servers Application-centric storage, not infrastructure-centric storage Is application-centric storage a storage system or a storage hypervisor? Critical piece of SDDC design How about we use this resource to do something good for the software defined datacenter? How about we take this opportunity to not just make faster and cheaper storage arrays but take this opportunity to create an application-centric storage system, as opposed to infrastructure centric storage. How about we stop talking about creating an fast storage system that provide catch-all NFS mount points or a collection fast iSCSI LUNS? How about we start talking about a faster policy-based system that exists only by the virtue of the existence of faster storage, faster media flash. © PernixData. All rights reserved.

PernixData FVP Platform

Decouple storage performance from capacity But actually when you look at flash and where do I place the flash resource, the placement becomes context sensitive. If I put flash in an array, primarily the performance of the data services improve as the array can process metadata updates faster. Snapshots become faster, replication can become faster. Question yourself, Will this improve application performance? If I put flash in the server the I/O has a shorter completion path so it will improve application performance. But should you use flash as a persistent datastore layer providing storage capacity? This behavior leads to two paradoxes Read & write acceleration of any VM workload © PernixData. All rights reserved.

PernixData FVP clustered Platform Application agnostic Flash technology agnostic Storage infrastructure agnostic FVP does that in a manner that require no change to the virtual machine, That means No drivers or changes to applications/VMs or static partitioning of server flash. It requires no change to your storage system and it requires, no new virtual appliance, no new datastores to manage. It’s the same old virtual machines that run on the same old datastores on the same old Storage Array except they are magically faster. And that magically faster applies to both read intensive virtual machines as write intensive virtual machines. © PernixData. All rights reserved.

Summary Next generation storage technologies belongs to server-tier Server-side storage intelligence abstracted away from storage systems Remove moving parts to let fast media really shine Maximize flash’s reach to every data center Short term benefits: Storage performance that scales like servers Long term benefits: Application-centric storage, not infrastructure-centric storage The net effect of this development is that you are able to get predictable and persistent microsecond storage performance. With new developments popping up in the industry every day, it is not weird to wonder when we will hit nano second latencies. When the industry is faced with the possibility of these types of speeds, we as PernixData belief that we can absolutely and fundamentally change what applications expect out of storage infrastructure. Applications used to expect to that storage platforms provided performance in the millisecond levels and use to give up improving their code as storage platforms were the bottlenecks. For the first time ever storage performance is not the bottleneck, and for the first time ever extremely fast storage is affordable with FVP and server side acceleration resources. Even an SMB-class platform can now have a million IOPS and super low latency if they want to. Now the real question for the next step becomes, if you can make a virtualized datacenter have a millions of IOPS at microsecond latency levels what would you do with that power? What new type of application will you develop; what new use cases would be possible with all that power? We at PernixData belief that if we can change the core assumption around the storage system and the way it performs, then we could see a new revolution in terms of application development and the way application actually use infrastructures. And we think that revolution is going to be very very exciting. © PernixData. All rights reserved.

References A Practical Implementation of Clustered Fault Tolerant Write Acceleration in a Virtualized Environment (FAST 2015) Virtual Machine File System (ACM SIGOPS Review) Better I/O through Byte-addressable persistent memory (SOSP 2009) Phase-change memory Resistive RAM (ReRAM) Magneto-resistive RAM (MRAM)

New Storage Technologies
Flash 10000 x faster than spindle (Mostly) consistent low latencies Flash in Storage Array Too far away from application Too many “moving parts” between flash and application Finite compute resources to utilize performance Need a new disk array to operationalize flash technology Emerging technologies Phase-change memory Memory channel storage © PernixData. All rights reserved.

vSphere Storage API for Array Integration
Storage primitives to offload storage operations to array Enable communication between ESXi Hypervisor and Array Offload Hypervisor and storage architecture Improve performance of storage intensive operations Storage cloning Storage Zeroing On disk locks (ATS) Thin Provisioning UNMAP Enable to introduce value-add into PSA stack asynchronously from hypervisor release cycle Effort on both storage vendor and VMware side Pluggable Storage Architecture in VMkernel Storage vendor update firmware to align with standard VASA framework Communicates features and functions of LUN to vSphere Feed into vSphere Storage Policy for alignment of required data services © PernixData. All rights reserved.

VMware Virtual Volumes
Application granular Data Management Context aware architecture VMDK 1st class citizen Policy driven QoS & Data Services Object based storage Solve data layout problem (logical/physical constructs) Store VMDK natively inside storage system No NFS/VMFS Redesign necessary of storage system © PernixData. All rights reserved.

Control Plane Move resources into hypervisor & Compute layer Context aware architecture Removing distance between resource and application Removal of scalability problems of infrastructure Owning complete development process and cycle © PernixData. All rights reserved.

Evolution of Flash-based storage in virtualized data centers

Similar presentations

Presentation on theme: "Evolution of Flash-based storage in virtualized data centers"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Evolution of Flash-based storage in virtualized data centers

Similar presentations

Presentation on theme: "Evolution of Flash-based storage in virtualized data centers"— Presentation transcript:

Similar presentations

About project

Feedback