Virtualization Technologies

Virtualization Technologies
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Virtualization Technologies Module 6.2

Virtualization Technologies
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Virtualization Technologies Upon completion of this module, you will be able to: Identify different virtualization technologies Describe block-level virtualization technologies and processes Describe file-level virtualization technologies and processes The objectives for this module are shown here. Please take a moment to read them. Virtualization Technologies

Lesson 1 – Virtualization – An Overview
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Lesson 1 – Virtualization – An Overview Upon completion of this lesson, you will be able to: Identify and discuss the various options for virtualization technologies The objectives for this lesson are listed here. Please take a moment to review them. Virtualization Technologies

Defining Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Defining Virtualization Virtualization provides logical views of physical resources while preserving the usage interfaces for those resources Virtualization removes physical resource limits and improves resource utilization Virtualization is not a new technology. In fact, it has existed in various forms for many years. One of the key components of virtualization is the ability to maintain the existing interfaces, which allows users to utilize current processes, technologies and systems. Virtualization allows you to overcome some of the physical limitations of technology. In certain situations, applications cannot share resources (server, network, storage) because of their requirements (compatibility, security, I/O, etc.). Virtualization allows administrators to virtually separate these applications without the need to purchase numerous instances of redundant equipment. Virtualization Technologies

What Makes Virtualization Interesting
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. What Makes Virtualization Interesting Potential Benefits: Higher rates of usage Simplified management Platform independence More flexibility Lower total cost of ownership Better availability Server Storage network Storage Virtualization technology provides a number of benefits at various levels of technology within an infrastructure. Virtualization allows more applications to share the same underlying physical technology, reducing hardware and operating costs, simplifying management of independent systems, and allowing the physical hardware to have higher rates of utilization. Availability and flexibility are achieved by being able to move virtual solutions across physical hardware; therefore reducing performance issues and unplanned downtime from faults. Virtualization Technologies

Virtualization Comes in Many Forms
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Virtualization Comes in Many Forms Each application sees its own logical memory, independent of physical memory Virtual Memory Each application sees its own logical network, independent of physical network Virtual Networks Each application sees its own logical server, independent of physical servers Virtual Servers As previously mentioned, virtualization has been in use for many years. Here are some examples of virtualization: Virtual memory has been used in the mainframe environment for more than 40 years and nearly 20 years in the open systems world. Virtual networks are VLANs (Virtual Local Area Networks), which provide an easy, flexible, less-costly way to manage logical groups in changing environments. VLANs make large networks more manageable by allowing centralized configuration of devices located in physically diverse locations. Virtual servers: Again, the mainframe environment has been using virtual servers for approximately 30 years. In the UNIX environment, LPARs (Logical PARtitions), have followed suit and provided virtual server capabilities. Products such as VMWare provide the same functionality for the Intel platforms. Virtual storage includes integrated cache-disk array, array-based logical-volume management, and integrated RAID. These technologies have been in place since the early 1990s. Each application sees its own logical storage, independent of physical storage Virtual Storage Virtualization Technologies 6

Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Virtualization Comes in Many Forms Each application sees its own logical memory, independent of physical memory Virtual Memory Physical memory Swap space App When computers were originally built, they were programmed based on physical memory. Memory was also expensive and scarce, because it tended to be hand-assembled. All of these factors led to computers being used for a single task - one application at a time. When they were out of memory, they were typically out of memory. Since then, memory technology has changed and prices have decreased. Virtual memory managers, part of the core operating system, emerged and computers can now utilize larger and larger amounts of memory. By allowing computers to access these extended quantities of memory, and by managing the usage of address spaces within that memory, multiple applications can now be run in parallel. Another important concept in virtual memory is a swap file. Even in today’s world, sometimes there is not enough physical memory in a computer to handle all of the tasks that it needs to process. A swap file (page file, swap space, etc.) is a portion of physical disk that is made to look like physical memory to the operating system. Because disk access is measured in milliseconds, whereas physical RAM (Random Access Memory) is measure in nanoseconds, swap space is orders of magnitude slower than physical memory. As a result, the operating system typically moves the least used data into swap so that RAM can be accessed by more active processes. Benefits of Virtual Memory Remove physical-memory limits Run multiple applications at once Virtualization Technologies 7

Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Virtualization Comes in Many Forms Each application sees its own logical network, independent of physical network Virtual Networks VLAN A VLAN B VLAN C VLAN trunk Switch Benefits of Virtual Networks Common network links with access-control properties of separate links Manage logical networks instead of physical networks Virtual SANs provide similar benefits for storage-area networks Virtual Local Area Networks (VLANs) allow an administrator to logically partition a physical network. Consider a geographically disparate company that has users from the same department scattered throughout the world. These users access the same resources, which are centrally located in one office. In a standard network implementation, these locations would most likely be separated into separate networks, connected together by routers. As previously seen, when network packets cross routers, latency is introduced and performance decreases. By using VLAN technology, these users with similar access requirements can be grouped together onto the same network, and thereby eliminate the need for network routing to occur. So, while they are physically located around the world, network-wise they appear to be all in the same location. In addition to improving network performance, VLANs can provide a means for security by isolating resources from unwanted access. They allow sensitive data to be kept isolated from other networks, and can restrict what computers have access to resources located within them. A more recent evolution of the VLAN is the Virtual SAN/Virtual Fabric. While the individual technologies vary from one manufacturer to another, conceptually they all function in the same way as a VLAN. Virtualization Technologies 8

Server-Virtualization Basics
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Server-Virtualization Basics Before Server Virtualization: Operating system Application Single operating system image per machine Software and hardware tightly coupled Running multiple applications on same machine often creates conflict Underutilized resources After Server Virtualization: Virtual Machines (VMs) break dependencies between operating system and hardware Manage operating system and application as single unit by encapsulating them into VMs Strong fault and security isolation Hardware-independent: They can be provisioned anywhere Virtualization layer Operating system App Server virtualization addresses a number of issues that exist in a physical server environment. One of the most prevalent was the need to isolate applications because of conflicts or requirements. In many cases, servers became stove-piped because application x had a file or resource conflict with application y. This problem became compounded with the requirements for high availability, two servers becomes four, four becomes eight, etc. Also, many applications do not take full advantage of today’s hardware capabilities, and resources go underutilized; processors, memory, storage, etc. As this problem grows, the requirements become staggering; space, power, maintenance, etc. Virtual servers provide a layer of abstraction between the operating system and the underlying hardware. Within a physical server, any number of virtual servers can be established, up to the hardware capabilities. Each virtual server looks like a physical machine to the operating system, but they all share the underlying physical hardware (in an isolated manner - e.g., physical memory is shared, but address space is not shared between virtual servers). Individual virtual servers can be restarted, upgraded, or even crash, without affecting the other virtual servers on the same host. While the operating system maintenance (patches, updates, etc.) is not affected by this, as they must all be maintained as normal, hardware-related issues are reduced. Instead of requiring 12 small servers to run a handful of applications, two large servers may be sufficient and provide high availability as well. In some cases, virtual machines can be moved from one piece of hardware to another, facilitating maintenance or recovery from a failure. Virtualization Technologies

Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved.
 Check Your Knowledge Define Virtualization What type of virtualization has existed for many years at the storage layer? What is a VSAN? Explain the concept of a swap file Explain the concept of server virtualization Virtualization provides logical views of physical resources while preserving the usage interfaces for those resources. Arrays have supported cache and LUNs, which make the underlying physical drives transparent. A virtual storage array network has the ability to segment a physical SAN fabric into multiple virtual fabrics. There is a portion of physical disk space that is utilized by the operating system as if it were RAM. Server virtualization allows an administrator to run multiple instances of an operating system (or different operating systems) on a single hardware instance. This allows the ability to overcome application interactions while still fully utilizing the hardware resources. Virtualization Technologies

Lesson 2 – Storage Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Lesson 2 – Storage Virtualization Upon completion of this lesson, you will be able to: Identify and discuss the various options for virtualization technologies The objectives for this lesson are listed here. Please take a moment to review them. Virtualization Technologies

Storage Functionality Today
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Storage Functionality Today Intelligence Lives Primarily on Servers and Storage Arrays Path management Volume management Replication Server Storage network Connectivity Volume management LUNs Access control Replication RAID Cache protection In a typical storage environment, the storage network provides only the communications mechanisms between the hosts and the arrays. All of the storage management functions exist on the arrays themselves, and the hosts are responsible for any redundant pathing and volume management of the raw disks presented by the arrays. Storage Virtualization Technologies

Storage Virtualization Requires a Multi-Level Approach
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Storage Virtualization Requires a Multi-Level Approach Intelligence Should Be Placed Closest to What it Controls Application functions Server Data-access functions Storage network We can characterize virtualization as a “device” or a “box.” What belongs at the server level? This is where application functions reside. What belongs at the storage-network level? This is where data-access functions reside. And what about the storage device? This is where data-preservation functions reside. An appropriate virtualization strategy is to distribute intelligence across all three, while centralizing the management and control of the entire environment. Data-preservation functions Storage Distributed intelligence / centralized management Virtualization Technologies

SNIA Storage Virtualization Taxonomy
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. SNIA Storage Virtualization Taxonomy What is created Where it is done: Host based server Based Virtualization Network based Virtualization Storage Device Storage subsystem Virtualization Storage virtualization comes in many flavors. The Storage Networking Industry Association (SNIA) has defined a storage virtualization taxonomy, which is helpful in surveying the landscape of storage virtualization. As a systematic classification, one of the questions that this storage virtualization taxonomy addresses is the type of virtualization. Immediately, most people think of block-virtualization, an important type of storage virtualization. However, there are also file virtualization, storage device virtualization, and three types of tape virtualization (including what we call virtual tape libraries). So storage virtualization cuts a wide swath. How it is implemented In-band Virtualization Out-of-band Virtualization Virtualization Technologies

Current Storage Virtualization Examples
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Current Storage Virtualization Examples Storage-Virtualization Examples Problem: MultiPathing Software I/O path performance and availability NAS Gateway Consolidated file-based storage Mgmt Station NAS Heads LAN File Block Storage network Virtualization is an enabler. But as an enabler, it is only as important as the problem it solves. This slide illustrates two areas where virtualization has been incorporated into a solution to address its customers’ storage-related challenges. As customers move to networked storage by deploying SANs, multipathing software abstracts the physical connections between servers and storage, and to provide important features such as transparent channel failover for better availability and I/O load balancing for optimized performance. NAS Gateways provide file-based virtualization that lets users consolidate multiple general-purpose file servers into a highly scalable, highly available NAS system with advanced management and functionality features critical to supporting high service levels. A NAS Gateway also consolidates NAS and SAN into a single infrastructure in order to take advantage of networked storage across more application requirements. Solution: Virtualization Technologies

Four Challenges of Storage Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Four Challenges of Storage Virtualization Scale: Virtualization technology aggregates multiple devices— must scale in performance to support the combined environment Functionality: Virtualization technology masks existing storage functionality— must provide required functions, or enable existing functions Management: Virtualization technology introduces a new layer of management —must be integrated with existing storage-management tools Shown here are some of the challenges of storage virtualization. Support: Virtualization technology adds new complexity into the storage network—requires vendors to perform additional interoperability tests Virtualization Technologies

The Scaling Challenge Standard environment Each array delivers units of performance (e.g., IOPS, SPEC-SFS, MB/s) 20,000 Before: Performance requirements are distributed across multiple storage arrays Application performance Replication performance Shown here is the “before” picture. This environment has no virtualization capabilities. There are several storage arrays, providing storage independently of one another. Each array can be managed independently, ensuring that each array can meet it’s requirements in terms of IOPS, capacity, etc. Virtualization Technologies

The Scaling Challenge Before: Performance requirements are distributed across multiple storage arrays Application performance Replication performance After: Storage network capabilities must support the aggregated environment Aggregate application performance Aggregate replication performance Virtualized environment Aggregated performance (e.g., IOPS, SPEC-SFS, MB/s) 100,000+ + 20,000 When virtualization technology is introduced, each storage device can no longer be viewed as an individual entity. The environment as a whole must now be analyzed. As a result, the infrastructure that is implemented both at a physical level and from a virtualization perspective must be able to adequately handle the new workload, which may consist of very different processing and traffic distribution. Greater care must be exercised to ensure that storage devices are performing to the appropriate requirements. Virtualization Technologies

The Functionality Challenge
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. The Functionality Challenge Standard environment Advanced array functionality: Mirrors, clones, and snapshots Protected and instant restores Synchronous and asynchronous replication Consistency Groups Before: Applications have access to rich array functionality Advanced local replication Advanced remote replication Array-level optimization In today’s typical environment, the storage array provides a range of advanced functionalities necessary for meeting an application’s service levels as defined by recovery-point objectives, recovery-time objectives, and uptime. There’s a wealth of rich, mature functionality in use today to achieve these service levels: local replication, extended-distance remote replication, high speed, disk-based restores, and the ability to provide application consistency across multiple volumes and arrays. Here you can see all the things that arrays are doing today; the investments made, and the dependencies in terms of protection and recoverability of information. Virtualization Technologies

The Functionality Challenge
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. The Functionality Challenge Before: Applications have access to rich array functionality Advanced local replication Advanced remote replication Array-level optimization After: Virtualization device must provide either required functionality or Specialized access to array functionality Virtualized environment Network functionality (depending on implementation) Advanced array functionality: Mirrors, clones, and snapshots Protected and instant restores Synchronous and asynchronous replication Consistency Groups ? In a virtualized environment, the virtual device must provide the same or better functionality than what’s currently on the storage array, and it must continue leveraging existing functionality on the arrays. That is, it should not keep from making use of existing investments in processes, skills, training, and people. Because storage virtualization devices are relatively new, they often have their own management functions and don’t integrate fully into existing tools. Over time, these tools will become more sophisticated, and become more transparent to an existing storage network, providing the same features and functionality that are offered on traditional storage devices. Virtualization Technologies

The Management Challenge
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. The Management Challenge Standard environment End-to-end management Before: Management tools provide integrated view of application to physical-storage mapping Monitoring and reporting Planning and provisioning A key advantage of today’s SRM (Storage Resource Management) tools is that they provide an end-to-end view that integrates everything in your environment. If you want monitoring, reporting, planning and provisioning services provided for your storage environment, efficiently and effectively, SRM is absolutely necessary. Virtualization Technologies

The Management Challenge
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. The Management Challenge Before: Management tools provide integrated view of application to physical-storage mapping Monitoring and reporting Planning and provisioning After: Storage network requires modification of management tools to support a virtualized environment Servers Networks Storage … and virtualization device Virtualized environment Server to virtualization device Virtualization device Virtualization device to physical storage When introducing a virtualization device, break the end-to-end SRM view into three distinct domains: the server to the virtualization device, the virtualization device to the physical storage, and the virtualization device itself. But none of these tools know how to work with one other. For example, if an array logs an error and sends a message for help, how will that message be picked up and reported to the other management tools sitting above the array? Virtualization Technologies

The Support Challenge Standard environment Interoperability: server types OS versions network elements storage-software products Before: Storage vendor must support complexity of multi-vendor network environments Servers and software Networks and software Arrays and software Virtualization is not a standalone technology, but something that has to work within an existing environment. This environment may be comprised of multiple vendor technology, in fact, it most likely will. Switch and storage devices from multiple vendors increases the complexity of an environment, often requiring multiple management tools and dealing with interoperability issues. Without a virtualization solution, many companies try to consolidate onto products from a single vendor, to ease these challenges. Virtualization Technologies

The Support Challenge Before: Storage vendor must support complexity of multi-vendor network environments Servers and software Networks and software Arrays and software After: Storage-virtualization vendor must provide additional support to address increased complexity New platforms New intelligence Interaction with existing infrastructure Virtualized environment Considerations: New hardware-qualification requirements Service and support ownership Problem escalation and resolution When a virtualization solution is introduced, the desire to standardize onto a single vendor is reduced or removed. In fact, one of the main advantages of a virtualization solution is to be able to utilize heterogeneous storage technologies. The virtualization device is a new platform with new intelligence, and it has to interact with everything already available. Who is responsible for new hardware-qualification requirements? How do any issues or problems get escalated and resolved? Who is responsible for service and support ownership? The answer: the virtualization provider. More complexity requires additional interoperability investments Virtualization Technologies

Virtual Storage Storage Virtualization: Block and File Level Storage network IP network Benefits of Virtual Storage Nondisruptive data migrations Access files while migrating Increase storage utilization Virtual storage is about applying the same idea as server virtualization; the separation of logical and physical resources, but at the storage level. This can be done with both SAN storage and NAS storage, virtualizing at both the block and the file level. The benefits here include nondisruptive data migrations, access to files and storage while migrations are in progress, and increased storage utilization. Each application sees its own logical storage, independent of physical storage Virtual Storage Virtualization Technologies 25

Comparison of Virtualization Architectures
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Comparison of Virtualization Architectures No state / no cache I/O at wire speed Full-fabric bandwidth High availability High scalability Value-add functionality Out-of-Band In-Band State / cache I/O latency Limited fabric ports More suited for static environments or environments with less growth Value-replace functionality Storage virtualization typically comes in two configurations: In-Band and Out-of-Band. In an Out-of-Band configuration, the virtualized environment configuration is stored externally to the data path and typically allows the environment to process data at line speed, with only minimal latency added for translation of the virtual configuration to the physical storage. Data is not cached, beyond what would normally occur in a typical SAN configuration. Because the switches are hardware based and optimized for fiber channel communications, they can be scaled significantly. Also, because only the data is included in the solution, many of the existing array features and functions can continue to be utilized in addition to new features offered by the virtualization solution. In an In-Band configuration, the configuration is placed directly in the data path. This is typically done with general-purpose servers or appliances, which function as the translation engine from the virtual configuration to the physical storage. Because this is typically a software implementation, the performance levels are not as high as an out-of-band solution and there is generally additional latency injected into the environment. Data is often cached on the device and then forwarded as processing cycles allow. This results in state occurring with data existing in the network for some time before being committed to disk, as opposed to an out-of-band solution that writes to disk after passing through the virtualization device. Virtualization Technologies

 Check Your Knowledge What are the four challenges of storage virtualization? At which level(s) is storage virtualization implemented? Control data in the data path is a feature of what type of virtualization architecture? Scale, Functionality, Management, Support Host, storage network, storage In-band Virtualization Technologies

Lesson 3 – Block-Level Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Lesson 3 – Block-Level Virtualization Upon completion of this lesson, you will be able to: Describe Block-Level Virtualization technologies and functionality The objectives for this lesson are listed here. Please take a moment to review them. Virtualization Technologies

Block-Level Storage Virtualization Basics
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Block-Level Storage Virtualization Basics Ties together multiple independent storage arrays Presented to host as a single storage device Mapping used to redirect I/O on this device to underlying physical arrays Deployed in a SAN environment Nondisruptive data mobility and data migration Multi-vendor storage arrays Storage-area network (SAN) Block-level virtualization provides a translation layer in the SAN between the hosts and the storage arrays. Instead of being directed to LUNs on individual storage arrays, hosts are directed to virtualized LUNs on the virtualization device. The virtualization device performs translations between the virtual LUNs and the physical LUNs on the individual arrays. In this manner, arrays from different vendors can be simultaneously utilized without having to deal with the interoperability issues, such as software incompatibles, LUN masking requirements, etc. As far as the host is concerned, all of the arrays look like a single target device and LUNs can be distributed, or even split, across multiple arrays. Virtualization Technologies

Usage Scenarios for Block-level Storage Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Usage Scenarios for Block-level Storage Virtualization Next-Generation Data Center Operations Heterogeneous Storage Consolidation Storage Utilization Extending Volumes Online Application Growth Business Continuity Nondisruptive Data Mobility Nondisruptive Data Mobility Scalability Block-level storage virtualization can be used for extending volumes online, resolving application-growth issues, and consolidation, to name a few uses. The next few slides focus specifically on nondisruptive data mobility. © Copyright 2006 EMC Corporation. All rights reserved. Virtualization Technologies 30

Block-Level Storage Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Block-Level Storage Virtualization Optimizes Resources and Improves Flexibility Before After Multi-vendor storage arrays Virtualization SAN SAN In addition to providing transparent volume access for heterogeneous storage arrays, block-level virtualization provides a non-disruptive data migration tool. In traditional SAN environments, when a LUN was migrated from one array to another, it would often become an offline event as the host had to be updated to reflect the new array configuration. In other instances, host CPU cycles were required to migrate the data from one array to the other, especially if migrating from one vendor to another. With a block-level virtualization solution in place, the virtualization engine handles the back-end migration of data. This allows LUNs to remain online and accessible while data is being migrated and allows the host to continue to access it while it is moved. Since the host is still pointed at the same virtual targets on the virtualization device, no changes are needed there. The only changes that need to be effected are the mappings on the virtualization device, which are dynamic. Since heterogeneous arrays can be used in a virtualized environment, it can easily facilitate an Information Lifecycle Management (ILM) strategy. By migrating older data from high-performance arrays to lower performing arrays or disks, significant cost savings can be achieved without impacting I/O. Multi-vendor storage arrays All applications have direct knowledge of storage location Simplify volume access Nondisruptive mobility Optimize resources Virtualization Technologies

Data Mobility How much data can you migrate on the weekend? Average migration rate using server-based copies is 4 GB per hour and with downtime At 48 hours of time per weekend, you can migrate 192 GB of data a week With network-based virtualization, data can be migrated at any time at much faster rates Nondisruptive data mobility matters because weekends are not enough. Performing data migrations on weekends has its limits. For server-based copy migrations, a typical rate is 4 GB per hour while the operation also suffers downtime. If you do the math, that’s 48 hours over two days, so at 4 GB/hour, you can migrate 192 GB per weekend with server-based migration. While this result depends on how many servers are moving the data, it’s not much in comparison with ever-increasing storage capacities. Data migrations can now be employed as routine steps in daily activities, rather than being put off until weekends, or your organization suffering long periods of downtime. This transforms data migration from a painful undertaking into a routine infrastructure-optimization technique that can be part of daily operations. Data mobility becomes a routine operation, making it a daily part of infrastructure optimization Virtualization Technologies

Block-Level Virtualization Example: EMC Invista
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Block-Level Virtualization Example: EMC Invista Inside the Intelligent Switch Host Mapped I/O streams Storage Mapping operation Input I/O stream Intelligent switch becomes storage target Intelligent Switches: Fibre Channel switches with custom hardware for enhanced processing Capable of performing operations on data streams at line speed Controlled by instructions from external management software (via APIs) Invista is based on intelligent switches: Fibre Channel switches with custom hardware for enhanced processing. These switches are capable of handling data operations at line speed. The switch reaches inside the I/O to examine not only what port is the I/O is being sent to, but also the logical volume and offset, all of which can be changed by the switch under control of the software. Everything starts with an input I/O stream that is sent by the host. The intelligent switch takes the I/O apart and figures out what its target is, what its logical unit number is, and the offset of the I/O within the logical unit. Then the I/O goes through a mapping operation to determine where that I/O sent by the host maps in physical storage. The resulting I/O is sent to the actual storage behind the switch. In this case, three separate arrays are shown behind the switch. All of this is transparent to the host, because the first principle of block-level storage virtualization is that the intelligent switch becomes the storage target. The result is that the host is picking up the storage resources from the intelligent switch. The host has no knowledge of the I/O-mapping operations and redirection that occur behind the scenes. Virtualization Technologies

Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Block-Level Virtualization Example: EMC Invista Inside the Intelligent Switch Host Mapped I/O streams Storage Mapping operation Input I/O stream Intelligent switch—data path Control processing Invista Control Path Cluster (CPC) Control Path Cluster—control path The CPC gets involved to make changes (e.g., allocate more storage), handle uncommon cases (e.g., SCSI inquires, SCSI reservations), perform control operations (e.g., cloning), etc. The second important principle of block-level storage-virtualization architecture is data and control separation. The previous example showed the data path - how I/O flows. Now we address the control path - how are the operations controlled? With Invista, a highly available control-path cluster becomes involved to make changes to storage allocation, issue commands for the migrations, and handle such uncommon I/O operations as SCSI inquiries. All of these are handled by software in the control path. Let’s take a look at how the control path and the data path work together to accomplish something important, using the example of data migration. Virtualization Technologies

Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Block-Level Virtualization Example: EMC Invista Inside the Intelligent Switch Host Mapped I/O streams Storage Mapping operation Mapping Operation Input I/O stream Control processing Invista Control Path Cluster (CPC) Requests received and dispatched to assigned storage location A new device is placed within the data center, and the CPC discovers the new device Here are the I/O requests coming into the picture. I/O is coming from the host on the left; it moves through the intelligent switch with its mapping process, and ultimately to the arrays on the right side. Let’s now consider what happens when a new storage array arrives. The new storage array is added and cabled up to the intelligent switch. The control path discovers that there is a new array. But, as the diagram shows, there is no I/O flowing to the new array yet. Rather, a management decision has to be made within the control path, with possible input from the Administrator, on how to put this new array into use. A management decision is made to update the mapping to determine how to put that new array into use. With the mapping done, I/O begins. Beyond bringing in a new array, perhaps there is a need to take out an old array. So the control path issues another set of instructions to move all of the storage off the yellow array and over to the light-blue array. The result is that more instructions are issued to the switch, the switch does the copying online, and the yellow array can be retired and moved out as planned. In summary, I/O is moved nondisruptively as needed, enabling you to nondisruptively incorporate new storage and nondisruptively retire old storage. The CPC creates the map and shows the host where the virtual volume exists; I/O begins Contents of an existing volume are moved online, and the volume is freed Virtualization Technologies

 Check Your Knowledge Consider that you have a leased storage array that is being replaced by a newer model. Place the relevant steps below in the correct order for data migration (hint: not all steps apply) Reconfigure LUN definitions on hosts Implement new array into existing environment Decommission legacy array Obtain new array Take host volumes offline for migration Migrate existing data from legacy array to new array Present storage on new array to virtualization engine Obtain new array Implement new array into existing environment Present storage on new array to virtualization engine Migrate existing data from legacy array to new array Decommission legacy array Virtualization Technologies

Lesson 4 – File Level Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Lesson 4 – File Level Virtualization Upon completion of this lesson, you will be able to: Describe File Level Virtualization technologies and functionality The objectives for this lesson are listed here. Please take a moment to review them. Virtualization Technologies

File-Level Virtualization Basics
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. File-Level Virtualization Basics Before File-level Virtualization: After File-level Server Virtualization: Break dependencies between end-user access and data location Storage utilization is optimized Nondisruptive migrations NAS devices/platforms IP network IP network NAS devices/platforms Before file-level virtualization, each NAS device or file server was physically and logically independent. Each host at the top knew exactly where it was getting each of its file-level resources. The result was underutilized storage resources and capacity problems, because the files were bound to a specific file server. Moving files to deal with this typically resulted in data-migration downtime, in addition to requiring host and application reconfiguration with new filenames. After file-level virtualization, the dependencies are broken between the data being accessed (file resources, in this case) and the location where the data is physically stored. This provides opportunities to optimize storage utilization and perform nondisruptive migration. These benefits should be familiar by now, as file-level virtualization has them in common with the two other virtualization technologies discussed, server virtualization and block-level storage virtualization. Every NAS device is an independent entity, physically and logically Underutilized storage resources Downtime caused by data migrations Virtualization Technologies

File-Level Storage Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. File-Level Storage Virtualization File Abstraction that Optimizes Resources and Improves Flexibility Before After Move data while writing and accessing existing data Update Global Namespace Multi-vendor NAS systems File Virtualization IP File systems Before virtualization, each of the clients shown on the left-hand side has direct knowledge of file locations, making it difficult to move the files to other NAS systems. After file virtualization is installed, the clients are insulated from the locations of the files. It becomes possible to move file systems across file servers online, and to reorganize the namespace without any disruption to the clients. While files are being moved, clients have continuous access to their files. They can read their files from the old location and write them back to the new location without realizing that the physical location has changed. File virtualization addresses this issue. Virtualizing the NAS environment allows users to access files from anywhere on the network by providing a Global Namespace. This simplifies file access while providing nondisruptive data mobility to help optimize resources. In this example, file virtualization allows files on the highly utilized NAS systems to be transparently moved to other NAS systems with lower utilization, which balances performance and optimizes the environment. In addition, multiple underused NAS systems can be consolidated into fewer NAS systems without affecting users or availability service levels. Multi-vendor NAS systems All users have direct knowledge of file locations Virtualization Technologies

Moving Files Online: A File Virtualization Example
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Moving Files Online: A File Virtualization Example DFS AD Automount NIS LDAP Global Namespace Manager NFS4 root DFS AD File Virtualization Appliance File-data migration File-data migration Global Namespace Manager Automount NIS LDAP Event Log File Virtualization inserted into I/O Client redirection Global Namespace updated Multiple clients (at the top of the diagram) are connected to multiple file servers (at the bottom). The file virtualization appliance (inside the black dotted line) performs this online movement of files. We have a file system we want to move. In this case, we want to move the blue file system off the yellow file server. File virtualization is triggered, and inserts itself into the I/O stream. This is accomplished by reconfiguring the virtual LANs to make the yellow file server visible only through the file virtualization service, which functions as a network bridge. As a result, the clients retain access to the files, but all that traffic now passes through the file virtualization system. Since file virtualization has control of this traffic, it can move the file system to its new location transparently to the clients, because the clients don’t see what is going on behind the scenes. Now the file system is in the new location, but all the I/O is still going through the file virtualization system because, at this stage, it is the only system that knows the new location of the file system. The file virtualization system proceeds to update the Global Namespace. This updates the namespaces on the clients, telling them about the new location of the blue file system. As the clients learn the new location of the blue file system, their I/O goes directly to that new location, removing the file virtualization from the I/O path. NFS4 Root NIS LDAP Admin Virtualization Technologies

Moving Files Online: A File Virtualization Example (continued)
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Moving Files Online: A File Virtualization Example (continued) DFS AD File Virtualization Appliance File-data migration Global Namespace Manager Automount NIS LDAP Event Log File virtualization inserted into I/O Client redirection Global Namespace updated Migration complete without downtime The migration is complete, without affecting clients or the system. File virtualization is now out of the I/O path and can be used for the next migration. When all the client sessions have been remapped to the new location, file virtualization device is now out of the data path. The big takeaway here is that the file virtualization device virtualizes an environment 100% of the time, with the namespace providing a logical abstraction layer. The file virtualization appliance selectively virtualizes traffic on the wire, based on the particular optimization or relocation events that need to take place. NFS4 Root NIS LDAP Admin Virtualization Technologies

Usage Scenarios for File-Level Storage Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Usage Scenarios for File-Level Storage Virtualization Next-Generation Data Center Operations Consolidation Consolidation Capacity Management Performance Management Global Namespace Management Business Continuity Tiered Storage Management This file-level storage virtualization technique has a lot of benefits: capacity management, tiered storage management, consolidation, and Global Namespace updates. Let’s talk about consolidation in more depth. © Copyright 2006 EMC Corporation. All rights reserved. Virtualization Technologies 42

Accelerated Consolidation
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Accelerated Consolidation Move Files Nondisruptively—with Continuous Access to Data Before: Too many file servers Buying more file servers for additional storage Complex migrations IP network File Virtualization After: Eliminate servers via migration to underutilized servers Maintain full read/write access during migration Transparent to clients and applications Increased utilization Eliminate file servers Average utilization Here is a typical scenario, with many file servers deployed to solve various problems. Some problems require a lot of file-server capacity, but others don’t require a dedicated file server. We’d like to consolidate them, but all of the clients know the locations of the files right now, and bringing them together may be too much trouble. With file virtualization, we can keep the clients from knowing which file server(s) houses the files. This enables file migration from File Servers 3 and 4 onto File Servers 1 and 2. File Servers 3 and 4 can be eliminated from this configuration to be used elsewhere, and File Servers 1 and 2 are brought up to a more effective level of utilization. This is all done transparently, because full file access is available during the migration. Once file virtualization is in place, it is easy to consolidate resources from aging servers or appliances without disrupting operations. Clients and applications never lose access to their resources during the move. You can scale existing servers further, or migrate to a new platform that meets more of your needs. Upgrades to existing platforms are now simple, rather than major events. Server 1 Server 2 Server 3 Server 4 Virtualization Technologies

Usage Scenarios for File-Level Storage Virtualization
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Usage Scenarios for File-Level Storage Virtualization Next-Generation Data Center Operations Consolidation Capacity Management Performance Management Global Namespace Management Global Namespace Management Business Continuity Tiered Storage Management Another important aspect of file virtualization is Global Namespace Management. © Copyright 2006 EMC Corporation. All rights reserved. Virtualization Technologies 44

Global Namespace Management
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Global Namespace Management Situation Billions of files with thousands to hundreds of thousands of clients Update namespace and retain access to files while migrating Scenario Update 1,000 client namespaces over the weekend 95% successful—50 typos or glitches 50 calls with 50 very angry employees There goes Monday...and Tuesday Wednesday: Start planning next set of changes Global Namespace Management is powerful. When there are millions of files and hundreds of thousands of clients, namespace updates can be risky. Suppose you need to update namespaces manually due to consolidation. Let’s say there are 1,000 namespaces to update over the weekend. This has been done before and will get it done before the weekend is over. But suppose there is a 95% success rate. That is good, but unfortunately for 1,000 clients, that means 50 mistakes. Fifty typos or glitches mean 50 employees who cannot access their files. That consumes Monday and Tuesday with Help Desk calls to deal with the resulting problems. On Wednesday, the planning begins for the next set of migrations, repeating the experience on the following weekend. Automating these updates and moving the files dynamically eliminates this scenario. Clients get full access to the files during migration, and weekends are reclaimed because migrations can be performed during normal working hours. Zero mistypes, 100% access during migration Virtualization Technologies 45

Simplified Namespace Management
Copyright © 2007 EMC Corporation. Do not Copy - All Rights Reserved. Simplified Namespace Management Access to Files and Folders Before: Complex file-server environments Namespace changes are time-consuming Multiple shares or mounts per client After: Multiple file systems appear as a single virtual file system via standard namespace Simplify management and ensure continuous access to files and folders Updates standard-namespace entries (UNIX, Linux, Windows) SHARE1 Windows T:\svr1\ SHARE2 NetApp S:\svr2\ SHARE3 Celerra H:\svr3\ SHARE4 UNIX G:\svr4\ In some environments, file servers have been added for so many uses that one runs out of drive letters—the X, the Y, and the Z drive are taken, and there are not any drives left. File virtualization allows transparent control of this situation by using the capabilities of the namespaces that are already in the clients. The eventual result is consolidation down to a smaller number of exported shares or file systems, each of which can be spread across file servers. Multiple file servers appear as a single virtual file system via a standard namespace. This reduction in the number of file systems (i.e., drive letters) simplifies management and gives clients an easier way to interact with the system. This is all done via standard-namespace mechanisms, such as Microsoft DFS and the UNIX Automounter, which are already in the clients. The big benefits of this deployment are: Multiple file systems appear as a single virtual file system Access to files and folders is simplified Administrators can manage the entire file-serving environment as a single entity Server 1 Server 2 Server 3 Server 4 Virtualization Technologies

 Check Your Knowledge Name two benefits of file-level virtualization What is a Global Namespace? Accelerated consolidation and Global Namespace management A universal set of file system locations/information that is maintained by a file virtualization solution and updated in many potential directories. Virtualization Technologies

Module Summary Key points covered in this module: Virtualization technologies Block-level virtualization technologies and processes File-level virtualization technologies and processes These are the key points covered in this module. Please take a moment to review them. Virtualization Technologies

Course Summary Key points covered in this course: Challenges found in today’s complex information management environment Storage technology solutions DAS NAS SAN Key business drivers for storage - Information Availability and Business Continuity Business continuity, managing and monitoring the data center, storage security, and virtualization Common storage management roles and responsibilities Key themes Technology Requirements Physical and Logical Elements Host – Interconnect – Storage Data Flow Storage Security These are the key points covered in this course. Please take a moment to review them. This concludes the training. Virtualization Technologies

Closing Slide Virtualization Technologies

Virtualization Technologies

Similar presentations

Presentation on theme: "Virtualization Technologies"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Virtualization Technologies

Similar presentations

Presentation on theme: "Virtualization Technologies"— Presentation transcript:

Similar presentations

About project

Feedback