Download presentation
Presentation is loading. Please wait.
Published byEmmeline McKinney Modified over 7 years ago
1
Virtual SANs (VSANs) Fabric Optimization Through Fabric Consolidation
Tom Nosella, P.Eng, B.Eng Sr. Manager – Technical Marketing Storage Technology Group CCIE #1395
2
Agenda Virtual SANs and their application How do Virtual SANs work?
3
Common Deployment Practice – SAN Islands
Today many SAN environments consist of numerous islands of connectivity Islands are physically isolated environments consisting of one or more interconnected switches Each island is typically dedicated to a single or multiple related applications Each island may be independently managed by a separate admin team Strict isolation from faults achieved through physical isolation ‘Fan-out’ SAN Islands ‘Departmental’ SAN Islands ‘Larger Core-Edge’ SAN Island
4
SAN Islands Have Purpose – At a Cost
SAN islands are built to address several technical and non-technical issues: Built to maintain isolation from fabric events or configuration errors Provide isolated and controlled management of island infrastructure Driven by bad experiences of large multi-switch fabrics However . . . Often over-provisioned port count for future growth – wasteful and costly Very widespread issue today – some analysts still recommending islands Island ‘A’ Island ‘B’ Island ‘C’
5
Introducing Virtual SANs (VSANs)
A Virtual SAN (VSAN) provides a method to allocate ports within a physical fabric to create virtual fabrics Analogous to VLANs in Ethernet Virtual fabrics created from larger cost-effective redundant physical fabric Reduces wasted ports of island approach Fabric events are isolated per VSAN – maintains isolation for HA (ie. RSCNs) Hardware-based isolation - traffic is explicitly tagged across inter-switch links with VSAN membership info Statistics can be gathered per VSAN Cisco MDS 9000 Family with VSAN Service Physical SAN islands are virtualized onto common SAN infrastructure
6
Virtualizing the Fabric – The Full Solution
To build a cost saving fabric virtualization solution, 7 key services are required: Virtual Fabric Attachment – the ability to assign virtual fabric membership at the port level Multiprotocol Extensions – the ability to extend virtual fabric service to iSCSI, FCIP, FICON, etc. Virtual Fabric Services – the ability to create fabric services per virtual fabric (routing, zones, RSCNs, QoS, etc.) Virtual Fabric Diagnostics – the ability to troubleshoot per virtual fabric problems Virtual Fabric Security – the ability to define separate security policies per virtual fabric Virtual Fabric Management – the ability to map and manage virtual fabrics independently Inter-Fabric Routing – the ability to provide connectivity across virtual fabrics – without merging the fabrics Virtual Fabric Service Model Virtualized Fabric Attachment Multiprotocol Transport Extensions Virtualized Fabric Services Virtualized Fabric Diagnostics Virtualized Fabric Security Policies Virtualized Fabric Management Inter-Virtual Fabric Routing ISL MDS 9000 Family Full Service End-to-End Virtual Fabric Implementation
7
Virtual SANs Reduce Infrastructure Costs
16 Port Switches BEFORE VSANs Virtual SANs allow dynamic provisioning and resizing of virtualized SAN islands Virtual fabrics are built to meet initial port requirements no stranded ports – need 40 ports? assign 40 ports not bound by fabric switch port count configurations Ports can be (re)assigned to VSANs non-disruptively ISLs become Enhanced ISLs (EISLs) carrying hardware tagged traffic from multiple VSANs ISL bandwidth is securely shared between VSANs – reduces cost of excessive ISLs EISLs only carry permitted VSANs – can limit reach of individual VSANs Ports Required: 70 Ports Deployed: 96 ISL Ports: 16 (7:1 fan-out) Ports Stranded: 10 Net: 96 ports for 70 used 32 Port Switches Ports Required: 40 Ports Deployed: 64 ISL Ports: 0 Ports Stranded: 24 Net: 64 ports for 40 used 70 Port Fabric – Red_VSAN 40 Port Fabric – Blue_VSAN WITH VSANs Ports Required: Ports Deployed: 128 ISL Ports: 0 Ports Assignable: 18 (able to add more switching modules too!) Net: 110 ports for 110 used
8
Virtual SANs Constrain Fault Impacts
Virtual SANs sectionalize fabric to increase availability High availability - all fabric services are replicated and maintained per VSAN (name services, zoning services, etc) Fabric events are isolated per VSAN – maintains isolation for HA Misbehaving HBA or controller Fabric rebuild event Zone set change etc. Fabric recovery from a disruptive event is ‘per VSAN’ resulting in faster reconvergence – smaller scope Future : Management segregation per VSAN Fact: VSANs, DO contain RSCNs within the VSAN AND zone they are generated – filtering is done in hardware – no leaking 8-link bundled EISL trunk (VSAN-enabled) !!Fabric event!! HBA generates erroneous control frames Faults are constrained to the extents of the VSAN and only affect devices within the VSAN
9
Differentiated Network Service – Per VSAN
Switch B Domain 100 Domain 200 Virtual SANs enable resource allocation and preference per virtual fabric Each VSAN runs a separate instance of FSPF routing – independent forwarding decisions per VSAN EISL trunk links can be tuned for preferential routing per VSAN Different recovery paths can be configured per VSANs VSANs can be carried securely across metro and wide area networks via FCIP, SONET, DWDM, or CDWM Quality of Service (QoS) per VSAN to give preferential treatment at points of congestion in network QOS Metric 50 - Red VSAN 16 Gbps EISL Trunk Metric Blue VSAN Metric 50 - Blue VSAN 8 Gbps EISL Trunk Metric Red VSAN QOS QOS Domain 104 Domain 204 Switch A How do I get from switch A to switch B? It depends on which VSAN. Preferential routes can be configured per-VSAN to engineer traffic patterns.
10
VSANs as Migration Tool to SAN
Virtual SANs can be used to safely migrate applications to the SAN Use any unused ports in the larger SAN fabric to assign to new application Isolate new SAN-attached devices to new VSAN to help restrict fault domain Using VSANs, control reach of traffic associated with new VSAN within larger fabric Merge VSAN into greater SAN after proper testing and evaluation has been completed Can simply use IVR as well to tie in new application to existing storage – no VSAN disruption Green VSAN is used to deploy new application Cisco MDS 9000 Family with VSAN Service Green VSAN is used to deploy new application
11
VSANs Allow Sharing of DR Facilities
Switch A Virtual SANs can be carried between data centers over various transport facilities Allows consolidation ($$$) of DR facilities while still maintaining isolation Can set preferential usage of DR transport per VSAN – routing metrics Various wide and metro area facilities can be used securely: FCIP (PoS, ATM, Metro Ethernet), SONET, DWDM, or CDWM, etc. Cisco MDS 9000 family provides traffic statistics per VSAN for chargeback possibility Full fabric discovery per-VSAN through Cisco Fabric Manager Domain 100 Domain 200 Metric 50 - Blue VSAN 1Gbps EISL Trunk Metric Red VSAN 8 Gbps EISL Trunk Metric 50 - Red VSAN Metric Blue VSAN DWDM or CWDM SONET or SDH IP Routed Network (FCIP) Domain 104 Domain 204 Switch B How do I get from switch A to switch B? Preferential routes can be configured per-VSAN helping utilize WAN/MAN transport facilities appropriately to meet the application needs. All this while still maintaining backup paths in case of path failures
12
iSCSI Adds to Connectivity Options
iSCSI offers additional connectivity options which help address mid-range server connection costs iSCSI is implemented on the MDS 9000 to mimic Fibre Channel attachment iSCSI hosts are discovered and displayed through the Cisco Fabric Manager iSCSI hosts are bound to assigned WWNs creating a static relationship enabling : Zoning of iSCSI initiators Accounting against iSCSI initiators Topology mapping of iSCSI initiators iSCSI is offered through the IPS module in the MDS 9000 Family of switches 8 x 1Gb Ethernet connections capable of wire-rate iSCSI and/or FCIP Configured through Cisco Fabric Manager Offer HA features such as VRRP Cisco IP Storage Switching Module iQN2 is mapped to an allocated pWWN and registered in the fabric IP-VRRP support for redundant IP gateway services iSCSI iQN2 = pWWN2 Cisco Catalyst 6500 Multilayer LAN Switches iSCSI Login registering iQN iSCSI qualified names are defined within iSCSI client iSCSI iSCSI iQN1 iQN2 iSCSI-enabled hosts
13
iSCSI Leveraging VSANs
VSANs are extended to iSCSI transparently through address mapping iSCSI is implemented on the MDS 9000 to mimic Fibre Channel attachment iSCSI hosts are discovered and displayed through the Cisco Fabric Manager iSCSI hosts are bound to assigned WWNs creating a static relationship enabling : iSCSI host VSAN membership Zoning of iSCSI initiators Accounting against iSCSI initiators Topology mapping of iSCSI initiators iSCSI is offered through the IPS module in the MDS 9000 Family of switches 8 x 1Gb Ethernet connections capable of wire-rate iSCSI and/or FCIP Configured through Cisco Fabric Manager Offer HA features such as VRRP Cisco IP Storage Switching Module IPS iSCSI Cisco Catalyst 6500 Multilayer LAN Switches iQN2 = pWWN2 iSCSI Login registering iQN iSCSI iSCSI iQN1 iQN2 iSCSI-enabled hosts
14
FICON Intermixing Leveraging VSANs
Industry First! FICON Intermixing Leveraging VSANs Application / Department based SAN Islands Z/OS (FICON) VSAN Cisco MDS 9000 Family FICON Z-Series Z/OS Applicaitons FICON Channel Mainframe Storage FICON FICON Common Storage Pool Shared Amongst VSANs FICON FC Z-Series Linux FICON FICON LINUX Applicaitons Fibre Channel Mainframe Storage FC Linux VSAN Open Systems VSAN FC Collapsed Fabric with VSANs Open Systems Servers and Applications Fibre Channel Open Systems Storage Clean partitioning of different operating environments (FICON, Z-Series Linux-FCP, Open Systems FCP) Significantly more stable and manageable than current zoning+best practices approach FC Separate physical fabrics Over-provisioning ports on each island High number of switches to manage
15
Inter-VSAN Routing (IVR) : Sharing Resources Across VSANs
Industry First! Inter-VSAN Routing (IVR) : Sharing Resources Across VSANs Allows sharing of centralized storage services such as tape libraries and disks across VSANs – without merging separate fabrics (VSANs) Provides high fabric resiliency and VSAN-based manageability Works for all MDS 9000 switches with a software upgrade to SAN-OS 1.3(1) Distributed, scaleable, and highly resilient architecture Transparent to third-party switches Enables blade-per-VSAN architecture for blade servers VSAN-specifc Disk Engineering VSAN_1 IVR Marketing VSAN_2 Blade Server IVR IVR Tape SAN_4 (access via IVR) HR VSAN_3 Blade Server VSAN_1 (access via IVR) Marketing VSAN_2 HR VSAN_3
16
Inter-VSAN Routing (IVR): Resilient SAN Extension Solutions
Industry First! Inter-VSAN Routing (IVR): Resilient SAN Extension Solutions Minimize the impact of change in fabric services across geographically dispersed sites Limit fabric control traffic such as SW-RSCNs and Build/Reconfigure Fabric (BF/RCF) to local VSANs Flexible connectivity with the highest availability Works with any transport service (FC, SONET, DWDM/CWDM, FCIP) Inter-VSAN Connection with Completely Isolated Fabrics EISL#1 in Port Channel Replication VSAN_1 Replication VSAN_4 Metro DWDM (or SONET/SDH or FCIP) IVR IVR Transit VSAN_3 (IVR) Local VSAN_2 Local VSAN_5 EISL#2 in Port Channel
17
Fabric-Based Virtualization Leveraging VSANs
Challenge: Optimize storage usage while supporting heterogeneous storage Virtual Targets with Virtual LUNs are built from discovered physical storage Virtual LUNs and targets can be zoned to destined host(s) Separate VSAN used to isolate physical storage Ability to virtualize across multiple vendors’ storage arrays Cisco working with several partners to deliver solutions Group ‘A’ Group ‘B’ TARG 1 TARG 2 TARG 3 TARG 1 TARG 2 50G 200G 100G 10G 200G 20G 300G 40G 240G 300G 50G 50G 125G . . . . . . Virtual Enclosure Storage VSAN Backup VSAN Shared Storage Pool Main Data Center
18
VSANs and Zones - Complimentary
Virtual SANs and fabric zoning are very complimentary Hierarchical relationship – First assign physical ports to VSANs Then configure independent zones per VSAN VSANs divide the physical infrastructure Zones provide added security and allow sharing of device ports VSANs provide traffic statistics VSANs only changed when ports needed per virtual fabric Zones can change frequently (eg. backup) Ports are added/removed non-disruptively to VSANs Relationship of VSANs to Zones Physical Topology VSAN 2 Disk2 Disk3 ZoneA Disk1 Host1 ZoneC Disk4 Host2 ZoneB VSAN 3 ZoneD Host4 ZoneA Host3 Disk5 Disk6
19
VSANs: Continual Innovation and Evolution
VSANs are key technology leveraged from large scale Ethernet datacenter expertise and technology (VLANs) VSANs represent key innovation in SAN design, provisioning and management Continual enhancements to further drive innovative cost-effective SAN solutions Cisco has proposed VSAN technology to the ANSI T11 FS-SP group SAN-OS v1.1 SAN-OS v1.3 SAN-OS v1.0 VSANs are extended to iSCSI and FCIP services VSAN-assigned iSCSI hosts VSAN-to-VLAN mapping VSAN trunking over FCIP VSANs enable cost-saving consolidation of SAN Islands Accounting per VSAN Separate services per VSAN zoneset, FSPF, RSCN, etc Separate policies per VSAN VSAN-based SPAN feature VSAN-based accounting SAN-OS v1.3 SAN-OS v1.2 FICON, fabric, and routing enhancements VSAN-based FICON services (FICON VSANs) FICON intermix with VSANs Inter-VSAN routing VSAN-based fabric timers VSAN-based QoS services SAN-OS v1.2 SAN-OS v1.1 VSAN-based management enhancements VSAN-specific roles - RBAC role assigned to VSAN scope - Admin only sees ports in assigned VSAN(s) SAN-OS v1.0
20
How Do Virtual SANs Work?
21
Two Primary Functions of VSANs
The Virtual SANs feature consists of two primary functions: Hardware-based frame tagging of traffic belonging to different VSANs – Hardware Isolation No special drivers or configuration required for end nodes (hosts, disks, etc) Traffic tagged at Fx_Port ingress and carried across EISL (enhanced ISL) links between switches Create independent partition of Fibre Channel services for each newly created VSAN – services include: Zone server, name server, management server, principle switch election, etc. Each service runs independently and is managed/configured independently Fibre Channel Services for Blue VSAN VSAN header is removed at egress point Fibre Channel Services for Red VSAN Cisco MDS 9000 Family with VSAN Service Trunking E_Port (TE_Port) Enhanced ISL (EISL) Trunk carries tagged traffic from multiple VSANs Trunking E_Port (TE_Port) VSAN header is added at ingress point indicating membership Fibre Channel Services for Blue VSAN No special support required by end nodes Fibre Channel Services for Red VSAN
22
EISLs and TE_Ports The Virtual SANs feature introduces two new SAN elements: The Trunking E_Port (TE_Port) Negotiated between MDS 9000 switches – default Carries tagged frames from multiple VSANs Can be optionally disabled to yield E_Port Only understood by Cisco MDS 9000 switches Also has a native VSAN assignment (for E_Port) Trunk all VSANs (1-4093) by default Not to be confused with Brocade ISL aggregation (trunking) The Enhanced ISL (EISL) link The resultant link created by two connected TE_Ports Superset of ISL functionality Carry individual control protocol information per VSAN (eg. zoning updates) Can be extended over distance (DWDM, FCIP, etc) Cisco MDS 9500 Director with VSAN Service Trunking E_Port (TE_Port) Enhanced ISL (EISL) Trunk carries tagged traffic from multiple VSANs Trunking E_Port (TE_Port) Cisco MDS 9216 Fabric with VSAN Service Trunking E_Port (TE_Port) Enhanced ISL (EISL) Trunk carries tagged traffic from multiple VSANs Notice: Blue VSAN doesn’t have to reside on switch for it to traverse switch
23
VSANs, EISLs, TE_ports and Port Channels
How do all these work together? Hierarchical relationship – Port Channels provide link aggregation to yield virtual ISL (E_Port) Single-link ISL or Port Channel ISL can be configured to become EISL – (TE_Port) VSANs can be selective grafted or pruned from EISL trunks All member links of a Port Channel must have same configuration prior to creating channel (eg. TE_Port or E_Port, VSANs enabled, etc) Port Channel technology provides high availability and fast recovery for VSAN trunk (EISL) Multiple Port Channels yield multiple paths for custom traffic engineering VSAN Metric VSAN Metric 8 Gbps PortChannel Trunking E_Port (TE_Port) VSAN 20 VSAN 10 Backup VSAN 10 Only E_Port Trunking E_Port (TE_Port) E_Port 4 link (8 Gbps) PortChannel configured as EISL
24
VSANs – The Technical Details
How is an EISL link formed? Negotiation details: Two interconnected switch ports conduct an ELP (Exchange Link Parameters) exchange – forms two E_Ports and ISL link (standards based negotiation) Two switches then conduct an ESC (Exchange Switch Capabilities) exchange – determines whether Cisco switch on other end or not capable of EISL (standards based negotiation) If NO – then remain as ISL If YES – then proceed to negotiate EISL/ISL If Cisco switches, two switches then conduct an EPP (Exchange Peer Parameters) exchange – determines whether to stay as ISL, move to EISL (VSAN-enabled), or isolate in case of mismatched port VSANs E E ELP Exchange ISL Formed* E E ELP Exchange (non-Cisco) ISL Formed* E E E E ESC Exchange Two Cisco Switches E E E E ESC Exchange (non-Cisco) DONE E E E E EPP Exchange Normal ISL OR TE TE EISL formed These modes are negotiated based on the configuration of the switches and the parameters of the ports. Isolation can occur if VSANs are mismatched. OR * Provided ELP parameters match such as timers and switches in interoperability mode if required Isolated
25
VSANs – The Technical Details – cont.
EISL Header – What’s inside? Several fields are added to the VSAN header that provide a variety of functions Each frame on a VSAN trunk carries an extra 8 bytes of header which includes: User priority – 3 bits – used for QoS functionality to designate priority of frame VSAN ID – 12 bits – used to mark the frame as part of a particular VSAN – supports up to 4096 VSANs MPLS flag – 1 bit – used to designate whether this frame is subject to Multi-Protocol Label Switching processing – future use Time-to-live (TTL) – 8 bits – used to help avoid routing loops – standard part of an IP frame Other misc. fields including version, frame type, and other reserved fields 4B SOF (Start of Frame) 8B EISL Header FC Header 24B Payload 0 -> 2112B 4B CRC 4B EOF (End of Frame)
26
VSAN Number Space VSAN numbering rules: VSAN 1 is the default VSAN
All ports are originally in VSAN1 VSAN 2 through 4093 can be assigned to ‘user’ VSANs – VSAN 0, 4094, 4095 are reserved A maximum of 1000 VSANs can be created from the range of Your mileage may vary based on other factors (eg. #zones, #zone sets, etc) VSAN 4094 is a reserved ‘special’ VSAN Called the ‘isolated VSAN’ Used to isolate ports who’s port-VSAN has been deleted Not propagated across switches Always present, can’t be deleted Configured VSANs VSAN 10 VSAN 20 VSAN 30 Cisco MDS 9000 Family with VSAN Service Trunking E_Port (TE_Port) Enhanced ISL (EISL) Trunk carries tagged traffic from multiple VSANs VSAN 30 is not propagated across EISL due to nonexistence on remote switch Trunking E_Port (TE_Port) Port is in VSAN 4094 (Isolated VSAN) VSAN 10 VSAN 20 Host is isolated from the fabric VSAN 30 Configured VSANs
27
VSANs and Domain IDs Recall: Each VSAN acts as a completely independent fabric Each VSAN has its own principle switch and domain_ID allocation policy (static or dynamic) Principle switches for different VSANs don’t have to reside on same physical switch Each switch will have a separate domain_ID for each active VSAN These domain_IDs can overlap between VSANs All ports are originally in VSAN1 Each VSAN can have a separate FC_ID allocation policy (static or dynamic) Domain 100 Domain 200 Domain 105 Domain 223 Domain 126 Domain 153 Domain 173 Domain 110 Domain 153 Domain 112 Domain 171 Domain 156 Domain 102 Domain 113 Domain 180 Domain 104 Domain 204 Domain 157 Domain 170 Domain 215 Domain 201 Domain 162 Each switch that has end ports in a particular VSAN will have a domain_ID assigned that that particular VSAN. Core switches that trunk these VSANs will also have assigned domain_IDs in these VSANs
28
VSANs and Non-Cisco Switches
The Virtual SANs feature involves a tagging mechanism which is not understood by 3rd party fabrics Cisco MDS 9000 Family switches do support heterogeneous switch interoperability – non VSAN aware Cisco “Interoperability Mode” (required) is configured per-VSAN – no loss of functionality in MDS 9000 switches Cisco MDS 9000 switches negotiate an E_Port with non-Cisco switches Cisco MDS 9000 E_Ports also have a port VSAN Therefore, the entire non-Cisco switch, including all its ports, will reside in the port VSAN of the connecting E_Port Enhanced ISL (EISL) trunks carrying numerous VSANs Simple ISL links E_Ports Non-Cisco Fabric Switches Each non-Cisco switch belongs to only one VSAN
29
Design Example Using VSANs
Challenge: Build a highly resilient, large port density and manageable SAN Leverage VSANs in replacement for multiple separate physical fabrics Use very high port-density platforms to minimize number of switches req’d Use link bundling and multipathing within fabric to optimize resource usage Use separate VSAN for backup traffic to isolate Three platforms to optimize port density Dept/App ‘A’ Dept/App ‘B’ FC FC FC FC FC FC FC FC FC FC FC FC FC FC Cisco MDS 9216 Cisco MDS 9509 VSAN Trunk Bundles VSAN Trunks Cisco MDS 9509 Shared Storage Pool Backup VSAN FC FC FC FC FC FC Main Data Center
30
Additional Information
Cisco Storage Networking Cisco Metro Optical Product Information Storage Network Industry Association (SNIA) Internet Engineering Task Force – IP Storage ANSI T11 – Fibre Channel
Similar presentations
© 2025 SlidePlayer.com Inc.
All rights reserved.