1. SAN Building Blocks SAN Building Blocks Introduction
This lesson describes all of the major building blocks of a Storage Area Network (SAN).
Importance
Cisco SEs need to be able to describe the building blocks of SAN networks to customers. They will also use these building blocks to design and deploy SAN solutions for customers.
© 2003, Cisco Systems, Inc. All rights reserved. 1

2. Lesson Objective
Upon completion of this lesson, you will describe all of the primary building blocks of a SAN network.
Performance Objective
Upon completion of this lesson you will describe all major aspects of a SAN network.
Enabling Objectives
Recall components of a SAN network and introduce concept of nodes
Recall advantages of SAN networking technology
List three Fibre Channel SAN topologies
Explain the role of the ports used in a SAN environment
Explain a point-to-point topology
Explain a Fibre Channel Arbitrated Loop topology
Explain a switched fabric topology
Describe the range of SAN interconnect devices
Explain how a SAN gateway device can be used to extend a SAN network
Describe how heterogeneous SANs can be interconnected with SAN gateway devices
Describe three types of SCSI transport protocols
Explain how SCSI applications use common sets of commands, regardless of transport protocol
Describe three functions of multiprotocol, multifunction devices
Describe an evolutionary migration to a multilayer storage utility model

3. Lesson Outline Recap of SAN Components Recap of San Advantages
SAN Network Topologies
SAN Port Types
SAN Point-To-Point Topology
Fibre Channel Arbitrated Loop Topology
Switched Fabric Topology
SAN Interconnect Devices
SAN Gateways
Heterogeneous SANs—FC + IP
SCSI Transport Protocols
SCSI Applications
Multiprotocol, Multifunction Devices
Evolution to a Multilayer Storage Utility Model
Prerequisites
Completion of the Unit 1 prerequisites

4. Recap of SAN Components
Internetworking
Gateways
Disk
Storage
Systems/ Servers
Disk
Storage
Recap of SAN Components
Objective
Recall components of a SAN network and introduce concept of nodes.
Introduction
A SAN network consists of a number of components. The end points of the network are nodes.
Facts
A SAN network consists of a SAN switch fabric connecting network nodes. SAN network nodes include:
Systems and servers
Disk storage arrays
SAN-attached disk drives
Tape devices
Nodes are the end points in the network—the sources and destination of information.
Switches
Tape
Storage

5. Recap of SAN Advantages
Storage consolidation
Highly scalable
Enables, off-site mirroring, backup and recovery
No single point of failure
LAN-free, server-less backup
Recap of SAN Advantages
Objective
Recall advantages of SAN networking technology.
Introduction
SAN networking technology has a number of advantages..
Facts
Advantages of a SAN network include:
Data can be consolidated into fewer, larger storage systems, providing centralized management
SANs are highly scalable, able to connect many more systems and storage devices
Enables off-site mirroring, backup and recovery
SANs link geographically dispersed sites, enabling off site mirroring, backup and recovery
SAN architecture provides redundancy of all critical hardware components
With SANs, data can be backed up without placing loads on LANs or servers

6. SAN Network Topologies
Point-to-Point
Arbitrated Loop
Switched
Fabric
Hub
SAN Network Topologies
Objective
List three Fibre Channel SAN topologies.
Introduction
Fibre Channel (FC) SANs have three main types of topology.
Definition
Topology refers to the geometric shape of a network, representing the way that devices are interconnected.
Examples
There are three main types of FC SANs:
Point-to-point, a dedicated connection between two nodes
Arbitrated Loop (ring), where all nodes arbitrate for access to the loop
Switched Fabric, a mesh of connections controlled by intelligent switches
Each topology provides distinct characteristics in terms of connectivity (port types and number of devices), bandwidth and latency, and failure modes.
Practice
What kinds of characteristics might be different among the different topologies—that is, how might they differ?

7. SAN Port Types Point-to-Point Switched Fabric Arbitrated Loop Hub
E/TE
F_port
N_port NL L FL
N_port
SAN Port Types
Objective
Explain the role of the ports used in a SAN environment.
Introduction
A variety of port types are used in a SAN environment, depending on the network component and type of network topology.
Definition
A node port is a hardware facility that connects a node to the topology.
Facts
Every entity connected to the SAN, or within the SAN, requires a physical interface or port
Each node requires one or more physical interfaces, depending on the network topology
In the Small Computer System Interface (SCSI) and FC environments, this function is referred to as a Host Bus Adapter or HBA
In the network world, the port function is referred to as a Network Interface Card or NIC
Practice
Is a port a logical or physical entity in a SAN?

8. SAN Point-To-Point Topology
N_port
SAN Point-to-Point Topology
Objective
Explain a point-to-point topology.
Introduction
Point-to-point is the simplest SAN topology.
Definition
Point-to-point is a topology that allows two devices to be directly connected together.
Facts
Point-to-point is the simplest topology and allows two devices to be connected. The point-to-point topology is supported by both FC and Ethernet-based SANs. Point-to-point is suitable for small scale networks, where storage devices are dedicated to file servers. Because the link is dedicated, devices use the full bandwidth of the link. However, several separate point-to-point fibre channel links are costly.
Port Type: Nodes connected in a point to point topology use ‘N’ port types.
Example
The figure illustrates a point-to-point topology.
Discussion
A memory stick attached to a USB port is a small scale example of a point to point topology.
Practice
How many devices can be connected using the point-to-point topology?
How would you expand a configuration using only the point-to-point topology?

9. Fibre Channel Arbitrated Loop Topology
Hub
NL_port
Fibre Channel Arbitrated Loop Topology
Objective
Explain a Fibre Channel Arbitrated Loop topology.
Introduction
Arbitrated loop is FC’s implementation of a ring topology.
Definition
Fibre Channel Arbitrated Loop (FC-AL) is based on a loop interconnection scheme, similar to token ring. Unlike token ring, however, control is managed using an arbitration protocol rather than a token-passing scheme.
Facts
Arbitrated Loop is a popular FC topology for disk (and tape) attachment due to its low cost. All devices share use of the same physical loop, and bandwidth is shared by all of the devices. However, the low-level protocols that are needed to manage access to the loop may adversely affect performance because they consume bandwidth.
The number of nodes on the loop directly affect performance, up to a maximum of 126 nodes. The maximum bandwidth is 100 MB/sec., which is shared among all nodes on loop. Once a node successfully arbitrates for control of the loop, it opens a channel to the target, transfers the data, then closes the channel. During this time, only one pair of devices can communicate on the loop.
Port type: Devices connect to the loop via the NL_Port (Node/Loop).
Continued …

10. Fibre Channel Arbitrated Loop Topology (cont.)
Arbitrated Loop Connected to Switch Fabric
Hub
NL_port
FL_port
Fibre Channel Arbitrated Loop Topology (cont.)
Facts
Loops may be connected to a switched fabric in a larger SAN environment using the FL_Port (Fabric/Loop) port type. This potentially represents the 127th loop port connection (126+1).
Example
The figure presents an example of an arbitrated loop topology.
Practice
Can you think of other cases where a loop or ring topology is used?
Do you think the arbitrated loop topology is a good topology for a SAN? Why?

11. Switched Fabric Topology
F_port
N_port
Switched Fabric Topology
Objective
Explain a switched fabric topology.
Introduction
The switched fabric topology is the basis for all large-scale SANs.
Definition
A switched fabric is an interconnection scheme based on one or more interconnected switches.
Facts
Switched topologies consist of one or more interconnected switches (that is, a switching fabric). A switched fabric allows multiple devices to communicate at the same time. This makes the switched fabric topology an ideal solution for large SAN configurations.
Both FC and Ethernet support the switched fabric topology. The theoretical size of the configuration is limited only by the addressing capabilities of the network. A switched fabric configuration may consist of thousands of devices.
Despite the fact that the switched fabric topology allows an interconnection of 2-to-the-24 ports, each individual port that is connected to the fabric is allocated 100 MBps full-duplex, dedicated bandwidth.
Any N_port (Node) can be attached to the fabric by way of a link. The connection at the switch end is an F_port (Fabric).

12. Switched Fabric Topology (cont.)
E_port
Arbitrated Loop Hub
FL_port
Switched Fabric Topology (cont.)
Facts
Other fabric switch port types include:
E_Port: (Expansion), connects to an E_Port on another switch.
G_Port: (Generic), a switch port that can operate as either an E_Port or an F_Port
Cisco MDS-specific switch port types include:
SD_Port (Span Destination)
TE_Port (Trunking Expansion, for Virtual SANs (VSANs)
The switched fabric topology may also be combined with other topologies, such as FC-AL. In this case the node loop port (NL_port) is attached to the fabric by way of a link. The connection on the switch end is called a fabric loop port (FL_port).
Example
The diagram illustrates a switched fabric topology.
Analogy
A switched fabric topology can be compared to a telephone network, where many phone calls can occur simultaneously.
Practice
Can you think of other cases where the switched fabric topology is used?
Do you think the switched fabric topology is a good topology for a SAN? Why?

13. SAN Interconnect Devices
Systems/Servers
Disk
Storage
Devices
Tape
Hub
Switches
SAN Internetworking
SAN Interconnect Devices
Objective
Describe the range of SAN interconnect devices.
Introduction
Interconnect devices are used to connect SAN devices to the SAN fabric. Their primary function is to transport data, delivering frames or packets from point A to point B.
Facts
SAN interconnect devices include:
SAN switches are the most common interconnects devices in large-scale SANs. The SAN switch also integrates routing functions. Note: IP switches are not by themselves SAN devices.
SAN Hubs are used in arbitrated loops SAN architectures.
SAN Bridges extend the SAN across other networks.
SAN Gateways provide the transition between physical interfaces, for example, FC-SCSI bus and SCSI to SCSI bus.
Note: Even though they may share the same name, Ethernet LAN devices do not have the same functionality as these SAN devices. The storage networking glossary at provides a good reference that highlights the differences.
Practice
What is the difference between a node and a switch?
Can you think of any other types of SAN interconnection devices besides those listed?

14. SAN Gateways Fibre Channel to SCSI bus FC to SCSI Bridge Fibre Channel
Tape
Storage
Fibre Channel to SCSI bus
FC to SCSI
Bridge
SCSI bus
Systems/ Servers
Fibre Channel
(FCP)
Disk
Array FC
SAN Gateways
Objective
Explain how a SAN gateway device can be used to extend a SAN network.
Introduction
This example shows how SAN gateway devices can connect a SCSI tape storage device to a FC Protocol (FCP) SAN network.
Many customers have existing installations using SCSI bus-based storage devices, such as disk arrays and tape drives or libraries. As they migrate to SAN environments, it is desirable to be able to continue to use those devices from the SAN. Gateway devices allow this to be done.
Definition
SAN gateways provide the transition between physical interfaces.
Example
This diagram illustrates a SAN gateway that interconnects FC to SCSI bus.
Disk
Array FC

15. Heterogeneous SANs—FC + IP
Fibre Channel
SAN
Ethernet/IP
SAN (iSCSI)
iSCSI to FCP
Gateway
SCSI Commands (iSCSI)
SCSI Commands (FCP)
Heterogeneous SANs–FC + IP
Objective
Describe how heterogeneous SANs can be interconnected with SAN gateway devices.
Introduction
A heterogeneous SAN interconnects an FC SAN with an IP SAN.
Definition
There are two fundamentally different SAN technologies:
FC SANs evolved from existing storage environments and are described with storage-based terminology
IP SANs evolved from networking technology and are described with networking terminology
Both FC and IP SANs enable multiple systems or servers to access multiple storage devices through a network-like infrastructure. Both support the transport of SCSI commands. FC uses the SCSI-FCP protocol, IP networks use the Internet SCSI (iSCSI) protocol.
Both support network configurations consisting of switches, routers, and hubs. However because FC and IP SAN technologies evolved from different backgrounds, the same term in one technology may have a different meaning in the other. The storage networking glossary at provides a good reference that highlights the differences.

16. SCSI Transport ProtocolsIP
iSCSI
TCP
Ethernet
Fibre Channel
FCP
(SCSI Over FC)
Parallel
SCSI
Transport
SCSI Transport Protocols
Layer 3 and 4
Network
Layer 2 Network
Fibre Channel Protocol (FCP)
SCSI Parallel Interface (SPI)
ISCSI (uses TCP for end-to-end error control)
SCSI Transport Protocols
Objective
Describe three types of SCSI transport protocols.
Introduction
The SCSI family of standards can support multiple SAN technologies and devices, through the implementation of three transport protocols.
Facts
SCSI is a family of standards broken up into different layers and command sets to allow for different physical transports, for example, Ethernet (iSCSI), FC, and Parallel SCSI.
iSCSI
Although not technically part of the family of SCSI standards, iSCSI defines the rules and processes to transmit and receive block storage applications over TCP/IP networks.
At the physical layer, iSCSI supports a Gigabit Ethernet interface so that systems supporting iSCSI interfaces can be directly connected to standard Gigabit Ethernet switches and/or IP routers. The iSCSI protocol sits above the physical and data-link layers and interfaces to the operating system's standard SCSI Access Method command set.
iSCSI enables SCSI-3 commands to be encapsulated in TCP/IP packets and delivered reliably over IP networks. iSCSI can be supported over any physical media that supports TCP/IP as a transport, but modern iSCSI implementations are on Gigabit Ethernet, which supports a theoretical limit of 4 billion ports (2^32) and a maximum cable length of 10 km.
Many experts predict that iSCSI enabled SANs will soon overtake FC based SANs.

17. SCSI Applications SCSI Commands, Data, and Status
SCSI Applications (File Systems, Databases)
SCSI Device-Type Commands
SCSI Block Commands
SCSI Stream Commands
Other SCSI Commands
SCSI Commands, Data, and Status
SCSI Generic Commands
SCSI Transport Protocols
iSCSI
FCP
(SCSI Over FC)
Parallel
SCSI
Transport
TCP
Layer 3 and 4
Network
Transport
SCSI Applications
Objective
Explain how SCSI applications use common sets of commands, regardless of transport protocol.
Introduction
Even though SCSI applications may make storage requests through a variety of devices and over a variety of transport protocols, the basic SCSI command set is the same.
Facts
User level applications (for example, Enterprise Resource Planning [ERP], Customer Relationship Management [CRM], and MS Office) are completely unaware of the underlying mechanism used to access physical storage devices.
Application requests to open, save, close or delete files and records are translated into SCSI commands such as copy, read, seek and write by the underlying operating system, file system or database. These commands in turn are relayed to the storage device by an underlying SCSI transport protocol over the data link and physical media layers.
In all cases, the commands are the same, regardless of which protocol is used, iSCSI, FC, or Parallel SCSI. Only the data structures and physical transport are different, no matter which protocol is used. IP
Layer 2 Network
Ethernet
Fibre Channel
Parallel SCSI

18. Multiprotocol, Multifunction Devices
Within a single device, multiple functions may be incorporated, including:
Multiprotocol switching
Encapsulation
Gateway functions
Multiprotocol, Multifunction Devices
Objective
Describe three functions of multiprotocol, multifunction devices.
Introduction
Multiprotocol or multi-function devices may provide valuable functionality in heterogeneous SANs.
Definition
A multiprotocol device is one that supports more than one protocol and/or physical interface.
Facts
When a SAN consists of both IP and FC portions, separate switches could be used for each portion. However, this requires the user to purchase two different switches, one for each portion. Instead, a single multi-protocol switch could be used for both portions. This could reduce the cost to provide the connectivity and simplify management of the SAN (one point of management rather than two).
A multiprotocol device may incorporate:
Multi-protocol switching, to support both FC and IP/Ethernet
Encapsulation, such as FC to IP (FCIP) encapsulation
Gateway functions, such as SCSI-FCP to iSCSI

19. Evolution to a multilayer storage utility model
Homogenous
“SAN Islands”
Midrange DAS
Engineering SAN
ERP SAN
HR SAN FC
Evolution to a Multilayer Storage Utility Model
Objective
Describe an evolutionary migration to a multilayer storage utility model.
Introduction
Storage is evolving to an on-demand, multilayer utility model. This is a business model of making storage available as a utility—such as water or electricity. The goal is to offer users the same type of plug-in, on-demand service associated with traditional telephone or electric utilities.
The diagram illustrates a migration path to a multilayer storage utility model in three phases.
Definitions
ERP, HR: Enterprise Resource Planning, Human Resources applications
DAS: Direct Attached Storage
VSANs: Virtual SANs, multiple virtual fabrics overlayed on the same physical structure
HA: High Availability, network availability prioritized by business application, according to the impact of network downtime on loss of revenue
LAN/MAN/WAN: Local, Metropolitan, Wide Area Network
FCIP: Fibre Channel over IP
QoS: Quality of Service, based on Service Level Agreements
HSM: Hierarchical Storage Management, a prioritization scheme based on storage cost and availability priority
Phase 0: Isolated SANs and Mid-range DAS

20. Evolution to a multilayer storage utility model (cont.)
Storage Network
Homogenous
“SAN Islands”
Phase 0: Isolated SANs and Mid-range DAS
Phase 1: High-end and Mid-range Consolidation
Midrange DAS
Engineering SAN
ERP SAN
HR SAN FC
Pooled Storage and Tape
Midrange
Hosts/Apps (MS Exchange)
Engineering, ERP, HR Hosts
Multilayer Storage Network
Security
VSAN
Scalability
QoS
WAN /
FCIP
multiprotocol
Diagnostics HA
Evolution to a Multilayer Storage Utility Model (cont.)
Example 2
Phase 1 involved consolidation of high-end and mid-range storage networking resources. Benefits of this phase include:
By interconnecting SANs, data access is improved data access.
Scalable VSANs enable a shared infrastructure and isolate fabric disruptions
There is a centralized, single point of management
A comprehensive suite of advanced, integrated diagnostics is available
Multiprotocol options provide cost effective alternative to FC for midrange applications
Disaster recovery is available through end-to-end transport across LAN/MAN/WAN

21. Evolution to a multilayer storage utility model (cont.)
Storage Network
Storage Utility
Homogenous
“SAN Islands”
Phase 0: Isolated SANs and Mid-range DAS
Phase 1: High-end and Mid-range Consolidation
Midrange DAS
Engineering SAN
ERP SAN
HR SAN FC
Multilayer Storage Network
Data Mobility
WAN /
FCIP
Storage Virtualization
Dynamic Provisioning
Heterogeneous Storage
LAN
Free Backup
HSM
Pooled Storage and Tape
Midrange
Hosts/Apps (MS Exchange)
Engineering, ERP, HR Hosts
Security
VSAN
Scalability
QoS
multiprotocol
Diagnostics HA
Evolution to a Multilayer Storage Utility Model (cont.)
Example 3
In phase 2, the final phase, the network is able to fully integrate and implement network hosted storage applications.
Benefits of this phase include:
Reduced management complexity by implementing virtualized storage
Improved performance and disaster recovery with data optimally distributed across the network
Support for heterogeneous storage environments
Reduction in overall storage cost through the implementation of HSM
Simplified storage administration through dynamic provisioning
Integrated content awareness, which enables administrators to offer policy based service levels
Practice
Based on your customer knowledge, what are some of the long-range benefits of migrating to a fully-integrated storage utility model?
Phase 2: Network Hosted Storage Applications

22. Lesson Practice Divide into three groups.
Referring to the lesson slide representing your group, create a hypothetical or real-world business profile of an enterprise whose needs would match the scale of the architectural solution. Link as many business processes to architectural components as possible.
Entry-level switched fabric topology architecture
Heterogeneous SAN architecture
Network-hosted storage application architecture

23. Summary This lesson presented these key points: SAN Network Topologies
SAN Port Types
SAN Interconnect Devices
Heterogeneous SANs
SCSI Transport Protocols
Multiprotocol, Multifunction Devices
Evolution to a Multilayer Storage Utility Model
Summary: SAN Building Blocks
This lesson described all the major building blocks of a Storage Area Network (SAN). Cisco SEs need to be able to describe the building blocks of SAN networks to customers. They will also use these building blocks to design and deploy SAN solutions for customers.