1. The Fibre Channel Protocol Stack
From FC-4 till end of unit 3

2. FC-4 and ULPs: application protocols
FC-0 to FC-3 solely serve to connect end devices by means of FC network.
Type of data exchanged? It depends on upper layer protocols, SCSI and IP.
FC-4 mappings task is to map the application protocols onto the underlying FC network.
Mappings support the API of existing protocols upwards in the direction of OS and realize these downwards in the direction of the medium.
Protocol mappings specify which service classes will be used and how application data will be projected onto exchange sequence frame mechanism.
Existing protocols? Install device drivers.

3. Fibre channel protocol (FCP) maps the SCSI protocol onto the underling FC network.
SCSI protocol operates as before via Fibre channel network. This is known as Fibre channel SAN
FCP should manage parallel to serial communication and daisy chain of SCSI bus onto FC topology.

4. IPFC uses a FC connection between two servers for IP data traffic.
Installing IPFC drivers in the OS.
Configure local IP to send IP packets to the FC HBA.
FICON (Fibre Connection) maps ESCON(Enterprise System Connection) used in Mainframes onto FC networks.

5. Fibre Channel SAN Point-to-point topology
How FC SAN is realized using the 3 topologies?
Point-to-point topology
Connects only two devices and is not expandable (Ex. Server and Storage devices).
Compared to SCSI cabling, it has advantages:
Greater cable lengths (10kms without repeaters)
Optic cables are robust and do not emit EM signals.
FC cables are simpler to lay.
Application server and shared data storage could be kept separate and protected.

6. Fabric topology Most flexible and scalable of the three types.
15.5 million devices can be connected in theory.
FC switches support cut-through routing.
Latency: the period of time required to transmit a signal or the period of time required to forward a frame.
Switch takes two to four micro seconds

7. Several devices can send and receive data simultaneously.
Each of the three logical connections get complete bandwidth (say, 200MByte/s).
Good design is necessary for full bandwidth availability.

8. Similar design but single switch is replaced by two switches.
ISL (Inter-switch link) connects the switches.
Limiting factor: all three connections pass through the same inter-switch link.
Only a third of the maximum bandwidth is available to servers.

9. Switches realize Aliasing, Name server and Zoning.
64 bit identifiers for WWPN and WWNN, and addressed by 24-bit port addresses. Administrator can make use of alias names to WWNs and ports.
New device reports and registers to name server. It can ask information about other devices.
Zoning helps in defining subnetworks within FC network. It has 2 advantages
Limits visibility (Protection of data)
Individual ports can reserve bandwidths for applications.

10. Soft Zoning Hard Zoning
If an end device asks the name server about other end devices in the FC network, it is only informed of the end device with which it shares at least one common zone.
If an end device knows the address (Port_Id) then it can still communicate with it. (OS stores everything)
Strictly only those devices that share at least one common zone can actually communicate.

11. Modern switches support LUN masking
Modern switches support LUN masking. Read the first bytes of the payload of each frame. Latency is slightly increased but this nothing compared to the latency at HBA.
In Virtual SAN, several ports (end devices) are grouped together to form a virtual fabric. Fabrics that are logically separate can be operated over one physical FC network.

12. Arbitrated loop topology
Connects servers and storage devices using a ring. (Data transfer in only one direction)
At any time, only two devices can exchange data, others have to wait. (If 6 servers are connected, then each server gets only one-sixth of the maximum bandwidth)
Hubs are used to simplify cabling.
To increase size, hubs can be cascaded.
It is less scalable and flexible, 126 devices.

14. Loops are subdivided into public loops and private loops.
Arbitrary loops does not support aliasing, routing, name server and zoning.
Loops are subdivided into public loops and private loops.
Private loop (Closed in on itself)
Public loop (connected to fabric by a switch)
A device in a public arbitrated loop can communicate with devices in fabric and with devices in the loop, if it controls both arbitrated loop protocol and the fabric protocol. (NL_PORT)

16. Devices with L-Port cannot communicate with the devices in the fabric.
Emulated loops (contain manufacturer specific upgrades) translate between arbitrary protocol and fabric protocol.

17. Hardware components of Fibre channel SAN
Servers and Storage devices are connected to FC network. HBAs are used in Servers and storage devices.
Cables (and different types), connectors are required for cabling.
An important device in FC network is Fibre Channel-to-SCSI bridge.
Creates a connection between FC and SCSI and has two important applications:
Old storage devices that cannot be converted from SCSI to FC can be used.
New tape libraries that support only SCSI.
SAN router, Storage gateway.

19. Newer switches have ports from 8 to 250 and support high speeds.
Enterprise class switches (directors) are designed to avoid single point of failure.
Small entry level SANs go for complementary switches.
Designers of large fibre channel SANs prefer directors. (Stock market or flight control)

20. Hubs are transparent to connected devices.
They bridge across defective and switched-off devices to maintain ring.
Unmanaged, managed and switched hubs.
Unmanaged hubs
Do not intervene if devices violate protocol.
Do not indicate the state of the hub or the state of the loop.
Managed hubs
Have administrative and diagnostic functions.
Monitor power supply, serviceability of fans, temperature and status of individual ports.
Intervene in higher level protocols also (deactivate a port).
Switched hub
Managed hub + exchange of data at full bandwidth.
FC hub vs. FC switch.

21. Link extenders can increase the maximum cable length by transmitting FC frames using MAN/WAN technique.
Latency increases and cannot be used in time critical applications.

22. InterSANs
Simple islands of SANs are easier to manage, data can be exchanged via LAN.

23. More and more applications are exploiting possibilities offered by FC SAN.
Backup, remote data mirroring, data sharing over FC SAN and storage virtualization require that all servers and storage devices are connected via a single SAN.

24. Interoperability of FC SAN
Interoperability of FCP
FCP is complex piece of software that is implemented as device driver.
Errors, should be extensively tested.
Issues leading to testing are Multitasking devices (race condition) and large number of components (50 combinations for connecting 1 server and 1 storage device).
Interoperability of end devices
Incompatible end devices (say switch) by manufacturers.
Testing should be done so that existing devices are also supported.

25. IP Storage Data transmission over TCP/IP and Ethernet.
IP Storage standards: iSCSI, iFCP, mFCP, FCIP and iSNS.
Storage data over TCP/IP: iSCSI, iFCP and FCIP.
Basic idea over iSCSI is to transmit SCSI protocol over TCP/IP.
iSCSI instead of FCP.

26. iSCSI HBAs to reduce server’s CPU load.
iSCSI and FC techniques can be used together.
Ex: Booting of diskless servers over iSCSI using iSCSI-to-Fibre Channel gateway.
Cost benefits are achieved.

27. iFCP defines the mapping of FC devices on TCP/IP.
To protect the investments that were made on FC.
To realize this, LAN switches must provide either F-port or FL-port.

28. Difference between mFCP (Metro FCP) and iFCP (Internet FCP) is that mFCP used UDP/IP and iFCP used TCP/IP.
mFCPs performance depends on the underlying network connection.
It is used only in low-error networks such as LANs.
Both mFCP and iFCP protect investments made in FC.
But they are very complex and extensive testing is required for cross manufacturer function.

29. Fibre Channel over IP (FCIP) was designed as supplement to FC.
Long distance data transfer (remote mirroring).
FCIP is tunnelling protocol that connects two FC islands over a TCP/IP route.
FCIP thus creates a point-to- point connection between two FC islands.
Encryption is added advantage in FCIP

30. iSNS (Internet Storage Name Service) is a service to scan for devices in the IP network.
It is a client-server application, clients register their attributes with the server.
iSNS is integrated in iSCSI and iFCP.
iSCSI, iFCP and FCIP are similar, compare protocol stack.

32. TCP/IP and Ethernet as an I/O technology
IP advantages over FC
Disadvantages of IP storage
Common network for LAN, MAN, SAN, etc.
Standardized and mature technology.
More people with TCP/IP knowledge.
No distance limits.
Cheaper hardware because of market competition.
Administration tools available.
Lack of standardization.
Lack of interoperability.
High CPU use for data traffic.
Greater TCP/IP protocol overhead.
High latency of TCP/IP switches.
Low exploitation of bandwidth.

33. IP network (Different purposes)
Common network for LAN, MAN, SAN, etc. But existing networks are working at their limit. Capacity must be increased.
IP network (Different purposes)
Telephone during day.
Data backup during night.

34. Standardization of all communications over TCP/IP/Ethernet ensures higher market volume, increases competition, results in reduced price.
Availability of personnel with knowledge because of decades of use.
Administration tools for TCP/IP networks. Tools to determine the load on storage devices, which servers use which storage device, etc.
Internet is growing and latency is increasing. Unsuitable for time critical I/O accesses (database transaction).
Serialize SCSI protocol and map it onto IP. Should be tested for interoperability.

35. TCP/IP data traffic is very CPU intensive.
In current network cards, large part of the protocol stack is processed by the CPU.
In FC, it is processed by the host bus adapters, hence freeing the CPU.
TCP/IP offload engines in modern network cards handle most of the protocol stack.

36. FC stack is a integrated whole, cut-through routing is simple.
TCP/IP and Ethernet were developed independently and not harmonized. Multiple layers are involved and hence cut-through routing is not as simple as FC.
Simultaneous transmissions leads to collisions and hence only 20-30% of the bandwidth is used. This is false. Full duplex without interference.

37. Migration from SCSI and Fibre Channel to IP Storage
Investment in technology that solves today’s problem and have long life cycle.
Interoperability issues until the technology gets standardized.

38. The NAS Architecture
NAS was introduced to provide shared storage for users on a network.
NAS is a specialized computer that provides file access and storage for a client/server network.
NAS is bundled solution that provides pre-packaged hardware and software.
Hardware components: Server part, computer system, storage adapters, storage controllers and disk storage.
Software components: OS, File system and network drivers.

39. A simple network with NAS

40. Bundling and configuration allows 5 minutes plug and play characteristic of NAS.
Enterprise-level solutions have much more thought put into planning and implementation.
NAS is optimized for file I/O, any other type of I/O will be problematic.
NAS is one of the easiest way to accommodate increases to storage capacity for LAN-based shared files.
Difficult to administer and manage when complexity increases.
NAS uses existing network resources (Ethernet-based TCP/IP).
Care should be taken for expansion.
NAS uses a RTOS, proprietary enhancements to kernel (Black box).

41. The NAS Software Architecture
NAS software consists of three major components:
The Micro-kernel OS
File System
Network Software Drivers
The Micro-Kernel OS Part
OS is a UNIX kernel derivative.
Designed to enable effective file I/O operations.
Memory management, process management and resource management are optimized to serve I/O requests.
Besides the management tools, other processing activities are not available.

43. Process management Memory management Resource management
File I/O requests at the highest priority to the I/O manager.
Scheduling methods and algorithms are optimized.
Other overhead tasks are managing network processes and handling TCP stack.
Memory management
Sufficient buffer space and cache management to facilitate the process management schedule.
Up-side, cache is used and down-side read-ahead buffer is utilized.
Resource management
Focused job, hence resource management is static job.
Operations of disk controllers and enhancements to controller processing play significant role in completion of I/O tasks.

44. NAS File Systems
Standard file systems supported are Common Internet File System (CIFs) and Network File System (NFS).
These standard file systems interface with NAS proprietary file system.
Heterogenous use of NAS device to support multiclient access is evolving from CIFs to SAMBA.

45. Network Connectivity
Supports standard TCP/IP communications with diverse transport media. (Popular: Ethernet)
TCP/IP creates overhead. (Processing of layers in server)
Performance issues are difficult to address because of
Black box nature of NAS
OEM supplier relationship of NIC components
TCP off-load engines (TOE) ensure processing of layers in NIC.

46. NAS as a Storage System
Putting all parts together, we get a highly optimized storage device to be placed on the network.
Minimum network disruption
Reduced server OS latency and cost.
Configuration flexibility and management is kept at a minimum.
They are available for a diversity of configurations and workloads:
Departmental solution.
Mid-range solution.
Enterprise-class solution.

47. The Departmental NAS Architecture
Often used to support the high growth of file servers that populate windows client-based networks.
Perfect solution for consolidating unoptimized servers. (Windows NT, Windows 2000 server, etc.)
The Internet NAS Architecture
A more scalable and cost-effective solution is to consolidate multiple servers into a single NAS device.
Perfect storage solution for internet connections to web-based file access.
The Enterprise NAS Architecture
Support of large capacities of data and high I/O content.
Structured as well as unstructured data.