1. Fibre Channel Physical Architecture
Introduction
Fibre Channel (FC) has characteristics of both I/O channels and data networks, and this unique blend of features is what makes FC ideal for storage area networks (SANs). This lesson, takes a close look at the features and capabilities of FC, and compares these features and capabilities with those of traditional I/O channels, primarily SCSI, and data networks such as Ethernet and ATM.
Importance
This lesson presents some basic information about FC’s physical capabilities and limitations that a designer must know in order to design valid SAN configurations. A basic understanding of SCSI will also be helpful because the SCSI bus is still widely in use. For example, SCSI is used as the internal bus that connects the disks inside of some SAN storage arrays.

2. Lesson Objective
Upon completion of this lesson, you will be able to compare Fibre Channel technology to SCSI technology and other data networks in terms of their architectures and performance characteristics.
Performance Objective
Upon completion of this lesson, you will be able to compare FC technology to SCSI technology and other data networks in terms of their architectures and performance characteristics.
Enabling Objectives
List the key characteristics and limitations of SCSI technology
List the advantages of serial network architectures
Identify FC link performance, throughput rates, and distance limitations
Compare FC with SCSI channels
Compare FC networks with other data networks

3. Outline SCSI Technology Parallel vs. Serial Architecture
Fibre Channel Performance
Fibre Channel vs. SCSI
Fibre Channel vs. other data networks
Prerequisites
All previous lessons in Curriculum Unit 2, Module 1.

4. SCSI Initiator (I/O Adapter)
SCSI Technology
Multidrop Topology
Data/Address Bus
SCSI Initiator (I/O Adapter)
Terminator
Control Signals
SCSI Technology
Objective
List the key characteristics and limitations of SCSI technology.
Introduction
This section discusses the key characteristics and limitations of SCSI, including SCSI topology, addressing, and allowable cable length.
Facts
All of the devices on a SCSI bus are connected to a single cable. This is called a multidrop topology:
Data bits are sent in parallel on separate wires. Control signals are sent on a separate set of wires.
Only one device at a time can transmit—a transmitting device has exclusive use of the bus.
A special circuit called a terminator must be installed at the end of the cable. The cable must be terminated to prevent unwanted electrical effects from corrupting the signal.
A multidrop topology has inherent limitations:
Parallel transmission of data bits allows more data to be sent in a given time period but complicates transmitter-receiver synchronization. The fact that control signals, such as clock signals, are sent on a separate set of wires also makes synchronization more difficult.
It is an inefficient way to use the available bandwidth, because only one communication session can exist at a time.
Termination circuits are built into most SCSI devices, but the administrator often has to set a jumper on the device to enable termination.
Incorrect cable termination can cause either a severe failure or intermittent, difficult-to-trace errors.
Interface
Interface
Interface

5. SCSI Technology (cont.)
Addressing
Data/Address Bus
SCSI Initiator (I/O Adapter)
Terminator
Control Signals
Facts
SCSI was designed to support a few devices at most, so its device addressing scheme is fairly simple—and not very flexible. SCSI devices use hard addressing:
Each device has a series of jumpers that determine the device’s physical address, or SCSI ID. The ID is software-configurable on some devices.
Each device must have a unique ID. Before adding a device to the cable, the administrator must know the ID of every other device connected to the cable and choose a unique ID for this new device.
The ID of each device determines its priority on the bus. For example, the SCSI target with ID 7 always has a higher priority than the SCSI initiator with ID 6. Because each device must have exclusive use of the bus while it is transmitting, ID 6 must wait until ID 7 has finished transmitting. Fixed priority makes it more difficult for administrators to control performance and quality-of-service.
Interface
Interface
Interface ID 7 ID 6 ID 5 ID
. . .
Priority

6. SCSI Technology (cont.)
25m (Fast Wide SCSI, 10-80MB/s)
12m (UltraSCSI, MB/s)
Fact
One of the primary disadvantages of the SCSI bus is its limited cable length.
The maximum SCSI cable length depends on the version of SCSI that is being used:
At fairly low speeds (10 to 80 MB/s) some versions of SCSI can operate over a total cable length of up to 25 meters (Fast Wide SCSI).
At the higher speeds required by modern applications (160 to 320 MB/s), this distance has been reduced to 12 meters total length (Ultra160 SCSI and Ultra320 SCSI).
Some older versions of SCSI require even shorter cables—down to 1.5 meters.

7. SCSI Technology (cont.)
Maximum allowable skew
Facts
The limited distance of the SCSI bus is inherent in the nature of parallel transmission. On a parallel bus, several bits of data are sent simultaneously down several wires. The bits must all arrive together within a very tight time period; if they do not arrive within that time period, the receiver cannot reconstruct the data.
The problem is that there are slight physical inconsistencies between the wires that can affect the speed at which electricity moves along the wire. As a set of data bits moves down the cable, the physical inconsistencies can cause the electrical signals to move at slightly different speeds in each write, and therefore to arrive at slightly different times. This effect is called skew. If the signal is too heavily skewed, the receiver cannot reconstruct the original data.
In the preceding graphic, the upper illustration depicts the allowable skew, and shows data that will arrive intact. The lower illustration shows data with too much skew to permit the receiver to reconstruct the data.
Maximum allowable skew

8. SCSI Technology (cont.)
Maximum allowable skew
Facts
The skew effect limits the distance over which high speed, parallel bus connections can be used. To meet the demand for faster and faster data transfer, the parallel bus approach has been to double the number of wires and speed up the rate at which signals are sent. Both of these modifications increase the skew effect:
More wires mean that there is more opportunity for the individual signals to get out of step with each other.
A faster signalling speed means that the allowable time frame for each bit of data shrinks, so the signals must be kept in closer step.
As SCSI bus speeds increase, skew increases while the maximum allowable skew decreases. The maximum length of the bus must therefore be reduced.
The preceding graphic depicts the increased skew that occurs when the number of wires are doubled and transmission speed increases.
Maximum allowable skew

9. Advantages of Serial Architecture
Benefits of serial architectures:
Reduces the cost and complexity of cabling
Use either copper cables or optical fiber
Clock synchronization and data transmission are performed in one signal
Simplifies product design, allowing faster evolution:
Wire speed of SCSI increases by 2x
Wire speed of Ethernet and FC increases by 10x
Advantages of Serial Architecture
Objective
List the advantages of serial network architectures.
Introduction
This section describes advantages of a serial architecture as compared to a parallel architecture.
Facts
In a serial data transmission, the data bits are sent sequentially along a single wire. This architecture offers several advantages over the parallel architecture:
A serial architecture reduces the cost and complexity of cabling. Unlike SCSI, FC does not require terminators, and it uses a network architecture (hubs and switches) rather than a multidrop (single cable) architecture.
Serial networks can use either copper cables or optical fiber. This allows customers to choose cheaper copper cables where distance is not a requirement, and to choose more expensive optical cable when longer distances must be supported.
Because clock synchronization and data transmission are performed in one signal, rather than on separate wires, synchronization can be more easily maintained at higher link rates and longer distances.
Overall, a serial architecture simplifies product design, allowing faster evolution. For example, the wire speed of SCSI doubles with each release, while the wire speed of Ethernet and FC increases by a factor of 10 with each release.
One of the most significant advantages of serial networks is that serial networks can support longer link distances. A single-mode fiber optic FC or Gigabit Ethernet link can support links over 400 times as long as on the longest SCSI bus.

10. Fibre Channel Performance
MB/s (sustained, each direction)
Bandwidth
Mode
Full duplex, serial
Maximum # of Nodes
126 arbitrated loop
~16 million switched fabric
Fibre Channel Performance
Objective
Identify FC link performance, throughput rates, and distance limitations.
Introduction
This section describes the performance characteristics of FC technology.
Facts
The performance characteristics of FCs are as follows:
Note that 100MB/s and 200MB/s are the half-duplex rates for Fibre Channel, but Fibre Channel is actually a full-duplex technology. In other words, Fibre Channel supports up to 200MB/s between two ports in both directions simultaneously.
Up to 30 m/link copper
Link Distance
Up to 10 km/link optical
Reliability
Bandwidth
MB/s (sustained, each direction)
Mode
Full duplex, serial
Maximum number of Nodes
126 arbitrated loop
>16 million switched fabric
Link Distance
Up to 30 m/link copper
Up to 10 km/link optical
Bit Error Ratio
< 10-12
Bit Error Ratio < 10-12

11. Fibre Channel Performance (cont.)
The Bit Error Ratio (BER) is calculated by dividing the number of erroneous bits by the total number of bits transmitted
A BER of corresponds to one error every 8 minutes at 2Gb/s
Due to some stringent applications, the industry is working on a BER of 10-15, or one error every 5.5 days at 2Gb/s
Facts
The Bit Error Ratio (BER) is calculated by dividing the number of erroneous bits by the total number of bits transmitted, received, or processed over some stipulated period. For example, 2.5 erroneous bits out of 100,000 bits transmitted would be 2.5 divided by 100,000 or 2.5 × 10-5.
The minimum and maximum values of average received power range determine the input power range required to maintain a BER less than This value takes into account worst case signal characteristics.
A BER of corresponds to one error every 8 minutes at 2Gb/s. This might seem like a very low error rate, but due to some stringent applications, the industry is working on achieving a BER of 10-15, which results in one error every 5.5 days at 2Gb/s.

12. Fibre Channel vs. SCSI Fibre Channel SCSI Full-duplex Serial
Objective
Compare FC and SCSI.
Introduction
This section compares the performance characteristics of FC technology with those of SCSI technology.
Facts
FC differs from traditional I/O buses such as SCSI in three significant respects:
Serial versus parallel architecture:
SCSI uses a parallel architecture in which data is sent simultaneously over multiple wires.
FC uses a serial architecture in which data transmitted one bit at a time, sequentially, over a single wire. This allows longer transmission distances.
Full-duplex versus half-duplex:
SCSI is half-duplex—data travels in one direction at a time.
FC is full-duplex—data travels in both directions simultaneously. A 1Gb FC link provides 2Gb/s (200MB/s) total aggregate bandwidth.
Packet-based versus dedicated connection:
On a SCSI bus, a device must assume exclusive control over the bus in order to communicate. (SCSI is sometimes referred to as a “simplex” channel because only one device can transmit at a time).
On an FC network, data is split into frames, allowing multiple devices to communicate simultaneously over a single data path.
Fibre Channel
SCSI
Full-duplex
Serial
Packet-based
Half-duplex
Parallel
Shared bus

13. Fibre Channel vs. SCSI (cont.)
SCSI Bus
Fibre Channel
Bandwidth
MB/s (burst)
MB/s (sustained)
Half duplex,
Full duplex,
Mode
parallel
serial
Maximum # of Nodes
126 Arbitrated Loop 16
>16 million switched
Facts
The table compares the characteristics of FC to those of SCSI. Significant differences between FC and SCSI include:
Bandwidth: Though the latest incarnation of SCSI (operating at 320MB/s) actually has a higher published bit rate than FC (200MB/s), FC is capable of delivering the published data rates in a sustained manner. The maximum SCSI bit rate is the peak rate, and cannot be sustained for long periods of time.
Mode: SCSI uses a parallel bus, with half duplex capability (transmission in one direction at a time), while the FC serial connection has full duplex capability.
Maximum number of nodes: 16 for SCSI, up to 16 million for FC.
The SCSI cable length limitations results in a maximum link distance of 25 meters, while FC, using optical cable, has a maximum link distance of 10 kilometers.
Software protocols: FC supports multiple protocols simultaneously. A version of the SCSI command set is often used on FC SANs, but the same SAN can simultaneously carry IP traffic and other protocols.
Note that the storage market typically measures data rates in megabytes-per-second (MB/s), whereas the network market typically measures data rates in megabits-per-second (Mb/s) or gigabits-per-second (Gb/s). The Fibre Channel market measures data rates in both MB/s and Gb/s, so you must be able to quickly translate between both units of measure. In Fibre Channel, 100MB/s equals 1Gb/s. Note that this conversion assumes that each byte equals 10 bits. This is actually true—Fibre Channel uses a bit encoding scheme in which each 8-bit byte is encoded as 10 bits for transmission.
Up to 30 m/link copper
Link Distance
1.5–25 m
Up to 10 km/link optical
Software
SCSI, IP, ESCON,
SCSI
Protocols
VI, etc

14. Fibre Channel vs. Other Networks
Ethernet
ATM
Fibre Channel
Bandwidth
MB/s
1.5, 6, or 25 MB/s
MB/s
Mode
Half or Full Duplex
Full Duplex
Full Duplex
Average Cont.
Data Flow
~ 40%
~40-85%
~ 95%
Maximum No.of Nodes
Arbitrated Loop: 126
Millions
Millions
Fibre Channel vs. Other Data Networks
Objective
Compare FC with other data networks.
Introduction
This section compares the characteristics of FC technology with those of other data networks.
Facts
The table compares the characteristics of FC to those of Ethernet and ATM, two other common networks.
One notable difference is in the Average Continuous Data Flow of each network. This figure represents how well the different technologies utilize their link bandwidth, and is stated as a percentage of the maximum link bandwidth. Both Ethernet and ATM typically have significant system overheads in processing the data from high speed links, so the Average Continuous Data Flow is typically far less than the maximum bandwidth. FC, however, maximizes efficiency by implementing many functions in hardware instead of in its software drivers, and is able to achieve an Average Continuous Data Flow of up to 95 percent of maximum bandwidth.
Note that all link distances stated here are according to the specifications. Many vendors support longer distances. For example, Finisar sells long-wave GBICs that support up to 30km on single-mode optical fiber.
FC and Gigabit Ethernet support similar link distances. However, IP is the most common protocol used on Ethernet networks, and there is a global WAN infrastructure that supports IP, so people tend to think of Ethernet as spanning longer distances than FC. FC’s “distance barrier” is not its physical specification but its incompatibility with the global IP infrastructure. Today, however, emerging technologies like FCIP allow IP networks to carry FC data, breaking down that distance barrier.
Switched Fabric: 16M
Typical No. Connected
1000’s
1000’s
10’s to 100’s
100m copper
5km optical
30m copper
10km optical
Link Distance
>100km
Protocols
Multiple protocols
Multiple protocols
Multiple protocols

15. Fibre Channel vs. Other Networks (cont.)
1995
1997
1999
2001
2003
2005
2007
2009
2011 10 20 30 40 50 60 70 80 90
100
Fibre Channel
Ethernet: 1Gb/s (2002)
Ethernet
Gb/s
Fibre Channel: 2Gb/s (2002)
Facts
Modern FC components support up to 2Gb/s of bandwidth in each direction:
The first generation of FC components, available commercially in 1995, operated at 1Gb/s (100MB/s in each direction)
2Gb/s (200MB/s) FC components became available in 2001
10Gb/s (1000MB/s) FC is planned for 2003 or 2004
FC and Ethernet offer similar link rate roadmaps:
Gigabit Ethernet, released in 2001, operates at 1Gb/s
The 10Gb Ethernet specification was completed in 2002, and 10Gb Ethernet products are expected to appear in 2003 or 2004
As the preceding graph shows, Fibre Channel and Ethernet are nearly parallel in the bandwidth race.
Although the published link rates are very similar for FC and Ethernet, 1Gb/s FC typically provides more usable bandwidth than 1Gb/s Ethernet, because FC has less overhead, lower latency, and more effective congestion control than Ethernet. New IP storage technologies such as Internet SCSI (iSCSI) are driving the development of more efficient Ethernet components, such as Ethernet adapters with TCP Offload Engine (TOE) chips, but FC’s underlying design still gives FC a lead on performance.

16. Lesson Review
Fill in the labels with the maximum link speeds supported by products available today.
320MB/s
SCSI Tape Library
200MB/s
FC-SCSI Bridge
SCSI
Practice
1. Fill in the labels in the diagram with the maximum link speeds supported by modern products. FC
Fibre Channel Fabric FC
FC Host
IP Network
Host IP FC FC FC
FC RAID Array
1000MB/s
200MB/s

17. Lesson Review (cont.)
Comparing Fibre Channel to SCSI, which offers the highest possible link rate?
Comparing 1Gb/s Fibre Channel to 1Gb/s Ethernet, which typically provides the highest actual throughput?
Compared to other networks, Fibre Channel offers the greatest advantage with respect to which features?
2. Comparing FC to SCSI, which offers the highest possible link rate?
a. FC
b. SCSI
c. No difference
3. Comparing 1Gb/s FC to 1Gb/s Ethernet, which typically provides the highest actual throughput?
b. Ethernet
4. Compared to other networks, FC offers the greatest advantage with respect to which features?
a. Protocols supported
b. Link distance
c. Efficient use of bandwidth
d. Maximum number of nodes

18. Summary
SCSI uses a parallel, shared-bus, multidrop topology that limits cable lengths and complicates management.
Serial networks can support longer distances than parallel buses because the skew effect is not a factor on a serial link.
SCSI is a parallel, half-duplex, shared bus, whereas FC is a serial, full-duplex, packet-based network.
FC supports either 100 or 200MB/s transmission speed with up to 95% Average Continuous Data Flow.
FC links can span up to 10Km depending on the type of cabling used.
Summary: Fibre Channel Physical Architecture
In this lesson, you learned the advantages of FC technology compared to the characteristics and limitations of SCSI technology and other data networks.