Ordered Sets Ordered Sets Introduction Fibre Channel (FC) defines a few simple control signals to perform frequently required control functions, such as marking the start and end of frames and maintaining link synchronization. These simple control signals are called Ordered Sets. This lesson describes the role of Ordered Sets in FC, and the purpose of each Ordered Set. Importance Ordered Sets are special FC data units that are used in critical control functions, including flow control. You need to be able to recognize the Ordered Sets and understand the function of each Ordered Set before you can learn about functions such as flow control. In addition, being able to quickly recognize Ordered Sets will allow you to effectively use protocol traces to identify and resolve problems on a SAN.

Lesson Objective Upon completion of this lesson, you will be able to describe the role of Ordered Sets in Fibre Channel and recognize some of the most commonly used Ordered Sets. Performance Objective Upon completion of this lesson, you will be able to describe the role of Ordered Sets in FC and recognize some of the most commonly used Ordered Sets. Enabling Objectives Define an Ordered Set State the purpose of the K28.5 character Identify the three types of Ordered Sets State the purpose of Frame Delimiters State the purpose of Primitive Signals State the purpose of Primitive Sequences

Outline What are Ordered Sets? The K28.5 Character Types of Ordered Sets What are Frame Delimiters? What are Primitive Signals? What are Primitive Sequences? Prerequisites All previous lessons in Curriculum Unit 2, Module 1.

What are Ordered Sets? An Ordered Set is a command signal that: Is a single word Can be sent outside of a frame Is not addressable What are Ordered Sets? Objective Define an Ordered Set Introduction This section defines the concept of an Ordered Set. Definition When a node transmits data to another node, FC packages that data into frames. Frames are used to transport both data and FC control signals. However, there are a few simple command signals that ports must use frequently to perform basic link synchronization and flow control functions. Wrapping an entire frame around each one of these command signals would waste bandwidth and increase latency. In FC, the smallest unit of data is the word, which is a 4-byte unit of data. All of the data that flows across an FC network—frame headers, control signals, and even the application data that is being sent from node to node—consists of a series of 4-byte words. FC designates special words to be used as simple command signals. These signals are called Ordered Sets. Facts Ordered Sets do not have to be encapsulated in a frame. Note, however, that the four bytes in an Ordered Set do not provide space for addressing or routing information. Therefore, Ordered Sets that are sent outside of a frame can only be used for link management functions between two directly connected ports. Example The preceding graphic shows an example of an Ordered Set, displayed in green, that occurs outside of a frame. Frames Ordered Set

The K28.5 Character Frames Ordered Set Byte Byte Data Word Ordered Set Objective State the purpose of the K28.5 character Introduction This section explains the relevance of the K28.5 character to Ordered Sets. Fact The 8b/10b encoding scheme provides 1024 transmission characters. The 8b/10b encoding scheme reserves some of these characters as special characters that differentiate control information from data. In FC, a special character called the K28.5 character is used to distinguish FC control information from data. An Ordered Set consists of a single transmission word that starts with the K28.5 special character. When a port receives the K28.5 character, it knows that the three bytes that follow are part of an Ordered Set. The second byte identifies the Ordered Set, and the third and fourth bytes identify subtypes of that Ordered Set. The K28.5 character is also known as the “comma character.” Byte Byte K28.5 Data Word Ordered Set

Types of Ordered Sets Transmission Word Ordered Set Data Word K28.5, Dxx.y, Dxx.y, Dxx.y Data Word Dxx.y, Dxx.y, Dxx.y, Dxx.y Primitive Signal Types of Ordered Sets Objective Identify the three types of Ordered Sets Introduction This section introduces the three types of Ordered Sets. Facts There are three types of Ordered Sets: Frame Delimiters are used to mark the beginning and end of frames. Primitive Signals are used to initiate, synchronize, and terminate communication sessions, and to maintain synchronization when no other information is being transmitted on the link. The two types of Primitive Signals are fill words and control signals. Primitive Sequences are similar to Primitive Signals, but are transmitted repeatedly until a response is received. They are used for link and loop initialization. Frame Delimiter Fill Word Control Signal Primitive Sequence Start-of-Frame End-of-Frame Idle Arbitrate Receiver Ready Virtual Circuit Ready Close Open Dynamic Half-Duplex Mark Synchronize Non-Operational State Offline State Link Reset Link Reset Response Loop Initialization Loop Port Bypass Loop Port Enable

What are Frame Delimiters? Frame Delimiters mark the beginning and end of frames SOF Start of Frame EOF End of Frame What are Frame Delimiters? Objective State the purpose of Frame Delimiters Introduction This section explains the type of Ordered Set called Frame Delimiters. Definition Frame Delimiters are Ordered Sets that are used to mark the beginning and the end of each frame. Facts There are two Frame Delimiters: Start-of-Frame (SOF) is used to mark the beginning of a frame and also indicates whether this is the first frame in a sequence and which class of service is in operation. End-of-Frame (EOF) is used to mark the end of a frame and also indicates whether this is the last frame in the sequence and which class of service is in operation. EOF has both negative disparity and positive disparity variants to ensure that every frame ends with negative running disparity. Example The preceding diagram shows an example of the SOF and EOF frame delimiters that mark the beginning and end of a frame. S O F E O F Header Payload CRC Frame

What are Frame Delimiters? (cont.) SOFix SOF Initiate Class x SOFnx SOF Normal Class x EOFn EOF Normal EOFt EOF Terminate EOFa EOF Abort Facts There are multiple subtypes of both SOF and EOF. These subtypes are used to convey information about the status of the frame. For example: SOFix (SOF Initiate) precedes the first frame of a sequence; x is a number that indicates the Class of Service. SOFnx (SOF Normal) precedes frames other than the first frame of a sequence; x is a number that indicates the Class of Service. EOFn (EOF Normal) concludes a frame that is not the last frame of a sequence. EOFt (EOF Terminate) concludes a frame that is the last frame of a sequence. EOFa (EOF Abort) concludes a frame that was aborted abnormally during transmission. EOFt SOFn3 EOFn SOFn3 EOFn SOFi3 Class 3 Sequence

Frame Delimiters (cont.) Example The image illustrates a captured analyzer trace that displays a single Class 2 frame and its SOF and EOF Frame Delimiters. Note to Developer This screencap should be replaced with a screencap from Cisco Fabric Analyzer.

What are Primitive Signals? Fill words maintain link synchronization IDLE Idle—sent between frames ARB Arbitrate for loop (FC-AL) What are Primitive Signals? Objective State the purpose of Primitive Signals Introduction This section explains the type of Ordered Set called Primitive Signals. Definition Primitive Signals are Ordered Sets that are used to signal events at the transmitting port. Primitive Signals are used for flow control and clock synchronization. On arbitrated loops, Primitive Signals are also used for virtual circuit management. Example The preceding graphic illustrates an FC sequence consisting of a frame preceded and followed by IDLE fill words. Facts: Fill Words To maintain synchronization, the FC-1 layer generates continuous transmissions over each link. This minimizes latency and reduces the rate of errors on the link. These continuous transmissions consist of fill words, which are a type of Primitive Signal. There are two fill words: The IDLE signal assists with clock synchronization and retiming. At least six idles are transmitted between frames, and additional idles may be added between frames to assist in synchronization and retiming. Idles are sent continuously when no frames are being transmitted to maintain synchronization. The ARB (Arbitrate) signal is used only on arbitrated loops. ARB is part of the mechanism that loop ports use to decide which port has control over the loop at any particular time. S O F E O F IDLE IDLE Header Payload CRC IDLE IDLE Frame

What are Primitive Signals? (cont.) Control signals are used in flow control R_RDY Receiver Ready OPN Open Port (FC-AL) CLS Close Port (FC-AL) SYN Clock Synchronize VC_RDY Virtual Circuit Ready DHD Dynamic Half-Duplex Facts: Control Signals In addition to fill words, there is another type of Primitive Signal called control signals. Control signals are simple commands that are sent between ports primarily to perform flow control functions. The preceding graphic displays an example of the Primitive Signal R_RDY. The target port sends R_RDY to inform the initiator that the target port has a free buffer. The control signals defined by the FC specification include: R_RDY (Receiver Ready) is sent by a port when the port is ready to receive data. R_RDY is used in buffer-to-buffer flow control. OPN (Open Port) is sent by an initiator on an arbitrated loop when the initiator wants to open a connection with a target port. CLS (Close Port) is sent by an initiator on an arbitrated loop when the initiator wants to close a connection with a target port. SYN (Clock Synchronization) is used to maintain clock synchronization in point-to-point and switched fabric configurations. VC_RDY (Virtual Circuit Ready) is used for flow control in Class 4 virtual-circuit connections. DHD (Dynamic Half-Duplex) is transmitted to change a full-duplex transfer to a half-duplex transfer between two L_Ports on an arbitrated loop. FC FC Frame R_RDY Initiator Target

Primitive Signals (cont.) Example The image displays a sample analyzer trace taken from an FC-AL. The trace shows examples of the Primitive Signals R_RDY, IDLE, CLS, ARB, and OPN. The lower window displays a detailed view of the highlighted item in the upper window—in this case, the R_RDY Ordered Set. Note that Byte 0 of the R_RDY ordered set is the K28.5 character. Note to Developer This screencap should be replaced with a screencap from Cisco Fabric Analyzer.

What are Primitive Sequences? Primitive Sequences are used for link control: They are transmitted repeatedly until a response is received Ports must receive three consecutive primitives before responding What are Primitive Sequences? Objective State the purpose of Primitive Sequences Introduction This section explains the type of Ordered Set called Primitive Sequences. Definition Primitive Sequences are another type of Ordered Set. Primitive Sequences are simple link-level protocols that are used for link control. The difference between Primitive Signals and Primitive Sequences is that Primitive Signals are transmitted once, while Primitive Sequences are transmitted repeatedly until a response is received. Ports are required to receive a minimum of three consecutive occurrences of the same Primitive Sequence before acknowledging its validity and responding accordingly.

What are Primitive Sequences? (cont.) NOS Non-Operational State OLS Offline State LR Link Reset LRR Link Reset Response LIP Loop Initialization Primitive LPB Loop Port Bypass LPE Loop Port Enable Used on fabrics Used on loops Example The preceding graphic shows an example of the NOS-OLS Primitive Sequence exchanged between a host N_Port and a switch F_Port. Facts The Primitive Sequences are: NOS (Non-Operational State) and OLS (Offline State). NOS is sent by a fabric port that is in an offline condition. The receiving port responds with OLS to begin the link initialization process. LR (Link Reset) and LRR (Link Reset Response). LR is sent by a fabric port that wants to initiate a link reset operation; the receiving port responds with LRR. LIP (Loop Initialization). LIP is sent by a loop port to begin the loop initialization process. LPB (Loop Port Bypass), and LPE (Loop Port Enable). When a port receives LPB, it places itself in bypass mode. While in bypass mode, the port retransmits all received transmission words but does not participate in loop operations or process data. LPE re-enables a bypassed port. FC NOS OLS Switch Host

Primitive Sequences (cont.) Example The image shows an analyzer trace that displays the initial bootstrap sequence between an N_Port and F_Port in a switch. The top of the trace displays the OLS, LR, LRR link initialization sequence during login. Note that the link initialization sequence is followed by a series of IDLEs to establish clock synchronization before the Fabric Login (FLOGI) command sequence begins. Note to Developer This screencap should be replaced with a screencap from Cisco Fabric Analyzer.

Lesson Review Practice Identify the Ordered Sets displayed in the screen image. State the type of each Ordered Set and identify its function. Note to Developer This screencap should be replaced with a screencap from Cisco Fabric Analyzer.

Summary Ordered Sets are single-word control signals that are sent outside of frames The K28.5 character is used to distinguish Ordered Sets from data words The three types of Ordered Sets are: Frame Delimiters: Mark the beginning and end of frames Primitive Signals: Initiate, synchronize, and terminate communication sessions; maintain link synchronization Primitive Sequences: Link control; transmitted repeatedly (at least 3 times) until a response is received Summary: Ordered Sets In this lesson, you learned how to describe the role of Ordered Sets in FC and recognize some of the most commonly used Ordered Sets, including Frame Delimiters, Primitive Signals, and Primitive Sequences.