Presentation is loading. Please wait.

Presentation is loading. Please wait.

Flow Control in Fibre Channel (BB_Credits)

Similar presentations


Presentation on theme: "Flow Control in Fibre Channel (BB_Credits)"— Presentation transcript:

1 Flow Control in Fibre Channel (BB_Credits)
All presenters should use this section (Notes Pages) to provide additional information and details about the bullet points above. These notes will help remind students of key points when reviewing materials in their office. It will also help students conduct more standardized and effective train-the-trainer sessions when they return to their offices. Lincoln Dale

2 Topology Hosts connected to a single switch
Storage connected to a second switch Switches connected to each other Hosts SAN Switches Storage FC

3 Buffer Credits (BB_Credits) at FC Class 3
BB_Credits are the “admission control” mechanism in FC to ensure that FC switches don’t run out of buffers (FC Switches cannot drop frames) For Devices operating at FC Class 3 (most devices), Buffer Credits are negotiated on a per-hop basis at device FLOGI. BB_Credits are the only flow-control mechanism for FC Class 3. 11 12 available BB_Credits 15 16 15 16 available BB_Credits available BB_Credits FC 12 available BB_Credits 16 Frame towards Disk shelf available BB_Credits Return BB_Credit token

4 Buffer Credits: The Good and The Bad
Since BB_Credits are per-hop for FC Class 3: Any speed mismatch (1G/2G) between senders & receivers and/or Insufficient numbers of BB_Credits in a device can result in Congestion due to Head of Line (HoL) Blocking This Congestion can impact other ports & devices in a SAN FC Disk shelf capable of sustaining 200MB/sec Tape capable of sustaining 15MB/sec

5 Buffer Credits (BB_Credits): Utopia
DO NO DELETE THIS STUFF OFF THE LEFT OF THE SLIDE. IT IS USED FOR THE ANIMATIONS!! Buffer Credits (BB_Credits): Utopia Frames to switchA 1 1 1 1 2 2 2 2 3 3 3 3 In an ideal FC network, there is no blocking in any device connected to the fabric. (all devices can process frames at the same rate and negotiate equal levels of BB_Credits) 4 4 4 4 5 5 5 5 6 6 6 6 FC 7 7 7 7 8 8 8 8 Disk shelf capable of sustaining 200MB/sec Frames to device Frames across switchA 12/16 16/16 13/16 14/16 15/16 11/16 1 1 1 1 1 1 Available BB_Credits 2 2 2 2 2 2 13/16 16/16 10/16 11/16 8/16 7/16 15/16 12/16 14/16 9/16 Available BB_Credits 3 3 3 3 3 3 4 4 4 4 4 4 5 5 5 5 5 5 11/16 14/16 15/16 13/16 12/16 16/16 6 6 6 6 6 6 Available BB_Credits 7 7 7 7 7 7 Tape capable of sustaining 15MB/sec 8 8 8 8 8 8 Frame towards Disk shelf Frame towards Tape / Return BB_Credit token

6 Buffer Credits (BB_Credits): Real world impact on performance
DO NO DELETE THIS STUFF OFF THE LEFT OF THE SLIDE. IT IS USED FOR THE ANIMATIONS!! Buffer Credits (BB_Credits): Real world impact on performance Frames to switchA 1 1 1 1 2 2 2 2 In reality, not all devices can process frames at the same rate and different devices negotiate different BB_Credits This means that a single device is capable of causing HOL blocking across a FC switch or even across a SAN! 3 3 3 3 Frames to other devices/ports backlog behind blocked Frame at Head-of-Queue 4 4 4 4 5 5 5 5 6 6 6 6 FC 7 7 7 7 7 Frames Blocked 2 Frames Blocked Uncongested Fabric 8 Frames Blocked 1 Frame Blocked 3 Frames Blocked 6 Frames Blocked 4 Frames Blocked 5 Frames Blocked 8 8 8 8 Disk shelf capable of sustaining 200MB/sec Frames to device Frames across switchA 15/16 14/16 13/16 16/16 1 1 1 1 1 1 Available BB_Credits 2 2 2 2 2 2 11/16 12/16 16/16 14/16 13/16 7/16 15/16 5/16 8/16 9/16 6/16 10/16 While this example isn’t perhaps the most realistic case (flooding FC frames at a device with only 2 BB_Credits), it does show the ramifications of Fibre Channel standards mandating “frames must not be dropped”. Available BB_Credits 3 3 3 3 3 3 FC Frame blocks at head of Ingress port due to 0 available BB_Credits on egress port 4 4 4 4 4 4 5 5 5 5 5 5 2/2 1/2 0/2 6 6 6 6 6 6 Available BB_Credits 7 7 7 7 7 7 Tape capable of sustaining 15MB/sec 8 8 8 8 8 8 Frame towards Disk shelf Frame towards Tape / Return BB_Credit token

7 Buffer Credits (BB_Credit) impact on performance
While Head-of-Line Blocking is a transitory event (until some BB_Credits are returned on the blocked port), performance can be adversely affected across an entire multi-switch FC Fabric by a single blocking port The Cisco MDS 9000 Series has multiple features to help alleviate the problem: Virtual Output Queueing (VoQ) on all ports Deep Buffers – 255 BB_Credits per port Fibre Channel Congestion-Control (FCC) – Detects congested ports and throttles the port causing the congestion at its origin

8 Virtual Output Queueing (VoQ)
In a typical FC switch, each port has an input queue for frames arriving on a port All frames, regardless of destination, are queued in the order they’re received This can potentially cause Head-of-Line blocking should one of the destination ports become congested Frames will begin to backup in the queue and the sourcing devices will eventually have to stop transmitting data. Switch without VoQ Frame to Port 5 Frame to Port 6 Frame to Port 4 Input Queue at Port 1 Top of Queue

9 Virtual Output Queueing (VoQ)
Virtual Output Queuing Model Frame to Port 5 Frame to Port 6 Frame to Port 4 Input Queue at Port 1 Top of Virtual Queue The MDS 9000 series utilizes Virtual Output Queues (VoQ) to eliminate Head-of-Line blocking. Each Destination Port is given its own Virtual Queue on the Input Port. Congestion on any Destination Port has no effect on traffic destined for other ports. Every Source Port on the MDS 9000 series has four Virtual Output Queues for every Destination Port – to cover 4 QoS levels per-port across a chassis

10 Deep Buffers – 255 Buffer-to-Buffer Credits (BB_Credits) per port
Typical rule-of-thumb is: 2 BB_Credits are required for every 1gbit/s 4 BB_Credits are required for every 2gbit/s Other Fibre Channel switches in the marketplace support up to 60 Buffer-to-Buffer Credits (BB_Credits) Performance is limited to 2gbit/s maximum over <15km Performance is limited to 1gbit/s maximum over <30km The MDS 9000 Series has up to 255 Buffer-to-Buffer Credits (BB_Credits) per port Wire-rate (2gbit/s) achievable for 64+km 1gbit/s achievable for 127+km

11 Fibre Channel Congestion Control (FCC)
Consider the following topology: Host A is issuing write operations to 100MB/sec Host B is issuing writes operations to 50MB/sec Congestion on Switch B’s port connecting the tape can cause a performance degredation for Host A writing to Disk FCC mitigates the congestion by throttling the traffic at the originating port 1. Switch B detects congestion on port connecting to Tape FC Disk shelf capable of sustaining 200MB/sec 2. Switch B signals Switch A to quench the initiator A 3. Switch A rate-limits incoming traffic Switch A Switch B Congestion Congestion Tape capable of sustaining 15MB/sec Port Rate-Limited on VoQ Limit HostBTape to 15MB/sec B

12 12 © 2001, Cisco Systems, Inc. All rights reserved.


Download ppt "Flow Control in Fibre Channel (BB_Credits)"

Similar presentations


Ads by Google