1. Wide Area InfiniBand What it is, and why it is
Obsidian Strategics Inc.
David Southwell
ESCC/ Internet2 Winter 2008 Joint techs
Jan 22 Honolulu, HI 1 1

2. InfiniBand 101
Open standard for high bandwidth, low latency, low CPU overhead, highly scalable interconnect.
1X, 4X, 8X or 12X channels at 2.5, 5.0 or 10.0 Gbits/s per link (less 20% for 8B/10B encoding).
1X SDR = 2Gbits/s, 12X QDR = 96Gbits/s (silicon Q1'08) - today, mostly 8 or 16Gbits/s deployed.
Cut-through switches – 60ns latency, scaling to thousands of ports.
Open stacks for Linux (Windows) – Open Fabrics Alliance
Low CPU loading due to Remote Direct Memory Access (RDMA)‏
Lossless flow control using buffer-credit scheme – very efficient but limits link lengths!
Host Channel Adapter A
Host Channel Adapter B 2
Across IB
link
Update
credit
From user-space memory 4
Into user-space memory 3 1
TIP2008
Obsidian Strategics 2

3. Equipment for range-extended InfiniBand
Wide Area
- SDR 4X IB + dual GbE payloads
- Global reach over 10G WANs:
OC-192 or 10Gbase-LW/LR
SONET/ ATM/ GFP-F/ IPv6 POS
- 2-port IB switch or router modes
Campus Area
- 10/40km dark fiber (LR/ER optics)‏
- 2-port IB switch mode only
Secure Wide Area (functional prototype stage)‏
- Global reach over 10GbE WANs
- 2-port IB switch or router modes
- Line-rate AES-192 crypto
TIP2008
Obsidian Strategics 3

4. Applications
TIP2008
Obsidian Strategics 4