Presentation is loading. Please wait.

Presentation is loading. Please wait.

Moving to FDR and EDR Speeds and Exascale Computing

Similar presentations


Presentation on theme: "Moving to FDR and EDR Speeds and Exascale Computing"— Presentation transcript:

1 Moving to FDR and EDR Speeds and Exascale Computing
2010 Sonoma Cable Panel Moving to FDR and EDR Speeds and Exascale Computing Moderator: Rupert Dance Date: 3/15/2010

2 Panelists Cinch Cables FCI Mellanox Luxtera Molex Volex
Merrick Moeller: FCI Jim David: Mellanox John Monson: Luxtera Marek Tlalka: Molex Jay Neer: Volex Toney Chew:

3 Agenda InfiniBand Road Map Current Market conditions Connectors Cables
Specification & Design Considerations Deployment Issues Compliance and Interoperability Testing

4 InfiniBand Roadmap

5 Current Market conditions
Connectors MicroGiGaCN (aka CX4) – 4X SDR and DDR QSFP 4X SDR, DDR and QDR, 8X QDR and 12X QDR CXP – 12X QDR Cables using QSFP connectors Passive Copper – up to 7 meters in length Active Copper – up to 15 meters in length Active Fiber – up to 300 meters in length Multi-Mode Fiber cables up to 100 meters Single-Mode Fiber cables up to 4 kilometers Gauge – copper cables 24 AWG for max length 32 AWG minimum gauge – used in 3 meters and less Marek Multi-Mode Fiber cables up to 100 meters Single-Mode Fiber cables up to 4 kilometers      (note our cables can go up to 4kmn but system buffering for latency limits the reach to under 1KM due to latency)

6 Connectors - FDR (14G) & EDR (25G)
QSFP Connectors Can QSFP support FDR Can QSFP support any higher speeds CXP Connectors Can CXP support FDR Can CXP support EDR 4x Connectors for EDR What is required for 4X connectors to support 25G speeds

7 Cable requirements for EDR
Cable design goals Length supported Gauge supported Loss Budget Cross talk Insertion Loss Return Loss Manufacturing issues Product Mix Copper versus Fiber Active versus passive Jim: I think the copper raw cable manufacturers will struggle to produce an EDR compatible 24 AWG cable. It will be very difficult to manufacture and, if it can’t be done, we’re looking at 26 AWG as the largest size which further limits the useful max length that a copper based cable can go.

8 Specification & Design Considerations
Convergence of IB and Ethernet specification Leverage volumes across markets for reduced cost IB specification for optical connectors for EDR Current IB spec does not cover optical spec Currently working with OIF Compatibility and legacy support 8b/10b versus 64b/66b What laser technology will be used for EDR VCSEL vs. EML lasers Link Integrity Issues DDR and QDR - lessons learned Allocating overall loss budget Attenuation based tuning Importance of EEPROM programming Optical Internetworking Forum (OIF)

9 Deployment Issues Power requirements Cooling requirements
Exascale Computing with 10,000+ nodes Cable management Distance limitation Other Considerations Cost 2011 EDR availability

10 Compliance and Interoperability
Compliance test methodology TDR versus VNA WDP – pros and cons IEEE 802.3ba System Interoperability testing End to end testing Switch to Switch testing Short trace and long trace testing Heterogeneous testing MPI based Interop testing

11 CXP Compliance Test Boards
- Module Compliance Board (MCB) - (1) Req’d if testing with HCB - (2) Req’d if testing with Cables - Host Compliance Board (HCB) 11

12 Cable Sponsors of the OFA Workshop

13 Mellanox Vendor Presentation

14 Addressing High Bandwidth Cable Challenges
Author: John Monson Date: March 15, 2010

15 Challenge – Cabling at 14Gb/s
Today’s QSFP suitable for FDR Target lengths Passive copper up to 5 meters Active copper up to 10 meters Optical up to 100 meters Active cable power similar to QDR Cable plants will look similar to today’s QDR clusters

16 Challenge – Cabling at 26Gb/s
Enhanced QSFP required for EDR Need to improve insertion loss Optimize impedance matching Target lengths Passive copper up to 3 meters Active copper up to 7-10 meters Optical up to 100 meters Power targets not significantly higher than QDR More clusters will now require active cables

17 Total Solution Required
I/O drivers must consider media type and length Transmitter must compensate for signal degradation over copper Receiver must compensate with increased sensitivity EDR will require better PCB material Switch traces up to 17 inches

18 Molex Vendor Presentation

19 InfiniBand Interconnect Infrastructure
Scaling the Current Ecosystem to 25G Jay Neer Strategic Product Manager – Industry Standards March 14, 2010

20 The Highest Density I/O Solutions
QSFP+ and CXP The Highest Density I/O Solutions RJ Channels 480 Gb/s SFP+ 48 Channels MicroGiga 112 Ch. 1120Gb/s iPass 160 Channels 1600 Gb/s QSFP+ 144 Channels 1440 Gb/s MicroGiga – 192 Ch. 960 Gb/s CXP – 384 Channels 3840 Gb/s 20

21 High Density InfiniBand Switch Qlogic – SC09
288 QSFP+ Ports 21

22 4x QSFP+ Family of Products
Single SMT Connector Single Cage Ganged Cages Stacked, Ganged Cages Light Pipe Options Thermal Options EMI Solution Options Direct Attach Copper Cables Copper SFP+ Breakout Cables Copper Loopbacks 12 Fiber MTP Optical Cables Optical LC Breakout Cables Optical Loopbacks

23 4x QSFP+ Family of Products
- The QSFP+ optical cable interface is defined as the industry standard 12 fiber MTP connector - Round bulk cable provides improved cable management within a rack - 3mm diameter cable for short runs - 4.5mm diameter for runs up to 25 meters - The cable assembly utilizes the exterior 4 fibers on each side of a 12 fiber ribbon cable; bulk cable with the center 4 fibers missing is also acceptable - Breakout cables to LC cables are used to connect to SFP+ applications 23

24 Sun Constellation System World’s Largest InfiniBand Switch
3,456 InfiniBand Channels per System with CXP 12x I/O’s 24

25 12x CXP Family of Products
Ecosystem Shipping: Single Integrated Connector Stacked Integrated Connectors Light Pipe Options Thermal Options EMI Solution Options Direct Attach Copper Cables Copper QSFP+ Breakout Cables Copper Loop-backs 24 Fiber MTP Optical Cables Optical QSFP+ Breakout Cables 25

26 12x CXP Family of Products
CXP / CXP CXP / MicroGiga 12x CXP / QSFP+ CXP / MicroGiga 4x 26

27 12x CXP Family of Products
CXP / QSFP CXP / CXP Pluggable Optical Module w/ Passive Optical Cable Assembly 27

28 12x CXP Family of Products
Thermal Solutions Single Port w/out Top Cover Access area to Top of Plug Heat Sinks Optional & Application Dependent - Can be extended to the rear or to the sides - Can be square or round pins or fins - Heights based on application Clip-on Cover w/vent holes Standard Capable of cooling W Devices 28

29 CXP Development / Compliance / Test Boards
- Module Compliance Board (MCB) - (1) Req’d if testing with HCB - (2) Req’d if testing with Cables - Host Compliance Board (HCB) 29

30 InfiniBand Roadmap Plenty of time to get to 25G 30


Download ppt "Moving to FDR and EDR Speeds and Exascale Computing"

Similar presentations


Ads by Google