Chiaro’s Enstara™ Summary Scalable Capacity –6 Tb/S Initial Capacity –GigE  OC-192 Interfaces –“Soft” Forwarding Plane With Network Processors For Maximum Flexibility Full protocol suite –Unicast: BGP, OSPF, IS-IS –Multicast: PIM, MBGP, MSDP –MPLS: RSVP-TE, LDP, FRR Stateful Assured Routing (STAR™) –Provides Service Continuity During Maintenance and Fault Management Actions –Stateful Protocol Protection Extended to BGP, ISIS, OSPF, Multicast, and MPLS Partitions –Abstraction Permitting Multiple Logical Classes Of Routers To Be Managed As If Separate Physical Routers –Each Partition Has Its Own CLI, SNMP, Security, and Routing Protocols Instances

We Chose OptIPuter for Fast Switching and Scalability Lithium Niobate ElectricalFabrics Bubble Packet Switching Speeds (ns) Switching Speeds (ms) Small Port Count Large Port Count ChiaroOpticalPhasedArray MEMS MEMS ElectricalFabrics

Input Optical Fiber WG #1 WG #128 Optical Phased Array – Multiple Parallel Optical WaveguidesOutputFibers Air Gap GaAs Waveguides

Chiaro Has a Scalable, Fully Fault Tolerant Architecture Significant Technical Innovation –OPA Fabric Enables Large Port Count –Global Arbitration Provides Guaranteed Performance –Fault-Tolerant Control System Provides Non-stop Performance Smart Line Cards –ASICs With Programmable Network Processors –Software Downloads For Features And Standards Evolution NetworkProc.LineCard NetworkProc.LineCard GlobalArbitration OpticalElectrical Chiaro OPA Fabric NetworkProc.LineCard NetworkProc.LineCard

Cluster – Disk Disk – Disk Viz – Disk DB – Cluster Cluster – Cluster OptIPuter LambdaGrid Enabled by Chiaro Networking Router switch Medical Imaging and Microscopy Chemistry, Engineering, Arts San Diego Supercomputer Center Scripps Institution of Oceanography Chiaro Enstara Image Source: Phil Papadopoulos, SDSC

½ Mile The UCSD OptIPuter Deployment SIO SDSC CRCA Phys. Sci - Keck SOM JSOE Preuss 6 th College Phase I, Fall-Winter 2002 Phase II, 2003 SDSC Annex Collocation point Node M The OptIPuter Experimental UCSD Campus Optical Network Earth Sciences SDSC Arts Chemistry Medicine Engineering High School Undergrad College SDSC Annex To CENIC Collocation Chiaro Router (Installed Nov 18, 2002) Production Router (Planned) Source: Phil Papadopoulos, SDSC; Greg Hidley, Cal-(IT) 2 Roughly, $0.20 / Strand-Foot UCSD New Cost Sharing Roughly $250k of Dedicated Fiber

UCSD Chiaro Estara Full Layer-3 Router Operating at 10ns –Chiaro Estara Router Chassis –Installed November 18, 2002 –24 1 GigE Network Interfaces –Spring 2003 Expandable to: –320 x 10 GigE Network Interfaces = 6.4 Terabit/s –All Capable Or Concurrent Routing At Full Line Speed Webcam –

The Center of the Network

Status as of about 15 minutes ago The hardest thing in networking – getting fiber in the ground and terminated –4 pair single mode fiber per site –Fiber terminated Monday (3 days ago) – fiber “polishing” still ongoing –First light/first packet yesterday (wed) at about 6pm –Ran linpack at about 6:05pm Currently two sites are connected –Linux Cluster at each site as a baseline. Sizes/capability/architecture vary as needed (and $$) change. –4 x 1 CSC –4 x 1 SDSC Chiaro –Serial #1 production router. “Tiny size”: –One redundant pair of optical cores – 640 gigabits. Can grow to 6+ Terabits –We have alpha/beta gigE blades from Chiaro –OptIPuter getting these 3 – 4 months ahead of schedule –Chose multiple gigE striped physically because –Cost at the site end –Start parallel –From the endpoint view – Chiaro works just like a standard router.

Netperf Numbers (data at first light) 2 streams  more than 1 gigabit (aggregate)

Linpack Numbers 4 processor Linpack –Run through local copper gigE switch –Run through Chiaro Router