1. May 2002© 2002 Yoram Ofek1 Satisfying the Requirements of Applications on a Single Packet Network Yoram Ofek Synchrodyne Networks, Inc. E-mail: ofek@synchrodyne.com Phone: (917) 601-7180

2. May 2002© 2002 Yoram Ofek2 Applications - Generic Traffic Types Playback:Machine-to-Person SinglePacketNetwork Person-to-PersonCommunications Typically with Rate Machine-to-MachineCommunications Typically No Rate

3. May 2002© 2002 Yoram Ofek3 Person-to-Person: with Rate ä Applications with some notion of rate: ä Most demanding: interactive - streaming media - voice/video ä end-to-end delay < 100 ms ä continuous play - i.e., periodic Machine-to-Person  Will satisfy also: non-interactive: playback, large file transfers - Machine-to-Person ä The transition from circuit to packet switching the rate per person will increase 3-4 orders of magnitude: ä from 10 4 b/s to 10 8 b/s

4. May 2002© 2002 Yoram Ofek4 Machine-to-Machine: No Rate ä (Computing) Machines are still evolving rapidly ä e.g., capabilities: “Moore’s Law” - new applications ä General characteristic: bursty - unpredictable in time/space ä All bits should be transferred correctly with no “shaping”: ä max. throughput (burst) AND min. delay & loss ä e.g., distributed/parallel computing, data processing t Traffic shape at the source t Traffic shape at the Destination t Transfer with Minimum Distortions No“Shaping”No“Shaping”

5. May 2002© 2002 Yoram Ofek5 Outline ä How to support the two generic traffic types: ä 1. Ring networks ä 2. Convergence routing ä 3. Time-driven - switched networks ä 4. Dynamic optical networking Machine-to-Machine Person-to-Person Machine-to-Machine Person-to-Person Integration of Machine-to-Machine using UTC Person-to-Person

6. May 2002© 2002 Yoram Ofek6 Rings ä First token ring was introduced (e.g., IBM, FDDI) ä Why token rings? ä Can support: ä 1) Bursty data (asynchronous) with ä no rate, (no) loss, (low) latency, fairness, multicast ä 2) Periodic real-time ä with rate and delay guarantees, multicast Machine-to-Machine Person-to-Person

7. May 2002© 2002 Yoram Ofek7 Rings with Spatial Bandwidth Reuse ä Packet are removed at destinations: slotted or insertion ring ä Concurrent transmission ä Throughput grows with locality ä all nodes can transmit simultaneously to their neighbors 1 2 3 4 5 6 7 8 9 10 11 12

8. May 2002© 2002 Yoram Ofek8 ä SAT (token) gives predefined transmission quota ä Rotates in the opposite direction ä Held intermittently if the node is not SATisfied Node 1 IB Node 6 IB Node 3 IB Node 2 IB SAT IB - Insertion Buffer IB - Insertion Buffer MetaRing: Fairness with Spatial Bandwidth Reuse Slotted or insertion ring Node 5 Node 4

9. May 2002© 2002 Yoram Ofek9 MetaRing: SAT Fairness Properties ä Equal throughput after each SAT rotation - with multiple variants ä Multiple SATs operations for simple fault recovery ä SAT/SAT’ for graceful degradation to (multi) bus operation ä SAT signal provides for: ä Bounded delay with no loss of bursty data - ä Integration of real-time traffic with known rate - ä MetaRing is the underlying network for IBM storage area network (SAN) products (also ANSI SSA - X3T10 standard) Spatial reuse rings are currently very active: Cisco SRP/DPT and IEEE 802.17 Multi-billion business for IBM Machine-to-Machine Person-to-Person

10. May 2002© 2002 Yoram Ofek10 Switched Network ä Is MetaRing panacea? NO

11. May 2002© 2002 Yoram Ofek11 Traffic with No Rate over Switched Network ä To transfer with max. throughput (burst) AND min. delay and loss t Traffic shape at the source t Traffic shape at the Destination t Transfer with Minimum Distortions No“Shaping”No“Shaping” ä TCP/IP: unstable/unpredictable throughput/delay/loss ä Cannot be done with over fixed routes (congestion and loss) ä Dynamic routing: ä e.g., “Hot-Potato” ( P. Baran ), Manhattan Street Network - deflection routing ( N. Maxemchuk )

12. May 2002© 2002 Yoram Ofek12 MetaNet Convergence Routing with Sense of Direction ä Invented by Yoram Ofek and Moti Yung ä Virtual ring embedding ä Link types: ä Ring - part of virtual ring/s ä Thread - all other links ä Embeddings methods - e.g.,: ä Simple Hamiltonian Circuit ä Euler tree traversal VN1 VN3 VN2 VN0 VN9 VN7 VN14 VN15 VN6 VN8 Sequential Numbering of Virtual Nodes: VN0, VN1, VN2, … AGF C B D E H I ä Multiple partial virtual rings

13. May 2002© 2002 Yoram Ofek13 ä Packet routing paradigm: ä 1. Packets are forwarded to idle output link “closer” to their destinations with: “sense of direction” - along virtual ring(s) ä 2. Virtual (buffer insertion) ring traffic gets priority to continue on the virtual ring links MetaNet Convergence Routing over Switched Network

14. May 2002© 2002 Yoram Ofek14 VN1 VN3 VN2 VN0 VN9 VN7 VN14 VN15 VN6 VN8 ä SHORT-CUT Routing: ä Example: packet arrives to VN6 on node C with destination H, can short- cut to VN8 ä Diametric routing in light load AGF C B D E H I Short-cut MetaNet Convergence Routing over Switched Network

15. May 2002© 2002 Yoram Ofek15 VN3 VN1 VN2 VN0 VN9 VN7 VN14 VN15 ä Broadcast-with-feedback: ä Requirements: ä asynchronous - without arbitration ä losslessness ä correctness ä complete coverage ä packet copied only once ä complete feedback to the source ä When short-cuts or jumps are possible the packets are DUPLICATED AGF C B D E H I MetaNet Convergence Routing over Switched Network

16. May 2002© 2002 Yoram Ofek16 ä Summary: ä Support traffic from bursty sources with no rates: ä No packet loss ä Bounded delay ä Fairness ä Broadcast and multicast (with feedback) However, still limitations: 1) on size - it is not a global network! 2) does not support person-to-person communications with known rates MetaNet Convergence Routing over Switched Network Machine-to-Machine Person-to-Person

17. May 2002© 2002 Yoram Ofek17 Outline ä How to support the two generic traffic types: ä 1. Ring networks ä 2. Convergence routing ä 3. Time-driven - switched networks ä 4. Dynamic optical networking Machine-to-Machine Person-to-Person Machine-to-Machine Person-to-Person Integration of Machine-to-Machine using UTC Person-to-Person

18. May 2002© 2002 Yoram Ofek18 Time-Driven Priority over Switched Network ä How to support communications with known rate on a global network? ä Flow (congestion) control methods: ä Rate control at the network’s boundaries - e.g., ATM (J. Turner) ä with statistical multiplexing inside the network ä Inside the network with local clocks scheduling - deadline scheduling (D. Ferari), GPS (A. Parekh, R. Gallager) ä Inside the network with scheduling based on global time: TIME-DRIVEN PRIORITY ä UTC - Coordinated Universal Time: TIME-DRIVEN PRIORITY ä Based on pipeline forwarding Person-to-Person

19. May 2002© 2002 Yoram Ofek19 Pipeline: optimal method - independent of a specific realization - successfully deployed with optimal efficiency in ä Factory (automotive), Computers (CPU) NOW pipeline in global networks! Thanks to GPS that provides UTC Pipeline: From Henry Ford to the Internet Time-of-day or UTC – coordinated universal time - with accuracy of  5  s 12 1000 Time Cycle0 12 1000 Time Cycle1 12 1000 Time Cycle 79 Super-cycle - UTC second with 80k Time-frames Time-of-Day or UTC 0 beginning of a UTC second 1 beginning of a UTC second Time Driven Priority

20. May 2002© 2002 Yoram Ofek20 Time-driven Priority - Forwarding t ii48 1212 Time Cycle t ii48 1212 Time Cycle i+1 i+2 Arbitrary Immediate 2-frame Arrive to OutputPort Forward from OutputPort 1. Immediate forwarding 2. 2-frame forwarding 3. Arbitrary forwarding h k k h kk k+1 2k k k

21. May 2002© 2002 Yoram Ofek21 Time-driven Priority for Videoconferencing Time driven priority videoconferencing with complex periodicity scheduling ä Face-to-face quality ä Scale the globe Time driven priority videoconferencing with complex periodicity scheduling ä Face-to-face quality ä Scale the globe Video- conferencing ä Sender-receiver synchronization Node C Receiver Node B Sender UTC from GPS Video Frame Capture Video Frame Display t real MPEG I picture MPEG MPEG MPEG P pictures MPEG MPEG ä Complex Periodicity Scheduling: The size of successive packets of the same flow changes in a repetitive manner

22. May 2002© 2002 Yoram Ofek22 ä IP and “Best Effort” service are unchanged ä Time-driven priority scales the globe: ä Jitter: bounded by 2*T f : ä Independent of the network size, traffic load, flow rate ä End-to-end delay: 2* h*Tf + prop. Delay ä No loss ä Can easily integrated with: ä MetaNet convergence routing ä Optimized for interactive streaming media Time-driven Priority - Summary Person-to-Person Machine-to-Machine

23. May 2002© 2002 Yoram Ofek23 ä Objective: to utilize UTC in the optical domain static optical ä In static optical networking all data units on the optical channel are switched in the same way while, dynamic optical ä In dynamic optical networking each data unit on the optical channel may be switched differently Fractional Switching for Dynamic Optical Networking

24. May 2002© 2002 Yoram Ofek24 Problems of Static Switching: N 2 ’s 5 s NYC LA SF SEA STL

25. May 2002© 2002 Yoram Ofek25 Save s & Grooming & Small or No Memory What: Fractional  Switching Save s & Grooming & Small or No Memory 1  5 Fractional  Pipes (F Ps) NYC LA SF SEA STL Number of s =

26. May 2002© 2002 Yoram Ofek26 IP Time-based Grooming and Degrooming IP/MPLS ADSL DSLAM (central office) Cable Modem Head-end Server Farm (web, VoD) Wireless Base Station Small  fractions   Switching Large   fractions Edge/Access Router (POP) Header processing only at the edges

27. May 2002© 2002 Yoram Ofek27 ä Pipeline forwarding of whole time frames ä No header processing ä Banyan based switch structure - optimal Dynamic: Fractional  Switching See pipeline forwarding - PF animation over FlPs 12 1000 Time Cycle0 12 1000 Time Cycle1 12 1000 Time Cycle 79 Super-cycle - UTC second with 80k Time-frames Time-of-Day or UTC 0 beginning of a UTC second 1 beginning of a UTC second

28. May 2002© 2002 Yoram Ofek28 Why: Dynamic: Fractional Switching The Optical Links are Memory UTC ä A mesh of linear delay lines ä How to preserve pipeline forwarding? ä Delay between switches = integer number of time frame

29. May 2002© 2002 Yoram Ofek29 Input 1 Optical Alignment Optical Switching Fabric Optical Alignment Input N Output 1 Output N t+1 Time-of-Day or UTC t-1t-2 t-3 t t+2 Idle time: Safety margin between two time frames Switch Controller Time-of-Day or UTC : Time frame payload – with a predefined number of data units : Time frame The Optical Links are the Memory

30. May 2002© 2002 Yoram Ofek30 Fractional λ Switching Properties ä Low time accuracy ä Microsecond with time frame delimiters ä Extremely simple switching ä Slow reconfiguration ä Predefined switching pattern ä Low complexity switch architectures ä High scalability ä Low cost ä Low complexity switching fabric ä Higher scalability ä Lower cost

31. May 2002© 2002 Yoram Ofek31 Multiple time frames  low blocking probability DWDM  many parallel routes  low blocking probability Multistage Crossbar Switching elements a*N*lg a N N 2 For N=256, a=4 4K 64K For N=1024, a=4 20K 1,000K (factor of 16) (factor of 50) Low Complexity Switching Fabric Optimal Speedup of 1 Scalability Blocking

32. May 2002© 2002 Yoram Ofek32 Blocking Probability

33. May 2002© 2002 Yoram Ofek33 Scheduling and Switching with UTC Alignment TF1 TF2 TF3 TF4 TF5 TF6 TF7 TF8 TF1 TF2 TF3 TF4 TF5 TF6 TF7 TF8 Periodic Schedule on Switch i Periodic Schedule on Switch j Always aligned with a bounded error (typically < 1  second) Thus, delay (memory) per switch = 1 TF Schedule s

34. May 2002© 2002 Yoram Ofek34 Scheduling and Switching without UTC Alignment Circuit Switching, e.g., SONET TF1 TF2 TF3 TF4 TF5 TF6 TF7 TF8 TF1 TF2 TF3 TF4 TF5 TF6 TF7 TF8 No alignment Thus, delay (memory) per switch = 1 Time Cycle Periodic Schedule on Switch i Periodic Schedule on Switch j Schedule s

35. May 2002© 2002 Yoram Ofek35 Service Interfaces Network Processor (MPLS) F P 2 F P 1 F P i Port SONET DMUX (STS-1) F P 2 F P 1 F P i Port IP/MPLS SONET OC-12/OC-48 MPLS Packets SONET STS-1 frames See Animation UTC

36. May 2002© 2002 Yoram Ofek36 Conclusion SingleNetwork Machine-to-Machine Typically No Rate Person-to-Person Typically with Rate Fractional Switching Time-driven Priority MetaNet Convergence Routing UTC can be used as the “GLUE” for combining: Person-to-Person, Machine-to-Machine, TCP/IP “best effort” …