1. ENGS 116 Lecture 191 Interconnection Networks Vincent H. Berk November 24, 2008 Reading for today: Sections 6.1 – 6.4 Reading for Monday: Sections 6.5 – 6.9

2. ENGS 116 Lecture 192 Project Reports Due by Beginning of class on Monday, December 1 st Content –Introduction and description of the topic –Coverage of topic: breadth/depth, appropriate background information –Analysis and discussion –References: correct citations, proper form Writing –Spelling –Grammar –Style and presentation Assume that the reader is familiar with basic architecture concepts Must use appropriate citations. Argument all your decisions. Email all source code to

3. ENGS 116 Lecture 193 Project Presentations 16 minutes – absolutely no more – Practice your timing! All group members talk 1 st (3) of December, 3 rd (3) of December Present: –research question –approach –results –conclusions EVERYONE ATTENDS these presentations

4. General Announcement Hardware Security Project starting this Winter term Design and implementation of Hardware Based Profiler Need: 1 or 2 Masters or PhD students Contact: Steve Taylor or Vincent Berk

5. ENGS 116 Lecture 195 Networks Common topics of conversation: –direct (point-to-point) vs. indirect (multi-hop) –topology (e.g., bus, ring, directed acyclic graph, star) –routing algorithms –switching (aka multiplexing) –wiring (e.g., choice of media, copper, coax, fiber) What really matters: –latency –bandwidth –cost –reliability

6. ENGS 116 Lecture 196 ABCs of Networks Starting point: Send bits between 2 computers Queue on each end Can send both ways (“Full Duplex”) Rules for communication? “protocol” –Inside a computer: Loads/Stores: Request (Address) & Response (Data) Need request & response signaling –Name for standard group of bits sent: packet

7. ENGS 116 Lecture 197 A Simple Example What is the format of a packet? (Protocol) –Fixed? Number of bytes? Request/ Response Address/Data 1 bit 32 bits 0: Please send data from address 1: Packet contains data corresponding to request

8. ENGS 116 Lecture 198 Questions About Simple Example What if more than 2 computers want to communicate? –Need computer address field (destination) in packet What if packet is garbled in transit? –Add error detection field in packet (e.g., CRC) What if packet is lost? –More elaborate protocols to detect loss (e.g., NAK, ARQ, time outs) What if multiple processes per machine? –Queue per process Questions such as these lead to more complex protocols and packet formats

9. ENGS 116 Lecture 199 A Simple Example Revisited What is the format of a packet? –Fixed? Number of bytes? Request/ Response Address/Data 2 bits32 bits 00: Request—Please send data from address 01: Reply—Packet contains data corresponding to request 10: Acknowledge request 11: Acknowledge reply 4 bits CRC

10. ENGS 116 Lecture 1910 Additional Background Connection of 2 or more networks: Internetworking 3 cultures for 3 classes of networks –SAN: server (storage) networks, performance –LAN: workstations, cost –WAN: telecommunications, long range Cost Performance (BW, latency) Reliability

11. ENGS 116 Lecture 1911 Interconnections (Networks) Examples: –SAN networks (infiniband): 100s nodes; ≤ 10 meters per link –Local Area Networks (Ethernet): 100s nodes; ≤ 100 meters –Wide Area Network (ATM): 1000s nodes; ≤ 5,000,000 meters Interconnect SW Interface HW Interface Node Link SW Interface HW Interface Node Link SW Interface HW Interface Node Link SW Interface HW Interface Node Link...

12. ENGS 116 Lecture 1912 Software to Send and Receive SW Send steps 1: Application copies data to OS buffer 2: OS calculates checksum, starts timer 3: OS sends data to network interface HW and says start SW Receive steps 3: OS copies data from network interface HW to OS buffer 2: OS calculates checksum, if matches send ACK; if not, deletes message (sender resends when timer expires) 1: If OK, OS copies data to user address space and signals application to continue Sequence of steps for SW: protocol –Example similar to UDP/IP protocol in UNIX

13. ENGS 116 Lecture 1913 Network Performance Measures SW Interface HW Interface Node Link Bandwidth Overhead Link SW Interface HW Interface Node Link Bandwidth Overhead Link Bisection Bandwidth Interconnect Latency...

14. ENGS 116 Lecture 1914 Universal Performance Metrics Sender Receiver Sender Overhead Transmission time (size ÷ bandwidth) Transmission time (size ÷ bandwidth) Time of Flight Receiver Overhead Transport Latency Total Latency = Sender Overhead + Time of Flight + Message Size ÷ BW + Receiver Overhead Includes header/trailer in BW calculation? Total Latency (processor busy) (processor busy)

15. ENGS 116 Lecture 1915 Simplified Latency Model Total Latency ≈ Overhead + Message Size / BW Overhead = Sender Overhead + Time of Flight + Receiver Overhead Example: show what happens as we vary the following –Overhead: 1, 25, 500 µsec –BW: 10, 100, 1000 Mbit/sec (factors of 10) –Message Size: 16 Bytes to 4 MB (factors of 4) If overhead is 500 µsec, how big a message is needed to get > 10 Mb/s of bandwidth?

16. ENGS 116 Lecture 1916 0.01 0.1 1 10 100 1000 1664 256 10244096 1638465536 262144 10485764194304 Message Size (bytes) Effective Bandwidth (Mbits/sec) o1, bw10 o1, bw100 o1, bw1000 o25, bw10 o25, bw100 o25, bw1000 o500, bw10 o500, bw100 o500, bw1000 o1, bw1000 o500, bw1000 o25, bw1000 o500, bw100 o500, bw10 bw1000 bw100 bw10 o1, bw100 Overhead, Bandwidth, Size

17. ENGS 116 Lecture 1917 Measurement: Sizes of Message for NFS 95% messages, 30% bytes for packets ≤ 200 bytes > 50% data transferred in packets = 8KB Packet size Cumulative % 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 010242048307240965120614471688192 Msgs Bytes Why?

18. ENGS 116 Lecture 1918 HW Interface Issues Where to connect network to computer? –Cache consistent to avoid flushes? (  memory bus) –Latency and bandwidth? (  memory bus) –Standard interface card? (  I/O bus) –MPP  memory bus; LAN, WAN  I/O bus $ CPU L2 $ Memory Bus MemoryBus Adaptor I/O bus I/O Controller I/O Controller Network ideal: high bandwidth, low latency, standard interface Network

19. ENGS 116 Lecture 1919 SW Interface Issues How to connect network to software? –Programmed I/O? (low latency) –DMA? (best for large messages) –Receiver interrupted or received polls? Things to avoid –Invoking operating system in common case –Operating at uncached memory speed (e.g., check status of network interface)

20. ENGS 116 Lecture 1920 CM-5 Software Interface CM-5 example (MPP) –Time per poll 1.6  secs; time per interrupt 19  secs –Minimum time to handle message: 0.5  secs –Enable/disable 4.9/3.8  secs As rate of messages arriving changes, use polling or interrupt? –Solution: Always enable interrupts, have interrupt routine poll until no messages pending –Low arrival rate  interrupt –High arrival rate  polling 0 10 20 30 40 50 60 70 80 90 100 0102030405060708090100 message interarrival (µsecs) message overhead (µsecs) Polling Interrupts Time between messages Overhead

21. ENGS 116 Lecture 1921 Network Media Copper, 1mm thick, twisted to avoid antenna effect (telephone) Used by cable companies: high BW, good noise immunity 3 parts are cable, light source, light detector. Twisted Pair: Coaxial Cable: Copper core Insulator Braided outer conductor Plastic Covering Fiber Optics Transmitter – L.E.D – Laser Diode Receiver – Photodiode light source Silica Total internal reflection Air

22. ENGS 116 Lecture 1922 Connecting Multiple Computers Shared Media vs. Switched: pairs communicate at same time, “point-to-point” connections Aggregate BW in switched network is many times that of shared –point-to-point faster since no arbitration, simpler interface Arbitration in shared network? –Central arbiter for LAN? –Listen to check if being used (“Carrier Sensing”) –Listen to check if collision (“Collision Detection”) –Random resend to avoid repeated collisions; not fair arbitration; –OK if low utilization (a.k.a. data switching interchanges, multistage interconnection networks, interface message processors) Switch Node Switched Media (CM-5, Fast-Ethernet) Node Shared Media (Ethernet)

23. ENGS 116 Lecture 1923 Switch Topology Structure of the interconnect Determines –Degree: number of links from a node –Diameter: max number of links crossed between nodes –Average distance: number of hops to random destination –Bisection: minimum number of links that separate the network into two halves (worst case) Warning: these three-dimensional drawings must be mapped onto chips and boards which are essentially two-dimensional media –Elegant when sketched on the blackboard may look awkward when constructed from chips, cables, boards, and boxes (largely 2D)

24. ENGS 116 Lecture 1924 Figure 8.15 A ring network topology. A Simple Example

25. ENGS 116 Lecture 1925 Examples of Static Interconnection Network Topologies c) Star a) Bus b) Linear array e) Tree f) Near-neighbor mesh d) Ring g) Completely connected h) 3–cube (hypercube)

26. ENGS 116 Lecture 1926 Figure 8.16: Network topologies that have appeared in commercial MPPs.

27. ENGS 116 Lecture 1927 Important Topologies N = 1024 TypeDegreeDiameterAvg DistBisectionDiamAvg D 1D mesh≤ 2N-1N/31 2D mesh≤ 42(N 1/2 - 1)2N 1/2 / 3N 1/2 6321 3D mesh≤ 63(N 1/3 - 1)3N 1/3 / 3N 2/3 ~ 30~ 10 Ring2N / 2N/42 2D torus4N 1/2 N 1/2 / 22N 1/2 3216 Hypercubenn = LogNn/2N/2 105

28. ENGS 116 Lecture 1928 Figure 8.14 A fat-tree topology for 16 nodes. 0 1 2 3 0 0 0 1 4 5 6 7 0 1 1 1 8 9 10 11 0 2 2 1 12 13 14 15 03 3 1 0 0 0 1 0 2 0 3 0

29. ENGS 116 Lecture 1929 Figure 8.13 Popular switch topologies for eight nodes.

30. ENGS 116 Lecture 1930 Examples of dynamic interconnection network topologies = processor= switch a) Crossbar switch b) 8  8 Baseline

31. ENGS 116 Lecture 2031 Connection-Based vs. Connectionless Telephone: operator sets up connection between the caller and the receiver –Once the connection is established, conversation can continue for hours Share transmission lines over long distances by using switches to multiplex several conversations on the same lines –“Time division multiplexing” divide B/W transmission line into a fixed number of slots, with each slot assigned to a conversation Problem: lines busy based on number of conversations, not amount of information sent Advantage: reserved bandwidth

32. ENGS 116 Lecture 2032 Connection-Based vs. Connectionless Connectionless: every package of information must have an address => packets –Each package is routed to its destination by looking at its address –Analogy, the postal system (sending a letter) –Also called “Statistical multiplexing” Each packet requires a new/separate routing decision Depending on implementation the switching stations may also be called routers.

33. ENGS 116 Lecture 2033 Routing Messages Within a network: –Shared media: Broadcast to everyone Internetwork routing. Options: –Source-based routing: message specifies path to the destination (changes of direction) –Virtual circuit: circuit established from source to destination, message picks the circuit to follow –Destination-based routing: message specifies destination, switch must pick the path: deterministic vs. non-deterministic deterministic: always follow same path adaptive: pick different paths to avoid congestion, failures randomized routing: pick between several good paths to balance network load

34. ENGS 116 Lecture 2034 mesh: dimension-order routing –(x 1, y 1 )  (x 2, y 2 ) –first  x = x 2 – x 1, –then  y = y 2 – y 1, hypercube: edge-cube routing –X = x o x 1 x 2... x n  Y = y o y 1 y 2... y n –R = X xor Y –Traverse dimensions of differing address in order tree: common ancestor Deadlock free? Deterministic Routing Examples 001 000 101 100 010 110 111 011

35. ENGS 116 Lecture 2035 Store and Forward vs. Cut-Through Store-and-forward policy: each switch waits for the full packet to arrive in switch before sending to the next switch (good for WAN) Cut-through routing or wormhole routing: switch examines the header, decides where to send the message, and then starts forwarding it immediately –In wormhole routing, when head of message is blocked, message stays strung out over the network, potentially blocking other messages (needs only buffer the piece of the packet that is sent between switches). CM-5 uses it, with each switch buffer being 4 bits per port. –Cut-through routing lets the tail continue when head is blocked, “accordioning” the whole message into a single switch. (Requires a buffer large enough to hold the largest packet).

36. ENGS 116 Lecture 2036 Congestion Control Packet switched networks do not reserve bandwidth; this leads to contention (connection-based limits input) Solution: prevent packets from entering until contention is reduced (e.g., freeway on-ramp metering lights) Options: –Packet discarding: If packet arrives at switch and no room in buffer, packet is discarded (e.g., UDP) –Flow control: between pairs of receivers and senders; use feedback to tell sender when allowed to send next packet Back-pressure: separate wires to tell to stop Window: give original sender right to send N packets before getting permission to send more; overlaps latency of interconnection with overhead to send & receive packet (e.g., TCP), adjustable window –Choke packets: aka “rate-based”; each packet received by busy switch in warning state sent back to the source via choke packet. Source reduces traffic to that destination by a fixed % (e.g., ATM, ICMP source quench)

37. ENGS 116 Lecture 2037 Practical Issues for Interconnection Networks Standardization advantages: –low cost (components used repeatedly) –stability (many suppliers to chose from) Standardization disadvantages: –Time for committees to agree –When to standardize? Before anything built? => Committee does design? Too early suppresses innovation Perfect interconnect vs. Fault Tolerant? –Will SW crash on single node prevent communication? (MPP typically assumes perfect) Reliability (vs. availability) of interconnect Most successful system is not always the best design.

38. ENGS 116 Lecture 2038 Practical Issues InterconnectionMPPLANWAN ExampleCM-5EthernetATM StandardNoYesYes Fault Tolerance?NoYesYes Hot Insert?NoYesYes Standards: required for WAN, LAN! Fault Tolerance: Can nodes fail and still deliver messages to other nodes? Required for WAN, LAN! Hot Insert: If the interconnection can survive a failure, can it also continue operation while a new node is added to the interconnection? Required for WAN, LAN!

39. ENGS 116 Lecture 2039 Inter-Network-Routing Connecting >2 networks together. Requires: –Addressing Hierarchy –Common Protocols –Courtesy and Security Each step in a route (hop) decides: –What first? –Where next? Transparent or explicit

40. ENGS 116 Lecture 2040 Bridging (transparent routing)

41. ENGS 116 Lecture 2041 OSI model This one has to be in every network presentation Electrical signals Ethernet Frame Packet IP TCP Network library interface Web browser 1. Physical 2. Data Link 3. Network 4. Transport 5. Session 6. Presentation 7. Application

42. ENGS 116 Lecture 2042 Networking Protocols: HW/SW Interface Internetworking: allows computers on independent and incompatible networks to communicate reliably and efficiently; –Enabling technologies: SW standards that allow reliable communications without reliable networks –Hierarchy of SW layers, giving each layer responsibility for portion of overall communications task, called protocol families or protocol suites Transmission Control Protocol/Internet Protocol (TCP/IP) –This protocol family is the basis of the Internet –IP makes best effort to deliver; TCP “guarantees” delivery –TCP/IP used even when communicating locally: NFS uses IP even though communicating across homogeneous LAN

43. ENGS 116 Lecture 2043 Protocol Key to protocol families is that communication occurs logically at the same level of the protocol, called peer-to-peer, but is implemented via services at the lower level Danger is each level increases latency if implemented as hierarchy (e.g., multiple check sums) Logical T H T H T H T H T H T H Actual T H THT H THT H THT H TH T T T T HH Message

44. ENGS 116 Lecture 2044 IP, TCP, and UDP IP = internet protocol, used at network layer –IP routes datagrams to destination machine, makes best effort to deliver packets but does not guarantee delivery or order of datagrams –For IP, every host and router must have unique IP address IPv4 uses 32-bit addresses IPv6 uses 16-byte addresses (not that straight forward, though!!!) TCP = transmission control protocol, used at transport layer –TCP is connection-oriented, makes guarantee of reliable, in-order delivery –Up to 4 retries on failure to deliver (or acknowledge!) UDP = user data protocol, used at transport layer –Connectionless protocol, makes no guarantees of delivery

45. ENGS 116 Lecture 2045 Packet Formats Fields: Destination, Checksum (C), Length (L), Type (T) Data/Header Sizes in bytes: (4 to 20)/4, (0 to 1500)/26, 48/5 L T Route Data (4 - 20) C 32 bits CM-5ATM Destination 32 bits C Data (48) T 32 bits Preamble Source Destination Data (0 - 1500) Pad (0 -46) Checksum Length Destination Source Ethernet

46. ENGS 116 Lecture 2046 Networking Summary Protocols allow heterogeneous networking Protocols allow operation in the presence of failures Routing issues: store and forward vs. cut-through, congestion,... Standardization key for LAN, WAN Internetworking protocols used as LAN protocols  large overhead for LAN Integrated circuit revolutionizing networks as well as processors Switch is a specialized computer

47. ENGS 116 Lecture 2047 Cluster (Multicomputer) A collection of low-cost nodes connected by a fast network. Applications: –Less synchronization required than for MP applications –Less need for communication –No need for one large homogeneous memory –Many copies of one application run in parallel Each node: –cheap –redundant Easily expandable –Scales if the software application scales

48. ENGS 116 Lecture 2048 Applications Distributed Database –Each node works as the query engine for data on local disk(s) –All nodes together implement redundancy: Failure of 1 or more nodes doesn’t damage the database Scientific applications: –Nuclear or Oceanographic simulations –Diskless nodes. Each node uses NFS (of similar SAN-based system) to access central data repository. –Applications are started over the network. –Think SETI@home (BOINC)