1. MCT620 – Distributed Systems
Workshop 1 – Lecture 3
Introduction to Networks

2. Introduction to Distributed Systems
Lecture 1 – Introduction to Distributed Systems
Lecture 2 – History of Distributed Systems
Lecture 3 – Introduction to Networks
In Lectures 1 and 2 we laid some foundations. We got an introduction to the core concepts of Distributed Systems and we looked at the history of these technologies.
The network is the medium on which distributed systems are built. In this lecture, we look at some of the defining qualities of networks and how these influence the way systems are designed.

3. In this Lecture …. Protocol Hierarchies Architecture of the Internet
IP Addressing & Classes
Subnetting
Transmission Control Protocol - TCP
User Datagram Protocol - UDP

4. Protocol Hierarchies
To reduce complexity of design, networks are organized as layers, each one built upon the one below it. The number of layers, the name of each layer, the contents and function of each layer differ from network to network.
The purpose of each layer is to create services for the layers above, hiding to those layers the details of how those services are actually implemented. The fundamental idea is that a particular piece of software (or even hardware) provides a service to its users, but keeps the details of its internal state and algorithms hidden from them.
Layer n on a machine carries a conversation with layer n on another machine. The rules and conventions used in this conversation are known as layer n protocol. In essence, a PROTOCOL is an agreement between the communicating parties on how communication is to proceed. The entities that implement the protocol at different layers level are called PEERS. It is peers that communicate using the protocol.
In reality, no data is directly transferred from layer n on one machine on layer n on the other machine. In effect, each layer passes data and control information to the layer below it, until the lowest layer is reached. Below Layer 1 is the physical medium, through the communication occurs.
Between each pair of adjacent layers there is an interface. The interface defines which primitive operations and services the lower layer makes available to the upper one. Most difficult design issue is to define clean interfaces between layers.
A set of layers and protocols is called a NETWORK ARCHITECTURE (it has to contain enough information to allow hardware and software engineers to design hardware and software that would obey the right protocol). A list of protocols used by certain systems, one protocol per layer, is called a PROTOCOL STACK.

5. Protocol Hierarchies (2)
The philosopher-translator-secretary architecture
Layer3 – two philosophers (peer processes), one speaks English and the other one speaks French. Since they have no common language, they engage a translator (peer processes at Layer 2). The translators, each contact a secretary (peer process Layer 1). Translators have agreed on a common language that both know (Dutch).
Note that each protocol is completely independent of the other ones. If at any time, the translators decide to change the language, all they have to do is to agree between each other. None of the interfaces with layer 3 or layer 1 will be changed. Similarly, the secretary could choose to use a different transmission medium (say ), without disturbing or even informing the other layers.

6. Protocol Hierarchies (3)
A message is produced by an application process running at layer 5. Message M is then given to layer 4 for transmission. Layer 4 puts a header in the front of the message to identify the message and pass the result to layer 3. The header includes control information, such as sequence numbers to allow layer 4 on destination to deliver messages in the right order, if the lower layers do not maintain sequence. In some layers, headers can also contain sizes, times and other control fields.
At layer 3 there is a limit on the size of the packet that can be transmitted. So layer 3 will break the incoming message into smaller parts, packets, adding header H3 corresponding to layer 3 on each packet. In this example, message M is split into M1 and M2.
Layer 3 decides which outgoing lines to use and passes the message to layer 2. Layer 2 adds not only a header to each piece, but also a trailer and gives the resulting units to layer 1 for physical transmission.
At the receiving machine, the message moves upwards, from layer to layer, with headers being stripped off as it progresses.

7. Architecture of the Internet
The INTERNET is not a network at all, but a vast collection of different networks that use common protocols and provide common services. The Internet was not planned and is not controlled by anyone. It all started back in late 1950’s when the US DoD realized that all communication is based on telephony systems with little or no redundancy. Therefore, a highly distributed, fault tolerant system needed to be in place. In the late 1950’s ARPA (Advanced Research Project Agency) was formed. In 1967 ARPA’s interest turned to networking … ARPANET, first network of computers was built. The subnet would consist of minicomputers (called IMPs – INTERNET MESSAGE PROCESSORS), interconnected by 56kb/s transmission lines. For reliability, each IMP would be connected to at least two another IMPs. The subnet was a datagram subnet, so if some lines and IMPs were destroyed, the messages could automatically be rerouted along the alternative paths.
Client at home calls its ISP over a dialup line. The packets carried over the modem are transferred to the ISPs POP (POINT OF PRESENCE) where they are removed from the telephone system and injected into the ISP regional network. From this point on, the system is fully digital and packet switched. The regional ISP consists of several routers, located in main cities where the ISP operates. The regional ISP subnet is connected to a backbone, that usually run between states or countries (even continents). The backbones are usually made of high bandwidth optical fiber lines, interconnected by routers. NAPs interconnect backbones (very often belonging to different competing ISPs).

8. TCP/IP Reference Model (1)
The TCP/IP protocol stack is used in the Internet, packet switching network (of networks) based on a connectionless internetwork layer. The Internet Protocol (IP) is the glue that holds together the Internet. It provides a best-efforts (not guaranteed) way to transport datagrams from source to destination.
Like most network protocols, TCP/ IP is a layered protocol. Each layer builds upon the layer below it, adding new functionality. The lowest level protocol is concerned purely with the business of sending and receiving data - any data - using specific network hardware. At the top are protocols designed specifically for tasks like transferring files or delivering . In between are levels concerned with things like routing and reliability. The benefit that the layered protocol stack gives you is that, if you invent a new network application or a new type of hardware, you only need to create a protocol for that application or that hardware: you don’t have to rewrite the whole stack. 
Workflow:
The transport layer takes the data streams and breaks them into datagrams (in theory they can be up to 64KB, but in practice they are no more than 1500 bytes).
Each datagram is transmitted through the Internet (possibly being fragmented into smaller pieces as it goes).
When all pieces get to the destination machine, they are reassembled by the network layer into the original datagram.
This datagram is handed to the transport layer which inserts it into the receiving process input stream.

9. Format of the IP datagram
IP packet is transferred in big endian : from left to right, with the high order bit of the version control going first. All 8086 based machines are little endian, so whenever sending or receiving, a conversion is required
Versioning (keeps track of the versioning control). Currently we can have IPV4 (0100) or IPV6 (0110)
Header Length – how long the header is in 32 bit-words; the minimum value is 5, which applies when no option is present; maximum value is 15 giving a maximum of 60 bytes for the header, when options are present (thus options field is limited to 40 bytes)
Type of service – defines how the datagram should be handled; it includes bits that define priority of the datagram; it also include bits that define the type of service the sender desires, such as level of throughput, reliability and delay; most of the times, this filed is completely ignored by routers
Total length – includes everything in the datagram, both header and data; the maximum length is 65,535 bytes; at this stage, this limit is OK, but with future gigabit networks, larger datagrams may be needed
Identification – is needed to allow the destination host to determine which datagram a newly arrived fragment belongs to. All the fragments of the datagram contains the identification field.
DF – bit specifying Don’t Fragment; this is helpful for systems that can’t put back together the fragments of a datagram
MF – stands for More Fragments; all fragments except the last one have this bit set. It is needed to know when the fragments of a datagram have arrived
Fragment offset – tells where in the current datagram the current fragments belongs. All fragments, except the last one have to be multiple of 8 bytes (elementary fragment unit); since 13 bits are provided, there are a number of 8192 fragments per datagram, giving a maximum datagram length of bytes
Time to live – is a counter used to limit the packets life times; it is suppose to count time in seconds, allowing a maximum life time of 255 seconds, but in practice it counts only hops. When it hits 0, the packet is discarded and a warning packet is sent back to the host; this feature prevents datagrams from going around forever.
Protocol – tells which transport process to give it to…in other words is specifies the transport layer protocol (TCP, UDP, etc…)
Header checksum – verifies only the header; the algorithm is to add up all 16 bit half words as they arrive, using one’s complement arithmetic and then take one’s complement result; the header checksum must be computed at each hope, since there are values in the header that modifies (i.e. hops count)
Options – was designed to allow designers and implementers of network protocols an escape mechanism; this option value allows for subsequent version of the protocols to include information not present in the original specs; the options are variable length, each of them beginning with one byte identifying the option.

10. The big picture
The Internet has two main protocols at the transport layer:
Connectionless protocol: UDP (User Datagram Protocol)
Connection oriented protocol: TCP (Transport Control Protocol)
Other useful protocols at the Network Layer:
ICMP – Internet Control and Messaging Protocol
IGMP – Internet Group Management Protocol – used for router discovery and multicasting

11. User Datagram Protocol (1)
Simple transport layer protocol described in RFC 768, in essence just a IP datagram with a short header
Provides a way to send encapsulated raw IP datagrams without having to establish a connection
For example, a UDP client can create a socket and send a datagram to a given server and then immediately send another datagram on the same socket to a different server; similarly, UDP server can receive multiple datagrams from different sources on the same single UDP socket

12. User Datagram Protocol (2)
The application writes a datagram to a UDP socket, which is encapsulated as either IPv4 or IPv6 datagram that is sent to the destination; there is no guarantee that UDP datagram ever reaches its final destination
Many client-server application that have one request – one response use UDP
Each UDP datagram has a length; if a UDP datagram reaches its final destination correctly, then the length of the datagram is passed onto the receiving application
UDP provides a connectionless service as there is no need for a long term relation-ship between a client and the server
With UDP no effort is made to make sure it arrives at the destination, or that all packets arrived that were sent. UDP is generally used for real-time applications like Internet radio and online gaming, where dropped packets need not be resent, and would probably be too old if they were. UDP is also used when upper-layer protocols do their own flow control and data stream checking and correcting.

13. UDP header
UDP segment consists of 8 bytes header followed by the data:
The source port and destination port identify the end points within the source and destination machines; without the port fields, the transport would not know what to do with the packet; with them, it delivers the packets correctly to the application attached to the destination port
The UDP length field includes the 8 byte header and the data
The UDP checksum includes the 1 complement sum of the UDP data and header. It is optional, if not computed it should be stored as all bits “0”.
UDP does not provide Flow control, Error control or Retransmission upon receipt of a bad segment. It does provide an interface to the IP protocol, with the added feature of demultiplexing multiple processes using the ports. One area where UDP is useful is client server situations where the client sends a short request to the server and expects a short replay. If the request or reply is lost, the client can timeout and try again.
DNS (Domain Name System) is an application that uses UDP: DNS is used to lookup the IP address of some host name; DNS sends and UDP packet containing the host name to a DNS server, the server replies with an UDP packet containing the host IP address. No setup is needed in advance and no release of a connection is required. Just two messages go over the network. UDP is also widely used in client server RPC and in real time multimedia applications.

14. Transmission Control Protocol (1)
Provides a reliable end to end byte stream over unreliable internetwork (different parts may have different topologies, bandwidths, delays, packet sizes, etc…)
Accepts user data streams from local processes, breaks them into pieces (not larger than 64KB) and sends each piece as a separate IP datagram- tt the receiving end, the IP datagrams that contains TCP packets, are delivered to the TCP transport entity, which reconstructs the original byte stream
The IP layer gives no guarantee that the datagrams will be delivered properly, so it is up to TCP to time out and retransmit them as the need arises
Datagrams that arrive, may be in the wrong order; it is up to TCP to reassemble them into messages in the proper sequence
When an application wants to communicate with another application via TCP, it sends a communication request. This request must be sent to an exact address. After a "handshake" between the two applications, TCP will setup a "full-duplex" communication between the two applications. The "full-duplex" communication will occupy the communication line between the two computers until it is closed by one of the two applications.
UDP occupies a similar place in the protocol stack to TCP, but UDP is much simpler and is not reliable.

15. Transmission Control Protocol (2)
Each byte on a TCP connection has its own 32 bit sequence number, used for various purposes (re- arrangement of out of sequence segments; identification of duplicate segments, etc…)
Sending and receiving TCP entities exchange data in the form of segments. A segment consists of a fixed 20 byte fixed header (plus an optional part) followed by zero or more data bytes
The TCP software decides how big segments should be; it can accumulate data from several writes to one segment or split data from one write over multiple segments.
There are two effective limits that restrict the maximum segment size:
1: Each segment including the TCP header must fit the byte IP payload
2: Each network has an maximum transfer unit (MTU) and each segment must fit in the MTU (in practice the MTU is a few thousand bytes and therefore sets the upper bound on the segment size)

16. TCP Service Model (1)
Provides connections between clients and servers, both the client and the server create end points, called sockets
Each socket has a number (address) consisting of the IP address of the host and a 16 bits number, local to that host, called port (TCP name for TSAP)
To obtain a TCP service, a connection has to be explicitly established between the two end points
A socket may be used for multiple connections at the same time
Two or more connections can terminate in the same socket; connections are identified by socket identifiers at both ends (socket1, socket2)
Provides reliability
When TCP sends data to the other end it requires acknowledgement in return; if ACK is not received, the TCP entity retransmits automatically the data and waits a longer amount of time
TCP contains algorithms to estimate round-trip time between a client and a server to know dynamically how long to wait for an acknowledgement
TCP provides flow control – it tells exactly to its peer how many bytes of data is willing to accept from its peer.
Port numbers bellow 1024 are called well known ports and are reserved for standard services
Port 21 used by FTP (File Transfer Protocol)
Port 23 used by Telnet (Remote Login)
Port 25 used by SMTP ( )
Port 69 used by TFTP (Trivial File Transfer Protocol)
Port 79 used by Finger (lookup for a user information)
Port 80 used by HTTP (World Wide Web)
Port 110 used by POP3 (remote access)
Port 119 used by NNTP (Usenet news)

17. TCP Service Model (2)
All TCP connections are full duplex and point to point (it doesn’t support multicasting or broadcasting)
A TCP connection is a byte stream not a message stream (message boundaries are not preserved)
i.e. if a process is doing four writes of 512 bytes to a TCP stream, data may be delivered at the other end as four 512 bytes reads, two 1024 bytes reads or one read
When an application sends data to TCP, TCP entity may send it immediately or buffer it (in order to collect a larger amount of data to send at once)
Sometimes the application really wants the data to be sent immediately (i.e. after a command line has been finished); to force the data out, applications can use the PUSH flag (which tells TCP not to delay transmission)
Urgent data – is a way of sending some urgent data from one application to another over TCP
i.e. when an interactive user hits DEL or CTRL-C key to break-off the remote computation; the sending app puts some control info in the TCP stream along with the URGENT flag; this causes the TCP to stop accumulating data and send everything it has for that connection immediately; when the urgent data is received at the destination, the receiving application is interrupted (by sending a break signal in Unix), so it can read the data stream to find the urgent data

18. TCP IP Client Server Interaction
We can see in this diagram there is a lot of overhead with TCP/IP in terms of managing the connection: (Watch out for this sequence of events in this week’s vLab activities.)
Client initiates a connection using a three-way handshake protocol. (messages 1-3)
Client sends its request (message 4)
Client requests server to close the connection (FIN) (message 5)
Server acknowledges the requests (message 6)
Server then completes the request, sends an answer (message 7) and requests that the connection be finished (message 8).
Client acknowledges and finishes connection (message 9)
Tannenbaum (2000)

19. IP Addressing
Three fields of variable sizes (dependent on the class of the address):
1: Class Type field defines the class (5 possible classes that an internet address is part off)
2: Network ID – up the class type, this field can be anywhere between 7 and 24 bits
3: Host ID – up to the class type it can be anywhere between 8 and 24 bits
Internet addresses are 32 bits long, written as four bytes separated by periods (full stops). They can range from to It’s worth noting that IP addresses are stored in big-endian format, with the most significant byte first, read left to right. This contrasts with the little-endian format used on Intel- based systems for storing 32- bit numbers. This minor point can cause a lot of trouble for PC programmers and others working with raw IP data if they forget.  IP addresses comprise two parts, the network ID and the host ID. An IP address can identify a network (if the host part is all zero) or an individual host. The dividing line between the network ID and the host ID is not constant. Instead, IP addresses are split into three classes which allow for a small number of very large networks, a medium number of medium- sized networks and a large number of small networks. 
ICANN (Internet Corporation for Assigned Names and Numbers), is a non profit corporation that manages the assignment of IP address space to various regional authorities that deal with IP address assignment.

20. Internet Classes Some examples of IP Address (show as 32 bit values):
(Class B)
(Class C)
(Class A)
(Class D)
(Class E)
Class A, B are full; class C still has available addresses; D is reserved for multicasting and class E is reserved for future use

21. Dotted decimal notation
Dotted Decimal Notation is used to make the form shorter and easier to read
Internet addresses are usually written using this form
Looking at the first byte of an address in decimal form will allow us to determine which class the particular address belongs (for the example shown it belongs to class B)

22. Class ranges for Internet addresses
Class A addresses have a first byte in the range 1 to 126. The remaining three bytes can be used for unique host addresses. This allows for 126 networks each with up to 16m hosts.  Class B addresses can be distinguished by first byte values in the range x. x to x. x. In these addresses, the first two bytes are used for the net ID, and the last two for the host ID, giving addresses for 16,000 networks, each with up to 16,000 hosts.  Class C addresses are in the range x to x. Here, the first three bytes identify the network, leaving just one byte for the individual hosts. This provides for 2 million networks of up to 254 hosts each. 

23. Campus network example
Consider a large organization with class B addresses ( )
With two level addressing schema, the organization can’t have more than one physical network.
Solution: allow subnets, allow a network to be split into several parts for internal use, but still act as a single network to the outside world.

24. Subnets (1)
Instead of having a single class B address with 14 bits for network and 16 bits for host number, some bits are taken away from the host number to create a subnet number
For example, if the university (large organization) has 35 departments it could use 6 bits for the subnet number and a 10 bit host number, allowing for up to 64 Ethernets, each with a maximum of 1022 hosts (all 0 and all 1 are not allowed); this split can be changed later if it proves to be wrong.

25. Subnets (2) In our example, the subnets are as follows:
Subnet mask:
First host in the subnet 1:
Subnet 2: |
Subnet:
First host in the subnet 2:
Subnet 3: |
Subnet:
First host in the subnet 3:
To implement subnetting, the main router will need a subnet mask that indicates the split between network + subnet number and host number
The masking process extracts the address of the physical network from an IP address (bitwise AND between the IP and mask); masking can be done either we have subnetting or not
Subnet mask is also written in dotted decimal notation or as a slash followed by the number of bits in the network
Subnet mask for this example is ; an alternative notation is /22 to indicate that the subnet mask is 22 bit long, so we can have /22

26. CIDR
Basic idea is to allocate the remaining IP addresses in variable-sized blocks, without regard to the classes.
If a site needs, say 2000 addresses, it is given a block of 2048 addresses on a byte boundary
Dropping classes makes the routing more complicated, the old routing algorithm is not working anymore
IPv4 is running out of addresses:
Class A networks (with 16 million host addresses) is too big for most of the organizations
Class C networks (with 256 host addresses) is too small for most of the organizations
Class B networks (with 65,536 host addresses) is about right for a medium sized organizations
In reality a class B address is too large for most of the organizations; studies show that half of the class B networks have less than 50 hosts….
Two solutions to cope with the shortage problem
Use of CIDR (Classless InterDomain Routing)
Use of NAT (Network Address Translation)

27. CIDR practical example (1)
Addresses are available at
Suppose that:
Cambridge needs 2048 addresses and it is assigned through with mask
Oxford asks for 4096 addresses and it is assigned through with mask
Edinburgh asks for 1024 addresses and it is assigned through with mask x

28. NAT
It comes into play whenever a need for more hosts than real IP addresses are available (i.e. an ISP may have a class B address, having a 65,534 possible hosts, but has more customers than that)
NAT (RFC 3022) – basic idea is to assign each company a single IP address (or a small number of them) for Internet traffic. Within company, every computer gets a unique IP address, which is used for routing internal traffic; when a packet exits the company and goes to the ISP, an address translation takes place
To make this thing possible, three ranges of IP addresses have been declared as private, companies can use them internally as they wish; the only rule is that no packets containing these addresses may appear on the internet itself. These address ranges are:
– /8 (16,777,216 hosts)
– /12 (1, 048,576 hosts)
– (65,536 hosts)

29. IPv6
CIDR and NAT may “buy” a few more years, but the days of IPv4 are numbered (address shortage problem)
1990, IETF started to work on IPv6, with a number of ambitious goals
However, it’s still not widely deployed
Goals of IPv6:
Support billion of hosts
Reduce the size of the routing tables
Simplify the protocol, to allow routers to process faster
Provide better security (auth and privacy) than IPv4
Pay more attention to type of service (for real time data)
Aid multicasting (by allowing scopes to be specified)
Make roaming possible without change of address
Allow protocol expansion
Permit the old and new protocols to coexist for years

30. Summary
In this lecture we have looked at some of the aspects of networking that impact on the way that Distributed Systems are designed.
We have seen how Protocol Hierarchies allow us to reduce complexity by abstracting detail into different layers.
The Architecture of the Internet has an ever growing influence on Distributed System design and we have seen how it is organised.
TCP & UDP are important protocols for internet based systems – we meet them again next week in the context of socket programming.