1. Computer Networks with Internet Technology William Stallings
Chapter 03
Traditional Applications

2. Terminal Access – Telnet History
Oldest Internet application
Demonstrated on four-node ARPANET deployed in 1969
Two years to expand protocol sufficiently to make it useful and to work out bugs
First published version RFC 97
"First Cut at a Proposed Telnet Protocol," February 1971
1983 final form issued as RFC 854 and RFC 855
(Get and study these RFCs – see last slide)
Still useful Internet application
Also pioneering effort application-level protocol design
Basis of many newer protocols
HTTP

3. Remote Terminal Access
Early motivation for networks was remote access to interactive systems
Dumb terminals
Keyboard and screen with primitive comms hardware
Stream of character data transmitted in each direction
Local host computer or terminal controller establish connection to remote host
Local user can use remote host
Hosts handle particular set of characteristics

4. Figure 3.1 (a) Telnet Operational Environment on Arpanet

5. Network Virtual Terminals
Virtual terminal protocol (VTP)
Transform characteristics of real terminal into standardized form
Network virtual terminal (NVT)
Imaginary device
Well defined set of characteristics
Both sides generate data and control signals in native language
Each side translates these to NVT and translates incoming NVT to native signals

6. Figure 3.2 Network Virtual Terminal Concept

7. Phases of operation Connection management Negotiation Control Data
Connection request and termination
Telnet uses TCP
Negotiation
To determine mutually agreeable set of characteristics
NVT has range of capabilities and features
Real terminal more limited
NVT has options, such as line length
Control
Exchange of control information and commands
e.g., end of line, interrupt process
Data
Transfer of data between two correspondents
For Telnet, control and data conveyed in single stream

8. Current Use of Telnet
Original environment for Telnet little relevance today
Still used and included in the TCP/IP suite
Availableon PCs for use over the Internet
PC includes Telnet software
Telnet protocol and translation between PC keyboard/display and NVT
Not GUI
Services available include United States Library of Congress
locis.loc.gov

9. Figure 3.1(b) Telnet Operational Environment on Internet

10. Figure 3.3 (Incomplete) A Telnet Session
telnet locis.loc.gov
Trying
Connected to locis.loc.gov.
Escape character is '^]'.
L O C I S: LIBRARY OF CONGRESS INFORMATION SYSTEM
To make a choice: type a number, then press ENTER
1 Copyright Information files available and up-to-date
2 Braille and Audio files frozen mid-August 1999
3 Federal Legislation files frozen December 1998
* * * * * * * * * * * * * * *
The LC Catalog Files are available at:
8 Searching Hours and Basic Search Commands
9 Library of Congress General Information
10 Library of Congress Fast Facts
12 Comments and Logoff
Choice:

11. Telnet NVT Lowest common denominator of existing systems (at the time)
Includes options and negotiation for more advanced equipment
Between two terminals, two processes, or terminal and process
Server module and user module map characteristics of terminal to NVT
NVT bi-directional character device with display and keyboard
Keyboard can generate all 128 ASCII codes plus user control signals

12. Telnet Transfer Protocol
Data sent half-duplex
Terminal-to-process, newline signifies end of user input
Process-to-terminal, Telnet Go Ahead used
Underlying TCP full duplex
Control signals sent any time regardless of current data direction
Data sent as stream of 8-bit bytes
No other formatting
Control signals and other non-data information sent as Telnet commands
Byte strings embedded in data stream
User control signals, commands between Telnet processes as part of protocol and option negotiation and sub-negotiation

13. Interpret as Command (IAC)
Delimited by Interpret as Command (IAC) char (all 1s, FF, 255)
If bit pattern 255 is part of the data stream, it must also be preceded by an IAC. i.e. IAC 255 or
Commands 2 bytes long
Option negotiation commands 3 bytes long
Third byte option identifier
Option sub-negotiation commands vary in length
Begin 3-byte sequence (IAC SB option-id)
End 2-byte sequence (IAC SE)
Protocol minimizes transmission overhead
No message headers
Processing overhead is high
Must process stream one character at a time
Data translation (between NVT and native)
Scan for Telnet commands
Classic tradeoff between message and stream protocols

14. TCP Urgent Data Pointer
Tells receiver there is urgent data further along data stream
TCP does not define what user about this
Receiving process will usually process urgent data quickly
Prevents TCP buffering holding up urgent data

15. Telnet Synch Mechanism
Allows user to communicate urgent command to server process
E.g. Interrupt Process (IP) or Abort Output (AO) command
Partially alleviates problems caused by time delays over network connections
Consists of Telnet command Data Mark (DM) in TCP segment with TCP urgent notification
Telnet sends urgent command such as AO, or IP, followed by DM sequence (IAC DM) as urgent data
Receiver should immediately scan data stream for Telnet commands as normal
Discard all data
Telnet commands handled normally
Continues until DM found
Then processing returns to normal

16. Telnet Options
Enable two sides connection to use capabilities beyond default NVT
Negotiation allows one side to request an option
Otherside may accept or reject
If accepted, effective immediately
Negotiation any time after connection established
Usually as soon as connected
Options not part of Telnet protocol specification
Published in RFCs

17. Telnet Options Categories
Three major categories
Change, enhance, or refine NVT characteristics
E.g. further definition of printer characteristics
Change transfer protocol
E.g. Suppress Go Ahead option (3), making data transfer protocol full duplex instead of half duplex
Allow other information to be defined and passed over connection
E.g. Status option (5) requests remote side to send status of all options negotiated
Most options may be single sided
Two negotiations required for both sides to use such options

18. Option Negotiation Either side may initiate negotiation
Can ask that option be enabled or that currently enabled option be disabled
You may always reject a request to enable an option
You must always accept a request to disable an option
Options are not enabled until the negotiation is complete
Never negotiate (either request or respond) about something that is already true; that is, do not initiate or respond to a request to initiate an option that is already in effect

19. WILL WONT DO DONT
Interpretation of commands depends on context (see next slide)
Unambiguous if both sides make same request and messages pass on network
A wishes B to implement option
A issues DO
If B agrees, it responds with WILL
B wishes to implement option on its (B’s) side, it issues WILL
If A agrees, it responds with DO
Both A and B request B to implement option at about same time
A issues DO and B issues WILL
Commands cross in network
Both sides receive response to their command
May be sub-negotiation to define details

20. Figure 3.4 Negotiation Messages in Telnet

21. The Longevity of Telnet
Telnet is older than most of its users
(But not most lecturers!)
Telnet is simple
RFC 854 is 15 pages
HTTP (see later lecture) is 176 pages
Simple job done by simple protocol
Telnet can evolve
Option negotiation was brilliant
Not common in IETF protocol designs until late 1980s
Enables Telnet to evolve to meet new demands without endless new versions of basic protocol
Currently over 100 RFCs on Telnet and its options
3% of the entire body of RFCs
Most recent RFC 2953, Telnet Encryption, September 2000

22. FILE TRANSFER—FTP FTP evolved from an era of radically diverse systems
Has obsolete commands, transfer modes, and data representations
Objectives:
Promote sharing of files (computer programs and/or data)
Encourage indirect or implicit (via programs) use of remote computers
Shield user from variations in file storage systems among hosts
Transfer data reliably and efficiently
File systems, rather than just files
Single file viewed as set of bits with name
Trivial File Transfer Protocol (TFTP) does this
Send request header to read or write file with some name
Stream bits across
11 pages to define
FTP deals with metadata such as file pathnames, file organization, access control, and data representation
Accordingly, RFC 959 is 69 pages long

23. FTP Model User FTP entity and Server FTP entity
Initiating host is user
Chooses file name and options
Server accepts or rejects request
Based on its file system protection and options requested
If accepted, server responsible for establishing and managing transfer
Operates on two levels (Figure 3.5)
Establish TCP connection
Exchange control information (commands and replies)
Second TCP connection established for data transfer
FTP user interface enables human user or program to access User FTP

24. FTP Commands Specify parameters for data connection
Data port, transfer mode, representation type, and structure
Nature of file system operation
Store, retrieve, append, delete
User data transfer protocol should "listen" on specified data port
Server initiates data connection and data transfer
FTP uses Telnet protocol on control connection
FTP user protocol or FTP server protocol may implement Telnet rules directly
FTP user protocol or FTP server protocol may use existing Telnet module

25. Figure 3.5 FTP Model

26. Figure 3.6 Overview of an FTP Transfer

27. Options FTP assumes files are objects in mass storage
Share some properties regardless of machine
Files uniquely identified by symbolic names
Files have owners and protection against unauthorized access
Files may be created, read from (copied from), written into, or deleted (within protection rules)
To support specific computers and operating systems, FTP can negotiate options in three dimensions
Datatype, file type, and transfer mode
Systems programmer on each system determines
How particular file can be mapped to standard file type
Using one of standard data types
Transferred using standard mode
Such that it is useful at destination

28. Data Types ASCII, EBCDIC, image, and logical byte size
Text files normally stored as character string
8-bit ASCII on most machines
If ASCII option used, no character code conversion required at either end in most cases
EBCDIC appropriate if both machines IBM hosts
ASCII or EBCDIC files may have further line or page printer specification
Nonprint: Suitable for files not destined for a line printer
Telnet formatting: Embedded control characters
Character control formatting: Formatting conventions from FORTRAN 
Image transfer is bit-by-bit replication of file from the source machine on the target machine
Logical byte size type used when data unit size must be preserved
Specifies byte size (need not be 8 bits)

29. File Types File structure, record structure, and page structure
To promote convenient, efficient interface to file system
Not possible to address idiosyncrasies of all operating systems 
File structure
String of bytes (defined by data type option) that terminates in an end of file marker
Most transfers use this type
Record structure
Sequence of records
Causes transmission of individual records, separated by standard End of Record marker for specified data type
Page structure
For files not stored contiguously on disk
Page structure needs to be preserved

30. Figure 3.7 FTP File Types

31. Transmission Modes – Stream Mode
Optimise use of network
Stream mode (default)
Raw data sent
Least computational burden on user and server systems
No restriction on file type
Record-structure files, 2-byte control code for EOR and EOF

32. Transmission Modes – Block Mode
Provides for restarting failed or interrupted transfer
Source encapsulates data into blocks (see Figure 3.8a)
Block begins with two field header
Descriptor may indicate zero of more of:
Last block in record: If record structure used each record consists of one or more blocks
Last block in file
Suspect data: data may contain errors
Not intended for error control within FTP
Allows sites to exchanging data (e.g., seismic or weather) to send and receive all data despite local errors (such as "magnetic tape read errors"), but showcertain portions are suspect
Restart marker: marks checkpoint in data stream
Receiver marks corresponding position and returns this
May restart from last correctly received marker
Count field indicates total length of data block in bytes

33. Transmission Modes – Compressed Mode
Allows source to squeeze sequences of same character into a shorter coded sequence
Uncompressed data
Replicated byte: Up to 63 of specified bytes
Filler string: Up to 63 of filler characters inserted at destination
Escape sequence: Byte of all zeros followed by descriptor code byte, as in block mode

34. Figure 3.8 FTP Transmission Mode Formats

35. Electronic Mail Most heavily used application on any network
Simple Mail Transfer Protocol (SMTP)
TCP/IP
Delivery of simple text messages
Multi-purpose Internet Mail Extension (MIME)
Delivery of other types of data
Voice, images, video clips

36. SMTP RFC 821 Not concerned with format of messages or data
Covered in RFC 822 (see later)
SMTP uses info written on envelope of mail
Message header
Does not look at contents
Message body
Except:
Standardize message character set to 7 bit ASCII
Add log info to start of message
Shows path taken

37. Basic Operation Mail created by user agent program (mail client)
Message consists of:
Header containing recipient’s address and other info
Body containing user data
Messages queued and sent as input to SMTP sender program
Typically a server process (daemon on UNIX)

38. Mail Message Contents Each queued message has:
Message text
RFC 822 header with message envelope and list of recipients
Message body, composed by user
A list of mail destinations
Derived by user agent from header
May be listed in header
May require expansion of mailing lists
May need replacement of mnemonic names with mailbox names
If BCCs indicated, user agent needs to prepare correct message format

39. SMTP Sender Takes message from queue
Transmits to proper destination host
Via SMTP transaction
Over one or more TCP connections to port 25
Host may have multiple senders active
Host should be able to create receivers on demand
When delivery complete, sender deletes destination from list for that message
When all destinations processed, message is deleted

40. Optimization
If message destined for multiple users on a given host, it is sent only once
Delivery to users handled at destination host
If multiple messages ready for given host, a single TCP connection can be used
Saves overhead of setting up and dropping connection

41. Possible Errors Host unreachable Host out of operation
TCP connection fail during transfer
Sender can re-queue mail
Give up after a period
Faulty destination address
User error
Target user changed address
Redirect if possible
Inform user if not

42. SMTP Protocol - Reliability
Used to transfer messages from sender to receiver over TCP connection
Attempts to provide reliable service
No guarantee to recover lost messages
No end to end acknowledgement to originator
Error indication delivery not guaranteed
Generally considered reliable

43. SMTP Receiver Accepts arriving message
Places in user mailbox or copies to outgoing queue for forwarding
Receiver must:
Verify local mail destinations
Deal with errors
Transmission
Lack of disk space
Sender responsible for message until receiver confirm complete transfer
Indicates mail has arrived at host, not user

44. SMTP Forwarding
Mostly direct transfer from sender host to receiver host
May go through intermediate machine via forwarding capability
Sender can specify route
Target user may have moved

45. Conversation SMTP limited to conversation between sender and receiver
Main function is to transfer messages
Rest of mail handling beyond scope of SMTP
May differ between systems

46. Figure 3.9 SMTP Mail Flow

47. SMTP System Overview
Commands and responses between sender and receiver
Initiative with sender
Establishes TCP connection
Sender sends commands to receiver
e.g. HELO<SP><domain><CRLF>
Each command generates exactly one reply
e.g. 250 requested mail action ok; completed

48. SMTP Replies Leading digit indicates category
Positive completion reply (2xx)
Positive intermediate reply (3xx)
Transient negative completion reply (4xx)
Permanent negative completion reply (5xx)

49. Operation Phases Connection setup Exchange of command-response pairs
Connection termination

50. Connection Setup Sender opens TCP connection with receiver
Once connected, receiver identifies itself
220 <domain> service ready
Sender identifies itself
HELO
Receiver accepts sender’s identification
250 OK
If mail service not available, step 2 above becomes:
421 service not available

51. Mail Transfer Sender may send one or more messages to receiver
MAIL command identifies originator
Gives reverse path to used for error reporting
Receiver returns 250 OK or appropriate fail/error message
One or more RCPT commands identifies recipients for the message
Separate reply for each recipient
DATA command transfers message text
End of message indicated by line containing just period (.)

52. Closing Connection Two steps Sender sends QUIT and waits for reply
Then initiate TCP close operation
Receiver initiates TCP close after sending reply to QUIT

53. Format for Text Messages RFC 882
Message viewed as having envelope and contents
Envelope contains information required to transmit and deliver message
Message is sequence of lines of text
Uses general memo framework
Header usually keyword followed by colon followed by arguments

54. Example Message Date:Tue, 16 Jan 1996 10:37:17 (EST)
From: “William Stallings”
Subject:The syntax of RFC 822
To:
Cc:
This is the main text, delimited from the header by a blank line.

55. Multipurpose Internet Mail Extension (MIME)
Extension to RFC822
SMTP can not transmit executables
Uuencode and other schemes are available
Not standardized
Can not transmit text including international characters (e.g. â, å, ä, è, é, ê, ë)
Need 8 bit ASCII
Servers may reject mail over certain size
Translation between ASCII and EBCDIC not standard
SMTP gateways to X.400 can not handle none text data in X.400 messages
Some SMTP implementations do not adhere to standard
CRLF, truncate or wrap long lines, removal of white space, etc.

56. Overview of MIME Five new message header fields
MIME version
Content type
Content transfer encoding
Content Id
Content Description
Number of content formats defines
Transfer encoding defined

57. Content Types Text body Multipart Message Image Video Audio
Mixed, Parallel, Alternative, Digest
Message
RFC 822, Partial, External-body
Image
jpeg, gif
Video
mpeg
Audio
Basic
Application
Postscript
octet stream

58. MIME Transfer Encodings
Reliable delivery across wide largest range of environments
Content transfer encoding field
Six values
Three (7bit, 8bit, binary) no encoding done
Provide info about nature of data
Quoted-printable
Data largely printable ASCII characters
Non-printing characters represented by hex code
Base64
Maps arbitrary binary input onto printable output
X-token
Named nonstandard encoding

59. Figure 3.10 Printable Encoding of Binary Data into Radix-64 Format

60. Required Reading
Stallings chapter 03
RFCs from