1 Computer Networks An Open Source Approach Chapter 6: Internet Services Chapter 6: Application Layer.

1 Computer Networks An Open Source Approach Chapter 6: Internet Services Chapter 6: Application Layer

2 Content 6.1 Issues 6.2 Domain Name System 6.3 E-Mail 6.4 World Wide Web 6.5 File Transfer Protocol 6.6 Simple Network Management Protocol 6.7 Voice over IP (VoIP) 6.8 Streaming 6.9 Peer-to-Peer Applications (P2P) 6.10 Summary Chapter 6: Application Layer

3 6.1 Issues How ports work? How servers start? Classification of servers Characteristics of application layer protocols Chapter 6: Application Layer

4 How Ports Work? Well known ports: 0~1023 Registered ports: 1024~49151 Dynamic and/or private ports: 49152~65535 Chapter 6: Application Layer

5 How Servers Start? Is a server better to run as standalone or via (x)inetd? Standalone:  requires explicit restarting for configuration changes to take effect  crashed standalone server will probably stay unnoticed and the service will become unavailable  better performance (x)inetd:  reads configuration files each time a client attempts to access their information  fresh copies will be spawned on demand Chapter 6: Application Layer

6 Classification of Servers Concurrent connectionless server Concurrent connection-oriented server Iterative connectionless server Iterative connection-oriented server Chapter 6: Application Layer

Underlying Transport Protocols ApplicationApplication layer protocolUnderlying transport protocol electronic mailSMTP, POP3, IMAPTCP remote terminal accessTelnetTCP file transferFTPTCP webHTTPTCP web cachingICPtypically UDP name resolutionDNStypically UDP network file systemNFStypically UDP network managementSNMPtypically UDP routing protocolRIPtypically UDP Internet telephonyproprietary (e.g. Vocaltec)typically UDP streaming multimediaproprietary (e.g. RealNetworks)typically UDP P2Pproprietary (e.g. BitTorrent, eDonkey)UDP for queries and TCP for data transfer Chapter 6: Application Layer 7

8 Iterative Connectionless Server Chapter 6: Application Layer

9 Concurrent Connection-Oriented Server Chapter 6: Application Layer

10 Characteristics of Application Layer Protocols Variable message formats and length  Message formats and length of application layer protocols vary with different applications and requirements Variable data types  Messages can be transmitted in textual or non-textual formats.  E.g., a Web server replies textual Web pages and binary images Statefulness  The server retains information of the session with the client  E.g., an FTP server remembers the client’s current working directory Chapter 6: Application Layer

11 6.2 Domain Name System Domain name space and domain delegation Zones and name servers Zone data files and resource records Resolvers and Name resolution Open source: BIND Chapter 6: Application Layer

12 Top Level Domains Domain Description com Commercial organizations, such as Intel (intel.com). org Non-profit organizations, such as WWW consortium (w3.org). gov Government organizations, reserved for U.S government such as National Science Foundation (nsf.gov). edu Educational organizations, such as UCLA (ucla.edu). net Networking organizations, such as Internet Assigned Numbers Authority which maintains the DNS root servers (gtld-servers.net). int Organizations established by international treaties between governments. For example, International Telecommunication Union (itu.int). Mil Reserved exclusively for the United States Military. For example, Network Information Center, Department of Defense (nic.mil). Two-letter country code The two-letter country code top level domains (ccTLDs) are based on the ISO 3166-1 two-letter country codes. Examples are tw (Taiwan), uk (United Kingdom). arpa Mostly unused now, except for the in-addr.arpa domain, which is used to maintain a database for reverse DNS queries. Others Such as.biz (business),.name (for individuals),.info (similar with.com). Chapter 6: Application Layer

13 Domain Delegation root domain comedu tw edu com org nctu nthunsysu cis csie ee cis.nctu.edu.tw uk co Easier management Load distribution Chapter 6: Application Layer

14 Zones and Name Servers Differences between a zone and a domain  A zone contains some part of the domain name space  A domain is a subtree in the domain name space  A name server may be authoritative for multiple zones Zone data files in a name server  Contains Resource Records (RR) describing all the hosts within that zone  Six major types of RR: “SOA”, “NS”, “A”, “PTR”, “CNAME” and “MX” Master and slave name servers  Availability and load balance  Data duplication by zone transfer Request from slave Notify by master Chapter 6: Application Layer

15 Master/Slave Name Servers slave 1. Redundancy for heavy load 2. Availability achieved master master (A) | slave (B) zone Azone B df of A df: zone data file df of B df of A and B A large domain name space zone transfer clients Chapter 6: Application Layer

16 Resource Records SOA (Start Of Authority)  Specify a best name server to be authoritative (master) for a zone  Ex: cis.nctu.edu.tw. 86400 IN SOA cisserv.cis.nctu.edu.tw.  Options for refresh, retry, expire..etc are supported NS (Name Server)  Specify name servers for a domain  Ex: cis.nctu.edu.tw. 86400 IN NS cisserv.cis.nctu.edu.tw. A (Address)  Map names to addresses,  Example (multi-homed): linux.cis.nctu.edu.tw. 86400 IN A 140.113.168.127 linux.cis.nctu.edu.tw. 86400 IN A 140.113.207.127 Chapter 6: Application Layer

17 Resource Records (cont) CNAME (Canonical Name)  Create name-to-name alias  Ex: www.cis.nctu.edu.tw. 86400 IN CNAME cache.cis.nctu.edu.tw. PTR (Pointer)  Point addresses to names (canonical name only)  Ex: 10.23.113.140.in-addr.arpa. 86400 IN PTR laser0.cis.nctu.edu.tw. MX (Mail Exchanger)  Provide mail-routing information  The one of best (lowest) preference value is chosen cis.nctu.edu.tw.86400 INMX0 mail.cis.nctu.edu.tw. cis.nctu.edu.tw.86400 INMX10 mail1.cis.nctu.edu.tw. Chapter 6: Application Layer

18 Name Resolution Query type  Forward query – name-to-address  Reverse query – address-to-name Resolution method  Iterative (by referral, commonly adopted)  Recursive (site-by-site recursion) Caching for speeding up the lookup process  Expire according to TTL  Tradeoff between performance and consistency Chapter 6: Application Layer

19 Name Resolution (Iterative) client (resolver) local name server query for “www.dti.gov.uk” root name server 1 2 3 4 6 5 “uk” name server ”gov.uk” name server other candidate name servers “dti.gov.uk” name server 7 8 www.dti.gov.uk. 5M IN A 164.36.253.20 www.dti.gov.uk. 5M IN A 164.36.164.20 Chapter 6: Application Layer

20 Message Format ID QRQR OpcodeA TCTC RDRD RARA ReservedRcode QDCOUNT ANCOUNT NSCOUNT ARCOUNT Question Answer Authority Additional 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 bit Header Chapter 6: Application Layer

21 Execution Flowchart of Named Cache initialization Wait for signals Command line parsing chroot and daemonize Initialize server configurations and data structures Create managers Install a handler for signals 1. Task manager 2. Timer manager 3. Socket manager For possible restart or termination signals Chapter 6: Application Layer

23 Basic Data Structure in Named view1 … viewlist zone_table zone1 zone3zone2 zoneN zone_table zone3 zone4zone8 … Red/Black Tree for zones RR1 RR2 RRN Red/Black Tree of RRs for zone4 zone_table … viewNview2 NULL Chapter 6: Application Layer

24 Domain Information Gopher (DIG) ; > DiG 9.2.0 > www.nctu.edu.tw ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 26027 ;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 3, ADDITIONAL: 3 ;; QUESTION SECTION: ;www.nctu.edu.tw. IN A ;; ANSWER SECTION: www.nctu.edu.tw. 259200 IN A 140.113.250.5 ;; AUTHORITY SECTION: nctu.edu.tw. 259200 IN NS ns.nctu.edu.tw. nctu.edu.tw. 259200 IN NS ns2.nctu.edu.tw. nctu.edu.tw. 259200 IN NS ns3.nctu.edu.tw. ;; ADDITIONAL SECTION: ns.nctu.edu.tw. 259200 IN A 140.113.250.135 ns2.nctu.edu.tw. 259200 IN A 140.113.6.2 ns3.nctu.edu.tw. 259200 IN A 163.28.64.11 Chapter 6: Application Layer

25 6.3 E-Mail Introduction Message Formats Internet Mail Protocols Open Source: Qmail Chapter 6: Application Layer

26 Introduction What is E-mail?  A method of sending messages from one user to another via computer networks Internet mail addressing:  format: user@host.network  e.g. ydlin@cis.nctu.edu.tw Chapter 6: Application Layer

27 Introduction (cont.) Components of Internet mail system:  Mail User Agent (MUA)  Mail Transfer Agent (MTA)  Mail Delivery Agent (MDA)  Mail Retrieval Agent (MRA) Chapter 6: Application Layer

28 Message Formats Internet Message Format (RFC 822) Multipurpose Internet Mail Extensions (RFC 2045-2049) Chapter 6: Application Layer

29 RFC 822 – Internet Message Format A message consists of:  An envelope which contains information needed to accomplish transport and delivery  Contents which compose the object to be delivered to the recipient Chapter 6: Application Layer

30 RFC 822 – Internet Message Format (cont.) Common message header fields: Chapter 6: Application Layer

31 RFC 822 – Internet Message Format (cont.) An example of message header: From: "Dr. Ying-Dar Lin" To: juvenia@cissun53.cis.nctu.edu.tw Date: Mon, 30 Jul 2001 02:42:34 +0800 (CST) Subject: paper Return-Path: Received: (from root@localhost) by mail.cis.nctu.edu.tw (8.11.4/8.11.4) id f6TIgct33454 for gis89813@cis.nctu.edu.tw; Mon, 30 Jul 2001 02:42:38 +0800 (CST) (envelope-from ydlin@cis.nctu.edu.tw) Received: from cissun53.cis.nctu.edu.tw (ydlin@cissun53.cis.nctu.edu.tw [140.113.23.53]) by mail.cis.nctu.edu.tw (8.11.4/8.11.4av) with ESMTP id f6TIgaI33445 for ; Mon, 30 Jul 2001 02:42:36 +0800 (CST) (envelope-from ydlin@cis.nctu.edu.tw) Received: (from ydlin@localhost) by cissun53.cis.nctu.edu.tw (8.9.3/8.9.3) id CAA16191 for juvenia; Mon, 30 Jul 2001 02:42:34 +0800 (CST) Message-Id: X-Virus-Scanned: by AMaViS perl X-UIDL: F5%#!88!"!>RG"!l!G!! Chapter 6: Application Layer

32 MIME What is MIME?  MIME is a specification for enhancing the capabilities of conventional Internet message format. MIME enables email messages to have:  textual header information and message bodies in character sets other than 7-bit ASCII  multiple objects within a single message  binary or application-specific files  multimedia files, such as images, audio, and video files Chapter 6: Application Layer

33 MIME (cont.) MIME message header: Chapter 6: Application Layer

34 MIME (cont.) The MIME content type set: Chapter 6: Application Layer

35 MIME (cont.) Possible values for the Content-Transfer-Encoding: field:  Quoted-Printable  Base64  7bit  8bit  Binary  X-Encoding Chapter 6: Application Layer

36 MIME (cont.) An example of MIME message: From: 'Ching-Ming Tien' To: ydlin@cis.nctu.edu.tw Subject: Cover MIME-Version: 1.0 Content-Type: image/jpg; name=cover.jpg' Content-Transfer-Encoding: base64 Content-Description: The front cover of the book Chapter 6: Application Layer

37 Internet Mail Protocols Simple Mail Transfer Protocol (RFC 2821) Post Office Protocol (RFC 1939) Internet Message Access Protocol (RFC 2060) Chapter 6: Application Layer

38 Simple Mail Transfer Protocol What is SMTP?  A standard host-to-host mail transport protocol and traditionally operates over TCP on port 25 Chapter 6: Application Layer

39 Simple Mail Transfer Protocol (cont.) Important SMTP commands: SMTP replies: Chapter 6: Application Layer ResponseDescription 2xxCommand accepted and processed. 3xxGeneral flow control. 4xxCritical system or transfer failure. 5xxErrors with the SMTP command.

40 Simple Mail Transfer Protocol (cont.) An SMTP transaction scenario: Chapter 6: Application Layer

41 Post Office Protocol What is POP?  A protocol used to retrieve e-mail from a mail server. POP3 Session States: 1. Authorization: Must log in with password before entering transaction state. 2. Transaction: Client can request actions of server, get mail for example. 3. Update: Updates mail box to reflect actions taken in transaction state. Chapter 6: Application Layer

42 Post Office Protocol (cont.) Minimal POP3 commands: Chapter 6: Application Layer

43 Post Office Protocol (cont.) POP3 replies:  +OK, -ERR Example POP session: Chapter 6: Application Layer

44 Internet Message Access Protocol What is IMAP?  A replacement for the POP3 protocol Differences between IMAP4 and POP3:  IMAP4 allows messages being stored and manipulated on the mail system  POP3 only allows users to download their messages and store and manipulate messages on the client’s machines Chapter 6: Application Layer

45 Internet Message Access Protocol (cont.) Four states in the IMAP4 server:  Non-authenticated state When a connection is established between the server and client  Authenticated state When a pre-authenticated connection starts, When acceptable authentication credentials have been provided After an error in selecting a mailbox  Selected state When a mailbox has been successfully selected  Logout state When the client asks to exit the server Chapter 6: Application Layer

46 Internet Message Access Protocol (cont.) IMAP4 command summary: Chapter 6: Application Layer

47 Internet Message Access Protocol (cont.) An IMAP4 transaction scenario: Chapter 6: Application Layer

49 Open Source: qmail Introduction to qmail:  qmail is a secure, reliable, efficient, simple MTA designed for Unix-like operating systems  qmail is a replacement for the sendmail  qmail is the second most common SMTP server  qmail has by far the fastest growth of any SMTP server Chapter 6: Application Layer

50 qmail System Structure Core modules of qmail: Chapter 6: Application Layer

51 qmail Data Flow Chapter 6: Application Layer

52 qmail Control Files Control files are in /var/qmail/control Some control files of qmail: Chapter 6: Application Layer

53 qmail Queue Structure The qmail queue directory is in /var/qmail/quque Subdirectories in the qmail queue and contents : Chapter 6: Application Layer

54 How Messages Pass Through the qmail Queue Chapter 6: Application Layer

55 6.4 World Wide Web Introduction Web Naming and Addressing Web Data Formats Hypertext Transfer Protocol Open Source: Apache Chapter 6: Application Layer

56 Introduction What is WWW?  A system of Internet servers that support specially formatted documents. How does it work? Chapter 6: Application Layer

57 Web Naming and Addressing Uniform Resource Identifier (RFC 2396) Uniform Resource Locator (RFC 1738) Uniform Resource Name (RFC 2141) Chapter 6: Application Layer

58 Uniform Resource Identifier What is URI?  A compact string of characters for identifying an abstract or physical resource. URI syntax:  Absolute URI: :  Generic URI: :// ? URI examples:  http://speed.cis.nctu.edu.tw/~ydlin/index.html#Books  http://www.google.com/search?q=linux  ftp://ftp.cis.nctu.edu.tw/Documents/IETF/rfc2300~2399/rfc2396.txt  mailto: ydlin@cis.nctu.edu.tw  news: comp.os.linux  telnet://bbs.cis.nctu.edu.tw/ ../icons/logo.gif Chapter 6: Application Layer

59 Uniform Resource Locator What is URL?  A compact string representation of the location for a resource that is available via the Internet URL syntax:  // : @ : / Chapter 6: Application Layer

60 Uniform Resource Locator (cont.) Some URL examples:  http://www.cis.nctu.edu.tw/chinese/ccg/titleMain.gif  ftp://john:secret@ftp.cis.nctu.edu.tw/projects/book.txt  nntp://news.cis.nctu.edu.tw/cis.course.computer- networks/5238  telnet://mail.cis.nctu.edu.tw:110/ Chapter 6: Application Layer

61 Uniform Resource Name What is URN?  A name that identifies a resource of unit of information independent of its location URN syntax:  ::= "urn:" ":"  NID: Namespace Identifier  NSS: Namespace Specific String URN examples:  urn:path:/A/B/C/doc.html  urn:ans:cis.nctu.edu.tw/ydlin/Resource  urn:isbn:0-201-56317-7 URN resolutioin:  http://www.isbn.com/0-201-56317-7 Chapter 6: Application Layer

62 Web Data Formats Evolution of Web Data Formats  SGML  HTML  XML  XHTML Chapter 6: Application Layer

63 Standard Generalized Markup Language What is SGML?  A system for organizing and tagging elements of a document. Characteristics of SGML  Descriptive Markup  Types of Document  Data Independence Chapter 6: Application Layer

64 HyperText Makeup Language What is HTML?  The authoring language used to create documents on the World Wide Web. A brief history of HTML  HTML 2.0, 3.0, 3.2, 4.0 HTML document structure  HTML version information  Header  Body Chapter 6: Application Layer

65 Extensible Markup Language What is XML?  A pared-down version of SGML, designed especially for Web documents. Why XML? How to use XML?  Traditional data processing  Document-driven programming (DDP)  Archiving  Binding Chapter 6: Application Layer

66 Extensible HyperText Markup Language What is XHTML?  A hybrid between HTML and XML specifically designed for Net device displays. Why XHTML? Using XHTML with other W3C tag sets:  XHTML for structural markup of documents  SMIL for multimedia  MathML for mathematics  SVG for scalable vector graphics  XForms for smart web forms Chapter 6: Application Layer

67 Hypertext Transfer Protocol What is HTTP? HTTP Conversation Client Request Server Response HTTP 1.1 Chapter 6: Application Layer

68 What is HTTP? A comprehensive addressing scheme Client-Server Architecture The HTTP protocol is connectionless and stateless An extensible and open representation for data types Chapter 6: Application Layer

69 HTTP Conversation Client request Request headers Request body Server status Response headers Requested data Disconnected Chapter 6: Application Layer

70 Client Request HTTP request methods: Chapter 6: Application Layer

71 Client Request Request example: Chapter 6: Application Layer

72 Server Response Response example: Server status codes: Chapter 6: Application Layer

74 HTTP 1.1 HTTP 1.1 features:  Persistent connections  Chunked encoding  byte ranges  Caching  Non-IP virtual hosting Chapter 6: Application Layer

Web Caching Web caching is a mechanism to expedite document downloading in WWW. To achieve maximum satisfaction from Web caching, some aspects need to be considered.  Candidates to be cached  Content replacement  Cache coherence 75 Copyright Reserved 2010 75 Chapter 6: Application Layer

Transparent Proxy A cache server can also act as a proxy server With port redirection, transparent proxy does not require manual configuration Two types of transparent proxy  integrated within a gateway  standalone server box 76 Copyright Reserved 2010 76 Chapter 6: Application Layer

77 Open Source: Apache Introduction to Apache:  Open-Source Web server originally based on NCSA server  Available on over 160 varieties of Unix -- and Windows NT  Over 58% of Internet Web servers run Apache or an Apache derivative Chapter 6: Application Layer

Chapter 6: Application Layer 79

Ration of sendfile ( ) to ap_invoke_handler ( ) File size1 kb4 kb16 kb64 kb256 kb1024 kb # of calling sendfile ( )1112715 Total execution time (μs) of sendfile ( ) 37 4278215527 Time ratio of sendfile ( ) to ap_invoke_handler ( ) 35%38%40%53%77%87% Chapter 6: Application Layer 84

85 Apache Server Life Cycle On Unix systems, Apache creates multiple processes to handle requests. The Windows and OS/2 ports are multithreaded.. Chapter 6: Application Layer

86 The Request Processing Cycle Chapter 6: Application Layer

87 6.5 File Transfer Protocol File transfer service Operation model Inside the connections Open source: wu-ftpd Chapter 6: Application Layer

88 File Transfer Service Goal  File sharing  Data replication for backup  Efficiency and reliability during transmission Use client-server model based on TCP/IP Authenticated and anonymous accesses Relationship between FTP and Telnet Access an ftp server with browser  ftp://ftp.cis.nctu.edu.tw (anony)  ftp://gis88559@cissol2.cis.nctu.edu.tw (authen) Chapter 6: Application Layer

89 Some Application FTP Commands Command Description OPENConnect to a remote host CATView a file in a remote host GETRetrieve files in a remote host RENAMEChange the name of a file in a remote host RMDelete a file in a remote host QUITTerminate an FTP session Chapter 6: Application Layer

90 Operation Model Client as both control host and receiver PORT: Send the IP and port of the client to which the data is retrieved Chapter 6: Application Layer

91 Inside the Connections Establishing control/data connections  Active Mode Control connection initiated by client Data connection initiated by server  Passive Mode When client is behind a firewall Both control/data connections are initiated by client FTP Reply Example FTP session Error Recovery Chapter 6: Application Layer

92 Active/Passive Mode Active mode Passive mode clientserver Request (Ask server to listen) Reply (Listening on port P of host H) Initiate a data connection firewall clientserver Initiate a data connection firewall Request (Connect to me at port P of host H) Control Data Chapter 6: Application Layer

93 Some Proper FTP Commands Command DescriptionType USERSend the user nameAccess Control PASSSend the passwordAccess Control PORTSend the IP and port of the client to which the data is retrieved Transfer Parameter PASVTell the server to listen on a data port rather than initiate a data connection Transfer Parameter RETRAsk server to transfer a copy of the requested file to the client File service STORCause the server to accept and receive the data and store it as a file File service RNFRSpecify the path of a source file to rename fromFile service RNTOSpecify the path of a destination file to rename toFile service ABORTell the server to abort the previous command and the corresponding data transfer File service Chapter 6: Application Layer

94 FTP Reply Reply DescriptionType 1yzThe requested action is being initiated; expect another reply before proceeding with a new command. Positive Preliminary reply 2yzThe requested action has been successfully completed.Positive Complete reply 3yzThe command has been accepted, but the requested action is being held, waiting for further information from another command. Positive Intermediate reply 4yzThe command is not accepted the and the requested action did not take place. The action can be requested again. Transient Negative Completion reply 5yzSimilar with 4yz, except that the error condition is permanent so that the action cannot be requested again. Permanent Negative Completion reply Chapter 6: Application Layer

95 Error Recovery The restart mechanism  Sender inserts ‘marker’ (used to identify the checkpoint) in the data stream  Receiver marks the position of the marker and reply the latest marker position of both sender and receiver to user  When error, user issues ‘restart’ with the position of the marker to the sender * User (control host) and receiver may/may not exist in the same machine Chapter 6: Application Layer

96 Example FTP Session STATUS:>Connecting to www.cis.nctu.edu.tw (ip = 140.113.166.122) STATUS:>Socket connected. Waiting for welcome message... 220 www.cis.nctu.edu.tw FTP server (Version wu-2.6.0(1) Mon Feb 28 10:30:36 EST 2000) ready. COMMAND:>USER www 331 Password required for www. COMMAND:>PASS ******** 230 User www logged in. COMMAND:>TYPE I 200 Type set to I. COMMAND:>REST 100 350 Restarting at 100. Send STORE or RETRIEVE to initiate transfer. COMMAND:>REST 0 350 Restarting at 0. Send STORE or RETRIEVE to initiate transfer. COMMAND:>pwd 257 "/home/www" is current directory. COMMAND:>TYPE A 200 Type set to A. COMMAND:>PORT 140,113,189,29,10,27  tell the server where to connect to 200 PORT command successful. COMMAND:>LIST  retrieve directory listing 150 Opening ASCII mode data connection for /bin/ls.  File status okay; about to open data connection …….list of files…. COMMAND:>TYPE I 200 Type set to I. COMMAND:>PORT 140,113,189,29,10,31 200 PORT command successful. COMMAND:>RETR test  retrieve the file “test” 150 Opening BINARY mode data connection for test (5112 bytes). Chapter 6: Application Layer

97 Open Source: wu-ftpd Introduction to wu-ftpd  Originally developed at Washington University  Most popular ftp daemon  Maintained by WU-FTPD Development Group. Features and configurations  Virtual ftp servers  On-the-fly compression  Important configuration files Inside wu-ftpd Chapter 6: Application Layer

98 Inside wu-ftpd read ACL files listen to requests connection accepted? fork a handler stand-alone? fork off (parent exits) service initialization reverse DNS check parse & execute commands yes no (under (x)inetd) yes no start ftp server with some options loop until termination signal is received Chapter 6: Application Layer

99 Virtual FTP Servers 1. Manage two or more FTP servers in one machine 2. Support guest groups for different virtual FTP servers … FTP server Configuration file of each virtual server clients ftp.site1.com.tw Lookup the ftpaccess file ftp.site2.com.tw ftp.site3.com.tw # Virtual Server setup for ftp.site1.com.tw virtual ftp.site1.com.tw root /var/ftp/virtual/site1 virtual ftp.site1.com.tw banner /var/ftp/virtual/site1/banner.msg virtual ftp.site1.com.tw logfile /var/log/ftp/virtual/site1/xferlog # Virtual Server setup for ftp.site2.com.tw … A rule segment in ftpaccess Chapter 6: Application Layer

100 On-the-Fly File Compressions Server compress a file (or directory) right before retrieved by users Example User ynlin logged in. Logged in to wwwpc.cis.nctu.edu.tw. ncftp /home/ynlin > ls 1.tar.gz Desktop/ ucd-snmp-4.2.1/ ncftp /home/ynlin > get ucd-snmp-4.2.1.tar.gz ucd-snmp-4.2.1.tar.gz: 7393280 bytes 552.83 kB/s ncftp /home/ynlin >lls -l drwxr-xr-x 24 gis88559 gis88 3584 Oct 8 12:18. drwxr-xr-x 88 root gis88 2048 Sep 10 17:48.. -rw-r----- 1 gis88559 gis88 7393280 Oct 8 12:18 ucd-snmp-4.2.1.tar.gz Chapter 6: Application Layer

101 Important Configuration Files File name Description ftpaccessUsed to configure the operations of the ftp daemon. ftpconversionsSpecify the postfix of a retrieved file and its corresponding operations. ftphostsUsed to deny/allow some hosts to login as certain accounts. ftpserversList the virtual servers and the corresponding directories containing their own configuration files. Chapter 6: Application Layer

102 6.6 Simple Network Management Protocol Background Architectural framework MIB SMI SNMP, SNMPv2, SNMPv3 Open source: NET-SNMP Chapter 6: Application Layer

103 Background Control over network systems are demanded Small tools: ping, traceroute, netstate..etc (base on ICMP) SNMP:  Remote control without being physically attached to managed entities  Exchange management information between network devices Short history  SNMPv1 (1989)– Management framework  SNMPv2 (1993)– Functionality enhancement  SNMPv3 (1998)– Security add-on Chapter 6: Application Layer

104 Architectural Framework trap or response management station MIB agent & managed device in one machine master agent subagent (managed device ) poll or request MIB SNMP Management station Agent Managed device Managed object (specified in MIB) Management protocols Protocols simpler than SNMP Chapter 6: Application Layer

105 Management Information Base (MIB) Tree-like virtual information store Identified by object identifiers Extensible in “experimental” and “private” branches  Register at IANA (http://www.iana.org/) MIB-II: for network management of TCP/IP-based internets (RFC 1213) Chapter 6: Application Layer

106 MIB-II iso (1) org (3) dod (6) …. internet (1) directory (1)mgmt (2)experimental (3)private (4) …. mib-2 (1) system (1)interface (2)at (3)icmp (5)tcp (6)udp (7)egp (8)ip (4) …. transmission (10)snmp (11) OBJECT IDENTIFIER: 1.3.6.1.2.1.4 security (5)snmpv2 (6) …. Chapter 6: Application Layer

107 Object Groups in Mib-2 Group Description systemGeneral information about the managed system interfaceConfiguration information and statistics of each physical interface atAddress translation between network address and physical address ipInformation of implementation and operation of IP in a local system. For example, routing table, default TTL. icmpInformation about the implementation and operation of ICMP. For example, number of messages ICMP sent and received. tcpInformation about the implementation and operation of TCP. For example, the number of maximum and active connections in the system. udpInformation about the implementation and operation of UDP. For example, the number of datagrams sent. egpInformation about the implementation and operation of EGP. tranmissionRelated information and statistics of different transmission schemes. snmpInformation about the accesses and errors of SNMP operations. Chapter 6: Application Layer

108 Structure of Management Information (SMI) Define the structure of a particular MIB  Place restrictions on the types of the objects allowed in the MIB Abstract Syntax Notation one (ASN.1)  Exchange of information between application components in different systems  Syntax – integer, octet string, object identifier  Encode the object values (by Basic Encoding Rule) Define the way data is represented during transmission Three categories of data types  Simple: Primitive ASN.1 data types  Application-wide: Special data types used in particular Applications  Simply constructed: table, row Chapter 6: Application Layer

109 TCP Connection Table (in MIB-II) -- the TCP Connection table -- The TCP connection table contains information about this -- entity's existing TCP connections. tcpConnTable OBJECT-TYPE SYNTAX SEQUENCE OF TcpConnEntry ACCESS not-accessible STATUS mandatory DESCRIPTION "A table containing TCP connection-specific information." ::= { tcp 13 } tcpConnEntry OBJECT-TYPE SYNTAX TcpConnEntry ACCESS not-accessible STATUS mandatory DESCRIPTION "Information about a particular current TCP connection. An object of this type is transient, in that it ceases to exist when (or soon after) the connection makes the transition to the CLOSED state." INDEX { tcpConnLocalAddress, tcpConnLocalPort, tcpConnRemAddress, tcpConnRemPort } ::= { tcpConnTable 1 } TcpConnEntry ::= SEQUENCE { tcpConnState INTEGER, tcpConnLocalAddress IpAddress, tcpConnLocalPort INTEGER (0..65535), tcpConnRemAddress IpAddress, tcpConnRemPort INTEGER (0..65535) } tcpConnState OBJECT-TYPE SYNTAX INTEGER { closed(1), listen(2), synSent(3), synReceived(4) established(5), finWait1(6), finWait2(7), closeWait(8), lastAck(9), closing(10), timeWait(11), deleteTCB(12) } ACCESS read-write STATUS mandatory DESCRIPTION "The state of this TCP connection.." ::= { tcpConnEntry 1 } tcpConnLocalAddress OBJECT-TYPE SYNTAX IpAddress ACCESS read-only STATUS mandatory DESCRIPTION "The local IP address for this TCP connection. In the case of a connection in the listen state which is willing to accept connections for any IP interface associated with the node, the value 0.0.0.0 is used." ::= { tcpConnEntry 2 } tcpConnLocalPort OBJECT-TYPE SYNTAX INTEGER (0..65535) ACCESS read-only STATUS mandatory DESCRIPTION "The local port number for this TCP connection." ::= { tcpConnEntry 3 } tcpConnRemAddress OBJECT-TYPE SYNTAX IpAddress ACCESS read-only STATUS mandatory DESCRIPTION "The remote IP address for this TCP connection." ::= { tcpConnEntry 4 } tcpConnRemPort OBJECT-TYPE SYNTAX INTEGER (0..65535) ACCESS read-only STATUS mandatory DESCRIPTION "The remote port number for this TCP connection." ::= { tcpConnEntry 5 } Chapter 6: Application Layer

110 TCP Connection Table (tabular view) tcpConnTable (1.3.6.1.2.1.6.13) Listen0.0.0.0230.0.0.00 Listen0.0.0.01610.0.0.00 closeWait127.0.0.1161127.0.0.11029 established140.113.88.16423140.113.88.1743082 tcpConnState (x.1.1) tcpConnLocalAddress (x.1.2) tcpConnLocalPort (x.1.3) tcpConnRemoteAddress (x.1.4) tcpConnRemotePort (x.1.5) x.1 tcpConnEntry = (x.1) x.1 INDEX Chapter 6: Application Layer

111 Basic Operations in SNMP PDUDescriptionsVersion GetRequest Retrieve the value of a leaf objectV1 GetNextRequest Get the object lexicographically next to the one specifiedV1 SetRequest Set (update) a leaf object with a valueV1 GetResponse Response for GetRequest (value) or SetRequest (ACK)V1 Trap Issued by agent to notify the management station of some significant event asynchronously V1 GetBulkRequest Retrieve large blocks of data, such as multiple rows in a table. V2 InformRequest Allows one MS to send trap information to another MS and receive a response V2 PDU: Basic data unit in SNMP operations MS : Management Station Variable-binding list: A list of variables and corresponding values in a PDU Chapter 6: Application Layer

112 TCP Connection Table (lexicographical view) tcpConnTable (1.3.6.1.2.1.6.13=x) tcpConnState (x.1.1) tcpConnLocalAddress (x.1.2) tcpConnLocalPort (x.1.3) tcpConnEntry = (x.1) Listen (x.1.1.0.0.0.0.23.0.0.0.0.0) Listen (x.1.1.0.0.0.0.161.0.0.0.0.0) closeWait (x.1.1.127.0.0.1.161. 127.0.0.1.1029) 0.0.0.0 (x.1.2.0.0.0.0.23.0.0.0.0.0) established (x.1.1.140.113.88.164.23. 140.113.88.174.3082) 0.0.0.0 (x.1.2.0.0.0.0.161.0.0.0.0.0) 127.0.0.1 (x.1.2.127.0.0.1.161. 127.0.0.1.1029) 140.113.88.164 (x.1.2.140.113.88.164.23. 140.113.88.174.3082) 23 (x.1.3.0.0.0.0.23.0.0.0.0.0) 161 (x.1.3.0.0.0.0.161.0.0.0.0.0) 161 (x.1.3.127.0.0.1.161. 127.0.0.1.1029) 23 (x.1.3.140.113.88.164.23. 140.113.88.174.3082) … Traverse the tree using Depth First Search Chapter 6: Application Layer

113 User-based Security Model (USM, RFC2574) Four major threats  Modification of Information (between agent and MS)  Masquerade (pretend to be an authorized user)  Disclosure (eavesdropping)  Message Stream Modification (reorder, delay, replay) Solutions  Timeliness module (for 1st, 3rd and 4th threats)  Authentication protocol—use MD5 (for the 2nd threat)  Privacy protocol—Use DES (for 3rd) Chapter 6: Application Layer

114 View-based Access Control Model (VACM RFC 2575) Access control  MIB view-- a collection of MIB objects Elements in the model  Group– Categorization of managers  securityLevel– Help distinguish the access rights of a group  Context– A collection of managed object accessible by an snmp entity.  MIB view– A subset of a particular context  Access policy– Decide the final access rights of a group to a context Chapter 6: Application Layer

115 An SNMP Entity and its Component (RFC 2571) Dispatcher Message Processing Subsystem Access Control Subsystem Security Subsystem SNMP engine Command Generator Command Responder Notification Receiver Notification Originator Proxy Forwarder Other Application(s) SNMP entity Chapter 6: Application Layer

116 Comparisons Between Three SNMP Versions VersionDescription and improvement SNMPv1 1. Define the SMI (RFC1155) 2. A more concise MIB definition (RFC 1212) 3. SNMP framework and its related operations (RFC 1157) 4. Concept of security (authentication) is proposed SNMPv2 1. Improved SMI (support 64-bit counter, and other types of address than IP) 2. Inclusion of ‘GetBulkRequest’ to improve the efficiency for retrieving large blocks of data 3. ‘InformRequest’ for communication between management stations SNMPv3 1. Security and administration add-on’s (RFC2571) 2. Multi-version SNMP message processing and dispatch capability (RFC 2572) 3. The five type of applications within an SNMP engine (RFC 2573) 4. User-based security model (RFC 2574) 5. View-based access control (RFC 2575) Chapter 6: Application Layer

117 Open Source: Net-SNMP Introduction to Net-SNMP Some commands for query, set and trap Extensible architecture  Ways of extending the MIB  Subagent protocol– AgentX (RFC2741)  Build and include a private MIB Inside Net-SNMP  Snmpd and snmptrapd Chapter 6: Application Layer

118 Introduction Started at CMU, moved to UCDavis (1995) and is now based at SourceForge (2000~) What it provides:  An extensible agent  SNMP library for further development  Tools to get or set information from SNMP agents  Tools to generate and handle SNMP traps Support SNMP V1, V2 and V3 Runs on many Unix-like systems and Windows Chapter 6: Application Layer

119 Some Commands for Query, Set and Trap NameDescription and examplePDU used SNMPGET Retrieve the value of a leaf object using get GetRequest SNMPSET Set (update) a leaf object with a value SetRequest SNMPBULKGET Get multiple objects at a time. Possibly under different subtrees GetBulkReques t SNMPWALK Explore all the objects under a subtree of the MIB GetNextReques t SNMPTRAP Uses the TRAP Request to send information to a network manager. More than one object identifiers can be applied as arguments Trap SNMPSTATUS Used to retrieve several important statistics from a network entity. Errors will also be reported, if any SNMPNETSTAT Displays the values of various network- related information retrieved from a remote system using the SNMP protocol Chapter 6: Application Layer

120 Command Line Examples $ /usr/local/bin/snmpbulkwalk -v 3 -u ynlin -l authNoPriv –a MD5 -A ynlinpasswd localhost system system.sysDescr.0 = Linux ynlin2.cis.nctu.edu.tw 2.4.14 #5 SMP 週四 11 月 22 23:6 system.sysObjectID.0 = OID: enterprises.ucdavis.ucdSnmpAgent.linux system.sysUpTime.0 = Timeticks: (30411450) 3 days, 12:28:34.50 system.sysContact.0 = gis88559 system.sysName.0 = ynlin2.cis.nctu.edu.tw system.sysLocation.0 = ynlin2 system.sysORLastChange.0 = Timeticks: (0) 0:00:00.00 system.sysORTable.sysOREntry.sysORID.1 = OID: ifMIB system.sysORTable.sysOREntry.sysORID.2 = OID:.iso.org.dod.internet.snmpV2.snmpB system.sysORTable.sysOREntry.sysORID.3 = OID: tcpMIB system.sysORTable.sysOREntry.sysORID.4 = OID: ip system.sysORTable.sysOREntry.sysORID.5 = OID: udpMIB.. $ snmpget -v 3 -u ynlin -l authNoPriv -a MD5 -A ynlinsnmp localhost system.sysContact.0 system.sysContact.0 = ynlin@cis.nctu.edu.tw $ snmpset -v 3 -u ynlin -l authNoPriv -a MD5 -A ynlinsnmp localhost system.sysContact.0 s gis88559 system.sysContact.0 = gis88559 $ snmpget -v 3 -u ynlin -l authNoPriv -a MD5 -A ynlinsnmp localhost system.sysContact.0 system.sysContact.0 = gis88559 Chapter 6: Application Layer

121 Snmpd and Snmptrapd Snmpd  Process snmp packets and respond with the required information  Can be either master agent or subagent  Basic access control with a port/addr pair  Load dynamical MIB module at start Snmptrapd  Receive and log trap messages  Configured to launch an external program when receiving a trap matching a particular object identifier Chapter 6: Application Layer

122 Processing flow inside Net-SNMP Chapter 6: Application Layer

123 Data Structure of a Session Chapter 6: Application Layer

124 6.7 VoIP Introduction Elements in a VoIP environment Underlying protocols  H.323  SIP Challenges for VoIP technology Open source: openphone Chapter 6: Application Layer

125 Introduction Three ways of transferring voice  Circuit switched  Voice over frame relay  Voice over IP Pros and cons of VoIP compared with PSTN  Inexpensive cost  Simplicity  Less bandwidth consumption  Extensibility Chapter 6: Application Layer

126 VoIP Environment (H.323 adopted) Gateway Telephone Router H.323 gatekeeper H.323 terminal MCU Packet switched networkCircuit switched network Chapter 6: Application Layer

127 H.323 Protocol Stack Q.931H.245 G.711 G.723 G.729 H.263 RTCP RAS (H.225.0) Control (by GK) A / V control AudioVideoControl TCP UDP IP RTP T.120 Data Recommendation Description RAS(H.225.0) Stands for Registration, Admission and Status. H.245 Control protocol for capability negotiation and channel setup. Q.931Used for call setup and teardown between two terminals. T.120Data protocols for multimedia conferencing. (application sharing, whiteboarding) RTP/RTCP Used for real time traffic synchronization and transportation. Chapter 6: Application Layer

128 Setup Procedure of an H.323 Call Registration and admission Call setup Terminal capability negotiation,channel setup and master-slave detection Stable call established and proceeds Close channel Call teardown Disengagement RAS Q.931 H.245 RTP/RTCP H.245 Q.931 RAS Chapter 6: Application Layer

129 Session Initialization Protocol (SIP) SIP  Targeted to replace the H.323 Simplicity Compatibility  Characterize a multimedia session  With supplementary SDP/SAP protocols HTTP-like characteristics  Text-based protocol  Message syntax and header fields identical to HTTP/1.1  Client-server scheme Chapter 6: Application Layer

130 Elements in an SIP Environment Local proxy server Remote proxy server Redirect server Location server User Agent Server (UAS) User Agent Client (UAC) Internet Chapter 6: Application Layer

131 Protocol Stacks of SIP TCPUDP IP SAP/SDPRTPRTCP SIP Multimedia Traffic Control planeData plane Chapter 6: Application Layer

132 Operators and Reply Codes in SIP OperatorsDescription INVITEInvite a user to a call ACKConfirmation for the final response BYETerminate a call between endpoints CANCELTerminate the search for a user or request for a call OPTIONSFeatures supported for a call REGISTERRegister current location of the client with location server INFOUse for mid-session signaling Reply CodeDescription 1xx (Informational)Trying, ringing and queued 2xx (Successful)The request was successful 3xx (Redirection) Give information about the receiver ’ s new location 4xx (Request Failures)Failure responses from a particular server 5xx (Server Failures)Failure responses given when a server itself has erred 6xx (Global Failures)Busy, decline, requests not acceptable Chapter 6: Application Layer

133 Example Operation in SIP 1.UAC INVITE UAS with the SIP URL 2.Destination IP address known => Send the request directly to the destination unknown => Redirected to the local proxy server (with location server). 3. In redirection mode, send back the callee’s new location 4. In proxy mode, forward the request to the destination. 5.The UAS answers 200 (OK), then the UAC ACKs 6.Session established Chapter 6: Application Layer

134 H.323 V.S. SIP Simplicity in operation and implementation Message encoding Capability exchange Supported data types Capability of handling a conference Chapter 6: Application Layer

135 Open Source: Asterisk An integrated PBX (Private Branch eXchange) system Chapter 6: Application Layer

136 Block Diagram Asterisk provides a framework to build a customized VoIP system.  Flexible to add/remove modules to establish the basic transport service 136 Chapter 6: Application Layer

137 Data Structures Two concepts, context and extension, are introduced 137 Chapter 6: Application Layer

138 Call flow 138 Chapter 6: Application Layer

140 6.8 Streaming Introduction to streaming  Availability  Short fetching time  Avoid storage overhead  Live broadcasting Architecture and components for streaming  Compression scheme  Protocol stack for streaming  QoS and synchronization mechanism Open source: Darwin streaming server Chapter 6: Application Layer

141 Architecture and Components for Streaming Video Files Audio Files Transport Protocols QoS Control Server Video DecoderAudio Decoder Transport Protocols QoS Control Client synchronization Video/Audio raw data Internet Chapter 6: Application Layer

142 Compression Schemes Temporal or spatial  Temporal – describe changes between frames  Spatial – describe similar patterns in a frame Lossless or lossy  Recoverability when decompressed  GIF (lossless) and JPG (lossy) Symmetrical or Asymmetrical  The time for compression and decompression  According to the load of the server Chapter 6: Application Layer

143 Protocol Stacks of Streaming TCPUDP IP RTSPRTCP RTP Control planeData plane RTSP (Real Time Streaming Protocol)  VCR-Style remote control functionalities  Establish and control streams Description file before a session Methods in a session (SETUP, PLAY, TEARDOWN)  Reuse of HTTP concepts (Ex: ASCII messages) HTTP  Mostly by buffering  Low performance HTTP Chapter 6: Application Layer

144 QoS and Synchronization Mechanism QoS mechanism  Source-based rate control  Receiver-based rate control Synchronization mechanism  Intra-stream synchronization Continuity of audio/video data frames inside a stream  Inter-stream synchronization Consistency between cooperative streams  Inter-object synchronization Consistency between streams and other objects like text and still images Chapter 6: Application Layer

145 Open Source: Darwin Streaming Server Open source version of Apple's QuickTime Streaming Server (QTSS) Deliver streaming media with RTP and RTSP Provide both live and on-demand programs Supports many formats: H.264/MPEG-4 AVC, MPEG-4 Part 2, 3GP and MP3 Provide Web-based administration, authentication, relay support, and integrated broadcaster administration, etc. Chapter 6: Application Layer

146 Block Diagram Two parts: core server and modules  Core server provides task scheduling  Module provides specific functions 146 Chapter 6: Application Layer

147 Data Structures 147 Chapter 6: Application Layer

148 Call Flow: Task Handling 148 Chapter 6: Application Layer

149 RTSP Handling State Transition Diagram 149 Chapter 6: Application Layer

150 6.9 Peer-to-Peer Applications (P2P) Introduction P2P Architectures Performance Issues of P2P Applications Case Study: BitTorrent Open source: Vuze (BitTorrent Client) 150 Chapter 6: Application Layer

151 Introduction to P2P A distributed network architecture in which participants act as both a client and a server Participants construct a virtual overlay network at the application layer on top of the underlying IP network Node: peers Edge: TCP/UDP connection 151 Chapter 6: Application Layer

Introduction to P2P Operations in P2P systems  Joining the P2P overlay network,  Resource discovery The most challenging problem for P2P applications  Resource retrieval Chapter 6: Application Layer 152

P2P Architectures Way of forming a P2P overlay network could be classified into three categories  Centralized  Decentralized and unstructured  Decentralized but structured Chapter 6: Application Layer 153

Centralized P2P Operation overview Chapter 6: Application Layer 154

Centralized P2P Utilizes a central directory server for locating objects in the P2P network  Peers join by registering to the directory server  Peers inform the directory server of objects to be shared  A peer sends query to the directory server to search an object  The peer receives reply from the directory server  The peer selects one or more peer in the reply to download the object directly from  Example: Napster Chapter 6: Application Layer 155

Centralized P2P Advantages  Simple  Easy to implement  Support various kinds of search such as keyword, full-text, and metadata search Disadvantages  Not a true P2P system as it relies on a central directory server which becomes performance bottleneck, single point of failure, not scalable and vulnerable to DoS attacks Chapter 6: Application Layer 156

Decentralized and Unstructured P2P To get rid of server, this approach floods query messages to peers to search for shared objects  Limited-scope flooding is adopted to reduce flooding messages  A query hit message is returned along the reverse path back to the inquirer Chapter 6: Application Layer 157 Example: Gnutella

Decentralized and Unstructured P2P Join procedure  A peer needs to know at least one of the peers already on the overlay network.  The peer sends a join message to a peer already on the overlay.  The existing peer then replies its identity as well as a list of its neighbors. It may also forward the join message to its neighbors.  Upon receiving join reply messages, the newcomer knows more peers on the overlay. Chapter 6: Application Layer 158

Decentralized and Unstructured P2P Hierarchical overlay with super peers  Flooding is apparently not scalable  FastTrack adopts a hierarchical overlay  A super peer acts as a local directory database which stores the indexes of objects shared by ordinary peers  Two-level hierarchical overlay The lower level adopts the central server approach The upper level (super peers) adopts the decentralized and unstructured approach. Chapter 6: Application Layer 159

Decentralized and Unstructured P2P Advantages  Fully distributed  Reliable, fault-tolerant  No single point of failure Disadvantages  Excessive query traffic make it not scalable  May fail to find content that is actually in the system  Super peer may become overloaded or been attacked Chapter 6: Application Layer 160

Decentralized but Structured Combine the distributed directory service with an efficient query routing scheme Key ideas  For distributed directory service, a hash function maps peers and objects into the same address space so that objects can be deterministically assigned to peers in a distributive manner.  For efficient query routing, peers are organized into a structured overlay based on their positions in the address space. Chapter 6: Application Layer 161

Decentralized but Structured Operations overview  Each peer generates its own node ID by a predefined hash function.  For each object being held and to be shared, the peer generates the object ID by the same or another hash function.  For each object, the peer sendd a register message to the node that has the node ID same as the object’s ID.  If a peer wants to query an object, it uses the hash function to generate the object ID and sends the query message to the node that hosts the object’s ID. Chapter 6: Application Layer 162

Decentralized but Structured Message routing (use Chord as an example)  Key idea: have each peer maintain a specially designed routing table such that every peer could forward the arriving message to a neighboring peer with node ID that is further closer to the destination.  Consider a 10-node Chord overlay in a 6-bit address space  Chord views its address space as a one- dimensional circular space such that peers in the space form a ring overlay. Chapter 6: Application Layer 163

Message Routing in Chord The routing table in Chord is called a finger table. For an m-bit address space, the finger table of a node with ID=x consists of at most m entries and the i-th entry points to the first node with ID following the ID of x+2 i-1 modulo 2 m, for 1≤i≤m. Chapter 6: Application Layer 164

Finger Table of Chord Finger table of node N8, where m =6. Chapter 6: Application Layer 165

Routing a Query Message Routing a query message for object 54 from N8 Chapter 6: Application Layer 166

167 Performance Issues of P2P Applications Free Riding Flash Crowd Topology Awareness NAT Traversal Churn Security Copyright Infringement Copyright Reserved 2010 167 Chapter 6: Application Layer

Free Riding Scalability of P2P systems relies on the contribution from peers  If a peer only consumes but contributes little or no resources, it becomes a free rider of the system.  85% of peers share no files in Gnutella in 2005 A common solution is to implement some incentive mechanisms.  For example, tit-for-tat in BitTorrent. Other solutions: reward-based and credit- based mechanisms Chapter 6: Application Layer 168

Flash Crowd A sudden, unanticipated growth in the demand of a particular object  e.g., a new release of a DVD video or mp3 file Issues  how to deal with a sudden large amount of query messages  how long to find and download the object within a short time period Solutions  Cache, duplicating popular objects Chapter 6: Application Layer 169

Topology Awareness A virtual link in a P2P overlay could be a long end-to-end connection across continents or a short one within a local area network  How to avoid serious topology mismatch Solutions  Many route-proximity and neighbor-proximity enhancements for P2P overlay systems have been proposed based on RTT measurement, preference of routing domain or ISP, or geographical information. Chapter 6: Application Layer 170

NAT Traversal Basic requirement for a P2P system is to provide peers with NAT traversal mechanisms  If both peers are behind NAT devices, they cannot connect to each other without help from other peers or STUN servers Solutions  In most cases, NAT traversal is solved by relay peers or super peers that have public IP addresses Chapter 6: Application Layer 171

Churn Churn refers to the phenomenon that peers dynamically join and leave the system at will.  high churn rate seriously affects the stability and scalability of a P2P system.  e.g., a high churn rate may cause a tremendous overlay maintenance overhead and dramatic routing performance degradation in DHT-based system Solutions  Avoid a rigid structure or relation among peers  Peers maintain a list of potential neighbors for quick and dynamic neighbor replacement Chapter 6: Application Layer 172

Security Security issues  P2P programs with back hole (Trojan Horse), spurious content, leaking of files not to be shared. Solutions to content pollution  protect the content with message digest such as MD5 In BitTorrent, the MD5 digest of each piece of a shared file is stored in the metadata file  peer reputation system  object reputation system Chapter 6: Application Layer 173

Copyright Infringement Sharing copyrighted objects through P2P systems is a serious problem which hinders the promotion and usage of P2P systems. Not only P2P users are responsible for copyright infringement, so are the companies that host P2P applications  Especially in the case where P2P systems will not be able to exist without their serves, e.g., Napster. Chapter 6: Application Layer 174

175 Case Study: BitTorrent BitTorrent (BT) is originally designed by Bram Cohen in 2001 Well thought-out protocol with several unique features  use tit-for-tat as an incentive mechanism to cope with free riders  use out-of-band search to avoid copyright infringement issue  use pull-based swarming for load balancing  use hash check to prevent propagation of spurious pieces  after a peer has successfully downloaded a file, it becomes a seeder to distribute the file Copyright Reserved 2010 175 Chapter 6: Application Layer

BT Terminologies A file is cut into pieces of a fixed size. A piece is further divided into chunks, the basic data unit for a peer to request for content. The integrity of a piece is protected by a SHA-1 A peer becomes a seeder if it has successfully downloaded the file. A tracker for each file or group of files to be shared. The tracker tracks the downloading peers and seeds, and coordinates the file distribution among peers. Chapter 6: Application Layer 176

BT Operation Overview Chapter 6: Application Layer 177

Piece Selection Random first piece selection  For the first few pieces, the client just randomly selects a piece to download. Rarest first policy  Selects the most scarce piece to download first End-game mode  To speed up the completion of a file download at the end, a peer with only a few pieces missing will send requests for all missing pieces to all the peers Chapter 6: Application Layer 178

Peer Selection Choking/unchoking  Choking refers to a temporal refusal to upload to a peer.  At the beginning, all peers are chocked  Tit-for-tat algorithm selects a fixed number of peers from which the peer downloaded most to unchoke Optimistic unchoking  new peer needs to move its first step when initially joined the system  select one peer at random Anti-snubbing  If a peer is choked by all of its peers (snubbed), it is better to run optimistic unchoking more often to explore more peers that are willing to cooperate. Chapter 6: Application Layer 179

180 Open source: Vuze (BitTorrent Client) Many BT client programs are open-source Some popular client programs  Vuze  uTorrent  BitComet Copyright Reserved 2010 180 Chapter 6: Application Layer

Core Packages Most of Vuze’s core packages are located under the.\com\aelitis\azureus\core directory Chapter 6: Application Layer 181

Data Structure Classes for the peer and piece objects Chapter 6: Application Layer 182

Algorithm Implementation Main program for controlling the piece and peer selection is the PEPeerControlImpl class Chapter 6: Application Layer 183

Implementation of Peer and Piece Selection Unchoking algorithms are implemented in DownloadingUnchocker.java and SeedingUnchocker.java  tit-for-tat is implemented in calculateUnchokes()  Optimistic unchoking is implemented in UnchokerUtil.getNextOptimisticPeer() getRequestCandidate() defined in PiecePickerImpl.java is the core method for deciding which block to download Chapter 6: Application Layer 184

185 6.10 Summary General issues  well-known ports, concurrent connection-oriented servers and iterative connectionless servers, statefulness/statelessness Major application protocols  DNS, SMTP, POP3, IMAP4, HTTP, FTP, SNMP Real time, multimedia application protocols  SIP, RTP, RTCP, RTSP P2P protocols  Napster, Gnutella, DHT, Chord, BT Chapter 6: Application Layer

1 Computer Networks An Open Source Approach Chapter 6: Internet Services Chapter 6: Application Layer.

Similar presentations

Presentation on theme: "1 Computer Networks An Open Source Approach Chapter 6: Internet Services Chapter 6: Application Layer."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

1 Computer Networks An Open Source Approach Chapter 6: Internet Services Chapter 6: Application Layer.

Similar presentations

Presentation on theme: "1 Computer Networks An Open Source Approach Chapter 6: Internet Services Chapter 6: Application Layer."— Presentation transcript:

Similar presentations

About project

Feedback