1. Web Application Development Instructor: Matthew Schurr

2. Components of the Web BrowserServer HTTP Request HTTP Response Internet  Browsers make requests over the Internet using HTTP. HTTP = Hyper Text Transfer Protocol  Servers respond with files that your browser displays.  Servers are simply computers optimized for serving files.

3. Web Browsers  Locally stored and executed software  Used for traversing, retrieving, and presenting resources on the Internet over HTTP  Combine HTML markup, CSS markup, JavaScript code, images, and other assets to create the inter-active visual display seen by users  In Practice: Many different implementations of same specifications ○ Firefox, Chrome, Safari, Opera, Internet Explorer, etc. Many different user hardware configurations ○ Desktop with large displays vs. mobile phones Diversity leads to compatibility challenges

4. Web Servers  Any software that allows a computer to deliver content to clients over the Internet using HTTP.  May deliver static content or content generated dynamically by a program  Server-side programs can be used to add interactivity and user-based customization to websites Forms, file uploads, etc. Unique experiences for each user (e.g. Facebook) Computational resources (e.g. Wolfram Alpha, Google)

5. Static vs. Dynamic Content  Static Content (pre-written, re-usable files) (most) images Most content in ‘90s was static Examples: a logo, a PDF document  Dynamic Response built on the fly by a program Most content today is dynamic Examples: Reddit, Facebook News Feed, Google Search  Server-Side Programs A program that exists on a web server, speaks HTTP, and generates content dynamically. A fundamental component of interactive web applications.

6. Hyper Text Transfer Protocol  Communication protocol through which browsers and servers interact. Messages are formatted to be human-readable text. Capable of transferring any type of binary data.  HTTP is a request-response protocol. Browser sends a single request to a server, server sends a single response back.  Servers are response-only. Can only respond to requests initiated by browser. Question: What limitations does this create?

7. HTTP Request HTTP Response Additional Back-End Services (Database, Files, Cache, etc.) BROWSER Intranet (Private) Web Server Server-Side Program Internet (Public) INPUT OUTPUT

8. Aside: Limitations of HTTP  A page view is a full request-response cycle.  Pages may issue background requests to pull new information.  Question: How would you implement a real time chat application? Users should not need to reload the page to get new messages. But the servers cannot push data to the browser since they are response-only.

9. Aside: Limitations of HTTP  Solution: Polling Periodically make requests to the server to update the display, regardless of whether or not new information is available.  Problem: Efficiency >99% of the time there will be no new data.  Alternate Solution: Web Sockets Allow for two-way communication similar to C sockets Browser support growing, but many people don't update

10. Internet Basics  All machines connected to the Internet are given a unique IP address.  Internet provides unreliable delivery of data packets (chunks of bits) from one IP address to another.  TCP/IP (Transmission Control Protocol) provides reliable, in-order delivery of messages over the network. HTTP messages are encapsulated within TCP/IP packets. Key Idea: HTTP messages are transmitted and received in entirety and without error.

11. Internet Protocol Address (IP)  A unique, numerical address assigned to each device connected to the Internet. Caution: an IP address may be assigned to your router, not your individual computer. Implication: two distinct computers behind the same router may have the same public IP. ○ IP addresses are not a reliable unique identifier  IPv4: 32-bits (pretty-printed) 156.73.123.10  IPv6: 128-bits 2001:db8:85a3:8d3:1319:8a2e:370:7348

12. Domain Name System (DNS)  Idea: large binary numbers are difficult for humans to remember. Prefer easy to remember names like "rice.edu".  Domain Name System – system responsible for translating easily memorized hierarchical names to numerical IP addresses e.g. "rice.edu"  128.42.204.44 Available to all computers connected to the Internet. Take COMP 429 for implementation details.

13. Uniform Resource Locator (URL)  A well-defined string format for specifying the location of a remote object. e.g. https://www.facebook.com/profile?id=101293123  Contains several parts: Protocol Host Port (optional) Path (optional) Query String (optional) Fragment (optional)  Best demonstrated by an example.

14. URL: Protocol  Tells the browser what message format should be used when sending requests. On the Internet today, primarily http:// or https://.http://https://  Will discuss differences between HTTP and HTTPS later. http://www.google.com:80/search?q=cat+pictures#result-10

15. URL: Host  Tells the browser where on the Internet to send requests. Can be a domain name or an IP address. ○ Recall: Packets are addressed to an IP address. ○ DNS can translate domains to IP addresses.  Domain names are hierarchical. Components are separated by dots. Top-Level Domain: right-most component (e.g. "com") Root Domain: second right-most component (e.g. "google") Subdomain(s): all other parts of the domain name, may have arbitrary depth (e.g. "www") http://www.google.com:80/search?q=cat+pictures#result-10

16. URL: Port (Optional)  Identifies a port on the remote machine. Default: 80 for HTTP, 443 for HTTPS  TCP/IP packets are addressed to both an address and port. Ports are simply a 16-bit unsigned integer. Ports identify a process (program) on the server. Enables single machine to host multiple services. ○ via demultiplexing of incoming data streams. http://www.google.com:80/search?q=cat+pictures#result-10

17. URL: Path (Optional)  Refers to a file location on the remote server. Allows the server to decide how to process the incoming request.  Older servers: refers to a file location relative to some "root" directory that was made public by the server.  Modern servers: virtual directory structure that maps paths to procedures in a program http://www.google.com:80/search?q=cat+pictures#result-10

18. URL: Query String (Optional)  A set of key-value pairs attached to the request. May be used by the server to decide what to do with the incoming request.  Commonly referred to as "GET parameters".  Provide extra information server may need when servicing the request. http://www.google.com:80/search?q=cat+pictures#result-10

19. URL: Fragment (Optional)  A string that refers to a specific scrolling point (anchor) on a web page. May also be used to indicate state for JavaScript applications.  Not transmitted over the network as part of an HTTP request – purely client-side. http://www.google.com:80/search?q=cat+pictures#result-10

20. HTTP Request Cycle  Question: What do you think happens when you enter a URL into your browser?  Parse Extract needed information from the URL  Resolve Translate domain name into IP  Request Establish connection, create HTTP request, and send it  Process Receive HTTP response, process response, close connection

21.  Protocol: HTTP  Host: www.google.com  Port: 80  Path: /search  Query String: ?q=cat+pictures  Fragment: result-10 ParseResolveRequestProcess http://www.google.com:80/search?q=cat+pictures#result-10

22.  The provided host ("www.google.com") is not an IP address. Can be translated to an IP by DNS. Use the "nslookup" utility in terminal. Choose any returned address and continue. ○ Question: Why are multiple addresses returned? ParseResolveRequestProcess ~$ nslookup www.google.com Non-authoritative answer: Name: www.google.com Addresses: 2607:f8b0:4002:c01::68 74.125.137.106 74.125.137.147 74.125.137.104 74.125.137.105 74.125.137.99 74.125.137.103

23.  URL specifies to send a HTTP message.  Build request, establish connection to the address on port 80, and send the request.  Assembled request (sent over the Internet): ParseResolveRequestProcess http://www.google.com:80/search?q=cat+pictures#result-10 GET /search?q=cat+pictures HTTP/1.1 User-Agent: Mozilla/5.0 Firefox/22.0 Host: google.com Connection: close Accept-Encoding: none Accept: */*

24. HTTP Request Structure  There are two main methods in HTTP: GET – you want to retrieve information from the server POST – you want to send information to the server (the request body) and get back a response. Used for file uploads and sending form data.  The request contains the path and query string from the URL. The server uses this when deciding how to service the request. Notice: this does not contain the host or fragment! METHOD PATH+QUERY_STRING HTTP/1.1 Header: Value (Request Body *Optional)

25. HTTP Request Structure  Contains headers which provide the remote server with information about the capabilities of the client browser.  May contain a body. When sending information to the server (POST requests), the body contains the data (a set of key-value pairs in a format similar to that of the query string).  Note: In actuality, end of the headers section is indicated by a double new line. METHOD PATH+QUERY_STRING HTTP/1.1 Header: Value (Request Body *Optional)

26. GET Parameters  Key-value pairs that are encoded into the URL. Located in the query string. Value is optional and can be left blank. Values must be URI encoded (more on this later).  Available on both POST and GET requests.  Number of pairs limited by maximum URL length. Usually ~2kb, varies from server to server.  Example: ?key=value&key2=value2… ?key&key2=value&key3…

27.  Wait for response from server and then close the connection.  Process the returned response to create a visual display in the browser.  Response may look like: ParseResolveRequestProcess HTTP/1.1 200 OK Server: Google Web Server Connection: Close Content-Type: text/html Content-Length: 3538 …

28. HTTP Response Structure  Code – indicates the status of the response  Status – standardized string describing code  Headers – describe the content and set rules for behavior between the client and browser  Body – the content itself. May be an HTML document, an image, or other binary data HTTP/1.1 CODE STATUS Header: Value (Response Body)

29. HTTP Status Codes  Predefined integer codes that tell browser status of request.  Examples: 200 OK 400 Bad Request 401 Unauthorized 403 Forbidden 404 Not Found 405 Method Not Allowed 500 Internal Server Error 501 Not Implemented

30. HTTP Headers  Content-Type – describes the content by its MIME type (a short, standardized string). Tells browser type of data the response contains. Enables browser to properly display data. Required by protocol.  Content-Length – describes the length of the content in bytes.  There are many more… you can guess at meaning of most or look them up.

31. HTTP MIME Types  Short, standardized strings that describe the type of content contained in response.  Response body is just a sequence of bits. MIME tells browser meaning of the bits. Image? PDF document? HTML document? Enables the browser to properly display the content.  When opening file locally, done by looking at the file extension. HTTP paths may not contain file extensions! Interpretation depends solely upon Content-Type header.

32. HTTP MIME Examples MIME Type StringDescription text/plainPlain Text text/htmlHTML Mark-Up (Text) text/cssCSS Mark-Up (Text) application/zipZIP Archive application/javascriptJavaScript Code (Text) image/jpegJPEG Image Format image/pngPNG Image Format image/gifGIF Image Format application/pdfAdobe PDF Document

33. HTTP Host Header  Recall: Host is not sent with path and query string at top of the request.  The same physical server (IP address) may host web sites for multiple domains. Host header provides way to distinguish between traffic destined for one domain vs. another. Mandatory in HTTP/1.1.

34. HTTP Headers: Additional Uses  Set rules for behavior of connection between browser and server.  Inform server about capabilities of browser and vice-versa.  Synchronization of dates between browser and server.  Facilitate caching and compression to optimize bandwidth usage.  Redirect clients to alternate locations  Debugging

35. Stateless Protocols  HTTP is a stateless protocol.  Each request-response cycle is treated as an independent transaction. Each transaction is unrelated to any previous ones.  The protocol implementation does not retain session information or status of communication partners for the duration of several requests.  Every single HTTP request is stand-alone and must contain all of the information necessary to properly service the request.

36. Stateless Protocols  Question: How do modern web applications maintain state?  Answer: The maintenance of any state between requests is handled entirely by the server-side program(s) servicing those requests. State is NOT handled by implementations of HTTP. State is NOT a requirement of HTTP. Provides maximum flexibility to application developers.  Key Idea: Statelessness is a fundamental design component of scalable applications.

37. GET /login HTTP/1.1 HTTP/1.1 200 OK Content-Type: text/html (login form HTML) POST /login HTTP/1.1 Content-Type: (data) user=(un)&pass=(pw) FETCH DATA HTTP/1.1 302 FOUND Location: /restricted-page Set-Cookie: sessionid=XXXXXXXX GET /restricted-page HTTP/1.1 Cookie: sessionid=XXXXXXXX STORE FETCH DATA HTTP/1.1 200 OK Content-Type: text/html (restricted access page HTML) Note: "sessionid" will be attached to all future requests. Statelessness in Action SERVER-SIDE PROGRAM PERSISTENT STORAGE SYSTEM

38. Cookies  The HTTP specification allows for the server to store small pieces of data (<4kb) with the client. These pieces of data are called cookies.  After a server stores a cookie with a client, the client transmits the cookie back to the server on each subsequent request. Server can then access the value and make use of it.  The setting and sending of cookies occurs using headers.

39. Uses of Cookies  Tracking a user's visits Advertisers Analytics  Identifying a user's session or login Set a long, random, unique identifier that will be attached to each future request  Storing any other small piece of data

40. Cookies and Domains  Cookies are stored on a per-domain basis. Only attached to requests for domain(s) on which they are set. Prevents applications running on one domain from accessing cookies for another domain. Subdomains may access cookies for parent domains if explicitly declared when setting the cookie.  Implications: Prevents google.com (or any other domain) from accessing your cookies for facebook.com. Prevents one web application from stealing your credentials to another.

41. Limitations of Cookies  Cookies are stored completely client-side. Malicious users can tamper with the values, delete cookies, or create cookie with any properties on a whim (will discuss how to deal with this later).  Support for cookies is not a requirement of HTTP. Users can disable cookies in their browser options. No guarantee that a cookie you request to set will be sent back on subsequent requests. Server-side programs must account for this.

42. Properties of Cookies  Name – the name of the cookie  Value – the data you want to store  Expiration – the time at which the cookie should cease to be attached to future requests  Path – the path(s) on the server at which the cookie should be accessible  Domain – the domain(s) at which the cookie should be accessible  Secure – whether the cookie should only be sent over encrypted HTTPS connections  HTTP Only – whether the cookie should only be sent over HTTP (no JavaScript access)

43. POST /login HTTP/1.1 Host: www.yoursite.com User-Agent: Chrome/32.2 Content-Type: application/json Content-Length: 39 {'user': 'admin', 'password': 'qwerty'} HTTP/1.1 200 OK Server: nginx Date: Sat, 20 Dec 2014 00:00:00 GMT Content-Type: text/html Set-Cookie: sessionid=XXXXXX; path=/; expires=01-Jan-2015 00:00:00 GMT; domain=.yoursite.com; httponly (HTML response redacted) Web Server Server-Side Program Additional Back-End Services (Database, Files, Cache, etc.) BROWSER Intranet (Private)

44. Firefox Developer Tools

45. Firefox Web Console

46. Firefox Cookie Viewer

48. Firefox Network Tool  You should also take a look at the Network tab on the web developer tools console.

49. HOMEWORK : Assignment 1  You can view the assignment on the course website (or in the syllabus).  Complete the assignment and turn it in on Owl-Space within two weeks from today.

50. Next Week: Intro to MVC Software Design HTML (Hyper Text Markup Language)

51. Aside: HTTPS  Process for sending a request over HTTPS is almost exactly the same as for HTTP!  The HTTP packets are assembled as normal but, before transmission, the entire packet is encrypted.  Upon arrival, the entire packet is decrypted before processing.

52. Aside: HTTPS  Packet is only visible in its encrypted form as it traverses the Internet. Only intended recipient can decrypt it! No one can eavesdrop on the packet (which may contain passwords, credit card info, etc.).  Protocol is largely transparent to programmers! Encryption/decryption processes occur automatically. Don't need any extra code to deal with HTTPS.

53. Aside: Email  Email is not sent over HTTP; email has its own protocols and client/server implementations. Outbound: SMTP, SMTPS Inbound: POP3, POP3S, IMAP, IMAPS  Process very similar to HTTP, but transmitted message is in SMTP, POP3, or IMAP. Messages are transmitted to the IP of email server for sender's domain. Email server will handle delivery by forwarding message to recipient domain's server.

54. Aside: G-Mail  A server-side application that acts as an intermediary between us (using HTTP) and Google's mail server (using SMTP/IMAP).  Viewing Message: HTTP GET request to Gmail, Gmail downloads messages from mail server over IMAP, Gmail formats messages into HTML, Gmail sends formatted HTML in HTTP response  Sending Message: HTTP POST request to Gmail, Gmail translates into SMTP request to the mail server, mail server delivers message

55. Web Application Development Instructor: Matthew Schurr

56. Model-View-Controller Design  Key Idea: Data and visualizations of data are separated by a level of indirection. Can re-use complicated UI elements. Can re-use model code to make multiple front-ends. View (User Interface) Controller Model (App Data) User Interaction Update Notify Update

57. Model  A bundle of an application's data and the algorithms that act upon that data.  The storage system that manages all of an application’s information. On Facebook: Users, Comments, Posts, Photos…  In this class: Relational Database Systems, SQL, and ORM

58. View  Any visual representation of application. Presented to the user. May be interactive.  Bundles visual representations together with algorithms that create them. e.g. a drop-down selection box ○ Contains a lot of functionality abstracted into a simple to use interface; often support for events and/or delegates.  Composition: views may be placed inside of other views. e.g. a form is a combination of labels, text input boxes, buttons, etc.  Inheritance: views may inherit functionality from other views.

59. View (Example): Selection Box  What does a selection box fundamentally need to know in order to do its job? A mapping of options (IDs/keys) to strings. A way to notify when and which option is selected

60. View (Example): Selection Box  Does a selection box need to know anything about the purpose of what it is displaying?  Does a selection box need to know anything about the underlying data of the application?  This is the power of MVC. Created a high-level UI widget with lots of functionality Widget knows nothing but what is absolutely necessary Widget is highly re-usable

61. View  Everything you see in your browser window is the top level view.  In this class: PHP HTML (Hyper Text Markup Language) CSS (Cascading Style Sheets) JavaScript Images, videos, files, etc.

62. Controller  Provides a level of indirection that provides compatibility between a model and a view.  Responsibilities: Mediates input so that it can be understood by the model. Listens for changes to the model and updates the view. Converts data from the model to a format that can be understood by the view and vice-versa.

63. Controller  In this class: JavaScript (Client-Side Programming) PHP (Server-Side Programming) HTTP (Controller and View Communication)  Our controllers (written in PHP) will generate views to send to the client by reading from the model and make changes to the model based on incoming user input.

64. The Big Debate: What You Know  MVC is an object-oriented design pattern.  On the web, the display in the browser is produced by parsing HTML markup, CSS markup, and images.  Where does this markup come from? From HTTP responses (containing the HTML/CSS markup) that were generated by a program running server-side. ○ C, PHP, Python, C++, Scala, Java, Ruby, etc. By being dynamically created by code that runs client-side (in the browser). ○ JavaScript code is downloaded over the Internet from the server and then executed locally to generate an HTML document (the result of parsed HTML and CSS).

65. The Big Debate: Client-Server MVC  Question: What functionality should live on the client, and what should live on the server? Should an MVC system live client-side, server-side, or on both? If both, should they be running the same or different MVC system? Should views be generated client-side, server-side, or by some combination of both?

66. Fundamental Questions  During the course, we'll focus on: How is data exchanged between the client and server? What are the capabilities of the client (browser)? What are the capabilities of server-side infrastructure? What are the security implications of implementing certain functionality client-side vs. server-side? What are the scalability implications of implementing certain functionality client-side vs. server-side?  At the end of the course, we'll discuss: What functionality should be implemented server-side? Why? What functionality should be implemented client-side? Why?