1. Lecture 5 Dynamic Web Servers CPE 401 / 601 Computer Network Systems slides are modified from Dave Hollinger

2. Web Server Talks HTTP Looks at METHOD, URI to determine what the client wants. For GET, URI often is just the path of a file – relative to some directory on the web server Dynamic Web Servers 2

3. GET /foo/blah Dynamic Web Servers 3 usrbinwwwetcfoofungif / blah

4. Dynamic Documents Dynamic Documents can provide: – automation of web site maintenance – customized advertising – database access – shopping carts – date and time service – … Dynamic Web Servers 4

5. Web Programming Writing programs that create dynamic documents has become very important There are a number of general approaches: – Create custom server for each service desired Each is available on different port. – Develop a real smart web server Server Side Includes, scripting, server APIs – Have web server run external programs Dynamic Web Servers 5

6. Custom Server Write a TCP server that watches a “well known” port for requests Develop a mapping from http requests to service requests Send back HTML (or whatever) that is created/selected by the server process Have to handle http errors, headers, etc Dynamic Web Servers 6

7. Drawbacks to Custom Server Approach We might have lots of ideas custom services – Each requires dedicated address (port) – Each needs to include: basic TCP server code parsing HTTP requests error handling headers access control Dynamic Web Servers 7

8. Smart Web Server Take a general purpose Web server (that can handle static documents) and – have it process requested documents as it sends them to the client The documents could contain commands that the server understands – the server includes some kind of interpreter Dynamic Web Servers 8

9. Example Smart Server Have the server read each HTML file as it sends it to the client The server could look for this: some command The server doesn’t send this part to the client, instead it interprets the command and sends the result to the client Everything else is sent normally Dynamic Web Servers 9

10. Server Side Includes Server Side Includes (SSI) provides a set of commands that a server will interpret Typically the server is configured to look for commands only in specially marked documents – so normal documents aren’t slowed down SSI commands are called directives – Directives are embedded in HTML comments Dynamic Web Servers 10

11. SSI Directives A comment looks like this: A directive looks like this: SSI servers keep a number of useful things in environment variables: DOCUMENT_NAME, DOCUMENT_URL echo: inserts the value of an environment variable into the page This page is located at Dynamic Web Servers 11

12. SSI Directives include: inserts the contents of a text file. flastmod: inserts the time and date that a file was last modified. Last modified: exec: runs an external program and inserts the output of the program. Current users: Dynamic Web Servers 12 Danger! Danger! Danger!

13. More Power Some servers support elaborate scripting languages Scripts are embedded in HTML documents, the server interprets the script: – Microsoft Active Server Pages (ASP) JScript, VBScript, PerlScript – Netscape LiveWire JavaScript, SQL connection library. – Many others… Dynamic Web Servers 13

14. Server Mapping and APIs Some servers include a programming interface that allows to extend the capabilities of the server by writing modules Specific URLs are mapped to specific modules instead of to files Dynamic Web Servers 14

15. External Programs Another approach is to provide a standard interface between external programs and web servers – We can run the same program from any web server – The web server handles all the http, we focus on the special service only – It doesn’t matter what language we use to write the external program Dynamic Web Servers 15

16. Common Gateway Interface CGI is a standard interface to external programs supported by most (if not all) web servers – CGI programs are often written in scripting languages (perl, tcl, etc.), The interface that is defined by CGI includes: – Identification of the service (i.e.,external program) – Mechanism for passing the request to the external program Dynamic Web Servers 16

17. Common Gateway Interface CGI is a standard mechanism for: – Associating URLs with programs that can be run by a web server – A protocol (of sorts) for how the request is passed to the external program – How the external program sends the response to the client CGI 17

18. CGI Programming CGI 18 CLIENT HTTP SERVER CGI Program http request http response setenv(), dup(), fork(), exec(),...

19. CGI URLs There is mapping between URLs and CGI programs provided by a web sever – The exact mapping is not standardized web server admin can set it up Typically: – requests that start with /CGI-BIN/, /cgi-bin/ or /cgi/, etc. not to static documents CGI 19

20. HTTP Server - CGI Interaction CGI 20 HTTP SERVER CGI Program stdin stdout Environment Variables

21. Environment Variables The web server sets some environment variables with information about the request The web server fork()s and the child process exec()s the CGI program The CGI program gets information about the request from environment variables CGI 21

22. STDIN, STDOUT Before calling exec(), the child process sets up pipes so that – stdin comes from the web server and – stdout goes to the web server In some cases part of the request is read from stdin Anything written to stdout is forwarded by the web server to the client CGI 22

23. Request Method: Get GET requests can include a query string as part of the URL: GET /cgi-bin/login?mgunes HTTP/1.0 CGI 23 Request Method Resource Name Delimiter Query String

24. Simple GET queries - ISINDEX You can put an tag inside an HTML document – The browser will create a text box that allows the user to enter a single string If an ACTION is specified in the ISINDEX tag, when the user presses Enter, – a request will be sent to the server specified as ACTION CGI 24

25. ISINDEX Example Enter a string: Press Enter to submit your query. If you enter the string “blahblah”, – the browser will send a request to the http server at foo.com that looks like this: GET /search.cgi?blahblah HTTP/1.1 CGI 25

26. What the CGI sees The CGI Program gets REQUEST_METHOD using getenv: char *method; method = getenv(“REQUEST_METHOD”); if (method==NULL) … /* error! */ CGI 26

27. Getting the GET If the request method is GET: if (strcasecmp(method,”get”)==0) The next step is to get the query string from the environment variable QUERY_STRING char *query; query = getenv(“QUERY_STRING”); CGI 27

28. Send back http Response and Headers CGI program can send back a http status line : printf(“HTTP/1.1 200 OK\r\n”); and headers: printf(“Content-type: text/html\r\n”); printf(“\r\n”); CGI 28

29. Important! CGI program doesn’t have to send a status line – HTTP server will do this for you if you don’t CGI program must always send back at least one header line indicating the data type of the content – usually text/html The web server will typically throw in a few header lines of it’s own – Date, Server, Connection CGI 29

30. Security!!! It is a very bad idea to build a command line containing user input! What if the user submits: “ ; rm -r *;” grep ; rm -r *; /usr/dict/words CGI 30

31. Beyond ISINDEX - Forms Many Web services require more than a simple ISINDEX HTML includes support for forms: – lots of field types – entire contents of form must be stuck together and put in QUERY_STRING by the Web server CGI 31

32. Form Fields Each field within form has a name and a value The browser creates a query that – includes a sequence of “name=value” substrings and – sticks them together separated by the ‘&’ character If user types in “Mehmet H.” as the name and “none” for occupation, – the query would look like this: “name=Mehmet+H%2E&occupation=none” CGI 32

33. HTML Forms Each form includes a METHOD that determines what http method is used to submit the request Each form includes an ACTION that determines where the request is made CGI 33

34. An HTML Form Name: Occupation: CGI 34

35. What a CGI will get query (from the environment variable QUERY_STRING) will be – a URL-encoded string containing the name,value pairs of all form fields The CGI must decode the query and separate the individual fields CGI 35

36. HTTP Method: POST GET method delivers data as part of URI POST method delivers data as the content of a request If REQUEST_METHOD is a POST, – the query is coming in STDIN The environment variable CONTENT_LENGTH tells us how much data to read CGI 36

37. Possible Problem char buff[100]; char *clen = getenv(“CONTENT_LENGTH”); if (clen==NULL) /* handle error */ int len = atoi(clen); if (read(0,buff,len)<0) … /* handle error */ pray_for(!hacker); CGI 37

38. GET vs. POST When using forms it’s generally better to use POST: – there are limits on the maximum size of a GET query string environment variable – a post query string doesn’t show up in the browser as part of the current URL CGI 38

39. CGI Sessions

40. Typical FORM CGI setup User fills out a form and presses submit CGI program gets a set of name,value pairs – one for each form field CGI decides what to do based on the name,value pairs – sometimes creates a new form based on the submission CGI Sessions 40

41. Sessions Many web sites allow you to establish a session – you identify yourself to the system – now you can visit lots of pages, add stuff to shopping cart, establish preferences, etc CGI Sessions 41

42. State Information Each HTTP request is unrelated to any other – as far as the Web server is concerned Each new request to a CGI program starts up a brand new copy of the CGI program Providing sessions requires keeping state information CGI Sessions 42

43. Session Conversation CGI Sessions 43 Client Client Hi! I'm Joe. Server Server Hi Joe (it's him again) Welcome Back... Hi Joe (it's him again) Welcome Back... I wanna buy a cookie. OK Joe, it will be there tomorrow. CGI1 CGI2

44. Hidden Field Usage One way to propagate state information is to use hidden fields User identifies themselves to a CGI program – fills out a form CGI sends back a form that contains hidden fields that identify the user or session CGI Sessions 44

45. Hidden does not mean secure! Anyone can look at the source of an HTML document – hidden fields are part of the document! If a form uses GET, all the name/value pairs are sent as part of the URI – URI shows up in the browser as the location of the current page CGI Sessions 45

46. Revised Conversation Initial form has field for user name GET /cgi1?name=joe HTTP/1.0 CGI1 creates order form with hidden field GET/cgi2?name=joe&order=cookie HTTP/1.0 CGI Sessions 46

47. Session Keys Many Web based systems use hidden fields that identify a session When the first request arrives, system generates a unique session key and stores it in a database Session key can be included in all forms/links generated by the system – as a hidden field or embedded in a link CGI Sessions 47

48. Session Key Properties Must be unique Should expire after a while Should be difficult to predict – typically use a pseudo-random number generator seeded carefully CGI Sessions 48

49. HTTP Cookies A "cookie' is a name,value pair that a CGI program can ask the client to remember Client sends this name,value pair along with every request to the CGI We can also use "cookies" to propagate state information CGI Sessions 49

50. Set-Cookie Header Options Cookies are set using HTTP headers The general form of the Set-Cookie header is: Set-Cookie: name=value; options The options include: – expires=... – domain=... – path=... CGI Sessions 50

51. Set-Cookie Fields Many options can be specified – separated by ";" Set-Cookie: a=blah; path=/; domain=.cse.unr.edu; expires=Thursday, 10-May-2010 12:00:00 2010 CGI Sessions 51 All must be on one line!

52. CGI cookie creation A CGI program can send back any number of HTTP headers – can set multiple cookies Content-Type is required! printf("Content-Type: text/html\r\n"); printf("Set-Cookie: prefs=nofrms\r\n"); printf("Set-Cookie: Java=yes\r\n"); printf("\r\n"); … now sends document content CGI Sessions 52

53. Getting HTTP Cookies Browser sends each cookie as a header: Cookie: prefs=nofrms Cookie: Java=OK Web server gives cookies to CGI program via an environment variable – or STDIN CGI Sessions 53

54. Multiple Cookies There can be more than one cookie Web Server puts them all together prefs=nofrms; Java=OK and puts this string in the environment variable: HTTP_COOKIE Each cookie can be up to 4k bytes One "site" can store up to 20 cookies on a user's machine CGI Sessions 54

55. Cookies and Privacy Cookies can't be used to: – send personal information to a web server without the user knowing about it – be used to send viruses to a browser – find out what other web sites a user has visited* – access a user's hard disk * although they can come pretty close to this! CGI Sessions 55

56. Some Issues Persistent cookies take up space on user's hard disk Can be used to track your behavior within a web site – This information can be sold or shared Cookies can be shared by cooperating sites – advertising agencies do this CGI Sessions 56