Zero Latency HTTP the comet technique

Who am I Tomcat Committer / ASF member Co-designed the Comet implementation Implemented NIO connector in 6 Responsible for session replication and clustering Been involved with ASF since 2001

What we will cover Brief History of HTTP How HTTP is used today Introduction to AJAX Beyond AJAX, The Comet Technique Comet Implementation – Under the hood Uni- vs. Bidirectional Comet Problems and their solutions Demo Q & A

The History of HTTP 1.0 (7th release) RFC 1945 in 1996 Co-authored by ASFs Roy T Fielding Current RFC 2616 Lead by W3C Activity on specs are closed Standard has been achieved

What is HTTP HyperText Transfer Protocol Text based protocol Request/Response semantics Stateless Most commonly run over TCP/IP networks Protocol used for much more than just hypermedia information sharing

What is HTTP Text based protocol – CRLN delimited GET /Protocols/History.html HTTP/1.1 Host: User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv: ) Gecko/ Firefox/ Accept: text/xml,application/xml,application/xhtml+xml,text/h tml;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO ,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive

What is HTTP Request/Response Semantics GET /Protocols/History.html HTTP/1.1 Host: Keep-Alive: 300 Connection: keep-alive HTTP/1.x 200 OK Date: Mon, 19 Mar :46:17 GMT Server: Apache/ (Unix) PHP/4.4.5 Keep-Alive: timeout=2, max=100 Content-Length: Connection: Keep-Alive Request Response

What is HTTP Stateless –TCP session can end after each request –State based on cookies TCP/IP transport –No limitations for transporting with other protocols Delivers both text and binary data

HTTP Limitations Request/Response –Change one field on a page requires a reload of the entire page –Client always has to initiate the request Stateless –Server required to keep state –State is timed out, if client is not cancelling it Finer grained communication is needed –AJAX –Rich Clients (FLEX, OpenLaszlo, …)

AJAX Asynchronous JavaScript +(and) XML Uses HTTP Asynchronous data processing Able to request data from server and update a page in the browser Common examples –maps.google.com

Benefits of AJAX Performance –More can be accomplished in less amount of transactions Less user interaction –Program can make intelligent decision about when a request needs to happen and what data it needs to fetch Sample Application for view of benefits –

Beyond AJAX What did AJAX not accomplish? –Still client poll based –Server push can be accomplished by a client poll followed by a delayed response –Traditional web/servlet containers are thread-per-request based –Server resource tied up for the duration of the request

Introducing Comet The answer to our problems –Q: What problems? –A: The fact that port HTTP/80 has been replacing every decent TCP protocol over the years. –Q: What do you mean? –A: Ill explain…

Picture borrowed from: dojotoolkit.org

3. Register with background thread How it Works Timeline Server Client 1. HTTP request 2. BEGIN Event 5. HTTP Response OK – Tx Enc: Chunked 4. Event Complete Webapp Async Thread 6. Server data push 7. Encode and transfer8. Another server push9. Client Push10. READ Event11. One more client push Webapp Worker Thread 12. And on it goes…

Under the hood Open HTTP request, leave it open ended Instant or delayed response(s), leave it open ended Tomcat processes just like servlet –CometProcessor extends HttpServlet(or should) –CometFilter extends Filter Connection remains open and is writable

Under the Hood Open ended request POST /load/echo HTTP/1.1 User-Agent: Filips Comet Client/1.0 Host: :8080 Transfer-Encoding: chunked 10 test data test 1 Open ended allows for new data without the overhead for a new request –Zero Latency HTTP

Under the Hood Open ended response HTTP/ OK Server: Apache-Coyote/1.1 Transfer-Encoding: chunked Content-Type: text/plain;charset=UTF-8 Transfer-Encoding: chunked Date: Mon, 19 Mar :25:36 GMT 12 test data test 1 Open ended allows for new data without the overhead for a new request –Zero Latency HTTP

Under the Hood CometProcessor –Extends HttpServlet to provide a fall back option should the connector not support Comet void event(CometEvent event)… –Types of Events: BEGIN READ END ERROR A request is a Comet request if the URL maps to a servlet that implements the CometProcessor interface

Under the Hood CometFilter –Extends javax.servlet.Filter to provide fall back option should the Connector not support Comet void doFilterEvent(CometEvent event, CometFilterChain chain) … Same mapping rules as CometProcessor

Under the Hood BEGIN –New connection created –Request headers fully parsed and mapped to a servlet –Partial body may or may not have been read –Processed by a worker thread

Under the Hood READ –New data available on the connection –Processed by worker thread –Read now to avoid repeated read events –Read using servlet input stream event.getHttpServletRequest.getInputStrea m()

Under the Hood END –The request has naturally come to an end –Server may shutdown, and needs to close connections –Only if Connector shutdown prior to app, piggy back on HttpServlet.destroy for most apps instead ERROR –When the connection times out or an error has occurred during invokation

Bidirectional Comet Tomcat supports bidirectional comet communication –Server can push data to the client –Client can push data to the server Timeout values can be set on a per- connection basis Unidirectional Comet means that only the server can push data to client

Scalability? No more thread per connection or thread per request One asynch thread can handle writes for thousands of Comet connections No overhead of request/response headers for each request No overhead for TCP setup/breakdown Memory overhead for open connections

Examples of use Rich Clients –Mail –Maps –Online conferences Clients with need for server push –Stock tickers –Auction sites –Chat

Future Improvements Non blocking reads Non blocking writes Bayeux protocol implementation End Request – AJAX over Comet Keepalive Connections Comet through HTTP proxy(?)

Demo Instructions for demo usage Demo consists of an interactive application Use this application to evaluate the session you just listened to …. …..