Jigsaw Performance Analysis Potential Bottlenecks

June 28, 2000 CHEN Ge CSIS HKU Core Components of Jigsaw httpd SocketClientFactory (The Thread Pool) serverpor :8001 Incoming Requests ResourceStoreManage (Resource Cache) Jigsaw

June 28, 2000 CHEN Ge CSIS HKU Core Components of Jigsaw The HTTP Daemon – The HTTP Daemon in Jigsaw is an instance of the httpd Class – The httpd Object contains a Thread Pool called SocketClientFactory, which actually handle the incoming requests – The httpd Object also contains a resource cache called ResourceStoreManager

June 28, 2000 CHEN Ge CSIS HKU Core Components of Jigsaw The SocketClientFactory – The SocketClientFactory contains a pool of threads (default is 45 threads) to be bound to incoming request – SocketClientFactory is also responsible for maintaining the server’s load information

June 28, 2000 CHEN Ge CSIS HKU Core Components of Jigsaw The ResourceStoreManager – Jigsaw use the concept resource refer to all the objects accessible through the Web Server – A resource maybe as simple as file in the file system. It also may represent some dynamic content generating program like CGI scripts, servlets, etc. – ResourceStoreManage stores referred resource in StoreEntry objects for caching purpose. When the server is shutdown, it will deploy the resources to persistent storage for reloading at the next startup time

June 28, 2000 CHEN Ge CSIS HKU Request Handling Stages SocketClient Thread Bind – The httpd runs in an infinite loop waiting for incoming socket connection – When a socket connect comes, httpd will pass the incoming SocketClient to SocketClientFactory to find out a thread in the thread pool to handle the request – The httpd thread will block until the SocketClientFactory either find a thread to handle the request or reject the request

June 28, 2000 CHEN Ge CSIS HKU Request Handling Stages public void run () { … while ( ( ! finishing) && ( socket != null ) ) { Socket ns = null ; try { ns = socket.accept() ; ns.setTcpNoDelay(true); } catch (IOException e) { … } if ( (socket != null) && (ns != null) && (factory != null) ) factory.handleConnection (ns) ; } // Our socket has been closed, perform associated cleanup. cleanup(restarting) ; } httpd factory manager

June 28, 2000 CHEN Ge CSIS HKU Request Handling Stages SoketClient Thread Bind –The SocketClientFactory will try to assign an thread from the pool to the incoming SocketClient. if server exceeds the predefined maximum load, the incoming request will be rejected(No response) –During the binding procedure, the SocketClientFactory will update many shared variables in the pool to refresh the server’s status through synchronized methods

June 28, 2000 CHEN Ge CSIS HKU Request Handling Stages Resource Lookup – When the SocketClientFactory successfully binds a thread to a incoming socket connection, the bound thread will proceed to handle the request – When it get the request content, for example, a URI for a html document, the thread will begin a complex procedure of ‘Resource Lookup’ – The previous requested URIs are stored in the ResourceStoreManager

June 28, 2000 CHEN Ge CSIS HKU Request Handling Stages Resource Emitting – After getting the requested resource information from the lookup stages, the thread gets all the necessary information about the resource requested – After it gets the necessary information, the thread emit the reply via socket

June 28, 2000 CHEN Ge CSIS HKU Major Methods Called org.w3c.jigsaw.httpd.run() --- the main thread public void run () { … while ( ( ! finishing) && ( socket != null ) ) { Socket ns = null ; try { ns = socket.accept() ; ns.setTcpNoDelay(true); } catch (IOException e) { … } if ( (socket != null) && (ns != null) && (factory != null) ) factory.handleConnection (ns) ; //This the Thread Binding process } …. }

June 28, 2000 CHEN Ge CSIS HKU Major Methods Called org.w3c.jigsaw.http.socket.SocketClientFacotory.handl eConnection() --- Thread Binding public void handleConnection (Socket socket) { …//Try to adjust client pool status according to the server’s load switch(loadavg) { case AVG_LIGHT: // Free list is non empty, be fast: if ( decrFreeCount() ) cs = (SocketClientState) freeList.removeTail(); break; case AVG_NORMAL: case AVG_HIGH: // Free list is non empty, but we try killing a client: killSomeClients(); if ( decrFreeCount() ) cs = (SocketClientState) freeList.removeTail(); break; case AVG_DEAD: break; }

June 28, 2000 CHEN Ge CSIS HKU org.w3c.jigsaw.http. socket.SocketClientFacotory.handleConnection() --- Thread Binding … if ( cs != null ) { … cs.status = SocketClientState.C_BUSY; cs.client.bind(socket); } else { … try { socket.close(); } catch (IOException ex) { } server.errlog(socket.getInetAddress()+" refused (overloaded)."); } return; }

June 28, 2000 CHEN Ge CSIS HKU Major Methods Called Org.w3c.jigsaw.http.Client.startconnection() --- the abstract class define the basic request handling interface protected boolean startConnection(InputStream in, DataOutputStream out) throws ClientException { // Process request, and time it: tstart = System.currentTimeMillis() ; reply = processRequest (request) ; tend = System.currentTimeMillis() ; //By Robin - 19/06/2000 lookupTime += tend-tstart; numOfLookupRequest ++; //End of By Robin … sent = emitReply(reply) ; … }

June 28, 2000 CHEN Ge CSIS HKU Major Methods Called org.w3c.jigsaw.httpd.perform() --- looking up the resource – This method is where Jigsaw try to find out the requested resource through a serials of lookup process – The previous introduced method Client.startconnection() calls this method through Client.processRequest() method

June 28, 2000 CHEN Ge CSIS HKU org.w3c.jigsaw.httpd.perform() --- looking up the resource public ReplyInterface perform(RequestInterface req) throws ProtocolException, ResourceException{ … // Create a lookup state, and a lookup result: LookupState ls = new LookupState(request); LookupResult lr = new lookupResult(root.getResourceReference()); // Run the lookup algorithm of root resource: try { if ( root.lookup(ls, lr) ) { if (lr.hasReply()) return lr.getReply(); } } catch (ProtocolException ex) { … } …

June 28, 2000 CHEN Ge CSIS HKU org.w3c.jigsaw.httpd.perform() --- looking up the resource // Let the target resource perform the method ResourceReference target = lr.getTarget(); Reply reply = null; … //perform the request: if ((error == null) && (target != null)) { request.setFilters(filters, infilter); request.setTargetResource(target); try { FramedResource res = (FramedResource)target.lock(); reply = (Reply)res.perform(request); … } return reply; }

June 28, 2000 CHEN Ge CSIS HKU How do we test the Performance Breakdown Insert some System.currentTimeMillis() call in the methods introdueced above Divide the whole request process time into three stages: thread binding time, resource lookup time, reply emitting time

June 28, 2000 CHEN Ge CSIS HKU Thread Binding Time In httpd.run() //By Robin - 19/06/2000 { long tstart = 0; long tend = 0; tstart = System.currentTimeMillis(); factory.handleConnection (ns) ; tend = System.currentTimeMillis(); requestNum ++; totalHandleTime = totalHandleTime + (tend-tstart); } /* Original Code Before By Robin - 19/06/2000 factory.handleConnection (ns) ; */ //end of By Robin - 19/06/2000

June 28, 2000 CHEN Ge CSIS HKU Resource Lookup Time In order to avoid the synchronization overhead during the timing process, all the threads first calculate their own values, then, the server will collect all the values of each thread every 6 (predefined, changeable) minutes

June 28, 2000 CHEN Ge CSIS HKU Resource Lookup Time For individual thread, in Client.startConnect() // Process request, and time it: tstart = System.currentTimeMillis() ; reply = processRequest (request) ; tend = System.currentTimeMillis() ; //By Robin - 19/06/2000 lookupTime += tend-tstart; numOfLookupRequest ++; //End of By Robin

June 28, 2000 CHEN Ge CSIS HKU Resource Lookup Time For the whole value, in SocketClientFactory.doBreakdownLog(), it get each value of the individual thread, and calculate out the average Lookup Time by divide the sum of all the threads lookup time by the number of total lookup processed

June 28, 2000 CHEN Ge CSIS HKU SocketClientFactory.doBreakdownLog() SocketClientState cs = csList; while (cs != null) { … totalHandleTime += cs.totalHandleTime; totalLookupTime += cs.totalLookupTime; totalEmitTime += cs.totalEmitTime; totalLookupRequest += cs.numOfLookupRequest; totalRequest += cs.numOfRequest; totalEmittedRequest += cs.numOfEmittedRequest; cs = cs.csnext; }

June 28, 2000 CHEN Ge CSIS HKU SocketClientFactory.doBreakdownLog() if (totalRequest!= 0) { aveHandleTime = totalHandleTime/totalRequest; } if (totalLookupRequest != 0) { aveLookupTime = totalLookupTime/totalLookupRequest; aveHandleTimeByLookupNum = totalHandleTime/totalLookupRequest; } if (totalEmittedRequest != 0) { aveEmitTime = totalEmitTime/totalEmittedRequest; }

June 28, 2000 CHEN Ge CSIS HKU Reply Emitting Time The Reply Emitting Time is tested similarly with that of Lookup Time, in Client.startConnection() … tend = System.currentTimeMillis() ; … // Inital keep alive check, emit the reply: … //By Robin - 20/06/2000 tendEmit = System.currentTimeMillis(); emitTime += tendEmit-tend; numOfEmittedRequest ++; //End of By Robin

June 28, 2000 CHEN Ge CSIS HKU Performance Breakdown Average Bind Time and Lookup Time

June 28, 2000 CHEN Ge CSIS HKU Performance Breakdown Total Connection Replied / Total Connection

June 28, 2000 CHEN Ge CSIS HKU Performance Breakdown Individual Process Stage Time/ Total Handle Time

June 28, 2000 CHEN Ge CSIS HKU Performance Breakdown Individual Process Stage Time/ Total Handle Time

June 28, 2000 CHEN Ge CSIS HKU Performance Breakdown – The Bind Time and Lookup Time increases significantly as the load increases – Except for the strange behavior of Jigsaw when the workload becomes very heavy, at which point the emit time suddenly increases dramatically, the Lookup Time counts for more than half of the total process time – The bind time is very irregular

June 28, 2000 CHEN Ge CSIS HKU Problems – Jigsaw sometimes hangs when the workload become heavy, even not that heavy as the most one in our test – Traceinfo shows that the loopup stage have some problem, it generate a lot excepts and errors when the workload increase, and Jigsaw will unable to find some really exists resources – The emit time become unexpectedly high when the work load very heavy, is it the problem of underlying socket problem? Still do no clear how Java implement the Socket interface? – Our previous speculation that the thread pool may be a potential bottleneck seems partially correct, if the irregular behavior of Binding Time can be explained by this – The loopup stage (including the cache) seems another bottleneck in Jigsaw from the tested data