1. JAWS (A Framework for High-Performance Web Servers) TP Version 1.0 YoungSu, Son only2u4u@devpia.com Microsoft MVP Devpia Architecture&Design Sysop Samsung Electronics Home Solution Group YoungSu, Son only2u4u@devpia.com Microsoft MVP Devpia Architecture&Design Sysop Samsung Electronics Home Solution Group

2. 6/9/2016 Devpia A&D EVA 2 Motivation Throughput ReliableTransaction Programmer Extensibility Short Response Time What’s the High Performance Web Server?

3. 6/9/2016 Devpia A&D EVA 3 Motivation Concurrency I/O Protocol&Filter Cache

4. 6/9/2016 Devpia A&D EVA 4 Motivation Thread-Per-Request Thread-Per-Session Thread Pool Concurrency Caching Structured Caching Hinted Caching LFU LRU Asynchronous I/O Reactive I/O Synchronous I/O I/O Parse Headers Parse Request Read Request Protocol & Filter Perform Request Log Request

5. 6/9/2016 Devpia A&D EVA 5 Seed Paper James C. Hu, Douglas Schmidt, JAWS: A Framework for High-Performance Web Servers, Domain-Specific Application Frameworks: Frameworks Experience By Industry, John Wiley & Sons, October, 1999.

6. 6/9/2016 Devpia A&D EVA 6 Agenda Motivation Background Info Basic Patterns Applying Patterns & Frameworks to Web Server Common Pitfalls of Developing Web Server Overcoming Web Server Pitfalls Relationship Between Frameworks, Patterns & Other Reuse Technique The JAWS Web Server Overview of the JAWS Framework Overview of the Design Pattern in JAWS Concurrency Strategy I/O Strategy Protocol Pipeline Strategy File Caching Strategy Web Server Benchmarking Concluding Remarks

7. 6/9/2016 Devpia A&D EVA 7 1. Background Information Core Components of SW Impl. Basic Patterns Strategy Pattern Template Method Pattern Component Configuration Pattern Factory Pattern Singleton Pattern

8. 6/9/2016 Devpia A&D EVA 8 1.1 Core Components of S/W Impl. Web Server Supports the Configuration of Various Web Server Great Architect CommonPart ConfigurablePart VariablePart

9. 6/9/2016 Devpia A&D EVA 9 ConfigurablePart VariablePart Core Components of S/W Impl. ComponentConfigurator Factory Strategy <<LINKS>> <<CREATES>> CommonPart(Modularity) Log Security Transaction AOP

10. 6/9/2016 Devpia A&D EVA 10 ConfigurablePart VariablePart ComponentConfigurator Factory Strategy <<LINKS>> <<CREATES>> CommonPart(Modularity) Log Security Transaction AOP

11. 6/9/2016 Devpia A&D EVA 11 1.2 Basic Patterns GoF 2 Principles Strategy Pattern Template Method Pattern Component Configurator Pattern Factory Pattern Observer Pattern Chain of Responsibility

12. 6/9/2016 Devpia A&D EVA 12 GoF 2 Principles Program to an interface, not to an implementation Favor Object Composition over Class Inheritance

13. 6/9/2016 Devpia A&D EVA 13 Strategy Pattern Intents satelliteSuperComputer Do you think that the same logic use between satellite and super computer?

14. 6/9/2016 Devpia A&D EVA 14 Strategy Pattern in GoF void main() { //pSort 라는 인터페이스를 선언한다 CSort *pSort; if (GetSystemInfo() == Low ) { // 인공위성에서는 Bubble Sorting 사용 pSort = new CBubble(); } else { // 슈퍼 컴퓨터에는 Quick Sorting 사용 pSort = new CQuick(); } pSort->Sorting(); delete pSort; } CSort +Sorting() CBubble +Sorting() CQuick +Sorting()

15. 6/9/2016 Devpia A&D EVA 15 Template Method Pattern Intents ODBC(JDBC) API Application ODBC (JDBC) Driver Manager MSSQL Driver Oracle Driver

16. 6/9/2016 Devpia A&D EVA 16 Template Method Pattern in GoF CQueryTemplate +doQuery() #FormatConnect() #FormatSelect() COracleQT #FormatConnect() #FormatSelect() CSqlSvrQT #FormatConnect() #FormatSelect() class CQueryTemplate {public void doQuery() { string dbCommand; dbCommand = FormatConnect(); dbCommand = FormantSelect(sql); }... } void main() {//pQT 라는 인터페이스를 선언한다 CQueryTemplate *pQT; Sql sql = “select * from AA”; if (GetDBProductInfo() = Oracle) pQT = new COracleQT(); else pQT = new CSqlSvrQT(); pQt->doQuery(sql); delete pQt; } Inversion of Control (Hollywood Principle)

17. 6/9/2016 Devpia A&D EVA 17 Component Configurator Pattern Intents ConfigurationFile HTTPProtocol Text Based LogComponent DESRSA XML Based LogComponent FTP Protocol When Change Config File Info, Component Configuration (Attribute) dynamically Changes! Great Architect

18. 6/9/2016 Devpia A&D EVA 18 Component Configurator in POSA2 Component +Init() +Fini() +Suspend() +Resume() +Info() Core Component Component Repository Component Configurator

19. 6/9/2016 Devpia A&D EVA 19 Component Configurator in POSA2

20. 6/9/2016 Devpia A&D EVA 20 Component Configurator Sample HTTP XML DES Static void main() { CMessage *pMessage = new CMessage(); if (GetComponentInfo(“Sort”) == HTTP) pMessage->Protocol = HTTP; else pMessage->Protocol = FTP; if (GetComponentInfo(“Log”) == XML) pMessage->Protocol = XML; else pMessage->Protocol = TEXT;... pMessage->Send(“Hello”); delete pMessage; } Advanced Topic Reflection

21. 6/9/2016 Devpia A&D EVA 21 ConfigurablePart VariablePart Motivation ComponentConfigurator Factory Strategy <<LINKS>> <<CREATES>> CommonPart(Modularity) Log Security Transaction

22. 6/9/2016 Devpia A&D EVA 22 Factory Pattern Intents Main Function Based Coding Object Oriented Programming

23. 6/9/2016 Devpia A&D EVA 23 Factory Pattern (Abstract Factory) Intents OO Design Advanced

24. 6/9/2016 Devpia A&D EVA 24 Factory Pattern in GOF Abstract Factory +CrateProduct() Client Concrete Factory Abstract Product Concrete Product static void main() { CMessage *pMessage = new CMessage(); if (GetComponentInfo(“Sort”) == HTTP) pMessage->Protocol = HTTP; else pMessage->Protocol = FTP; if (GetComponentInfo(“Log”) == XML) pMessage->Protocol = XML; else pMessage->Protocol = TEXT;... pMessage->Send(“Hello”); delete pMessage; }

25. 6/9/2016 Devpia A&D EVA 25 Factory Pattern in GOF Abstract Factory +CrateProduct() Client Concrete Factory Abstract Product Concrete Product class CFactory { CMessage* GetInstance() { CMessage *pMessage = new CMessage(); if (GetComponentInfo(“Sort”) = HTTP) pMessage->Protocol = HTTP; else pMessage->Protocol = FTP; if (GetComponentInfo(“Log”) = XML) pMessage->Protocol = XML; else pMessage->Protocol = TEXT;... return pMessage; } Reminds Component Configurator!! Reflection

26. 6/9/2016 Devpia A&D EVA 26 3 Foe Patterns ConfigurablePart VariablePart ComponentConfigurator Factory Strategy <<LINKS>> <<CREATES>> CommonPart(Modularity) Log Security Transaction

27. 6/9/2016 Devpia A&D EVA 27 Observer Pattern (Misconception) Change 1. Observation 2. Notification Data Misconception!! Or Prejudices ( 편견 !)

28. 6/9/2016 Devpia A&D EVA 28 Observer is Notifieer!! Observer in Observer Pattern is Notifieer

29. 6/9/2016 Devpia A&D EVA 29 Applicants Manager Don’t Call us!! Hollywood Principle

30. 6/9/2016 Devpia A&D EVA 30 Applicants Manager Don’t Call us! We Call You!!! Hollywood Principle

31. 6/9/2016 Devpia A&D EVA 31 Flow of Observer Pattern Data Client (Type1) Client (Type2) Client (Type3) Manager Client (Source) 1. Change 2. Change 3. Notification Observer is Notifieer Change

32. 6/9/2016 Devpia A&D EVA 32 Observer Pattern in GoF Subject +Attach (Observer o) +Detach (Observer o) +Notify() Concrete Subject Observer +Update () Concrete Observer

33. 6/9/2016 Devpia A&D EVA 33 Chain of Responsibility Intents

34. 6/9/2016 Devpia A&D EVA 34 Chain of Responsibility The Last Solution 바퀴벌레 팩 http://ajujoa.com/2460506

35. 6/9/2016 Devpia A&D EVA 35 Chain of Responsibility in GoF ClientHandler +HandleRequest() Concrete Handler +successor

36. 6/9/2016 Devpia A&D EVA 36 Design Strategy & Known Use Design Strategy The First Handler Consider Probability In order of Frequency of Use, Consist of Handler Sequences Role of The Last Handler My Parents Solve every problems! Known Use Exception Handling Mechanism Liskov Substitution Principle (LSP)

37. 6/9/2016 Devpia A&D EVA 37 2. Applying Patterns & Frameworks to Web Server Common Pitfalls of Developing Web Server Software Overcoming Web Server Pitfalls with Patterns and Frameworks Relationship between Frameworks, Patterns, and Other Reuse Techniques

38. 6/9/2016 Devpia A&D EVA 38 2.1 Common Pitfalls of Developing Web Server SW Excessive low-level detail Continuous rediscovery & reinvention of incompatible higher-level programming abstraction. incompatible higher-level programming abstraction. High potential for errors Lack of portability Steep learning curve Inability to handle increasing complexity

39. 6/9/2016 Devpia A&D EVA 39 2.2 Overcoming Web Server Pitfalls with Patterns & Frameworks Limitations of Class Library do not capture the control flow and collaboration the control flow and collaboration among families of related SW components. among families of related SW components. Class Library reuse often re-invent and re-implement re-invent and re-implement the overall Software architecture the overall Software architecture for each new Application. for each new Application.

40. 6/9/2016 Devpia A&D EVA 40 2.2 Overcoming Web Server Pitfalls with Patterns & Frameworks The Benefits of Patterns Expert’s Design Experiences Good Methods to use Framework The infrastructure of Framework The Benefits of Frameworks Expert’s intellectual products Help developers avoid costly re-invention

41. 6/9/2016 Devpia A&D EVA 41 2.3 Relationship Between Frameworks, Patterns, and Other Reuse Techniques Class Libraries VS Framework App Specific Logic OO Design EventLoop DATABASEADTs MATHNETWORKING GRAPHICSGUI SingletonStrategy ReactorAdapter State Active Object Class Library Component Architecture Invocations Selections Class Library Design Pattern DATABASE Singleton GRAPHICS Adapter EventLoop Reactor NETWORKING Active Object GUI State App Specific Logic MATH ADTs Callbacks Invocations Application Framework Component Architecture

42. 6/9/2016 Devpia A&D EVA 42 2.3 Relationship Between Frameworks, Patterns, and Other Reuse Techniques Frameworks define “semi-complete” application that embody domain-specific object structures that embody domain-specific object structures and functionality. Class Libraries don’t offer explicit guidance to system design Components in a Framework manage the canonical flow of control within App Frameworks are active and exhibit “IoC (Inversion of Control)” at runtime. Template Method Pattern (Hollywood Principle) In Practice, Frameworks, Class Libraries, and Components are complementary technologies.

43. 6/9/2016 Devpia A&D EVA 43 3. The JAWS Adaptive Web Server Overview of the JAWS Framework, Overview Design Pattern in JAWS. 4 Core Components in JAWS Concurrency.I/O.Protocol/Pipeline. File Caching. JAWS Framework Revisited

44. 6/9/2016 Devpia A&D EVA 44 3.1 Overview of the JAWS Framework

45. 6/9/2016 Devpia A&D EVA 45 3.2 Overview Design Pattern in JAWS Strategic Pattern AcceptorReactorProactor Active Object Service Configurator Tactical Pattern StrategyAdapterStateSingleton

46. 6/9/2016 Devpia A&D EVA 46 3.2.1 Strategic Pattern AcceptorReactorProactor Active Object Service Configurator

47. 6/9/2016 Devpia A&D EVA 47 Acceptor/Connector Pattern

48. 6/9/2016 Devpia A&D EVA 48 Acceptor Pattern Event Dispatcher Concurrency Task Acceptor peer_acceptor_accept() Protocol Handler peer_stream_open() Protocol Handler Acceptor is Factory. Creates, Accepts and Activates a new Protocol Handler Whenever the Event Dispatcher notifies Acceptor Whenever the Event Dispatcher notifies Acceptor that a connection has arrived from a Client.

49. 6/9/2016 Devpia A&D EVA 49 Reactor Pattern Reactor handle_events() register_handler(in handle) remove_handler(in handle) Event Handler handle_event(in type) get_handle() Synchronous Event Demultiplexer select() Concrete Event Handler handle_event(in type) get_handle() Concrete Event Handler handle_event(in type) get_handle() Handle <<uses>> handleset notifies owns dispatch

50. 6/9/2016 Devpia A&D EVA 50 Reactor Pattern InputOutput Handler Synchronous IO Reactor send_file() recv_file() send_data() recv_data() Protocol Pipeline InputOutput Asynchronous IO Reactive IO Initiation Dispatcher handle_event() register_handler(h) remove_handler(h) Timer_Queue schdule_timer(h) cancel_timer(h) expire_timers(h) Filecache Handle Reactive IO Handler handle_input() handle_output() IO Handle decouples the synch demultiplexing and dispatching logic. the Single-Thread Web Server Concurrency model

51. 6/9/2016 Devpia A&D EVA 51 Proactor Pattern Proactor handle_events() Asynchronous Event Demultiplexer get_completion_event() Concrete Completion Handler Handle <<uses>> * <<enqueues>> is associated with Completion Event Queue Asynchronous Operation Processor execute_async_operation() Asynchronous Operation async_operation() Initiator Completion Handler handle_event() <<executes>> <<dequeues>> <<uses>><<invokes>> <<uses>> <<demultiplexes & dispatches>>

52. 6/9/2016 Devpia A&D EVA 52 Proactor Pattern (IOCP) Socket A Socket B Socket C Completion Port 2.2 Create Completion Queue Thread Thread Thread 1. Create Socket 2.1 Create Completion Port 4. Link Socket and Port 5. 1 Send Completion Socket Info 5. 2 Send Completion Packet Thread Pool 3. Create Thread 6. Process Completion Events 7. Thread Returns Thread Pool

53. 6/9/2016 Devpia A&D EVA 53 Proactor Pattern InputOutput Handler Synchronous IO Proactor send_file() recv_file() send_data() recv_data() Protocol Pipeline InputOutput Reactive IO Asynchronous IO Completion Dispatcher handle_event() register_handler(h) remove_handler(h) Timer_Queue schdule_timer(h) cancel_timer(h) expire_timers(h) Filecache Handle Proactive IO Handler handle_read_file() handle_write_file() IO Handle decouples the Asynch demultiplexing and dispatching logic. The Asynch variant of the Thread Pool Use IOCP (IO Completion Port) Asynch Op open() cancel()

54. 6/9/2016 Devpia A&D EVA 54 Active Object Pattern Future Scheduler Dispatch() Insert() MethodRequest Can_run() Call() * Proxy method_1() method_n() Activation Queue Insert() Remove() Servant method_1() method_n() createscreatesmaintains Concrete MethodRequest 1 Concrete MethodRequest N Client write to obtain result from invoke 11 11

55. 6/9/2016 Devpia A&D EVA 55 Active Object Pattern decouples method invocation from method execution. Thread-per-Request, Thread Pool, Thread-per-Session Model use Active Object pattern. Scheduler http_request() ftp_request() snmp_request() dispatch() Activation Queue insert() remove() Method Object Protocol Handler http_request() ftp_request() snmp_request() Resource Representation Protocol Pipeline delegates for(;;){ m = aq->remove(); dispatch(m); } aq->insert(http)

56. 6/9/2016 Devpia A&D EVA 56 Service Configurator Pattern Component +Init() +Fini() +Suspend() +Resume() +Info() Core Component Component Repository Service Configurator

57. 6/9/2016 Devpia A&D EVA 57 Service Configurator Pattern Provides System Configuration at Runtime. Dynamically optimize, control and reconfigure the behavior of Web server strategies a installation-time or during run-time. Service +init() +fini() +suspend() +resume() +info() Filecache Protocol Handler Service Repository Filter Repository Cache Strategy Repository Protocol Pipeline DLL Cache Strategy Read RequestFilterLRU StrategyLFU Strategy Parse RequestLog Request

58. 6/9/2016 Devpia A&D EVA 58 3.2.2 Tactical Patterns Strategy Configure Cache replacement strategies without afffecting the core SW Architecture. Adapter Uniformly encapsulate the operation of synch, asynch and reactive I/O. State Event Dispatcher use State to seamlessly support different concurrency strategies and both synch and asynch. Singleton To provide a global point to access object. Exist one copy of Cached Virtual File System.

59. 6/9/2016 Devpia A&D EVA 59 Review

60. 6/9/2016 Devpia A&D EVA 60 3.3 4 Components in JAWS Thread-Per-Request Thread-Per-Session Thread Pool Concurrency Caching Structured Caching Hinted Caching LFU LRU Asynchronous I/O Reactive I/O Synchronous I/O I/O Parse Headers Parse Request Read Request Protocol & Filter Perform Request Log Request

61. 6/9/2016 Devpia A&D EVA 61 3.3 Concurrency Strategy Design Challenge A large portion of non-I/O related Web server overhead is due to the Web server’s concurrency strategy. Key Overhead Synchronization Thread/Process Creation Context Switching Choosing an efficient concurrency strategy is crucial to achieve high-performance

62. 6/9/2016 Devpia A&D EVA 62 3.3 Concurrency Strategy Alternative Solution Strategies Thread-per-RequestThread-per-Session Thread Pool Single-ThreadExperiments JAWS Concurrency Strategy Framework

63. 6/9/2016 Devpia A&D EVA 63 3.3.1 Alternative Solution Strategies Concurreny Strategy of Web Servers Single Thread RoxenProcess-BasedApachZeusMultiThreadPHTTDJAWS Static Factors Dynamic Factors H/W Configuration OS Platform Web Server Use Case Machine Load # of simultaneous requests Memory Use Memory Workload

64. 6/9/2016 Devpia A&D EVA 64 Thread-per-Request Allows each client request to run concurrently in a separate thread. Each Request arrives, a new thread is crated to process request. This design appropriate synch I/O mechanisms Event Dispatcher Concurrency Task Protocol Handler Acceptor Protocol Handler

65. 6/9/2016 Devpia A&D EVA 65 Thread-per-Request (cont’d) AdvantageSimplicity Exploit parallelism on multi-processor platform Limitation The # of running threads may grow without bound, exhausting available memory and CPU resources Compatible Fields Use Light loaded servers with low latency Don’t use time consuming task based system

66. 6/9/2016 Devpia A&D EVA 66 Thread-per-Session Allows each client connection to run concurrently All of these requests are submitted through a connection between each client and a separate thread in the web server process Advantage Does not spawn a separate thread for each request. Limitation This model still vulnerable to unbounded resource consumption as the # of clients grows.

67. 6/9/2016 Devpia A&D EVA 67 Thread-per-Session (cont’d) Suited for HTTP 1.1 but not HTTP 1.0 HTTP 1.0 support open/operation/close One URL Fetch per Connection Needs Reconnection Transporting Data Size Limitation URL Size Limitation HTTP 1.1 Persistent Connection Pipeline Functionality Many Requests -> Support Serial Response Compressed Data Transporting Use Proxy Server and Cache

68. 6/9/2016 Devpia A&D EVA 68 Thread Pool Allows up to N requests to execute concurrently within a pool of threads. Support prespawning of Thread Creation Event Dispatcher Concurrency Task Protocol Handler Acceptor Protocol Handler

69. 6/9/2016 Devpia A&D EVA 69 Thread Pool (cont’d) Advantage Lower overhead that Thread-per-Request. Thread Prespawning thread creation cost is amortized Pool Size is Fixed Resource Consume is bounded Limitation Small Pool Size New incoming requests be dropped or wait indefinitely. Large Pool Size Resource Consumption may be no better than using Thread-Per-Request

70. 6/9/2016 Devpia A&D EVA 70 Single-Threaded All connections and requests are handled by the same thread of control. Single-Threaded servers process requests iteratively. Compatible Filed Asynch or Reactive I/O Multiple Request Inadequate for high volume server Subsequent requests are blocked until they are reached

71. 6/9/2016 Devpia A&D EVA 71 Experiments Static adaptivity Select the concurrency strategy that best meets The static requirements of the system. multi-threaded concurrency More suited to A multiple-processor machinethan a uni-processor machine Dynamic adaptivity Adapt dynamically to current server condition In order to accommodate Unanticipated load Support to increase the # of available threads in the Pool

72. 6/9/2016 Devpia A&D EVA 72 JAWS Concurrency Strategy Framework Event Dispatcher dispatch() Concurrncy dispatch() Task accept() activiate() enque() deque() svc() Thread Pool dispatch() Thread-per-Connection dispatch() Protocol Handler open() Acceptor accept() task->accept() task->activate() task->enque() acceptor->accept() server->dispatch() server task acceptor handler

73. 6/9/2016 Devpia A&D EVA 73 3.4 I/O Strategy Design Challenge is to devise efficient data retrieval and delivery strategies, collectively referred to as I/O. Key Overhead Arranging Multiple I/O operation. ExampleSynch Web Transaction involving monetary fund transfer ASynch Web access to static information (Web page – ASP..)

74. 6/9/2016 Devpia A&D EVA 74 3.4 I/O Strategy Alternative Solution Strategies The Synchronous I/O Strategy The Reactive I/O Strategy The Asynchronous I/O Strategy Experiments JAWS I/O Strategy Framework

75. 6/9/2016 Devpia A&D EVA 75 The Synch I/O Strategy Synch I/O describes the model of I/O interaction between a Web server process and kernel. Kernel doesn’t return the thread of control to the server until the requested I/O operation either complete, completes partially, or fails. Advantage (Easy Use) Disadvantage (Resource Exhaust) If combined with a single threaded concurrency strategy, It is not possible to perform multiple synch I/O operation simultaneously. It is not possible to perform multiple synch I/O operation simultaneously. When using multiple thread, It is still possible for an I/O request to block indefinitely. It is still possible for an I/O request to block indefinitely.

76. 6/9/2016 Devpia A&D EVA 76 The Reactive I/O Strategy Reactive I/O alleviates the blocking problems of synchronous I/O without resorting to polling. of synchronous I/O without resorting to polling. Web Server uses an OS event demultiplexing system call. select() in UNIX WaitForMultipleObjects() in Win32 Refers Reactor Pattern Disadvantage May not make effective use of multiple CPU.

77. 6/9/2016 Devpia A&D EVA 77 The Asynch I/O Strategy Asynch I/O simplifies the de-multiplexing of multiple events the de-multiplexing of multiple events in one or more thread of control in one or more thread of control without blocking the Web server. without blocking the Web server.Advantage The Web Server not block on I/O request. Disadvantage Asynch I/O is not available on many OS Platform. more complicated than writing synch program.

78. 6/9/2016 Devpia A&D EVA 78 Experiments The Result reveal that each I/O strategy behaves differently each I/O strategy behaves differently under different load condition. under different load condition. JAWS I/O Strategy Framework addresses this issue addresses this issue by allowing the I/O Strategy by allowing the I/O Strategy to adapt dynamically to adapt dynamically to run-time server condition. to run-time server condition.

79. 6/9/2016 Devpia A&D EVA 79 JAWS I/O Strategy Framework Perform Request svc() fh InputOutput Handler receive_file() receive_data() send_file() send_data() complete() error() InputOutput receive_file() receive_data() send_file() send_data() Synch IO receive_file() receive_data() send_file() send_data() Asynch IO receive_file() receive_data() send_file() send_data() Proactor Reactive IO receive_file() receive_data() send_file() send_data() Reactor Filecache Handle ioh io io io->send_file(“index.html”) write(fh) Proactor-> send_file(fh) Reactor-> send_file(fh)

80. 6/9/2016 Devpia A&D EVA 80 3.5 Protocol Pipeline Strategies Design Challenge Early Web Server Simply retrieved the requested file. Transferred its contents to the request. Modern Web Server Support Various File Format text, image, audio, video. Introducing Server Side Engine (ASP, JSP..) search engine, map generation, interfacing database, security, financial transaction.

81. 6/9/2016 Devpia A&D EVA 81 3.5 Protocol Pipeline Strategy Alternative Solution Strategies Pipeline. Reflection + Component Configurator. Examples Advanced Search Engine. Image Servers. Adaptive Web Contents. JAWS Protocol Pipeline Strategy Framework

82. 6/9/2016 Devpia A&D EVA 82 Using Pipeline PerformRequestLogRequest HTTP/ 1.0 200 OK Date: Thu, 09 Oct 01:26:00 GMT Server: Eva/1.0 Last-Modified: Wed, 08 Oct 1997.. Content-Length : 12345 Content-Type : text/html Homepage.. any.net - - [09/Oct/1997:01:26:00 -0500] “GET /~jxh/home.html HTTP/1.0 ReadRequest GET /~jxh/hoem.html HTTP/1.0 Connection: Keep-Alive User-Agent: Mothra/0.1 Host: any.net Accept: image/gif, image/jpeg, */* ParseRequest GET /users/jxh/public_thml/home.html HTTP /1.0 ParseHeaders CONNECTION USER_AGENT HOST ACCEPT Keep-Alive Mothra/0.1 any.net image/gif image/jpeg */*

83. 6/9/2016 Devpia A&D EVA 83 Static vs Dynamic Static Configuration HTTP 1.0 Request limited # of processing operation Marsharing & Demarsharing Demutlipelxing Dynamic Configuration Unlimited # of Processing operation Requires a lot of process arranging cost.

84. 6/9/2016 Devpia A&D EVA 84 Example Advanced Search Engine May construct data filters dynamically. “(perfomrnace AND (NOT symphonic))” Requesting for Web pages containing the word “performance” but not the word “symphonic”. Could be implemented as a pipeline of a positive match component coupled with a negative match component

85. 6/9/2016 Devpia A&D EVA 85 Example (cont’d) Image Servers User may request that Image be cropped, scaled, rotated, and dithered. Adaptive Web Contents Dynamic Delivery for End User. PDA User Web Content Overviews and Samller Image. Web Content Overviews and Samller Image. Workstation & Personal Computer Full Multimedia Enriched Pages. Full Multimedia Enriched Pages.

86. 6/9/2016 Devpia A&D EVA 86 JAWS Protocol Pipeline Strategy Protocol Handler InputOutput Handler Protocol Pipeline svc() Read Request svc() Filter svc() Filter svc() parse() Parse Headers svc() parse() Perform Request svc() perform() Log Request svc() log() component->svc() io->receive_data() Filter::svc() parse() Filter::svc() perform() Filter::svc() log() pipe io component

87. 6/9/2016 Devpia A&D EVA 87 3.6 File Caching Strategies Desigh Challenges Accessing the File System is a significant source of web server overhead. a significant source of web server overhead. Cache is a storate medium that provides more efficent retrieval. Alternative Solution Strategies Least Recently Used (LRU) Caching. Least Frequently used (LFU) Caching. Hinted Caching. Structured Caching. JAWS Cached Virtual File System Framework

88. 6/9/2016 Devpia A&D EVA 88 Alternative Solution Strategies Least Recently Used Caching Least Frequently Used Caching Structured Caching Refers the Next Page. Hinted Caching Analysis of Web page retrieval pattern. Web pages have spatial locality. Using statistical information about links (hinted). Using Prefetching Method. Refers “Hinted Caching in the Web”

89. 6/9/2016 Devpia A&D EVA 89 Alternative Solution Strategies Structured Caching

90. 6/9/2016 Devpia A&D EVA 90 JAWS Cached Virtual FS Framework InputOutput receive_file() send_file() filename InputOutput handle() mapaddr() name() size() open() Filecache Object handle() mapaddr() name() size() HashTable Filecache find() fetch() remove() create() findish() rw_lock[] Cache Strategy insert() LRU Strategy insert() LFU Strategy insert() fh fo replace least recently used Replace least Frequently used fh->open(file) cvf->open(file,fo) htcache cvf

91. 6/9/2016 Devpia A&D EVA 91 JAWS Cached Virtual FS Framework Perform Request svc() fh InputOutput Handler receive_file() receive_data() send_file() send_data() complete() error() InputOutput receive_file() receive_data() send_file() send_data() Synch IO receive_file() receive_data() send_file() send_data() Asynch IO receive_file() receive_data() send_file() send_data() Proactor Reactive IO receive_file() receive_data() send_file() send_data() Reactor Filecache Handle ioh io io io->send_file(“index.html”) write(fh) Proactor-> send_file(fh) Reactor-> send_file(fh)

92. 6/9/2016 Devpia A&D EVA 92 3.7 The JAWS Framework Visited Thread-Per-Request Thread-Per-Session Thread Pool Concurrency Caching Structured Caching Hinted Caching LFU LRU Asynchronous I/O Reactive I/O Synchronous I/O I/O Parse Headers Parse Request Read Request Protocol & Filter Perform Request Log Request

93. 6/9/2016 Devpia A&D EVA 93 4. Web Server Benchmarking Hardware Testbed Software Request Generator Experimental Results A Summary of Techniques for Optimizing Web Servers.

94. 6/9/2016 Devpia A&D EVA 94 4.1 Hardware Testbed System Specification Two Micron Millerinia PRO2 plus workstation 128M RAM Client – 200 MHZ Server – 180 MHZ ENI-155P-MF-S ATM Card ATM Network running through a FORE Systems ASX-200BX (bandwith 622 Mbps) Limitation of LAN emulation mode = 120 Mbps

95. 6/9/2016 Devpia A&D EVA 95 4.2 Software Request Generator Experiment Purpose Using WebSTONE Stress Tool Collect Client and Server Side Metric average server throughput, client latency Gather Loads and Statistics using various particular files sizes Refers the next page. Using various concurrency & dispatching strategies research the impacts of results.

96. 6/9/2016 Devpia A&D EVA 96 4.2 Software Request Generator Document Size 500 bytes 5 Kbytes 50 Kbytes 5M Bytes Frequency 35% 50% 14% 1% File Access Patterns

97. 6/9/2016 Devpia A&D EVA 97 4.3 Experimental Results Using 3 Models Definition of Terms Throughput Comparisons Latency Comparisons Summary of Benchmark Results A Summary of Techniques for Optimizing Web Servers

98. 6/9/2016 Devpia A&D EVA 98 4.3 Experimental Results Using 3 Models Synchronous Thread-per-Request Using Synchronous I/O Synchronous Thread Pool Using Synchronous I/O Asynchronous Thread Pool Using TransmitFile() for Asynchronous I/O TransmitFile() is a custom Win32 Function()

99. 6/9/2016 Devpia A&D EVA 99 4.3 Experimental Results Definitions of Terms Throughput The average # of bits received per second by client. Start Time HTTP Request received by client. End Time Connection is closed at the client end. Latency The average amount of delay in milliseconds Start Time HTTP GET Request received by client. End Time Completely Received File.

100. 6/9/2016 Devpia A&D EVA 100 4.3 Experimental Results Throughput Comparisons. Connection ↑ - Throughput ↓ As expected, The throughput for each connection degrades for each connection degrades as the connections per second increase.

101. 6/9/2016 Devpia A&D EVA 101 4.3 Experimental Results Throughput Comparisons. Refer Figure 3. Experiment Results from 50K Files. Thread-Per-Request Model Connection load ↑ -> Throughput ↓ rapidly Incurs higher thread creation overhead. Synchronous Thread Pool Connection load ↑ -> Throughput ↓ more gracefully Uses Pre-spawning thread.

102. 6/9/2016 Devpia A&D EVA 102 4.3 Experimental Results Throughput Comparisons. Refer Figure 1 through 5. TransmitFile() performs extremely poorly for small files ( i.e. < 50K bytes) TransmitFile() is forced to wait while the kernel services incoming request. As the Size of the file grows, TransmitFile() rapidly outperforms the synch model.

103. 6/9/2016 Devpia A&D EVA 103 4.3 Experimental Results Throughput Comparisons. Fig 1. 500 Byte File Fig 2. 5K Byte File

104. 6/9/2016 Devpia A&D EVA 104 4.3 Experimental Results Throughput Comparisons. Fig 3. 50K File Fig 4. 500K File

105. 6/9/2016 Devpia A&D EVA 105 4.3 Experimental Results Throughput Comparisons. Fig 5. 5M File

106. 6/9/2016 Devpia A&D EVA 106 4.4 A Summary of Techniques (for Optimizing Web Server) Lightweight Concurrency Don’t use Process-based concurrency Mechanism. Using POSIX thread. Specialized OS Feature Using IOCP Request Lifecycle system call overhead. Reducing synchronization. Caching files. Reducing Iterative System I/O Call (open, read, write..) Using “gather-write” Using writev (Unix System Call) Pre-computation HTTP response

107. 6/9/2016 Devpia A&D EVA 107 4.4 A Summary of Techniques (for Optimizing Web Server) Logging Overhead File System access Makes a significant number of I/O Calls. Synchronization overhead Thread/Process are required to log request to a common shared log file, to a common shared log file, access to the log file needs to be synchronized. access to the log file needs to be synchronized. Reverse hostname lookups Hostname is typically more useful information in the log file. Ident Lookups For identifying User Name Make setting up a new TCP/IP Connection

108. 6/9/2016 Devpia A&D EVA 108 4.4 A Summary of Techniques (for Optimizing Web Server) Transport layer optimizations The listen backlog Give Queue Size Higher. Socket Send buffers Should be set to the highest permissible limit. On Solaris, 64KB limit Nagle’s Algorithms Congestion Control Algorithm Server latency critical App (such as X-Windows) disable this algorithm. disable this algorithm.

109. 6/9/2016 Devpia A&D EVA 109 5. Concluding Remarks

110. 6/9/2016 Devpia A&D EVA 110 References Pattern Oriented Software Architecture Volume 2, Wiley, Douglas Schmidt et al. 서버 성능 분석 보고서, http://www.javaservice.net/~java/bbs/read.cgi?m=qn a&b=consult&c=r_p&n=1013740705 http://www.javaservice.net/~java/bbs/read.cgi?m=qn a&b=consult&c=r_p&n=1013740705 http://www.javaservice.net/~java/bbs/read.cgi?m=qn a&b=consult&c=r_p&n=1013740705 Douglas C. Schmidt, http://www.cs.wustl.edu/~schmidt/ http://www.cs.wustl.edu/~schmidt/ JAWS http://www.dre.vanderbilt.edu/JAWS/ http://www.dre.vanderbilt.edu/JAWS/