Practical Session 12 Reactor Pattern

Disadvantages of Thread per Client It's wasteful – Creating a new Thread is relatively expensive. – Each thread requires a fair amount of memory. – Threads are blocked most of the time waiting for network IO. It's not scalable – A Multi-Threaded server can't grow to accommodate hundreds of concurrent requests – It's also vulnerable to Denial Of Service attacks (an attempt to make a machine or network resource unavailable to its intended users). Poor availability – It takes a long time to create a new thread for each new client. – The response time degrades as the number of clients rises. Solution – The Reactor Pattern is another (better) design for handling several concurrent clients.

Reactor Pattern It's based on the observation that if a thread does not need to wait for Network IO, a single thread could easily handle tens of client requests alone. Uses Non-Blocking IO, so threads don't waste time waiting for Network. Have one thread in charge of the network: – Accepting new connections and handling IO. – As the Network is non-blocking, read, write and accept operations "take no time", and a single thread is enough for all the clients. Have a fixed number of threads, which are in charge of the protocol. – These threads perform the message framing (Tokenization) – Perform message processing (what to do with each message). Note that unlike the Multi-Threaded server, in this design a single thread may handle many clients.

How Is It Done? We require non-blocking network access. We talked about: – Channel (wrap a socket, decouples open and connect; supports concurrent threads, and concurrent reading and writing; only one thread reading at a time; only one thread writing at a time). – Buffer (need it to read/write to channels) We will talk about: – Selector

Selectors When we perform non-blocking IO, read(), we read currently available bytes and return the result. This means we can read or write 0 bytes, and this is what will probably happen most of the time. We would like a way for our thread to wait until any of our channels is ready, and only then perform the read(), write() or accept(), only on the ready channel. This is what the Selector class does. Tutorial:

Selector A Selector allows a single thread to handle multiple Channel's. Handy if your application has many connections (Channels) open, but only has low traffic on each connection. For instance, in a chat server.

Workflow Example: Initial State Server listens to some port. Selector registered to retrieve ACCEPT events when requested. Attachment ConnectionAcceptor used to accept incoming connection.

Client 1: Attempts to connect to server

ACCEPT Event is raised

Server selects events, handles them.

Client 1: Sends data. Client 2: Connection request.

New events raised.

Events Selected and handled.

New Events raised – Write Events are selectable!

Write event for client 1 handled. For client 2 ignored

Client 3: Connection request. Client 2/3: Write

Events raised!

And handled…

Registering Selector to Channel The Selector registers itself to a Channel. Registration means that the channel informs the Selector when some event happens. This event can be: – Channel is ready to be read from. [incoming message] – Channel can now be written to. [outgoing message] – Channel can now accept a new connection. [incoming connection request] During registration a selection key is given to the channel, and the channel is told which events to monitor: – Selector selector = Selector.open(); // a new Selector – ConnectionHandler connectionHandler = new ConnectionHandler(); Or ConnectionAccepter connectionAcceptor = new ConnectionAcceptor(); – socketChannel.register(selector, SelectionKey.OP_READ, anAttachmemt); A channel can monitor OP_READ, OP_WRITE, OP_ACCEPT, OP_CONNECT In order to support more than one command, we use bitwise or: – OP_READ | OP_WRITE Attachment: – Done when registering a selector to a channel. – An attachment is an object that we will have access to when the event occurs – Usually, this object is associated with a task that should be performed.

Registration Example Adding a selector for the server channel which accepts new connections:

register() Registers this channel with the given selector, returning a selection key. This method first verifies that this channel is open and that the given initial interest set is valid. If this channel is already registered with the given selector, the selection key representing that registration is returned after setting its interest set to the given value. Otherwise, The resulting key is added to this channel's key set before being returned.

select() Once a Selector is registered to some channels, one can use the "select()" method. This method blocks until at least one of the channels is ready for the registered event. Then, a list of SelectionKeys is returned. Each Selectionkey is associated with one channel, and holds: – The interest set (events we’re registered to) – The ready set (events the channel is ready for) – The Channel – The Selector – the attachment registered on the channel (optional).

Three Different Server Events Event is isAcceptable() – OP_ACCEPT – We retrieve ConnectionAcceptor object attached. – We run its accept() method, which accepts a new incoming connection through that channel. – Note: ConnectionAcceptor contains a reference to its channel (done upon construction). Event is isReadable() – OP_READ – We retrieve ConnectionHandler object attached. – We run its.read() method, which attempts to read data received by the channel from client. Event is isWritable() – OP_WRITE – We retrieve ConnectionHandler object attached. – We run its.write() method, which attempts to write data to the channel in order to send it from server to client.

ConnectionAcceptor Used only for ServerSocketChannel in order to accept new connections. Accept workflow: Selector now listens to OP_READ events on this channel in anticipation of new data.

Accepting a new connection ConnectionAcceptor method. In ConnectionHandler.create() we attach the ConnectionHandler to the SelectorKey. In register(): 0 indicates that initially it does not listen to any type of event. switchToReadOnlyMode() changes the Selector to listen to SelectionKey.OP_READ The “key” returned from the register() is the token [id] of this registration (one key per channel). Accessing the selector can be done via this “key”.

ConnectionHandler ConnectionHandler contains: outData: Vector - contains data to be sent to client. – SocketChannel it needs to read from and write to. read() from channel to (local) ByteBuffer. write() from outData in channel. – MessageTokenizer in order to tokenize received data. void addBytes(ByteBuffer buffer) – adds (the local) ByteBuffer to StringBuffer Boolean hasMessage() – returns true if a complete message exists in inData String nextMessage() – removes and returns a complete message from inDat – ServerProtocol in order to process message, and create response. String processMessage(String message) – processes message, creates response. Boolean isEnd(String message) – checks whether message is shutdown request. – ProtocolTask is our runnable. One runnable per one connection handler. We run the same task once per event handling. – Uses MessageTokenizer in order to retrieve a complete message. – Uses ServerProtocol in order to process message and create response. » Response stored in outData

ConnectionHandler: read()

ConnectionHandler: write() outData = Vector (). Each message to be sent is stored in ByteBuffer. All messages to be sent are stored in outData.

Reactor main loop. Scans the channels and retrieves the new events. Initiates the appropriate function of the retrieved attachment depending on event type.

The different parts The Reactor: – Our driver [main] – In charge of handling reading, writing and accepting new connection events. Reading and Writing are encapsulated in the ConnectionHandler class Accepting is encapsulated in the ConnectionAcceptor class. – Holds a thread-pool of workers (an Executor). [inside ReactorData object] These worker-threads are in charge of the protocol handling. They receive the bytes read from the network, and apply the protocol on the received data. ProtocolTask: [ our runnable] – The class which represents tasks submitted to the executor. – The ProtocolTask object is created along with the ConnectionHandler. – Task object is created once, and used every time new data is received. Checks if a complete message is found Applies protocol on received data Adds response to outData vector

Continued ConnectionHandler holds for each connection: – The SocketChannel for a specific connection – Vector outData – the data to be sent – ProtocolTask which includes MessageTokenizer which stores and handler received data. Important: – We don't have one thread per connection. – We have one ConnectionHandler object per connection. – We have one ConnectionAcceptor object for the ServerChannel. – Each channel is associated with SelectionKey – ProtocolTask is our runnable, and executed whenever new data is received, stores received data inside it and applies our MessageTokenizer on the processed data, the response generated is sent to the outData to be sent.

AsyncServerProtocol Has same functionality of ServerProtocol Has two additions: – boolean shouldClose()://like thread.shouldStop() Returns true after the terminating message has been encountered. This is used to signal the connection handler that the protocol has done its work and it's time to send all the remaining bytes and quit. – void connectionTerminated(): Used to notify the protocol the client closed the connection, or that the network connection was lost. This is called by the ConnectionHandler when it identifies a network error.