Andreas Ulbrich SDE – Microsoft Robotics

 Why CCR?  Hello, World!  Message-Based Coordination  CCR Examples  Asynchronous Programming Model (APM)  User Interfaces  Error Handling

 C  Concurrency  Process many task simultaneously  Scalability, Responsiveness  Leverage parallelism  C  Coordination  Exercise control  Orchestrate asynchronous operations  Handle (partial) failures  R  Runtime  Scheduler, Extensibility

 Asynchronous message passing (in-process)  No explicit threads, locks, semaphores!  Task scheduled based on message availability  Data-dependency scheduler  Models concurrency  Coordination primitives (join, choice, …)  Composition of data-driven components  Iterative tasks  Express sequential control flow of async. tasks

var queue = new DispatcherQueue(); var port = new Port (); port.Post("Hello, World!"); Arbiter.Activate(queue, Arbiter.Receive( false, port, message => Console.WriteLine(message) ) ); Port: channel for sending and receiving messages Post: sends a message Task: delegate that handles the message (message is consumed) Receive arbiter: coordination primitive Task queue: dispatcher schedules queues RR, task must be activated on a queue Port on which to receive the message Not persistent: handles only one message

Dispatcher Port Dispatcher Queues Threads Arbiter Handler Arbiter is attached to port Arbiter is activated on queue Dispatcher schedules items from its queues round-robin to run in its threads. Post places a message on the port Arbiter checks whether it can consume the message. Creates work item from message and handler Creates work item from message and handler Enqueue work item Thread calls handler with message as arg. Scheduler picks next work item to execute

Handler Dispatcher Port Dispatcher Queues Threads Arbiter Handler There can be many of everything

 Message triggers an operation  PortSet  Message carries parameter for operation  Decouples sender from actual implementation  Message signals completion of operation  Port, PortSet, …  Port is stand-in for actual result  Port can be send to other tasks  Coordinate tasks with messages

Single-Port Primitives Task-Scheduling (non port-specific) Multi-Port Primitives

 Executes at most one of its child-tasks PortSet resultPort = … Arbiter.Activate( queue, Arbiter.Choice Arbiter.Choice( resultPort, result => Console.WriteLine("result: " + result), exception => Console.WriteLine("exception“) ) ); Handler if string received Handler if exception received

 Executes when all of its branches can execute  Coordinate on completion of concurrent tasks  Atomic consumption to prevent deadlocks Arbiter.Activate(queue, Arbiter.JoinedReceive Arbiter.JoinedReceive (false, resultPort1, resultPort2, (result1, result2) => { Console.WriteLine(“done”); } ) ); Ports on which to receive messages Handler receives both results as arguments

 Executes on reception of multiple items of the same type  Scatter/Gather, coordinate completion of parallel tasks  Atomic consumption to prevent deadlocks Arbiter.Activate( queue, Arbiter.MultiplePortReceive Arbiter.MultiplePortReceive( false, resultPorts, results => { Console.WriteLine(results.Length); } ) ); Array of ports on which to receive messages Handler receives array of results as argument

 Why CCR?  Hello, World!  Message-Based Coordination  CCR Examples  Asynchronous Programming Model (APM)  User Interfaces  Error Handling

 BCL: Asynchronous versions for many operations  BeginOperation, EndOperation pair  Callback when operation completed/failed  BeginRead(buffer, offset, count, callback, state)  Returns IAsyncResult (moniker for pending result)  Also passed to callback  EndRead(asyncResult)  Returns result of operation

 Callback maybe called from any thread, e.g.  Calling thread (synchronous completion)  Thread pool (potential to starve)  Depends on implementation of Begin/End  Coordination is clumsy  Sequential (continuation passing, nested delegates)  Recurrence (ping pong between callbacks)  Scatter/Gather, Partial failure

 Use CCR to  Model concurrency of application  Coordinate asynchronous operations  Getting out of APM is easy  In the callback post IAsyncResult to a CCR port

IEnumerator CcrReadFileAsync(string file) { var resultPort = new Port (); using (var fs = new FileStream(file,…,FileOptions.Asynchronous)) { var buf = new byte[fs.Length]; fs.BeginRead(buf, 0, buf.Length, resultPort.Post, null); IAsyncResult result = null; yield return Arbiter.Receive(false, resultPort, ar => { result = ar; }); try { fs.EndRead(result); ProcessData(buf); } catch { // handle exception } } Iterative task: models sequential control flow 1)Begin read operation 2)“wait” for result of read 3)Process read data Iterative task: models sequential control flow 1)Begin read operation 2)“wait” for result of read 3)Process read data Use port to coordinate on completion of async. operation Callback: simply post IAsyncResult as a message Yield receiver task, Task is activated by dispatcher, Dispatcher calls MoveNext() when task complete. Does not block thread! Yield receiver task, Task is activated by dispatcher, Dispatcher calls MoveNext() when task complete. Does not block thread!

IEnumerator CcrReadFileAsync(string file) { var resultPort = new Port (); using (var fs = new FileStream(file,…,FileOptions.Asynchronous)) { var buf = new byte[fs.Length]; fs.BeginRead(buf, 0, buf.Length, resultPort.Post, null); yield return (Receiver)resultPort; var result = (IAsyncResult)resultPort; try { fs.EndRead(result); ProcessData(buf); } catch { // handle exception } } Simplified notation

 Directly new IterativeTask  queue.Enqueue( new IterativeTask(“test.txt”, CcrReadFileAsync) );  Yielded from an iterative task new IterativeTask(…);  yield return new IterativeTask(…);  As task conditioned by an arbiter Arbiter.ReceiveWithIterator  Arbiter.Activate(queue, Arbiter.ReceiveWithIterator( false, port, CcrReadFileAsync ) )

 Simplified code  Access to locals, no need to pass async. state  Sequential control flow, e.g. loops  No nesting, ping/pong of callbacks using try/finally  Ability to use using, try/finally  Simplifies resource management

 Block-copy large file from input stream to output stream  Sequential control flow with repeated async. operations  while (input not empty): read block from input write block to output

public static PortSet Read( Stream stream, byte[] buffer, int offset, int count) { var resultPort = new PortSet (); stream.BeginRead(buffer, offset, count, asyncResult => { try { resultPort.Post(stream.EndRead(asyncResult)); } catch (Exception e) { resultPort.Post(e); } }, null); return resultPort; } Port set: collection of ports, one for each possible result of the operation Post result or exception

static IEnumerator CopyStream(Stream source, Stream dest) { try { var buffer = new byte[blocksize]; int read = 0; do { Exception exception = null; yield return Arbiter.Choice( StreamAdapter.Read(source, buffer, 0, buffer.Length), r => { read = r; }, e => { exception = e; } ); if (exception == null) { // write to dest Choice arbiter: executes only one of its branches depending on the received message Port set on which to receive message Handler if int is received Handler if Exception is received

static IEnumerator CopyStream(Stream source, Stream dest) { try { var buffer = new byte[blocksize]; int read = 0; do { var readResult = StreamAdapter.Read( source, buffer, 0, buffer.Length); yield return (Choice)readResult; var exception = (Exception)readResult; if (exception == null) { // write to dest read = (int)readResult; Simplified notation

 See the complete sample in Visual Studio

 Read and write are sequential  Modify to exploit parallelism  Read the next block while writing  Have to coordinate  “wait” until read succeeds and write succeeds or read fails or write fails Receivers Join Choice

while (write > 0) { var writeResult = StreamAdapter.Write(dest, bufferB, 0, write); if (read > 0) { // read new bytes and write existing buffer readResult = StreamAdapter.Read(source, …); yield return Arbiter.Choice( Arbiter.JoinedReceive (false, readResult, writeResult, (r, s) => { read = r; } ), Arbiter.Receive (false, readResult, e => { exception = e; }), Arbiter.Receive (false, writeResult, e => { exception = e; }) ); Choice arbiter: only one of its branches will execute Receiver branches for exceptions Join branch: receives int on readResult and EmptyValue on writeResult Join handler delegate gets both messages are parameters

 See the complete sample in Visual Studio

 Concurrency desirable for responsivenes  UI often single threaded or has thread affinity  WinForms, WPF  Manual thread management, cross-thread calls  Use CCR to decouple UI from application logic  Adapter for handling cross thread calls  messages

CCR to UI adapter (WPF or WinForms) CCR to UI adapter (WPF or WinForms) Application Logic (CCR Tasks) Application Logic (CCR Tasks) UI sends message to trigger application logic Application logic sends message to indicate completion or status CCR adapter handles message in UI context

 Concurrent, async: try/catch wont do the job  Method 1: explicit handling  Exception caught at origin and posted as message for coordination  See CopyStream sample  Method 2: Causalities  Captures all exception thrown by tasks that are executed as result of the same cause (transitive)   Nested exception handling in concurrent, asynchronous programs!

void ExecuteWithCausality() { var causality = new Causality("my causality"); Dispatcher.AddCausality(causality); var port = new Port (); port.Post(42); Arbiter.Activate(queue, Arbiter.Receive(false, port, HandleMessage) ); Arbiter.Activate(queue, Arbiter.Receive(false, (Port )causality.ExceptionPort, Console.WriteLine ) ); } void HandleMessage(int message) { throw new Exception("Catch me if you can!"); } Create a new causality and add it to the active causalities of the current task Send a message and activate a receiver. All active causalities travel with the message to the tasks that handle the message. Handle the exception thrown within the causality.

 Manage queue of tasks  Tasks are picked round-robin from queues in scheduler  Parameters  Execution policy, queue length, scheduling rate  Use separate queues to  Separate tasks with significantly different arrival rates  Enforce scheduling policies

 Manage a thread pool  Default: 1 per core, or 2 on single core  Parameters  Number of threads, STA/MTA, Priority, Background  Use separate dispatchers  Interop with legacy components (COM)  Isolate “uncooperative” components  Separate tasks with significantly different lengths

 Asynchronous message passing  Easy coordination of concurrent, asynchronous task  Compositional Arbiters  Sequential control flow with iterators  Exploits parallelism  Easy integration into.NET applications

 Microsoft CCR and DSS Toolkit 2008   Microsoft Robotics Developer Studio 2008   Express Version (free for non-commercial use)  Standard Version (free for academic institutions)  Internal on \\products

 Derive from CcrServiceBase  Stores reference to dispatcher queue  Overloads for Activate, TimeoutPort, etc  Model operations as messages  PortSet for response  Expose operations port  Activate persistent receivers for operation messages

 Protect state with Interleave arbiter  TearDownReceiverGroup  Terminates Interleave  ExclusiveReceiverGroup  Message handlers that modify state  ConcurrentReceiverGroup  Message handlers that read state  Efficient reader/writer lock (writer biased)  Interleave protection spans iterative tasks!

public CcrComponent(DispatcherQueue queue) : base(queue) { Activate( Arbiter.Interleave( TeardownReceiverGroup new TeardownReceiverGroup( Arbiter.Receive (false, _operationsPort, StopHandler)), ExclusiveReceiverGroup new ExclusiveReceiverGroup( Arbiter.ReceiveWithIterator (true, _operationsPort, WriteHandler)), ConcurrentReceiverGroup new ConcurrentReceiverGroup( Arbiter.Receive (true, _operationsPort, ReadHandler)) ) ); }

 Culture, user principal, …  Explicit  Make it part of your message  Dispatcher  Useful if set of context is limited  E.g. one dispatcher per culture  Custom receivers  Capture context on post  Set it on execute

 Intel Xeon 5150 (2x 2.66 GHz), 4 GByte  Persisted signal item receiver latency  Send message  schedule task  run task  1.67 µs  Iterator latency  Spawn  schedule  run task  2.1 µs