1. .NET Core Summer event 2019 – Brno, CZ
Async demystified
.NET Core Summer event 2019 – Brno, CZ
Karel Zikmund – @ziki_cz

2. Agenda History of async patterns in .NET History and evolution of Task
async-await
Based on internal talk from author of Task, async and all good things around – Stephen Toub, architect of BCL

3. APM pattern: Asynchronous Programming Model
.NET Framework 1.0/1.1 …
interface IAsyncResult {
bool IsCompleted;
bool IsCompletedSynchronously;
object AsyncState;
WaitHandle AsyncWaitHandle; }
Across BCL:
IAsyncResult BeginFoo(..., AsyncCallback callback, object state);
void EndFoo(IAsyncResult iar);
IAsyncResult
AsyncWaitHandle – ManualResetEvent or AutoResetEvent
Across BCL
Usage either:
Wait for callback to be called, or
Call EndFoo which will block until completed

4. APM pattern Synchronous call: Achieving the same:
IAsyncResult BeginFoo(..., AsyncCallback callback, object state);
void EndFoo(IAsyncResult iar);
Synchronous call:
Foo();
Achieving the same:
EndFoo(BeginFoo(..., null, null));
Leveraging asynchronous calls:
BeginFoo(..., iar => {
T val = EndFoo(iar);
// do stuff ...
});
Single operation works fine, but in reality you do more – e.g. in a loop

5. APM – Example Copy stream to stream: int bytesRead;
while ((bytesRead = input.Read(buffer)) != 0) {
output.Write(buffer, 0, bytesRead); }

6. APM – Nesting problem
BeginRead(..., iar => { int bytesRead = EndRead(iar); input.BeginWrite(..., iar2 => { int bytesWritten2 = EndWrite(iar2); BeginRead(..., iar3 => { int bytesRead3 = EndRead(iar3); BeginWrite(..., iar4 => { // ... again and again });
Manually it does not work – somehow turn it into loop
It’s possible but extremely long and tricky
Further complications with IsCompletedSynchronously

7. APM – IsCompletedSynchronously
IAsyncResult r = BeginRead(..., iar => {
if (!iar.IsCompletedSynchronously) {
// ... asynchronous path as shown earlier }
});
if (r.IsCompletedSynchronously) {
// ... Synchronous path
Even worse in loop
Overall very complicated
Queueing on ThreadPool much simpler
For perf reasons
In the loop it is even more complicated
However:
On MemoryStream, the data is already available … instead of ThreadPool, call delegate immediately
-> Leads to recursive calls -> 10K StackOverflow
Bottom part:
Even BCL lots of wrappers (e.g. in Networking: LazyAsyncResult) with lots of specializations
Very complicated

8. EAP: Event-based Asynchronous Pattern
.NET Framework 2.0
obj.Completed += (sender, eventArgs) => {
// ... my event handler }
obj.SendPacket(); // returns void
Did not solve multiple-calls problem, or loops
Introduced context
Straightforward idea – Completed event
Kick off operation, then Completed handler is invoked (generally on ThreadPool)
5-10 classes in BCL … like SmtpMail, TcpClient, BackgroundWorker
Downsides:
We shipped it in .NET Framework 2.0 and quickly realized that it is interesting experiment, but not exactly addressing real needs

9. Task .NET Framework 4.0 MSR project – parallel computing
Divide & conquer efficiently (e.g. QuickSort)
Shaped Task – similar to today
Task – represents general work (compute, I/O bound, etc.)
= promise / future / other terminology
Task / Task<T> – operation (with optional result T)
T … in the case of Task<T>
State related to synchronization
State related to callback
90% right, 10% keeps Toub awake at night even after 10 years and would love to change it
NOT tied to ThreadPool – not tied to executing delegate
Shove result into it
Can be completed
Can wake up someone waiting on it

10. Task / TaskCompletionSource
Here's a callback, invoke it when you're done, or right now if you've already completed
I want to block here, until your work is done
Cannot be completed by user directly
TaskCompletionSource … wrapper for Task
Holds Task internally and operates on it via internal methods
Methods:
SetResult
SetException
SetCancelled
Task – something to consume - hook up to, not to change directly (no control)
TaskCompletionSource – can alter state of Task … has control over Task
Lazy initialization (something over network) … you are in charge who can change complete the work

11. Task – Consumption Either: Or: Even multiple times: ContinueWith:
Task<T> t;
Either:
t.Wait(); // Blocks until Task is completed
Or:
t.ContinueWith(callback); // Will be executed after Task is completed
Even multiple times:
t.ContinueWith(callback2);
t.ContinueWith(callback3);
ContinueWith:
Does not guarantee order of executions
Always asynchronous (queued to ThreadPool/scheduler in general)
Wait - creates ManualResetEvent which will be signaled when one of the SetResult/SetException/SetCancelled is called
Option: TaskExecutionOption to do it synchronously
APM - IAsyncResult … no shared implementation
Everyone had to have their own implementation
Task model - you don't pass delegate at creation, but you can walk up on any of them and say "call me when you're done"
Abstractions enabled - like async-await
await hooks up the callback

12. Task.Run We complicated things  Task<T> Task.Run(delegate d)
Adds field to Task with ‘d’
Queues work to ThreadPool
Thread grabs it, executes it, marks task completed
Sets completed = execute callback, waking up things waiting on it, etc.

13. Task.Run implementation
Task<T> Run(Func<T> f) { var tcs = new TaskCompletionSource<T>(); ThreadPool.QueueUserWorkItem(() => { try { T result = f(); tcs.SetResult(result); } catch (ex) { tcs.SetException(ex); } }); return tcs.Task;
TaskCompletionSource creates Task
Returns Task to be awaited on, etc.
Now we implemented Task.Run without storing any delegate on Task

14. async-await .NET Framework 4.5 / C# 5 Example of asynchronous code:
Task<int> GetDataAsync();
Task PutDataAsync(int i);
Code:
Task<int> t = GetDataAsync();
t.ContinueWith(a => {
var t2 = PutDataAsync(a.Result);
t2.ContinueWith(b => Console.WriteLine("done"));
});
GetData/PutData … maybe across the wire

15. async-await C# 5 with async-await helps us:
Task<int> t = GetDataAsync();
t.ContinueWith(a => {
var t2 = PutDataAsync(a.Result);
t2.ContinueWith(b => Console.WriteLine("done"));
});
C# 5 with async-await helps us:
int aResult = await t;
Task t2 = PutDataAsync(aResult);
await t2;
Console.WriteLine("done");
Compiler translates it to the code above (hand-waving involved)
Compiler does not treat Task specially, but it just looks for pattern (awaiter pattern)

16. Awaiter pattern Translated to: int aResult = await t;
var $awaiter1 = t.GetAwaiter();
if (! $awaiter1.IsCompleted) { // returns bool
// ... }
int aResult = $awaiter1.GetResult(); // returns void or T
// If exception, it will throw it
Bold methods are pattern matching

17. Awaiter pattern – details
void MoveNext() { if (__state == 0) goto label0; if (__state == 1) goto label1; if (__state == 42) goto label42; if (! $awaiter1.IsCompleted) { __state = 42; $awaiter1.OnCompleted(MoveNext); return; } label42: int aResult = $awaiter1.GetResult();
“! IsCompleted” part is complicated – I have to hook up code that comes back here when task completes
All of it is part of MoveNext method -- it is a state machine, every await in method is state in state machine (hand waving a bit)
OnCompleted has slightly more complicated signature

18. State Machine State machine: string x = Console.ReadLine();
int aResult = await t;
Console.WriteLine("done" + x);
State machine:
struct MethodFooStateMachine {
void MoveNext() { ... }
local1; // would be ‘x’ in example above
local2;
params;
_$awaiter1; }
How does ‘x’ survive continuation? (it is just on stack) – need to capture it
Same in continuations - C# compiler lifts it to keep it on heap allocated object (floats through closures)
Same in state machine
Compiler optimizes - stores here things only crossing await boundary
In debug -- struct is class -- for debuggability, but for perf struct
Why? These async methods often complete synchronously – example:
BufferedStream … large buffer behind with inner stream
If I ask for 1B, but it reads 10K in bulk, then lots of calls end up synchronously
If it was class, then we would allocate per call

19. State Machine – Example
public async Task Foo(int timeout) { await Task.Delay(timeout); } public Task Foo(int timeout) { FooStateMachine sm = default; sm._timeout = timeout; sm._state = 0; sm.MoveNext(); return ???;
struct FooStateMachine {
int _timeout; // param
// locals would be here too
void MoveNext() { ... }
int _state;
TaskAwaiter _$awaiter; }

20. State Machine – Example
public Task Foo(int timeout) { FooStateMachine sm = default; sm._tcs = new TaskCompletionSource(); sm._timeout = timeout; sm._state = 0; sm.MoveNext(); return sm._tcs.Task; } AsyncValueTaskMethodBuilder.Create(); _tcs.Task -> _builder.Task;
struct FooStateMachine {
int _timeout; // param
// locals would be here too
void MoveNext() {
// ...
_tcs.SetResult(...); }
int _state;
TaskAwaiter _$awaiter;
TaskCompletionSource _tcs;
_tcs on state machine is logically there
problem 2 allocations – TaskCompletionSource and Task (inside)
For the synchronous case we want 0 allocations ideally (BufferedStream example)
Even the Task/TaskCompletionSource is problematic, because it is anything Task-like
Each Task-like type (except Task) has attribute defining builder - builder pattern
ValueTask has one -> AsyncValueTaskMethodBuilder
instead of new TaskCompletionSource() -> AsyncValueTaskMethodBuilder.Create();
We have internally in System.Runtime.CompilerServices structs:
AsyncTaskMethodBuilder, AsyncTaskMethodBuilder<T>, AsyncVoidMethodBuilder
instead of _tcs.Task -> _builder.Task
We eliminated TaskCompletionSource allocation
What about the Task?

21. State Machine – Summary
What about Task allocation?
Builder can reuse known tasks
Task.CompletedTask (without value)
boolean – True/False
int … <-1,8>
LastCompleted (e.g. on MemoryStream)
Does not work on SslStream (alternates headers and body)
Size: 64B (no value) / 72B (with value)
Azure workloads OK (GC will collect)
Hot-path: up to 5%-10% via more GCs

22. ValueTask
.NET Core 2.0
Also as nuget package down-level
struct ValueTask<T> { T;
Task<T>; }
Only one of them: T+null or default+Task<T>
NET Core 2.1
ValueTask<int> Stream.ReadAsync(Memory<byte>, ...)
Methods are 1-liners (if Task<T> == null, do something, else something else)
Nicely handles synchronously completing case
Note: Non-generic ValueTask does not make sense - only Task inside … we have CompletedTask
.NET Core 2.1
Luckily we introduced Memory<T> at the same time, as we cannot overload on return type
That's why sometimes in PRs we wrap byte[] in Memory first … to use the ValueTask
Design-guidelines: Start with Task … use ValueTask only in hot-path scenarios

23. ValueTask – Can we improve more?
What about the 1% asynchronous case?
.NET Core 2.1
struct ValueTask<T> { T;
Task<T>;
IValueTaskSource<T>; }
struct ValueTask {
Task;
IValueTaskSource;
IValueTaskSource – complicated interface
almost the awaiter pattern:
Are you completed?
Hook up a call back
Get a result
All implementations on ValueTask are now ternary
Value: You can implement it however you want, incl. reset (reuse)
Complicated to do, so not everywhere
Socket.SendAsync/ReceiveAsync
Object per send/receive … if one at a time (typical)
0 allocation for loop around Send/Receive on Socket
Same: NetworkStream, Pipelines, Channels

24. Summary @ziki_cz APM pattern = Asynchronous Programming Model
.NET Framework 1.0/1.1 ( )
IAsyncResult, BeginFoo/EndFoo
Limited nesting / loops
EAP = Event-based Asynchronous Pattern (.NET Framework 2.0)
Events – similar problems as APM
Task (.NET Framework 4.0)
Wait / ContinueWith
TaskCompletionSource (for Write)
async-await (.NET Framework 4.5 / C# 5)
Awaiter pattern, state machine
ValueTask (.NET Core 2.0)
Don’t use unless you are on hot-path
Hyper-optimizations possible, stay away! 
@ziki_cz