1. Async #2
Lucian Wischik Senior Program Manager Microsoft
NDC 2012

2. Plan of talk Slides and code here – http://blogs.msdn.com/lucian
DPE Metro Template
4/11/2019
Plan of talk
1. Async Design Patterns
TAP and Progress
Task.Yield
IAsyncEnumerable
When to expose async APIs
When not to expose them
Fire-and-forget methods
Void-returning asyncs
2. Integrating Async into Existing Code
Call Sync from Async with Task.Run
Call Async from Sync with pump
Turn events into tasks
User-defined awaitables
Console application
3. Async Codegen and Performance
Codegen
Optimize network Big-O
Client vs Server perf
Avoid needless async methods
Hand-optimize the hot path
ConfigureAwait(false)
Slides and code here –
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

3. Sequential composition – the “semicolon operator” await Statement1();}
If statements
if (b)
await Statement1a();
else
await Statement1b();
await Statement2();
if (b) {
Statement1a( () => {
Statement2();
});
} else {
Statement1b(() => { }
While statements
foreach (var i in cc) {
await Statement1(); }
await Statement2();
using (var ii=cc.GetEnumerator()) {
Action ffor = null;
ffor = () => {
if (!ii.MoveNext()) {
Statement2();
return;
Statement1(() => {
ffor();
}); };

4. 1. Async Design Patterns, and API Design
Task Asynchronous Pattern
Async methods have suffix “Async” and return Task or Task<T>
Progress
Use IProgress<T>, with “new Progress<T>(delegate)”
Task.Yield
Doesn’t yield as you’d expect; but within WPF, Dispatcher.Yield does.
IAsyncEnumerable<T> pattern
Must be produced by hand, but is user-friendly to consume
Don’t wrap sync APIs as async, nor vice versa
Let the caller of your library know the truth
Fire-and-forget APIs
Void-returning asyncs are only for top-level event handlers

5. 2. Integrating Async into Existing Code
Async infects the callstack up and down
When you make a method async, its callers should be made async too
To call Sync from Async:
Easy: use Task.Run where needed (if code is okay on a threadpool thread)
To call Async from Sync:
Hard: to make it truly synchronous, need a nested message-loop (BAD!)
To use an event-based API with async:
Use TaskCompletionSource to generate a task from an event
To await a non-Task type directly:
Define a GetAwaiter method on it, which returns TaskAwaiter
Console apps
You can’t have async void Main. But it’s a multithreaded sync context…

6. 3. Async Performance and CodeGen
Optimize the slow things first
Don’t focus on optimizing CPU if your algorithm is network-bound
Server vs client: perf goal is responsiveness vs throughput
Server: fine to block your thread. Client: absolutely not okay.
Don’t create tasks or async methods needlessly
If the only await is in “return await <expr>” then method shouldn’t be async
Hand-optimize the hot path
If data is likely already present, use Task.FromResult rather than async method
Factor out CPU-bound work away from awaits
Any scope with an “await” will lift its variables; much more costly
Use .ConfigureAwait(false)
If your library doesn’t need to return to the UI thread, then don’t!

7. 3. Async Performance and CodeGen
async Task FooAsync() {
BODY }
struct FooAsync_StateMachine : IAsyncStateMachine { //(1,2,3) private int _state; public AsyncTaskMethodBuilder _builder;  
private void MoveNext() { try { switch (_state) { TRANSFORMED_BODY } } catch (Exception ex) { _builder.SetException(ex); return; } _builder.SetResult(); }
private void SetStateMachine(IAsyncStateMachine sm) {
_builder.SetStateMachine(sm); }
Task FooAsync() {
var sm = new FooAsync_StateMachine();
sm._state = -1;
... copy params & this if needed into S.M.
sm._builder = AsyncTaskMethodBuilder.Create();
sm._builder.Start(ref sm);
return sm._builder.Task; }
(1) The compiler implicitly generates a state machine for each async method. Each state corresponds to a piece of code between await statements. The MoveNext() method will advance it to the next state.
(2) The state-machine is a struct, for efficiency reasons -- so that on the “fast path” where no awaits were actually needed, then it doesn’t need to be allocated on the heap.
(3) The “builder” is much the same as the TaskCompletionSource we saw earlier
Note that the “Task FooAsync()” method that we generate has exactly the same metadata-signature as what the user wrote.

8. 3. Async Performance and CodeGen
Console.WriteLine("a");
await Task.Delay(100);
Console.WriteLine("b");
switch (_state) {
case 0: goto AFTERAWAIT0;
case -1: // fallthrough }
Console.WriteLine("a");
TaskAwaiter tmp = Task.Delay(100).GetAwaiter();  //(4)
if (!tmp.IsCompleted)
{  //(5)
_state = 0;
_awaiter = tmp;
_builder.AwaitOnCompleted(ref tmp, ref this);  //(8)
return;
AFTERAWAIT0:  //(9)
tmp = (TaskAwaiter)_awaiter;
_awaiter = default(TaskAwaiter);
tmp.GetResult();  //(6)
tmp = default(TaskAwaiter);  //(7)
Console.WriteLine("b");
(4) The await operator is pattern-based. For “await t”, the compiler makes a call to t.GetAwaiter() to get an awaiter. For instance, you could make an extension method “MyAwaiter GetAwaiter(this int i)” to be able to await integers – in which case tmp would have type MyAwaiter rather than TaskAwaiter. WinRT uses this, so you can await an IAsyncInfo.
(5) In this case we awaited Task.Delay(100), which won’t have completed yet. But imagine if the task had already completed. Then it would go straight to calling tmp.GetResult(), with no need for heap allocations.
(6) The job of tmp.GetResult() is to throw any exceptions from the task (if any), and to return a value (if any).
(7) We null-out the temporary variable immediately so it can be garbage-collected.

9. 3. Async Performance and CodeGen
int x = 10;
await t1;
int z = 10; {
int y = 15;
Console.Write(x + y + z); }
class FooAsync_StateMachine:
private int _x, _z;
MoveNext:
this._x = 10;
await t1; this._z = 10;
{ int y = 15;
Console.Write(this._x + y + this._x); }
Ideally it’d work like this: “If a local is written before an await, and read after an await, then it must be lifted into the state-machine (either by permanently locating it in the state machine -- or by putting there just before an await and restored afterwards, whichever is more efficient).”
In practice: If a local’s scope includes an await, then C# will permanently relocate the local into the state machine. This includes “z” in the above code. VB will relocate ALL locals into the state machine.
Cost to access a field is about 3x as high as a local.

10. 3. Async Performance and CodeGen
int[] a; int i;
a[i].CompareTo(await t);
[[PUSH a]] ; (int[])
[[PUSH i]] ; (int[], int)
DUP0:DUP1 ; (int[], int, int[], int)
LDELEM ; (int[], int, int)
POP ; (int[], int)
TUPLE.NEW ; (Tuple<int[], int>)
STFLD this._stack
... RETURN ; do awaiter pattern...
AFTERAWAIT0:
LDFLD this._stack ; (Tuple<int[], int>)
MAKE_STACK ; (int[], int)
CALL t.GetResult ; (int[], int, int)
MAKE_LVALUES ; (&int, int)
CALL CompareTo ; (bool)
Normally, the compiler will evaluate each sub-expression in turn (pushing it onto the stack), then it will call the desired operation.
If there is an “await” which doesn’t take the fast-path, then it’ll have to save the stack into the state machine before returning. That’s because the stack has to be empty when at each RETURN operaton. The compiler saves the stack into a Tuple of the appropriate type, stored in a state-machine field “_stack” of type Object.
But if there were any managed addresses on the stack, then this isn’t allowed. In the above code, it would normally want to evaluate a[i] as a managed address (lvalue) before calling the CompareTo method; all struct methods are similarly invoked on managed addresses in case the method mutates the struct.
In such cases the compiler has to avoid pushing the managed address onto the stack in the first place. Instead it pushes the constituent parts (in this case “a” and “i”). These can be saved into the Tuple okay. Later on, after the “await” has finished and immediately prior to the call to “CompareTo”, it reconstitutes those constituent parts into the address. Note (*): it still had to issue a dummy LDELEM call in advance, to shake out any ArrayIndexExceptions or NullReferenceExceptions that might arise.
Managed addresses can come only from “LOCAL”, “rvalue.FIELD”, “rvalue[rvalue]” and from ByRef parameters (disallowed in async methods). Managed addresses are only ever consumed by “lvalue.M(…)”, “lvalue=rvalue”, “lvalue+=rvalue”, “lvalue++”, and passing an lvalue ByRef.

11. 3. Async Performance and CodeGen
Optimize “Network Big-O” first
Strive to do network requests in parallel
async Task f() {
await LoadAsync(x);
await LoadAsync(y); }
Task f()
{ var t1 = LoadAsync(x);
var t2 = LoadAsync(y); await Task.WhenAll(t1, t2);

12. 3. Async Performance and CodeGen
ASP experiment: server makes backend query to get RSS, parses it, and returns a page with the feed’s title.

13. 3. Async Performance and CodeGen
Measure perf first. Only if perf is a problem, here are ideas…
The pattern “return await <expr>” is often pointless
async Task f() {
return await LoadAsync(); }
Task f()
return LoadAsync();
You can often hand-optimize the “hot” path to avoid async methods
async Task<int> GetNextAsync() { if (i < buf.Length) return buf[i++]; buf = await LoadBufAsync(); i = 0; return buf[i++]; }
Task<int> GetNextAsync() {
if (i < buf.Length) return Task.FromResult( buf[i++]);
return GetNextAsyncInternal(); }
async Task<int> GetNextAsync() { buf = await LoadBufAsync(); i = 0; return buf[i++]; }
If await and CPU-bound work share same scope, then factor them out
async Task f()
{ var buf = await LoadAsync(); for (int i=0; i++; i < buf.Length) { buf[i] *= 2; } }
{ var buf = await LoadAsync(); Compute(buf); } void Compute(int[] buf) { for (int i=0; i++; i < buf.Length) { buf[i] *= 2; }
If you don’t need to return to UI thread, use ConfigureAwait(false) to avoid it
async Task f()
{ while (!eof) { await LoadMoreAsync(); } }
{ while (!eof) { await LoadMoreAsync() ConfigureAwait(false); } }

14. Q & A Downloads from this talk
(next week)

15. 4/11/ :17 AM
© 2012 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
© 2010 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.
The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.