Presentation is loading. Please wait.

Presentation is loading. Please wait.

Everything you always wanted to know about threading....... BUT WERE AFRAID TO ASK ELLIOT H. OMIYA (EHO) PRINCIPAL SDE WINDOWS DEVELOPER EXPERIENCE.

Similar presentations

Presentation on theme: "Everything you always wanted to know about threading....... BUT WERE AFRAID TO ASK ELLIOT H. OMIYA (EHO) PRINCIPAL SDE WINDOWS DEVELOPER EXPERIENCE."— Presentation transcript:

1 Everything you always wanted to know about threading....... BUT WERE AFRAID TO ASK ELLIOT H. OMIYA (EHO) PRINCIPAL SDE WINDOWS DEVELOPER EXPERIENCE

2 Agenda A short history of threading in Windows Threading in the Windows Runtime Threading and UI programming Async

3 Short History of Threading in Windows Anyone remember _beginthread(ex)?? Going back to VC++ 1.0 and the early to mid-90s (16-bit real mode Windows) Eventually we sorted out CreateThread versus _beginthread issues and interactions with the CRT. Inevitably this led to an explosion of threads in Windows programs. Everyone started hand-rolling their own thread pools until we shipped ThreadPool in Windows NT. Dedicated threads (i.e. CreateThread) are still popular but harder for application to manage.

4 ThreadPools Simple: queue, n threads, and an algorithm to dequeue work and run it. Not simple: the actual implementation Job 1: queue (actually, this part is relatively simple) If you have priorities, one queue per priority level is a fine implementation Job 2: threads First decision: how many threads? One per core?

5 Worker threads for thread pool One per core is fine unless you have: IO Threads that can block on event/semaphore A busy thread (waiting on IO or event) can do no more work Result: you need a pool of threads greater than the # of cores because every worker thread can potentially block

6 Work types Turns out you need work types (in the internal implementation) TP_WORK TP_WAIT TP_IO TP_TIMER etc. Basically these are execution triggers – how they start executing. You could also explicitly set up work items that you know do IO or will wait on an event/semaphore. So the API just speaks of work items, but there is additional complexity in the actual implementation.

7 Thread explosion There are several classic programming problems that can lead to thread explosion. Simple example: web server servicing web requests: n worker threads servicing incoming requests One worker thread to read actual html file and associated resources from disk (into cache) Simple programming error can lead to disastrous results.

8 Work Item Queue Work Item... Worker Threads Work Item Event (web page read) Work Item Every worker thread is blocked on an event

9 Work Item Queue Work Item... Worker Threads Work Item Event (web page read) Work Item But the worker thread that signals the event is in the queue! (Bad code!)

10 Work Item Queue Work Item... Worker Threads Work Item Event (web page read) Work Item So you have to create another thread in the pool...

11 Work Item Queue Work Item... Worker Threads Work Item Event (web page read) Work Item But you could have a large number of requests for the same web page...

12 Work Item Queue Work Item... Worker Threads Work Item Event (web page read) Work Item Result: thread explosion

13 Thread explosion Turns out there are many variations of this problem. Another example: freezing threads for a GC operation We call this programming defect popular internal dependency Solution: create a short (or long) delay – (yes, a heuristic) Algorithm assumes the resource dependency will resolve (i.e. does not cover the programming error we previously described) Alleviates the thread explosion during this dependency resolution

14 Fairness Recap: n priority work queues and a system that does IO and has events/semaphores OS has a highly efficient signaling mechanism: IO completion ports – kernel knows when IO completes. Can also be used for events. So kernel knows immediately when a work thread that was blocked is ready to run (ReadyThread). It can release work to the user mode side of the threadpool when it is idle.

15 Fairness Windows 7: TP_WORK always trumped TP_IO and TP_WAIT So lots of TP_WORK could starve work items that unblocked (i.e. unfair) A fairer algorithm blends TP_WORK, TP_IO, and TP_WAIT. Servicing IRPs is a kernel mode concept so the worker thread factory must be in kernel mode.

16 Kernel User Worker Thread Factory Work IO Wait... Work Item Queue Work Item Work Item Queue Work Item

17 Kernel User Worker Thread Factory Work IO Wait... Work Item Queue Work Item Work Item Queue Work Item IRPs complete

18 Kernel User Worker Thread Factory Work IO Wait... Work Item Queue Work Item Work Item Queue Work Item Transported and running

19 Fairness Fairness could only be implemented with kernel mode managing work items, IO, timers, and event waiters. Bonus: IRPs complete rapidly, we can batch up and do a single transport for multiple work items (fewer ring transitions) Very difficult to balance throughput, resource management, and fairness in user mode only. This is a C++ conference, why do you care?

20 Your unique value add is the diverse types of work you do in ISO C++ On Windows, think of how your library functions can benefit from a highly optimized threadpool. On desktop, you can access CreateThread and the latest TP APIs On WinRT, you only have the threadpool. This is where you throw tomatoes at me.

21 I need dedicated threads I can control... We hear this a lot. Not every workload is amenable to (hopefully short) chunks of work items. I have long running work. Examples: Populate off-screen parts of game board (viewport scrolling optimization) Lazy layout (optimize start-up performance) (insert your favorite scenario here, you all have one or three)

22 Time-sliced The Windows Threadpool does have a dedicated thread model: time- sliced. Trade-off of latency versus throughput (and resource consumption) Using work items is efficient, many work items processed by relatively few threads. (batched mode). Using time-siiced has less latency but burns an expensive resource: a thread. (quantum mode) Plug: This is a summary of Pedro Texeiras excellent talk on the threadpool on MSDN. Many more details there.Pedro Texeiras excellent talk

23 WinRT threadpool Windows::System::Threading::ThreadPool has support for: Run a single work item - RunAsync(WorkItemHandler^) Run a single work item based on timer - CreateTimer(TimerElapsedHandler^, delay) Run a periodic work item based on a timer – CreatePeriodicTimer(TimerElapsedHandler^, period) auto WorkItem = ref new WorkItemHandler( [&](IAsyncAction^ workItem) {..... // background work }); IAsyncAction ^ ThreadPoolWorkItem = Windows::System::Threading::ThreadPool::RunAsync(WorkItem); Eventually, background work is reflected in UI, result(s) need to run on the correct UI thread: use CoreWindow Dispatcher

24 Agile Objects (are your friend)

25 In WinRT objects are objects And threads are threads. Forgive me while I digress into COM for a bit (you will forgive me ). How many people in this audience know what a COM apartment is? (If you do, how do you like dealing with apartments?) Better question: how many people want to know what a COM apartment is?

26 Traditional view of multi-threading Objects can be accessed from any thread The hard part is dealing with multi-thread contention, resource management, locking, deadlock prevention, etc. The hard part is hard enough. You already deal with context on a thread no matter what platform you run on: at a minimum, you deal with the context that is UI. But, you can set up your code so that most objects can run on any thread. And you want multi-threading to be no harder than this.

27 Agile Objects WinRT objects are agile by default. This bit of magic (1) allows WinRT objects to just be multi-threaded objects (as you have traditionally known them). They can run anywhere (hence the term agile) and you have the job of dealing with multi-threaded resource contention. Period. This is the technique that allows you to ignore that there is a thing called an apartment. There are also UI-affine objects in WinRT but you are used to dealing with these on whatever platform(s) you code for. (1)Actually not magic: uses FTM, lousy name, but it saves you from knowing anything about apartments. FTM (2)Deep dive on this from Martyn Lovells talk at build 2013. And our Channel9 at build 2013Channel9 video

28 UI and ASTA

29 UI Threads All mainstream systems bind a single thread to UI operations. Implicitly or explicitly a UI thread has a context into which it is bound. In the 90s we created the notion of apartment which to this date almost no one understands. But just think of it as a context. Very popular other OS says: It is strongly recommended not to update UI controls etc from a background thread (e.g. a timer, comms etc). This can be the cause of crashes which are sometimes very hard to identify. In other words, dont perform UI operations out of context.

30 UI Contexts In this world of many threads, were still stuck with a single thread and UI. It gets worse: turns out rather than multi-threaded races and deadlocks, UI threads can suffer from re-entrancy. Re-entrancy: Im in the middle of doing thing A, why am I all of a sudden doing thing B? Re-entrancy is the cause of a large number of crashes and deadlocks. UI has two models: re-entrancy == surprises, non-re-entrancy == potential deadlocks.

31 How to safely update UI 101 Option 1: Background threads call directly into UI threads. Only safe if you are REALLY careful. (See slide on re-entrancy). Option 2: Background threads post notifications that UI threads process when they are ready. (Dispatcher model). Can UI threads call directly into objects running on background threads? Only safe if you are somewhat careful. Oh, and dont take too long to update UI. What is definition of too long? In the last few years it was all about frame rate. But now, it means be immediately responsive to input.

32 Pretend you have the chance to define a new platform... What choices do you make? One of the first questions: Does your UI model allow re-entrancy? Next: do you allow arbitrary threading? How much synchronous UI update do you allow in your platform? (i.e. how much risk of spinning donuts do you want?)

33 We ended up calling that new platform: WinRT (Windows Runtime) Choice #1: Non-re-entrancy (ASTA) Choice #2: ThreadPool APIs but not arbitrary threading Choice #3: No spinning donuts: async across the API surface Choice #4: No arbitrary window creation (and you dont have to know what an HWND is) We still have single-threaded UI frameworks: HTML/CSS and XAML And there are plenty of things you can do synchronously in these frameworks, e.g. toggle button state or update textbox/listbox/etc. contents.

34 Tying it together Turns out that you have to tie together the following elements for the UI and programming model to work: Window creation and behavior Window event (message) processing Dispatcher processing (remember those UI notifications we need?) Call processing (incoming and outgoing) Input And you have to prioritize these with respect to each other.

35 The entity that ties these together is ASTA ASTA == UI context (forget that one of the As is apartment) One thread per window, windows are created by contract activations (e.g. click on a tile or share something from an app). The main thread of a WinRT application runs in a multi-threaded context but is not a UI thread. i.e. the main thread is not a UI thread. As a result, of course, there is a main UI window / thread. ASTAs are non re-entrant. When an outgoing call is in progress, an incoming call blocks. This requires careful planning, but there are no surprises. (Re-entrancy is nearly always a surprise).

36 Async (no spinning donuts) There has been a concerted effort to make UI responsive, this is key to the fast and fluid platform promise. Every API was (and is) reviewed. Any synchronous call that takes longer than the low 10s of milliseconds to execute must be async. Every async API conforms to a uniform pattern. The language projections (C++, C#/VB.Net, Javascript) all have built- in async support.

37 WinRT Async 101 All WinRT async operations are hot start, as soon as they are produced, they are running. You can attach to a running async operation by supplying a completion handler (put_Completed). When the operation completes, it fires a completion callback. The type of each callback is the result type of the async operation. Of course, when you supply the completion callback, it may fire completion immediately, i.e. the operation completed already.

38 WinRT Async 101 When an async operation completes, its status is reported: Complete Canceled (user request to cancel was honored) Error WinRT async operations are a one-way state machine: AsyncMethod => Running => Terminal State {Completed|Error|Canceled} Processing always occurs on background thread (usually a TP work item) Pattern is completely documented in code: see WRLs async.h

39 Async and Agile Async Operation objects are a good example of agile objects Mostly designed to be called from UI threads But do all of their work on background (TP) threads As a result, async operation objects are agile (and therefore directly callable from UI threads)

40 Async support (C++ / PPL) Language projections are key to productivity in producing WinRT apps. Natural and familiar is a key design point: make the experience natural and familiar for the language that is being used. For C++, the natural means of consuming async is based on the Concurrency Runtime (PPL). The model is continuation-based (.then() + lambda/functor/function) with support for exception handling and cancellation.

41 Async and contexts Consuming async operations is often done for the benefit of UI create_task takes note of the originating context, i.e. what kind of thread made the original call. (use_default).use_default create_task will return the result of an async operation (if any) to that originating context by default. C++ developers of course have control over this (e.g. you may want to de-bounce transitions back to the UI thread until the end of a series of continuations). Well talk about CoreWindows Dispatcher method soon.

42 CoreWindow Dispatcher One of the things that the ProcessEvents loop on a CoreWindow schedules is dispatcher work items. CoreWindowDispatcher::RunAsync() schedules a work item to run on the UI thread associated with the CoreWindow. Since the CoreWindow processing loop processes multiple types of items, a rich priority scheme is supported.

43 CoreWindow Dispatcher Priority Normal priority (default) means: run dispatcher callbacks in FIFO order, cooperative with input (and window management events). (Everyone runs cooperatively). Low priority means: Dispatcher callbacks run when there is no input pending. (Input beats dispatcher, i.e. app is responsive to input). Idle priority means: Dispatcher callbacks run when there is nothing else in any queue. (everything beats dispatcher). High priority means: dispatcher callbacks run ahead of everything else. (dispatcher beats everything). Docs say: dont use this (more on this in a second).

44 Rendering UI This is not your fathers message loop. Everything used to happen on the UI thread: message processing, paint, computation, animation, alpha blending, etc. As applications became more graphically intensive, everything became a slave to frame rate (60 fps == 16.667ms window). Get everything done by deadline. Very difficult model to get right. WinRT render threads are separate from ASTA UI threads. Composition is separate from render. Goodness: relieves pressure on the (precious) UI thread

45 Responsive UI The paint beat is tied to the frame rate. The frameworks (WinJS and xaml) take care of this. Most UI operations are tweaks to layout (change text in a text box, scroll, get image ready to render, etc.). These occur directly on UI thread. Initial layout and large changes to layout (e.g. navigation) are critical and time-sensitive. You have to beat the next vsync. If you cant beat the next vsync (common) then you can create a transition animation while the next layout is being prepared. (independent animations do not run on the UI thread)

46 Responsive UI What does this have to do with threading? CoreWindowDispatcher priority High serves a couple of different system purposes: In WinJS – High priority is used for layout changes. Think: layout beats everything In CoreApplication, the app object in every WinRT application, the suspend notification is delivered on UI threads via the Dispatcher (for apps, responding to Suspend is highly time critical) If you choose to use High priority, remember that it is an extremely sharp knife: highly effective and dangerous when used improperly.

47 Tying it all back together UI threads are created by the system and managed by CoreWindow processing loop. Background threads run via WinRT ThreadPool or language projection components (sitting on WinRT ThreadPool). CoreWindow Dispatcher schedules work on UI threads, with a rich priority scheme. Responsive and useful app UI means balancing: CPU utilization on background threads UI thread processing that is responsive to input and never blocks (including never running long workloads)

48 Which means... Responsive programs are sequences of async processing followed by a return to UI thread either by: Direct context capture (return to point of origination); or Explicit call to CoreWindow Dispatcher

49 Async Investments Microsoft continues to invest heavily and innovate in the async space. C# introduced await keyword, dramatically simplifying async programming. Await allows async consumption code to read in a more logical flow. It looks like synchronous code, but the block of code following an await statement executes later. Think you dont need/want this? Go to Herb Sutters //build 2013 talk, and fast forward to about 50 minutes in.Herb Sutters //build 2013 talk And then make sure you attend Deons talk later today where we tell you Everything You Ever Wanted To Know about C++ await.

50 In Summary... Probably not everything you always wanted to know The valuable assets you have in your C++ code come to light in UI And the threading and coding rules for UI are different than they were 5-10 years ago (more input responsiveness, more async, etc.) Optimize background processing in units of work and think carefully about the relative priorities of that work. (And dont reinvent the threadpool!) You do really want await (dont miss Deons talk).

Download ppt "Everything you always wanted to know about threading....... BUT WERE AFRAID TO ASK ELLIOT H. OMIYA (EHO) PRINCIPAL SDE WINDOWS DEVELOPER EXPERIENCE."

Similar presentations

Ads by Google