Improving Rotor for Dynamically Typed Languages Fabio Mascarenhas and Roberto Ierusalimschy

PUC-Rio Improving Rotor for Dynamically Typed Languages Introduction 1 st RFP: Compiling Lua to the CLR Good performance, integration with rest of the CLR Issues found Inefficient coroutines Wrong weak map semantics Heavy codegen CLR 2.0 has lightweight codegen, but other issues remain

PUC-Rio Improving Rotor for Dynamically Typed Languages Coroutines Lightweight threads: no concurrent execution, explicit context switching Compared to full threads: easier to use (no race conditions to worry about), better performance Applications: producer/consumer, generators, iterators, event-based programming, cooperative multitasking Full coroutines: first-class, can switch context inside functions called by the main coroutine function

PUC-Rio Improving Rotor for Dynamically Typed Languages Symmetric x Asymmetric Symmetric coroutines: transfer operation switches to another coroutine Asymmetric coroutines: resume operation switches to a coroutine, yield switches back You can implement symmetric coroutines with asymmetric, and vice versa We decided to have asymmetric coroutines as the basic abstraction

PUC-Rio Improving Rotor for Dynamically Typed Languages Using Coroutines Creating Coroutine c = new Coroutine(obj.CoBody); Resuming (or starting) retval = c.Resume(obj); Yielding retval = Coroutine.Yield(obj); Coroutine body is a method that receives an object and returns an object Argument to Resume is passed to the body when the coroutine is started, and returned from Yield when resuming Argument to Yield is returned from Resume when yielding Uncaught exceptions propagate up the yield chain

PUC-Rio Improving Rotor for Dynamically Typed Languages Example – Tree traversal static object InOrder(object o) { if(o) { Node n = (Node)o; InOrder(n.Left); Coroutine.Yield(n.Key); InOrder(n.Right); } object o; Coroutine c = new Coroutine(Tree.InOrder); while(o = c.Resume(tree)) Console.WriteLine(o);

PUC-Rio Improving Rotor for Dynamically Typed Languages Implementation Issues Each coroutine needs its own stack Allocating and switching among these stacks: fiber API on Windows, ucontext functions on BSD/Linux/MacOS Coroutines x TLS Create a logical thread (CLR thread without an OS thread) for each coroutine? – heavy, coroutines are not threads, this is a hack Add a Coroutine native class to the runtime, to store coroutine- specific state, and modify the Thread class and the threading subsystem to work with it Propagating uncaught exceptions Enclose the coroutine body in a try/catch block and rethrow the exception inside Resume Implementation just started

PUC-Rio Improving Rotor for Dynamically Typed Languages Flexible GC Semantics You cant reproduce the semantics of the GC of some languages in the CLR Lua and its weak maps and reference queues Java and its phantom references Python and its weak maps and per-instance finalizers Weak maps can be used to implement property tables Phantom references/per-instance finalizers useful if you want to know when an object is unreachable but do not want to implement Finalize (and pay the associated cost)

PUC-Rio Improving Rotor for Dynamically Typed Languages Extending the Rotor GC Add per-instance callbacks Unreacheability callback without resurrection »Collect object after running callback Unreacheability callback with resurrection »Same as class finalizer Post-collection callback Reference queues and weak maps implemented with callbacks Transparent weak references?

PUC-Rio Improving Rotor for Dynamically Typed Languages Final Remarks Results still months away Changes for working with Rotor 2 should be minimal Update the Lua compiler to take advantage of the new features Contact: