Integrating Bigloo into the.NET Platform Séminaire du club INTech Grenoble 15/05/2003

Outline Introduction Scheme, Functional Programming and Bigloo The.NET Framework Motivations for a Bigloo.NET back-end How do we implement such a different language than C#/J#/… in.NET? Separate compilation A different notion of polymorphism Mapping of types and constructs Closures Tail recursion Continuations Inlining Conclusions

Scheme A functional language (strict evaluation, side-effects), derived from L ISP First class data types: Numerals, strings, symbols, pairs, vectors (arrays), lambdas High-order language functions are first-class objects Dynamic type-checking Very powerful macro system + eval at run-time Prefixed notation n-ary functions made coherent (and cond1 cond2 cond3 cond4 …) A standard: R5RS (Revised 5 Report on Scheme)

Functional Programming int fac( int n ) { int result= 1; while (n > 0) { result*= n; --n; } return result; } (define (fac n) (if (zero? n) 1 (* n (fac (- n 1))))) Imperative programming: Functional programming:

Bigloo An extension of Scheme R5RS Separate compilation (modules) Object extensions à la C LOS /Meroon + extensible classes Optional type annotations for compile-time strong typing Fair-Threads … Functional programming often thought to have bad performances Bigloo generates C code whose performance is close to human-written C code Bigloo JVM programs are about 2 times slower than Bigloo C programs An optimizing Scheme compiler to C code, JVM bytecode and now.NET bytecode Support for continuations on the C back-end, only No arbitrary-precision integer arithmetic

The.NET Framework Similar to the Java platform: A stack-based register-less Virtual Machine (the Common Language Runtime) Memory is garbage collected Bytecode is Just-In-Time compiled to native code (can be precompiled in.NET) Microsoft (Commercial + Rotor): de facto reference implementations DotGNU (Portable.NET): interpreter only, missing parts in runtime, many tools 3~4 implementations of.NET so far: Hopes of higher performances: Bytecode is fully linked within assemblies Ximian (Mono): JIT compiler, runtime almost complete, few compilation tools At least, applications start faster: building.NET shell tools is realistic A portable bytecode (Common Intermediate Language), verified then executed Bytecode is systematically JITed: more efficient than looking for hot-spots? At least, more deterministic run times

Motivations for a Bigloo.NET back-end Not a simple mapping of Java constructs to JVM bytecode As for Java:.NET is more language-agnostic Bigloo generates C code whose efficiency is close to human-written C code Can we achieve the same in the.NET Framework? Better interoperability with other languages Ability to use the most suitable language for different parts of an application Can we get a.NET back-end at low cost once we have a JVM back-end? "Write Once, Run Everywhere" promise Big sets of features provided by the runtime libraries Lots of services provided by the VM: RTTI, GC, subtyping polymorphism, ….NET has better performances?

··· Bigloo and its back-ends OS Bigloo C Runtime OS JVM Bigloo JVM Runtime Bigloo JVM App OS CLR Bigloo.NET RT Bigloo CIL App C# App Eiffel# App Bigloo Application Bigloo Compiler Bigloo Runtime Bigloo Runtime Bigloo C App

Compiler and runtime code sizes (lines) Compiler Common code % C back-end3 0005% JVM back-end5 5009%.NET back-end4 5007% Runtime Common code % C runtime % Boehm GC % JVM runtime7 0008%.NET runtime7 5008%

Outline Introduction Scheme, Functional Programming and Bigloo The.NET Framework Motivations for a Bigloo.NET back-end How do we implement such a different language than C#/J#/… in.NET? Separate compilation A different notion of polymorphism Mapping of types and constructs Closures Tail recursion Continuations Inlining Conclusions

Separate compilation A Bigloo application is made of several modules (one per file), a module may define several classes, the graph of module references can be cyclic 2 problems: MS ilasm wants types to be fully-qualified by the module defining the type MS ilasm cant compile to object files no C# and IL mixing Solution: Code is compiled using PNet ilasm, which performs separate compilation Problem: Binaries generated by PNet are not recognized by MS VMs and not signed (yet) Solution: Binaries can be disassembled, reassembled and signed by MS tools Finding the module that defines a type should be the job of the linker The Bigloo.NET runtime is a mix of C# sources and Bigloo generated IL Each module is compiled separately into a.il file

A different notion of polymorphism In C++/Java/C#/Eiffel/…/CLR object systems: Classes encapsulate both data and behaviour (methods), different visibility levels In C LOS object system: Classes only encapsulate data, no visibility concept Behaviour is implemented through generic functions Generic function dispatch based on dynamic type of arguments Bigloo performs single dispatch, i.e. dispatch based on first argument only First idea: use.NET object system All generics must be defined on the root class No more separate compilation Our approach: All Bigloo types are indexed, all instances have a type index Dispatch based on the type index, using compressed vptr tables Very large and sparse vptr tables Method dispatch based on dynamic or static type of objects

Mapping of types and constructs pair integer string symbol int32, possibly boxed byte[] (for mutability) bigloo.symbol bigloo.pair real float64, possibly boxed method module function class (singleton) class static method or inlined loop class method closure bigloo.procedure Boxing is reduced as much as possible thanks to optional type annotation and type inference system (SUA) Unboxed numerals are passed on the stack Boxed numerals are allocated in the heap, and a reference is passed on the stack

Closures used as first-class values (define (inc x) (lambda (y) (+ x y))) First idea: each closure is a singleton instance of a class Too much of a load on the type system (>4000 real closures in the compiler!) Even worse with JVM, where each class is a file! Most of non-escaping closures (not used as values) are simplified by the compiler: Inlined as loops when called as tail-recursive functions Turned into regular functions otherwise (map (inc 5) '( )) ( ) Comparable with Java inner anonymous classes with a single method

Closures Current implementation from the Java back-end, which lacks of function pointers: Each closure is an instance of a bigloo.procedure : Object[] environment; int arity, index; public Object apply( Object argument_list ) { switch( index ) { case 0: call closure body 0 case 1: call closure body 1 … Problems: Cost of virtual function apply Loss of potential type information for environment variables Cost of body dispatching depending on closure index Delegates ( ) could simplify the dispatch.NET possible improvement? Delegate invocation seems to be 2x slower than indexed call…

Tail Recursion In functional programming, iterations are most of the time written as recursions: (define (fac n) (if (zero? n) 1 (* n (fac (- n 1))))) (fac 3) (define (fac n) (define (iter n acc) (if (zero? n) acc (iter (- n 1) (* acc n)))) (iter n 1)) (fac 3) R5RS requirement: tail-calls should not add activation blocks (* 3 ) (fac 2) (* 2 ) (fac 1) (* 1 ) (fac 0) (iter 3 1)(iter 2 3)(iter 1 6)(iter 0 6)6 Tail-recursive n!: Recursive n!: Continuation of fac call No continuation for iter call

Tail Recursion Self-recursive or co-recursive function calls implemented as goto s (define (even? n) (or (zero? n) (odd? (- n 1)))) (define (odd? n) (and (not (zero? n)) (even? (- n 1)))) What about functions called as tail recursion ? In Java: stack consumption In.NET: call instructions can be flagged as tail. Security issue: security grant/denial frames are dropped Probably not a real concern for Bigloo users Significant performance penalty! Tail recursion not feasible in C and Java

Continuations call/cc (call with current continuation) captures the continuation of a computation (+ 3 (call/cc (lambda (k) (* 3 4)))) 15 (define kont '?) (+ 3 (call/cc (lambda (k) (set! kont k) (* (/ 5 (k 8)) 4)))) 1 (kont 10) 13 Ability to restore the stack state save by a call/cc at any time R5RS requirement: continuations are first-class values The stack can only be introspected in.NET! Simplification: continuations can only be called within the call/cc dynamic extent Continuations become simple computation escapes Perfectly handled by exceptions holding the call/cc return value Continuations are generalization of escapes: Abort a computation (exception) + resume a computation (stack restoration)

Inlining Scheme/Bigloo programs use a lot of small functions like cons, car, cdr, … Such functions must be inlined ! As for the JVM JIT compiler, inlining doesnt seem to be performed as expected Bigloo already performs inlining of Bigloo functions What about Bigloo.NET runtime functions? Inlining must be done "by hand", by injecting IL code instead of function calls

Conclusions A new back-end for Bigloo, to embrace the "Common Language Runtime" initiative Enlargement of Functional Programming targets A work still in progress Tail calls not yet investigated Recette succeeds at 100% on MS platforms The Bigloo compiler has succeeded the bootstrap proof of concept Could we handle continuations within the CLR? No benchmark yet available