Intraprocedural Optimizations Jonathan Bachrach MIT AI Lab

Outline Goal: eliminate abstraction overhead using static analysis and program transformation Topics: –Intraprocedural type inference –Static method selection –Specialization and Inlining –Static class prediction –Splitting –Box/unboxing –Common Subexpression Elimination –Overflow and range checks –Partial evaluation revisited Partially based on: Chambers’ “Efficient Implementation of Object- oriented Programming Languages” OOPSLA Tutorial

Running Example (dg + ((x ) (y ) => )) (dm + ((x ) (y ) => ) (%ib (%i+ (%iu x) (%iu y))) (dm + ((x ) (y ) => ) (%fb (%f+ (%fu x) (%fu y))) (dm x2 ((x ) => ) (+ x x)) (dm x2 ((x ) => ) (+ x x)) Anatomy of Pure Proto Arithmetic –Dispatch –Boxing –Overflow checks –Actual instruction C Arithmetic –Actual instruction

Biggest Inefficiencies Method dispatch Method calls Boxing Type checks Overflow and range checks Slot access Object creation

Intraprocedural Type Inference Goal: determine concrete class(es) of each variable and expression Standard data flow analysis through control graph –Propagate bindings b -> { class … } –Sources are literals, isa expressions, results of some primitives, and type declarations –Form unions of bindings at merge points –Narrow sets after typecases –Assumes closed world (or at least final classes)

Type Inference Example (set x (isa …)) ;; x in { } (set y (table-growth-factor x));; y in { } (set z (if t x y));; z in { }

Narrowing Type Precision (if (isa? x ) (+ x 1) (+ x 37.0)) (if (isa? x ) (let (([x ] x)) (+ x 1)) (let (([x ! ] x)) (+ x 37.0)))

Static Method Selection (set x (isa …)) ;; x in { } (set y (table-growth-factor x));; y in { } (print out y) If only one class is statically possible then can perform dispatch statically: (set y ( :table-growth-factor x)) If a couple classes are statically possible then can insert typecase: (sel (class-of y) (( ) ( :print y)) (( ) ( :print y)))

Type Check Removal Type inference can clearly be used to remove type checks and casts (set x (isa …)) ;; x in { } (if (isa? x ) (go) (stop)) ==> (set x (isa …)) ;; x in { } (go)

Intraprocedural Type Inference Critique Pros: –Simple –Fast –Fewer dependents Cons: –Limited type precision No result types Incoming arg types No slot types Etc.

Specialization Q: How can we improve intraprocedural type inference precision? A: Specialization which is the cloning of methods with narrowed argument types Improves type precision of callee by contextualizing body: (dm sqr ((x ) (y )) (* x y)) ==> (dm sqr ((x ) (y )) (* x y)) Must make sure super calls still mean same thing

Specialization of Constructors Crucial to get object creation to be fast Specialization can be used to build custom constructors (def (isa )) (slot thingy-x 0) (slot (t ) thingy-tracker (+ (thingy-x t) 1)) (slot thingy-cache (fab )) (df thingy-isa (x tracker cache) (let ((thingy (clone ))) (unless (== x nul) (set (%slot-value thingy thingy-x) x)) (set (%slot-value thingy thingy-tracker) (if (== tracker nul) (+ (thingy-x p) 1) tracker)))) (set (%slot-value thingy thingy-cache) (if (== cache nul) (fab ) cache))))

Inlining Q: Can we do better? A: Inlining can improve specialization by inserting specialized body Improves type precision at call-site by contextualizing body (includes result types): (dm f ((x ) (y )) (+ (g x y) 1)) (dm g (x y) (+ x y)) ==> (dm f ((x ) (y )) (+ (+ x y) 1))

Synergy: Method Selection + Inlining (df f ((x ) (y )) (+ x y)) ;; method selection (df f ((x ) (y )) ( :+ x y)) ;; inlining (df f ((x ) (y )) (%ib (%i+ (%iu x) (%iu y))))

Pitfalls of Inlining and Specialization Must control inlining and specialization carefully to avoid code bloat Inlining can work merely using syntactic size trying never to increase size over original call Class-centric specialization usually works by copying down inherited methods tightening up self references (harder for multimethods) Can run inlining/specialization trials based on –Final static size –Performance feedback

Class Centric Specialization (def (isa )) (slot (point-x ) 0) (dm point-move ((p ) (offset )) (set (point-x p) (+ (point-x p) offset))) (def (isa )) ==> (dm point-move ((p ) (offset )) (set (point-x p) (+ (point-x p) offset)))

Static Class Prediction Can improve type precision in cases where for a given generic a particular method is much more frequent Insert type check testing prediction –Can narrow type precision along then and else branches Especially useful in combination with inlining

Static Class Prediction Example (df f (x) (let ((y (+ x 1))) (+ y 2))) (df f (x) (let ((y (if (isa? x ) (+ x 1) (+ x 1)))) (if (isa? y ) (+ y 2) (+ y 2))))) (df f (x) (let ((y (if (isa? x ) ( :+ x 1) (+ x 1)))) (if (isa? y ) ( :+ y 2) (+ y 2)))))

Synergy: Class Prediction + Method Selection + Inlining (df f (x) (let ((y (if (isa? x ) (+ x 1) (+ x 1)))) (if (isa? y ) (+ y 2) (+ y 2))))) ;; method selection (df f (x) (let ((y (if (isa? x ) ( :+ x 1) (+ x 1)))) (if (isa? y ) ( :+ y 2) (+ y 2))))) ;; inlining (df f (x) (let ((y (if (isa? x ) (%ib (%i+ (%iu x) %1)) (+ x 1)))) (if (isa? y ) (%ib (%i+ (%iu y) (%iu 2))) (+ y 2)))))

Splitting Problem: Class prediction often leads to a bunch of redundant type tests Solution: Split off whole sections of graph specialized to particular class on variable –Can split off entire loops –Can specialize on other dataflow information

Splitting Example (df f (x) (let ((y (+ x 1))) (+ y 2))) (df f (x) (if (isa? x ) (let ((y (+ x 1))) (+ y 2)) (let ((y (+ x 1))) (+ y 2)))) (df f (x) (if (isa? x ) (let ((y ( :+ x 1))) ( :+ y 2)) (let ((y (+ x 1))) (+ y 2))))

Splitting Downside Splitting can also lead to code bloat Must be intelligent about what to split –A priori knowledge (e.g., integers most frequent) –Actual performance

Box / Unboxing (df + ((x ) (y ) => ) (%ib (%i+ (%iu x) (%iu y)))) (df f ((a ) (b ) => ) (+ (+ a b) a)) ;; inlining + (df f ((a ) (b ) => ) (%ib (%i+ (%iu (%ib (%i+ (%iu a) (%iu b)))) (%iu a)))) ;; remove box/unbox pair (df f ((a ) (b ) => ) (%ib (%i+ (%i+ (%iu a) (%iu b)) (%iu a))))

Synergy: Splitting + Method Selection + Inlining + Box/Unboxing (df f (x) (if (isa? x ) (let ((y (+ x 1))) (+ y 2)) (let ((y (+ x 1))) (+ y 2)))) ;; method selection (df f (x) (if (isa? x ) (let ((y ( :+ x 1))) ( :+ y 2)) (let ((y (+ x 1))) (+ y 2)))) (df f (x) (if (isa? x ) ( :+ ( :+ x 1) 2) (let ((y (+ x 1))) (+ y 2)))) ;; inlining (df f (x) (if (isa? x ) (%ib (i+ (%iu (%ib (%i+ (%iu x) %1)))) %2)) (let ((y (+ x 1))) (+ y 2)))) ;; box/unbox (df f (x) (if (isa? x ) (%ib (%i+ (%i+ (%iu x) %1)) %2)) (let ((y (+ x 1))) (+ y 2))))

Common Subexpression Elimination (CSE) Removes redundant computations –Constant slot or binding access –Stateless/side-effect-free function calls Examples (or (elt (cache x) ‘a) (elt (cache x) ‘b)) ==> (let ((t (cache x))) (or (elt t ‘a) (elt t ‘b)) (if (< i 0) (if (< i 0) (go) (putz)) (dance)) ==> (if (< i 0) (go) (dance))

Overflow and Bounds Checks aka “Moon Challenge” Goal: –Support mathematical integers and bounds checked collection access –Eliminate bounds and overflow checks Strategy: –Assume most integer arithmetic and collection accesses occur in restricted loop context where range can be readily inferred –Perform range analysis to remove checks Bound from above variables by size of collection Bound from below variables by zero Induction step is 1+

Range Check Example (rep (((sum ) 0) ((i ) 0)) (if (< i (len v)) (let ((e (elt v i))) (rep (+ sum e) (+ i 1))) sum)) ;; inlining bounds checks (rep (((sum ) 0) ((i ) 0)) (if (< i (len v)) (let ((e (if (or (< i 0) (>= i (len v))) (sig...) (vref v i)))) (rep (+ sum e) (+ i 1))) sum)) ;; CSE (rep (((sum ) 0) ((i ) 0)) (if (< i (len v)) (let ((e (if (< i 0) (sig...) (vref v i)))) (rep (+ sum e) (+ i 1))) sum)) ;; range analysis (rep (((sum ) 0) ((i ) 0)) (if (< i (len v)) (let ((e (vref v i))) (rep (+ sum e) (+ i 1))) sum))

Overflow Check Removal aka “Moon Challenge” Critique Pros: –simple analysis Cons: –could miss a number of cases but then previous approaches (e.g., box/unbox) could be applied

Advanced topic: Representation Selection Embed objects in others to remove indirections Change object representation over time Use minimum number of bits to represent enums Pack fields in objects

Advanced Topic: Algorithm Selection Goal: compiler determines that one algorithm is more appropriate for given data –Sorted data –Biased data Solution: –Embed statistics gathering in runtime –Add guards to code and split

Rule-based Compilation First millennium compilers were based on special rules for –Method selection –Pattern matching –Oft-used system functions like format Problems –Error prone –Don’t generalize to user code Challenge –Minimize number of rules –Competitive compiler speed –Produce competitive code

Partial Evaluation to the Rescue Holy grail idea: –Optimizations are manifest in code –Do previous optimizations with only p.e. Simplify compiler based on limited moves –Static eval and folding –Inlining Eliminate –Custom method selection –Custom constructor optimization –Etc.

Partial Eval Example (dm format (port msg (args …)) (rep nxt ((I 0) (ai 0)) (when (< I (len msg))) (let ((c (elt msg I))) (if (= c #\%) (seq (print port (elt args ai)) (nxt (+ I 1) (+ ai 1)))) (seq (write port c) (nxt (+ I 1) ai))))))) (format out “%>? ” n) First millennium solution is to have a custom optimizer for format (seq (print port n) (write port “> “)) Second millennium solution with partial evaluation (nxt 0 0) (seq (print port n) (nxt 1 1)) (seq (print port n) (seq (write port #\>) (nxt 2 1))) (seq (print port n) (seq (write port #\>) (seq (write port #\space))))

Partial Eval Challenge Inlining and static eval are slow –“Running” code through inlining –Need to compile oft-used optimizations Residual code is not necessarily efficient –Sometimes algorithmic change is necessary for optimal efficiency Example: method selection uses class numbering and decision tree whereas straightforward code does naïve method sorting Perhaps there is a middle ground

Open Problems Automatic inlining, splitting, and specialization Efficient mathematical integers Constant determination Representation selection Algorithmic selection Efficient partial evaluation Super compiler that runs for days

Reading List Chambers: “Efficient Implementation of Object-oriented Programming Languages” OOPSLA Tutorial Chambers and Ungar: SELF papers Chambers et al.: Vortex papers