Data Flow Analysis for Software Prefetching Linked Data Structures in Java Brendon Cahoon Dept. of Computer Science University of Massachusetts Amherst, MA Kathryn S. McKinley Dept. of Computer Sciences University of Texas at Austin Austin, TX

Motivation Object-oriented languages are mainstream Key performance issues –Same old: processor-memory gap, parallelism  Combination of modern processors and languages results in poor memory performance

RSIM Performance Compiled Java (Vortex) – no GC

Prefetching Arrays vs. Objects Most prior work concentrates on arrays –Compilers directly prefetch any element –Loop transformations enable effective scheduling –Successful results using both hardware and software Cannot use same techniques on linked data structures –Objects are small and disjoint –Access patterns are less regular and predictable –Only know the address of directly connected objects

Software Data Prefetching for Java Hide memory latency from linked structure traversals Introduced by Luk and Mowry for C programs: –We add data flow and interprocedural analysis –Identify pointer structures from declaration –Find pointer chasing in loops and self recursive calls Challenges introduced by Java –Dynamically allocated objects make analysis difficult –Small methods obscure context

Outline Data flow analysis for identifying linked structures –New intra and interprocedural analysis Greedy prefetching Jump-pointer prefetching Experimental results

Identifying Linked Structure Traversals We define a data flow solution: –Intraprocedural for loops –Interprocedural for recursion Benefits: –Independent of program representation –Many compilers use data flow frameworks –May be composed with other analyses Loop while (o != null) { t = o; … o = t.next; } Recursion method visit() { …. if (this.next != null) visit(this.next); }

Data Flow Analysis Data flow information –Sets of tuples: Status values: not recurrent, possibly, recurrent Not recurrent : initial value Possibly : first use of a field reference Recurrent : an object accessed in linked structure traversal Intraproceedural: forward, flow-sensitive, may analysis Interprocedural: bidirectional, context-sensitive

Analysis Examples while (o != null) { s1: t = o.next; s2: o = t; } while (o != null) { s1: o = o.next; s2: o = bar(); } s1: o = o.next; s2: o = o.next; 1 st Iteration s1: o is not recurrent, set t to possibly s2: t is possibly, set o to possibly s1: set o to possibly s2: set o to possibly 1 st Iteration s1: set o to possibly s2: set o to not recurrent 2 nd Iteration s1: o is possibly, set t to recurrent s2: t is recurrent, set o to recurrent

Analysis Extensions for Common Idioms Track objects in fields or arrays –Class based field assignments –Arrays are monolithic Indirect recurrent objects –Unique objects referenced by linked structures while (e.f != null) { o = e.f; e.f = o.next; o.compute(); } while (e.hasMoreElements()) { o = (ObjType)e.nextElement(); o.compute(); }

Greedy Prefetching Prefetch directly connected objects Algorithm consists of two steps: –Detect accesses to linked structures –Schedule prefetches When object is not null Completely hiding latency is difficult

Greedy Prefetching Example Doubly linked list int sum (Dlist l) { int s = 0; while (l != null) { s =+ l.data; l = l.next; } return s; }

Greedy Prefetching Example Doubly linked list Greedy prefetching int sum (Dlist l) { int s = 0; while (l != null) { prefetch(l.next); s += l.data; l = l.next; } return s; }

Jump-Pointer Prefetching Prefetch indirectly connected objects –Tolerates more latency than greedy prefetching Algorithm contains three steps: –Find linked data structure traversal and creation sites –Create jump-pointers When creating or traversing the linked structure –Schedule prefetches Prefetch special jump-pointer field

Inserting Jump-Pointers at Creation Time Void add(ObjType o) { ListNode n = new ListNode(o); jumpObj = jumpQueue[i]; jumpObj.jmp = n; jumpQueue[i++%size] = n; if (head == null) { head = n; } else { tail.next = n; } tail = n; } jumpObj n

Jump-Pointer Prefetching Example int sum (Dlist l) { int s = 0; while (l != null) { prefetch(l.jmp); s += l.data; l = l.next; } return s; } Doubly linked list Jump-pointer prefetching

Experimental Results Object-oriented Olden benchmarks in Java Simulation using RSIM –Out-of-order, superscalar processor Compile programs using Vortex –Translate Java programs to Sparc assembly –Contains object-oriented, traditional optimizations –Linked structure analysis, greedy and jump-pointer prefetching

Prefetching Performance health mst perimtr treeadd bh bisort tsp voronoi em3d power

Prefetch Effectiveness health mst perimtr treeadd bh bisort tsp voronoi em3d power

Static Prefetch Statistics ProgramInterproceduralIntra- Procedural Fields MonoPoly Health815 Mst3 Perimeter98 Treeadd2 BH16810 Bisort44 Tsp614 Voronoi141 Em3d20 Power4

Contributions and Future Work New interprocedural data flow analysis for Java Evaluation of prefetching on Java programs Prefetching hides latency, but Room for improvement Other uses for analysis (work in progress) –Garbage collection: prefetching, object traversal –Prefetching arrays of objects