1. Pointer and Escape Analysis for (Multithreaded) Programs Martin Rinard MIT Laboratory for Computer Science

2. Traditional Escape Analysis Procedures Dynamically allocated objects Does an object escape the allocating procedure? Is the lifetime of object contained in lifetime of procedure? If so, can stack allocate object Sequential programs

3. Modern Escape Analysis Region of Program Procedure Thread Group of Threads Multiple-Entry Component Dynamically allocated objects Is object “captured” within region?

4. Uses of Escape Information Negative Interaction Information Past: Traditional Compiler Optimizations Stack Allocation Synchronization Elimination Variety of Dynamic Check Eliminations

5. Foundation for Interaction Analyses Systems built from groups of reconfigurable components Important to understand interactions Safety of composition Transform to enhance fast reconfiguration, fault-tolerance, predictability, performance Escape analysis focuses interaction analyses Eliminates host of potential interactions Cuts away large parts of program Enables use of more powerful interaction analyses

6. Importance of Interaction Analysis Need a global information about properties of potential component compositions Interaction patterns Inter-component dependences Theme: see across component boundaries Payoff Information about behavior of potential composed systems Customized implementations Functionality-improving transformations Reduce vulnerability to failures Improve adaptation response times

7. Key Requirements Rapid reconfiguration, adaptability, customization Huge range of potential customized combinations Envisioned large-scale transformations impractical to perform manually Need to automate interaction analysis and subsequent transformations Distributed systems inherently concurrent Analyze multithreaded programs Characterize and exploit interactions between threads

8. Outline Combined Pointer and Escape Analysis Sequential Programs Multithreaded Programs Implementation in Flex Compiler Experimental Results

9. Points-to Escape Graph in Example vectorelementData [ ] this enum e dotted = outside solid = inside void computeMax() { int max = 0; Enumeration enum = database.elements(); while (enum.hasMoreElements()) { Employee e = enum.nextElement(); if (max < e.salary()) { max = e.salary(); highestPaid = e; } } database highestPaid

10. Definitions: node types N I = inside nodes represent objects created within the computation of the method one inside node for each object creation site; represents all objects created at site N O = outside nodes represent objects created outside of the computation of the method

11. Definitions: outside node types N P = parameter nodes represent objects passed as incoming parameters N L = load nodes one load node for each load statement in method represents objects loaded from an escaped node N CL = class nodes node from which static variables are accessed

12. Points-to Escape Graph in Example vectorelementData [ ] this enum e dotted = outside solid = inside void computeMax() { int max = 0; Enumeration enum = database.elements(); while (enum.hasMoreElements()) { Employee e = enum.nextElement(); if (max < e.salary()) { max = e.salary(); highestPaid = e; } } database highestPaid

13. Points-to Escape Graph in Example red = escaped white = captured void computeMax() { int max = 0; Enumeration enum = database.elements(); while (enum.hasMoreElements()) { Employee e = enum.nextElement(); if (max < e.salary()) { max = e.salary(); highestPaid = e; } } dotted = outside solid = inside vectorelementData [ ] this enum e database highestPaid

14. Escaped nodes parameter nodes class nodes thread nodes nodes in return set nodes reachable from other escaped nodes captured is the opposite of escaped

15. Dataflow Analysis Computes a points-to escape graph for each program point Points-to escape graph is a triple I - set of inside edges O - set of outside edges e - escape function

16. Dataflow Analysis Initial state: I :formals point to parameter nodes, classes point to class nodes O: Ø Transfer functions: I´ = (I – Kill I ) U Gen I O´ = O U Gen O Confluence operator is U

17. Intraprocedural Analysis Must define transfer functions for: copy statement l = v load statement l 1 = l 2.f store statement l 1.f = l 2 return statement return l object creation site l = new cl method invocation l = l 0.op(l 1 …l k )

18. copy statement l = v Kill I = edges(I, l ) Gen I = { l } × succ(I, v ) I´ = (I – Kill I )  Gen I l v Existing edges

19. copy statement l = v Kill I = edges(I, l ) Gen I = { l } × succ(I, v ) I´ = (I – Kill I )  Gen I Generated edges l v

20. load statement l 1 = l 2.f S E = {n 2  succ(I, l 2 ). escaped(n 2 )} S I =  {succ(I, n 2,.f ). n 2  succ(I, l 2 )} case 1: l 2 does not point to an escaped node (S E = Ø ) Kill I = edges(I, l 1 ) Gen I = { l 1 } × S I l1l1 l2l2 Existing edges f

21. load statement l 1 = l 2.f S E = {n 2  succ(I, l 2 ). escaped(n 2 )} S I =  {succ(I, n 2,.f ). n 2  succ(I, l 2 )} case 1: l 2 does not point to an escaped node (S E = Ø ) Kill I = edges(I, l 1 ) Gen I = { l 1 } × S I Generated edges l1l1 l2l2 f

22. load statement l 1 = l 2.f case 2: l 2 does point to an escaped node (S E  Ø ) Kill I = edges(I, l 1 ) Gen I = { l 1 } × (S I  {n}) Gen O = (S E × {f}) × {n} l1l1 l2l2 Existing edges

23. load statement l 1 = l 2.f case 2: l 2 does point to an escaped node (S E  Ø ) Kill I = edges(I, l 1 ) Gen I = { l 1 } × (S I  {n}) Gen O = (S E × {f}) × {n} Generated edges l1l1 l2l2 f

24. store statement l 1.f = l 2 Gen I = (succ(I, l 1 ) × { f }) × succ(I, l 2 ) I´ = I  Gen I l2l2 Existing edges l1l1

25. store statement l 1.f = l 2 Gen I = (succ(I, l 1 ) × { f }) × succ(I, l 2 ) I´ = I  Gen I Generated edges l2l2 l1l1 f

26. object creation site l = new cl Kill I = edges(I, l ) Gen I = { } l Existing edges

27. object creation site l = new cl Kill I = edges(I, l ) Gen I = { } Generated edges l

28. Method call Transfer function for method call: Take points-to escape graph before the call site Retrieve the points-to escape graph from analysis of callee Map callee graph into caller graph Result is the points-to escape graph after the call site

29. Interprocedural Mapping Set up a mapping between caller and callee outside nodes in the callee may refer to any number of inside nodes in the caller add all reachable inside edges from callee’s graph into caller’s graph outside edges from a node in the callee need to be added to the mapped caller node if it escapes

30. Interprocedural Mapping Example void printStatistics() { BufferedReader r = new BufferedReader( new InputStreamReader(System.in)); EmployeeDatabase e = new EmployeeDatabase(r); e.computeMax(); System.out.println(“max salary = “ + e.highestPaid); }

31. Interprocedural Mapping Example e graph before call site elementData [ ] database void printStatistics() { BufferedReader r = new BufferedReader( new InputStreamReader(System.in)); EmployeeDatabase e = new EmployeeDatabase(r); e.computeMax(); System.out.println(“max salary = “ + e.highestPaid); }

32. Interprocedural Mapping Example callee graph graph before call site Enum object is not present because it was captured in the callee. elementData [ ] this database e elementData [ ] database void printStatistics() { BufferedReader r = new BufferedReader( new InputStreamReader(System.in)); EmployeeDatabase e = new EmployeeDatabase(r); e.computeMax(); System.out.println(“max salary = “ + e.highestPaid); } highestPaid

33. Step 1: Map formals to actuals graph before call site e elementData [ ] database void printStatistics() { BufferedReader r = new BufferedReader( new InputStreamReader(System.in)); EmployeeDatabase e = new EmployeeDatabase(r); e.computeMax(); System.out.println(“max salary = “ + e.highestPaid); } callee graph elementData [ ] this database highestPaid

34. Step 2: Match edges to extend mapping graph before call site e elementData [ ] database void printStatistics() { BufferedReader r = new BufferedReader( new InputStreamReader(System.in)); EmployeeDatabase e = new EmployeeDatabase(r); e.computeMax(); System.out.println(“max salary = “ + e.highestPaid); } callee graph elementData [ ] this database highestPaid

35. graph before call site e elementData [ ] database callee graph elementData [ ] this database highestPaid Step 3: Map nodes and edges to construct new graph graph after call site e elementData [ ] database highestPaid

36. Key Feature Even if an object escapes one method Often possible to recapture object in caller methods Common in practice Iterators Objects that hold multiple return values

37. Life is not so Simple Dependences between phases Mapping best framed as constraint satisfaction problem Solved using constraint satisfaction

38. Algorithm Features Compositional Analyze each method once Obtain parameterized result Reuse result at different call sites Independent preanalysis of libraries Partial Can analyze method without analyzing methods it invokes Useful if not all code available in analyzable form

39. Incrementalized Analysis Compositional + Partial Enables Incrementalization Choose object to attempt to capture Analyze method containing allocation site Track where object escapes Specific call sites Caller Incrementally analyze only those parts Usual Result Significant reduction in analysis time Almost all benefit of full analysis

40. Key Limitation (so far) No analysis of interactions between threads Objects that escape from allocating thread are NEVER recaptured Solution: extend algorithm to analyze interactions between threads Challenge: avoid analyzing all interleavings of statements from parallel threads

41. Interactions Between Threads badc Heap

42. Interactions Between Threads badc Heap White ThreadYellow Thread

43. Interactions Between Threads badc Heap White Thread a.f = b; Yellow Thread

44. Interactions Between Threads badc Heap White Thread a.f = b; t = a.f; Yellow Thread t

45. Interactions Between Threads badc Heap White Thread a.f = b; t = a.f; t.f = c; Yellow Thread t

46. Interactions Between Threads badc Heap White Thread a.f = b; p = b.f; t = a.f; t.f = c; Yellow Thread p

47. Interactions Between Threads badc Heap White Thread a.f = b; p = b.f; p.f = d; t = a.f; t.f = c; Yellow Thread p

48. Interactions Between Threads badc Heap White Thread a.f = b; p = b.f; p.f = d; t = a.f; t.f = c; Yellow Thread

49. Important Properties Result Depends on Specific Interleaving Analyze all Interleavings? Iterate Across Threads to Fixed Point?

50. Important Properties Result Depends on Specific Interleaving Analyze all Interleavings? Iterate Across Threads to Fixed Point? Analyze Each Thread Once Parameterized Analysis Result Characterizes All Potential Interactions Combine Analysis Results From Different Threads to Compute Actual Interactions Compositional Analysis

51. Our Approach Build Points-To Escape Graph Result Characterizes Potential Interactions Inside Edges - Represent References Created By Currently Analyzed Thread Outside Edges - Represent References Created By Parallel Threads Inside Nodes - Represent Objects Created By Current Analyzed Thread Outside Nodes - Represent Objects Accessed Via Outside Edges

52. Analysis of Each Thread t = a.f; t.f = c; Yellow Thread Analysis Result for Yellow Thread Analysis Result for White Thread White Thread a.f = b; p = b.f; p.f = d; ba

53. Analysis of Each Thread t = a.f; t.f = c; Yellow Thread Analysis Result for Yellow Thread Analysis Result for White Thread White Thread a.f = b; p = b.f; p.f = d; ba1 Outside Edge

54. Analysis of Each Thread t = a.f; t.f = c; Yellow Thread Analysis Result for Yellow Thread Analysis Result for White Thread White Thread a.f = b; p = b.f; p.f = d; bad1

55. Analysis of Each Thread t = a.f; t.f = c; Yellow Thread 2a Analysis Result for Yellow Thread Analysis Result for White Thread White Thread a.f = b; p = b.f; p.f = d; bad1

56. Analysis of Each Thread t = a.f; t.f = c; Yellow Thread 2ac Analysis Result for Yellow Thread Analysis Result for White Thread White Thread a.f = b; p = b.f; p.f = d; bad1

57. Combining Analysis Results Match Corresponding Edges Map Outside Nodes to Nodes that They Represent During the Analysis Use Mapping to Combine Graphs 2ac bad1 Analysis Results From Threads

58. Combining Analysis Results Match Corresponding Edges Map Outside Nodes to Nodes that They Represent During the Analysis Use Mapping to Combine Graphs 2ac bad1 Mapping Analysis Results From Threads

59. Combining Analysis Results Match Corresponding Edges Map Outside Nodes to Nodes that They Represent During the Analysis Use Mapping to Combine Graphs 2ac bad1 Mapping Analysis Results From Threads

60. Combining Analysis Results Match Corresponding Edges Map Outside Nodes to Nodes that They Represent During the Analysis Use Mapping to Combine Graphs 2ac bad1 Mapping Analysis Results From Threads

61. Combining Analysis Results Match Corresponding Edges Map Outside Nodes to Nodes that They Represent During the Analysis Use Mapping to Combine Graphs 2ac bad1 Mapping Combined Result Analysis Results From Threads ba

62. Combining Analysis Results Match Corresponding Edges Map Outside Nodes to Nodes that They Represent During the Analysis Use Mapping to Combine Graphs 2ac bad1 Mapping Combined Result Analysis Results From Threads bac

63. Combining Analysis Results Match Corresponding Edges Map Outside Nodes to Nodes that They Represent During the Analysis Use Mapping to Combine Graphs 2ac bad1 Mapping Combined Result Analysis Results From Threads badc

64. Combining Analysis Results Match Corresponding Edges Map Outside Nodes to Nodes that They Represent During the Analysis Use Mapping to Combine Graphs 2ac bad1 Mapping Combined Result Analysis Results From Threads badc 2 1

65. Recapturing Nodes If a, b, c, and d may be recaptured after analyzing interactions between threads Have complete points-to information for recaptured nodes Combined Result After Interthread Analysis badc 2 1 Combined Result After Recapturing Nodes badc

66. Common Usage Pattern Current Thread Creates and Initializes Objects (Synchronization) Passes Objects As Parameters to New Thread Never Accesses Objects Again New Thread Starts Running Accesses Objects (Synchronization)

67. Key Enhancement Instrument Analysis to Record Actions on Nodes synchronization operations reads and writes Ordering Between Actions and Thread Starts Use Ordering to Rule Out Potential Interactions Synchronizations from different threads temporally separated by thread creation events

68. Server Benchmark Characteristics IR Size (instrs) Number of Methods Pre Analysis Time (secs) echo 4,63913128 time 4,57313629 http 10,643292103 phone 9,54726775 Intra Thread Analysis Time (secs) Inter Thread Analysis Time (secs) 74 70 199 191 73 74 269 256

69. Percentage of Eliminated Synchronization Operations 0 20 40 60 80 100 httpphonetimeechomtrt Interprocedural Only Interthread

70. Complications Using Connectivity Information to Define Concepts of Escaped and Captured Nodes Recapturing Nodes After Combining Results Treating Escaped and Captured Nodes Differently During Analysis and Combination Escaped Nodes Can Have Outside Edges Captured Nodes Have No Outside Edges Recursively Generated Threads Accurate Call Graph Modeling Caller/Callee Interaction

71. Application to Real-Time Java Real-Time Java has Scoped Memory (regions) Motivation Keep Java’s safe memory model (no explicit deallocation) Obtain predictable allocation times (no garbage collection) Solution: Preallocate a subregion of memory Task allocates its objects in predictable time from that region Region deallocated as a unit when done

72. Scoped Memory Model Scoped Memory is a separate object Can run a computation in a Scoped Memory (restore old memory when finished) Get a tree of nested computations, each with its Scoped Memory Interaction with Threading New thread can use same scoped memory as parent thread Or can use new scoped memory

73. Nested Scoped Memories Scoped Memory Object

74. Referencing Constraints Scoped Memory Object Referencing Down Scopes Is NOT OK Referencing Up Scopes Is OK

75. Preventing Downward References Reference Checks Done Dynamically At every write of a reference into an object field or array element Check that written object is allocated in a scope below that of referred object If not, throw an exception Drawbacks Dynamic checking overhead Errors when program runs

76. Escape Analysis Escape analysis provides information about how lifetimes of objects relate to computation Use escape analysis to automatically insert scoped memories Use escape analysis results to check programs with explicit scoped memories Check that NO object escapes computation that executes in Scoped Memory If no object escapes, then program will never violate referencing constraints Eliminate dynamic checks Eliminate potential source of errors Analyzing interactions between threads may be crucial (depending on usage patterns)

77. Implementation Status FLEX compiler infrastructure Full Java compiler Lots of utilities and packages Support for deep program analyses and transformations Implemented Scoped Memory checks Implemented combined points-to escape analysis Used analysis results to eliminate scoped memory checks Benchmarks Matrix Multiply (with Integers) Linked List Sum (with Integers)

78. Results Checks No Checks Matrix Multiply 35.631.7 Linked List Sum 4.003.05 Verified that programs do not violate ScopedMemory constraints Execution Times (seconds)

79. Conclusion Combined Points-to and Escape Analysis Compositional, Partial, Incrementalized Implemented Transformations Stack Allocation, Private Heap Allocation, Synchronization Elimination, Scoped Memory Check Elimination Foundation for Interaction Analysis Information about system behavior Customized implementations Functionality-improving transformations Optimized adaptation handling