1. Randomized Accuracy Aware Program Transformations for Efficient Approximate Computations Sasa Misailovic Joint work with Zeyuan Allen ZhuJonathan KelnerMartin Rinard MIT CSAIL

2. … … … … … Nodes represent computation Edges represent flow of data

3. … Functions – process individual data Reduction nodes – aggregate data … … … …

4. … min avg Functions – process individual data Reduction nodes – aggregate data … … … …

5. … min avg f2f2 f3f3 f1f1 … … … …

6. … min avg … … … …

7. Sampling inputs of reduction nodes Reductions consume fewer inputs … min avg … … … …

8. … min avg Sampling inputs of reduction nodes Reductions consume fewer inputs … … … …

9. Tradeoff Space Error Time

10. Tradeoff Space Error Time

11. Optimal Tradeoff Curve Error Time Using the tradeoff curve: Minimize time subject to error bound Minimize error subject to time bound

12. Our Result AnalysisOptimization Original program Error bound Optimized program Transformations

13. Outline Model of Computation Tradeoff Curve Construction Optimized Program Selection Related Work

14. Model of Computation … ff t uv w t uv w ffgg gg … … … … min avg

15. Model of Computation … ff t uv w t uv w ffgg gg … … … … min avg min 1 n n n f g t uv w avg mm

16. Structure of Computation Computation nodes DAGs of functions Functions: arbitrary code Process individual inputs Reduction nodes Aggregation functions Average, min, max, sum… Computation Tree Computation nodes and reduction nodes min 1 n n n f g t uv w avg mm

17. min Accuracy-Aware Transformations 1 n n n avg mm

18. min Accuracy-Aware Transformations 1 n n n avg mm …

19. min Accuracy-Aware Transformations 1 n n n avg mm …

20. min Accuracy-Aware Transformations 1 n n n avg mm …

21. min Program Configuration Vector Defines transformed program Functions: probability of executing each version Reductions: number of elements to sample 1 n n n avg mm

22. min Configuration Vector Specifies program version Functions: probability of executing each version Reductions: number of elements to sample 1 n n n avg mm Find optimal program = Find configuration vector that achieves optimal accuracy vs. performance tradeoff Find optimal program = Find configuration vector that achieves optimal accuracy vs. performance tradeoff

23. min 1 n n n avg mm Tradeoff Curve Construction: Algorithm

24. min Tradeoff Curve Construction: Algorithm 1 n n n avg mm

25. Tradeoff Curve Construction: Algorithm 1 mm n n n min avg

26. Tradeoff Curve Construction: Algorithm 1 mm n n n min avg

27. Tradeoff Curve Construction: Algorithm 1 n n n min avg m

28. Tradeoff Curve Construction: Algorithm 1 n n n min

29. Tradeoff Curve Construction: Algorithm 1 n n n min

30. Tradeoff Curve Construction: Algorithm 1 n n n min

31. Tradeoff Curve Construction: Algorithm 1 n min

32. Tradeoff Curve Construction: Algorithm

33. Computation Node Optimization (E 2,T 2 ) (E 0,T 0 )(E 1,T 1 ) Linear program

34. Computation Node Optimization (E 2,T 2 ) (E 0,T 0 )(E 1,T 1 ) Linear program

35. Computation Node Optimization (E 2,T 2 ) (E 0,T 0 )(E 1,T 1 ) Linear program

36. Computation Node Optimization (E 2,T 2 ) (E 0,T 0 )(E 1,T 1 ) Linear program

37. The Algorithm: Reduction Nodes m avg

38. The Algorithm: Reduction Nodes m avg

39. The Algorithm: Reduction Nodes m avg

40. The Algorithm: Reduction Nodes m avg

41. Approximate Tradeoff Curve Error Time Bidimensional Discretization

42. Approximating Tradeoff Curve Error Time Bidimensional Discretization

43. Approximating Tradeoff Curve Error Time

44. Approximating Tradeoff Curve Error Time

45. Approximating Tradeoff Curve Error Time

46. Approximating Tradeoff Curve Error Time

47. Properties of the Algorithm

48. Obtaining Optimized Programs

49. Related Work Accuracy-aware transformations Empirical justification: training/test input set [Rinard ICS ‘06, Rinard OOPSLA ’07, Ansel et al. PLDI ‘09, Misailovic et al. ICSE ’10, Baek & Chilimbi PLDI ‘10 Hoffmann et al. ASPLOS ‘11, Sidiroglou et al. FSE ‘11] Probabilistic accuracy analysis for loop perforation [Misailovic et al. SAS ‘11, Chaudhuri et al. FSE ‘11]

50. Related Work Accuracy-aware transformations Empirical justification: training/test input set [Rinard ICS ‘06, Rinard OOPSLA ’07, Ansel et al. PLDI ‘09, Misailovic et al. ICSE ’10, Baek & Chilimbi PLDI ‘10 Hoffmann et al. ASPLOS ‘11, Sidiroglou et al. FSE ‘11] Probabilistic accuracy analysis for loop perforation [Misailovic et al. SAS ‘11, Chaudhuri et al. FSE ‘11] Ensuring safety of transformed programs Separating critical and approximate parts of program [Carbin & Rinard ISSTA ’10, Sampson et al. PLDI ’11] Verifying relaxed semantics of programs [Carbin et al. CSAIL-TR ‘11] Analytic properties of programs [Majumdar & Saha RTSS ‘09, Chaudhuri et al. POPL’10, Ivancic et al. MEMOCODE ‘10, Reed & Pierce ICFP ’10, Chaudhuri & Solar-Lezama PLDI ’10, Chaudhuri et al. FSE ’11]

51. Summary Model of Computation Accuracy-aware program transformations Effects on overall accuracy and execution time Explore and Exploit Optimal Tradeoffs Approximate optimal tradeoff curve construction Polynomial, dynamic programming algorithm Randomized program configurations to achieve tradeoffs Envisioned Applications Image and video processing, numerical algorithms, queries on big data sets, machine learning, … Optimization, fault tolerance, dynamic adaptation