1. Swarat Chaudhuri Rice University Biplab Deka Rakesh Kumar UIUC Principled Approximations in Probabilistic Programming

2. The computing stack (approximation) Algorithms Compiler and runtime Architecture The APPROX view: with probabilities and approximations!

3. The computing stack Algorithms Compiler and runtime Architecture The APPROX view: with probabilities and approximations! Probabilistic program Program synthesis Hardware for sampling

4. DPMM Clustering In each iteration, update the cluster assignments of data points one at a time For each data point, compute the probability of it belonging to all existing clusters and an unseen cluster Sample from this probability distribution Gaussian Dirichlet Process Mixture Model (DPMM) Gibbs Sampling Algorithm Cluster 3 Cluster 2 Cluster 1 Cluster 2 Cluster 1 Cluster 2 Cluster 1 New Cluster Model

5. Sampling Probabilities A Distribution 123 4 5 670 5 Call to Sampler A Sample 3 2 7 2 4 Samples Specified Distribution

6. Sampling in Hardware Prefix Sum Comparators PRNG Encoder CLK Probabilities Sample 32b 35b 1b 3b 32b Idea: exploit errors!

7. Robustness to hardware faults Stuck-at Faults Transient Faults Sampler Clustering using DPMM Deka, Biplab. “On Fault Tolerance of Hardware Samplers”. Masters Thesis, University of Illinois at Urbana Champaign, 2014

8. Voltage-Error Rate Tradeoff Sloan, J.; Kesler, D.; Kumar, R.; Rahimi, A., "A numerical optimization-based methodology for application robustification: Transforming applications for error tolerance," Dependable Systems and Networks (DSN), 2010

9. Approximating compilers Traditional compiler: Input: Deterministic program Goal: Executable semantically equivalent to source Method: Syntax-guided translation Approximating compiler: Input: Probabilistic program Goal: –Satisfy basic boolean invariants –Minimize quantitative error Method: Program synthesis

10. Probabilistic programs heightMan = Gaussian(177,64); heightWoman = Gaussian(164,64); assume(heightWoman > heightMan); return heightMan Source: Tutorial on Infer.NET by John Winn and Tom Minka Addition: assertions, angelic nondeterminism More complex example: clustering

11. Probabilistic programming ++ Random variables X: range over distributions Deterministic variables y –Either holes or temporaries Functions f(X 1,..., X k, y 1,..., y k ) –Can map random variables to deterministic ones Expectation, probability Assertions Pareto-optimality goals

12. Example X = Gaussian(??, 10); assume (X < 10); c = Pr(X > 0); assert (c > 0.7); minimize (c); One synthesis algorithm in [CCS14] Based on probabilistic abstract interpretation Hole [CCS14] Chaudhuri, Clochard, Solar-Lezama. Bridging boolean and quantitative synthesis using smoothed proof search. POPL 2014.

13. Use in approximation Holes = degree of approximation Assertions = invariants, hard bounds Declarative error minimization Deterministic temporaries track resource consumption Synthesis = Compilation

14. Thank you! Questions?