Swarat Chaudhuri Rice University Biplab Deka Rakesh Kumar UIUC Principled Approximations in Probabilistic Programming.

Slides:

Advertisements

Similar presentations

Program Verification using Probabilistic Techniques Sumit Gulwani Microsoft Research Invited Talk: VSTTE Workshop August 2006 Joint work with George Necula.

Advertisements

Program Analysis using Random Interpretation Sumit Gulwani UC-Berkeley March 2005.

Slide 1 of 18 Uncertainty Representation and Reasoning with MEBN/PR-OWL Kathryn Blackmond Laskey Paulo C. G. da Costa The Volgenau School of Information.

Scaling Up Graphical Model Inference

Gated Graphs and Causal Inference

Less is More Probabilistic Model for Retrieving Fewer Relevant Docuemtns Harr Chen and David R. Karger MIT CSAIL SIGIR2006 4/30/2007.

A Tutorial on Learning with Bayesian Networks

Probabilistic models Jouni Tuomisto THL. Outline Deterministic models with probabilistic parameters Hierarchical Bayesian models Bayesian belief nets.

Variational Methods for Graphical Models Micheal I. Jordan Zoubin Ghahramani Tommi S. Jaakkola Lawrence K. Saul Presented by: Afsaneh Shirazi.

Programming Abstractions for Approximate Computing Michael Carbin with Sasa Misailovic, Hank Hoffmann, Deokhwan Kim, Stelios Sidiroglou, Martin Rinard.

Fast Bayesian Matching Pursuit Presenter: Changchun Zhang ECE / CMR Tennessee Technological University November 12, 2010 Reading Group (Authors: Philip.

CS420 lecture one Problems, algorithms, decidability, tractability.

Smooth Interpretation Swarat Chaudhuri and Armando Solar-Lezama.

Machine Learning Week 2 Lecture 1.

Probability theory and average-case complexity. Review of probability theory.

Randomized Accuracy Aware Program Transformations for Efficient Approximate Computations Sasa Misailovic Joint work with Zeyuan Allen ZhuJonathan KelnerMartin.

Probabilistic Graph and Hypergraph Matching

CS 355 – Programming Languages

Part IV: Monte Carlo and nonparametric Bayes. Outline Monte Carlo methods Nonparametric Bayesian models.

DATAFLOW PROCESS NETWORKS Edward A. Lee Thomas M. Parks.

Discovering Affine Equalities Using Random Interpretation Sumit Gulwani George Necula EECS Department University of California, Berkeley.

Modeling Pixel Process with Scale Invariant Local Patterns for Background Subtraction in Complex Scenes (CVPR’10) Shengcai Liao, Guoying Zhao, Vili Kellokumpu,

Tracking with Online Appearance Model Bohyung Han

Complexity 19-1 Complexity Andrei Bulatov More Probabilistic Algorithms.

EECS 598: Background in Theory of Computation Igor L. Markov and John P. Hayes

Maximum Likelihood (ML), Expectation Maximization (EM)

Weikang Qian The Synthesis of Stochastic Logic to Perform Multivariate Polynomial Arithmetic Abstract Ph.D. Student, University of Minnesota Marc D. Riedel.

Computer vision: models, learning and inference Chapter 10 Graphical Models.

Chapter 5. Operations on Multiple R. V.'s 1 Chapter 5. Operations on Multiple Random Variables 0. Introduction 1. Expected Value of a Function of Random.

. PGM 2002/3 – Tirgul6 Approximate Inference: Sampling.

Binary Variables (1) Coin flipping: heads=1, tails=0 Bernoulli Distribution.

Complexity Theory Lecture 2 Lecturer: Moni Naor. Recap of last week Computational Complexity Theory: What, Why and How Overview: Turing Machines, Church-Turing.

Correlation testing for affine invariant properties on Shachar Lovett Institute for Advanced Study Joint with Hamed Hatami (McGill)

Topic Models in Text Processing IR Group Meeting Presented by Qiaozhu Mei.

Probabilistic Reasoning for Robust Plan Execution Steve Schaffer, Brad Clement, Steve Chien Artificial Intelligence.

Stochastic DAG Scheduling using Monte Carlo Approach Heterogeneous Computing Workshop (at IPDPS) 2012 Extended version: Elsevier JPDC (accepted July 2013,

Direct Message Passing for Hybrid Bayesian Networks Wei Sun, PhD Assistant Research Professor SFL, C4I Center, SEOR Dept. George Mason University, 2009.

Weikang Qian. Outline Intersection Pattern and the Problem Motivation Solution 2.

4.1 Probability Distributions NOTES Coach Bridges.

Mixture of Gaussians This is a probability distribution for random variables or N-D vectors such as… –intensity of an object in a gray scale image –color.

Semantics In Text: Chapter 3.

Other NN Models Reinforcement learning (RL)

Dr. Sudharman K. Jayaweera and Amila Kariyapperuma ECE Department University of New Mexico Ankur Sharma Department of ECE Indian Institute of Technology,

Non-Approximability Results. Summary -Gap technique -Examples: MINIMUM GRAPH COLORING, MINIMUM TSP, MINIMUM BIN PACKING -The PCP theorem -Application:

Random Interpretation Sumit Gulwani UC-Berkeley. 1 Program Analysis Applications in all aspects of software development, e.g. Program correctness Compiler.

CS246 Latent Dirichlet Analysis. LSI  LSI uses SVD to find the best rank-K approximation  The result is difficult to interpret especially with negative.

1/17/2016CS225B Kurt Konolige Probabilistic Models of Sensing and Movement Move to probability models of sensing and movement Project 2 is about complex.

This Week Lecture on relational semantics Exercises on logic and relations Labs on using Isabelle to do proofs.

Additional NN Models Reinforcement learning (RL) Basic ideas: –Supervised learning: (delta rule, BP) Samples (x, f(x)) to learn function f(.) precise error.

CS Statistical Machine learning Lecture 25 Yuan (Alan) Qi Purdue CS Nov

Probability theory and average-case complexity. Review of probability theory.

A Collapsed Variational Bayesian Inference Algorithm for Latent Dirichlet Allocation Yee W. Teh, David Newman and Max Welling Published on NIPS 2006 Discussion.

Genetic Algorithm. Outline Motivation Genetic algorithms An illustrative example Hypothesis space search.

PROBABILITY AND COMPUTING RANDOMIZED ALGORITHMS AND PROBABILISTIC ANALYSIS CHAPTER 1 IWAMA and ITO Lab. M1 Sakaidani Hikaru 1.

Learning Deep Generative Models by Ruslan Salakhutdinov

SOFT COMPUTING.

Haim Kaplan and Uri Zwick

Compiler Lecture 1 CS510.

Clustering (3) Center-based algorithms Fuzzy k-means

Adaptation Behavior of Pipelined Adaptive Filters

Latent Variables, Mixture Models and EM

Latent Dirichlet Analysis

Towards Next Generation Panel at SAINT 2002

Akio Utsugi National Institute of Bioscience and Human-technology,

Synthesis via Numerical optimization

Transformation-invariant clustering using the EM algorithm

Bayesian Inference for Mixture Language Models

CS246: Latent Dirichlet Analysis

Junghoo “John” Cho UCLA

Topic Models in Text Processing

Presentation transcript:

Swarat Chaudhuri Rice University Biplab Deka Rakesh Kumar UIUC Principled Approximations in Probabilistic Programming

The computing stack (approximation) Algorithms Compiler and runtime Architecture The APPROX view: with probabilities and approximations!

The computing stack Algorithms Compiler and runtime Architecture The APPROX view: with probabilities and approximations! Probabilistic program Program synthesis Hardware for sampling

DPMM Clustering In each iteration, update the cluster assignments of data points one at a time For each data point, compute the probability of it belonging to all existing clusters and an unseen cluster Sample from this probability distribution Gaussian Dirichlet Process Mixture Model (DPMM) Gibbs Sampling Algorithm Cluster 3 Cluster 2 Cluster 1 Cluster 2 Cluster 1 Cluster 2 Cluster 1 New Cluster Model

Sampling Probabilities A Distribution Call to Sampler A Sample Samples Specified Distribution

Sampling in Hardware Prefix Sum Comparators PRNG Encoder CLK Probabilities Sample 32b 35b 1b 3b 32b Idea: exploit errors!

Robustness to hardware faults Stuck-at Faults Transient Faults Sampler Clustering using DPMM Deka, Biplab. “On Fault Tolerance of Hardware Samplers”. Masters Thesis, University of Illinois at Urbana Champaign, 2014

Voltage-Error Rate Tradeoff Sloan, J.; Kesler, D.; Kumar, R.; Rahimi, A., "A numerical optimization-based methodology for application robustification: Transforming applications for error tolerance," Dependable Systems and Networks (DSN), 2010

Approximating compilers Traditional compiler: Input: Deterministic program Goal: Executable semantically equivalent to source Method: Syntax-guided translation Approximating compiler: Input: Probabilistic program Goal: –Satisfy basic boolean invariants –Minimize quantitative error Method: Program synthesis

Probabilistic programs heightMan = Gaussian(177,64); heightWoman = Gaussian(164,64); assume(heightWoman > heightMan); return heightMan Source: Tutorial on Infer.NET by John Winn and Tom Minka Addition: assertions, angelic nondeterminism More complex example: clustering

Probabilistic programming ++ Random variables X: range over distributions Deterministic variables y –Either holes or temporaries Functions f(X 1,..., X k, y 1,..., y k ) –Can map random variables to deterministic ones Expectation, probability Assertions Pareto-optimality goals

Example X = Gaussian(??, 10); assume (X < 10); c = Pr(X > 0); assert (c > 0.7); minimize (c); One synthesis algorithm in [CCS14] Based on probabilistic abstract interpretation Hole [CCS14] Chaudhuri, Clochard, Solar-Lezama. Bridging boolean and quantitative synthesis using smoothed proof search. POPL 2014.

Use in approximation Holes = degree of approximation Assertions = invariants, hard bounds Declarative error minimization Deterministic temporaries track resource consumption Synthesis = Compilation

Thank you! Questions?