1. Genetic Programming on Program Traces as an Inference Engine for Probabilistic Languages Vita Batishcheva, Alexey Potapov elokkuu@gmail.com, potapov@aideus.com 2015 AGI @ Berlin

2. Key Idea 2 We believe that cognitive architectures and universal algorithmic intelligence should be united in order to build AGI Probabilistic programming could become a suitable basis for developing such a uniting framework We propose the novel approach to inference in PPLs based on genetic programming (and simulated annealing), which is applied to probabilistic program (computation) traces Program traces are used as a “universal genetic code” for arbitrary generative models, and it is enough to only specify such a model in the form of a probabilistic program to perform evolutionary computations

3. 3 Probabilistic + Genetic programming Probabilistic programming provides the possibility to define generative models declaratively PPLs are very convenient to use Many solutions utilize efficient inference techniques for particular types of generative models Performance of generic inference methods in PPLs can be rather low even for models with a small number of random choices -> more appropriate inference methods are needed (like GP) Wide and easy applicability of inference in PPLs is also desirable by evolutionary computations It is interesting to combine generality of inference over declarative models in Turing-complete PPLs and strength of genetic programming

4. Background: Probabilistic programs 4 Some PPLs extend existing languages Programs typically include calls to (pseudo-) random functions PPLs use an extended set of random functions corresponding to different common distributions Programs in PPLs are treated as generative models defining distributions over possible return values, and their direct evaluation can be interpreted as taking one sample from corresponding distributions Multiple evaluation of a program can be used to estimate an underlying distribution PPLs support programs defining conditional distributions A strict condition can be converted into a soft one or even can initially have a task with a goal to optimize some function

5. Implemented Language 5 Since no language supports flexible enough external control of evaluation process, it was easier for us to reproduce (using Scheme as the host language) some basic functionality of Church Our implementation consists of - some basic functions (+, -, *, /, and, or, not, list, car, cdr, cons, etc.), - several random functions (flip, random_integer, gaussian, multinomial), - declaration of variables and functions (define, let), - function calls with recursion and -“quote” and “eval” functions - optimization queries

6. (define x (gaussian 0 10)) x (* (- x 3.7) (- x 3.7)) (evolve (init-pop prg 100) 100 0.03 100) —> x = 3.6959 6 Implemented Language Example

7. (define xs '(-831 253 -307 444 -533 412 393 804 125 -83 71 774 322 -543 -504 -254 613 -884 723 234 -333 234 764 645 345 -23 87)) (define (summ xs ws) (if (null? xs) 0 (+ (if (car ws) (car xs) 0) (summ (cdr xs) (cdr ws))))) (define (gen-ws n) (if (equal? n 0) '() (cons (flip) (gen-ws (- n 1))))) (define ws (gen-ws (length xs))) ws (abs (- (summ xs ws) 1)) (evolve (init-pop prg 100) 100 0.03 100) —> ws = '(#t #t #t #f #f #t #t #f #f #t #f #f #f #t #f #f #f #t #t #t #t #f #f #t #t #t #f) 7

8. (define (f) (if (flip) ‘() (cons (flip) (f))))) 8 Genetic Operators For Computation Traces: Mutations 1 (flip) = #f (flip) = #t (flip) = #f (flip) = #t ‘(#t #f) 2 (flip) = #f (flip) = #t ‘(#t) mutation The program terminates when the green flip value results in #t. The result – list of blue values – is a list of random #t and #f of random length.

9. (define (tree) (if (flip 0.7) (random-integer 10) (list (tree) (tree)))) 9 Crossover '(6 9) '(8 0) '(7 6) '(8 (6 2)) 3 '(8 (6 2)) '(7 9) '((0 7) (7 4)) '(7 (7 4)) '((3 (7 (1 7))) 5) '((5 2) 2) '((4 (7 (1 7))) 2)

10. Task RMSE mh-query annealing- query evolution- query 4x 2 +3x xs=(0 1 2 3) 1.710.2170.035 4x 2 +3x xs = (0 0.1 0.2 0.3 0.4 0.5) 0.940.4250.010 0.5x 3 –x xs = (0 0.1 0.2 0.3 0.4 0.5) 0.4670.1690.007 10 Empirical Evaluation: Curve Fitting TABLE I: AVERAGE RMSE

11. ys Correct answers, % mh-query annealing- query evolution- query '(0 1 2 3 4 5)90%100% '(0 1 4 9 16 25)20%100% '(1 2 5 10 17 26)10%70%80% '(1 4 9 16 25 36)0%90%80% '(1 3 11 31 69 131)0%90%60% 11 Integer Number Sequence Prediction TABLE II: PERCENTAGE OF CORRECT SOLUTIONS

12. Conclusion 12 Methods of simulated annealing and genetic programming over probabilistic program traces were developed In spite of simplicity of the used meta-heuristic search methods, they outperformed the standard mh-query Other types of genetic operators are to be implemented Possibly, one general inference method cannot be efficient in all problem domains, so it should be automatically specialized Optimization queries can be useful to extend semantics of PPLs

13. Thank you for attention Vita Batishcheva, Alexey Potapov elokkuu@gmail.com, potapov@aideus.com 2015 AGI @ Berlin