1. Using Parallelism to Improve Theorem Prover Interactivity David L. Rager May 17, 2010 Committee Members: Warren A Hunt Jr. (Chair), Matt Kaufmann, J Strother Moore, James C Browne, Emmett Witchel

2. 2 Project Goal  Reduce the latency between when a user submits a conjecture and when the user receives useful feedback concerning that conjecture’s provability

3. 3 Outline  Introduction The Automated Proof Process – the Waterfall The Automated Proof Process – the Waterfall The ways ACL2 currently uses parallelism The ways ACL2 currently uses parallelism  Research Contributions  Key Steps Removing sequential dependencies Removing sequential dependencies Introducing Parallelism into the Proof Process Introducing Parallelism into the Proof Process Adding futures to the underlying implementation languageAdding futures to the underlying implementation language Adding parallelism abstractions to the logicAdding parallelism abstractions to the logic Using parallelism in the waterfallUsing parallelism in the waterfall Managing output Managing output Managing user interrupts Managing user interrupts  Evaluating the Soundness and Performance of our Approach

4. 4 The Proof Process  Named “the waterfall”  We hope to parallelize the application of all but the induction heuristic

5. 5 The Proof Process  Each proof obligation that is not the original goal that needs to go through the waterfall is called a subgoal  We parallelize at the subgoal level because: Highest level of granularity available without considering parallelizing the proofs of theorems themselves Highest level of granularity available without considering parallelizing the proofs of theorems themselves Parallelizing the proofs of goals is thought to be much less usefulParallelizing the proofs of goals is thought to be much less useful Already attempted parallelizing the rewriter without much practical gains Already attempted parallelizing the rewriter without much practical gains The waterfall is mostly functional The waterfall is mostly functional in nature, whereas the code above the waterfall (which includes the induction code) tends to have more side-effects

6. 6 Current Use of Parallelism in ACL2  Process-level parallelism GNU’s “make –j#” and cert.pl GNU’s “make –j#” and cert.pl Certifies the regression suite in parallel Certifies the regression suite in parallel  Plet/pargs/pand/por User level parallelism User level parallelism Useful for improving performance of proofs by simulation Useful for improving performance of proofs by simulation

7. 7 Research Contributions  Maintain Interactivity Continue to support ACL2 users’ ability to use the prover despite parallel execution in its proof process Continue to support ACL2 users’ ability to use the prover despite parallel execution in its proof process  Mechanisms for Early Feedback Provide feedback to the theorem prover user asap Provide feedback to the theorem prover user asap Could result in super-linear speedup Could result in super-linear speedup

8. 8 Research Contributions  Improve Support for Lisp-Level Programming Interface that unifies multi-threading libraries Interface that unifies multi-threading libraries  Provide Parallelism Abstractions Enrich the built-in theory with primitives that allow parallel execution Enrich the built-in theory with primitives that allow parallel execution spec-mv-let, a better plet, etc.spec-mv-let, a better plet, etc.  Evaluate our Approach Determine the usefulness of parallelizing a modern semi-automatic theorem prover at the subgoal level Determine the usefulness of parallelizing a modern semi-automatic theorem prover at the subgoal level SpeedupSpeedup How we present non-deterministic outputHow we present non-deterministic output

9. 9 Key Steps  Create version of ACL2 without sequential dependencies (e.g., the modification of state and pspv) in the main proof process (the waterfall)  Introduce raw Lisp primitives and ACL2 abstractions necessary to evaluate the waterfall in parallel  Reincorporate the interactive portion of the waterfall (output and interrupts)  Evaluate the soundness and performance of our approach

10. 10 Removing Sequential Dependencies from the Waterfall  State-based Challenges State is a special type of variable in ACL2 State is a special type of variable in ACL2 Used when performing I/O, when performing system calls, etc.Used when performing I/O, when performing system calls, etc. Contains the “logical story” for these side-effectsContains the “logical story” for these side-effects ACL2 restricts the use of state in the following two ways ACL2 restricts the use of state in the following two ways ACL2 restricts the name “state” from being used a variable anywhere that it doesn’t represent this one particular instanceACL2 restricts the name “state” from being used a variable anywhere that it doesn’t represent this one particular instance ACL2 requires that if state is modified, that the modified state be returned as part of the return valueACL2 requires that if state is modified, that the modified state be returned as part of the return value So, if we remove the modification of state from the waterfall, we will know that the waterfall is (mostly) side-effect free. So, if we remove the modification of state from the waterfall, we will know that the waterfall is (mostly) side-effect free. Allows us to more easily find the big “gotchas” (e.g., I/O)Allows us to more easily find the big “gotchas” (e.g., I/O)

11. 11 Removing Sequential Dependencies from the Waterfall  State-based Solutions Preliminary step: remove I/O from the waterfall Preliminary step: remove I/O from the waterfall Disable proof techniques that require modifying state (e.g., clause processors and computed hints) Disable proof techniques that require modifying state (e.g., clause processors and computed hints) Skip the proofs of libraries that require those techniquesSkip the proofs of libraries that require those techniques Results in skipping about 7% of the regression suite Results in skipping about 7% of the regression suite Our current thoughts are that we can reinstate most of these techniques as demand occurs Our current thoughts are that we can reinstate most of these techniques as demand occurs

12. 12 Removing Sequential Dependencies from the Waterfall  PSPV-based Challenges The Prover SPecial Variables (PSPV) data container acts as an accumulator for changes to variables that would be global, if ACL2 were written in a non-functional manner. The Prover SPecial Variables (PSPV) data container acts as an accumulator for changes to variables that would be global, if ACL2 were written in a non-functional manner. Need to find a way to combine these changes that does not negatively affect the soundness or performance of ACL2 Need to find a way to combine these changes that does not negatively affect the soundness or performance of ACL2

13. 13 Removing Sequential Dependencies from the Waterfall  PSPV-based Solutions If we know how to combine the changes between the two proof steps, do so. If we know how to combine the changes between the two proof steps, do so. Otherwise, terminate the latter proof step and restart it with the intermediate PSPV value Otherwise, terminate the latter proof step and restart it with the intermediate PSPV value Results in a computation exactly the same as the serial computation, without a need to combine PSPVs Results in a computation exactly the same as the serial computation, without a need to combine PSPVs

14. 14 Component 2: Introducing Parallelism  Adding futures to the underlying implementation language (Lisp)  Adding parallelism abstractions to the logic (ACL2’s logic)  Managing output  Managing user interrupts

15. 15 Adding Futures to the Underlying Implementation Language  Create functions for spawning, reading, and terminating the evaluation of futures (future x) :: X -> Future-structure (future x) :: X -> Future-structure (future-read x) :: Future-structure -> X (future-read x) :: Future-structure -> X (future-abort x) :: Future-structure -> C (future-abort x) :: Future-structure -> C  Relies upon our multi-threading interface that unifies CCL and SBCL features

16. 16 Adding Parallelism Abstractions to the Logic  Modify plet to support multiple values and speculative evaluation Give the ACL2 user a means to specify that a variable is unused in a particular expression Give the ACL2 user a means to specify that a variable is unused in a particular expression Therefore, once evaluation enters a branch with such a specified branch, plet can terminate the evaluation of the unnecessary variable values Therefore, once evaluation enters a branch with such a specified branch, plet can terminate the evaluation of the unnecessary variable values Already defined plet differently in raw Lisp and the ACL2 logic Already defined plet differently in raw Lisp and the ACL2 logic This is an enhancement of what we already created [Rager 2008, Rager and Hunt 2009]This is an enhancement of what we already created [Rager 2008, Rager and Hunt 2009] Example usage: Example usage: (plet (((x y) (mv 3 4)) (plet (((x y) (mv 3 4)) ((q r) (mv 8 9))) ((q r) (mv 8 9))) (if (equal q 8) (if (equal q 8) (check-vars-unused-and-kill (x y) (check-vars-unused-and-kill (x y) (+ q r)) (+ q r)) (+ x y q r))) (+ x y q r)))

17. 17 Adding Parallelism Abstractions to the Logic  By using this enhanced version of plet, we can create new abstractions, e.g. spec-mv-let  Spec-mv-let automatically performs the check that certain variables are unused and automatically terminates unnecessary computations  Example annotated usage: (spec-mv-let (x y) (spec-mv-let (x y) ;; speculatively evaluate (mv 3 4) ;; speculatively evaluate (mv 3 4) (mv 3 4) (mv 3 4) (mv-let (q r) (mv-let (q r) (mv 8 9) (mv 8 9) (if (equal q 8) (if (equal q 8) ;; the speculative evaluation is irrelevant, return ;; the speculative evaluation is irrelevant, return ;; a value that doesn’t use those results ;; a value that doesn’t use those results (+ q r) (+ q r) ;; the speculative evaluation is useful, return a ;; the speculative evaluation is useful, return a ;; value that uses those results ;; value that uses those results (+ x y q r)))) (+ x y q r))))

18. 18 Component 3: Managing Interactivity  Original Goal: maintain output consistent between serial and parallel proofs of subgoals Store the output in a data structure and signal when the next piece of output is ready for printing Store the output in a data structure and signal when the next piece of output is ready for printing I believe this to be feasible, but we would like to do more I believe this to be feasible, but we would like to do more

19. 19 Managing Output  New Goal: print meaningful output as it becomes available Print the proof checkpoints as soon as they are computed Print the proof checkpoints as soon as they are computed There is usually a very minimal amount of checkpoint output compared to the amount of output that occurs with a full proof attempt’s narrativeThere is usually a very minimal amount of checkpoint output compared to the amount of output that occurs with a full proof attempt’s narrative Users suggest that in the relatively rare case that a user wants to see the full narrative, that they would likely be satisfied with replaying the proof in a serial manner. Users suggest that in the relatively rare case that a user wants to see the full narrative, that they would likely be satisfied with replaying the proof in a serial manner.

20. 20 Managing Output  It is unclear what type of interface changes will be necessary to accommodate non- deterministic output  An investigation into such accommodations is a planned part of this work  An example of one potential interface follows

21. 21 / 29

22. 22 Managing User Interrupts  When a user aborts a proof, all subgoal computations will be gracefully terminated and the parallelism state of ACL2 will be reset to its initial state  When a user attempts to debug a proof: Automatically interrupt and pause other threads Automatically interrupt and pause other threads Automatically order those threads to resume evaluation Automatically order those threads to resume evaluation Users also have access to these functions that pause and resume subgoal computations Users also have access to these functions that pause and resume subgoal computations

23. 23 Component 4: Evaluating our Approach  Soundness  Performance

24. 24 Evaluating Soundness  A large subset of the regression suite (93%) will be able to pass with parallelism enabled  We rely on our strategic development and use of macros as our main source of credibility  Insert assertions that check for unexpected changes in the program’s state.  This branch of the build will be tuned for interactivity. Those desiring the assurance level of non-parallel ACL2 can run the serial version.

25. 25 Evaluating Performance  Goal: Reduce the latency between when a user submits a conjecture and when the user receives useful feedback concerning that conjecture’s provability  We do not try to improve the performance of proof attempts that take < 1 second

26. 26 Evaluating Performance  We are not concerned with proof attempts that take less than a second to compute, because anything between 0.10 seconds and 1.0 seconds feels the same to a user.  Example performance result for certifying one of the ACL2 libraries: Summary for making books/ordinals/ordinal-addition.cert: Average sequential time was: 6.22s Average parallel time was: 5.09s Sequential minimum was: 6.12s Parallel minimum was: 4.87s Of 10 iterations, the parallel version was faster than the sequential version 10 times.

27. 27 What We’ve Done so Far  Removed the modification of state and output from the waterfall  Modified Lisp library and ACL2 to permit parallel evaluation of the waterfall  Run some preliminary performance benchmarks

28. 28 What’s Left  Reintroducing output in a helpful way  Improving performance  Evaluating our solution

29. 29 Outline  Introduction The Automated Proof Process – the Waterfall The Automated Proof Process – the Waterfall The ways ACL2 currently uses parallelism The ways ACL2 currently uses parallelism  Research Contributions  Key Steps Removing sequential dependencies Removing sequential dependencies Introducing Parallelism into the Proof Process Introducing Parallelism into the Proof Process Adding futures to the underlying implementation languageAdding futures to the underlying implementation language Adding parallelism abstractions to the logicAdding parallelism abstractions to the logic Using parallelism in the waterfallUsing parallelism in the waterfall Managing output Managing output Managing user interrupts Managing user interrupts  Evaluating the Soundness and Performance of our Approach