1. 1 Distributed BDD-based Model Checking Orna Grumberg Technion, Israel Joint work with Tamir Heyman, Nili Ifergan, and Assaf Schuster CAV00, FMCAD00, CAV01, CAV03, CHARME05,ATVA05 PDMC July 14, 2011

2. 2 Model Checking Specification Model Checker Counter Example + System Model

3. 3 BDD-Based Model Checking Model checking often suffers from the state-explosion problem which refers to its high space requirements One of the first solution proposed for this problem: symbolically representing the model using BDDs Model checking is then done by manipulations on BDDs

4. 4 Binary decision diagrams (BDDs) Data structure for representing Boolean functions –Often concise in memory They are suitable for representing sets of states Most Boolean operations can be done efficiently with BDDs Yet, they still cannot handle large systems

5. 5 Our solution: Distributed Model checking Using the accumulative computation power and memory of a number of machines working in parallel Enables model checking of models with higher memory requirements –For large models, time requirements become a problem as well

6. 6 Next solution: Asynchronous Distributed Model Checking Obtaining high speedups by using an asynchronous distributed algorithm for Reachability Analysis –Computing Reachability for models with high time and memory requirements

7. 7 Reachability Analysis Reachability Analysis - computing the set of states that can be reach from the set of initial states The verification of most temporal safety properties can be reduced to Reachability Analysis

8. 8 Sequential Reachability Algorithm

9. 9 R S0S0

10. Image Computation The computation of the set of successors of a given set of states Image can be obtained by micro steps –Partitioned transition relations Each partition defines the transition for one variable The conjunction of all the partitions give the transition of all variables. –Each micro step adds to the intermediate results one more partition

11. 11 The Distributed Approach

12. 12 The state space on which the Reachability analysis is performed is partitioned into slices Each slice is owned by one process A set of Boolean window functions: w 1 …w n, defines for each process the slice it owns State Space Partition

13. 13 State Space Partition – Cont. The set of window functions is complete and disjoint: The state space S is partitioned so that S W1W1 W2W2 W3W3 W4W4 W5W5

14. The Basic Distributed Algorithm The algorithm works iteratively Each iteration consists of two phases

15. The Basic Distributed Algorithm The algorithm works iteratively Each iteration consists of two phases –Phase 1 – Image Computation Each of the active workers applies image computation to states it owns, found in the previous iteration

16. The Basic Distributed Algorithm The algorithm works iteratively Each iteration consists of two phases –Phase 1 – Image Computation Each of the active workers applies image computation to states it owns, found in the previous iteration Some of the computed states may not own by it

17. The Basic Distributed Algorithm The algorithm works iteratively Each iteration consists of two phases –Phase 1 – Image Computation Each of the active workers applies image computation –Phase 2 – Exchange The active workers exchange the states they do not own with the other workers

18. The Basic Distributed Algorithm The algorithm works iteratively Each iteration consists of two phases –Phase 1 – Image Computation Each of the active workers applies image computation –Phase 2 – Exchange The active workers exchange the states they do not own with the other workers Phase 2 starts only after all the active workers finished phase 1

19. 19 The Basic Algorithm ImageIteration 1Exchange P1P1 P2P2...........

20. Using workers “by need” There is a pool of non-active free workers Workers join and leave the computation as needed 20

21. Splitting If the worker ’ s memory overflows during image computation The image computation stops with an intermediate results The overflowed worker gets a free worker from the pool and splits the computation into two parts –Each worker then continues the computation starting from its part of the intermediate result

22. Splitting (cont.) Overflow may occur also in the exchange phase Splitting is applied and then exchange continues among the new set of workers, according to their new windows

23. 23 Collect Small Workers If a worker does not have enough work –It delivers its ownership and owned states to a colleague worker –It joins the pool of free workers (and may participate in the computation later on, if needed)

24. 24 High Time Requirements Observations: The iterations are synchronized “Fast” workers have to wait until all workers complete the image phase before they can proceed to the exchange phase –Unnecessary idle time for “fast” processes

25. 25 High Time Requirements Observations: The iterations are synchronized “Fast” workers have to wait until all workers complete the image phase before they can proceed to the exchange phase –Unnecessary idle time for “fast” processes In reachability analysis states can be found in any order as long as they are all found

26. 26 The Asynchronous Approach The iterations are no longer synchronized among processes Each process can execute the image computation at its own pace while sending and receiving states "in the background“ –Image computation and state exchange become concurrent operations

27. 27 Challenge 1 – Concurrency Computing image and sending/receiving non-owed states is done concurrently –Receiving and sending non-owned states packages is done “in the background” (asynchronous send/receive) –Transforming packages back to BDDs and packaging BDDs to be sent, is done between micro steps

28. 28 Challenge 2 – Package Forwarding Because of splitting during the exchange phase, some non-owned states may reach the wrong process Solution: Each process “takes” from the package the states it owns and forwards it to other processes which also have states in this package

29. 29 P 2, W 2 P2,W’2P2,W’2 P3,W’3P3,W’3 Package Forwarding P1P1 P 2,W 2 P 3,W 2  W ’ 3

30. 30 Package Forwarding (cont.) The algorithm assures that each non-owned state will eventually reach its owner

31. 31 Challenge 3 - Termination Asynchronous distributed termination detection in an environment in which processes may join and leave the computation –Extension to the two phase Dijkstra termination detection algorithm with dynamic number of processes The termination detection algorithm works “in the background”

32. 32 Implementation Division, developed by Tamir Heyman, is a generic platform for the study of distributed symbolic model checking –Division requires a model checker as an external module We use Intel’s model checker, Forecast The distributed asynchronous algorithm has been developed and implemented by Nili Ifergan, on top of the Division framework

33. 33 Implementation issues The communication between the processes was done through MPI over fast Ethernet –The communication module allows for true asynchronous message passing We conducted our experiments on a parallel testbed –It included a maximum of 55 PC machines, each consisting of dual 2.4GHz Xeon processors with 4GB memory

34. Summary We developed a distributed BDD- based model checking –The state space is partitioned, not the BDDs We obtained significant memory and time reductions We verified huge industrial designs, mostly taken from Intel 34

35. Summary (cont.) The project stopped because SAT-based (bounded) model checking became a standard tool –Fast and easy to use The advantage of our approach was (and maybe still is) in the verification of extremely large hardware designs –At the time there was no interest 35

36. The future Modern technologies and infrastructures may make distributed BDD-based model checking attractive again ? 36

37. 37 The End