1. Parallel Exact Stochastic Simulation in Biochemical Systems
Jiangtian Li
7/9/2019
csc801 system research seminar

2. csc801 system research seminar
Introduction
Overview of parallel exact stochastic simulation
My summer work at Oak Ridge National Lab
Joint work with Mike McCollum, Nagiza Samatova
One of two research projects this summer
7/9/2019
csc801 system research seminar

3. csc801 system research seminar
Motivation for PESS
Model large-scale biochemical systems such as gene regulatory network
A single cell cycle of E.coli requires 1014~1016 executions of reactions
Accelerate simulation by parallelizing executions while preserving accuracy of stochastic simulation
7/9/2019
csc801 system research seminar

4. Gillespie’s stochastic simulation
Event-driven simulation of random process
Modeling spatially uniform mixture of N species (state variables) through M reaction channels (events)
Probability of reaction scheduled is characterized by propensity function
Simulation ends when species populations converge
7/9/2019
csc801 system research seminar

5. Simulation Framework Simulation Application Simulation Executive
• state variables
• code modeling system behavior
• I/O and user interface software
State variables
Species A=1000;
Species B=500;
Species C=100; …
calls to schedule events
calls to event handlers
Event list (reactions)
A + B -> C
C + D -> E …
Simulation Executive
• event list management
• managing advances in simulation time
Global clock = 3:00
7/9/2019
csc801 system research seminar

6. Issues in Parallelization
Inter-processor synchronization to avoid causality errors 12 17 23 35
affect
processed event 14 26 42
unprocessed event
7/9/2019
csc801 system research seminar

7. Issues in Parallelization
Model subdivision and processor mapping
Reducing communications
Load balancing
7/9/2019
csc801 system research seminar

8. Inter-processor Synchronization
Optimistic synchronization protocol – Time Warp
To ensure correct execution order in whole simulation
To exploit more parallelism in situation where causality errors don’t occur
natural and elegant implementation
7/9/2019
csc801 system research seminar

9. csc801 system research seminar
Time Warp
Roll back state variables
when a straggler message is received
record a log of the modified state variables
restore the state variables
straggler message 18
processed event 12
A=1
B=2
C=3 21
A=4 35
A=5
C=9
unprocessed event 41
state variables log
A=0
B=0
C=0
A=1
A=4
C=3
A=1
A=4
C=3
A=1
B=2
C=3
A=5
B=2
C=9
rollback
7/9/2019
csc801 system research seminar

10. csc801 system research seminar
Time Warp (cont.)
Unsend messages
use anti-message – logically identical copy of original message, with opposite sign
anti-message stored in sender’s output queue and resent when rollback
message/anti-message annihilated in receiver’s input queue
7/9/2019
csc801 system research seminar

11. Example: unsend message
Input queue
processed event 12 21 35 41
unprocessed event
saved state
State queue
anti-message
Anti-message
queue 19 42 18
send anti-message
Sender
Straggler message, rollback 42
Anti-message arrives, rollback
Annihilate message and anti-message
Input queue 42 27 45 55
State queue
Receiver
Anti-message
queue 33 57
send anti-message
7/9/2019
csc801 system research seminar

12. csc801 system research seminar
Compute G.V.T
Global virtual time
Defined as minimum among unprocessed and partially processed events and anti-messages
Serve as lower bound on time stamp of any future rollback
Compute
synchronous algorithm
handle transient message – message acknowledgement
7/9/2019
csc801 system research seminar

13. csc801 system research seminar
Model subdivision
Goal - Efficient parallel execution
Reduce communications
Reduce rollbacks
Maintain load balance
Approach - Analyze interdependence among reactions
7/9/2019
csc801 system research seminar

14. Model subdivision (cont.)
7/9/2019
csc801 system research seminar

15. Model subdivision (cont.)
Approach - Construct graph model
vertex – reaction
vertex weight – propensity function
edge weight – sum of vertices weights
7/9/2019
csc801 system research seminar

16. Model subdivision (cont.)
Partition graph –
Objective - minimize messages communicated (cut-edge weights)
Constraint – balance executions
Metis – graph partition package
University of Minnesota
7/9/2019
csc801 system research seminar

17. csc801 system research seminar
Results
Tested on SGI Altix (Ram) at ORNL
256 Intel Itanium2 processors running at 1.5 GHz
8 GB of memory per processor
7/9/2019
csc801 system research seminar

18. csc801 system research seminar
Results
7/9/2019
csc801 system research seminar

19. csc801 system research seminar
Results
7/9/2019
csc801 system research seminar

20. csc801 system research seminar
Results
7/9/2019
csc801 system research seminar

21. csc801 system research seminar
Results
7/9/2019
csc801 system research seminar

22. csc801 system research seminar
Future work
Dynamic model subdivision during simulation
Trade-off – how much is overhead?
7/9/2019
csc801 system research seminar

23. csc801 system research seminar
Conclusion
Parallel Exact Stochastic Simulation has demonstrated to be efficient and effective in modeling biochemical systems
Proper model subdivision significantly enhanced performance
7/9/2019
csc801 system research seminar

24. csc801 system research seminar
Any question?
7/9/2019
csc801 system research seminar