10/26/20151 GC16/3011 Functional Programming Lecture 21 Parallel Graph Reduction

10/26/20152 Contents  Sequential evaluator  Parallel graph reduction  Lazy/Strict/Non-strict evaluation  Strictness analysis  Elements of parallel graph reduction  Issues

10/26/20153 Sequential evaluator  A sequential evaluator  Normal order (lazy)  Algebraic type = tree  Tree reduction  Can make tree into a graph  Graph reduction  Parallel GR: many sequential evaluators all working on one graph  With additional support for synchronisation and communication

10/26/20154 Making things easier  Turn all functions into combinators  No free variables!  “Lift” all lambdas to top level  Named functions have no embedded lambdas  So no name clashes, no free variable capture  Easy beta reduction!  “supercombinator reduction”

10/26/20155 Lazy/Strict/Non-strict evaluation  Normal order reduction (“lazy evaluation”) is SEQUENTIAL  Alternative: Applicative order (“strict evaluation”)  Parallel GR CANNOT be sequential!  Does PGR therefore lose benefits of lazy evaluation?  No! As long as it only evaluates those bits of the graph that normal order reduction would have evaluated  “non-strict” evaluation  Use “Strictness Analysis”

10/26/20156 Strictness analysis  Done at compile time  For (f x y z) which of x, y and/or z does f always evaluate using normal order reduction?  If (f endless y z) = endless, then assume f MUST evaluate x  Halting problem?  Use approximation technique  Abstract Interpretation

10/26/20157 Elements of parallel GR  Many processors running sequential evaluators  Graph exists in shared heap(s)

10/26/20158 Processor Shared heap Processor Processor + 3 x

10/26/20159 Elements of parallel GR  Processors  Each evaluates a subgraph to (weak head) normal form  Slightly modified for Inter-Process Communication and synchronisation  Values communicated via graph  Two processors might evaluate the same sub-graph  just wasted effort, cannot affect correctness  But prefer not to waste the effort!

10/26/ Elements of parallel GR  A processor evaluates one or more tasks  Tasks/threads “to be done” in shared task pool(s)  Synchronisation required whenever a task requires the value of a subgraph that is currently being evaluated by another task  First task blocks, waiting for second task to finish  Processor need not block (can start another task)

10/26/ Processor Processor Processor Processor Processor Shared x5 + 3 x Shared task pool

10/26/ Sparking  Strictness analysis tells us all of the subgraphs that definitely need to be evaluated  Graph is annotated  At start, just one task descriptor is placed in task pool  One evaluator will start evaluating it  Evaluators check annotations and place task descriptors (sparks) into the task pool  Other evaluators will start work on the sparked tasks

10/26/ Issues  How things are represented in the heap  How the evaluators work  Shared memory (including Virtual SM) or distributed memory  Synchronising tasks (e.g. block and resume)  Scheduling, task distribution and load balancing

10/26/ Summary  Sequential evaluator  Lazy/Strict/Non-strict evaluation  Strictness analysis  Elements of parallel graph reduction  Issues