Presentation is loading. Please wait.

Presentation is loading. Please wait.

Fine-grained Visualization Pipelines and Lazy Functional Languages D.J. Duke 1, M. Wallace 2, R. Borgo 1, & C. Runciman 2 1 Visualization and Virtual Reality.

Similar presentations


Presentation on theme: "Fine-grained Visualization Pipelines and Lazy Functional Languages D.J. Duke 1, M. Wallace 2, R. Borgo 1, & C. Runciman 2 1 Visualization and Virtual Reality."— Presentation transcript:

1 Fine-grained Visualization Pipelines and Lazy Functional Languages D.J. Duke 1, M. Wallace 2, R. Borgo 1, & C. Runciman 2 1 Visualization and Virtual Reality Group, University of Leeds, UK 2 Programming Languages and Systems Group, University of York, UK

2 2 Overview Motivation why are we doing functional programming? a lazy polytypic grid Haskell 101 Marching cubes Functional approaches array-based streaming Evaluation performance software engineering Outlook

3 3 Why (pure) functional programming? Practical concern: grid-enabled visualization new technologies for generic programming (polytypism) staged computation laziness – natural demand-driven evaluation and streaming migration to parallel evaluation Theoretical concern: abstractions for software development problem decomposition program composition J. Hughes, Why Functional Programming Matters correctness – mathematically tractable, concise different way of thinking about problems

4 4 What is (pure) functional programming?

5 5 Type declaration - optional Higher-order: map takes another function as parameter Function body: list of equations Curried functions: a -> b -> c rather than (a,b) -> c makes partial application easy e.g. (+1), or map (+4) What is (pure) functional programming? Functions are first-class citizens passed as parameters and/or returned as results higher-order functions implement patterns of computation Expressive type systems Laziness Example map :: (a -> b) -> [a] -> [b] map f [] = [] map f (a:as) = (f a) : (map f as) Pattern matching: case analysis of list constructor Laziness: (f a) etc only evaluated as needed Type variables: this function works for any choice of types a and b

6 6 Haskell 101 Other useful higher-order functions (.) :: (b -> c) -> (a -> b) -> (a -> c) (f. g) x = f (g x) zipWith2 :: (a->b->c) -> [a] -> [b] -> [c] zipWith2 f [] _ = [] zipWith2 f _ [] = [] zipWith2 f (a:as) (b:bs) = (f a b):(zipWith f as bs) ($) :: (a -> b) -> a -> b f $ x = f x [Why? Write f. g $ x instead of (f. g) x ] Local definitions largest :: [a] -> a largest (a:as) = largest1 a as where largest1 a [] = a largest1 a (b:bs) | a > b = largest1 a bs | otherwise = largest1 b bs Type classes largest :: (Ord a) => [a] -> a

7 Pipelines and functions pipeline architecture widespread in visualization supports distribution and streaming However Streaming is ad-hoc and coarse grained Algorithms depend on mesh type Data traversed multiple times reader ozone levels isosurfacenormals isosurfac e reade r temperatu re displaygeo-reference Note: analogy of pipeline composition and function composition: f. g ?

8 In the future... a lazy polytypic grid Grid enabling: distribution of the run-time system and on-demand streaming of arbitrary data. Through fusion laws, multiple traversals on a single resource are folded into one pass. 2 readerozone levelsisosurfacenormals isosurfac e reade r temperatu re geo-referencedisplay Algorithms: written once, based on generic pattern of data types, then instantiated for any type. 1 3 Specialization: adapt programs to utilize resources available – data or computational.

9 9 Marching Cubes Why do it? explore functional visualization well known, important algorithm For each cell compare point samples with threshold generate case-index lookup table to find intersected edges interpolate surface-edge intersection group intersection points into triangles Ambiguity problem – various solutions MC33 approach tri-linear interpolant \

10 10 Functional arrays Basic types type XYZ = (Int,Int,Int) type Num a => Dataset a = Array XYZ a type Cell a = (a,a,a,a,a,a,a,a) Dataset traversal isoA :: (Ord a, Integral a) => a -> Dataset a -> [Triangle] isoA th arr = concat $ zipWith1 (mcubeA th lookup) addrs where lookup arr (x,y,z) = (arr!(x,y,z), arr!(x+1,y,z),.., arr!(x+1,y+1,z+1)) addrs = [ (i,j,k) | k <- [1..ksz-1], j <- [1..jsz-1], i <- [1..isz-1]] (isz,jsz,ksz) = bounds arr Cell-surface intersection mcubeA th lookup xyz = group3. map (interp th cell xyz). mctable!. toByte. map8 (>th) $ cell where cell = lookup xyz not pseudocode

11 11 Can we do better? entire dataset must be in-core repeated threshold comparison (once per cell with common vertex) repeated computation of edge interpolant interpolant used by more than one triangle in cell surface intersects edge of more than one cell

12 12 A window onto samples Insight – we only need a constant-sized window onto dataset window size = plane + line + 1 = (jsz+1)*isz + 1 move from cell-to-cell => advance window by 1 values read lazily as ne eded – and gc'd automatically Haskell representation - a list of values data Num a => D XYZ [a]

13 13 Streams of cells stream :: XYZ -> [a] -> [Cell a] stream (isz,jsz,ksz) origin = zip8 origin (drop 1 origin) (drop (line+1) origin) (drop line origin) (drop plane origin) (drop (plane+1) origin) (drop (planeline+1) origin) (drop planeline origin) where line = isz plane = isz * jsz planeline = plane + line 8-tuple... cells

14 14 Discontinuities Two apparently different solutions: rewrite mkStream to test for dataset boundary separately strip phantom cells from output disContinuities :: XYZ -> [b] -> [b] disContinuities (isz,jsz,ksz) = step (0,0,0) where step (i,j,k) (x:xs) | i==(isz-1) = step (0,j+1,k) xs | j==(jsz-1) = step (0,0,k+1) (drop (isz-1) xs) | k==(ksz-1) = [] | otherwise = x : step (i+1,j,k) xs cellStream = disContinuities size. Stream Fusion rule in compiler can merge generation & test

15 15 From array to stream isoA :: (Ord a, Integral a, Fractional b) => a -> Dataset a -> [Triangle b] isoA th sampleArr = concat $ zipWith1 (mcubeA th lookup) addrs where lookup arr (x,y,z) = (arr!(x,y,z), arr!(x+1,y,z),.., arr!(x+1,y+1,z+1)) addrs = [ (i,j,k) | k <- [1..ksz-1], j <- [1..jsz-1], i <- [1..isz-1]] mcubeA th lookup xyz = group3. map (interp th cell xyz). mctable!. toByte. map8 (>th) $ cell where cell = lookup xyz Array isoS th samples = concat $ zipWith2 (mcubeS th) addrs cells where cells = stream size samples addrs = [ (i,j,k) | k <- [1..ksz-1], j <- [1..jsz-1], i <- [1..isz-1]] mcubeS :: (Ord a, Integral a, Fractional b) => a -> XYZ -> Cell a -> [Triangle b] mcubeS th xyz cell = group3. map (interp th cell xyz). mctable!. toByte. map8 (>th) $ cell Stream

16 16 Sharing vertex comparison isoS th samples = concat $ zipWith2 (mcubeS th) addrs cells where cells = stream size samples addrs = [ (i,j,k) | k <- [1..ksz-1], j <- [1..jsz-1], i <- [1..isz-1]] mcubeS :: (Ord a, Integral a, Fractional b) => a -> XYZ -> Cell a -> [Triangle b] mcubeS th xyz cell = group3. map (interp th cell xyz). mctable!. toByte. map8 (>th) $ cell Array Stream isoT th samples = concat $ zipWith3 (mcubeT th) addrs cells indices where indices = map toByte. stream. map (>th) cells =... addrs =... mcubeT :: (Ord a, Integral a, Fractional b) => a -> XYZ -> Cell a -> Byte -> [Triangle b] mcubeT th xyz cell index = group3. map (interp th cell xyz). mctable! $ index Indices

17 17 Sharing edge interpolants isoT th samples = concat $ zipWith3 (mcubeT th) addrs cells indices where indices = map toByte. stream. map (>th) cells =... addrs =... mcubeT :: (Num a, Fractional b) -> XYZ -> Cell a -> Byte -> [Triangle b] mcubeT th xyz cell index = group3. map (interp th cell xyz). mctable! $ index Indices Interpolants isoI th (D size samples) = concat $ zipWith3 mcubeI addrs indices edges where edges = disContinuities size. mkCellEdges th size indices =... addrs =... mcubeI :: (Fractional b) => CellEdge b -> Byte -> XYZ -> [Triangle b] mcubeI xyz index edges = group3. map (selectEdge edges xyz). (mctable!) $ index type CellEdge a = (a, a, a, a, a, a, a, a, a, a, a, a) mkCellEdges :: (Integral a, Fractional b) => a -> XYZ -> [a] -> [CellEdge b] mkCellEdges thresh (XYZ isz jsz ksz) stream = zipWith12 CellEdge inter_x (drop line inter_x) (drop plane inter_x) (drop (plane+line) inter_x) inter_y (drop 1 inter_y) (drop plane inter_y) (drop (plane+1) inter_y) inter_z (drop 1 inter_z) (drop line inter_z) (drop (line+1) inter_z) where line = isz plane = isz*jsz offset d = zipWith2 interp stream d inter_x = offset (drop 1 stream) inter_y = offset (drop line stream) inter_z = offset (drop plane stream) interpolate v0 v1 = fromIntegral (thresh-v0) / fromIntegral (v1-v0)

18 18 Performance - time Comparison with VTK pipeline [raw reader] -> vtkMarchingCubes; outputs retained forced evaluation of Haskell output comparison f(surface size)

19 19 Performance - space Peak live memory usage Haskell usage measured by heap profiling Stream space use f(window size) Scalabilty without moving to out-of-core methods

20 20 Ifs, buts, and maybes Comparison of apples and oranges? only intended as a reality check... vtkMarchingCubes not the fastest VTK surface extraction filter VTK pipeline could be fine-tuned, e.g. drop intermediate results other implementations However... our approach can also be tuned further... elegant Haskell not necessarily efficient Haskell compiler work, e.g. fusion laws: (map f).(map g) = map (f.g)

21 21 Other criteria Streaming criteria (Law et.al., Proceedings Vis'99) caching: streamed data access + memoisation demand-driven: natural product of call-by-need (lazy evaluation) data pulled in – and transformed – only to the extent needed hardware architecture independence polymorphic types supported by type predicates Component based: library of fine-grained visualization combinators Clarity algorithms used in decision making should be evidently correct strong typing, concise expression, equational reasoning

22 22 Outlook Achievements novel re-construction of fundamental algorithms fine-grained streaming - scalability demonstrated that FP can be (surprisingly) efficient Current work extending stream approach to other mesh organizations exploring polytypism to abstract from organization distributing (functional) pipeline over a grid exiting distributed FP approaches – GRID-GUM, Eden York bytecode compiler (YHC)

23 23 Finally Thanks to: EPSRC Fundamental Computing for e-Science Programme Anonymous reviewers Further information VVR group: Plasma group: Repository: hackage.haskell.org/trac/PolyFunViz/ Haskell: Questions?


Download ppt "Fine-grained Visualization Pipelines and Lazy Functional Languages D.J. Duke 1, M. Wallace 2, R. Borgo 1, & C. Runciman 2 1 Visualization and Virtual Reality."

Similar presentations


Ads by Google