2. Partial Differential Equations Introduction, Adam Zornes Discretizations and Iterative Solvers, Chenfang Chen Parallelization, Dr. Danny Thorne

3. What do You Stand For? A PDE is a Partial Differential Equation This is an equation with derivatives of at least two variables in it. In general, partial differential equations are much more difficult to solve analytically than are ordinary differential equations

4. What Does a PDE Look Like Let u be a function of x and y. There are several ways to write a PDE, e.g., –u x + u y = 0 –  u/  x +  u/  y = 0

5. The Baskin Robin’s esq Characterization of PDE’s The order is determined by the maximum number of derivatives of any term. Linear/Nonlinear –A nonlinear PDE has the solution times a partial derivative or a partial derivative raised to some power in it Elliptic/Parabolic/Hyperbolic

6. Six One Way Say we have the following: Au xx + 2Bu xy + Cu yy + Du x + Eu y + F = 0. Look at B 2 - AC – < 0 elliptic – = 0 parabolic – > 0 hyperbolic

7. Or Half a Dozen Another A general linear PDE of order 2: Assume symmetry in coefficients so that A = [aij] is symmetric. Eig(A) are real. Let P and Z denote the number of positive and zero eigenvalues of A. –Elliptic: Z = 0 and P = n or Z = 0 and P = 0.. –Parabolic: Z > 0 (det(A) = 0). –Hyperbolic: Z=0 and P = 1 or Z = 0 and P = n-1. –Ultra hyperbolic: Z = 0 and 1 < P < n-1.

8. Elliptic, Not Just For Exercise Anymore Elliptic partial differential equations have applications in almost all areas of mathematics, from harmonic analysis to geometry to Lie theory, as well as numerous applications in physics. The basic example of an elliptic partial differential equation is Laplace’s Equation –u xx - u yy = 0

9. The Others The heat equation is the basic Hyperbolic –u t - u xx - u yy = 0 The wave equations are the basic Parabolic –u t - u x - u y = 0 –u tt - u xx - u yy = 0 Theoretically, all problems can be mapped to one of these

10. What Happens Where You Can’t Tell What Will Happen Types of boundary conditions –Dirichlet: specify the value of the function on a surface –Neumann: specify the normal derivative of the function on a surface –Robin: a linear combination of both Initial Conditions

11. Is It Worth the Effort? Basically, is it well-posed? –A solution to the problem exists. –The solution is unique. –The solution depends continuously on the problem data. In practice, this usually involves correctly specifying the boundary conditions

12. Why Should You Stay Awake for the Remainder of the Talk? Enormous application to computational science, reaching into almost every nook and cranny of the field including, but not limited to: physics, chemistry, etc.

13. Example Laplace’s equation involves a steady state in systems of electric or magnetic fields in a vacuum or the steady flow of incompressible non-viscous fluids Poisson’s equation is a variation of Laplace when an outside force is applied to the system

14. Poisson Equation in 2D

15. Example: CFD

16. Computational Fluid Dynamics CFD can be defined narrowly as confined to aerodynamic flow around vehicles but it can be generalized to include as well such areas as weather and climate simulation, flow of pollutants in the earth, and flow of liquids in oil fields (reservoir modelling). Involve Huge PDE’s Computational Science only Realistic Solution

17. Links http://www.npac.syr.edu/users/gcf/cps713overI94/ http://www.cse.uiuc.edu/~rjhartma/pdesong.html http://www.maths.soton.ac.uk/teaching/units/ma274/nod e2.html http://www.npac.syr.edu/projects/csep/pde/pde.html http://mathworld.wolfram.com/PartialDifferentialEquation. html

18. Discretization and Iterative Method for PDEs Chunfang Chen Danny Thorne Adam Zornes

19. Classification of PDEs Different mathematical and physical behaviors:  Elliptic Type  Parabolic Type  Hyperbolic Type System of coupled equations for several variables:  Time : first-derivative (second-derivative for wave equation)  Space: first- and second-derivatives

20. Classification of PDEs (cont.) General form of second-order PDEs ( 2 variables)

21. PDE Model Problems  Hyperbolic (Propagation) Advection equation (First-order linear) Advection equation (First-order linear) Wave equation (Second-order linear ) Wave equation (Second-order linear )

22. PDE Model Problems (cont.)  Parabolic (Time- or space-marching) Burger’s equation(Second-order nonlinear) Burger’s equation(Second-order nonlinear) Fourier equation (Second-order linear ) Fourier equation (Second-order linear ) (Diffusion / dispersion)

23. PDE Model Problems (cont.)  Elliptic (Diffusion, equilibrium problems) Laplace/Poisson (second-order linear) Laplace/Poisson (second-order linear) Helmholtz equation (second-order linear) Helmholtz equation (second-order linear)

24. PDE Model Problems (cont.)  System of Coupled PDEs Navier-Stokes Equations Navier-Stokes Equations

25. Well-Posed Problem Numerically well-posed Numerically well-posed  Discretization equations  Auxiliary conditions (discretized approximated) the computational solution exists (existence) the computational solution exists (existence) the computational solution is unique (uniqueness) the computational solution is unique (uniqueness) the computational solution depends continuously on the approximate auxiliary data the computational solution depends continuously on the approximate auxiliary data the algorithm should be well-posed (stable) also the algorithm should be well-posed (stable) also

26. Boundary and Initial Conditions R s n RR  Initial conditions: starting point for propagation problems  Boundary conditions: specified on domain boundaries to provide the interior solution in computational domain

27. Numerical Methods  Complex geometry  Complex equations (nonlinear, coupled)  Complex initial / boundary conditions  No analytic solutions  Numerical methods needed !!

28. Numerical Methods Objective: Speed, Accuracy at minimum cost  Numerical Accuracy (error analysis)  Numerical Stability (stability analysis)  Numerical Efficiency (minimize cost)  Validation (model/prototype data, field data, analytic solution, theory, asymptotic solution)  Reliability and Flexibility (reduce preparation and debugging time)  Flow Visualization (graphics and animations)

29. computational solution procedures Governing Equations ICS/BCS Discretization System of Algebraic Equations Equation (Matrix) Solver Approximate Solution Continuous Solutions Finite-Difference Finite-Volume Finite-Element Spectral Boundary Element Hybrid Discrete Nodal Values Tridiagonal ADI SOR Gauss-Seidel Krylov Multigrid DAE U i (x,y,z,t) p (x,y,z,t) T (x,y,z,t) or  ( , , ,  )

30. Discretization  Time derivatives almost exclusively by finite-difference methods almost exclusively by finite-difference methods  Spatial derivatives - Finite-difference: Taylor-series expansion - Finite-difference: Taylor-series expansion - Finite-element: low-order shape function and - Finite-element: low-order shape function and interpolation function, continuous within each interpolation function, continuous within each element element - Finite-volume: integral form of PDE in each - Finite-volume: integral form of PDE in each control volume control volume - There are also other methods, e.g. collocation, - There are also other methods, e.g. collocation, spectral method, spectral element, panel spectral method, spectral element, panel method, boundary element method method, boundary element method

31. Finite Difference  Taylor series  Truncation error  How to reduce truncation errors? Reduce grid spacing, use smaller  x = x-x o Reduce grid spacing, use smaller  x = x-x o Increase order of accuracy, use larger n Increase order of accuracy, use larger n

32. Finite Difference Scheme  Forward difference  Backward difference  Central difference

33. Example : Poisson Equation (1,1) (-1,-1) (1,-1) (-1,1) x y

34. Example (cont.)

35. Rectangular Grid After we discretize the Poisson equation on a rectangular domain, we are left with a finite number of gird points. The boundary values of the equation are the only known grid points

36. What to solve? Discretization produces a linear system of equations. The A matrix is a pentadiagonal banded matrix of a standard form: A solution method is to be performed for solving

37. Matrix Storage We could try and take advantage of the banded nature of the system, but a more general solution is the adoption of a sparse matrix storage strategy. We could try and take advantage of the banded nature of the system, but a more general solution is the adoption of a sparse matrix storage strategy.

38. Limitations of Finite Differences  Unfortunately, it is not easy to use finite differences in complex geometries.  While it is possible to formulate curvilinear finite difference methods, the resulting equations are usually pretty nasty.

39. Finite Element Method  The finite element method, while more complicated than finite difference methods, easily extends to complex geometries.  A simple (and short) description of the finite element method is not easy to give. PDE Weak Form Matrix System

40. Finite Element Method ( Variational Formulations )  Find u in test space H such that a(u,v) = f(v) for all v in H, where a is a bilinear form and f is a linear functional.  V(x,y) =  j V j  j (x,y), j = 1,…,n  I(V) =.5  j  j A ij V i V j -  j b i V i, i,j = 1,…,n  A ij =  a(   j,   j ), i = 1,…,n  B i =  f  j, i = 1,…,n  The coefficients V j are computed and the function V(x,y) is evaluated anyplace that a value is needed.  The basis functions should have local support (i.e., have a limited area where they are nonzero).

41. Time Stepping Methods  Standard methods are common: –Forward Euler (explicit) –Backward Euler (implicit) –Crank-Nicolson (implicit) θ = 0, Fully-Explicit θ = 1, Fully-Implicit θ = ½, Crank-Nicolson

42. Time Stepping Methods (cont.)  Variable length time stepping –Most common in Method of Lines (MOL) codes or Differential Algebraic Equation (DAE) solvers

43. Solving the System  The system may be solved using simple iterative methods - Jacobi, Gauss-Seidel, SOR, etc. Some advantages: Some advantages: - No explicit storage of the matrix is required - No explicit storage of the matrix is required - The methods are fairly robust and reliable - The methods are fairly robust and reliable Some disadvantages Some disadvantages - Really slow (Gauss-Seidel) - Really slow (Gauss-Seidel) - Really slow (Jacobi) - Really slow (Jacobi)

44. Solving the System  Advanced iterative methods (CG, GMRES) CG is a much more powerful way to solve the problem. Some advantages: Some advantages: –Easy to program (compared to other advanced methods) –Fast (theoretical convergence in N steps for an N by N system) Some disadvantages: Some disadvantages: –Explicit representation of the matrix is probably necessary –Applies only to SPD matrices

45. Multigrid Algorithm: Components  Residual  compute the error of the approximation  Iterative method/Smoothing Operator  Gauss-Seidel iteration  Restriction  obtain a ‘coarse grid’  Prolongation  from the ‘coarse grid’ back to the original grid

46. Residual Vector  The equation we are to solve is defined as:  So then the residual is defined to be:  Where u q is a vector approximation for u  As the u approximation becomes better, the components of the residual vector(r), move toward zero

47. Multigrid Algorithm: Components  Residual  compute the error of your approximation  Iterative method/Smoothing Operator  Gauss-Seidel iteration  Restriction  obtain a ‘coarse grid’  Prolongation  from the ‘coarse grid’ back to the original grid

48. Multigrid Algorithm: Components  Residual  compute the error of your approximation  Iterative method/Smoothing Operator  Gauss-Seidel iteration  Restriction  obtain a ‘coarse grid’  Prolongation  from the ‘coarse grid’ back to the original grid

49. The Restriction Operator  Defined as ‘half-weighted’ restriction.  Each new point in the courser grid, is dependent upon it’s neighboring points from the fine grid

50. Multigrid Algorithm: Components  Residual  compute the error of your approximation  Iterative method/Smoothing Operator  Gauss-Seidel iteration  Restriction  obtain a ‘coarse grid’  Prolongation  from the ‘coarse grid’ back to the original grid

51. The Prolongation Operator  The grid change is exactly the opposite of restriction

52. Prolongation vs. Restriction  The most efficient multigrid algorithms use prolongation and restriction operators that are directly related to each other. In the one dimensional case, the relation between prolongation and restriction is as follows:

53. Full Multigrid Algorithm 1.Smooth initial U vector to receive a new approximation U q 2. Form residual vector: R q =b -A U q 3. Restrict R q to the next courser grid  R q-1 4. Smooth Ae= R q-1 starting with e=0 to obtain e q-1 5.Form a new residual vector using: R q-1 = R q-1 -A e q-1 6. Restrict R 2 (5x? where ?  5) down to R 1 (3x? where ?  3) 7. Solve exactly for Ae= R 1 to obtain e 1 8. Prolongate e 1  e 2 & add to e 2 you got from restriction 9. Smooth Ae= R 2 using e 2 to obtain a new e 2 10. Prolongate e q-1 to e q and add to U q

54. Reference http://csep1.phy.ornl.gov/CSEP/PDE/PDE.htm l www.mgnet.org/www.ceprofs.tamu.edu/hchen/ www.cs.cmu.edu/~ph/859B/www/notes/multig rid.pdf www.cs.ucsd.edu/users/carter/260 www.cs.uh.edu/~chapman/teachpubs/slides04 -methods.ppt http://www.ccs.uky.edu/~douglas/Classes/cs5 21-s01/index.html

55. Parallelization HPC Issues in PDEs, Part 3 Danny Thorne, CS521, University of Kentucky, April 9, 2002

56. Parallel Computation Serious calculations today are mostly done on a parallel computer. The domain is partitioned into subdomains that may or may not overlap slightly. Goal is to calculate as many things in parallel as possible even if some things have to be calculated on several processors in order to avoid communication. ”Communication is the Darth Vader of parallel computing.”

57. Example: Original Mesh Consider solving a PDE on this grid. Suppose that only half of the nodes fit on a processor.

58. Example: Mesh on Two Processors Divide into two connected subsets and renumber within the subdomains. Communication occurs between neighbors that cross the processor boundary. Ghost points (or overlap) can reduce communication sometimes at the expense of extra computation. Computation is communication per word.

59. Mesh Decomposition Goals are to maximize interior while minimizing connections between subdomains. That is, minimize communication. Such decomposition problems have been studied in load balancing for parallel computation. Lots of choices: METIS, ParMETIS -- University of Minnesota.METISParMETIS PARTI -- University of Maryland,PARTI CHACO -- Sandia National Laboratories,CHACO JOSTLE -- University of Greenwich,JOSTLE PARTY -- University of Paderborn,PARTY SCOTCH -- Université Bordeaux,SCOTCH TOP/DOMDEC -- NAS at NASA Ames Research Center.TOP/DOMDEC

60. Graph Partitioning Software CHACO Bruce Hendrickson and Robert Leland Sandia National Laboratories.CHACO JOSTLE Chris Walshaw University of Greenwich.JOSTLE METIS George Karypis and Vipin Kumar University of Minnesota.METIS ParMETIS George Karypis, Kirk Schloegel, and Vipin Kumar University of Minnesota.ParMETIS PARTY Robert Preis and Ralf Diekmann University of PaderbornPARTY SCOTCH François Pellegrini Université BordeauxSCOTCH TOP/DOMDEC Horst D. Simon and Charbel Farhat NAS at NASA Ames Research CenterTOP/DOMDEC http://www.erc.msstate.edu/~bilderba/research/other_sites.html

61. Mesh Decomposition Quality of mesh decomposition has a highly significant effect on performance. Arriving at a “good” decomposition is a complex task. “Good” may be problem/architecture dependent. A wide variety of well-established methods exist. Several packages/libraries provide implementations of many of these methods. Major practical difficulty is differences in file formats. http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-11.html

62. Decomposition Goals Load balance. Distribute elements evenly across processors. Each processor should have equal share of work. Communication costs should be minimized. Minimize sub-domain boundary elements. Minimize number of neighboring domains. Distribution should reflect machine architecture. Communication versus calculation. Bandwidth versus latency. Note that optimizing load balance and communication cost simultaneously is an NP-hard problem. http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-13.html

63. Static Mesh versus Dynamic Mesh Static mesh Decomposition need only be carried out once. Static decomposition may therefore be carried out as a preprocessing step, often done in serial. Dynamic mesh Decomposition must be adapted as underlying mesh changes. Dynamic decomposition therefore becomes part of the calculation itself and cannot be carried out solely as a pre- processing step. http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-14.html

64. Piston

65. Rod Impact Notice that the mesh changes dynamically. http://www.amtec.com/plotgallery/animation/movies/t-rod.gif

66. Frisbee http://www.llnl.gov/CASC/Overture/henshaw/figures/pib.mpg Need a dynamic mesh for this?

67. Dynamic Decomposition To redistribute a mesh A new decomposition must be found and required changes made (shuffling of data between processors). The time taken may outweigh the benefit gained so an assessment of whether it is worthwhile is needed. The decomposition must run in parallel and be fast. Take into account the old decomposition. Make minimal changes. http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-15.html

68. Dual Graphs To make a decomposition we need A representation of the basic entities being distributed (e.g. the elements, in the case of a FE mesh), An idea of how communication takes place between them. A dual graph, based on the mesh, fills this role. Vertices in the graph represent the entities (elements). Edges in the graph represent communication. Graph depends on how data is transferred. For finite elements it could be via nodes, edges or faces. So a single mesh can have more than one dual graph. Edges (communications) or vertices (calculations) may be weighted. http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-16.html

69. 2D Example http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-17.html

70. Graph Partitioning This problem occurs in several applications. VLSI circuit layout design, Efficient orderings for parallel sparse matrix factorization, Mesh decomposition. This problem has been extensively studied. A large body of literature already exists. Much work is still being produced in the field. http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-18.html

71. Graph Partitioning Given a graph we want to partition it into parts such that: Each part has roughly the same number of vertices, Number of edges that cross partitions is minimized. http://www-users.cs.umn.edu/~karypis/publications/Talks/multi-constraint/sld002.htm

72. Mesh Decomposition View mesh decomposition as having two aspects: Partitioning a graph, Mapping the sub-domains to processors. Algorithms may Address both issues, Or simplify the problem by ignoring the latter. Whether this is an issue depends on target machine. With very fast communications, e.g. the T3D, it's probably less important. On a cluster of workstations it may be absolutely critical. http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-22.html

73. Generalization

74. Decomposition Algorithms Global methods Direct k-way partitioning. Recursive application of some -way technique, e.g. bisection. Local refinement techniques Incrementally improve quality of an existing decomposition. Multi-level variants Work with coarse approximation of graph to increase performance. Hybrid techniques Using various combinations of above. http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-24.html

75. Examples Simple, direct k-way algorithms Random, Scattered & Linear Bandwidth Reduction The Greedy Algorithm Optimization algorithms Simulated Annealing Chained Local Optimization Genetic Algorithms Geometry based recursive algorithms Coordinate Partitioning Inertial Partitioning http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-25.html

76. Examples Graph based recursive algorithms Layered Partitioning Spectral Partitioning Local refinement algorithms Kernighan and Lin Jostle Multi-level and hybrid variants Multi-level Kernighan and Lin Partitioning Multi-level Spectral RQL / Symmlq Partitioning http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/MeshDecomp-26.html

77. Space Filling Curves Locality preserving. Each point lies a unique distance along the curve.

78. Space Filling Curves Optimal load balance. Subdomain boundaries are sub-optimal. Recall: Optimizing load and comm is NP-hard.

79. Space Filling Curves The main advantages of this partition method are: It is fast compared to graph partitioning heuristics, It runs in parallel, It requires no administration and no storage of processor neighborhoods. The knowledge of the separators is enough to compute where to find a node and which processor to ask for it.

80. Parallel PDE Tools Parallel ELLPACK -- Purdue UniversityParallel ELLPACK POOMA -- Los Alamos National LaboratoryPOOMA Overture -- Lawrence Livermore National LaboratoryOverture PADRE – LLNL, LANLPADRE PETSc -- Argonne National LaboratoryPETSc Unstructured Grids -- University of HeidelbergUnstructured Grids

81. Links A Software Framework for Easy Parallelization of PDE Solvers -- http://www.ifi.uio.no/~xingca/PCFD2000/Slides/pcfd2000/http://www.ifi.uio.no/~xingca/PCFD2000/Slides/pcfd2000/ Numerical Objects Online -- http://www.nobjects.com/Diffpack/http://www.nobjects.com/Diffpack/ Xing Cai's home page at University of Oslo -- http://www.ifi.uio.no/~xingca/http://www.ifi.uio.no/~xingca/ Related Grid Decomposition Sites -- http://www.erc.msstate.edu/~bilderba/research/other_sites.htmlhttp://www.erc.msstate.edu/~bilderba/research/other_sites.html Domain Decomposition Methods -- http://www.ddm.org/http://www.ddm.org/ People working on Domain Decomposition -- http://www.ddm.org/people.htmlhttp://www.ddm.org/people.html Unstructured Mesh Decomposition -- http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/http://www.epcc.ed.ac.uk/epcc-tec/documents/meshdecomp-slides/ HPCI Seminar: Parallel Sparse Matrix Solvers -- http://www.epcc.ed.ac.uk/computing/research_activities/HPCI/sparse_seminar.html http://www.epcc.ed.ac.uk/computing/research_activities/HPCI/sparse_seminar.html METIS: Family of Multilevel Partitioning Algorithms -- http://www-users.cs.umn.edu/~karypis/metis/index.htmlhttp://www-users.cs.umn.edu/~karypis/metis/index.html Multilevel Algorithms for Multi-Constraint Graph Partitioning -- http://www-users.cs.umn.edu/~karypis/publications/Talks/multi-constraint/sld001.htm http://www-users.cs.umn.edu/~karypis/publications/Talks/multi-constraint/sld001.htm Structured Versus Unstructured Meshes -- http://www.epcc.ed.ac.uk/epcc-tec/courses/images/Meshes.gifhttp://www.epcc.ed.ac.uk/epcc-tec/courses/images/Meshes.gif Domain Decomposition Methods for elliptic PDEs -- http://www.cs.purdue.edu/homes/mav/projects/dom_dec.htmlhttp://www.cs.purdue.edu/homes/mav/projects/dom_dec.html Multiblock Parti library -- http://www.cs.umd.edu/projects/hpsl/compilers/base_mblock.htmlhttp://www.cs.umd.edu/projects/hpsl/compilers/base_mblock.html Chaco: Software for Partitioning Graphs -- http://www.cs.sandia.gov/~bahendr/chaco.htmlhttp://www.cs.sandia.gov/~bahendr/chaco.html A Portable and Efficient Parallel Code for Astrophysical Fluid Dynamics -- http://astro.uchicago.edu/Computing/On_Line/cfd95/camelse.html http://astro.uchicago.edu/Computing/On_Line/cfd95/camelse.html Links for Graph Partitioning -- http://rtm.science.unitn.it/intertools/graph-partitioning/links.htmlhttp://rtm.science.unitn.it/intertools/graph-partitioning/links.html A Multilevel Algorithm for Partitioning Graphs -- http://www.chg.ru/SC95PROC/509_BHEN/SC95.HTMhttp://www.chg.ru/SC95PROC/509_BHEN/SC95.HTM Multigrid Workbench: The Algorithm -- http://www.mgnet.org/mgnet/tutorials/xwb/mgframe.htmlhttp://www.mgnet.org/mgnet/tutorials/xwb/mgframe.html Domain DecompositionParallelization of Mesh Based Applications -- http://www.hlrs.de/organization/par/par_prog_ws/online/DomainDecomposition/index.htm http://www.hlrs.de/organization/par/par_prog_ws/online/DomainDecomposition/index.htm Parallel ELLPACK Elliptic PDE Solvers -- http://www.cs.purdue.edu/homes/markus/research/pubs/pde-solvers/paper.htmlhttp://www.cs.purdue.edu/homes/markus/research/pubs/pde-solvers/paper.html POOMA -- http://www.acl.lanl.gov/pooma/http://www.acl.lanl.gov/pooma/ Overture -- http://www.llnl.gov/CASC/Overture/http://www.llnl.gov/CASC/Overture/ PADRE: A Parallel Asynchronous Data Routing Environment -- http://www.c3.lanl.gov:80/~kei/iscope97.1/iscope97.1.htmlhttp://www.c3.lanl.gov:80/~kei/iscope97.1/iscope97.1.html PETSc: The Portable, Extensible Toolkit for Scientific Computation -- http://www-fp.mcs.anl.gov/petsc/http://www-fp.mcs.anl.gov/petsc/ UG Home Page -- http://cox.iwr.uni-heidelberg.de/~ug/http://cox.iwr.uni-heidelberg.de/~ug/ Space-filling Curves -- http://mrl.nyu.edu/~dzorin/sig99/zorin3/sld036.htmhttp://mrl.nyu.edu/~dzorin/sig99/zorin3/sld036.htm Separate IMAGE for Basic foil 30 Two Space Filling Curves -- http://www.npac.syr.edu/users/gcf/cps615gemoct98/foilsepimagedir/030IMAGE.html http://www.npac.syr.edu/users/gcf/cps615gemoct98/foilsepimagedir/030IMAGE.html Using Space-filling Curves for Multi-dimensional Indexing -- http://www.dcs.bbk.ac.uk/~jkl/BNCOD2000/slides.htmlhttp://www.dcs.bbk.ac.uk/~jkl/BNCOD2000/slides.html The origins of fractals -- http://plus.maths.org/issue6/turner1/http://plus.maths.org/issue6/turner1/ Adaptive Parallel Multigrid Methods -- http://wissrech.iam.uni-bonn.de/research/projects/zumbusch/hash.htmlhttp://wissrech.iam.uni-bonn.de/research/projects/zumbusch/hash.html