1. Jens Krüger Technische Universität München
Linear Algebra on GPUs
Jens Krüger Technische Universität München

2. Why LA on GPUs? Why should we care about Linear Algebra at all?
Use LA to solve PDEs
solving PDEs can increase realism for VR, Education, Simulations, Games, …

3. Why Linear Algebra on GPUs?
… and why do it on the GPU?
The GPU is a fast streaming processor
LA operations are easily “streamable”
The result of the computation is already on the GPU and ready for display

4. Getting started … Computer graphics applications GPU as workhorse
Visual simulation
Visual computing
Education and Training
Basis linear algebra operators
General linear
algebra package
GPU as workhorse
for numerical computations
High bandwidth
Parallel computing
Programmable GPUs

5. Representation Vector representation 2D textures best we can do
Per-fragment vs. per-vertex operations
High texture memory bandwidth
Read-write access, dependent fetches 1 N 1 N

6. Representation (cont.)
Dense Matrix representation
treat a dense matrix as a set of column vectors
again, store these vectors as 2D textures
Matrix N i
N Vectors
... N 1 i
N 2D-Textures
... 1 i N

7. Representation (cont.)
Banded Sparse Matrix representation
treat a banded matrix as a set of diagonal vectors
Matrix
2 Vectors N 1 2 i
2 2D-Textures 1 2 N N

8. Representation (cont.)
Banded Sparse Matrix representation
combine opposing vectors to save space i
Matrix
2 Vectors N 1 2
2 2D-Textures 1 2 N
N-i N

9. Operations Vector-Vector Operations Reduced to 2D texture operations
Coded in vertex/fragment shaders
return tex0 OP tex1

10. Operations (cont.) Vector-Vector Operations
Reduce operation for scalar products
original Texture
... st
1 pass
...
2 pass nd
...
Reduce m x n region
in fragment shader

11. Operations (cont.)
In depth example: Vector / Banded-Matrix Multiplication A b x =

12. Example (cont.)
Vector / Banded-Matrix Multiplication A b A b x =

13. Example (cont.) Compute the result in 2 Passes Pass 1: b A x =
multiply

14. Example (cont.) Compute the result in 2 Passes Pass 2: b A b‘ x =
shift x
multiply

15. Representation (cont.)
Random sparse matrix representation
Textures do not work
Splitting yields highly fragmented textures
Difficult to find optimal partitions
Idea: encode only non-zero entries in vertex arrays
Column as TexCoord 1-4 N
Matrix
Result
Vector
Matrix = ×
Values as TexCoord 0
Vertex Array 1
Vertex Array 2
Vector
Result
Tex0
Pos
Pos
Tex1-4
Tex0
Pos
Pos
Tex1-4
Tex1-4 =
Row Index as Position

16. Building a Framework Presented so far: representations on the GPU for
single float values
vectors
matrices
dense
banded
random sparse
operations on these representations
add, multiply, reduce, …
upload, download, clear, clone, …

17. Building a Framework Example: CG
Encapsulate into Classes for more complex algorithms
Example use: Conjugate Gradient Method, complete source:
void clCGSolver::solveInit() {
m_clMatrix->matrixVectorOp(CL_SUB,m_clvX,m_clvB,m_clvR); // R = A*x-b m_clvR->addVector(m_clvR,m_clvR,-1.0f,0.0f); // R = -R
m_clvR->addVector(m_clvR,m_clvP,1.0f,0.0f); // P = R
m_clvR->reduceAdd(m_clvR, clfRho); // rho = sum(R*R); }
void clCGSolver::solveIteration() {
m_clMatrix->matrixVectorOp(CL_NULL,m_clvP,NULL,m_clvQ); // Q = Ap;
m_clvP->reduceAdd(m_clvQ,clfTemp); // temp = sum(P*Q);
clfRho->divZ(clfTemp,clfAlpha); // alpha = rho/temp;
m_clvX->addVector(m_clvP,m_clvX,1,clfAlpha); // X = X + alpha*P
m_clvR->subtractVector(m_clvQ,m_clvR,1,clfAlpha); // R = R - alpha*Q
m_clvR->reduceAdd(m_clvR,clfNewRho); // newrho = sum(R*R);
clfNewRho->divZ(clfRho,clfBeta); // beta = newrho/rho
m_clvR->addVector(m_clvP,m_clvP,1,clfBeta); // P = R+beta*P;
clFloat *temp; temp=clfNewRho;
clfNewRho=clfRho; clfRho=temp; // swap rho and newrho pointer
void clCGSolver::solve(int maxiter) {
solveInit();
for (int i = 0;i< maxiter;i++) solveIteration();
int clCGSolver::solve(float rhoTresh, int maxiter) {
solveInit(); clfRho->clone(clfNewRho);
for (int i = 0;i< maxiter && clfNewRho.getData() > rhoTresh;i++) solveIteration();
return i;

18. Building a Framework Example: Jacobi
The Jacobi method for a problem can be expressed with matrices as follows: U D L A b x + = ×
where ( ) 1 k x U L b D × + - =
If the Matrix is stored in the diagonal form the matrix D-1 can be computed on the fly by inverting the diagonal elements during the vector-vector product. So one Jacobi step becomes one matrix-vector product, one vector-vector product and one vector subtract.

19. Example 1 2D Waves (explicit)
usually you would compute:
you need to write a custom shader for this filter
think about this as a matrix-vector operation!

20. Example 2 2D Waves (implicit)
2D wave equation
Finite difference discretization
Implicit Crank-Nicholson scheme
Key Idea: Rewrite as Matrix-Vector Product

21. Example 2: 2D wave equation (cont.)

22. Navier Stokes on GPUs

23. The Equations ) ( Re 1 = Ñ - + × V div p f v t u d
The Navier-Stokes Equations for 2D:
Advection ) ( Re 1 2 = Ñ - + × V
div p f v t u y x d
Diffusion
Pressure
External Forces
Zero Divergence

24. NSE Discretization ( ) Diffusion Advection Pressure y p g v x uv t ¶ -+ ÷ ø ö ç è æ = 2 Re 1
Diffusion
Advection
Pressure ( ) 2 1 , y v x j i d - + = ú û ù ê ë é ( ) ÷ ø ö ç è æ - + = ú û ù ê ë é 2 1 , j i v u x uv d a y p j i d , 1 - = ú û ù ê ë é +

25. Navier-Stokes Equations (cont.)
Rewrite the Navier Stokes Equations
where
now F and G can be computed ( ) 1 , - = + p y t G v j i d x F u ( ) ; Re 1 , 2 ÷ ø ö ç è æ + ú û ù ê ë é - = y j i g v x uv t G u F

26. Navier-Stokes Equations (cont.)
Problem: Pressure is still unknown!
From derive:
..to get this Poisson Equation: ( ) 1 , - = + p x t F u j i d ( ) 1 , - = + p y t G v j i d ) ( = V
div u t + 1 v t + 1 G t 2 p t + 1 F t 2 p t + 1 = + = - D t + - D t x y x x 2 y y 2 ( + ) - ( + ) + ( + ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) p n 1 2 p n 1 p n 1 p n + 1 - 2 p n + 1 + p n + 1 æ 1 F n - F n G n - G n ö i + 1 , j i , j i - 1 , j + i , j + 1 i , j i , j - 1 = ç i , j i - 1 , j + i , j i , j - 1 ÷ ( ) ( ) ç ÷ d x 2 d y 2 d t d è x d y ø

27. Navier-Stokes Equations (cont.)
The basic algorithm:
Compute F and G
add external forces
advect
diffuse
solve the Poisson equation
update velocities
easy 
semi-lagrange [Stam 1999]
explicit
use the CG solver
subtract pressure gradient

28. Multigrid on GPUs

29. Multigrid “in english”
do a few Jacobi/Gauss-Seidel iterations on the fine grid
Jacobi/G-S eliminate high frequencies in the error
conjugate gradient does not have this property !!!
compute the residuum of the last approximaton
propagate this residuum to the next coarser grid
can be done by the means of a matrix multiplication
solve the coarser grid for the absolute error
for the solution you can use another multigrid step
backpropagate the error to the finer grid
can be done by another matrix multiplication (transposed matrix from 3.)
use the error to correct the first approximation
do another few Jacobi/Gauss-Seidel iterations to remove noise introduced by the propagation steps

30. ( ) Multigrid “in greek” { x b = × ¢ - + 4 3 1 h A e I r 2
Consider this problem: x is the solution vector of a set of linear equations
this equation holds for current approximation x’ with the error e
rearranging leads to the residual equation with residuum r
now multiply both sides with a non quadratic interpolation matrix and replace the error with an error times the transposed interpolation matrix
Let A2h be the product of the Interpolation Matrix, A and the transposed Interpolation matrix.
Finally we end up with a new set of linear equations with only half the size in every dimension of the old one. We solve this set for the error at this grid level. Propagating the error the above steps we can derive the error for the large system and use it to correct out approximation ( ) { h A e I r x b 2 = × - + 4 3 1

31. Multigrid (cont.)
„fine grid“, „coarse grid“ only makes sense if the problem to solve corresponds to a grid 
this is the case in the finite difference methods described before 
need to find an “interpolation matrix” for the propagation step to generate the coarser grid (for instance simple linear interpolation)
need an “extrapolation matrix” to move from the coarse to the fine grid
the coarse grid matrices can be pre-computed

32. Multigrid on GPUs Observation:
you only need matrix-vector operations and a “Jacobi smoother” to do multigrid
to put it on the GPU simply use the matrix-vector operations from the framework
Improvement:
to do the interpolation an extrapolation steps we can use the fast bilinear interpolation hardware of the GPU instead of a vector-matrix multiplication

33. Multigrid on GPUs (cont.)
We can embed the new “rescale” easily into the vector representation by simply rendering one textured quad.
Vector Texture
Render Target
Interpolation
Extrapolation
Render Target
Vector Texture

34. Selected References
Chorin, A.J., Marsden, J.E. A Mathematical Introduction to Fluid Mechanics. 3rd ed. Springer. New York, 1993
Briggs, Henson, McCormick A Multigrid Tutorial, 2nd ed. siam, ISBN
Acton Numerical Methods that Work, The Mathematical Association of America ISBN
Krüger, J. Westermann, R. Linear algebra operators for GPU implementation of numerical algorithms, In Proceedings of SIGGRAPH 2003, ACM Press / ACM SIGGRAPH
Bolz , J., Farmer, I., Grinspun, E., Schröder, P. Sparse Matrix Solvers on the GPU: Conjugate Gradients and Multigrid, In Proceedings of SIGGRAPH 2003, ACM Press / ACM SIGGRAPH
Hillesland, K. E. Nonlinear Optimization Framework for Image-Based Modeling on Programmable Graphics Hardware, In Proceedings of SIGGRAPH 2003, ACM Press / ACM SIGGRAPH
Fedkiw, R., Stam, J. and Jensen, H.W. Visual Simulation of Smoke. In Proceedings of SIGGRAPH , ACM Press / ACM SIGGRAPH
Stam, J. Stable Fluids. In Proceedings of SIGGRAPH 1999, ACM Press / ACM SIGGRAPH,
Harris, M., Coombe, G., Scheuermann, T., and Lastra, A. Physically-Based Visual Simulation on Graphics Hardware.. Proc SIGGRAPH / Eurographics Workshop on Graphics Hardware 2002.