Optimization

How to Optimize Code Conventional Wisdom: Don't do it (For experts only) Don't do it yet

Why Not??? You can't optimize something that doesn't work Biggest improvement is picking the right algorithm 90 / 10 rule 90% of the time is spent in 10% of the code Need to be able to verify speedup Optimized code is usually: Less readable/maintainable More complex/brittle

Code Tricks Prefer ints to floats Minimize float->int conversions Use multiplication instead of division

Strength Reductions Avoid costlier operations (divide/multiply) by using cheaper ones (add/subtract/shift) int total = 0; for (i = 0; i < 1000; i++) total += i * 3; int total = 0; for (i = 0; i < 3000; i+=3) total += i;

Fancy Tricks Make if the common case: If path usually gets unconditional branch Static branch prediction usually skips forward branches Else relies on conditional branch

Fancy Tricks Loop unrolling : Reducing iterations by increasing work done per step Less Overhead Gives compiler/processor more to rearrange for (i = 1; i <= 30; i++) a[i] = a[i] + b[i] * c; for (i = 1; i <= 30; i+=3) { a[i] = a[i] + b[i] * c; a[i+1] = a[i+1] + b[i+1] * c; a[i+2] = a[i+2] + b[i+2] * c; }

Fancy Tricks Loop fusion: Combine loops doing unrelated jobs to avoid overhead Will they fight over cache??? for (i = 0; i < N; i++) C[i] = A[i] + B[i]; D[i] = E[i] + C[i]; for (i = 0; i < N; i++) C[i] = A[i] + B[i]; D[i] = E[i] + C[i]; }

Fancy Tricks Loop peeling: Break up loops to avoid branches within them for (i = 1; i <= N; i++) { if (i==1) A[i] = 0; else if (i == N) A[i] = N; else A[i] = A[i] + 8; } A[1] = 0; for (i = 2; i < N; i++) A[i] = A[i] + 8; A[N] = N;

How To Optimize - Branching Pipelined processors HATE branch miss- predictions 15+ cycle penalty

How To Optimize - Branching May be able to rearrange data for consistent branching Some compilers can use runtime profiles to reorder code Come compilers allow "expect" – hints about how to order instructions

How To Optimize - Branching Maybe better off without it:

How To Optimize - Branching Maybe better off without it:

How To Optimize - Cache Cache: Modern CPU's live/die by cache Like contiguous data Avoid repeated swapping between multiple items https://www.youtube.com/watch?v=rX0ItVEVjHc&start=1555 25:00-35:00 and beyond

How to Optimize Right Rules Start with clean, working code and good tests Test any optimizations If it doesn't help, take it out

Compiler Optimization Compilers can Eliminate unneeded code Do strength reductions Unroll loops https://godbolt.org/g/z3OUvL http://goo.gl/rvJ7rq Compiler likes specific information Use constants Make variables as local as possible

Samples O0 : Faithful (but efficient) line by line translation O1 : Try to do work in registers instead of stack frame Remove obviously worthless code O2 : Aggressively do work ahead of time

Samples C++ Code: