Optimization

How to Optimize Code Conventional Wisdom: 1.Don't do it 2.(For experts only) Don't do it yet

Why Not??? You can't optimize something that doesn't work Biggest improvement is picking the right algorithm 90 / 10 rule – 90% of the time is spent in 10% of the code Need to be able to verify speedup Optimized code is usually: – Less readable/maintainable – More complex/brittle

Code Tricks Prefer ints to floats Minimize float->int conversions Use multiplication instead of division Put common branches first in if/else sequences

Fancy Tricks Loop unrolling : – Reducing iterations by increasing work done per step Less Overhead Gives compiler/processor more to rearrange for (i = 1; i <= 30; i+=3) { a[i] = a[i] + b[i] * c; a[i+1] = a[i+1] + b[i+1] * c; a[i+2] = a[i+2] + b[i+2] * c; } for (i = 1; i <= 30; i++) a[i] = a[i] + b[i] * c;

Fancy Tricks Loop fusion: – Combine loops doing unrelated jobs to avoid overhead Will they fight over cache??? for (i = 0; i < N; i++) C[i] = A[i] + B[i]; for (i = 0; i < N; i++) D[i] = E[i] + C[i]; for (i = 0; i < N; i++) C[i] = A[i] + B[i]; D[i] = E[i] + C[i]; }

Fancy Tricks Loop peeling: – Break up loops to avoid branches within them A[1] = 0; for (i = 2; i < N; i++) A[i] = A[i] + 8; A[N] = N; for (i = 1; i <= N; i++) { if (i==1) A[i] = 0; else if (i == N) A[i] = N; else A[i] = A[i] + 8; }

Strength Reductions Avoid costlier operations (divide/multiply) by using cheaper ones (add/subtract/shift) int total = 0; for (i = 0; i < 1000; i++) total += i * 3; int total = 0; for (i = 0; i < 3000; i+=3) total += i;

Compiler Optimization Compilers can – Eliminate unneeded code – Do strength reductions – Unroll loops Compiler likes specific information – Use constants – Make variables as local as possible

Compiler Optimization gcc has multiple optimizations levels – 0 = no optimization – 1 = optimize, but don't spend too much time – 2 = spend lots of time optimizing – 3 = aggressively optimize for speed and allow code size to increase if need be Activate with command line flag like -O2 Oh-2

Samples C++ Code:

Samples O0 : Faithful (but efficient) line by line translation O1 : – Try to do work in registers instead of stack frame – Remove obviously worthless code O2 : – Aggressively do work ahead of time

How To Optimize - Branching Pipelined processors HATE branch mispredications – 15+ cycle penalty

How To Optimize - Branching May be able to rearrange data for consistent branching Some compilers can use runtime profiles to reorder code Come compilers allow "expect" – hints about how to order instructions

How To Optimize - Branching Maybe better off without it:

How To Optimize - Branching Maybe better off without it:

How To Optimize - Cache Cache: – Modern CPU's live/die by cache Like contiguous data Avoid repeated swapping between multiple items

Bad Situations for Cache Data with poor locality – Complex object oriented programming Large 2D arrays traversed in column major order… Row Major Access Col Major Access

How to Optimize Right Rules 1.Start with clean, working code and good tests 2.Test any optimizations 3.If it doesn't help, take it out