1. Optimization

2. How to Optimize Code Conventional Wisdom: 1.Don't do it 2.(For experts only) Don't do it yet

3. Why Not??? You can't optimize something that doesn't work Biggest improvement is picking the right algorithm 90 / 10 rule – 90% of the time is spent in 10% of the code Need to be able to verify speedup Optimized code is usually: – Less readable/maintainable – More complex/brittle

4. Code Tricks Prefer ints to floats Minimize float->int conversions Use multiplication instead of division Put common branches first in if/else sequences

5. Fancy Tricks Loop unrolling : – Reducing iterations by increasing work done per step Less Overhead Gives compiler/processor more to rearrange for (i = 1; i <= 30; i+=3) { a[i] = a[i] + b[i] * c; a[i+1] = a[i+1] + b[i+1] * c; a[i+2] = a[i+2] + b[i+2] * c; } for (i = 1; i <= 30; i++) a[i] = a[i] + b[i] * c;

6. Fancy Tricks Loop fusion: – Combine loops doing unrelated jobs to avoid overhead Will they fight over cache??? for (i = 0; i < N; i++) C[i] = A[i] + B[i]; for (i = 0; i < N; i++) D[i] = E[i] + C[i]; for (i = 0; i < N; i++) C[i] = A[i] + B[i]; D[i] = E[i] + C[i]; }

7. Fancy Tricks Loop peeling: – Break up loops to avoid branches within them A[1] = 0; for (i = 2; i < N; i++) A[i] = A[i] + 8; A[N] = N; for (i = 1; i <= N; i++) { if (i==1) A[i] = 0; else if (i == N) A[i] = N; else A[i] = A[i] + 8; }

8. Strength Reductions Avoid costlier operations (divide/multiply) by using cheaper ones (add/subtract/shift) int total = 0; for (i = 0; i < 1000; i++) total += i * 3; int total = 0; for (i = 0; i < 3000; i+=3) total += i;

9. Compiler Optimization Compilers can – Eliminate unneeded code – Do strength reductions – Unroll loops http://goo.gl/TK2k06 http://goo.gl/rvJ7rq http://goo.gl/TK2k06 http://goo.gl/rvJ7rq Compiler likes specific information – Use constants – Make variables as local as possible

10. Compiler Optimization gcc has multiple optimizations levels – 0 = no optimization – 1 = optimize, but don't spend too much time – 2 = spend lots of time optimizing – 3 = aggressively optimize for speed and allow code size to increase if need be Activate with command line flag like -O2 Oh-2

11. Samples C++ Code:

12. Samples O0 : Faithful (but efficient) line by line translation O1 : – Try to do work in registers instead of stack frame – Remove obviously worthless code O2 : – Aggressively do work ahead of time

13. How To Optimize - Branching Pipelined processors HATE branch mispredications – 15+ cycle penalty

14. How To Optimize - Branching May be able to rearrange data for consistent branching Some compilers can use runtime profiles to reorder code Come compilers allow "expect" – hints about how to order instructions

15. How To Optimize - Branching Maybe better off without it:

16. How To Optimize - Branching Maybe better off without it:

17. How To Optimize - Cache Cache: – Modern CPU's live/die by cache Like contiguous data Avoid repeated swapping between multiple items https://www.youtube.com/watch?v=rX0ItVEVjHc&start=1555

18. Bad Situations for Cache Data with poor locality – Complex object oriented programming Large 2D arrays traversed in column major order… 012 0 768276 1 838494 2 889383 012345678 768276838494889383 Row Major Access Col Major Access

19. How to Optimize Right Rules 1.Start with clean, working code and good tests 2.Test any optimizations 3.If it doesn't help, take it out