1. Algorithms: Design and Analysis
, Semester 2,
0. Preliminaries
Please ask
questions
Who I am: Andrew Davison WiG Lab
Objective
to give some background on the course

2. 1. What is a Algorithm?
An algorithm is a finite set of unambiguous instructions for solving a problem.
An algorithm is correct if on all legitimate inputs, it outputs the right answer in a finite amount of time
Can be expressed as
pseudocode
flow charts
text in a natural language (e.g. English)
computer code

3. The Importance of Algorithms
Their impact is broad and far-reaching.
Internet. Web search, packet routing, distributed file sharing, ...
Biology. Human genome project, protein folding, ...
Computers. Circuit layout, file system, compilers, ...
Computer graphics. Movies, video games, virtual reality, ...
Security. Cell phones, e-commerce, voting machines, ...
Multimedia. MP3, JPG, DivX, HDTV, face recognition, ...
Social networks. Recommendations, news feeds, advertisements, ...
Physics. N-body simulation, particle collision simulation, ...

4. The Top 10 Algorithms of the 20th Century
Ten algorithms having "the greatest influence on the development and practice of science and engineering in the 20th century".
Dongarra and Sullivan Top Ten Algorithms of the Century Computing in Science and Engineering January/February 2000
Barry Cipra The Best of the 20th Century: Editors Name Top 10 Algorithms SIAM News Volume 33, Number 4, May 2000

5. What are the Top 10?
1946: The Metropolis (Monte Carlo) Algorithm. Uses random processes to find answers to problems that are too complicated to solve exactly.
1947: Simplex Method for Linear Programming.  A fast technique for maximizing or minimizing a linear function of several variables, applicable to planning and decision-making.
1950: Krylov Subspace Iteration Method.  A technique for rapidly solving the linear equations that are common in scientific computation.

6. 1951: The Decompositional Approach to Matrix Computations
1951: The Decompositional Approach to Matrix Computations. A collection of techniques for numerical linear algebra.
1957: The Fortran Optimizing Compiler. 
1959: QR Algorithm for Computing Eigenvalues. A crucial matrix operation made swift and practical. Application areas include computer vision, vibration analysis, data analysis.
1962: Quicksort Algorithm. 
We will look at this.

7. 1965: Fast Fourier Transform (FFT)
1965: Fast Fourier Transform (FFT). It breaks down waveforms (like sound) into periodic components. Used in many different areas (e.g. digital signal processing , solving partial differential equations, fast multiplication of large integers.)
1977: Integer Relation Detection. A fast method for finding simple equations that explain collections of data.
1987: Fast Multipole Method. Deals with the complexity of n-body calculations. It is applied in problems ranging from celestial mechanics to protein folding.

8. 2. Some Algorithm Types Simple recursive algorithms Backtracking
Brute force
Divide and conquer
Dynamic programming
Greedy algorithms
Randomized algorithms

9. Simple Recursive A simple recursive algorithm: Examples:
Non-recursive base case
Recurs with a simpler subproblem
Examples:
Count the number of occurrence of an element in a tree
Test if a value occurs in a list
Fibonnaci number calculation
Several of the other algorithm types use recursion in more complex ways

10. Fibonnaci numbers The Fibonacci sequence:
1, 1, 2, 3, 5, 8, 13, 21, 33, ...
Find the nth Fibonacci number:
fib(int n)
if (n <= 1)
return 1;
else return fib(n-1) + fib(n-2);

11. lots of repeated work,
that only gets worse
for bigger n
Execution of fib(5):

12. Backtracking
Backtracking algorithms use a depth-first recursive search
involves choice (non-determinism)
backtracking means "go back to where you came from"
Examples:
search a maze
color a map with no more than four colors

13. Brute Force
A brute force algorithm tries all possibilities until a solution is found.
often this is implemented using backtracking
Brute force algorithms usually have exponential running time (very large time, so very slow)
Various heuristics and optimizations can be used
heuristic means “a rule of thumb”
e.g. look for sub-optimal solutions (not the best), which can be calculated more quickly than the best one

14. Divide and Conquer A divide and conquer algorithm : Examples:
Divide the problem into smaller subproblems
Combine the solutions to the subproblems into a solution to the original problem
Examples:
quicksort
merge sort
binary search 2

15. Dynamic Programming
A dynamic programming algorithm remembers past results and uses them to find new results.
often use a table (2D array) to store results
memorization = remember results in a table
the problen contains repeating subproblems
since past results are stored, they can be reused instead of being calculated again

16. Fibonacci numbers Again
Finding the nth Fibonacci number involves lots of repeated work that can be avoided using memorization:

17. Greedy algorithms
At each step use the best solution you can get right now, and don't allow for backtracking to change the choice later
no backtracking makes the coding easier and faster
You hope that by choosing the best at each step, you will end up with the best combined final solution 2

18. Example: Counting Money
Count out some money using the fewest possible notes and coins
A greedy algorithm will take the largest possible note or coin at each step
Example: count out $6.39 using:
a $5 bill
a $1 bill // to make $6
a 25¢ coin // to make $6.25
a 10¢ coin // to make $6.35
four 1¢ coins // to make $6.39 3

19. Greedy Algorithms Can 'Fail'
“Krons” money come in 1, 7, and 10 coins
Count out 15 krons:
A 10 kron piece
Five 1 kron pieces, for a total of 15 krons
This requires 6 coins, but a better solution is two 7 kron pieces and one 1 kron piece (3 coins)
The greedy algorithm 'fails' because its final solution is not the best 4

20. Randomized Algorithms
A randomized algorithm uses a random number to make a choice during the computation
faster than calculating a choice
the choice may be just as good
Examples:
Quicksort randomly chooses a pivot
Factor a large number by choosing random numbers as possible divisors

21. 3. Analysis of Algorithms
The theoretical study of algorithm performance and resource usage. Performance isn't the only important things for code: • modularity • user-friendliness • correctness • programmer time • maintainability • simplicity • functionality • extensibility • robustness • reliability
This subject isn't about
these things.

22. Why study Algorithm Analysis?
• It help us to understand algorithm scalability. • Performance often draws the line between what is feasible and what is impossible. • Algorithmic mathematics provides a precise way to talk about program behavior. • The lessons of program performance generalize to other computing resources.

23. 4. Course Structure Running time of programs Backtracking
Divide-and-conquer; quicksort
Linear sorting
Hashing
Dynamic programming
Graph algorithms
search, shortest path
not minimum spanning trees, which was done in Discrete Maths
Heaps
P and NP problems

24. Meeting Times / Locations
Monday 9:00 – 10:30 IDL Wednesday 9:00 – 10:30 IDL
I want to change these times to be 3 classes/week, each of 1 hour.
Tell me your preferences.

25. Workload Mid-term exam: 35% (2 hours) Final exam: 45% (3 hours)
week 9
Final exam: 45% (3 hours)
weeks 17-18
Two exercises: 20% (2*10)
weeks 7-8 and weeks 15-16

26. Non-Attendence Penalty
I may take registration at the start of a class.
If someone is not there, they lose 1% (unless they have a good excuse).
A maximum of 10% can be lost
deducted from your final mark

27. Course Materials
All the handouts (and other materials) will be placed on-line at
Software.coe/Algorithms/
Print 6 slides-per-page, grayscale, and bring to class.

28. 5. Books
Intro to Java Programming, Comprehensive, 10th ed. Y. Daniel Liang
Topics
Generics, Collections (lists, stack, queues,sets, maps)
Sorting, searching
graph algorithms (2 chapters)

29. Algorithms, 4th ed. Robert Sedgewick, Kevin Wayne
Introduction to Algorithms, 3rd ed. Thomas Cormen, Charles Leiserson, Ronald Rivest, Clifford Stein
lots of resources online; see video section

30. การออกแบบและวิเคราะห์อัลกอริทึม ผู้แต่ง : สมชาย ประสิทธิ์จูตระกูล
โครงสร้างข้อมูลและอัลกอริทึม Data Structures and Algorithms)
ผู้แต่ง : สุธี พงศาสกุลชัย & ณัฐพงษ์ วารีประเสริฐ

31. Fun Overviews
Algorithms Unlocked Thomas H. Cormen MIT Press, March 2013
Nine Algorithms that Changed the Future John MacCormick Princeton University Press, 2011
search engine indexing, pagerank, public key cryptography, error-correcting codes, pattern recognition, data compression, databases, digital signatures, computablity

32. Algorithmic Puzzles Anany Levitin, Maria Levitin Oxford University Press, , 2011
Algorithmics: The Spirit of Computing David Harel, Yishai Feldman Addison-Wesley; 3 ed., 2004 (and Springer, 2012)

33. The New Turing Omnibus: Sixty-Six Excursions in Computer Science A. K
The New Turing Omnibus: Sixty-Six Excursions in Computer Science A. K. Dewdney Holt, 1993
66 short article; e.g. detecting primes, noncomputable functions, self-replicating computers, fractals, genetic algorithms, Newton-Raphson Method, viruses

34. 6. Videos MIT 6.046J / 18.410J Intro. to Algorithms, Fall 2005
original course website for Cormen book
video and slides side-by-side
category/introduction-to-algorithms
notes taken while watching the videos

35. Skiena's Algorithms Lectures
1997, 2007, 2012

36. Hi-tech Trek Royal Institution Christmas Lectures 2008
A hi-tech trek through the world of computer science by Professor Chris Bishop; aimed at school kids.
Lecture 1: Breaking the speed limit
Lecture 2: Chips with everything
Lecture 3: Ghost in the machine
Lecture 4: Untangling the web
Lecture 5: Digital intelligence
2008/
I have copies on a DVD

37. 7. Web Sites Algorithm Tutorials Algorithmy Algorithmist
d1=tutorials&d2=alg_index
Algorithmy
brief explanations and code
Algorithmist
explanations and code
Wikipedia page for an algorithm
e.g.