1. Chapter 3 The Efficiency of Algorithms
Algorithmic problem solving
Chapter 3
The Efficiency of Algorithms

2. Attributes of Algorithms
Are some algorithms better than others?
We expect correctness from our algorithms
Ease of understanding; Elegance
Analysis of Algorithms
Efficiency
Term used to describe an algorithm’s careful use of resources
Benchmarks
Useful for rating one machine against another and for rating how sensitive a particular algorithm is with respect to variations in input on one particular machine

3. Invitation to Computer Science, 5th Edition
Measuring Efficiency
Analysis of algorithms
The study of the efficiency of algorithms
An important part of computer science
Invitation to Computer Science, 5th Edition

4. Invitation to Computer Science, 5th Edition
Sequential Search
Search for NAME among a list of n names
Start at the beginning and compare NAME to each entry until a match is found
Invitation to Computer Science, 5th Edition

5. Invitation to Computer Science, 5th Edition
Figure 3.1 Sequential Search Algorithm
Invitation to Computer Science, 5th Edition

6. Invitation to Computer Science, 5th Edition
Figure 3.2 Number of Comparisons to Find NAME in a List of n Names Using Sequential Search
Invitation to Computer Science, 5th Edition

7. Order of Magnitude - Order n
Order of magnitude n
Anything that varies as a constant times n (and whose graph follows the basic shape of n)
Sequential search
An Θ(n) algorithm in both the worst case and the average case
Invitation to Computer Science, 5th Edition

8. Invitation to Computer Science, 5th Edition
Figure 3.3 Work = 2n
Invitation to Computer Science, 5th Edition

9. Invitation to Computer Science, 5th Edition
Figure 3.4 Work = cn for Various Values of c
Invitation to Computer Science, 5th Edition

10. Invitation to Computer Science, 5th Edition
Figure 3.5 Growth of Work = cn for Various Values of c
Invitation to Computer Science, 5th Edition

11. Invitation to Computer Science, 5th Edition
Selection Sort
Selection sort algorithm
Sorts in ascending order
Subtask within selection sort
Task of finding the largest number in a list
When selection sort algorithm begins
The largest-so-far value must be compared to all the other numbers in the list
If there are n numbers in the list, n – 1 comparisons must be done
Invitation to Computer Science, 5th Edition

12. Invitation to Computer Science, 5th Edition
Figure 3.6 Selection Sort Algorithm
Invitation to Computer Science, 5th Edition

13. Invitation to Computer Science, 5th Edition
Figure 3.7 Comparisons Required by Selection Sort
Invitation to Computer Science, 5th Edition

14. Selection Sort (continued)
Selection sort algorithm
Does n exchanges, one for each position in the list to put the correct value in that position
Space efficiency of the selection sort
Original list occupies n memory locations
Storage is needed for:
The marker between the unsorted and sorted sections
Keeping track of the largest-so-far value and its location in the list
Invitation to Computer Science, 5th Edition

15. Invitation to Computer Science, 5th Edition
Figure 3.8 An Attempt to Exchange the Values at X and Y
Invitation to Computer Science, 5th Edition

16. Invitation to Computer Science, 5th Edition
Figure 3.9 Exchanging the Values at X and Y
Invitation to Computer Science, 5th Edition

17. Order of Magnitude - Order n2
Order of magnitude n2, or Θ(n2)
An algorithm that does cn2 work for any constant c
Selection sort
An Θ(n2) algorithm (in all cases)
Sequential search
An Θ(n) algorithm (in the worst case)
Invitation to Computer Science, 5th Edition

18. Invitation to Computer Science, 5th Edition
Figure 3.10 Work 5 cn2 for Various Values of c
Invitation to Computer Science, 5th Edition

19. Invitation to Computer Science, 5th Edition
Figure 3.11 A Comparison of n and n2
Invitation to Computer Science, 5th Edition

20. Invitation to Computer Science, 5th Edition
Figure 3.12 For Large Enough n, 0.25n2 Has Larger Values Than 10n
Invitation to Computer Science, 5th Edition

21. Invitation to Computer Science, 5th Edition
Figure 3.13 A Comparison of Two Extreme Q(n2) and Q(n) Algorithms
Invitation to Computer Science, 5th Edition

22. Analysis of Algorithms
Data cleanup algorithms
The Shuffle-Left Algorithm
The Copy-Over Algorithm
The Converging-Pointers Algorithm
Invitation to Computer Science, 5th Edition

23. The Shuffle-Left Algorithm
Scans list from left to right
When a zero is found, copy each remaining data item in the list one cell to the left
Value of legit
Originally set to the length of the list
Is reduced by 1 every time a 0 is encountered
Invitation to Computer Science, 5th Edition

24. Invitation to Computer Science, 5th Edition
Figure 3.14 The Shuffle-Left Algorithm for Data Cleanup
Invitation to Computer Science, 5th Edition

25. The Shuffle-Left Algorithm (continued)
To analyze time efficiency:
Identify the fundamental units of work the algorithm performs
Copying numbers
Best case occurs when the list has no 0 values because no copying is required
Worst case occurs when the list has all 0 values
Invitation to Computer Science, 5th Edition

26. The Shuffle-Left Algorithm (continued)
Worst case
Occurs when the list has all 0 values
An Θ(n2) algorithm
Space-efficient
Only requires four memory locations to store the quantities n, legit, left, and right
Invitation to Computer Science, 5th Edition

27. The Copy-Over Algorithm
Scans the list from left to right, copying every legitimate (non-zero) value into a new list that it creates
Every list entry is examined to see whether it is 0
Every non-zero list entry is copied once
Best case
Occurs if all elements are 0
Θ(n) in time efficiency
No extra space is used

28. Invitation to Computer Science, 5th Edition
Figure 3.15 The Copy-Over Algorithm for Data Cleanup
Invitation to Computer Science, 5th Edition

29. The Copy-Over Algorithm (continued)
Worst case
Occurs if there are no 0 values in the list
Algorithm copies all n non-zero elements into the new list and doubles the space required
Θ(n) in time efficiency
Time/space tradeoff
You gain something by giving up something else
Invitation to Computer Science, 5th Edition

30. The Converging-Pointers Algorithm
Swap zero values from left with values from right until pointers converge in the middle
Best case
A list containing no 0 elements
Worst case
A list of all 0 entries
Θ(n) in time efficiency
Is space-efficient
Invitation to Computer Science, 5th Edition

31. Invitation to Computer Science, 5th Edition
Figure 3.16 The Converging-Pointers Algorithm for Data Cleanup
Invitation to Computer Science, 5th Edition

32. Invitation to Computer Science, 5th Edition
Figure 3.17 Analysis of Three Data Cleanup Algorithms
Invitation to Computer Science, 5th Edition

33. The Converging-Pointers Algorithm (continued)
In an Θ(n) algorithm:
The work is proportional to n
In an Θ(n2) algorithm:
The work is proportional to the square of n
Invitation to Computer Science, 5th Edition

34. Invitation to Computer Science, 5th Edition
Binary Search
Procedure
First looks for NAME at roughly the halfway point in list
If name equals NAME, search is over
If NAME comes alphabetically before name at halfway point, search is narrowed to the front half of list
If NAME comes alphabetically after name at halfway point, search is narrowed to the back half of the list
Algorithm halts when NAME is found
Invitation to Computer Science, 5th Edition

35. Invitation to Computer Science, 5th Edition
Figure 3.18 Binary Search Algorithm (list must be sorted)
Invitation to Computer Science, 5th Edition

36. Binary Search (continued)
Logarithm of n to the base 2
Number of times a number n can be cut in half and not go below 1
As n doubles:
lg n increases by only 1, so lg n grows much more slowly than n
Worst case and average case
Θ(lg n) comparisons
Works only on a list that has already been sorted

37. Invitation to Computer Science, 5th Edition
Figure 3.19 Binary Search Tree for a 7-Element List
Invitation to Computer Science, 5th Edition

38. Invitation to Computer Science, 5th Edition
Figure 3.20 Values for n and lg n
Invitation to Computer Science, 5th Edition

39. Invitation to Computer Science, 5th Edition
Figure 3.21 A Comparison of n and lg n
Invitation to Computer Science, 5th Edition

40. Invitation to Computer Science, 5th Edition
Pattern-Matching
Usually involves a pattern length that is short compared to the text length
That is, when m is much less than n
Best case
Θ(n)
Worst case
Θ(m * n)
Invitation to Computer Science, 5th Edition

41. When Things Get Out of Hand
Polynomially bound algorithms
Work done is no worse than a constant multiple of n2
Graph
A collection of nodes and connecting edges
Hamiltonian circuit
A path through a graph that begins and ends at the same node and goes through all other nodes exactly once
Invitation to Computer Science, 5th Edition

42. Invitation to Computer Science, 5th Edition
Figure 3.22 Order-of-Magnitude Time Efficiency Summary
Invitation to Computer Science, 5th Edition

43. Invitation to Computer Science, 5th Edition
Figure 3.23 Four Connected Cities
Invitation to Computer Science, 5th Edition

44. Invitation to Computer Science, 5th Edition
Figure 3.24 Hamiltonian Circuits among All Paths from A in Figure 3.23 with Four Links
Invitation to Computer Science, 5th Edition

45. When Things Get Out of Hand
Exponential algorithm
An Θ(2n) algorithm
Brute force algorithm
One that beats the problem into submission by trying all possibilities
Intractable problem
No polynomially bounded algorithm exists
Approximation algorithms
Do not solve the problem, but provide a close approximation to a solution

46. Invitation to Computer Science, 5th Edition
Figure 3.25 Comparisons of lg n, n, n2, and 2n
Invitation to Computer Science, 5th Edition

47. Invitation to Computer Science, 5th Edition
Figure 3.26 Comparisons of lg n, n, n2, and 2n for Larger Values of n
Invitation to Computer Science, 5th Edition

48. Invitation to Computer Science, 5th Edition
Figure 3.27 A Comparison of Four Orders of Magnitude
Invitation to Computer Science, 5th Edition