1. 14 SORTING AND SEARCHING CHAPTER
Copyright © 2013 by John Wiley & Sons. All rights reserved.
Slides by Rick Giles

2. Chapter Goals To study several sorting and searching algorithms
To appreciate that algorithms for the same task can differ widely in performance
To understand the big-oh notation
To estimate and compare the performance of algorithms
To write code to measure the running time of a program
Copyright © 2013 by John Wiley & Sons. All rights reserved.

3. Contents Selection Sort Profiling the Selection Sort Algorithm
Analyzing the Performance of the Selection Sort Algorithm
Merge Sort
Analyzing the Merge Sort Algorithm
Searching
Problem Solving: Estimating the Running Time of an Algorithm
Sorting and Searching in the Java Library
Copyright © 2013 by John Wiley & Sons. All rights reserved.

4. 14.1 Selection Sort
Sorts an array by repeatedly finding the smallest element of the unsorted tail region and moving it to the front
Example: sorting an array of integers 11 9 17 5 12
Copyright © 2013 by John Wiley & Sons. All rights reserved.

5. Sorting an Array of Integers
Find the smallest and swap it with the first element
Find the next smallest. It is already in the correct place
Find the next smallest and swap it with first element of unsorted portion
Repeat
When the unsorted portion is of length 1, we are done 5 9 17 11 12 5 9 17 11 12 5 9 11 17 12 5 9 11 12 17 5 9 11 12 17
Copyright © by John Wiley & Sons. All rights reserved.

6. SelectionSorter.java Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

7. SelectionSorter.java (cont.)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

8. SelectionSortDemo.java
Copyright © 2013 by John Wiley & Sons. All rights reserved.

9. ArrayUtil.java Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

10. ArrayUtil.java (cont.)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

11. SelectionSortDemo.java
Copyright © 2013 by John Wiley & Sons. All rights reserved.

12. 14.2 Profiling the Selection Sort Algorithm
Want to measure the time the algorithm takes to execute
Exclude the time the program takes to load
Exclude output time
Create a StopWatch class to measure execution time of an algorithm
It can start, stop and give elapsed time
Use System.currentTimeMillis method
Create a StopWatch object
Start the stopwatch just before the sort
Stop the stopwatch just after the sort
Read the elapsed time
Copyright © 2013 by John Wiley & Sons. All rights reserved.

13. StopWatch.java Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

14. StopWatch.java (cont.) Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

15. StopWatch.java (cont.) Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

16. SelectionSortTimer.java Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

17. SelectionSortTimer.java (cont.)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

18. Selection Sort on Various Array Sizes*
Milliseconds
10,000
786
20,000
2,148
30,000
4,796
40,000
9,192
50,000
13,321
60,000
19,299
* Obtained with a 2GHz Pentium processor, Java 6, Linux
Copyright © 2013 by John Wiley & Sons. All rights reserved.

19. Time Taken by Selection Sort
Copyright © 2013 by John Wiley & Sons. All rights reserved.

20. 14.3 Analyzing the Performance of the Selection Sort Algorithm (1)
In an array of size n, count how many times an array element is visited
To find the smallest, visit n elements + 2 visits for the swap
To find the next smallest, visit (n - 1) elements + 2 visits for the swap
The last term is 2 elements visited to find the smallest + 2 visits for the swap
Copyright © 2013 by John Wiley & Sons. All rights reserved.

21. 14.3 Analyzing the Performance of the Selection Sort Algorithm (2)
The number of visits:
n (n - 1) (n - 2)
This can be simplified to n2 /2  +  5n/2  - 3
5n/2 - 3 is small compared to n2 /2 — so ignore it
Also ignore the 1/2 — it cancels out when comparing ratios
Copyright © 2013 by John Wiley & Sons. All rights reserved.

22. 14.3 Analyzing the Performance of the Selection Sort Algorithm (3)
The number of visits is of the order n2
Using big-Oh notation: The number of visits is O(n2)
Multiplying the number of elements in an array by 2 multiplies the processing time by 4
Big-Oh notation “f(n) = O(g(n))” expresses that f grows no faster than g
To convert to big-Oh notation: Locate fastest-growing term, and ignore constant coefficient
Copyright © 2013 by John Wiley & Sons. All rights reserved.

23. 14.4 Merge Sort Sorts an array by
Cutting the array in half
Recursively sorting each half
Merging the sorted halves
Much more efficient than selection sort
Copyright © 2013 by John Wiley & Sons. All rights reserved.

24. Merge Sort Example Divide an array in half and sort each half
Merger the two sorted arrays into a single sorted array
Copyright © 2013 by John Wiley & Sons. All rights reserved.

25. sort Method
Copyright © 2013 by John Wiley & Sons. All rights reserved.

26. MergeSorter.java Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

27. MergeSorter.java (cont.)
Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

28. MergeSorter.java (cont.)
Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

29. MergeSorter.java (cont.)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

30. MergeSortDemo.java
Copyright © 2013 by John Wiley & Sons. All rights reserved.

31. 14.5 Analyzing the Merge Sort Algorithm (1)
Merge Sort (milliseconds)
Selection Sort (milliseconds)
10,000 40
786
20,000 73
2,148
30,000
134
4,796
40,000
170
9,192
50,000
192
13,321
60,000
205
19,299
Copyright © 2013 by John Wiley & Sons. All rights reserved.

32. Analyzing the Merge Sort Algorithm (2)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

33. Analyzing the Merge Sort Algorithm (3)
In an array of size n, count how many times an array element is visited
Assume n is a power of 2: n = 2m
Calculate the number of visits to create the two sub-arrays and then merge the two sorted arrays
3 visits to merge each element or 3n visits
2n visits to create the two sub-arrays
total of 5n visits
Copyright © by John Wiley & Sons. All rights reserved.

34. Analyzing the Merge Sort Algorithm (4)
Let T(n) denote the number of visits to sort an array of n elements then
T(n) = T(n/2) + T(n/2) + 5n or
T(n) = 2T(n/2) + 5n
The visits for an array of size n/2 is:
T(n/2) = 2T(n/4) + 5n/2
So T(n) = 2 × 2T(n/4) +5n + 5n
The visits for an array of size n/4 is:
T(n/4) = 2T(n/8) + 5n/4
So T(n) = 2 × 2 × 2T(n/8) + 5n + 5n + 5n
Copyright © 2013 by John Wiley & Sons. All rights reserved.

35. Analyzing the Merge Sort Algorithm (5)
Repeating the process k times:
T(n) = 2kT(n/2k) +5nk
Since n = 2m, when k=m: T(n) = 2mT(n/2m) +5nm
T(n) = nT(1) +5nm
T(n) = n + 5nlog2(n)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

36. Analyzing the Merge Sort Algorithm (6)
To establish growth order
Drop the lower-order term n
Drop the constant factor 5
Drop the base of the logarithm since all logarithms are related by a constant factor
We are left with n log(n)
Using big-Oh notation: Number of visits is O(nlog(n))
Copyright © 2013 by John Wiley & Sons. All rights reserved.

37. Merge Sort vs Selection Sort
Selection sort is an O(n2) algorithm
Merge sort is an O(nlog(n)) algorithm
The nlog(n) function grows much more slowly than n2
Copyright © 2013 by John Wiley & Sons. All rights reserved.

38. 14.6 Searching
Linear search: Examines all values in an array until it finds a match or reaches the end
Also called sequential search
Number of visits for a linear search of an array of n elements:
The average search visits n/2 elements
The maximum visits is n
A linear search locates a value in an array in O(n) steps
Copyright © 2013 by John Wiley & Sons. All rights reserved.

39. LinearSearcher.java
Copyright © 2013 by John Wiley & Sons. All rights reserved.

40. LinearSearchDemo.java Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

41. LinearSearchDemo.java (cont.)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

42. Binary Search (1) Locates a value in a sorted array by
Determining whether the value occurs in the first or second half
Then repeating the search in one of the halves
Copyright © 2013 by John Wiley & Sons. All rights reserved.

43. Binary Search (2) To search 15: 15 ≠ 17: We don’t have a match
Copyright © 2013 by John Wiley & Sons. All rights reserved.

44. BinarySearcher.java Continued
Copyright © 2013 by John Wiley & Sons. All rights reserved.

45. BinarySearcher.java (cont.)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

46. Analyzing Binary Search (1)
Count the number of visits to search a sorted array of size n
We visit one element (the middle element) then search either the left or right subarray
Thus: T(n) = T(n/2) + 1
Using the same equation, T(n/2) = T(n/4) + 1
Substituting into the original equation: T(n) = T(n/4) + 2
This generalizes to: T(n) = T(n/2k) + k
Copyright © 2013 by John Wiley & Sons. All rights reserved.

47. Analyzing Binary Search (2)
Assume n is a power of 2, n = 2m, where m = log2(n)
Then: T(n) = 1 + log2(n)
Binary search is an O(log(n)) algorithm
Copyright © 2013 by John Wiley & Sons. All rights reserved.

48. 14.7 Problem Solving: Estimating the Running Time of an Algorithm
Differentiating between O(nlog(n)) and O(n2) has great practical implications
Being able to estimate running times of other algorithms is an important skill
Will practice estimating the running times of array algorithms
Copyright © 2013 by John Wiley & Sons. All rights reserved.

49. Linear Time (1)
An algorithm to count how many elements of an array have a particular value:
int count = 0;
for (int i = 0; i < a.length; i++) {
if (a[i] == value) { count++; } }
Copyright © 2013 by John Wiley & Sons. All rights reserved.

50. Linear Time (2) To visualize the pattern of array element visits:
Imagine the array as a sequence of light bulbs
As the ith element gets visited, imagine the ith light bulb lighting up
When each visit involves a fixed number of actions, the running time is n times a constant, or O(n)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

51. Linear Time (3)
When you don’t always run to the end of the array; e.g.:
boolean found = false;
for (int i = 0; !found && i < a.length; i++) {
   if (a[i] == value) { found = true; } }
Loop can stop in middle:
Still O(n), because in some cases the match may be at the very end of the array
Copyright © 2013 by John Wiley & Sons. All rights reserved.

52. Quadratic Time (1) What if we do a lot of work with each visit?
Example: find the most frequent element in an array
Obvious for small array
For larger array?
Put counts in a second array of same length
Take the maximum of the counts, 3, look up where the 3 occurs in the counts, and find the corresponding value, 7
Copyright © 2013 by John Wiley & Sons. All rights reserved.

53. Quadratic Time (2)
First estimate how long it takes to compute the counts
for (int i = 0; i < a.length; i++) {
   counts[i] = Count how often a[i] occurs in a }
Still visit each element once, but work per visit is much larger
Each counting action is O(n)
When we do O(n) work in each step, total running time is O(n2)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

54. Quadratic Time (3) Algorithm has three phases:
Compute all counts
Compute the maximum
Find the maximum in the counts
First phase is O(n2), second and third are each O(n)
The big-Oh running time for doing several steps in a row is the largest of the big-Oh times for each step
Thus algorithm for finding the most frequent element is O(n2)
Copyright © 2013 by John Wiley & Sons. All rights reserved.

55. Logarithmic Time (1)
An algorithm that cuts the size of work in half in each step runs in O(log(n)) time
E.g. binary search, merge sort
To improve our algorithm for finding the most frequent element, first sort the array:
Sorting costs O(n log(n)) time
If we can complete the algorithm in O(n) time, have a better algorithm that the pervious O(n2) algorithm
Copyright © 2013 by John Wiley & Sons. All rights reserved.

56. Logarithmic Time (2) While traversing the sorted array:
As long as you find a value that is equal to its predecessor, increment a counter
When you find a different value, save the counter and start counting anew
Copyright © 2013 by John Wiley & Sons. All rights reserved.

57. Logarithmic Time (3)
int count = 0; for (int i = 0; i < a.length; i++) { count++; if (i == a.length - 1 || a[i] != a[i + 1]) { counts[i] = count; count = 0; } }
Copyright © 2013 by John Wiley & Sons. All rights reserved.

58. Logarithmic Time (4)
Constant amount of work per iteration, even though it visits two elements:
2n is still O(n)
Entire algorithm is now O(log(n))
Copyright © 2013 by John Wiley & Sons. All rights reserved.

59. 14.8 Sorting and Searching in the Java Library
When you write Java programs, you don’t have to implement your own sorting algorithms
Arrays and Collections classes provide sorting and searching methods
Copyright © 2013 by John Wiley & Sons. All rights reserved.

60. Sorting The Arrays class contains static sort methods
To sort an array of integers:
int[] a = ... ; Arrays.sort(a);
That sort method uses the Quicksort algorithm (see Special Topic 14.3)
To sort an array of ArraysList use Collections.sort method:
ArrayList<String> names = ... ; Collections.sort(names);
That sort method uses the merge sort algorithm
Copyright © 2013 by John Wiley & Sons. All rights reserved.

61. Binary Search
Arrays and Collections classes contains static binarySearch methods
These methods implement the binary search algorithm, with a useful enhancement:
If the value is not found in the array, return -k -1, where k is the position before which the element should be inserted
E.g.
int[] a = { 1, 4, 9 };
int v = 7;
int pos = Arrays.binarySearch(a, v);
// Returns –3; v should be inserted before position 2
Copyright © 2013 by John Wiley & Sons. All rights reserved.

62. Comparing Objects (1)
Arrays and ArrayLists classes provide sort and binarySearch methods for objects
These methods cannot know how to compare arbitrary objects
Methods require that the objects belong to a class that implements the Comparable interface with a single method:
public interface Comparable {
int compareTo(Object otherObject); }
Copyright © 2013 by John Wiley & Sons. All rights reserved.

63. Comparing Objects (2) Call
a.compareTo(b)
must return a negative number if a should come before b, 0 if a and b are the same, and a positive number otherwise
Several classes in the standard Java Library implement the Comparable interface
E.g. String, Date
Copyright © 2013 by John Wiley & Sons. All rights reserved.

64. Comparing Objects (3)
You can implement the Comparable interface for your classes as well
E.g. to sort a collection of countries by area, class Country can implement this interface and supply a compareTo method:
Copyright © 2013 by John Wiley & Sons. All rights reserved.

65. Comparing Objects (3)
You can implement the Comparable interface for your classes as well
E.g. to sort a collection of countries by area, class Country can implement this interface and supply a compareTo method:
public class Country implements Comparable {
public int compareTo(Object otherObject)
Country other = (Country) otherObject;
if (area < other.area) { return -1; }
else if (area == other.area) { return 0; }
else { return 1; } }
Copyright © 2013 by John Wiley & Sons. All rights reserved.

66. Summary: Selection Sort Algorithm
The selection sort algorithm sorts an array by repeatedly finding the smallest element of the unsorted tail region and moving it to the front.
Copyright © 2013 by John Wiley & Sons. All rights reserved.

67. Summary: Measuring the Running Time of an Algorithm
To measure the running time of a method, get the current time immediately before and after the method call.
Copyright © 2013 by John Wiley & Sons. All rights reserved.

68. Summary: Big-Oh Notation
Computer scientists use the big-Oh notation to describe the growth rate of a function.
Selection sort is an O(n2) algorithm. Doubling the data set means a fourfold increase in processing time.
Insertion sort is an O(n2) algorithm.
Copyright © 2013 by John Wiley & Sons. All rights reserved.

69. Summary: Contrasting Running Times of Merge Sort and Selection Sort Algorithms
Merge sort is an O(n log(n)) algorithm. The n log(n) function grows much more slowly than n2.
Copyright © 2013 by John Wiley & Sons. All rights reserved.

70. Summary: Running Times of Linear Search and Binary Search Algorithms
A linear search examines all values in an array until it finds a match or reaches the end.
A linear search locates a value in an array in O(n) steps.
A binary search locates a value in a sorted array by determining whether the value occurs in the first or second half, then repeating the search in one of the halves.
A binary search locates a value in a sorted array in O(log(n)) steps.
Copyright © 2013 by John Wiley & Sons. All rights reserved.

71. Summary: Practice Developing Big-Oh Estimates of Algorithms
A loop with n iterations has O(n) running time if each step consists of a fixed number of actions.
A loop with n iterations has O(n2) running time if each step takes O(n) time.
The big-Oh running time for doing several steps in a row is the largest of the big-Oh times for each step.
A loop with n iterations has O(n2) running time if the ith step takes O(i) time.
An algorithm that cuts the size of work in half in each step runs in O(log(n)) time.
Copyright © vby John Wiley & Sons. All rights reserved.

72. Summary: Use the Java Library Methods for Sorting and Searching Data
The Arrays class implements a sorting method that you should use for your Java programs.
The Collections class contains a sort method that can sort array lists.
The sort methods of the Arrays and Collections classes sort objects of classes that implement the Comparable interface.
Copyright © 2013 by John Wiley & Sons. All rights reserved.