1. Searching and Sorting
Gary Wong

2. Prerequisite
Time complexity
Pseudocode
(Recursion)

3. Agenda Searching Sorting Linear (Sequential) Search Binary Search
Bubble Sort
Merge Sort
Quick Sort
Counting Sort

4. Linear Search
One by one...

5. Linear Search
Check every element in the list, until the target is found
For example, our target is 38: i 1 2 3 4 5
a[i] 25 14 9 38 77 45
Not found!
Found!

6. Linear Search Initilize an index variable i Compare a[i] with target
If a[i]==target, found
If a[i]!=target,
If all have checked already, not found
Otherwise, change i into next index and go to step 2

7. Linear Search Time complexity in worst case? Advantage? Disadvantage?
If N is number of elements,
Time complexity = O(N)
Advantage?
Disadvantage?

8. Binary Search
Chop by half...

9. Binary Search Given a SORTED list: (Again, our target is 38) i 1 2 3 4
Smaller!
Found!
Larger! i 1 2 3 4 5
a[i] 9 14 25 38 45 77 L R

10. Binary Search
Why always in the middle, but not other positions, say one-third of list?
1) Initialize boundaries L and R
2) While L is still on the left of R
mid = (L+R)/2
If a[mid]>Target, set R be m-1 and go to step 2
If a[mid]<Target, set L be m+1 and go to step 2
If a[mid]==Target, found

11. Binary Search Time complexity in the worst case? Advantage?
If N is the number of elements,
Time complexity = O(lg N)
Why?
Advantage?
Disadvantage?

12. Example: Three Little Pigs
HKOI 2006 Final Senior Q1
Given three lists, each with M numbers, choose one number from each list such that their sum is maximum, but not greater than N.
Constraints:
M ≤ 3000
Time limit: 1s

13. Example: Three Little Pigs
How many possible triples are there?
Why not check them all?
Time complexity?
Expected score = 50

14. Example: Three Little Pigs
A simpler question: How would you search for the maximum number in ONE SORTED list such that it is not greater than N?
Binary search!
With slight modification though
How?

15. Example: Three Little Pigs
Say, N=37 i 1 2 3 4 5
a[i] 9 14 25 38 45 77 L R

16. Example: Three Little Pigs
Let’s go back to original problem
If you have chosen two numbers a1[i] and a2[j] already, how would you search for the third number?
Recall: How would you search for the maximum number in ONE SORTED list such that it is not greater than N-a1[i]-a2[j]?

17. Example: Three Little Pigs
Overall time complexity?
Expected score = 90


18. Example: Three Little Pigs
Slightly harder: Given TWO lists, each with M numbers, choose one number from each list such that their sum is maximum, but not greater than N.
Linear search?
Sort one list, then binary search!
Time complexity = O(M lg M)
O(M2) if less efficient sorting algorithm is used
But, can it be better?

19. Example: Three Little Pigs
Why is it so slow if we use linear search?
If a1[i] and a2[j] are chosen, and their sum is smaller than N:
Will you consider any number in a1 that is smaller than or equal a1[i]?
If a1[i] and a2[j] are chosen, and their sum is greater than N:
Will you consider any number in a2 that is greater than or equal to a2[j]?

20. Example: Three Little Pigs
Recall: Why is it so slow if we use linear search?
Because you use it for too many times!
At which number in each list should you begin the linear search?
Never look back at those we do not consider!
Time complexity?
Expected score = 100 

21. What can you learn? Improve one ‘dimension’ using binary search
Linear search for a few times can be more efficient than binary search for many times!
DO NOT underestimate linear search!!!

22. Points to note
To use binary search, the list MUST BE SORTED (either ascending or decending)
NEVER make assumptions yourself
Problem setters usually do not sort for you
Sorting is the bottleneck of efficiency
But... how to sort?

23. How to sort? For C++: sort() Time complexity for sort() is O(N lg N)
which is considered as efficient
HOWEVER...
Problem setters SELDOM test contestants on pure usage of efficient sorting algorithms
Special properties of sorting algorithms are essential in problem-solving
So, pay attention! 

24. Smaller? Float! Larger? Sink!
Bubble Sort
Smaller? Float! Larger? Sink!

25. Bubble Sort Suppose we need to sort in ascending order...
Repeatedly check adjacent pairs sequentially, swap if not in correct order
Example:
The last number is always the largest 18 9 20 11 77 45
Incorrect order, swap!
Correct order, pass!

26. Bubble Sort
Fix the last number, do the same procedures for first N-1 numbers again...
Fix the last 2 numbers, do the same procedures for first N-2 numbers again...
...
Fix the last N-2 numbers, do the same procedures for first 2 numbers again...

27. Bubble Sort for i -> 1 to n-1 for j -> 1 to n-i
if a[j]>a[j+1], swap them
How to swap?

28. Many a little makes a mickle...
Merge Sort & Quick Sort
Many a little makes a mickle...

29. Merge Sort
Now given two SORTED list, how would you ‘merge’ them to form ONE SORTED list?
List 1: 8 14 22
List 2: 10 13 29 65
Temporary list: 8 8 10 10 13 13 14 14 22 22 29 29 65 65
Combined list:

30. Merge Sort
Merge
While both lists have numbers still not yet considered
Compare the current first number in two lists
Put the smaller number into temporary list, then discard it
If list 1 is not empty, add them into temporary list
If list 2 is not empty, add them into temporary list
Put the numbers in temporary list back to the desired list

31. Merge Sort
Suppose you are given a ‘function’ called ‘mergesort(L,R)’, which can sort the left half and right half of list from L to R:
How to sort the whole list?
Merge them!
How can we sort the left and right half?
Why not making use of ‘mergesort(L,R)’? 10 13 29 65 8 14 22 L
(L+R)/2
(L+R)/2+1 R

32. Merge Sort mergesort(L,R){ If L is equal to R, done; Otherwise, }
m=(L+R)/2;
mergesort(L,M);
mergesort(M+1,R);
Merge the lists [L,M] and [M+1,R]; }

33. Merge Sort
mergesort(0,6)
mergesort(0,1)
mergesort(2,3)
mergesort(4,5)
mergesort(6,6) 65 10 29 13 14 8 22
mergesort(0,0)
mergesort(1,1)
mergesort(2,2)
mergesort(3,3)
mergesort(4,4)
mergesort(5,5)
mergesort(0,3)
mergesort(4,6) 8 10 10 13 10 65 29 13 13 14 65 29 22 8 8 29 14 14 65 22

34. Merge Sort
Time complexity?
O(N lg N)
Why?

35. Quick Sort Choose a number as a ‘pivot’
Put all numbers smaller than ‘pivot’ on its left side
Put all numbers greater than (or equal to) ‘pivot’ on its right side 10 13 29 65 8 14 22 10 13 8 14 22 65 29

36. a[y] < pivot! shift x, swap!
Quick Sort
a[y] < pivot! shift x, swap!
How?
y shifts to right by 1 unit in every round
Check if a[y] < pivot
If yes, shift x to right by 1 unit and swap a[x] and a[y]
If y is at 2nd last position, swap a[x+1] and pivot
Time complexity? 10 13 29 65 8 14 22 x y
In fact, initial positions of x and y should be shifted to the left by 1 unit.

37. Quick Sort Use similar idea as in merge sort
If we have a function called ‘quicksort(L,R)’...
Make use of ‘quicksort(L,R)’ to sort the two halves! 10 13 8 14 22 65 29

38. Quick Sort quicksort(L,R){ If L is equal to R, done; Otherwise, }
Choose a number as a pivot, say a[p];
Perform pivoting action;
quicksort(L,p-1);
quicksort(p+1,R); }

39. Quick Sort Time complexity in worst case? O(N2) 
What is the worst case?
Preventions:
Choose a random number as pivot
Pick 3 numbers, choose their median
Under randomization, expected time complexity is O(N lg N)

40. Counting Sort
No more comparison...

41. Counting Sort Create a list, then tally!
Count the occurences of elements
Example:
Sort {2,6,1,4,2,1,9,6,4,1} in ascending order
A list from 1 to 9 (or upper bound of input no.)
Three ‘1’s, two ‘2’s, two ‘4’s, two ‘6’s and one ‘9’
List them all!

42. Counting Sort Time complexity? O(range * N)
Good for numbers with small range

43. More...
If we have time...

44. More... Lower bound of sorting efficiency?!
Guess the answer by binary search
Count the number of ‘inversions’
Find the kth largest number
Other sorting algorithms

45. Time for lunch
Yummy!