CSE 250 – Data Structures

Today’s Goals  First review the easy, simple sorting algorithms  Compare while inserting value into place in the vector  Go through unsorted data & select value to be added  Can sorting get faster? If so, are there limits?  What does this mean for sorting & first sorts  Divide-and-conquer approaches are faster how?  Use execution trees to show how merge sort works  Learn quick sort weakness and how it can be avoided

Simple Sorting Algorithms  Splits vector into ordered & unordered parts  Grows ordered part by 1 until entire vector sorted  Way in which values added depends on the sort: Insert Insert 1 st unordered value by shifting larger ordered values select From unordered values, select smallest to add at end

Simple Sorting Algorithms  Splits vector into ordered & unordered parts  Grows ordered part by 1 until entire vector sorted  Way in which values added depends on the sort: Insert Insert 1 st unordered value by shifting larger ordered values select From unordered values, select smallest to add at end  What is time complexity for each of these?

Simple Sorting Algorithms  Splits vector into ordered & unordered parts  Grows ordered part by 1 until entire vector sorted  Way in which values added depends on the sort: Insert Insert 1 st unordered value by shifting larger ordered values select From unordered values, select smallest to add at end  What is time complexity for each of these? O(n)

Simple Sorting Algorithms  Splits vector into ordered & unordered parts  Grows ordered part by 1 until entire vector sorted  Way in which values added depends on the sort: Insert Insert 1 st unordered value by shifting larger ordered values select From unordered values, select smallest to add at end  What is time complexity for each of these? O(n)

Simple Sorting Algorithms  Splits vector into ordered & unordered parts  Grows ordered part by 1 until entire vector sorted  Way in which values added depends on the sort: Insert Insert 1 st unordered value by shifting larger ordered values select From unordered values, select smallest to add at end  What is time complexity for each of these?  Complexity usually multiplied since actions are nested O(n)

Simple Sorting Algorithms  Splits vector into ordered & unordered parts  Grows ordered part by 1 until entire vector sorted  Way in which values added depends on the sort: Insert Insert 1 st unordered value by shifting larger ordered values select From unordered values, select smallest to add at end  What is time complexity for each of these?  Complexity usually multiplied since actions are nested O(n) * O(n) = O(n 2 ) O(n)

Can You Guess My Name? void __________(vector &vec) { for (int i=0; i < vec.size()-1; i++){ int j = i; for (int k=i+1; k < vec.size(); k++){ if (vec[k] < vec[j]) { j=k; } } swap(vec[i], vec[j]); } }

Can You Guess My Name? void __________(vector &vec) { for (int i=0; i < vec.size()-1; i++){ int j = i; for (int k=i+1; k < vec.size(); k++){ if (vec[k] < vec[j]) { j=k; } } swap(vec[i], vec[j]); } }

Can You Guess My Name? void __________(vector &vec) { for (int i=0; i < vec.size()-1; i++){ int j = i; for (int k=i+1; k < vec.size(); k++){ if (vec[k] < vec[j]) { j=k; } } swap(vec[i], vec[j]); } } /* Loops through vector 1 value at a time, but both sorts do this! */

Can You Guess My Name? void __________(vector &vec) { for (int i=0; i < vec.size()-1; i++){ int j = i; for (int k=i+1; k < vec.size(); k++){ if (vec[k] < vec[j]) { j=k; } } swap(vec[i], vec[j]); } }

Can You Guess My Name? void __________(vector &vec) { for (int i=0; i < vec.size()-1; i++){ int j = i; for (int k=i+1; k < vec.size(); k++){ if (vec[k] < vec[j]) { j=k; } } swap(vec[i], vec[j]); } } /* Selecting smallest value! */

Can You Guess My Name? void selectionSort(vector &vec) { for (int i=0; i < vec.size()-1; i++){ int j = i; for (int k=i+1; k < vec.size(); k++){ if (vec[k] < vec[j]) { j=k; } } swap(vec[i], vec[j]); } }

Insertion Sort void insertionSort(vector &vec) { for (int i=0; i 0 && tmp < vec[j-1]; j--){ vec[j] = vec[j-1]; } vec[j] = tmp; } }

Selection v. Insertion Sort Selection SortInsertion Sort  Selects smallest remaining  Ordered portion complete  Number shifts always O(n)  O(n 2 ) time complexity  In-memory -- O(1) space  Selects next unordered value  Not complete until end  Up to O(n 2 ) shifts needed  O(n 2 ) time complexity  In-memory -- O(1) space

Counting Comparisons decision tree  Consider sort as a path in a decision tree  Nodes are single decision needed for sorting yesno Is x i > x j ?

Counting Comparisons decision tree  Consider sort as a path in a decision tree  Nodes are single decision needed for sorting  Traveling from root to leaf sorts data  Tree’s height is lower-bound on sorting complexity

Decision Tree Height  Leaf needed for each order values could appear  Needed to ensure we sort different inputs differently  Consider 4, 5 as data to be sorted using a tree  Could be entered in 2 possible orders: 4, 5 or 5, 4  Need two leaves for this sort unless (4 < 5) == (5 < 4)

Decision Tree Height  Leaf needed for each order values could appear  Needed to ensure we sort different inputs differently  Consider 4, 5 as data to be sorted using a tree  Could be entered in 2 possible orders: 4, 5 or 5, 4  Need two leaves for this sort unless (4 < 5) == (5 < 4)  For sequence of n numbers, can arrange n! ways  Tree with n! leaves needed to sort n numbers  Given this many leaves, what is height of the tree?

Decision Tree Height  With n ! external nodes, height would be:

The Lower Bound  But what does O(log(n !) ) equal? n ! = n * n -1 * n -2 * n -3 * n /2 * … * 2 * 1 n ! ≤ ( ½* n ) ½* n (½ of series is larger than ½* n ) log(n!) ≤ log((½*n) ½*n ) log(n!) ≤ ½*n * log(½*n) O(log(n!)) ≤ O(½*n * log(½*n))

The Lower Bound  But what does O(log(n !) ) equal? n ! = n * n -1 * n -2 * n -3 * n /2 * … * 2 * 1 n ! ≤ ( ½* n ) ½* n (½ of series is larger than ½* n ) log(n!) ≤ log((½*n) ½*n ) log(n!) ≤ ½*n * log(½*n) O(log(n!)) ≤ O(½*n * log(½*n))

The Lower Bound  But what does O(log(n !) ) equal? n ! = n * n -1 * n -2 * n -3 * n /2 * … * 2 * 1 n ! ≤ ( ½* n ) ½* n (½ of series is larger than ½* n ) log(n!) ≤ log((½*n) ½*n ) log(n!) ≤ ½*n * log(½*n) O(log(n!)) ≤ O(n log n)

Lower Bound on Sorting  Smallest number of comparisons is tree’s height  Decision tree sorting n elements has n! leaves  At least log( n !) height needed for this many leaves  This simplifies to at most O(n log n) height  O(n log n) time needed to compare data!  Can we define sort that runs in minimum time?

Julius, Seize Her!  Formula to Roman success  Divide peoples before an attack  Then conquer weakened armies  Common programming paradigm  Divide: split into 2 partitions  Recur : solve for partitions  Conquer: combine solutions

Divide-and-Conquer  Like all recursive algorithms, need base case  Has immediate solution to a simple problem  Work is not easy and sorting 2+ items takes work  1 item sorted since it cannot be out of order  Sorting a vector with 0 items even easer  Recursive step simplifies problem & combines it  Begins by splitting data into two equal vector s  Merges sub vector s after they have been sorted

Execution Tree  Depicts divide-and-conquer execution  Recursive call represented by each oval node  Original vector shown at start  At the end of the oval, sorted vector shown  Initial call at root of the (binary) tree  Bottom of the tree has leaves for base cases

Execution Tree  Depicts divide-and-conquer execution  Recursive call represented by each oval node  Original vector shown at start  At the end of the oval, sorted vector shown  Initial call at root of the (binary) tree  Bottom of the tree has leaves for base cases

Merge Sort Execution  Not in a base case 

Merge Sort Execution  Not in a base case, so split into lefty & righty 

Merge Sort Execution  Not in a base case, so split into lefty & righty 

Merge Sort Execution  Recursively call merge-sort on lefty 

Merge Sort Execution  Recursively call merge-sort on lefty  

Merge Sort Execution  Not in a base case, so split into lefty & righty (again)  

Merge Sort Execution  Not in a base case, so split into lefty & righty (again)  

Merge Sort Execution  Recursively call merge-sort on lefty  

Merge Sort Execution  Recursively call merge-sort on lefty 7 2   

Merge Sort Execution  Still no base case, so split again & recurse on lefty 7    

Merge Sort Execution  Enjoy the base case – literally no work to do! 7  77    

7  77  7 Merge Sort Execution  Recurse on righty and solve for this base case 2  22    

Merge Sort Execution  Merge the two solutions to complete this call 7  77  72  22    

Merge Sort Execution  Recurse on righty and sort this vector 9 4   77  72  22    

Merge Sort Execution  Split into lefty & righty and solve the base cases 9  94  44       77  72  22  2

Merge Sort Execution  Merge the 2 solutions to sort this vector 9  94  44       77  72  22  2

Merge Sort Execution  Merge the 2 solutions to sort this vector 9  94  44       77  72  22  2

Merge Sort Execution  Merge the 2 solutions to sort this vector 9  94  44       77  72  22  2

Merge Sort Execution  Merge the 2 solutions to sort this vector 9  94  44       77  72  22  2

Merge Sort Execution  Merge the 2 solutions to sort this vector 9  94  44       77  72  22  2

Merge Sort Execution  Merge the 2 solutions to sort this vector 9  94  44       77  72  22  2

Merge Sort Execution  I feel an urge, an urge to merge 9  94  44       77  72  22  2

Merge Sort Execution  Let's do the merge sort again! (with righty )   94  44       77  72  22  2

Merge Sort Execution  Let's do the merge sort again! (with righty ) 3 8   88  83  33    94  44       77  72  22  2

Merge Sort Execution  Let's do the merge sort again! (with righty ) 6 1   66  61  11    88  83  33  39  94  44       77  72  22  

Merge Sort Execution  Let's do the merge sort again! (with righty ) 6 1   66  61  11    88  83  33    94  44       77  72  22  2

Merge Sort Execution  Merge the last call to get the final result 6 1   66  61  11    88  83  33    94  44       77  72  22  2

Quick Sort  Divide: Partition by pivot  L has values <= p  G uses values >= p  Recur: Sort L and G  Conquer: Merge L, p, G p

Quick Sort  Divide: Partition by pivot  L has values <= p  G uses values >= p  Recur: Sort L and G  Conquer: Merge L, p, G p

Quick Sort  Divide: Partition by pivot  L has values <= p  G uses values >= p  Recur: Sort L and G  Conquer: Merge L, p, G p L G p

Quick Sort  Divide: Partition by pivot  L has values <= p  G uses values >= p  Recur: Sort L and G  Conquer: Merge L, p, G p p L G p

Execution Example pivot  Each call starts by selecting the pivot 

Execution Example pivot  Each call starts by selecting the pivot 

Execution Example pivot  Split into L & G partitions around pivot 

Execution Example pivot  Split into L & G partitions around pivot 

Execution Example  Recursively solve L partition first 

Execution Example  Recursively solve L partition first  

Execution Example pivot  Select new pivot and split into L & G partitions  

Execution Example  Recursively solve for L 1   

Execution Example  Recursively solve for L & enjoy base case 1  11   

Execution Example  Now solve for G partition and select pivot  3 41  11   

Execution Example  Recursively solve for L   11   

Execution Example  Recursively solve for L & enjoy base case   11   

Execution Example  Now solve for G & enjoy base case 4  44    11   

Execution Example pivot  Add L, pivot, & G to complete previous call   11     44  4

Execution Example pivot  Add L, pivot, & G to complete previous call 1  11     44   3 4

Execution Example  Recursively sort G from original call 1  11    44    3 4

Execution Example  Recursively sort G from original call 1  11    44     3 4

Execution Example pivot  Select pivot & partition into L & G 1  11    44     3 4

Execution Example  Solve L recursively via base case & move to G 1  11    44      3 4

Execution Example pivot  Select pivot & partition into L & G 1  11    44      3 4

Execution Example  Solve through two base cases in L & G 9  99  9 1  11    44      3 4

Execution Example pivot  Add L, pivot, & G to complete the call 9  99  9 1  11    44      3 4

Execution Example pivot  Add L, pivot, & G to complete the call 9  99  9 1  11    44      3 4

Execution Example pivot  Add L, pivot, & G to complete final call   99  9 1  11  1 4  44      3 4

Execution Example only as good as pivot choice  Sorts data, but only as good as pivot choice   99  9 1  11  1 4  44      3 4

Pivot Choice Ideas  Choose first (or last) value in part of vector  Might work, unless vector already sorted

Pivot Choice Ideas  Choose first (or last) value in part of vector & often is  Might work, unless vector already sorted & often is

Pivot Choice Ideas  Choose first (or last) value in part of vector  Might work, unless vector already sorted & often is  People like order, so this is often worst choice

Pivot Choice Ideas  Choose first (or last) value in part of vector  Might work, unless vector already sorted & often is  People like order, so this is often worst choice  Could instead choose random value in partition

Pivot Choice Ideas  Choose first (or last) value in part of vector  Might work, unless vector already sorted & often is  People like order, so this is often worst choice  Could instead choose random value in partition

Pivot Choice Ideas  Choose first (or last) value in part of vector  Might work, unless vector already sorted & often is  People like order, so this is often worst choice  Could instead choose random value in partition  Worst case much less likely, but still not impossible  Best could be median of first, last, & middle values  Very unlikely for worst case running time to occur  But also adds much more complexity to your code

void ________(vector &vec) { if (vec.size() fst(vec.begin(),vec.begin()+n/2); vector snd(vec.begin()+n/2, vec.end()); ________(fst); ________(snd); conquer(vec, fst, snd); }

It’s Merge Sort! void mergeSort(vector &vec) { // Check for base case if (vec.size() fst(vec.begin(),vec.begin()+n/2); vector snd(vec.begin()+n/2, vec.end()); // Recursively solve smaller problems mergeSort(fst); mergeSort(snd); // Conquer by merging solutions merge (vec, fst, snd); }

Quick Sort v. Merge Sort Quick SortMerge Sort  Work mostly splitting data  Cannot guarantee even split  Should skip some comparisons  Does not need extra space  Less work allocating arrays  Blindly merges all the data  Data already in sorted order!  Blindly splits data in half  Always gets even split  Needs * to use other arrays  Wastes time in allocation  Complex workaround exists  Work mostly in merge step  Combines two (sorted) halves  Always skips some comparisons

Limit to Code  Want to write sort once – extra work never good  So far this lecture, sorts had int type hardcoded  template declares generic types for a function  Command for works above function using generics template  Sort reuse limited by for comparisons  Instead add comparison function as a parameter  * before function name otherwise listed like function

Fully Generic _______ Sort template void _____ (vector &vec, int (*cmp)(GenType, GenType)) { for (int i=1; i 0 && cmp(tmp, vec[j]) < 0) { vec[j] = vec[j-1]; j -= 1; } vec[j] = tmp; } }

For Next Lecture  Homework #1 due tonight  Deadline is very soon -- must be submitted by 2AM  Next week’s lectures on Sets and Maps