CS32 Discussion Section 1B Week 8 TA: Hao Yu (Cody)

Reminder & Agenda Time complexity Sorting algorithm Binary tree Project 3 due today 9:00 p.m. Homework 4 due next Tuesday 9:00 p.m.

Time Complexity – Motivation How to evaluate the efficiency of a program? Elapsed time? Assume same input size and machine Program 1: Program 2: Truth We need an objective, quantitive approach! FASTER? I am a worm~

Big-O Problem formulation Basic concept: step calculation Given an input of size n, how long does the algorithm take to finish the task in terms of n? Basic concept: step calculation How many steps (#statement) does the algorithm take to finish the task in terms of n? Example: 1 + 1 + 1 + n + n  3 + 2n 1 + 1 + n2  2 + n2

Big-O Calculation The upper-bound on the function’s growth Tips The constant doesn’t matter O(n) = O(5n) = O(18n) != O(kn) Only the highest degree matters O(10n3 + 7n2) = O(8n3) = O(2n3 + nlogn) Tips Find the most time-consuming loop for-loop, while-loop, recursive Compute its total #statement

Big-O Calculation: Exercise 1 int sum(int n) { int total = 0; for (int i = 0; i<=n; i++) total += i; return total; } n+1 O(n+1) = O(n) int sum2(int n) { int total = n*(n+1)/2; return total; } No loop! O(1)

Big-O Calculation: Exercise 2 void fun() { int m = 100; int n = 1000; for (int i = 0; i<= m; i++){ for (int j = 0; j <= n; j++) cout<< "Hello"<<endl; } (m + 1) x (n + 1) O(100 x 1000) = O(1) O((m+1) x (n+1)) = O(mn + m + n + 1) = O(mn) Tip: O((m+1) x (n+1)) ≈ O(m x n) = O(mn) void all_pairs(int arr[], int size) { for (int i = 0; i < size; i++) { for (int j = i; j< size; j++) { if (i != j) cout << i << " " << j << endl; } size x size O(size x size) = O(size2) = O(n2)

Big-O Calculation: Exercise 3 int outputQ(int n) { int counter = 0; for(int i=0 ; i < n ; i++ ) { for(int j=0 ; j < n ; j++ ) { counter++; cout << "Q"; break; } return counter; n x n n x 1! O(n x 1) = O(n)

Big-O Calculation: Exercise 3 int outputQ(int n) { int counter = 0; for(int i=0 ; i < n ; i++ ) { for(int j=0 ; j < n ; j++ ) { counter++; cout << "Q"; if (j > 10) break; } return counter; n x 10 O(n x 10) = O(n)

Big-O Calculation: Exercise 3 int outputQ(int n) { int counter = 0; for(int i=0 ; i < n ; i++ ) { for(int j=0 ; j < n ; j++ ) { counter++; cout << "Q"; if (i < 10) break; } return counter; 1 + 1 + … + 1 + n + n + … + n 10 + (n - 10) x n  10n2 – 100n O(10n2 - 100n) = O(n2) Tip: O(10 + (n-10) x n) ≈ O(n x n) = O(n2)

Big-O Calculation: Exercise 4 int countX(int n) { int counter = 0; for(int i=1 ; i <= n ; i*=2 ) for(int j=1; j <= n ; j++) for(int k=1; k <= n ; k++) counter++; return counter; } logn x n x n O(n2logn)

Big-O Calculation: Exercise 5 int binary_search(int arr[], int start, int end, int value) { if(start > end) return -1; int middle = start + ((end - start) / 2); if (arr[middle] == value) return middle; if(arr[middle] > value) return binary_search(arr, start, middle - 1, value); return binary_search(arr, middle + 1, end, value); } Terminal condition: length (end - start) <= 0 Increment condition: length = length / 2 A B for (i = length; i >= 0; i = i / 2) { // do something } A: New length = old length / 2 B: New length = old length / 2 O(logn + logn) = O(logn)

Order of Complexity Sorting these! O(1) O(log n) O(n log n) O(n2) O(nk), k >= 1 O(n)

Big-O – The Meaning We say an algorithm “efficient” if it runs fast with a large input  Similar to limit problem in math It implies the efficiency of an algorithm with a small input size is relatively not important We usually don’t care this part

Sorting Algorithms A typical problem but still gets attention Sorting contest http://sortbenchmark.org/ 2015: 15.9 TB/min Sorting algorithms https://en.wikipedia.org/wiki/Sorting_algorithm Developed based on a specific purpose and data This class selection sort, insertion sort, bubble sort quick sort, merge sort

Review Selection sort Bubble sort Insertion sort Find the minimum value Swap the minimum value and the current value Repeat until sorted Bubble sort Compare two consecutive values and swap if necessary Repeat until sorted Insertion sort Pick one value Insert it to the right position Repeat until sorted

Exercise 1 Sort (4, 3, 1, 5, 2) using Selection sort Insertion sort Bubble sort Write down every step!

Exercise 1 Sort (4, 3, 1, 5, 2) using Selection sort Insertion sort Bubble sort Write down every step! 4 3 1 5 2 1 3 4 5 2 1 2 4 5 3 1 2 3 5 4 Best case: O(n2) Average case: O(n2) Worst case: O(n2) 1 2 3 4 5

Exercise 1 Sort (4, 3, 1, 5, 2) using Selection sort Insertion sort Bubble sort Write down every step! 4 3 1 5 2 3 4 1 5 2 1 3 4 5 2 4 3 1 5 2 1 3 4 5 2 1 3 4 5 2 1 2 4 5 3 1 2 3 4 5 1 2 3 5 4 1 2 3 4 5 Selection sort Best case: O(n) Average case: O(n2) Worst case: O(n2)

Exercise 1 Sort (4, 3, 1, 5, 2) using Selection sort Insertion sort Bubble sort Write down every step! 3 4 1 5 2 3 1 4 5 2 3 1 4 5 2 3 1 4 2 5 1 3 4 2 5 1 3 4 2 5 1 3 2 4 5 1 3 2 4 5 1 3 2 4 5 1 2 3 4 5 Best case: O(n) Average case: O(n2) Worst case: O(n2) 1 2 3 4 5

Exercise 2 Which sorting algorithm will produce the following array after 2 iterations? Source: Formerly TA Kung-Hua Chang Unsorted array 3, 5, 4, 1, 2, 6, 7, 9, 8, 0 Array after 2 iterations 3, 1, 2, 4, 5, 6, 7, 0, 8, 9 Observation: If it uses selection sort, 0 and 1 must be sorted Array after 2 iterations: 0, 1, … If it uses insertion sort, first 3 values must be sorted Array after 2 iterations: 3, 4, 5, … If it uses bubble sort, small values would be pushed at most twice Array after 2 iterations: 3, 1, 2, …

Review Merge sort Quick sort Keep splitting the array until the length is 1 Merge Quick sort Pick a pivot and move smaller values to the left, larger values to the right Repeat step 1 to all subarrays

Exercise 3 Sort (3, 7, 6, 5, 8, 2, 1, 4) using Merge sort Quick sort Write down every step!

Exercise 3 Sort (3, 7, 6, 5, 8, 2, 1, 4) using Merge sort Quick sort Best case: O(nlogn) Average case: O(nlogn) Worst case: O(nlogn) Sort (3, 7, 6, 5, 8, 2, 1, 4) using Merge sort Quick sort Write down every step! 3 7 6 5 8 2 1 4 3 2 3 1 3 2 3 3 7 5 6 2 8 1 4 3 5 6 7 1 2 4 8 1 2 3 4 5 6 7 8

Exercise 3 Sort (3, 7, 6, 5, 8, 2, 1, 4) using Merge sort Quick sort Best case: O(n) Average case: O(nlogn) Worst case: O(n2) Sort (3, 7, 6, 5, 8, 2, 1, 4) using Merge sort Quick sort Write down every step! Pivot: The first value Pivot: The middle value 3 7 6 5 8 2 1 4 3 7 6 5 8 2 1 4 2 1 3 7 6 5 8 4 3 2 1 4 5 7 6 8 1 2 3 6 5 4 7 8 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8

Binary Trees Another data structure! Usually implemented using linked list class BinaryTree { BinaryTree() { m_root = nullptr; } ~BinaryTree() { freeTree(m_root); } void preOrder(Node *node); void inOrder(Node *node); void postOrder(Node *node); void levelOrder(Node *node); int numNode(Node *node); int numLeafNode(Node *node); int numNonLeafNode(Node *node); int height(Node *node); void freeTree(Node *node); Node *m_root; }; class Node { Node(int val) { value = val; left = nullptr; right = nullptr; } int value; Node *left, *right; };

Traverse Tree Nodes void preOrder(Node *node) { if (node == nullptr) return ; cout << node->value << " "; preOrder(node->left); preOrder(node->right); } 1 2 5 3 4 6 7 void inOrder(Node *node) { if (node == nullptr) return ; preOrder(node->left); cout << node->value << " "; preOrder(node->right); } 4 2 6 1 3 5 7 void postOrder(Node *node) { if (node == nullptr) return ; preOrder(node->left); preOrder(node->right); cout << node->value << " "; } 7 3 6 1 2 4 5

Traverse Tree Nodes 1 2 3 4 5 6 7 1 2 5 3 4 6 7 void levelOrder() { queue<Node*> q; q.push(m_root); while (!q.empty()) { Node *visited_node = q.front(); q.pop(); if (visited_node->left != nullptr ) q.push(visited_node->left); if(visited_node->right!= nullptr ) q.push(visited_node->right); cout << visited_node->value << " "; } 1 2 3 4 5 6 7 void someOrder() { stack<Node*> q; q.push(m_root); while(!q.empty()) { Node *visited_node = q.top(); q.pop(); if(visited_node->right != nullptr ) q.push(visited_node->right); if(visited_node->left!= nullptr ) q.push(visited_node->left); cout << visited_node->value << " "; } 1 2 5 3 4 6 7

Exercise 1 What’s the output? Preorder: Inorder: Postorder: Levelorder: 5  3  4  8  1  6  7  9  0  2 3  8  4  5  6  1  0  9  2  7 8  4  3  6  0  2  9  7  1  5 5  3  1  4  6  7  8  9  0  2 5 3 1 4 6 7 8 9 2

Traverse Tree – Node Numbers int numNode(Node *node) { if (node == nullptr) return 0; return 1 + nodeNum(node->left) + nodeNum(node->right); } int numLeafNode(Node *node) { if (node == nullptr) return 0; if (node->left == nullptr && node->right == nullptr) return 1; return numLeafNode(node->left) + numLeafNode(node->right); } int numNonLeafNode(Node *node) { if (node == nullptr) return 0; if (node->left == nullptr && node->right == nullptr) return 1 + numNonLeafNode(node->left) + numNonLeafNode(node->right); }

Traverse Tree – Others int height(Node *node) { if (node == nullptr) return 0; int leftHeight = height(node->left); int rightHeight = height(node->right); return 1 + max(leftHeight, rightHeight); } void freeTree(Node *node) { if (node == nullptr) return ; freeTree(node->left); freeTree(node->right); delete node; }