# Interaction public int countCookiesEaten(Bowl bowl) {

## Presentation on theme: "Interaction public int countCookiesEaten(Bowl bowl) {"— Presentation transcript:

Interaction public int countCookiesEaten(Bowl bowl) {
if (bowl.isEmpty()) {         return 0;     }     else {         this.eatCookie(bowl);         return this.countCookiesEaten(bowl) + 1; }

Big-O and Sorting Lecture 15

Importance of Algorithm Analysis 1/2
“Performance” of an algorithm often refers to how quickly it executes and/or how much memory it needs performance matters when the data grows in size! can observe and/or analyze, then tune or revise algorithm Algorithm analysis is so important that every Brown CS student is required to take at least one course covering it

Importance of Algorithm Analysis 2/2
Many important factors that make performance important computing resources implementation (like machine, language) size of data, denoted N number of elements to be sorted number of elements in ArrayList to iterate through ex: much faster to search through CS15 course-list than the Social Security Database This lecture: a brief introduction to Algorithm Analysis! Goal: learning how to maximize efficiency of an algorithm and conserve resources

Runtime The runtime of an algorithm varies with the input and typically grows with the input size In most of computer science we focus on the worst case running time. easier to analyze and important for unforeseen inputs Average case is what will happen most often. Best case requires the least amount of work and is the best situation you could have. average case is also important, best case is interesting but not insightful How to determine runtime? by inspecting pseudocode, can determine the number of statements executed by an algorithm as a function of input size allows us to evaluate approximate speed of an algorithm independent of the hardware or software environment memory use may be even more important for small and embedded devices Supplied by CS16, taught at Brown University with Prof. David Laidlaw

Elementary Operations
Algorithmic “time” is measured in elementary operations math (+, -, *, /, max, min, log, sin, cos, abs, ...) comparisons ( ==, >, <=, ...) function calls and value returns (not counting the body of the method) variable assignment variable increment or decrement array allocation creating a new object (careful, object's constructor may have elementary ops too!) For the purpose of algorithm analysis, we assume each of these operations takes the same time: “1 operation” – we are only interested in “asymptotic performance” for large data sets where small differences don’t matter – see S8 Supplied by CS16, taught at Brown University with Prof. David Laidlaw

Example: Constant runtime
public int addition(int x, int y) { return x+y; //call, add and return, 3 ops } always 3 operations performed How many operations performed if this function were to add ten integers? Would it still be constant runtime? Supplied by CS16, taught at Brown University with Prof. David Laidlaw

Example: Linear runtime
How many operations if the list has 1,000 elements? Worst case is an increasing list Worst case varies proportional to the size of the input list: 4n + 2 We’ll be in the for loop longer as the input list grows The runtime increases linearly //find max of a set of integers public int maxElement(int[] a) { int max = 0; //assignment, 1 op for (i=0; i<a.length; i++){ //1 op per loop if (a[i]>max){ //2 ops per loop max = a[i]; //1 op per loop, sometimes } return max; //1 op Supplied by CS16, taught at Brown University with Prof. David Laidlaw

public void printPossibleSums(int[ ] a, int arrayLength) { for (i=0; i<arrayLength; i++) { //1 op per loop for (j=0; j<arrayLength; j++) { //1 op per loop System.out.println(a[i] + a[j]); // 4 ops per loop } Requires about 5n2 + n operations (okay to approximate!) Number of operations executed grows quadratically! If add one element to list: element must be multiplied with every other element in list Notice that the linear algorithm on the previous slide had only one for loop, while this quadratic one has two for loops, nested Supplied by CS16, taught at Brown University with Prof. David Laidlaw

Big-O Notation - OrderOf()
But how to abstract from implementation? Big O notation O(N) means an operation is done on each element once N elements * 1 operation/element = N operations O(N2) means each element is operated on N times N elements * N operations/element = N2 operations Only consider “asymptotic behavior” i.e., when N>>1 (N is much greater than 1) N is unimportant compared to N2

1+2+3+…+N =  i = N(N+1)/2, which is O(N2)
Big-O Constants Remember, only largest N expression without constants matters We aren’t concerned about runtime with small numbers of data – we care about running operations on large amounts of inputs 3N2 and 500N2 are both O(N2) – unrealistic if N is small, of course N/2 is O(N) 4N2 + 2N is O(N2) Useful sum that recurs frequently in analysis: 1+2+3+…+N =  i = N(N+1)/2, which is O(N2) i =1 N

Social Security Database Example 1/3
Hundreds of millions of people in US, each have a number associated to them If 100,000 people named John Smith, each has an individual SSN If gov’t wants to look up information they have on John Smith, they use his SSN

Social Security Database Example 2/3
Say it takes 10-4 seconds to perform a constant set of operations on one SSN running an algorithm on 5 social security numbers may take 5x10-4 seconds, and running an algorithm on 50 will only take 5x10-3 seconds both are incredibly fast, a difference in runtime might not be noticeable by an interactive user this changes with large amounts of data, i.e., the actual SS Data Base

Social Security Database Example 3/3
Say it takes 10-4 seconds to perform a constant set of operations on one SSN to perform an algorithm with O(n) on 300 million people, it will take 8.3 hours O(n2) takes 285,000 years With large amounts of data, differences between O(n) and O(n2) are huge

Graphical Perspective 1/2
f(n) on linear graph paper

Graphical Perspective 2/2
f(n) on log-log graph paper Base for logs often depends on the data structures we use in our algorithms

Quiz! 1/3 What is the big-O runtime of this algorithm? Answer: O(n)
pubilc int sumArray(int[] array){ int sum = 0; for (int i=0; i<array.length; i++){ sum = sum + array[i]; } return sum; Answer: O(n)

Quiz! 2/3 What is the Big-O runtime of this algorithm?
Consider the getColor method in LiteBrite pubilc java.awt.color getColor(){ return _currentColor; } Answer: O(1)

Quiz! 3/3 What is the big-O runtime of this algorithm? Answer: O(n2)
public int sumSqaureArray(int dim, int[][] a){ int sum = 0; for (int i=0; i<dim; i++){ for (j=0; j<dim; j++){ sum = sum + a[j][i]; } return sum; Answer: O(n2)

Sorting We can use our analysis of runtimes to help choose the best algorithm to solve a problem A popular application is sorting, along with searching (future lecture) a huge consumer of computer time. They often go together…

Sorting – Social Security Numbers
Consider example where run-time influences approach How would you sort every SSN in the Social Security Database in increasing order? A number of ways to sort a list these ways vary in the amount of time they take

Sorting – Bubble Sort Iterate through sequence, compare each element to right neighbor Exchange adjacent elements if necessary End up with a sorted sub-array on the right. Each time we go through the list, need to switch one fewer item Images courtesy of Brown cs16 with David Laidlaw

Bubble Sort - Pseudocode
Iterate through sequence, compare each element to right neighbor Exchange adjacent elements if necessary End up with a sorted sub-array on the right. Each time we go through the list, need to switch one fewer item N is number of objects in sequence Pseudocode:

Bubble Sort - Runtime Worst-case analysis (sorted in inverse order):
the while-loop is iterated N-1 times iteration i has (i - 1) operations Total: 2+N+2(N-1)+6[(N-1) ]= 3N+6N(N-1)/2 = 3N2+... = O(N2) N is number of objects in sequence

Sorting – Insertion Sort
Like inserting a new card into a partially sorted hand by bubbling to the left into sorted subarray on left; little less brute-force than bubble sort Add one element a[i] at a time Find proper position, j+1, to the left by shifting to the right a[i-1], a[i-2], ..., a[j+1] left neighbors, until a[j] < a[i] Move a[i] into vacated a[j+1] After iteration i<n, the original a[0] ... a[i] are in sorted order, but not necessarily in final position

Pseudocode – Insertion Sort
for (int i = 1; i < n; i++) { int toInsert = a[i]; int j = i-1; while ((j >= 0) && (a[j] > toInsert) { move a[j] forward; j--; } move toInsert to a[j+1]; Images courtesy of Brown cs16 with David Laidlaw

Runtime – Insertion Sort
for (int i = 1; i < n; i++) { int toInsert = a[i]; int j = i-1; while ((j >= 0) && (a[j] > toInsert) { move a[j] forward; j--; } move toInsert to a[j+1]; We have a while loop inside our for-loop. The while loops calls on N-1 operations, then N-2 operations… The for-loop calls on that while loop N times. So O(N2) because we have to call on a while loop with around N operations N different times Reminder! Constants don’t matter with Big-O

Sorting – Selection Sort
Find smallest element and put it in a[0] Find 2nd smallest element and put it in a[1], etc Less data movement (no bubbling)

Pseudocode – Selection Sort
What we want to happen: for (int i = 0; i < n; i++) { find minimum element a[min] in subsequence a[i...n-1] swap a[min] and a[i] } Pseudocode: for (int i = 0; i < n-1; i++) { int min = i; for (int j = i + 1; j < n; j++) { if (a[j] < a[min]) { min = j; } temp = a[min]; a[min] = a[i]; a[i] = temp;

Run time of Selection Sort
Most executed instructions are those in inner for loop (if) Each such instruction is executed (N-1) + (N-2) times Time complexity: O(N2) Pseudocode: for (int i = 0; i < n-1; i++) { int min = i; for (int j = i + 1; j < n; j++) { if (a[j] < a[min]) { min = j; } temp = a[min]; a[min] = a[i]; a[i] = temp;

Comparison of Elementary Sorting Algorithms
The differences in Best and Worst case performance result from the state (ordering) of the input before sorting Selection wins on data movement But for small data, even the worst sort, Bubble is fine!

Sorting – Merge Sort

Outline of Recursive (Top Down) Merge Sort 1/2
Partition sequence into two sub-sequences of N/2 elements. Recursively partition and sort each sub-sequence. Merge the sorted sub-sequences.

Outline of Recursive (Top Down) Merge Sort 2/2
Partition sequence into two sub-sequences of N/2 elements. Recursively partition and sort each sub-arrays. Merge the sorted sub-arrays.

Merge Sort Pseudocode 1/3
public class Sorts { // other code here public void mergeSort( ItemSequence listSequence, int first, int last) { if (first < last) { int middle = (first + last) / 2; mergeSort(listSequence, first, middle); mergeSort(listSequence, middle+1, last); merge(listSequence, first, middle,last); } Note: •listSequence is sequence to sort. •first and last are smallest and largest indices of sequence. //recursively mergeSort sub-sequence //recursively mergeSort sub-sequence // merge sorted sub-sequence back // together

Merge Sort Pseudocode 2/3
public class Sorts { // other code here public void mergeSort( ItemSequence listSequence, int first, int last) { if (first < last) { int middle = (first + last) / 2; mergeSort(listSequence, first, middle); mergeSort(listSequence, middle+1, last); merge(listSequence, first, middle,last); } Photo from

Merge Sort Pseudocode 3/3
public ItemSequence merge(ItemSequence A, ItemSequence B){ result = new ItemSequence() int aIndex = 0; int bIndex = 0; while (aIndex < A.length and bIndex < B.length){ if (A[aIndex] <= B[bIndex]){ add A[aIndex] to result; aIndex++; } else { add B[bIndex] to result; bIndex++; if (aIndex < A.length){ result = result + A[aIndex...end]; if (bIndex < B.length){ result = result + B[bIndex...end]; } return result; Add the elements from the two sequences in increasing order If there are elements left that you haven’t added, add the remaining elements to your result

Consider it graphically
Each part of the tree performs n operations to merge the two subproblems below it Because we divide the sequence by two, the algorithm makes log2n merges. O(nlog2n) which is way better than O(N2) You will learn much more about how to find the runtime of these types of algorithms in cs16! Photo from

Comparing Algorithms Side by Side
Gifs courtesy of Bubble Sort – O(N2) Insertion Sort – O(N2) Merge Sort - (Nlog2N)

What’s next? Now that you've been introduced to how different approaches to solving problems can speed/slow up and dramatically affect speed of algorithm… Subsequent lectures will cover other data structures for collections beyond arrays and arraylists that you can use to significantly improve speed of your algorithm

Lecture 16 Linked Lists C S 1 5

Data Structures You have been introduced to techniques for storing a collection of elements using Arrays and ArrayLists, and for analyzing the run-time performance of an algorithm There are other ways to store a collection of elements. Choose which data structure to use based on the way data is accessed and stored in your algorithm The access and store operations of different data structures can have very different impacts on an algorithm’s overall efficiency

What is a Linked List? A Linked List is a collection of nodes that form a sequence of elements The Linked List holds a reference to the first node Each node holds an element and a reference to the next node in the list

Node<HTA> _first null Node<HTA> Node<HTA> _next HTA _data Node<HTA> Node<HTA> _next HTA _data Node<HTA> Node<HTA> _next HTA _data Ardra Gregory Sonia

indexed (allows explicit access to nth item) indices of other items don’t change if an item is replaced or deleted from a specific index, but will change if new item is inserted at an index before end cannot change size dynamically ArrayLists indexed (allows explicit access to nth item) indices of other items are updated following an inserted or deleted item can grow/shrink dynamically Java uses an Array as the underlying data structure Linked Lists not indexed. In order to access the nth element, must start at the beginning and go to the next node n times → no random access can grow/shrink dynamically use nodes instead of arrays can insert/remove in middle of list without data movement throughout rest of list

Find java.util implementation at List.html For now, we’ll define our own data structure MyLinkedList difference between MyLinkedList and Java implementation is that Java includes methods beyond basic LinkedList requirements MyLinkedList is a building block for more specialized versions: stacks, queues, sorted linked lists…

Constructor initializes instance variables _first is initially set to null addFirst creates first node and sets _first to reference it addLast appends a Node to the end of the list public class MyLinkedList<Type> { private Node<Type> _first; private int _size; public MyLinkedList() { ... } public void addFirst(Type el) { public void addLast(Type el) { // more on next slide Generic type parameter

removeFirst removes first Node removeLast remove last Node remove removes the node containing the element el public Type removeFirst() { ... } public Type removeLast() { public Type remove(Type el){

public boolean contains(Type el) { ... } public int size() { public boolean isEmpty() { contains checks if list contains specified element size returns the _size of the list isEmpty checks if the list is empty

public class MyLinkedList<Type> { private Node<Type> _first; private int _size; public MyLinkedList() { ... } public void addFirst(Type el) { public void addLast(Type el) { public Type removeFirst() { ... } public Type removeLast() { public Type remove(Type el){ public boolean contains(Type el) { public int size() { public boolean isEmpty() { Generic type parameter

The Node Class public class Node<Type> { private Type _element; private Node<Type> _next; public Node(Type element) { _element=element; _next=null; } public void setNext(Node<Type> next){ _next = next; public Node<Type> getNext() { return _next; /*similar mutator and accessor method for _element are elided.*/ Node class holds its element and a reference to the next Node Its methods are made up of mutators and accessors for these variables That’s it.

Announcements DoodleJump DQs due Sunday at 2PM
Help Session Sunday at 6PM in B&H 166 DoodleJump early handin is Friday 10/31 at 10PM. On time handin is Sunday 11/2 at 11:59PM Late handin is Tuesday 11/4 at 11:59PM