Analysis of Algorithms 7/2/2015CS202 - Fundamentals of Computer Science II1.

Slides:



Advertisements
Similar presentations
Md. Ahsan Arif, Assistant Professor, Dept. of CSE, AUB
Advertisements

HST 952 Computing for Biomedical Scientists Lecture 10.
Fundamentals of Python: From First Programs Through Data Structures
Algorithm Complexity Analysis: Big-O Notation (Chapter 10.4)
Fall 2006CENG 7071 Algorithm Analysis. Fall 2006CENG 7072 Algorithmic Performance There are two aspects of algorithmic performance: Time Instructions.
Chapter 10 Algorithm Efficiency
Introduction to Analysis of Algorithms
© 2006 Pearson Addison-Wesley. All rights reserved10-1 Chapter 10 Algorithm Efficiency and Sorting CS102 Sections 51 and 52 Marc Smith and Jim Ten Eyck.
Scott Grissom, copyright 2004 Chapter 5 Slide 1 Analysis of Algorithms (Ch 5) Chapter 5 focuses on: algorithm analysis searching algorithms sorting algorithms.
Complexity Analysis (Part I)
© 2006 Pearson Addison-Wesley. All rights reserved6-1 More on Recursion.
Cmpt-225 Algorithm Efficiency.
The Efficiency of Algorithms
Algorithm Analysis CS 201 Fundamental Structures of Computer Science.
The Efficiency of Algorithms
Algorithm Efficiency and Sorting Bina Ramamurthy CSE116A,B.
Liang, Introduction to Java Programming, Eighth Edition, (c) 2011 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
Analysis of Algorithm.
Analysis of Algorithms COMP171 Fall Analysis of Algorithms / Slide 2 Introduction * What is Algorithm? n a clearly specified set of simple instructions.
© 2006 Pearson Addison-Wesley. All rights reserved10 A-1 Chapter 10 Algorithm Efficiency and Sorting.
Analysis of Algorithms Spring 2015CS202 - Fundamentals of Computer Science II1.
Algorithm Analysis (Big O)
Liang, Introduction to Java Programming, Seventh Edition, (c) 2009 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
Time Complexity Dr. Jicheng Fu Department of Computer Science University of Central Oklahoma.
Analysis of Algorithm Lecture 3 Recurrence, control structure and few examples (Part 1) Huma Ayub (Assistant Professor) Department of Software Engineering.
Week 2 CS 361: Advanced Data Structures and Algorithms
1 Chapter 24 Developing Efficient Algorithms. 2 Executing Time Suppose two algorithms perform the same task such as search (linear search vs. binary search)
Introduction to complexity. 2 Analysis of Algorithms Why do we need to analyze algorithms? –To measure the performance –Comparison of different algorithms.
1 Recursion Algorithm Analysis Standard Algorithms Chapter 7.
{ CS203 Lecture 7 John Hurley Cal State LA. 2 Execution Time Suppose two algorithms perform the same task such as search (linear search vs. binary search)
Mathematics Review and Asymptotic Notation
Analysis of Algorithms
Copyright © 2007 Pearson Education, Inc. Publishing as Pearson Addison-Wesley. Ver Chapter 9: Algorithm Efficiency and Sorting Data Abstraction &
1 7.Algorithm Efficiency What to measure? Space utilization: amount of memory required  Time efficiency: amount of time required to process the data Depends.
1 COMP3040 Tutorial 1 Analysis of algorithms. 2 Outline Motivation Analysis of algorithms Examples Practice questions.
© 2011 Pearson Addison-Wesley. All rights reserved 10 A-1 Chapter 10 Algorithm Efficiency and Sorting.
Chapter 10 A Algorithm Efficiency. © 2004 Pearson Addison-Wesley. All rights reserved 10 A-2 Determining the Efficiency of Algorithms Analysis of algorithms.
CS1020 Data Structures and Algorithms I Lecture Note #11 Analysis of Algorithms.
1 7.Algorithm Efficiency What to measure? Space utilization: amount of memory required  Time efficiency: amount of time required to process the data.
CSS342: Algorithm Analysis1 Professor: Munehiro Fukuda.
Program Efficiency & Complexity Analysis. Algorithm Review An algorithm is a definite procedure for solving a problem in finite number of steps Algorithm.
Algorithm Efficiency and Sorting Data Structure & Algorithm.
Chapter 10 Algorithm Analysis.  Introduction  Generalizing Running Time  Doing a Timing Analysis  Big-Oh Notation  Analyzing Some Simple Programs.
Algorithm Analysis CS 400/600 – Data Structures. Algorithm Analysis2 Abstract Data Types Abstract Data Type (ADT): a definition for a data type solely.
Chapter 5 Algorithms (2) Introduction to CS 1 st Semester, 2015 Sanghyun Park.
Lecture 10 – Algorithm Analysis.  Next number is sum of previous two numbers  1, 1, 2, 3, 5, 8, 13, 21 …  Mathematical definition 2COMPSCI Computer.
Algorithm Analysis Part of slides are borrowed from UST.
1/6/20161 CS 3343: Analysis of Algorithms Lecture 2: Asymptotic Notations.
Algorithm Analysis (Big O)
Liang, Introduction to Java Programming, Sixth Edition, (c) 2007 Pearson Education, Inc. All rights reserved Chapter 23 Algorithm Efficiency.
E.G.M. PetrakisAlgorithm Analysis1  Algorithms that are equally correct can vary in their utilization of computational resources  time and memory  a.
DS.A.1 Algorithm Analysis Chapter 2 Overview Definitions of Big-Oh and Other Notations Common Functions and Growth Rates Simple Model of Computation Worst.
Complexity of Algorithms Fundamental Data Structures and Algorithms Ananda Guna January 13, 2005.
Analysis of Algorithms Spring 2016CS202 - Fundamentals of Computer Science II1.
GC 211:Data Structures Week 2: Algorithm Analysis Tools Slides are borrowed from Mr. Mohammad Alqahtani.
Chapter 15 Running Time Analysis. Topics Orders of Magnitude and Big-Oh Notation Running Time Analysis of Algorithms –Counting Statements –Evaluating.
Algorithm Complexity Analysis (Chapter 10.4) Dr. Yingwu Zhu.
1 7.Algorithm Efficiency These factors vary from one machine/compiler (platform) to another  Count the number of times instructions are executed So, measure.
Algorithm Analysis 1.
CS 302 Data Structures Algorithm Efficiency.
Introduction to Analysis of Algorithms
Analysis of Algorithms
Analysis of Algorithms
Time Complexity Analysis Neil Tang 01/19/2010
Algorithm Analysis (not included in any exams!)
CS 201 Fundamental Structures of Computer Science
Algorithm Efficiency and Sorting
Analysis of Algorithms
Algorithm Analysis Bina Ramamurthy CSE116A,B.
Analysis of Algorithms
Presentation transcript:

Analysis of Algorithms 7/2/2015CS202 - Fundamentals of Computer Science II1

Syllabus Introduction Important Notes 7/2/2015CS202 - Fundamentals of Computer Science II2

7/2/2015CS202 - Fundamentals of Computer Science II3 Algorithm: idea of a program

7/2/2015CS202 - Fundamentals of Computer Science II4 Algorithm An algorithm is a set of instructions to be followed to solve a problem. –There can be more than one solution (more than one algorithm) to solve a given problem. –An algorithm can be implemented using different prog. languages on different platforms. Once we have a correct algorithm for the problem, we have to determine the efficiency of that algorithm. – How much time that algorithm requires. – How much space that algorithm requires. We will focus on –How to estimate the time required for an algorithm –How to reduce the time required

7/2/2015CS202 - Fundamentals of Computer Science II5 Analysis of Algorithms How do we compare the time efficiency of two algorithms that solve the same problem? We should employ mathematical techniques that analyze algorithms independently of specific implementations, computers, or data. To analyze algorithms: –First, we start counting the number of significant operations in a particular solution to assess its efficiency. –Then, we will express the efficiency of algorithms using growth functions.

7/2/2015CS202 - Fundamentals of Computer Science II6 Analysis of Algorithms Simple instructions (+,-,*,/,=,if,call) take 1 step Loops and subroutine calls are not simple operations –They depend on size of data and the subroutine –“sort” is not a single step operation –Complex Operations (matrix addition, array resizing) are not single step We assume infinite memory We do not include the time required to read the input

7/2/2015CS202 - Fundamentals of Computer Science II7 The Execution Time of Algorithms Consecutive statements Times count = count + 1; 1 sum = sum + count; 1 Total cost =  The time required for this algorithm is constant Don’t forget: We assume that each simple operation takes one unit of time

7/2/2015CS202 - Fundamentals of Computer Science II8 The Execution Time of Algorithms If-else statements Times if (n < 0){ 1 absval = -n 1 cout << absval; 1 } else absval = n; 1 Total Cost <= 1 + max(2,1)

7/2/2015CS202 - Fundamentals of Computer Science II9 The Execution Time of Algorithms Single loop statements Times i = 1; 1 sum = 0; 1 while (i <= n) { n + 1 i = i + 1; n sum = sum + i; n } Total cost = (n + 1) + n + n  The time required for this algorithm is proportional to n

7/2/2015CS202 - Fundamentals of Computer Science II10 The Execution Time of Algorithms Nested loop statements Times i = 1; 1 sum = 0; 1 while (i <= n) { n + 1 j=1; n while (j <= n) { n * (n + 1) sum = sum + i; n * n j = j + 1; n * n } i = i +1; n } Total cost = (n + 1) + n + n * (n + 1) + n * n + n * n + n  The time required for this algorithm is proportional to n 2

7/2/ Algorithm Growth Rates We measure the time requirement of an algorithm as a function of the problem size. The most important thing is to learn how quickly the time requirement of an algorithm grows as a function of the problem size. An algorithm’s proportional time requirement is known as growth rate. We can compare the efficiency of two algorithms by comparing their growth rates. The time requirement as a function of the problem size n

7/2/2015CS202 - Fundamentals of Computer Science II12 Big Oh notation: anyone knows?

7/2/2015CS202 - Fundamentals of Computer Science II13 Order-of-Magnitude Analysis and Big-O Notation If Algorithm A requires time proportional to f(n), it is said to be order f(n), and it is denoted as O(f(n)). f(n) is called the algorithm’s growth-rate function. Since the capital O is used in the notation, this notation is called the Big-O notation.

7/2/2015CS202 - Fundamentals of Computer Science II14 Big-O Notation Definition: Algorithm A is order of if it requires no more than time units to solve a problem of size –There may exist many values of c and n 0 More informally, is an upper bound on

Big-O Notation Big-O definition implies: constant n 0 beyond which it is satisfied We do not care about small values of n 7/2/2015CS202 - Fundamentals of Computer Science II15 O(n) Ω(n) Θ(n) c*f(n) T(n) c*f(n) c 1 *f(n) c 2 *f(n) T(n)

7/2/2015CS202 - Fundamentals of Computer Science II16 Example Show that is order of –Show that there exist constants c and n 0 that satisfy the condition Try c = 3 and n 0 = 2

7/2/ A Comparison of Growth-Rate Functions

7/2/2015CS202 - Fundamentals of Computer Science II18

A Comparison of Growth-Rate Functions Any algorithm with n! complexity is useless for n>=20 Algorithms with 2 n running time is impractical for n>=40 Algorithms with n 2 running time is usable up to n=10,000 –But not useful for n>1,000,000 Linear time (n) and n log n algorithms remain practical even for one billion items Algorithms with log n complexity is practical for any value of n 7/2/2015CS202 - Fundamentals of Computer Science II19

7/2/2015CS202 - Fundamentals of Computer Science II20 Conclusions from prev table

7/2/2015CS202 - Fundamentals of Computer Science II21 Properties of Growth-Rate Functions 1.We can ignore the low-order terms –If an algorithm is O(n 3 +4n 2 +3n), it is also O(n 3 ) –Use only the highest-order term to determine its grow rate 2.We can ignore a multiplicative constant in the highest-order term –If an algorithm is O(5n 3 ), it is also O(n 3 ) 3.O( f(n) ) + O( g(n) ) = O( f(n) + g(n) ) –If an algorithm is O(n 3 ) + O(4n 2 ), it is also O(n 3 +4n 2 )  So, it is O(n 3 ) –Similar rules hold for multiplication

7/2/2015CS202 - Fundamentals of Computer Science II22 Some Useful Mathematical Equalities

7/2/2015CS202 - Fundamentals of Computer Science II23 Growth-Rate Functions Remember our previous examples Times i = 1; 1 sum = 0; 1 while (i <= n) { n + 1 i = i + 1; n sum = sum + i; n } Total cost = (n + 1) + n + n = 3 * n + 3  The time required for this algorithm is proportional to n  The growth-rate of this algorithm is proportional to O(n)

7/2/2015CS202 - Fundamentals of Computer Science II24 Growth-Rate Functions Times i = 1; 1 sum = 0; 1 while (i <= n) { n + 1 j=1; n while (j <= n) { n * (n + 1) sum = sum + i; n * n j = j + 1; n * n } i = i +1; n } Total cost = (n + 1) + n + n * (n + 1) + n * n + n * n + n Total cost = 3 * n * n + 3  The time required for this algorithm is proportional to n 2  The growth-rate of this algorithm is proportional to O(n 2 )

7/2/2015CS202 - Fundamentals of Computer Science II25 Conclusions from prev table

7/2/2015CS202 - Fundamentals of Computer Science II26 What to Analyze Worst-case performance –It is an upper bound for any input –Its use is more common than the others Best-case performance –This is useless! Why? Average-case performance –It is valid if you can figure out what the “average” input is –It is computed considering all possible inputs and their distribution –It is usually difficult to compute

7/2/2015CS202 - Fundamentals of Computer Science II27 Consider the sequential search algorithm int sequentialSearch(const int a[], int item, int n){ for (int i = 0; i < n; i++) if (a[i] == item) return i; return 0; } Worst-case: –If the item is in the last location of the array or –If it is not found in the array Best-case: –If the item is in the first location of the array Average-case: –How can we compute it?

How to find the growth-rate of C++ codes? 7/2/2015CS202 - Fundamentals of Computer Science II28

Some Examples Solved on the Board. 7/2/2015CS202 - Fundamentals of Computer Science II29

What about recursive functions? 7/2/2015CS202 - Fundamentals of Computer Science II30

Consider the problem of Hanoi towers void hanoi(int n, char source, char dest, char spare) { if (n > 0) { hanoi(n - 1, source, spare, dest); move from source to dest hanoi(n - 1, spare, dest, source); } How do we find the growth-rate of the recursive hanoi function? First write a recurrence equation for the hanoi function Then solve the recurrence equation –There are many methods to solve recurrence equations We will learn a simple one known as repeated substitutions 7/2/2015CS202 - Fundamentals of Computer Science II31

Let’s first write a recurrence equation for the hanoi function We will then solve it by using repeated substitutions 7/2/201532

More examples Factorial function Binary search Merge sort – later 7/2/2015CS202 - Fundamentals of Computer Science II33