CPSC 311 Section 502 Analysis of Algorithm Fall 2002 Department of Computer Science Texas A&M University

CPSC 311-502 Instructor: O. Burchan Bayazit (pronounced as bourch han) CPSC 311 O. Burchan Bayazit – Fall 02 CPSC 311-502 Instructor: O. Burchan Bayazit (pronounced as bourch han) Office: 407 E Harvey R. Bright Bldg. Email: burchanb@cs.tamu.edu Office Hours: 4:00pm-5:30pm Tuesdays Teaching Assistant: Kasthuri Srinivasan Kannan Email: ksk7657@cs.tamu.edu Office Hours: 4:00pm-5:30pm Thursdays

CPSC 311 O. Burchan Bayazit – Fall 02 Online Material Course homepage: http://parasol.tamu.edu/~burchanb/Courses/311/ Mailing List: (announcements only) cpsc-311-02c@listserv.tamu.edu Discussion Group: (announcements+discussions) news://news.tamu.edu/tamu.classes.cpsc311

Class Meeting: 9:35-10:50am, Tuesdays and Thursdays ZACH 105B CPSC 311 O. Burchan Bayazit – Fall 02 Course Information Class Meeting: 9:35-10:50am, Tuesdays and Thursdays ZACH 105B Textbook: Introduction to Algorithms, Cormen, Leiserson, Rivest, Stein, McGraw-Hill (and MIT Press), 2nd Edition (latest)

CPSC 311 O. Burchan Bayazit – Fall 02 Prerequisites CPSC 211 (Data Structures) and Math 302 (Discrete Mathematics). In particular, you should be familiar with: mathematical solutions to recurrence relations mathematical induction data structures including linked lists, arrays, trees, and graphs knowledge of a high level block structured language such as Pascal, C or Java

Mechanics Assignments (25%) Quizzes (15%) Exams (60%, 20% each) CPSC 311 O. Burchan Bayazit – Fall 02 Mechanics Assignments (25%) Homework (6-10) In-Class Programming Assignments CS Culture Activities Quizzes (15%) Exams (60%, 20% each) Two midterms and a final

OK, what about algorithms? CPSC 311 O. Burchan Bayazit – Fall 02 OK, what about algorithms? Algorithm: a sequence of computational steps that transform given input to the output

Course Goals At the end of the semester you should: CPSC 311 O. Burchan Bayazit – Fall 02 Course Goals At the end of the semester you should: be familiar with fundamental algorithms and algorithmic techniques, given a particular application, be able to decide which algorithm among a set of possible choices is best, be able to prove correctness and analyze the running time of a given algorithm, be able to design efficient algorithms for new situations, using as building blocks the techniques learned, be able to prove a problem is NP-complete using reduction.

CPSC 311 O. Burchan Bayazit – Fall 02 Course Outline Introduction and Mathematical Fundamentals - (Ch. 1, 2, 3, 4 ) Sorting - (Ch. 6, 7, 8) Selection - (Ch. 9) EXAM 1 Hashing - (Ch. 11) Basic Graph Algorithms - (Ch. 22 ) More Graph Algorithms - (Ch. 23, 24, 25) Union-Find/Disjoint Set Data Structure - (Ch. 21) Dynamic Programming - (Ch. 15) EXAM 2 NP-Completeness - (Ch. 34, 35) Special Topics TBD EXAM 3 (FINAL) (Friday December 13, 12:30-2:30pm)

Quiz Time !!!!! Fill out the personal information in the front page CPSC 311 O. Burchan Bayazit – Fall 02 Quiz Time !!!!! Fill out the personal information in the front page Answer the questions of the quiz in the back page

Answer to Question 1. Shown binary search for 13 1 2 3 4 5 6 7 8 9 10 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 1. Shown binary search for 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Answer to Question 1. Find the middle 1 2 3 4 5 6 7 8 9 10 11 12 13 14 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 1. Find the middle 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Answer to Question 1. Select the right side (8<13) 1 2 3 4 5 6 7 8 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 1. Select the right side (8<13) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Answer to Question 1. Select the right side (8<13) 1 2 3 4 5 6 7 8 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 1. Select the right side (8<13) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Answer to Question 1. Find the middle 1 2 3 4 5 6 7 8 9 10 11 12 13 14 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 1. Find the middle 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Assume we have only integers (11 is also correct, if you take floor of the mid-point)

Answer to Question 1. Select right side (12<13) 1 2 3 4 5 6 7 8 9 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 1. Select right side (12<13) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Answer to Question 1. Find the middle 1 2 3 4 5 6 7 8 9 10 11 12 13 14 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 1. Find the middle 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Answer to Question 1. Select the left side (13<14) 1 2 3 4 5 6 7 8 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 1. Select the left side (13<14) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Answer to Question 1. Find the middle = 13 1 2 3 4 5 6 7 8 9 10 11 12 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 1. Find the middle = 13 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Answer to Question 2. What is returned on input 20? CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. What is returned on input 20? Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) }

Answer to Question 2. x=20 Stack Subroutine whatami(integer x) { CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=20 Stack whatami(20)

Answer to Question 2. x=20 Stack Subroutine whatami(integer x) { CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=20 Stack whatami(20)

Answer to Question 2. x=20 Stack Subroutine whatami(integer x) { CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=20 Stack whatami(20)

Answer to Question 2. x=10 Stack Subroutine whatami(integer x) { CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=10 Stack whatami(10) whatami(20)

Answer to Question 2. x=10 Stack Subroutine whatami(integer x) { CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=10 Stack whatami(10) whatami(20)

Answer to Question 2. x=10 Stack Subroutine whatami(integer x) { CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=10 Stack whatami(10) whatami(20)

Answer to Question 2. x=5 Stack Subroutine whatami(integer x) { CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=5 Stack whatami(5) whatami(10) whatami(20)

Answer to Question 2. x=5 Stack Subroutine whatami(integer x) { CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=5 Stack whatami(5) whatami(10) whatami(20)

Answer to Question 2. x=5 Stack Subroutine whatami(integer x) { CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=5 Stack whatami(5) whatami(10) whatami(20)

Answer to Question 2. x=5 Stack Return 2 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=5 Stack whatami(5) whatami(10) whatami(20) Return 2

Answer to Question 2. x=10 Stack Return 2 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=10 Stack whatami(10) whatami(20) Return 2

Answer to Question 2. x=20 Stack Return 2 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 2. Subroutine whatami(integer x) { if (x is even) then return (whatami(x/2)) else return ((x-1)/2) } x=20 Stack whatami(20) Return 2

Answer to Question 3. Show the link list after execution d e a b c tmp CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. Show the link list after execution d e a b c tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2 tmp

Answer to Question 3. d e a b c tmp tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 3. d e a b c tmp tmp1 tmp2 tmp1=next(next(tmp)) tmp2=next(tmp1) next(tmp1)=next(tmp2) next(tmp2)=next(tmp) next(tmp)=tmp2

Answer to Question 4. Use induction to prove 1+2+…+n=n(n+1)/2 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 4. Use induction to prove 1+2+…+n=n(n+1)/2

Answer to Question 4. Step 1: Verify that eq. holds for n=1 CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 4. Step 1: Verify that eq. holds for n=1 Left hand side is 1 and right hand site 1*(1+1)/2, so the equation holds for n=1

Answer to Question 4. Step 2: assume that eq. holds for n=k CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 4. Step 2: assume that eq. holds for n=k 1+2+…+k= k(k+1)/2

CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 4. Step 3: using assumption at Step2, show that eq. holds for n=k+1 From Step 2: 1+2+…+k= k(k+1)/2 If we add k+1 to each side 1+2+….+k+ (k+1) = k(k+1)/2 + (k+1) or 1+2+….+k+ (k+1) = (k(k+1) + 2(k+1))/2

Answer to Question 4. Step 3: cont. CPSC 311 O. Burchan Bayazit – Fall 02 Answer to Question 4. Step 3: cont. 1+2+….+k+ (k+1) = (k(k+1) + 2(k+1))/2 Becomes 1+2+….+k+ (k+1) = (k+1)( k+2))/2 Hence Eq. Holds for n=k+1

CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort 1 2 3 4 5 6 5 2 4 6 1 3

Insertion Sort 5 2 4 6 1 3 Insert 2 into sorted part sorted CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort sorted 5 2 4 6 1 3 Insert 2 into sorted part

Insertion Sort 5 4 6 1 3 2 After Insertion sorted CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort sorted 5 4 6 1 3 2 After Insertion

Insertion Sort 5 4 6 1 3 2 Insert 4 into the sorted part sorted CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort sorted 5 4 6 1 3 2 Insert 4 into the sorted part

Insertion Sort 4 5 6 1 3 2 Insert 6 into the sorted part sorted CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort sorted 4 5 6 1 3 2 Insert 6 into the sorted part

Insertion Sort 4 5 6 1 3 2 Insert 1 into the sorted part sorted CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort sorted 4 5 6 1 3 2 Insert 1 into the sorted part

Insertion Sort 2 4 5 6 3 1 Insert 3 into the sorted part sorted CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort sorted 2 4 5 6 3 1 Insert 3 into the sorted part

CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort sorted 1 2 3 4 5 6

CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort 6 1 2 3 4 5

CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort How to insert? (assume we inserted previous elements and we are at the last element) j=6 (element’s position) 2 4 5 6 3 1 Start from i=j-1

Insertion Sort How to insert? 2 4 5 6 3 1 i=5 If(array[i]>Element) CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort How to insert? 2 4 5 6 3 1 i=5 If(array[i]>Element) switch Array[i] and Array [i+1]

Insertion Sort How to insert? 2 4 5 3 6 1 i=4 If(array[i]>Element) CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort How to insert? 2 4 5 3 6 1 i=4 If(array[i]>Element) switch Array[i] and Array [i+1]

Insertion Sort How to insert? 2 4 3 5 6 1 i=3 If(array[i]>Element) CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort How to insert? 2 4 3 5 6 1 i=3 If(array[i]>Element) switch Array[i] and Array [i+1]

Insertion Sort How to insert? 2 3 4 5 6 1 i=2 If(array[i]<=Element) CPSC 311 O. Burchan Bayazit – Fall 02 Insertion Sort How to insert? 2 3 4 5 6 1 i=2 If(array[i]<=Element) stop Insertion

Psuedo Algorithm for Insertion Sort CPSC 311 O. Burchan Bayazit – Fall 02 Psuedo Algorithm for Insertion Sort for j=2 to Length(Array) { key=Array[j] i=j-1 while (i>0 and A[i]>key) { A[I+1]=A[I] i=i-1 } A[i+1]=key

Psuedo Algorithm for Insertion Sort CPSC 311 O. Burchan Bayazit – Fall 02 Psuedo Algorithm for Insertion Sort cost for j=2 to Length(Array) { key=Array[j] i=j-1 while (i>0 and A[i]>key) { A[I+1]=A[I] i=i-1 } A[i+1]=key c1 c2 c3 c4 c5 c6 c7

Psuedo Algorithm for Insertion Sort CPSC 311 O. Burchan Bayazit – Fall 02 Psuedo Algorithm for Insertion Sort cost times for j=2 to Length(Array) { key=Array[j] i=j-1 while (i>0 and A[i]>key) { A[I+1]=A[I] i=i-1 } A[i+1]=key c1 n n-1 c2 c3 n-1 c4 sum(2,n)(tj) c5 sum(2,n)(tj-1) sum(2,n)(tj-1) c6 c7 n-1

Cost of Insertion Sort cost times T(n)=c1.n+c2.(n-1)+c2.(n-1)+ CPSC 311 O. Burchan Bayazit – Fall 02 Cost of Insertion Sort cost times c1 n n-1 c2 T(n)=c1.n+c2.(n-1)+c2.(n-1)+ c4. sum(2,n)(tj)+c5. sum(2,n)(tj-1)+ c6. sum(2,n)(tj-1)+c7.(n-1) c3 n-1 c4 sum(2,n)(tj) c5 sum(2,n)(tj-1) sum(2,n)(tj-1) c6 c7 n-1

Best-Case cost In the best case, the array is already sorted, hence CPSC 311 O. Burchan Bayazit – Fall 02 Best-Case cost In the best case, the array is already sorted, hence tj =1 cost times c1 n n-1 c2 c3 n-1 c4 sum(2,n)(tj) T(n)=c1.n+c2.(n-1)+c2.(n-1)+ c3.(n-1)+c4.(n-1)+c7.(n-1) = (c1+c2+c3+c4+c7)n-(c2+c3+c4+c7) c5 sum(2,n)(tj-1) sum(2,n)(tj-1) c6 c7 n-1 T(n) is a linear function of n

CPSC 311 O. Burchan Bayazit – Fall 02 Worst-Case cost In the worst case, the array is already sorted in the reverse order hence tj =j cost times c1 n n-1 c2 c3 n-1 c4 sum(2,n)(tj) sum(2,n)(tj) becomes sum(2,n)(j) =n*(n+1)/2-1 sum(2,n)(tj-1) becomes sum(2,n)(j-1) =n*(n-1)/2 c5 sum(2,n)(tj-1) sum(2,n)(tj-1) c6 c7 n-1 T(n) is a linear function of n

Worst-Case cost cost times T(n) is a quadratic function of n CPSC 311 O. Burchan Bayazit – Fall 02 Worst-Case cost cost times T(n)=c1.n+c2.(n-1)+c3.(n-1)+c4.(n.(n+1)/2-1) +c5.n.(n-1)/2+c6.n.(n-1)/2+c7.(n-1) =(c4+c5+c6)/2.n2+ (c1+c2+c3+(c4-c5-c6)/2+c7).n –(c2+c3+c4+c7) c1 n n-1 c2 c3 n-1 c4 sum(2,n)(tj) c5 sum(2,n)(tj-1) sum(2,n)(tj-1) c6 c7 n-1 T(n) is a quadratic function of n

CPSC 311 O. Burchan Bayazit – Fall 02 Average-Case cost In the average case, we need the search half of the sorted part of the array hence tj =j/2 cost times c1 n n-1 c2 c3 n-1 c4 sum(2,n)(tj) c5 sum(2,n)(tj-1) sum(2,n)(tj-1) c6 c7 n-1 T(n) is a quadratic function of n