1. SNU OOPSLA Lab. 1 Great Ideas of CS with Java Part 1 WWW & Computer programming in the language Java Ch 1: The World Wide Web Ch 2: Watch out: Here comes Java Ch 3: Numerical computation & Function Ch 4: Subroutines & Databases Ch 5: Graphics Ch 6: Simulation Ch 7: Software engineering Part 2 Understanding what a computer is and how it works Ch 8: Machine architecture Ch 9: Language translation Ch 10: Virtual Environment for Computing Ch 11: Security, Privacy, and Wishful thinking Ch 12: Computer Communication Part 3 Advanced topics Ch 13: Program Execution Time Ch 14: Parallel Computation Ch 15: Noncomputability Ch 16: Artificial intelligence

2. SNU OOPSLA Lab. Ch13. Program Execution Time Copyright © SNU OOPSLA Lab.

3. SNU OOPSLA Lab. 3 Table of Contents On the Limitations of Computer Science Program Execution Time Tractable Algorithms Selection Sort Quicksort Binary Search Intractable Algorithms Chess Some Practical Problems with Very Expensive Solutions Summary

4. SNU OOPSLA Lab. 4 On the Limitations of Computer Science Church-Markov-Turing thesis Any procedure that can be precisely explained can be coded in any PL But… There are many useful calculations that we cannot do Reasons for failure 1. The execution time may be too long real time requirements predicting yesterday's weather 2. Non-computable problems 3. The problems that we don't know the efficient algorithm yet Many of these are in the field of AI

5. SNU OOPSLA Lab. 5 Table of Contents On the Limitations of Computer Science Program Execution Time Tractable Algorithms Selection Sort Quicksort Binary Search Intractable Algorithms Chess Some Practical Problems with Very Expensive Solutions Summary

6. SNU OOPSLA Lab. 6 Program Execution Time Complexity, N Time Space Tractable computation Can be solved within reasonable amount of time Intractable calculations Can be solved, but require astounding amounts of computation for most cases

7. SNU OOPSLA Lab. 7 Table of Contents On the Limitations of Computer Science Program Execution Time Tractable Algorithms Selection Sort Quicksort Binary Search Intractable Algorithms Chess Some Practical Problems with Very Expensive Solutions Summary

8. SNU OOPSLA Lab. 8 Tractable Algorithms (1/4) Study of a Sorting Algorithm Sorting, Ordering, Alphabetizing Example (1/2) Collecting the names of all people with a specific height and weight The program will display the names of all people with the target values Figure 13.1

9. SNU OOPSLA Lab. 9 Tractable Algorithms (2/4) Time to take for searching a single record T = (where n is the number of people) Figure 13.2

10. SNU OOPSLA Lab. 10 Tractable Algorithms (3/4) The process of sorting numbers into ascending order If we use “quick sort” algorithm, T = Figure 13.3

11. SNU OOPSLA Lab. 11 Tractable Algorithms (4/4) Searching vs. Sorting Figure 13.4

12. SNU OOPSLA Lab. 12 Selection Sort (1) N items in an array named Data [ 2 | 4 | 7 | 3 | 1 | 8 | 5 ] Find smallest of elements 1 thru N of Data Interchange this with 1st element of array Data [ _ | _ | _ | _ | _ | _ | _ ] Find smallest of elements 2 thru N of Data Interchange this with 2nd element of array Data [ _ | _ | _ | _ | _ | _ | _ ] ….. Find smallest of elements K thru N of Data Interchange this with Kth element of array Data [ _ | _ | _ | _ | _ | _ | _ ] …….. Done when K = N [ _ | _ | _ | _ | _ | _ | _ ]

13. SNU OOPSLA Lab. 13 Selection Sort (2) How many comparison Operations? N-1 comparisons in first pass N-2 comparisons in second pass... 1 comparisons in last pass The above means N-1 + N-2 + N-3 +... 2 + 1 N*(N-1)/2 = N*N/2 - N/2

14. SNU OOPSLA Lab. 14 Selection Sort (3) What does “order” mean? Various values of N t = A * N * N = A *

15. SNU OOPSLA Lab. 15 Binary Search Assumes a list of sorted set like book pages Search a particular page (say page 69) in the 200 page book Step 1: Pick any page number P. If P is 69, you got it Step 2: If P is bigger than 69, the pages after P is ignored from now on Step 3: If P is smaller than 69, the pages before P is ignored from now on With the remaining pages, continue steps 1,2 and 3 T = A * log(N) // T (execution time), A (constant), N (book pages) Logarithmic execution time

16. SNU OOPSLA Lab. 16 Various Complexity (1) Linear Time Algorithms Add elements of an array Single loop algorithms t = A * N Cubic Time Algorithms matrix multiplication t = A * N^3 Polynomial Time t = A * N^K Faster machines make a lot of difference Polynomial time  tractable problems

17. SNU OOPSLA Lab. 17 Various Complexity (2) What does Order log(N) or N*log(N) Mean? Various values of N K = 1024; Mega = K * K; Giga = K * Mega; Tera = K * Giga Polynomial = Tractable

18. SNU OOPSLA Lab. 18 Table of Contents On the Limitations of Computer Science Program Execution Time Tractable Algorithms Intractable Algorithms Summary

19. SNU OOPSLA Lab. 19 Intractable computation (1) Definition: A computation is intractable if its execution time increases with increasing n faster than any polynomial Intractable problems are called “NP” problems or “NP-Complete” problems NP means non-polynomial Exponential = Intractable

20. SNU OOPSLA Lab. 20 Intractable Computation (2) Permutation problem: generating of all orderings of a set of n objects Set = { A, B, C}  ABC, ACB, BAC, BCA, CAB, CBA The Java program for the permutation problem must be straightforward. Then let’s measure the time for the following cases No of objects = 2,500 No of objects = 5,000 No of objects = 10,000 Wait 10 mins, wait 1 hour, wait 1 day, …….. What is going on?

21. SNU OOPSLA Lab. 21 Intractable Computation (3) Execution time of Permutation problem No of Objects (N) Execution Time (T sec.) 1, 2, 3, 4, 5 < 1 6 3.19 7 22.57 8 183.00 9 1,672.20 T = 4.6 X 10 ** (-3) X N! N = 10  T = 4.46 hours N = 15  T = 191 years N = 20  T = 354,875,356 YEARS

22. SNU OOPSLA Lab. 22 Intractable Computation (3) Permutation vs. Sorting vs. Searching Figure 13.4 permutation

23. SNU OOPSLA Lab. 23 Finding Minimum-Cost Path in a weight graph Figure 13.6 Shortest possible trip from A to A Traveling salesman’s problem Need to check all possible paths  (n-1)!/2

24. SNU OOPSLA Lab. 24 Traveling Salesperson Problem Visit N Cities in an optimal Order Optimality By: time distance cost N factorial (N!) Possibilities N! is roughly N^N Typical of some very practical problems, but sadly it is intractable! Representative problem in the class of NP-complete problems

25. SNU OOPSLA Lab. 25 The subset sum problem Consider {3, 21, 25, 31, 45, 57, 77, 87} Find a subset of integers that comes as close as possible to adding up to 110 but that does not exceed 110 Find all subsets of the given set and check them out! The number of subset: roughly 2^8  128 What about 100 elements: 2^100  waaoo! Big number!

26. SNU OOPSLA Lab. 26 Finding the Best Coverage of an Area: need to check all subsets of the given parts Figure 13.8

27. SNU OOPSLA Lab. 27 Finding the Best Coloring of a Graph Figure 13.9 No two connected nodes have the same color Find the minimum number of colors

28. SNU OOPSLA Lab. 28 Towers of Hanoi (1/8) Move the disks from tower1 to tower2 3 2 1 Tower 1Tower 2 Tower 3

29. SNU OOPSLA Lab. 29 Towers of Hanoi (2/8) Move the disks from tower1 to tower2 3 2 Tower 1Tower 2 Tower 3 1

30. SNU OOPSLA Lab. 30 Towers of Hanoi (3/8) Move the disks from tower1 to tower2 3 Tower 1Tower 2 Tower 3 1 2

31. SNU OOPSLA Lab. 31 Towers of Hanoi (4/8) Move the disks from tower1 to tower2 3 Tower 1Tower 2 Tower 3 1 2

32. SNU OOPSLA Lab. 32 Towers of Hanoi (5/8) Move the disks from tower1 to tower2 Tower 1Tower 2 Tower 3 1 23

33. SNU OOPSLA Lab. 33 Towers of Hanoi (6/8) Move the disks from tower1 to tower2 Tower 1Tower 2 Tower 3 123

34. SNU OOPSLA Lab. 34 Towers of Hanoi (7/8) Move the disks from tower1 to tower2 Tower 1Tower 2 Tower 3 1 2 3

35. SNU OOPSLA Lab. 35 Towers of Hanoi (8/8) Move the disks from tower1 to tower2 Tower 1Tower 2 Tower 3 1 2 3

36. SNU OOPSLA Lab. 36 Towers of Hanoi: execution time T= 5.49 X 10 ** (-3) X 2 ** (N) N disks T time

37. SNU OOPSLA Lab. 37 Chess Note: here N is number of moves looking ahead. We Have an Algorithm! Layers of look ahead: If I do this, then he does this,... Can Represent Possibilities by Tree Problem Solved (?!) Assume 10 Possibilities Each Move t = A * 10^N  !!!! Exponential What about the go game?

38. SNU OOPSLA Lab. 38 Table of Contents On the Limitations of Computer Science Program Execution Time Tractable Algorithms Intractable Algorithms Summary

39. SNU OOPSLA Lab. 39 Summary On the Limitations of Computer Science Some solvable, tractable problems Some solvable, but intractable problems Some unsolvable problems We want to solve the problem within a reasonable time limit But…. Some problems can be speed up by parallel computing techniques (Ch14) Some problems can be solved, but takes enormous amount time and the parallel computing in Ch14 cannot be helpful Some problems cannot be solved ever! (Ch 15) Some problems can be compromised by AI techniques (Ch 16)

40. SNU OOPSLA Lab. 40 Ch13: Program Execution Time Text Review Time