CES 512 Theory of Software Systems B. Ravikumar (Ravi) Office: 141 Darwin Hall Course Web site:

Course Goals: develop critical thinking about problem solving (in the context of software design) emphasis on innovation, avoiding dull and routine line of thinking whenever possible. become exposed to interesting problems (some for pure fun, others for purpose). focus on models – understanding the context of a problem: specialization, generalization. tools to create algorithmic solutions, implement and test etc.

Context of the course in Master’s program I created this course mainly because I felt that students who want to do a thesis in Software Design need to know this course material. (More specifically, the thesis topics I would like to supervise will require background similar to this.) The emphasis (creative thinking, awareness of trends and developments etc.) is based on my belief that one can stand out by consciously demanding high expectation of self. Emphasis on theoretical ideas and models. But at every stage, we will examine the relevance of the model to practical applications.

Course Goals: tools for algorithm design divide and conquer, induction graph searching, network flow, matching linear programming dynamic programming scheduling and other optimization problems heuristics and NP-completeness tools for analysis recurrence relations, summation of series reductions between problems experimental analysis

algorithmic modeling case studies formal systems and their applications logic, grammars, automata etc.

Goals for today’s lecture Course outline Discuss assigned work course project Cover Chapters 1-3 of the text.

Algorithm Design and Analysis Efficient algorithms are a crucial part of software design tinkering with implementation and coding details can’t get us very far. root of the problem may be an inefficient algorithm. optimization problems allow multiple “correct” solutions, but only one correct and optimal solution. correctness of algorithms can be tricky to establish. predicting the behavior of algorithm is an important part of software design.

Some standard approaches to systematic algorithm design identify commonly occurring problems (such as sorting, searching, multiplication, breadth-first search etc.) and design the most efficient algorithms for them. devise general-purpose techniques (induction, divide and conquer, backtracking, dynamic programming etc.) and study when they are effective by looking at many examples. create and study models (such as trees, graphs, probabilistic algorithms) that can be used to represent problems and solutions.

Analysis of algorithms time taken to solve a problem as a function of problem size usually behaves as a well- defined function (logarithmic, linear, quadratic, n log n, exponential, etc.) when the time complexity function is complicated, it can be approximated to a well-defined function. problems can be classified based on the time complexity into easy (polynomial time solvable), provably intractable (not polynomial time solvable) and hard (not known to be polynomial time solvable, and not likely to be so).

Formal Systems and Languages Specification of software describing the syntax of a programming language (or even parts of natural language to a computer) automata applications string processing problems protocol specification and verification machine learning

Example: Arrange numbers 1, 2,..., 15 in a cyclic order so that the sum of every three consecutive numbers is at most 24. Why was 24 chosen? Can we solve this problem with 23 replacing 24? Systematically trying all permutations. Why is this not a good idea? The problem has multiple solutions. So we can try some random sequences. Algorithm for random permutation generation.

Course overview: Mathematical preliminaries induction summation, approximation, estimation inequalities, upper and lower bounds O,  and  notation recurrence relations review of discrete math Sorting and related problems quadratic time sorting algorithms heap sort and merge sort quick sort selection problem non-comparison based algorithms

Searching binary search tree AVL trees application to a geometric problem hashing Design Techniques divide and conquer dynamic programming greedy method probabilistic method backtracking genetic algorithms, neural networks etc.(?)

Graph Problems basic definitions and models DFS, BFS and applications minimum spanning tree shortest path problem matching in a bipartite graph Formal Systems and Applications finite automata context-free languages predicate logic applications

NP-completeness, decision trees etc. Lower-bound - models reductions Miscellaneous Topics string-matching cryptography parallel computers

Insertion Sorting Efficient algorithm for sorting a small number of elements. Insert(A,i) {// pre-condition: A[0.. i-1] is sorted. //post-condition: A[0.. i] is sorted. temp = A[i]; while (A[i-1] > temp) {A[i] = A[i-1]; i--;} A[i] = temp; } InsertionSort(A) { for (j = 2; j<= A.size; ++j) Insert(A,j) }

Our pseudo-code style is a little different from that of the text. (Ours will be a lot closer to Java or C++.) Analysis of InsertionSort: Focus on a specific operation. (key comparison). Text considers every operation. Measured as a function of parameter(s). In this case, it is the size of the array. Cost of the operation is a constant (1). Cases of interest: worst-case best-case average-case

Insert(A,i) {// pre-condition: A[0.. i-1] is sorted. //post-condition: A[0.. i] is sorted. temp = A[i]; while (A[i-1] > temp) {A[i] = A[i-1]; i--;} A[i] = temp; } worst-case: i best-case: 1 average: will do later. InsertionSort(A) { for (j = 1; j<= A.size; ++j) Insert(A,j) } worst-case: n(n-1)/2 best-case: n

Review of background material Mathematical Preliminaries induction - proof technique for assertion of the form “ for all integer n,... ” Example 1: For all n, n 3 + 5n is divisible by 6. Example r + … + r k = (r k+1 – 1)/(r-1) Proof by induction on k.

Inequalities, O,  and  notation For any positive real constants c and d, c n > n d for all sufficiently large n. There are positive constants c 1 and c 2 such that for all n, c 1 n log n <= log n! <= c 2 n log n. Definition: Let f(n) and g(n) be such that f(n) <= c g(n) for all large enough n. Then, we say f(n) = O(g(n)). If =, we have f(n) =  (g(n)). If both hold, f(n) =  (g(n)).

From previous slide, we have: log n! =  (n log n) Exercise: 1) 1 + 1/2+ 1/ /n =  (log n) 2) 1 + 1/ / /n 2 =  (1) 3) Estimate 1/2 + 2/ / n/2 n Summation formulas for arithmetic, geometric and arithmetic-geometric series