1. BITS Pilani Pilani Campus Data Structure and Algorithms Design Dr. Maheswari Karthikeyan Lecture1

2. BITS Pilani, Pilani Campus o Data structures: conceptual and concrete ways to organize data for efficient storage and efficient manipulation o Employment of this data structures in the design of efficient algorithms Course Outline

3. BITS Pilani, Pilani Campus o Algorithm analysis techniques o Algorithm design techniques o Implementation/analysis of data structures: Lists, stacks, and queues Trees Hashing Priority queues (heaps) Sorting Graphs Methodology

4. BITS Pilani, Pilani Campus Any organization has a collection of records that can be searched, processed in any order, or modified. o Data structures organize data: Good choice: more efficient programs Bad choice: poor program performance –The choice can make a difference between the program running in a few seconds or many day o Characteristics of a problem’s solution efficient: if it solves problem within resource constraints –time –space Cost: amount of resources a solution consumes Why to Study Data Structures?

5. BITS Pilani, Pilani Campus Costs and Benefits A data structure requires a certain amount of: space for each data item it stores time to perform a single basic operation programming effort.

6. BITS Pilani, Pilani Campus Select a data structure as follows: 1.Analyze the problem to determine the resource constraints a solution must meet. 2.Determine the basic operations that must be supported. Quantify the resource constraints for each operation. 3.Select the data structure that best meets these requirements. Selecting a Data Structure

7. BITS Pilani, Pilani Campus o A logical view of the data objects together with specifications of the operations required to create and manipulate them. Describe an algorithm – pseudo-code Describe a data structure – ADT o A data structure is the physical implementation of an ADT Each ADT operation is implemented by one or more subroutines Data structures are used to organize data in main memory Abstract Data Types (ADT)

8. BITS Pilani, Pilani Campus ADT: Type Operations Data Items: Logical Form Data Items: Physical Form Data Structure: Storage Space Subroutines Data Type

9. BITS Pilani, Pilani Campus Programmers deal with: –problems, –algorithms and –computer programs. Problems, Algorithms and Programs

10. BITS Pilani, Pilani Campus Problems, Algorithms and Programs Problem: a task to be performed. –Best thought of as inputs and matching outputs. –Problem definition should include constraints on the resources that may be consumed by any acceptable solution. Problems  mathematical functions –A function is a matching between inputs (the domain) and outputs (the range). –An input to a function may be single number, or a collection of information. –The values making up an input are called the parameters of the function. –A particular input must always result in the same output every time the function is computed.

11. BITS Pilani, Pilani Campus Problems, Algorithms and Programs Algorithm: a method or a process followed to solve a problem. –A recipe: The algorithm gives us a “recipe” for solving the problem by performing a series of steps, where each step is completely understood and can be implemented. An algorithm takes the input to a problem (function) and transforms it to the output. –A mapping of input to output. A problem can be solved by many algorithms.

12. BITS Pilani, Pilani Campus For example, the problem of sorting can be solved by the following algorithms: Insertion sort Bubble sort Selection sort Shellsort Mergesort Others Problems, Algorithms and Programs

13. BITS Pilani, Pilani Campus An algorithm possesses the following properties: –It must be correct. –It must be composed of a series of concrete steps. –There can be no ambiguity as to which step will be performed next. –It must be composed of a finite number of steps. –It must terminate. A computer program is an instance, or concrete representation, for an algorithm in some programming language. Problems, Algorithms and Programs

14. BITS Pilani, Pilani Campus The design of algorithms is also an important focus. Types of algorithms: Greedy algorithms Divide and Conquer Dynamic programming Randomized algorithms Backtracking Algorithm Design Techniques

15. BITS Pilani, Pilani Campus Predict the amount of resources required: memory: how much space is needed? computational time: how fast the algorithm runs? FACT: running time grows with the size of the input Input size (number of elements in the input) –Size of an array, polynomial degree, # of elements in a matrix, # of bits in the binary representation of the input, vertices and edges in a graph Def: Running time = the number of primitive operations (steps) executed before termination Running time is expressed as T(n) for some function T on input size n. Algorithm Analysis

16. BITS Pilani, Pilani Campus Two approaches to obtaining running time: –Measuring under standard benchmark conditions. –Estimating the algorithms performance Estimation is based on: –The “size” of the input –The number of basic operations The time to complete a basic operation does not depend on the value of its operands. Algorithm Analysis

17. BITS Pilani, Pilani Campus Running Time (Example )

18. BITS Pilani, Pilani Campus k j # runs Number of executions Running Time (Example ) n 123….n 11,21,2,3…1,2,. n 123…

19. BITS Pilani, Pilani Campus # runs = 1 + 2 + 3 + 4..+ n = ∑ j ∑ j = T(n) = c1 + c2 (n+1) + c3(n 2 + 1) + c4 (n 2 ) = Order of n 2 n j=1 n Running Time (Example) = n 2 n(n+1)/2

20. BITS Pilani, Pilani Campus a) sum1 = 0; for (k=1; k<=n; k*=2) for (j=1; j<=n; j++) sum1++; b) sum2 = 0; for (k=1; k<=n; k*=2) for (j=1; j<=k; j++) sum2++; c) sum3 = 0; for (k=1; k<=n; k++) for (j=1; j<=n; j++) sum3++; What is the running time for the following codes? Running Time (Example)

21. BITS Pilani, Pilani Campus k j # runs Number of executions Running Time (Example a ) N x log N log n 124….n 1,2,..n 1,2..n…1,2,. n nnn…

22. BITS Pilani, Pilani Campus # runs = (1 +..N) log n = ∑ n ∑ n = n log n T(n) = Order of n log n. j=1 log n j=1 Running Time (Example a)

23. BITS Pilani, Pilani Campus k j # runs Number of executions Running Time (Example b ) log n 124….n 11,21,2,3,4…1,2,. n 124… 1 + 2 + 4 + 8 + 16 + …… n

24. BITS Pilani, Pilani Campus # runs = 1+2+4+8+16..+ n = 1 + 2 1 + 2 2 + 2 3 + 2 4 + …… + 2 logn ∑ 2 i = log n j=1 Running Time (Example b) 2n-1 T(n) = Order of n.

25. BITS Pilani, Pilani Campus The growth rate for an algorithm is the rate at which the cost of the algorithm grows as the size of the input grows. –Linear Growth T(n) = n –Quadratic Growth T(n) = n 2 –Exponential Growth T(n) = 2 n –Logarithmic Growth T(n) = n log n Growth Rate

26. BITS Pilani, Pilani Campus Not all inputs of a given size take the same time to run. Best case Worst case Average case Types of Analysis

27. BITS Pilani, Pilani Campus The best case running time of an algorithm is the function defined by the minimum number of steps taken on any instance of size n. The worst case running time of an algorithm is the function defined by the maximum number of steps taken on any instance of size n. The average-case running time of an algorithm is the function defined by an average number of steps taken on any instance of size n. Types of Analysis

28. BITS Pilani, Pilani Campus Worst case –Provides an upper bound on running time –An absolute guarantee that the algorithm would not run longer, no matter what the inputs are Best case –Input is the one for which the algorithm runs the fastest Average case –Provides a prediction about the running time –Assumes that the input is random Types of Analysis (e.g. numbers reversely ordered) (e.g., numbers already ordered) (general case)

29. BITS Pilani, Pilani Campus A way to describe behavior of functions in the limit –How we indicate running times of algorithms –Describe the running time of an algorithm as n grows to  O notation: asymptotic “less than”: f(n) “≤” g(n)  notation: asymptotic “greater than”: f(n) “≥” g(n)  notation: asymptotic “equality”: f(n) “=” g(n) Asymptotic Notations

30. BITS Pilani, Pilani Campus For each of the following pairs of functions, either f(n) is O(g(n)), f(n) is Ω(g(n)), or f(n) is Θ(g(n)). Determine which relationship is correct. –f(n) = log n 2 ; g(n) = log n + 5 –f(n) = n; g(n) = log n 2 –f(n) = log log n; g(n) = log n –f(n) = n; g(n) = log 2 n Asymptotic Notations - Examples f(n) =  (g(n) ) f(n) =  (g(n)) f(n) = O(g(n)) f(n) =  (g(n))

31. BITS Pilani, Pilani Campus Intuitively: O(g(n)) = the set of functions with a smaller or same order of growth as g(n) Asymptotic notations O-notation

32. BITS Pilani, Pilani Campus 3n + 2 = O(n) ; 3n + 2 = 2 3n + 3 = O(n) ; 3n + 3 = 3 100n + 6 = O(n) ; 100n + 6 = 6 Examples

33. BITS Pilani, Pilani Campus 3n + 2 = O(n) ; 3n + 2 = 2 3n + 3 = O(n) ; 3n + 3 = 3 100n + 6 = O(n) ; 100n + 6 = 6 10 n 2 + 4n + 2 Examples = O(n 2 ) 10 n 2 + 4n + 2 = 5

34. BITS Pilani, Pilani Campus  - notation Asymptotic notations (cont.) Intuitively:  (g(n)) = the set of functions with a larger or same order of growth as g(n)

35. BITS Pilani, Pilani Campus 3n + 2 = ? 3n + 3 = ? 100n + 6 = ? Examples 3n + 2 >= 3n for all n >= 1 3n + 3 >= 3n for all n >= 1 100n + 6 >= 100n for all n >= 1

36. BITS Pilani, Pilani Campus  -notation Asymptotic notations (cont.) Intuitively  (g(n)) = the set of functions with the same order of growth as g(n)

37. BITS Pilani, Pilani Campus Introduction to the course Algorithm Analysis Topics Covered