1. מבוא מורחב 1 Lecture 3 Material in the textbook Sections 1.1.5 to 1.2.1

2. מבוא מורחב 2 Today Continue Lecture 2 (sqrt) Refine our model  Applicative vs. Normal Order Understand how it captures the nature of processes  Recursive vs. Iterative processes  Orders of growth

3. Computing SQRT: A Numeric Algorithm To find an approximation of square root of x, use the following recipe: Make a guess G Improve the guess by averaging G and x/G Keep improving the guess until it is good enough G = 1X = 2 X/G = 2G = ½ (1+ 2) = 1.5 X/G = 4/3G = ½ (3/2 + 4/3) = 17/12 = 1.416666 X/G = 24/17G = ½ (17/12 + 24/17) = 577/408 = 1.4142156

4. (define (sqrt-iter guess x) (if (good-enough? guess x) guess (sqrt-iter (improve guess x) x))) (define (good-enough? guess x) (< (abs (- (square guess) x)) precision)) (define (improve guess x) (average guess (/ x guess))) (define (sqrt x) (sqrt-iter initial-guess x)) (define initial-guess 1.0) (define precision 0.0001)

5. Good programming Style 1. Divide the task to well-defined, natural, and simple sub-tasks. E.g: good-enough? and improve. Rule of thumb : If you can easily name it, it does a well-defined task. 2. Use parameters. E.g.: precision, initial-guess. 3. Use meaningful names.

6. Procedural abstraction It is better to: Export only what is needed Hide internal details. The procedure SQRT is of interest for the user. The procedure improve-guess is an internal detail. Exporting only what is needed leads to: A clear interface Avoids confusion

7. Rewriting SQRT (Block structure) (define (sqrt x) (define (good-enough? guess x) (< (abs (- (square guess) x)) precision)) (define (improve guess x) (average guess (/ x guess))) (define (sqrt-iter guess x) (if (good-enough? guess x) guess (sqrt-iter (improve guess x) x))) (define initial-guess 1.0) (define precision 0.00001) (sqrt-iter initial-guess x))

8. Further improving sqrt Note that in every application of sqrt we substitute for x the same value in all subsequent applications of compound procedures ! Therefore we do not have to explicitly pass x as a formal variable to all procedures. Instead, can leave it unbounded (“free”).

9. SQRT again, taking advantage of the refined substitution model (define (sqrt x) (define (good-enough? guess) (< (abs (- (square guess) x)) precision)) (define (improve guess) (average guess (/ x guess))) (define (sqrt-iter guess) (if (good-enough? guess) guess (sqrt-iter (improve guess)))) (define initial-guess 1.0) (define precision 0.00001) (sqrt-iter initial-guess))

10. SQRT (cont.) ==>(sqrt 2) (define (good-enough? guess) (< (abs (- (square guess) 2)) precision)) (define (improve guess) (average guess (/ 2 guess))) (define (sqrt-iter guess) (if (good-enough? guess) guess (sqrt-iter (improve guess)))) (define initial-guess 1.0) (define precision 0.00001) (sqrt-iter initial-guess))

11. Lexical Scoping - again The lexical scoping rules means that the value of a variable which is unbounded (free) in a procedure f is taken from the procedure in which f was defined. It is also called static scoping

12. Another example for lexical scope (define (proc1 x) (define (proc2 y) (+ x y)) (define (proc3 x) (proc2 x)) (proc3 (* 2 x))) Proc3.x Proc1.x (proc1 4) proc1.x = 4 (proc3 8) proc3.x = 8 (proc2 8) proc2.y = 8 proc2.x=proc1.x=4 12

13. מבוא מורחב 13 Applicative order vs. Normal Order Evaluation To Evaluate a combination: (other than special form) Evaluate all of the sub-expressions in any order Apply the procedure that is the value of the leftmost sub-expression to the arguments (the values of the other sub-expressions)

14. מבוא מורחב 14 Applicative order evaluation rules Combination... ( …… ) Evaluate to get the procedure and evaluate to get the arguments If is primitive: do whatever magic it does If is compound: evaluate body with formal parameters replaced by arguments

15. מבוא מורחב 15 Normal order evaluation Combination … ( …… ) Evaluate to get the procedure and evaluate to get the arguments If is primitive: do whatever magic it does If is compound: evaluate body with formal parameters replaced by arguments

16. מבוא מורחב 16 The Difference Applicative ((lambda (x) (+ x x)) (* 3 4)) (+ 12 12) 24 Normal ((lambda (x) (+ x x)) (* 3 4)) (+ (* 3 4) (* 3 4)) (+ 12 12) 24 This may matter in some cases: ((lambda (x y) (+ x 2)) 3 (/ 1 0)) Scheme is an Applicative Order Language!

17. מבוא מורחב 17 Compute a b (Recursive Approach) wishful thinking : base case: a b = a * a (b-1) a 0 = 1 (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1))))))

18. מבוא מורחב 18 Compute a b (Iterative Approach) Another approach: Operationally: Halting condition: product  product * a counter  counter - 1 counter = 0 a b = a 2 *a*…*a= a 3 *…*a Which is: a b = a * a * a*…*a b

19. מבוא מורחב 19 Compute a b (Iterative Approach) (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product))) (exp-iter a b 1)) Syntactic Recursion How then, do the two procedures differ? They give rise to different processes – lets use our model to understand how.

20. מבוא מורחב 20 Recursive Process (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1)))))) (exp-1 3 4) (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81

21. מבוא מורחב 21 Iterative Process (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 0 81) 81 (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product))) (exp-iter a b 1))

22. מבוא מורחב 22 The Difference (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 0 81) 81 (exp-1 3 4) (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81 Growing amount of space Constant amount of space

23. 23 Why More Space? Recursive exponentiation: (define exp-1 (lambda (a b) (if (= b 0) 1 (* a (exp-1 a (- b 1))))) operation pending Iterative exponentiation: (define (exp-2 a b) (define (exp-iter a counter product) (if (= counter 0) product (exp-iter a (- counter 1) (* a product)))) (exp-iter a b 1)) no pending operations

24. מבוא מורחב 24 Example: Factorial wishful thinking : base case: n! = n * (n-1)! n = 1 (define fact (lambda (n) (if (= n 1) 1 (* n (fact (- n 1)))))) Iterative or Recursive?

25. מבוא מורחב 25 Summary Recursive process num of deferred operations “grows proportional to b” Iterative process num of deferred operations stays “constant” (actually it’s zero) Can we better quantify these observations? Orders of growth…

26. 26 Order of Growth: Recursive Process (exp-1 3 4) (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81  4 (exp-1 3 5) (* 3 (exp-1 3 4)) (* 3 (* 3 (exp-1 3 3))) (* 3 (* 3 (* 3 (exp-1 3 2)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 1))))) (* 3 (* 3 (* 3 (* 3 3)))) (* 3 (* 3 (* 3 9))) (* 3 (* 3 27)) (* 3 81) 243  5 (* 3 (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 (* 3 1)))) Dependent on b

27. 27 Iterative Process (define (exp-2 a b) (define (exp-iter a b product) (if (= b 0) product (exp-iter a (- b 1) (* a product)))) (exp-iter a b 1) (exp-2 3 4) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 0 81) 81 (exp-2 3 5) (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 0 81) 243 (exp-iter 3 2 27) (exp-iter 3 0 243) Some constant, independent of b

28. מבוא מורחב 28 Orders of Growth Suppose n is a parameter that measures the size of a problem (the size of its input) R(n) measures the amount of resources needed to compute a solution procedure of size n. Two common resources are space, measured by the number of deferred operations, and time, measured by the number of primitive steps. The worst-case over all inputs of size n

29. מבוא מורחב 29 Orders of Growth Want to estimate the “order of growth” of R(n): R 1 (n)=100n 2 R 2 (n)=2n 2 +10n+2 R 3 (n) = n 2 Are all the same in the sense that if we multiply the input by a factor of 2, the resource consumption increases by a factor of 4 Order of growth is proportional to n 2

30. מבוא מורחב 30 Orders of Growth We say R(n) has order of growth  (f(n))  if there are constants c 1  0 and c 2  0 such that for all n c 1 f(n) <= R(n) <= c 2 f(n) R(n)  (f(n)) if there is a constant c  0 such that for all n c f(n) <= R(n) R(n)  O(f(n)) if there is a constant c  0 such that for all n R(n) <= c f(n)

31. מבוא מורחב 31 Orders of Growth t t f f 100n 2  O(n) 100n 2   (n) 100n 2   (n) 100n 2   (n 2 ) True or False? t t f f 2 100   (n) 2 100 n  O(n 2 ) 2 n   (n) 2 10   (1) True or False?

32. מבוא מורחב 32 Resources Consumed by EXP-1 (exp-1 3 4) “n”=b=4 (* 3 (exp-1 3 3)) (* 3 (* 3 (exp-1 3 2))) (* 3 (* 3 (* 3 (exp-1 3 1)))) (* 3 (* 3 (* 3 (* 3 (exp-1 3 0))))) (* 3 (* 3 (* 3 (* 3 1)))) (* 3 (* 3 (* 3 3))) (* 3 (* 3 9)) (* 3 27) 81 Space b <= R( b ) <= b which is  b  Time b <= R( b ) <= 2 b which is  b  Linear Recursive Process

33. מבוא מורחב 33 Resources Consumed by EXP-2 (exp-2 3 4) “n”=b=4 (exp-iter 3 4 1) (exp-iter 3 3 3) (exp-iter 3 2 9) (exp-iter 3 1 27) (exp-iter 3 0 81) 81 Space  1  Time  b  Linear Iterative Process

34. מבוא מורחב 34 Summary Trying to capture the nature of processes Quantify various properties of processes:  Number of steps a process takes (Time Complexity)  Amount of Space a process uses (Space Complexity)