1. 6.001 SICP – September 22 6001-Introduction
Trevor Darrell
32-D512
6.001 web page:
section web page:
Office Hours W11, 32-D512
Orders of Growth
Linear
Exponential
Logarithmic
Let
6.001 SICP

2. 6.001 SICP Orders of Growth Linear Exponential Logarithmic let

3. Orders of growth of processes
Suppose n is a parameter that measures the size of a problem
Let R(n) be the amount of resources needed to compute a procedure of size n.
We say R(n) has order of growth Q(f(n)) if there are constants k1 and k2 such that k1f(n)<= R(n)<= k2f(n) for large n
Two common resources are space, measured by the number of deferred operations, and time, measured by the number of primitive steps.
6.001 SICP

4. Orders of Growth N=2 N=10 N=100 Constant Q(1) 1 1 1
Logarithmic Q(log N)
Linear Q(n)
Quadratic Q(n2)
Exponential Q(2n) x1030
At 1 billion operations per second (current state of the art), if you were to run an exponential time algorithm in the lab on a data set of size n=100, you would be waiting for approximately 4x1011 centuries for the code to finish running!
6.001 SICP

5. Order of growth examples
For each, find simplest and slowest growing f for which R(n)=Q(f(n))
R(n)=6
R(n)=n2 + 3
R(n)=6*n3 + 3*n2 + 7n + 100
R(n)=23n+7
6.001 SICP

6. Order of growth examples
For each, find simplest and slowest growing f for which R(n)=Q(f(n))
R(n)=6
Q(1) * 1 <= 6 <= 6 * 1 for all n
R(n)=n2 + 3
Q(n2) 1*n2 <= n2 + 3 <= 2*n2 for all n > 2
R(n)=6*n3 + 3*n2 + 7n + 100
Q(n3) 1*n3 <= 6*n3 + 3*n2 + 7n <= 7*n3 for all n>100
R(n)=23n+7
Q(8n) 1*8n <= 23n+7 <= 28*8n for all n>0
6.001 SICP

7. Examples Time = Q(n) Space = Q(n) (define (factorial n) (if (= n 1) 1
(* n (factorial (- n 1)))))
(factorial 5)
(* 5 (factorial 4))
(* 5 (* 4 (factorial 3)))
(* 5 (* 4 (* 3 (factorial 2))))
(* 5 (* 4 (* 3 (* 2 (factorial 1)))))
(* 5 (* 4 (* 3 (* 2 1))))
(* 5 (* 4 (* 3 2)))
(* 5 (* 4 6))
120
Time = Q(n) Space = Q(n)
6.001 SICP

8. Examples Time = Q(n) Space = Q(1) (define (fact2 n)
(define (helper cur k)
(if (= k 1)
cur
(helper (* cur k) (- k 1))))
(helper 1 n))
(fact2 5)
(helper 1 5)
(helper 5 4)
(helper 20 3)
(helper 60 2)
(helper 120 1)
Time = Q(n) Space = Q(1)
6.001 SICP

9. Towers of Hanoi Three posts, and a set of different size disks
any stack must be sorted in decreasing order from bottom to top
the goal is to move the disks one at a time, while preserving these conditions, until the entire stack has moved from one post to another
6.001 SICP

10. Towers of Hanoi
(define move-tower (lambda (size from to extra) (cond ((= size 0) true) (else (move-tower (- size 1) from extra to) (print-move from to) (move-tower (- size 1) extra to from)))))
(define print-move (lambda (from to) (write-line ``Move top disk from ``) (write-line from) (write-line `` to ``) (write-line to)))
6.001 SICP

11. A tree recursion
Move 4
Move 3
Move 2
Move 1
6.001 SICP

12. Orders of growth for towers of Hanoi
Let tn be the number of steps that we need to take to solve the case for n disks. Then
 tn = 2tn = 2(2tn-2 +1) + 1 ….= 2n -1
 So in time we have Q(2n) -- exponential
In space, we have one deferred operation for each increment of the stack of disks -- Q(n) -- linear
6.001 SICP

13. Quiz
What is simplest expression for the order of growth of running time of procedure mul1 & mul2?
(define (mul1 n m) (if (= n 0) 0 (+ m (mul1 (- n 1) m))))
(define (mul2 n m) (define (help count ans) (if (= count 0) ans (help (- count 1) (+ m ans)))) (help n 0))
1. Theta(1)
2. Theta(2n)
3. Theta(n)
4. Theta(n/2)
6.001 SICP

14. Now write a procedure mul4 that computes m*n in Q(log n) time
Hint: Compare with a^b = (a^2)^(b/2) if b is even,
and a^b = a*a^(b-1) if b is odd,
n*m = 2 * ( (n / 2) * m) if n even,
n*m = m + ((n-1) * m) if n odd.
6.001 SICP

15. Local Variables Suppose we want to compute
F(x,y)=x(1+xy)2 + y(1-y) + (1+xy)(1-y)
Useful to name intermediate quantities:
A = 1 + xy
B = 1 - y
F(x,y) = xA2 + yB + AB
How can we do this?
Using a “helper function?”
Using a nested lambda?
Using Let…
6.001 SICP

16. Local variables with helper fcn. / lambda
A = 1 + xy; B = 1 - y; F(x,y) = xA2 + yB + AB
Can use an auxiliary procedure to bind the intermediate variables:
(define (f x y)
(define (f-helper a b)
(+ (* x (square a))
(* y b)
(* a b)))
(f-helper (+ 1 (* x y))
(- 1 y)))
6.001 SICP

17. Local variables with helper fcn. / lambda
A = 1 + xy; B = 1 - y; F(x,y) = xA2 + yB + AB
Or use lambda directly….
(define (f x y)
((lambda (a b)
(+ (* x (square a))
(* y b)
(* a b)))
(+ 1 (* x y))
(- 1 y)))
(define (f x y)
(define (f-helper a b)
(+ (* x (square a))
(* y b)
(* a b)))
(f-helper (+ 1 (* x y) )
(- 1 y)) )
6.001 SICP

18. Local variables with let
Let is a special form which is syntactic sugar for this use of lambda to create local variables
(let (( <var1> <exp1> )
( <var2> <exp2> ) …
( <varn> <expn> ))
<body> )
Which means: let <var1> be <exp1>, <var2> be <exp2>, <var3> be <exp3> in <body> , and return the value of the last expression in <body>
6.001 SICP

19. Local variables with let
A = 1 + xy; B = 1 - y; F(x,y) = xA2 + yB + AB
(define (f x y)
((lambda (a b)
(+ (* x (square a))
(* y b)
(* a b)))
(+ 1 (* x y))
(- 1 y)))
(define (f x y)
(let ((a (+ 1 (* x y))
(b (- 1 y)) )
(+ (* x (square a))
(* y b)
(* a b))))
6.001 SICP

20. Let vs. nested define
Use of nested defines for local variable definition is considered bad form, but is technically legal:
(define (f x)
(define k 2)
(+ x k))
Let can be used inside another expression:
(define x 5)
(+ (let ((x 3))
(+ x (* x 10))) 5)
(define (f x)
(let ((k 2))
(+ x k)))
6.001 SICP

21. Let*
Let first evaluates all of the values, then binds each to the variable names.
(define x 0)
(define y 0)
(let ((x 1)
(y 2)
(z (+ x y))) z)
=== 0!
Let* evaluates and binds each name/value pair in sequence, so that each definition is available to the next
(let* ((x 1)
=== 3!
6.001 SICP

22. Let/Let*
Write a single function using let or let* to solve (x- 5)^3 + 2(x-5)^2 where x = y – 1 and y = K*z and K = 39.2, where z is given.
(let* ((k 39.2)
(y (* k z))
(x (- y 1))
(a (- x 5)))
(+ (* a a a) (* 2 a a)))
6.001 SICP

23. Remember…
calendar…
6.001 SICP