1. can be solved on a computer in a reasonable time, using a reasonable
Theory Of Computation
What problems cannot be solved on a computer?
What problems cannot be solved on a computer in a “reasonable” amount of time?
These aren’t just philosophical questions; their answers will determine how practical it is to pursue computer-based solutions to real-world problems like hurricane prediction, disease control, and economic forecasting.
Computability:
What are the
limitations to
the problem-
solving abilities
of computers?
Complexity:
What kinds of problems
can be solved on a computer
in a reasonable time,
using a reasonable
amount of memory? CS
Chapter 12 – Theory Of Computation

2. Chapter 12 – Theory Of Computation
Computability
To examine the limits of what it is possible to do with a computer, Alan Turing ( ) developed a simplified mathematical model, called a Turing machine.
A Turing machine consists of three parts:
A tape of cells from which symbols can be read and into which symbols can be written,
Control Unit
A read/write head that moves back and forth across the tape, reading the symbol inside the current cell and/or writing a new symbol into the current cell, and
Read/Write Head
Tape
A control unit that keeps track of what “state” the machine is in, and uses that state and the current symbol under the read/write head to:
Determine which symbol to place in the current cell,
Determine whether to move the read/write head one cell to the left or right, and
Determine the next state of the machine. CS
Chapter 12 – Theory Of Computation

3. State Transition Diagram* 1
State:
START
A state transition diagram may be used to define a Turing machine. * 1
State:
ADD
Each // transition signifies reading  on the tape, replacing it with , and then moving the read/write head in the  direction. * 1
State:
CARRY * 1
State:
NO CARRY
START
ADD
CARRY
NO CARRY
OVERFLOW
HALT
RETURN
*/*/L
1/0/L
0/1/L
0/0/L,
1/1/L
*/1/L
*/*/R
*/*/-
0/0/R,
1/1/R * 1
State:
NO CARRY * 1
State:
RETURN * 1
State:
RETURN * 1
State:
RETURN
The state transition diagram above defines a Turing machine that increments a binary number on the tape by one. * 1
State:
RETURN * 1
State:
HALT CS
Chapter 12 – Theory Of Computation

4. The Halting Problem The Church-Turing Thesis
Computer scientists commonly accept the Church-Turing Thesis, which states that the set of functions that can be calculated on a computer is exactly the same as the set of functions for which a Turing machine can be devised.
There are problems that have been proven to be non-computable (i.e., no Turing machine can be devised to calculate their solutions).
One classical example:
The Halting Problem
Given a program with a set of input values, does the program halt on that input, or does it get stuck in an infinite loop? CS
Chapter 12 – Theory Of Computation

5. Complexity
The time complexity of an algorithm is a measure of how many steps are executed when the associated program is run.
void printA() {
cout << 0 << endl; }
Number of Output Statements Executed: 4
Time Complexity: O(1)
The “big-O” notation provides information regarding the program’s “order of complexity”.
O(1) indicates that the execution time doesn’t relate to the size of the number n.
O(n) indicates that the execution time increases linearly as n increases.
O(n2) indicates that the execution time increases quadratically as n increases.
void printB() {
int i;
for (i = 1; i <= 100; i++)
cout << 0 << endl; }
Number of Output Statements Executed: 100
Time Complexity: O(1)
void printC(int n) {
int i;
for (i = 1; i <= n; i++)
cout << 0 << endl; }
Number of Output Statements Executed: n
Time Complexity: O(n)
void printD(int n) {
int i,j;
for (i = 1; i <= n; i++)
for (j = 1; j <= n; j++)
cout << 0 << endl; }
Number of Output Statements Executed: n2
Time Complexity: O(n2) CS
Chapter 12 – Theory Of Computation

6. Logarithmic vs. Polynomial vs. Exponential
An algorithm is said to have logarithmic time complexity if the number of steps in its execution is bounded by some logarithmic function:
k log2(n)
Essentially, this means that doubling the size of the problem (n) would only increase the execution time by a constant amount (k).
An algorithm is said to have polynomial time complexity if the number of steps in its execution is bounded by some polynomial function:
aknk+ak-1nk-1+…+a2n2+a1n+a0
An algorithm is said to have exponential time complexity if the number of steps in its execution is bounded by some exponential function:
k(2n)
Essentially, this means that increasing the size of the problem (n) by one would double the execution time.
log2(n) n n2 n3 2n 2 5 25
125 31 3 10
100
1000
1024 4 20
400
8000 CS
Chapter 12 – Theory Of Computation

7. Chapter 12 – Theory Of Computation
Big-O
An algorithm’s time complexity is dominated by its most significant term.
For example, an algorithm that executes in time n2+10n is considered to be O(n2) because, as n increases, the n2 term ultimately dominates the n term.
Additional examples:
5n+n n3 is O(n3)
log2(n)+n2+2n is O(2n)
100n+max(n2,1000-n3) is O(n2)
n3 -⅞(n3-80n2-800n) is O(n3)
log2(n) is O(log2(n)) CS
Chapter 12 – Theory Of Computation

8. P and NP Problems
A problem is said to be a P problem if it can be solved with a deterministic, polynomial-time algorithm.
(Deterministic algorithms have each step clearly specified.)
A problem is said to be an NP problem if it can be solved with a nondeterministic, polynomial-time algorithm.
In essence, at a critical point in the NP problem’s algorithm, a decision must be made, and it is assumed that some magical “choice” function (also called an oracle) always chooses correctly.
For example, take the Satisfiability Problem:
Given a set of n boolean variables b1, b2, … bn, and a boolean function f (b1, b2, …, bn), are there any values that can be assigned to the variables so the function value will evaluate to TRUE?
To try every combination of boolean values would take exponential time, but the nondeterministic solution at right has polynomial time complexity.
for (i = 1; i <= n; i++)
bi = choice(true, false);
if (f(b1, b2,…, bn) == true)
satisfiable = true;
else satisfiable = false; CS
Chapter 12 – Theory Of Computation

9. The Knapsack Problem A Nondeterministic Polynomial Solution:
The Knapsack Problem involves taking n valuable jewels J1, J2,…,Jn, with respective weights w1, w2,…, wn, and prices p1, p2,…, pn, and placing some of them in a knapsack that is capable of supporting a combined weight of M.
The problem is to pack the maximum worth of gems without exceeding the capacity of the knapsack.
(It’s not as easy as it sounds; three lightweight $1000 gems might be preferable to one heavy $2500 gem, and one 20-pound gem worth a lot of money might be preferable to twelve 1-pound gems that are practically worthless.)
A Nondeterministic Polynomial Solution:
TotalWorth = 0;
TotalWeight = 0;
for (i = 1; i <= n; i++) {
bi = choice(true, false);
if (b1 == true)
TotalWorth+= pi;
TotalWeight += wi; }
if (TotalWeight <= M)
cout << “Woo-hoo!” << endl;
else
cout << “Doh!” << endl; CS
Chapter 12 – Theory Of Computation

10. Chapter 12 – Theory Of Computation
Cryptography
Networks are set up to send messages right past stations that aren’t authorized to read them, but what’s to prevent such unauthorized viewing?
message
from to
message
from to
message
from to
message
from to
The most common solution to this problem is encryption, where the message is coded in such a way that only the receiving station can decode it. CS
Chapter 12 – Theory Of Computation

11. Public-Key Encryption1.
Create Message
I have affixed to me the dirt and dust of countless ages!
Chuck mdbriugndlwg
Linus mamnsgfyddkd
Lucy qhgwdnchsgsh
Patty ahwbsgcydhzx 2.
Look Up Recipient’s Public Key 3.
Encrypt Message With Recipient’s Public Key
xsjb2dhdkWb$xzduYdm!dj5slLssghd8nd&hsnqabi?dsjsg% 4.
Transmit Encrypted Message
xsjb2dhdkWb$xzduYdm!dj5slLssghd8nd&hsnqabi?dsjsg% 5.
Decrypt Message With Recipient’s Private Key
I have affixed to me the dirt and dust of countless ages! CS
Chapter 12 – Theory Of Computation

12. Key-Based Authentication2.
Encrypt Message With Sender’s Private Key
Ma3ndhvyr#bcjaqwpfQkguiorkfohskxi8vce%fpgkjfhikfvdamxxyemfideychssfhsgdhahdm$dlglyn7buchso 1.
Create Message
I’m going to recruit that funny-looking kid who plays shortstop on Chuck’s team! 3.
Transmit Encrypted Message
Ma3ndhvyr#bcjaqwpfQkguiorkfohskxi8vce%fpgkjfhikfvdamxxyemfideychssfhsgdhahdm$dlglyn7buchso 4.
Decrypt Message With Sender’s Public Key
I’m going to recruit that funny-looking kid who plays shortstop on Chuck’s team! CS
Chapter 12 – Theory Of Computation