Limitations

Limitations of Computation Hardware precision Hardware precision Software - we’re human Software - we’re human Problems Problems –complex problem formulation (weather?) –complexity –inherent uncomputability

Hardware Read text section 17.1 arithmetic precision arithmetic precision –but use extra words –be attentive to numerical roundoff effects physical components physical components –build in redundancy, test, maintain communications communications –check-digits, parity bits, error-correcting codes are we reaching the limit of Moore’s law? are we reaching the limit of Moore’s law?

Software Read text section 17.2 Software complexity Software complexity –software engineering –formal i/o specification –code reviews, inspection, testing –formal verification  often too complex, but has been useful

Problems Complex problems Complex problems –many factors and variables –difficulty modeling –no “closed form” temp tomorrow = temp. today * pressure… –solved via large-scale simulation with lots of math dictating behavior of “cells” based on adjacent cells. turn crank, see what happens. –as moments increase, accuracy decreases

Problems Computational Complexity Running time of an algorithm is a function of input size Running time of an algorithm is a function of input size Some functions grow faster than others Some functions grow faster than others WAY faster WAY faster For some problems, we have efficient algorithms (functions that grow moderately). For some problems, we have efficient algorithms (functions that grow moderately). For others, the best known algorithms have running times that grow so fast that they are impractical, and always will be. For others, the best known algorithms have running times that grow so fast that they are impractical, and always will be. Some problems provably cannot be solved with a faster algorithm Some problems provably cannot be solved with a faster algorithm

Spreadsheet of running times and execution times at 1 billion steps/sec and execution times at 1 billion steps/secexercise: - create columns for n, n 2, n 3, 2 n, 3 n, n! - compute running times for n=10,20,…

UNCOMPUTABLE problems previous category were problems that could be solved, but known methods took too long previous category were problems that could be solved, but known methods took too long this category are problems for which we know: this category are problems for which we know: THEY CANNOT BE SOLVED BY ANY ALGORITHM WHATSOEVER

History Hilbert’s “program” Hilbert’s “program” Gödel’s undecidability theorem Gödel’s undecidability theorem Turing’s halting problem Turing’s halting problem Natural problems and consequences Natural problems and consequences

David Hilbert famous mathematician famous mathematician 23 problems 23 problems founder of founder of “metamathematics” “metamathematics” goal to “mechanize” goal to “mechanize”mathematics

Kurt Gödel Logician Logician Proved in 1931 …(age 25) Proved in 1931 …(age 25) “there are true propositions that cannot be proved” (i.e., “no” to Hilbert’s program) Shook the foundations of Shook the foundations of –mathematics –philosophy –science –everything

Alan Turing British Mathematician British Mathematician Seminal work in Computation, AI, Cryptanalysis Seminal work in Computation, AI, Cryptanalysis Subject of several books, movies Subject of several books, movies

A Halting Problem x = input (“please enter a positive number) WHILE x > 1: IF x is even: x = x/2 ELSE: x = 3x + 1

The Halting Problem Given: Program, Input to program Given: Program, Input to program Does the program on the input ever halt? Does the program on the input ever halt? Critical question that pervades all of computation Critical question that pervades all of computation Has much more broad consequences than it at first appears Has much more broad consequences than it at first appears

Hilbert’s 10th problem Finding integer solutions to a polynomial with integer coefficients: Finding integer solutions to a polynomial with integer coefficients: 15x 4 y 2 - 3xy 3 z 2 + 2x 2 z = 1 No algorithm exists for finding such solutions, or determining whether there are any. No algorithm exists for finding such solutions, or determining whether there are any.  in some cases you can find solutions, or tell that there is no solution. By “No algorithm exists” we mean there isn’t a general method that works for all possible polynomials.

Mathematical theorems… Given a possible theorem T, we’d like to know if T is true. Given a possible theorem T, we’d like to know if T is true. We need to find a proof of T. We need to find a proof of T. Can write a computer program P to enumerate all possible proofs for T, and check to see if any is a correct one. Can write a computer program P to enumerate all possible proofs for T, and check to see if any is a correct one. If P ever halts, it has found a correct proof. If P ever halts, it has found a correct proof. Thus, halting problem for P is question of whether or not T is provably true. Thus, halting problem for P is question of whether or not T is provably true.

Towards proving that the Halting Problem is unsolvable…

Warm-ups: Self-reference leads to paradox In a certain town, there is a barber who shaves all and only those who don’t shave themselves. In a certain town, there is a barber who shaves all and only those who don’t shave themselves. Homogenous words: self-describing Homogenous words: self-describing –“english”, “short”, “polysyllabic”, … Heterogenous words: non-self-describing –“spanish”, “long”, “monosyllabic”, …

Warm-ups: Self-reference leads to paradox I can force you to lie, if you answer my question with only “yes” or “no”: I can force you to lie, if you answer my question with only “yes” or “no”: Q: Will the next word you say be “no”? Q: Will the next word you say be “no”?

“Proof” of Halting problem Suppose there was an algorithm that could tell whether a program halted on given input……(and worked for all inputs) Suppose there was an algorithm that could tell whether a program halted on given input……(and worked for all inputs) We will show how this leads to a logical contradiction. When an assumption leads to a contradiction, the assumption must be false. We will show how this leads to a logical contradiction. When an assumption leads to a contradiction, the assumption must be false.

“Proof” of Halting problem Suppose there was an algorithm that could tell whether a program halted on given input…… program P data D “yes, P(D) halts” “no, P(D) doesn’t halt” halt-checker

Program Q program P data P “yes, P(P) halts” “no, P(P) doesn’t halt” program P purposely get into infinite loop!! halt and print “Q halts” halt-checker

Program Q when given itself program Q data Q yes, Q(Q) halts no, Q(Q) doesn’t halt program Q purposely get into infinite loop!! halt and print “Q halts” So, Program Q when run with program Q as input halts if and only if Program Q when run with program Q as input doesn’t halt! halt-checker This is a contradiction. Thus, no “halt-checker” can exist!!

Summary Impending hardware limitations ?? Impending hardware limitations ?? Software limitations due to how large and complex programs can become. Software limitations due to how large and complex programs can become. Running-time limitations because sometimes best algorithm runs too slow. Running-time limitations because sometimes best algorithm runs too slow. Some problems too complex to be modeled. Some problems too complex to be modeled. Some problems just have no algorithm. Some problems just have no algorithm.