0. ::ICS 804:: Theory of Computation - Ibrahim Otieno -
SCI/ICT Building Rm. G15

1. Course Outline Mathematical Preliminaries Turing Machines
Recursion Theory
Markov Algorithms
Register Machines
Regular Languages and finite-state automata
Aspects of Computability

2. Last week: Regular Languages and FSA
Regular Expressions and regular languages
Deterministic FSA
Non-deterministic FSA
Finite-state automata with epsilon moves
Generative grammars
Context-free, Context-sensitive languages
Chomsky Hierarchy

3. Course Outline Mathematical Preliminaries Turing Machines
Additional Varieties of Turing Machines
Recursion Theory
Markov Algorithms
Register Machines
Regular Languages and finite-state automata
Aspects of Computability

4. Aspects of Computability
Church-Turing Thesis
Rice’s Theorem
Wrap-Up

5. The Church-Turing Thesis

6. Church-Turing
Effectively calculable function: cultural notion; part of mathematician’s informal sense of what mathematics is about
Turing-computable function: not cultural or informal at all
Church-Turing Thesis tries to combine these two notions (focus on number- theoretic functions)

7. Church-Turing
Turing (1936): a number-theoretic function f is effectively calculable if and only if f is Turing-computable
Not possible to formally prove it. Why not?
Try to justify it
Two parts:
f is effectively calculable so f is Turing-computable
f is Turing-computable so f is effectively calculable
Obvious: there exists some algorithm, so it is calculable

8. Church-Turing
f is effectively calculable so f is Turing- computable: not trivial
Does there exist a function that is effectively calculable but not Turing- computable?
Many (very different) models of computation, all related (cf. equivalence results)
Turing computable functions, Markov-computable functions, …
Partial Recursive Functions!

9. Feasibility (recap)
Cobham–Edmonds thesis regarding computational feasibility: the problem of determining whether a given string is a member of a given language L is feasible if and only if L is in P
Complexity class P: class of all languages accepted in polynomial time by some (single-tape) Turing Machine

10. Problems with the Thesis
Computable by what? Focus on single-tape Turing machine
Which model of computation is the favored one?
Does it make a difference?

11. Problems with the Thesis
e.g. L = {w*|na(w) = nb(w)}
Register machine: O(n) steps
Single Tape Turing Machine: O(n2) steps
Multi-Tape Turing Machine: O(n) steps
Markov Machine: O(n2) steps

12. Polynomial Relatedness
Suppose that multitape MTM accepts language L in O(n) steps. Then there exists MRM that accepts L in time O([timeTM(n)]2) under the uniform cost assumption.
Suppose that MRM accepts language L in O(n) steps under the assumption of uniform cost. Then there exists multitape MTM that accepts L in time O([timeTM(n)]3).

13. Complexity (recap: MA)
Given Turing machine M accepting L, there exists a Markov algorithm AS that accepts L in O(timeM(n)) steps
Given Markov algorithm AS accepting L, there exists a Turing machine M that accepts L in O([timeAS(n)]4) steps

14. Time and Space (recap: TM)
Corollary Suppose that language L is accepted by k-tape Turing machine M. Then L is accepted by a single-tape Turing machine M1 such that timeM1(n) is O([timeM(n)]2)
Corollary Suppose that language L is accepted by k-tape Turing machine M. Then L is accepted by a single-tape Turing machine M´ such that spaceM´(n) is O(spaceM(n)).

15. Results Regarding Polynomial Relatedness of Models
Markov
Algorithm
Model
O(timeM(n))
O([timeAS(n)]4)
Multitape
Turing Machine model
O([timeTM(n)]3)
O(timeM(n))
O([timeM(n)]2)
Register Machine
Model
Single-tape Turing Machine model
O([timeTM(n)]2)

16. Reformulation
In invoking the Cobham-Edmonds thesis, it seems to make no difference which model we take
Reformulation:
A language-acceptance problem is feasible provided that the language in question is accepted in polynomially bounded time

17. More generally
Invariance principle for models of sequential Computation Let Model be some machine model of sequential computation. Then Model will be said to be a reasonable model of sequential computation if it is polynomially related to the multitape Turing machine model

18. Church-Turing
Over 7 decades, many different models of computation, but all can be related to the Turing-machine model
All seem to encompass the same class of functions: partial recursive functions
Hypothesis:
this must be a “natural” class
Mathematical properties of its members make it the natural outcome of diverse approaches
This class must be the class of effectively calculable functions
If one of the other models could compute some function that is not Turing-computable, then the class would not seem to be so properly defined

19. Church-Turing
So: no formal proof, but vast amount of empirical evidence
Thesis:
If function f is effectively calculable, then f is Turing computable. Equivalently, if a function f is not Turing computable, then f is not effectively calculable
But limits of human cognition?

20. Rice’s Theorem

21. Rice’s Theorem Preliminaries:
Every natural number is the encoding or gödel number of a unique single-tape deterministic Turing machine
Every Turing machine with input alphabet ={1} computes a unary partial recursive function
The class of Turing machines with input alphabet ={1} can be effectively enumerated

22. Recursive Set (recap)
A set S of natural numbers is recursive if its characteristic function S is (partial) recursive with S(n) = 1 if n  S 0 if n  S

23. Rice’s Theorem
Theorem: Let  be a set of unary partial recursive functions that is nontrivial (i.e. not empty or the entire class). Then let  be the set of all gödel numbers of the members of . Then  is not a recursive set of natural numbers

24. Rice’s Theorem Important ramifications for pragmatic computer science:
e.g. teacher trying to grade C programs for some assignment
Approach 1: check input/output
Approach 2: grading program that runs on source code to check whether it effectively computes some function
Following Rice’s Theorem (and Church-Turing) there cannot be such a grading program

25. Wrap-up

26. Paradigms Three computational paradigms:
Function computation
Language recognition
Transduction paradigm
Various ways of modeling these paradigms: Turing machines, Markov algorithms, register machines

27. Turing Starting point: single-tape Turing machines
Variations: one-way-infinite tape, halting state, multi-tape
Inconsequential from theoretical point of view
Variants: extensionally equivalent

28. Markov Algorithms Labeled and unlabeled
Like Turing machines, universal model of computation
Universal:
Not: can do everything
But: any function that is computable in any sense is computed by some Markov algorithm
Church-Turing: functions that are not Markov- computable are not computable in any intuitive sense either

29. Computability-in-principle
Computation without limitations in terms of time and resources
Some functions are computable in principle, but may take ages to finish
e.g. Busy Beaver is not Turing-computable

30. Busy Beaver
The Turing Machine in TMn that writes the maximum number of 1s for a number of states
Very hard to find for anything but the smallest of n’s
For any n: large number of possible Turing Machines
Some Turing Machines will loop (so (n) = 0)
Hard to distinguish between infinitely looping machine and one that is just writing a lot of 1s

31. Busy Beaver
Calculated by brute force

32. Busy Beaver Very theoretic, toy-like argument to non- computability
But: impact on pragmatic computer science as well
Rice’s theorem: various decision problems are not solvable by any Turing machine and thus not solvable at all
Various algorithms for proving program correctness do not exist: computer scientists with no background in theory can waste a lot of time trying to develop such an algorithm

33. Theory of Computation
Origins in 1920s/1930s by mathematical logicians and philosophers before the advent of the modern digital computer!!!
Theoretic computer science has been advancing independently from pragmatic computer science
But: theoretic work has influenced pragmatic developments and vice versa
e.g. Universal Turing Machine as a precursor to modern-day Computability theory influences design of programming languages and compilers
Register machines: reflect modern machine architecture

34. Theory of Computability
A side-branch of mathematics
But: mathematicians cannot axiomatically characterize mathematical truth
Church-Turing, Cobham-Edmonds, … try to define computability and seem to have largely succeeded
Applications in pragmatic computer science have validated these insights
But: theory of computation is a science in its own right (e.g. theoretical physics, theoretical linguistics, cognitive science, …)
What is computable?
What is feasible?
Basic philosophical questions!

35. Feasible Computation More interesting from a practical point of view
Complexity theory can aid during program design
Cobham-Edmonds Thesis: feasibility is modeled by a deterministic, single-tape Turing machine whose running time is bounded above by some polynomial in the size of the input
Complexity class P (and PNP): can be computed in polynomial time
Complexity class NP: requires exponential time
NP-complete problems not in the class of P are deemed to be computationally unfeasible

36. Feasible Computation
So when working on a problem: if you find the solution to not be solvable in polynomial time, direct your effort to a subset of the problem
E.g. traveling salesman problem for Weighted undirected graphs
 refocus the problem with some heuristics, e.g. nearest-neighbor traveling salesman problem
So complexity theory has had a huge impact outside of theoretical computing

37. Feasible Computation
P-completeness: although computation is feasible, the problem is unlikely to be solvable using a parallel computing approach
in other words: a parallel algorithm will not produce any improvement in worst-case execution time over the polynomially bounded non-parallel algorithm that already exists
So knowing this: if a problem is in complexity class P and is polynomially bounded in time, don’t waste time trying to parallelize the solution

38. Part II: The Chomsky Hierarchy
Accepted by nondeterministic push-down stack automaton
G: Context-Free Grammars
Accepted by linear-bounded automaton
G: context-sensitive grammars
Accepted by deterministic FSA
G: Right-linear grammars
Context-free
languages
Recursively
enumerable languages
Context- sensitive languages
Regular
languages
Accepted by deterministic 1tape Turing Machine

39. Open questions
P = NP ?
Can deterministic linear-bounded automaton accept the same language as a nondeterministic one?
Nondeterminism, although proven to be inconsequential for Turing Machines and finite- state automata, is still not fully understood for models of computation