1. Chapter 1 Introduction to the Theory of Computation

2. Theory of Computation Can we mathematically model the computation?
Automata Theory
Regular languages: DFA, NFA, Regular expressions
Context-free languages: Context-free grammar, pushdown automata
Turing machines: the most powerful model of computation
Can the computer solve all problems?
Computability Theory
Halting problem
How efficient the solution is?
Complexity Theory
Time and space complexity (Big O notation)

3. 1.1 Mathematical Preliminaries
Sets (1)
A set is a collection of objects
Order and repetition is irrelevant
X = {1, 2, 3}
X = {n | 1  n  3, n  N }
Y = { } =   Y is an empty set
The power set of X is the set of all subsets of X
(X) = 2X = { Y | Y  X}
|(X)| = 2|X|
Example X = {1, 2, 3}
2X = (X)={{},{1},{2},{3},{1,2},{1,3},{2,3}, {1,2,3}}
{1,2}  X is a proper subset of X
{1,2,3}  X is NOT a proper subset of X
{1,2,3} is a superset of {1, 2}

4. Sets (2) The  symbol signifies membership (belong to):
x  X indicates that x is a member or element of the set X
x  X indicates that x is not a member or element of the set X
Assume X = {1, 2, 3}, then
1  X, but 4  X
Two sets are equal if they contain the same members (elements)
If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4}
Then X = Y, but X  Z, and so Y  Z

5. Sets (3) Assume X = {1,2,3} and Y = {2,4} Union
X  Y = {z | z  X or z Y} = {1,2,3,4}
Intersection
X  Y = {z | z  X and z Y} = {2}
Difference
X - Y = {z | z  X and z  Y} = {1,3}

6. Sets (4) Complement Let X be a subset of Y
The complement of X ( X ) with respect to Y is the set of elements in Y but not in X
X = { z | z  Y and z  X } = Y - X
Assume X = {1,2,3} is a subset of Y = {1,2,3,4}
X = {4}
DeMorgan’s Laws
(X  Y) = X  Y
(X  Y) = X  Y

7. Inductive Proofs
Prove a statement S(X) about a family of objects X (e.g., integers, trees) in three parts:
Basis Step:
Prove for one or several small values of X directly
Inductive step:
Assume S(Y) for Y smaller than X
Prove S(X) using that assumption

8. Inductive Proofs (EX1) Prove by induction that
S: ……+ n = n (n + 1) / 2
Basis step:
n = 1
1 ( 1 + 1) / 2 = 1
Induction step (assumption step):
Assume that S is true for all series of length < n
Prove that S is true for a series of length n?
The series of length n is
1+2+3+…+n-1+n
By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2
Add n to the summation proves S.

9. Inductive Proofs (EX2) Example
A complete binary tree with n leaves has 2n - 1 nodes
Proof:
Formally, S(T): if T is a binary tree with n leaves, then T has 2n – 1 nodes
Induction is on the size = number of nodes of T

10. Inductive Proofs (EX2) Basis:
If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so OK
Induction:
Assume S(U) for trees with fewer nodes than T. In particular, assume for the subtrees of T
T must be a root plus two subtrees U and V.
If U and V have u and v leaves, respectively, and T has t leaves, then u + v = t.
Proof:
By the inductive hypothesis, U and V have 2u - 1 and 2v - 1 nodes, respectively.
Then T has 1 + (2u -1) + (2v - 1) = 2(u + v) - 1 = 2t -1
Proving the inductive step

11. If-And-Only-If Proofs
Often, a statement we need to prove is of the form
“X if and only if Y”
We are then required to do two things:
Prove the if-part: Assume Y and prove X
Prove the only-if-part: Assume X, prove Y
Remember:
The if and only-if parts are converses of each other
One part, say “if X then Y” says nothing about whether Y is true when X is false.
An equivalent form to “if X then Y” is “if not Y then not X”; the latter is the contra-positive of the former.

12. Equivalence of Sets
Many important facts in language theory are of the form that two sets of strings, described in two different ways, are really the same set
To prove sets S and T are the same, prove:
x is in S if and only if x is in T. That is:
Assume x is in S; prove x is in T .
Assume x is in T ; prove x is in S.

13. Functions
Let f be a function defined on a set A and taking values in a set B. Set A is called the domain of f; set B is called the range of f.
Function f is said to be one-to-one if, whenever f(x) = f(y), it must be the case that x = y.
In other words, f is one-to-one if it maps distinct objects to distinct objects.
Function f is said to be onto if, for any b  B, there exists an a  A for which f(a) = f(b).
A function is bijection if it is one-to-one and onto.
If function f is defined for each element of A, then f is said to be a total function; otherwise, f is said to be a partial function.

14. Relations
Given two sets, X and Y, a relation R is a set of ordered pairs (x, y) where x  X and y Y
Equivalence relations
Reflexive
R is reflexive if for every a  A, then (a, a)  R
Symmetric
R is symmetric if for every a and b in A, if (a, b)  R, then also (b, a)  R
Transitive
R is transitive if for any a, b, c, if (a, b) and (b, c)  R, then also (a, c)  R
R is an equivalence relation if R is reflexive, symmetric and transitive

15. 1.2: Strings and Languages (1)
Alphabet: Finite, nonempty set of symbols
Examples:
 = {0, 1}: binary alphabet
 = {a, b, c, …, z}: the set of all lower case letters
The set of all ASCII characters
String: Finite sequence of symbols from an alphabet 
01101 where  = {0, 1}
abracadabra where  = {a, b, c, …, z}

16. 1.2: Strings and Languages (2)
Empty String: The string with zero occurrences of symbols from  and is denoted e or 
Length of String: Number of symbols in the string
The length of a string w is usually written |w|
|1010| = 4
|e| = 0
|uv| = |u| + |v|
Reverse : wR
If w = abc, wR = cba

17. 1.2: Strings and Languages (3)
Concatenation: if x and y are strings, then xy is the string obtained by placing a copy of y immediately after a copy of x
x = a1a2 …ai, y = b1b2 …bj
xy = a1a2 …aib1b2 …bj
Example: x = 01101, y = 110, xy =
xe = ex = x

18. 1.2: Strings and Languages (4)
Power of an Alphabet: k = the set of strings of length k with symbols from S
Example:  = {0, 1}
S1 = S = {0, 1}
S2 = {00, 01, 10, 11}
S0 = {e}
Question: How many strings are there in S3?
The set of all strings over  is denoted *
S* = S0  S1  S2  S3  …
Also
S+ = S1  S2  S3  …
S* = S+  {e}
S+ = S* - {e}

19. 1.2: Strings and Languages (5)
Substring: any string of consecutive characters in some string w
If w = abc
e, a, ab, abc are substrings of w
Prefix and suffix:
if w = vu
v is a prefix of w
u is a suffix of w
Example
a, ab , abc are prefixes of w
c, bc, abc are suffixes of w

20. 1.2: Strings and Languages (6)
Suppose: S is the string banana
Prefix : ban, banana
Suffix : ana, banana
Substring : nan, ban, ana, banana
Proper prefix, suffix, or substring cannot be all of S

21. 1.2: Strings and Languages (8)
Language: set of strings chosen from some alphabet
A language is a subset of *
Example of languages:
The set of valid Arabic words
The set of strings consisting of n 0’s followed by n 1’s
{e, 01, 0011, , …}
The set of strings with equal number of 0’s and 1’s
{e, 01, 10, 0011, 0101, 1010, 1001, 1100, …}
Empty language:  = { }
The language {e} consisting of the empty string
Note:   {e}

22. 1.2: Strings and Languages (9)
Can concatenate languages
L1L2 = {xy | x  L1, y  L2}
Ln = L concatenated with itself n times
L0 = {e}; L1 = L
Star-closure
L* = L0  L1  L2  ...
L+ = L* - L0
Languages can be finite or infinite
L = {a, aba, bba}
L = {an | n > 0}

23. 1.2: Strings and Languages (10)
OPERATION
DEFINITION
union of L and M written L  M
L  M = {s | s is in L or s is in M}
concatenation of L and M written LM
LM = {st | s is in L and t is in M}
Kleene closure of L written L*
L* =  Li , i=0,.., 
L* denotes “zero or more concatenations of “ L
positive closure of L written L+
L+ =  Li, i=1,.., 
L+ denotes “one or more concatenations of “ L
L+ = LL*

24. 1.2: Strings and Languages (11)
L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9}
L  D = the set of letters and digits
LD = all strings consisting of a letter followed by a digit
L2 = the set of all two-letter strings
L4 = L2 L2 = the set of all four-letter strings
L* = { All possible strings of L plus  }, L+ = L* - 
D+ = set of strings of one or more digits
L (L  D ) = set of all strings consisting of a letter followed by a a letter or a digit
L (L  D )* = set of all strings consisting of letters and digits beginning with a letter

25. 1.2: Strings and Languages (12)
The language L consists of strings over {a,b} in which each string begins with an a should have an even length
aa, ab  L
aaaa,aaab,aaba,aabb,abaa,abab,abba,abbb  L
baa  L
A  L

26. 1.2: Strings and Languages (13)
The language L consists of strings over {a,b} in which each occurring of b is immediately preceded by an a
e  L
a  L
abaab  L
bb  L
bab  L
abb  L

27. 1.2: Strings and Languages (14)
Let X = {a,b,c} and Y = {abb, ba}. Then
XY = {aabb, babb, cabb, aba, bba, cba}
X0 = {e}
X1 = X = {a,b,c}
X2 = XX = {aa,ab,ac,ba,bb,bc,ca,cb,cc}
X3 = XXX = {aaa,aab,aac,aba,abb,abc,aca,acb,acc,baa,bab,bac,bba,bbb,bbc,bca,bcb,bcc,caa,cab,cac,cba,cbb,cbc,cca,ccb,ccc}

28. 1.2: Strings and Languages (15)
The language L = {a,b}*{bb}{a,b}* =(a|b)*bb(a|b)* =*bb*
consists of the strings over {a,b} that contain the substring bb
bb  L
abb  L
bbb  L
aabb  L
bbaaa  L
bbabba  L
abab  L
bab  L
b  L

29. 1.2: Strings and Languages (16)
Let L be the language that consists of all strings that begin with aa or end with bb
L1 = {aa}{a,b}*
L2 = {a,b}*{bb}
L = L1  L2 = {aa}{a,b}*  {a,b}*{bb}=aa* | *bb
bb  L
abb  L
bbb  L
aabb  L
bbaaa  L
bbabba  L
abab  L
bab  L
ba  L

30. 1.2: Strings and Languages (16)
Let L1 = {bb} and L2 = {e, bb, bbbb} over b
The languages L1* and L2* both contain precisely the strings consisting of an even number of b’s.
e , with length zero, is an element of L1* and L2*

31. 1.2: Strings and Languages (17)
What is the language of all even-length strings over {a,b}
L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)*
What is the language of all odd-length strings over {a,b}
L = {a,b}* - {aa, bb, ab, ba}* or
L = {a,b}{aa, bb, ab, ba}* or
L = {aa, bb, ab, ba}* {a,b} or
L = (a|b)(aa|bb|ab|ba)*