Chapter 1 Introduction to the Theory of Computation

Theory of Computation Can we mathematically model the computation? Automata Theory Regular languages: DFA, NFA, Regular expressions Context-free languages: Context-free grammar, pushdown automata Turing machines: the most powerful model of computation Can the computer solve all problems? Computability Theory Halting problem How efficient the solution is? Complexity Theory Time and space complexity (Big O notation)

1.1 Mathematical Preliminaries Sets (1) A set is a collection of objects Order and repetition is irrelevant X = {1, 2, 3} X = {n | 1  n  3, n  N } Y = { } =   Y is an empty set The power set of X is the set of all subsets of X (X) = 2X = { Y | Y  X} |(X)| = 2|X| Example X = {1, 2, 3} 2X = (X)={{},{1},{2},{3},{1,2},{1,3},{2,3}, {1,2,3}} {1,2}  X is a proper subset of X {1,2,3}  X is NOT a proper subset of X {1,2,3} is a superset of {1, 2}

Sets (2) The  symbol signifies membership (belong to): x  X indicates that x is a member or element of the set X x  X indicates that x is not a member or element of the set X Assume X = {1, 2, 3}, then 1  X, but 4  X Two sets are equal if they contain the same members (elements) If X = {1, 2, 3}, Y = {1, 2, 3}, and Z = {2, 3, 4} Then X = Y, but X  Z, and so Y  Z

Sets (3) Assume X = {1,2,3} and Y = {2,4} Union X  Y = {z | z  X or z Y} = {1,2,3,4} Intersection X  Y = {z | z  X and z Y} = {2} Difference X - Y = {z | z  X and z  Y} = {1,3}

Sets (4) Complement Let X be a subset of Y The complement of X ( X ) with respect to Y is the set of elements in Y but not in X X = { z | z  Y and z  X } = Y - X Assume X = {1,2,3} is a subset of Y = {1,2,3,4} X = {4} DeMorgan’s Laws (X  Y) = X  Y (X  Y) = X  Y

Inductive Proofs Prove a statement S(X) about a family of objects X (e.g., integers, trees) in three parts: Basis Step: Prove for one or several small values of X directly Inductive step: Assume S(Y) for Y smaller than X Prove S(X) using that assumption

Inductive Proofs (EX1) Prove by induction that S: 1 + 2 + 3 + ……+ n = n (n + 1) / 2 Basis step: n = 1 1 ( 1 + 1) / 2 = 1 Induction step (assumption step): Assume that S is true for all series of length < n Prove that S is true for a series of length n? The series of length n is 1+2+3+…+n-1+n By the assumption 1+2+…+n-1 = (n-1)((n-1)+1)/2 Add n to the summation proves S.

Inductive Proofs (EX2) Example A complete binary tree with n leaves has 2n - 1 nodes Proof: Formally, S(T): if T is a binary tree with n leaves, then T has 2n – 1 nodes Induction is on the size = number of nodes of T

Inductive Proofs (EX2) Basis: If T has 1 leaf, it is a one-node tree. 1 = 2*1 - 1, so OK Induction: Assume S(U) for trees with fewer nodes than T. In particular, assume for the subtrees of T T must be a root plus two subtrees U and V. If U and V have u and v leaves, respectively, and T has t leaves, then u + v = t. Proof: By the inductive hypothesis, U and V have 2u - 1 and 2v - 1 nodes, respectively. Then T has 1 + (2u -1) + (2v - 1) = 2(u + v) - 1 = 2t -1 Proving the inductive step

If-And-Only-If Proofs Often, a statement we need to prove is of the form “X if and only if Y” We are then required to do two things: Prove the if-part: Assume Y and prove X Prove the only-if-part: Assume X, prove Y Remember: The if and only-if parts are converses of each other One part, say “if X then Y” says nothing about whether Y is true when X is false. An equivalent form to “if X then Y” is “if not Y then not X”; the latter is the contra-positive of the former.

Equivalence of Sets Many important facts in language theory are of the form that two sets of strings, described in two different ways, are really the same set To prove sets S and T are the same, prove: x is in S if and only if x is in T. That is: Assume x is in S; prove x is in T . Assume x is in T ; prove x is in S.

Functions Let f be a function defined on a set A and taking values in a set B. Set A is called the domain of f; set B is called the range of f. Function f is said to be one-to-one if, whenever f(x) = f(y), it must be the case that x = y. In other words, f is one-to-one if it maps distinct objects to distinct objects. Function f is said to be onto if, for any b  B, there exists an a  A for which f(a) = f(b). A function is bijection if it is one-to-one and onto. If function f is defined for each element of A, then f is said to be a total function; otherwise, f is said to be a partial function.

Relations Given two sets, X and Y, a relation R is a set of ordered pairs (x, y) where x  X and y Y Equivalence relations Reflexive R is reflexive if for every a  A, then (a, a)  R Symmetric R is symmetric if for every a and b in A, if (a, b)  R, then also (b, a)  R Transitive R is transitive if for any a, b, c, if (a, b) and (b, c)  R, then also (a, c)  R R is an equivalence relation if R is reflexive, symmetric and transitive

1.2: Strings and Languages (1) Alphabet: Finite, nonempty set of symbols Examples:  = {0, 1}: binary alphabet  = {a, b, c, …, z}: the set of all lower case letters The set of all ASCII characters String: Finite sequence of symbols from an alphabet  01101 where  = {0, 1} abracadabra where  = {a, b, c, …, z}

1.2: Strings and Languages (2) Empty String: The string with zero occurrences of symbols from  and is denoted e or  Length of String: Number of symbols in the string The length of a string w is usually written |w| |1010| = 4 |e| = 0 |uv| = |u| + |v| Reverse : wR If w = abc, wR = cba

1.2: Strings and Languages (3) Concatenation: if x and y are strings, then xy is the string obtained by placing a copy of y immediately after a copy of x x = a1a2 …ai, y = b1b2 …bj xy = a1a2 …aib1b2 …bj Example: x = 01101, y = 110, xy = 01101110 xe = ex = x

1.2: Strings and Languages (4) Power of an Alphabet: k = the set of strings of length k with symbols from S Example:  = {0, 1} S1 = S = {0, 1} S2 = {00, 01, 10, 11} S0 = {e} Question: How many strings are there in S3? The set of all strings over  is denoted * S* = S0  S1  S2  S3  … Also S+ = S1  S2  S3  … S* = S+  {e} S+ = S* - {e}

1.2: Strings and Languages (5) Substring: any string of consecutive characters in some string w If w = abc e, a, ab, abc are substrings of w Prefix and suffix: if w = vu v is a prefix of w u is a suffix of w Example a, ab , abc are prefixes of w c, bc, abc are suffixes of w

1.2: Strings and Languages (6) Suppose: S is the string banana Prefix : ban, banana Suffix : ana, banana Substring : nan, ban, ana, banana Proper prefix, suffix, or substring cannot be all of S

1.2: Strings and Languages (8) Language: set of strings chosen from some alphabet A language is a subset of * Example of languages: The set of valid Arabic words The set of strings consisting of n 0’s followed by n 1’s {e, 01, 0011, 000111, …} The set of strings with equal number of 0’s and 1’s {e, 01, 10, 0011, 0101, 1010, 1001, 1100, …} Empty language:  = { } The language {e} consisting of the empty string Note:   {e}

1.2: Strings and Languages (9) Can concatenate languages L1L2 = {xy | x  L1, y  L2} Ln = L concatenated with itself n times L0 = {e}; L1 = L Star-closure L* = L0  L1  L2  ... L+ = L* - L0 Languages can be finite or infinite L = {a, aba, bba} L = {an | n > 0}

1.2: Strings and Languages (10) OPERATION DEFINITION union of L and M written L  M L  M = {s | s is in L or s is in M} concatenation of L and M written LM LM = {st | s is in L and t is in M} Kleene closure of L written L* L* =  Li , i=0,..,  L* denotes “zero or more concatenations of “ L positive closure of L written L+ L+ =  Li, i=1,..,  L+ denotes “one or more concatenations of “ L L+ = LL*

1.2: Strings and Languages (11) L = {A, B, …, Z, a, b, …z} D = {1, 2, …, 9} L  D = the set of letters and digits LD = all strings consisting of a letter followed by a digit L2 = the set of all two-letter strings L4 = L2 L2 = the set of all four-letter strings L* = { All possible strings of L plus  }, L+ = L* -  D+ = set of strings of one or more digits L (L  D ) = set of all strings consisting of a letter followed by a a letter or a digit L (L  D )* = set of all strings consisting of letters and digits beginning with a letter

1.2: Strings and Languages (12) The language L consists of strings over {a,b} in which each string begins with an a should have an even length aa, ab  L aaaa,aaab,aaba,aabb,abaa,abab,abba,abbb  L baa  L A  L

1.2: Strings and Languages (13) The language L consists of strings over {a,b} in which each occurring of b is immediately preceded by an a e  L a  L abaab  L bb  L bab  L abb  L

1.2: Strings and Languages (14) Let X = {a,b,c} and Y = {abb, ba}. Then XY = {aabb, babb, cabb, aba, bba, cba} X0 = {e} X1 = X = {a,b,c} X2 = XX = {aa,ab,ac,ba,bb,bc,ca,cb,cc} X3 = XXX = {aaa,aab,aac,aba,abb,abc,aca,acb,acc,baa,bab,bac,bba,bbb,bbc,bca,bcb,bcc,caa,cab,cac,cba,cbb,cbc,cca,ccb,ccc}

1.2: Strings and Languages (15) The language L = {a,b}*{bb}{a,b}* =(a|b)*bb(a|b)* =*bb* consists of the strings over {a,b} that contain the substring bb bb  L abb  L bbb  L aabb  L bbaaa  L bbabba  L abab  L bab  L b  L

1.2: Strings and Languages (16) Let L be the language that consists of all strings that begin with aa or end with bb L1 = {aa}{a,b}* L2 = {a,b}*{bb} L = L1  L2 = {aa}{a,b}*  {a,b}*{bb}=aa* | *bb bb  L abb  L bbb  L aabb  L bbaaa  L bbabba  L abab  L bab  L ba  L

1.2: Strings and Languages (16) Let L1 = {bb} and L2 = {e, bb, bbbb} over b The languages L1* and L2* both contain precisely the strings consisting of an even number of b’s. e , with length zero, is an element of L1* and L2*

1.2: Strings and Languages (17) What is the language of all even-length strings over {a,b} L = {aa, bb, ab, ba}* = (aa|bb|ab|ba)* What is the language of all odd-length strings over {a,b} L = {a,b}* - {aa, bb, ab, ba}* or L = {a,b}{aa, bb, ab, ba}* or L = {aa, bb, ab, ba}* {a,b} or L = (a|b)(aa|bb|ab|ba)*