1. Lecture 24 MAS 714 Hartmut Klauck
Theory of Computing
Lecture 24
MAS 714
Hartmut Klauck

2. Size of Automata?
We know L is regular, if there is a DFA with a finite number of states for L
Minimum number of states is the Myhill-Nerode Index
Questions:
How can we find a minimal DFA for L?
Is it unique (up to renaming states)?
Are NFA smaller than DFA?
Are two-way DFA smaller than DFA?

3. Question 4
Consider the language L={xi : x2{0,1}n and i2{0,1} log n and xi=1}
n is fix, this is a finite language
It is easy to see that there are 2n rows in the communication matrix
Rows labeled x contain the string on columns i=1…n
Hence any DFA for L has 2n states.
Can give a 2-way DFA with O(n2) states:
go to the right end of the input, read i into the state, move n-i steps left, accept if there is a 1

4. Question 3 Consider L={xy: x,y 2 {0,1}n, x  y}
This is a FINITE language, n is fixed
Exercise: there is an NFA with O(n2) states for L
DFA-size: the matrix has more than 2n distinct rows
DFA size is exponential
Hence NFA can be exponentially smaller than DFA for some languages
Example where they are not: complement of L

5. 2) Uniqueness
Uniqueness of the minimal DFA (up to vertex names) follows from the Myhill-Nerode characterization
The optimal DFA has exactly one state for each set of equal rows of the comm. matrix
Edges are also determined uniquely

6. 1) Optimal DFA
Theorem: For every regular L there is a unique DFA A that has minimum size. Given a DFA M for L we can find A in polynomial time.
Corollary: Given DFA A,B we can decide in polynomial time whether they compute the same language
Minimize DFA, compare graphs (note that this is not an instance of the (hard) graph isomorphism problem, since we know the starting state and edge labels)
Corollary: Given DFA A, we can decide in polynomial time whether LA is empty

7. Note The corresponding questions for Turing machines are undecidable
These properties (easy to find unique representation, easy to find small representation, easy to check for emptiness and easy to compare) make DFA a good data-structure for languages/Boolean functions

8. DFA Minimization
All algorithms try to recover equivalence classes in the Myhill-Nerode characterization
I.e., try to find the smallest partition of the rows of the communication matrix into equal rows
Input: DFA with n states and alphabet size k
Hopcroft: O(nk log n)
Moore: O(n2k)
Idea in both:
Start from partition of states into accepting/rejecting
Refine partition until no longer possible

9. Proof We are given a DFA M with n states and alphabet size k
Want to find an optimal DFA A
First: remove unreachable states
unreachable means in the graph, cannot be reached from q0 on any input
Identify them with DFS from the starting state
Equivalent states: q,q’ are equivalent, if all strings y, starting from q or q’ lead to the same result (acc/rej)
can assume |y|< n, otherwise y contains a loop in M
equivalent states have the same row in the communication matrix

10. A simple algorithm Algorithm: Remove unreachable states
Build a n£n matrix D that containing blank entries
For all rows q, and columns q’, where q2 F and q’ is not, set D[q,q’]=D[q’,q]=²
Iterate [until nothing to do, at most n times]
Loop over all letters a and all pairs q,q’ with D[q,q’] blank
If D[±(q,a),±(q’,a)] is not blank, set D[q,q’]=a
Join states q,q’ that have D[q,q’] still blank

11. A simple algorithm
D allows us to find a witness that q,q’ are not equivalent: D[q,q’]=a, then find recursively the witness y for ±(q,a) and ±(q’,a), witness for q,q’ is ay
Follow ay from q and from q’, one leads to accept, the other to reject
Witnesses need not be longer than n-1
If no witness exists, then q,q’ are equivalent
by definition of equivalence

12. Time Each iteration takes time O(n2k)
If q and q’ are not equivalent, a witness of length no more than n-1 can be found, i.e., number of iterations is at most n-1
typically much less: longest simple path in DFA

13. Correctness All q,q’ with D[q,q’] not blank are not equivalent
Claim: algo will find a witness for non-equivalent q,q’
Induction over length of witness
length 0: in first iteration
Assume length k witnesses are found in k iterations
If q,q’ are not equivalent, and there is a string ay of length k such that ±(q,ay) accepts, ±(q’,ay) rejects, then some witness y’ for ±(q,a) and ±(q’,a) of length k-1 is found earlier
ay’ is a witness and found in iteration k
Claim: all equivalent states will never be declared not equivalent
Clear from the witness computed

14. Minimizing NFA? NFA are frequently smaller than DFA
Can we minimize their size?
Theorem:
NFA-minimization is PSPACE-hard
This remains true if the input is a DFA but we search the smallest NFA
Even approximation is hard

15. Conclusion
DFA can be transformed into unique minimal DFA in polynomial time
Can then compare, decide emptiness etc.
NFA and 2-way DFA/NFA can be exponentially smaller, but cannot be minimized efficiently
Limits of DFA:
Communication method

16. Regular Languages (Definition 2)
Simple recursive description of languages
Definition: Regular expression over alphabet §
a2§ is a r.e.
The empty word ² is a r.e.
; is a r.e. [empty set]
R1[ R2 is regular [union] Notation: r1 | r2
R1R2 is regular [concatenation]
R1* is regular [Kleene]

17. Examples 0*10*: strings with one 1 §* 1§*: strings with at least one 1
(§§)*: even length
1*(011*)*: every 0 followed by a 1
(0*10*10*)*: even number of 1’s
0*|1*: contains only 0’s or only 1’s

18. Regular vs. DFA
Definition: a language describable by a regular expression is called regular
Theorem: The set of regular languages is the same set as the set of languages decidable by DFA
Proof:
We show
NFA can simulate regular expression
Regular expression can be constructed from DFA

19. Regular to NFA Consider an NFA with ² transitions
Edges can be labeled with ² (empty word)
Exercise: NFA with ² transitions can be transformed into NFA, same number of states
R a regular expression
Inductively find an NFA for R
Basis cases: R=², R=a2§, R=;
NFA is trivial (at most 3 states)
Induction: Find NFA for Union, Concatenation, Kleene Hull

20. Regular to NFA Union R1[ R2: Concatenation: Kleene:
NFA M1 and M2, starting state ²-move to start in M1 and to start in M2
Concatenation:
Kleene:

21. DFA to regular
Generalized DFA: edges can carry regular expressions instead of letters
Idea 1: start with a normal DFA, shrink to a generalized DFA with 1 edge
Idea 2: dynamic programming type argument
Ri,j,k regular expression for the set of strings that lead from qi to qj using q0,…, qk only

22. DFA to regular
R0,0,0=(a|b|c)* if there are edges labeled a,b,c from q0 to q0
Ri,i,0 = ² otherwise
Ri,j,0 = a|b etc., if there are edges labeled a and b from qi to qj
Ri,j,k = Ri,j,k-1 | Ri,k,k-1 (Rk,k,k-1)* Rk,j,k-1
Finally take union over all R0,i,n, where qi is accepting
Result: a reg. expr. for the DFA
Works even for NFA

23. Conclusion Theoretical Computer Science Algorithms Computability
Complexity
Machine Models
Formal Languages
More: Cryptography, Information/Communication Theory …

24. Topics: Algorithms: Other theory: Basic Algorithms [Sorting/Searching]
Graph Algorithms
Paradigms [Greedy, Dynamic Programming]
Linear Programming
Algorithms for hard problems [Approximation]
Other theory:
Hardness and reductions [eg. NP-completeness]
Computability [eg. Halting problem, Universality]
Time-Hierarchy [etc.]
Finite Automata