1. CSC312
Automata Theory
Lecture # 1
Introduction

2. Administrative Stuff Instructor: Dr. Mudasser Naseer
Cabin # 1, Faculty Room C7
Lectures: Sec-A: Tue 16:30 (C-6), Wed 16:30 (C-11) Sec-B: Mon 11:30 (C-13), Wed 15:00 (C-11)
Office Hrs: Tue & Thu 1400 – 1600 hrs
(or by appointment)
Prerequisite: CSC102 - Discrete Structures

3. Course Objectives:
To study and mathematically model various abstract computing machines that serve as models for computations and examine the relationship between these automata and formal languages.

4. Course Outline
Regular expressions, NFAs. Core concepts of Regular Languages and Finite Automata; Decidability for Regular Languages; Non-regular Languages; Context-free Languages and Pushdown Automata; Decidability for Context-free Languages; Non-context-free Languages; Turing Machines and Their Languages are important part of the course. Transducers (automata with output).

5. Course Organization
Text Book: i) Denial I. A. Cohen Introduction to Computer Theory, Second Edition, John Wiley & Sons, 1997.
Reference Books:
i) J. E. Hopcroft, R. Motwani, & J. D. Ullman Introduction to Automata Theory, Languages, and Computation, Third Edition, Pearson, 2008.
Instruments: There will be 2~3 assignments, 4~5 quizzes, Weights: Assignments 10% Quizzes 15% S-I 10%
S-II 15% Final Exam 50%

6. Schedule of Lectures
Lect.#
Topics/Contents 1
Introduction to Automata theory, Its background, Mathematical Preliminaries, Sets, Functions, Graphs, Proof Techniques 2
Formal Languages, Introduction to defining languages, alphabet, language, word, null string, length of a string, reverse of a string, Palindrome, Kleene closure. 3
Formal definition of Regular Expressions, Defining languages with regular expressions, Languages associated with regular expressions. 4
Equality of Regular Expressions, Introducing the language EVEN-EVEN. 5
More examples related to regular expressions. 6
Introducing Finite Automata., Defining languages using Finite Automata. Constructing Finite Automata for different languages. 7
Recognizing the language defined by the given Finite Automata. 8
More examples related to Finite Automata. 9
Transition Graphs with examples, Generalized Transition Graphs, Non-determinism in case of Transition Graphs. 10
Non-deterministic FA’s. Differences between FA, TG and NFA. 11
Sessional I

7. Schedule of Lectures
Lect.#
Topics/Contents 12
Kleene’s Theorem, Algorithm for turning TGs into REs 13
Kleene’s Theorem, Algorithm for turning REs into FAs 14
Nondeterminism, NFA, converting NFA into FA. 15
Finite Automata with output, Moore’s machines and Mealy machines with examples. 1’s Complement machine, Increment machine. 16
Theorems for Converting Moore machines into Mealy machines and vice versa. Transducers as models of sequential circuits. 17
Regular Languages, Closure properties (i.e. , Concatenation and Kleene closure) of Regular Languages with examples. 18
Complements and Intersections of Regular Languages, Theorems relating to regular languages and the related examples. 19
Non-Regular Languages, The pumping Lemma, Examples relating to Pumping Lemma. 20
Decidability, decision procedure, Blue-paint method, Effective decision procedure to prove whether two given RE’s or FA’s are equivalent. Myhill-Nerode theorem, Related Examples. 21
Sessional II

8. Schedule of Lectures
Lect.#
Topics/Contents 22
Context-Free Grammars, CFG’s for Regular Languages with examples. CFG’s for non-regular languages. 23
CFG’s of PALINDROME, EQUAL and EVEN-EVEN languages, Backus-Naur Form. 24
Parse Trees, Examples relating to Parse Trees, Lukasiewicz notation, Prefix and Postfix notations and their evaluation. 25
Ambiguous and Unambiguous CFG’s, Syntax tree, Total language tree. 26
Regular Grammars, Semi-word, Word, Working String, Converting FA’s into CFG’s. Constructing Transition Graphs from Regular Grammars. 27
Killing null productions. Killing unit productions, Chomsky Normal form with examples, Left most derivations. 28
Pushdown Automata, Constructing PDA’s for FA’s, Pushdown stack. 29
Examples related with PDA, PDA for Odd Palindrome, Even Palindrome, PalindromeX. 30
Nondeterministic PDA. Proving CFG = PDA with examples. 31
Constructing PDA from CFG in CNF with examples 32
Turing machines, Examples of Turing Machines with trace tables, Converting FA’s into Turing machines.

9. Some basics
Automaton = A self-operating machine or mechanism (Dictionary definition), plural is Automata.
Automata = abstract computing devices
Automata theory = the study of abstract machines (or more appropriately, abstract 'mathematical' machines or systems, and the computational problems that can be solved using these machines.
Mathematical models of computation
Finite automata
Push-down automata
Turing machines

10. History
1930s : Alan Turing defined machines more powerful than any in existence, or even any that we could imagine – Goal was to establish the boundary between what was and was not computable.
1940s/150s : In an attempt to model “Brain function” researchers defined finite state machines.
Late 1950s : Linguist Noam Chomsky began the study of Formal Grammars.
1960s : A convergence of all this into a formal theory of computer science, with very deep philosophical implications as well as practical applications (compilers, web searching, hardware, A.I., algorithm design, software engineering,…)

11. Courtesy Costas Busch - RPI
Computation
memory
CPU
Courtesy Costas Busch - RPI

12. Courtesy Costas Busch - RPI
temporary memory
input memory
CPU
output memory
Program memory
Courtesy Costas Busch - RPI

13. Courtesy Costas Busch - RPI
Example:
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
Courtesy Costas Busch - RPI

14. Courtesy Costas Busch - RPI
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
Courtesy Costas Busch - RPI

15. Courtesy Costas Busch - RPI
temporary memory
input memory
CPU
output memory
Program memory
compute
compute
Courtesy Costas Busch - RPI

16. Courtesy Costas Busch - RPI
temporary memory
input memory
CPU
Program memory
output memory
compute
compute
Courtesy Costas Busch - RPI

17. Courtesy Costas Busch - RPI
Automaton
temporary memory
Automaton
input memory
CPU
output memory
Program memory
Courtesy Costas Busch - RPI

18. Courtesy Costas Busch - RPI
Different Kinds of Automata
Automata are distinguished by the temporary memory
Finite Automata: no temporary memory
Pushdown Automata: stack
Turing Machines: random access memory
Courtesy Costas Busch - RPI

19. Courtesy Costas Busch - RPI
Finite Automaton
temporary memory
input memory
Finite
Automaton
output memory
Example: Vending Machines
(small computing power)
Courtesy Costas Busch - RPI

20. Courtesy Costas Busch - RPI
Pushdown Automaton
Stack
Push, Pop
input memory
Pushdown
Automaton
output memory
Example: Compilers for Programming Languages
(medium computing power)
Courtesy Costas Busch - RPI

21. Courtesy Costas Busch - RPI
Turing Machine
Random Access Memory
input memory
Turing
Machine
output memory
Examples: Any Algorithm
(highest computing power)
Courtesy Costas Busch - RPI

22. Courtesy Costas Busch - RPI
Power of Automata
Finite
Automata
Pushdown
Automata
Turing
Machine
Less power
More power
Solve more
computational problems
Courtesy Costas Busch - RPI

23. Mathematical Preliminaries
Sets
Functions
Relations
Graphs
Proof Techniques
Courtesy Costas Busch - RPI

24. Courtesy Costas Busch - RPI
SETS
A set is a collection of elements
We write
Courtesy Costas Busch - RPI

25. Courtesy Costas Busch - RPI
Set Representations
C = { a, b, c, d, e, f, g, h, i, j, k }
C = { a, b, …, k }
S = { 2, 4, 6, … }
S = { j : j > 0, and j = 2k for some k>0 }
S = { j : j is nonnegative and even }
finite set
infinite set
Courtesy Costas Busch - RPI

26. Courtesy Costas Busch - RPI1 2 3 4 5 A U 6 7 8 9 10
Universal Set: all possible elements
U = { 1 , … , 10 }
Courtesy Costas Busch - RPI

27. Courtesy Costas Busch - RPI
Set Operations
A = { 1, 2, 3 } B = { 2, 3, 4, 5}
Union
A U B = { 1, 2, 3, 4, 5 }
Intersection
A B = { 2, 3 }
Difference
A - B = { 1 }
B - A = { 4, 5 } A B 2 4 1 3 5 U 2 3 1
Venn diagrams
Courtesy Costas Busch - RPI

28. Courtesy Costas Busch - RPI
Complement
Universal set = {1, …, 7}
A = { 1, 2, 3 } A = { 4, 5, 6, 7} 4 A A 6 3 1 2 5 7
A = A
Courtesy Costas Busch - RPI

29. Courtesy Costas Busch - RPI
{ even integers } = { odd integers }
Integers 1
odd
even 5 6 2 4 3 7
Courtesy Costas Busch - RPI

30. Courtesy Costas Busch - RPI
DeMorgan’s Laws
A U B = A B U
A B = A U B U
Courtesy Costas Busch - RPI

31. Courtesy Costas Busch - RPI
Empty, Null Set:
= { }
S U = S
S =
S = S
- S = U
= Universal Set
Courtesy Costas Busch - RPI

32. Courtesy Costas Busch - RPI
Subset
A = { 1, 2, 3} B = { 1, 2, 3, 4, 5 }
A B U
Proper Subset:
A B U B A
Courtesy Costas Busch - RPI

33. Courtesy Costas Busch - RPI
Disjoint Sets
A = { 1, 2, 3 } B = { 5, 6}
A B = U A B
Courtesy Costas Busch - RPI

34. Courtesy Costas Busch - RPI
Set Cardinality
For finite sets
A = { 2, 5, 7 }
|A| = 3
(set size)
Courtesy Costas Busch - RPI

35. Courtesy Costas Busch - RPI
Powersets
A powerset is a set of sets
S = { a, b, c }
Powerset of S = the set of all the subsets of S
2S = { , {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c} }
Observation: | 2S | = 2|S| ( 8 = 23 )
Courtesy Costas Busch - RPI

36. Cartesian Product A = { 2, 4 } B = { 2, 3, 5 }
A X B = { (2, 2), (2, 3), (2, 5), ( 4, 2), (4, 3), (4, 5) }
|A X B| = |A| |B|
Generalizes to more than two sets
A X B X … X Z
Courtesy Costas Busch - RPI

37. Courtesy Costas Busch - RPI
FUNCTIONS
domain
range 4 A B
f(1) = a a 1 2 b c 3 5
f : A -> B
If A = domain
then f is a total function
otherwise f is a partial function
Courtesy Costas Busch - RPI

38. Courtesy Costas Busch - RPI
RELATIONS
Let A & B be sets. A binary relation “R” from A to B
R = {(x1, y1), (x2, y2), (x3, y3), …}
Where and
R ⊆ A x B
xi R yi to denote
e. g. if R = ‘>’: 2 > 1, 3 > 2, 3 > 1
Courtesy Costas Busch - RPI

39. Courtesy Costas Busch - RPI
Equivalence Relations
Reflexive: x R x
Symmetric: x R y y R x
Transitive: x R y and y R z x R z
Example: R = ‘=‘
x = x
x = y y = x
x = y and y = z x = z
Courtesy Costas Busch - RPI

40. Courtesy Costas Busch - RPI
GRAPHS
A directed graph e b
node d a
edge c
Nodes (Vertices)
V = { a, b, c, d, e }
Edges
E = { (a,b), (b,c), (b,e),(c,a), (c,e), (d,c), (e,b), (e,d) }
Courtesy Costas Busch - RPI

41. Courtesy Costas Busch - RPI
Labeled Graph 2 6 e 2 b 1 3 d a 6 5 c
Courtesy Costas Busch - RPI

42. Courtesy Costas Busch - RPI
Walk a b c d e
Walk is a sequence of adjacent edges
(e, d), (d, c), (c, a)
Courtesy Costas Busch - RPI

43. Courtesy Costas Busch - RPI
Path a b c d e
Path is a walk where no edge is repeated
Simple path: no node is repeated
Courtesy Costas Busch - RPI

44. Courtesy Costas Busch - RPI
Cycle e
base b 3 1 d a 2 c
Cycle: a walk from a node (base) to itself
Simple cycle: only the base node is repeated
Courtesy Costas Busch - RPI

45. Courtesy Costas Busch - RPI
Euler Tour 8
base e 7 1 b 4 6 5 d a 2 3 c
A cycle that contains each edge once
Courtesy Costas Busch - RPI

46. Courtesy Costas Busch - RPI
Hamiltonian Cycle 5
base e 1 b 4 d a 2 3 c
A simple cycle that contains all nodes
Courtesy Costas Busch - RPI

47. Courtesy Costas Busch - RPI
Trees
root
parent
leaf
child
Trees have no cycles
Courtesy Costas Busch - RPI

48. Courtesy Costas Busch - RPI
root
Level 0
Level 1
Height 3
leaf
Level 2
Level 3
Courtesy Costas Busch - RPI

49. Courtesy Costas Busch - RPI
Binary Trees
Courtesy Costas Busch - RPI

50. Courtesy Costas Busch - RPI
PROOF TECHNIQUES
Proof by induction
Proof by contradiction
Courtesy Costas Busch - RPI

51. Courtesy Costas Busch - RPI
Induction
We have statements P1, P2, P3, …
If we know
for some b that P1, P2, …, Pb are true
for any k >= b that
P1, P2, …, Pk imply Pk+1
Then
Every Pi is true
Courtesy Costas Busch - RPI

52. Courtesy Costas Busch - RPI
Proof by Induction
Inductive basis
Find P1, P2, …, Pb which are true
Inductive hypothesis
Let’s assume P1, P2, …, Pk are true,
for any k >= b
Inductive step
Show that Pk+1 is true
Courtesy Costas Busch - RPI

53. Courtesy Costas Busch - RPI
Example
Theorem: A binary tree of height n
has at most 2n leaves.
Proof by induction:
let L(i) be the maximum number of
leaves of any subtree at height i
Courtesy Costas Busch - RPI

54. Courtesy Costas Busch - RPI
We want to show: L(i) <= 2i
Inductive basis
L(0) = (the root node)
Inductive hypothesis
Let’s assume L(i) <= 2i for all i = 0, 1, …, k
Induction step
we need to show that L(k + 1) <= 2k+1
Courtesy Costas Busch - RPI

55. Courtesy Costas Busch - RPI
Induction Step
height k
k+1
From Inductive hypothesis: L(k) <= 2k
Courtesy Costas Busch - RPI

56. Courtesy Costas Busch - RPI
Induction Step
height
L(k) <= 2k k
k+1
L(k+1) <= 2 * L(k) <= 2 * 2k = 2k+1
(we add at most two nodes for every leaf of level k)
Courtesy Costas Busch - RPI

57. Courtesy Costas Busch - RPI
Remark
Recursion is another thing
Example of recursive function:
f(n) = f(n-1) + f(n-2)
f(0) = 1, f(1) = 1
Courtesy Costas Busch - RPI

58. Courtesy Costas Busch - RPI
Proof by Contradiction
We want to prove that a statement P is true
we assume that P is false
then we arrive at an incorrect conclusion
therefore, statement P must be true
Courtesy Costas Busch - RPI

59. Courtesy Costas Busch - RPI
Example
Theorem: is not rational
Proof:
Assume by contradiction that it is rational
= n/m
n and m have no common factors
We will show that this is impossible
Courtesy Costas Busch - RPI

60. = n/m m2 = n2
n is even
n = 2 k
Therefore, n2 is even
m is even
m = 2 p
2 m2 = 4k2
m2 = 2k2
Thus, m and n have common factor 2
Contradiction!