1. Regular Languages Sequential Machine Theory Prof. K. J. Hintz Department of Electrical and Computer Engineering Lecture 3 Comments, additions and modifications by Marek Perkowski

2. LanguagesLanguages Informal Languages –English –Body language –Bureaucratic conventions and procedures Formal Languages –1) Rule-based –2) Elements are decidable –3) No deeper understanding required

3. Formal Language All the Rules of the Language Are Explicitly Stated in Terms of the Allowed Strings of Symbols, e.g., –Programming languages, e.g., C, Lisp, Ada –Military communications –Digital network protocols

4. Alphabets Alphabet: a finite set of symbols, aka I,  –Roman: { a, b, c,..., z } –Binary:{ 0, 1 } –Greek:{  } –Cyrillic:{ Ж, Й, Њ, С, Р,... }

5. String String, word: a finite ordered sequence of symbols from the alphabet, usually written with no intervening punctuation –x 1 = “ t h e “ –x 2 = “ 0 1 0 1 1 0 “ –x 3 = “ “ –x 4 = “ Ж Й С Р “

6. String Reverse of String –The sequence of symbols written backwards Reverse of Concatenation –Strings themselves must be reversed

7. String Length or Size of String –The number of symbols

8. Strings Null String, Empty String, e,  –A string of length or size zero –The symbols e or , meant to denote the null string, are not allowed to be part of the language

9. Substring A String, v, Is a Substring of a String, w, iff There Are Strings x and y Such That –w = x v y –x is called the prefix –y is called the suffix –x and/or y could be 

10. Kleene Closure Set of All Strings,  *, I* –Order IS important –Not the same as, the powerset of the alphabet, since order is NOT important in the powerset

11. Concatenation Operator If x, y  I*, then the concatenation of x and y is written as –z = x  y –e.g., if x = “Red”| x | = 3 y = “skins”| y | = 5 z = x  y = “Redskins“| z | = 8

12. Concatenation Operator Concatenation of Any String With the Null String Results in the Original String –x  e = e  x = x –x   =   x = x Concatenation is Associative –x = abcy=defz= ghi –( x  y )  z = x  ( y  z ) –abcdefghi = abcdefghi

13. Language Language, L : Any Subset of the Set of All Strings of an Alphabet I*I* L1L1 L2L2

14. Classes of Languages Enumerated Languages –Defined by a List of All Words in the Language L e = { “quidditch”, “nimbus 2000”, } not very interesting Rule-based –Defined by Properties or a Set of Rules

15. Rule Based Languages A Test to Determine Whether a String Is a Member of a Language A Means of Constructing Strings That Are in the Language –Must be able to construct ALL strings in the language –Must be able to construct ONLY strings in the language

16. Rule-Based Language Example Let I = { a, b } A Language That Consists of All Two Letter Strings – L = { aa, ab, ba, bb } –  is not an element of the language

17. Empty Language Null Language, Empty Language,  : The Language With No Words in It –Not the same as  –  can be made into a language with words –A language consisting only of  is still a language

18. Kleene Star If is a language, then L * Is the Set of All Strings Obtained by Concatenating Zero or More Strings of L. Concatenation of Zero Strings Is  Concatenation of One String Is the String Itself L + = L * - {  }

19. Kleene Closure Example L = { 0, 1} L * = { , 0, 00, 000,..., 0*, 1, 11, 111,..., 1*, 01, 001, 0001,..., 0*1,... }

20. Kleene Closure Examples L = { ab, f } L * = { , ab, abf, fab, ffab, ffabf,... }  * = {  } if L = {  } then L * = {  }

21. Kleene Closure Examples Let I = { a, b } L = Language ( ( ab )* ) { , ab, abab, ababab,... } which is not the same as L = Language ( a* b* ) { , a, b, ab, aab, abb,... } The language of all strings of a’s and b’s in which the a’s, if any, come before the b’s

22. Recursive Language Definition Variation of Rule-Based Three-step Process 1. Specify some basic elements of the set 2. Specify the rules for forming new elements from old elements of the set 3. Specify that elements not in 1 or 2 above are NOT elements of the set

23. Recursive Example Two Equivalent Recursive Definitions of Rational Numbers –Rational #1 – we define set Rational#1 of rational numbers 1. Rat_1 = { - ,... -3, -2, -1, 1, 2, 3,...,  } 2. if p, q  Rat_1, then p/q  Rat_1 3. the only rational numbers are those generated by 1 and 2 above.

24. Recursive Example –Rational #2, we generate the set of rational numbers with different rules, but this is the same set. 1. Rat_2 = { -1, +1 } 2. if p, q  Rat_2, p,q != 0, then (p+q)/p  Rat_2 3. the only rational numbers are those generated by 1 and 2 above. e.g., generates all integers, similarly negative integers. Now we can generate any rational number Example. To create 2/3  take p=3, then take p+q=2 thus p=-1 which is negative integer, OK

25. Interest in Recursive Definitions Recursive definition allow us to prove some Statements About What Is Computable. Recursive definition leads to Proof by Induction

26. Principle of Mathematical Induction* Let A Be a Subset of the Natural Numbers 0  A, and for each natural number, n, –if { 0, 1,..., n }  A, –implies(n + 1)  A –then A = N * Lewis & Papadimitriou, pg. 24

27. Mathematical Induction In practice, mathematical induction is used to prove assertions of the form For all natural numbers, n, property P is true

28. Mathematical Induction Practice To prove statements of the form A = { n : P is true of n }, three steps 1. Basis Step: show that 0  A, i.e., P is true of n = 0 2. Induction Hypothesis: assume that for some arbitrary, but fixed n > 0, P holds for each natural number 0, 1,..., n

29. Mathematical Induction Practice 3. Induction Step: use the induction hypothesis (that P is true of n) to show that P is true of (n + 1) By the Induction Principle, Then A=N and Hence, P Is True of Every Natural Number.

30. Induction Example* * Lewis & Papadimitriou, pg. 25 1. Basis Step

31. Induction Example 2. Induction Hypothesis We just assume that the rule is true for certain m smaller than n

32. Induction Example 3. Induction Step

33. Induction Example

34. Another Induction Example Define set EVEN as 1. 0 is in EVEN 2. if x  EVEN then so is x + 2 3. The only elements of EVEN are those produced by 1 & 2 above. Prove by induction that all of elements of EVEN end in either 0, 2, 4, 6, or 8.

35. Induction Example (cont) Proof 1. Basis Step 0  EVEN by definition, therefore the property is true of the zero’th step since 0  { 0, 2, 4, 6, 8 } 2. Induction Hypothesis Assume that the last digit of (m+2)  { 0, 2, 4, 6, 8 } for 0 < m < n

36. Induction Example 3. Induction Step n  EVEN 02... 14n2n + 2 26n+1(2n+2)+2 38n+12(n+1) +2 410 ends in {0,2,4,6,8} by step 2 0+2=2, 2+2=4,4+2=6 6+2=8, 8+2=0  {0,2,4,6,8} Prove by induction that all of elements of EVEN end in either 0, 2, 4, 6, or 8. 2n+2 Thus if for n it ends wih 0,2,4,6,8 then for n+1 it also ends with 0,2,4,6,8

37. Regular Expressions Shorthand Notation for Concisely Expressing Languages Defined Recursively Lead to a Definition of Regular Languages Provide Finite Representation of Possibly Infinite Languages Lead to Lexical Analyzers

38. Regular Expressions Notation

39. Regular Expressions Over I  and  are regular expressions a is a regular expression for each a  I If r and s are regular expressions, then so are r  s, r  s, and r* No other sequences of symbols are regular expressions

40. Regular Expressions Alternative 1. L (  ) = {  } L ( a ) = { a } If p and q are regular expressions, then 2. L ( pq ) = L ( p ) L ( q ) 3. L ( p  q ) = L ( p )  L ( q ) 4. L ( p* ) = L ( p )* Regular expressions

41. Regular Expressions Example What is L 3 ( ( a  b )* a ) ? Observe that we do everything by completely formal transformations between expressions representing languages and sets (languages)

42. Regular Expressions Boolean OR Distributes over Concatenation –which is the language of all strings beginning with a, ending with b, and having none or more c’s in the middle, and, –all strings beginning and ending with b and having at least one c in the middle

43. Regular Expressions The Boolean OR Operator Can Distribute When It Is Inside a Kleene Starred Expression, but Only in Certain Ways Be very careful and do not invent or guess identities for tranformations, use only those that were proven and given here

44. Regular Expressions Useful String –( a + b )* = the set of all strings of a and b of any length – L = Language ( ( a + b )* ) –{ , a, b, ba, ab,… abab, abaab, abbaab, babba, bbb,... }

45. Regular Languages If L  I* is finite, then L is regular. If L 1 and L 2 are regular, so are – L 3 = L 1  L 2 – L 4 = L 1  L 2 = {x 1  x 2 | x 1  L 1, x 2  L 2 } If L is regular, then so is L *, where * is the Kleene Star

46. Regular Languages If L Is a Finite Language, Then L Can Be Defined by a Regular Expression. The Converse Is Not True. That Is, Not All Regular Expressions Represent Finite Languages. L = Language( ( a + b )* ) Is Infinite Yet Regular

47. Typical Homework Typical homework in this area may include the following: 1.Converting arbitrary regular expression to a graph and next converting this graph to a NDFA. 2.Creating a deterministic or non-deterministic stack-based automaton for language such as deterministic or non- deterministic palindromes or language similar to {a n b n | n = 0, 1,2,…..} 3.Example: Find a deterministic state machine for the following language “even number of zeros after odd number of ones or odd number of zeros that follow even number of ones” 4.You should be able to transit among regular expression, non- deterministic and deterministic automata for this expression and a corresponding regular grammar in any direction, for instance you may start from a regular grammar and write a regular expression, or start from a non-deterministic automaton and write the set of rules for the grammar of the language that this automaton accepts.