# 1 Algorithmic Problems Zeph Grunschlag. 2 Agenda Viewing algorithms as languages Algorithmic Decision Problems A TM, A CFG, A REX, E TM, … etc.

## Presentation on theme: "1 Algorithmic Problems Zeph Grunschlag. 2 Agenda Viewing algorithms as languages Algorithmic Decision Problems A TM, A CFG, A REX, E TM, … etc."— Presentation transcript:

1 Algorithmic Problems Zeph Grunschlag

2 Agenda Viewing algorithms as languages Algorithmic Decision Problems A TM, A CFG, A REX, E TM, … etc.

3 Knowing the Unknowable Arrived at important juncture in course. Now we will study one of the fundamental questions in Computer Science: What is the ultimate power of computational machinery? Of equal importance in human quest to understand self: What are the impenetrable limits of human knowledge?

4 Knowing the Unknowable Accepting the Church-Turing thesis implies that these are the same question! Church-Turing thesis is usually recognized as applying to human computations, and therefore human thought processes. Amazing Fact: It is possible to give precise examples of problems which are beyond the reach of mechanical computation as well as human intellect. We in fact “know the unknowable.”

5 Decision Problems A decision problem is a set of yes/no questions, each called an instance. For the decision problem to be interesting, the set of questions should be infinite. EG: “Does z equal the sum of x and y ?” is a good example. A computer program that can give the correct solution for any possible x,y,z intuitively must have figured out how to add numbers. EG: “Does 8 equal the sum of 3 and 5?” is a bad example. Could easily write program for this instance without knowing how to add!

6 Decision Problems Q: How about the problem “Does God Exist?” Can a computer solve this problem?

7 Decision Problems A: In the way defined above, the answer is yes! We give two computer programs, both of which accept the input “Does God exist?” One of these programs is correct! Therefore, that computer program has solved the problem of deciding if God exists! Q: Any misgivings? A(String question){ return “No!” } B(String question){ return “Yes!” }

8 Decision Problems A: Many possible misgivings. Here’s one: “Solution” to existence of God surely not what is meant by solving the problem. We expected the solution to be arrived at by careful consideration of physical evidence, or by theological arguments, or some other intricate process. CS viewpoint: Any algorithm which works only on a finite set of inputs is considered meaningless, as could always program a look- up table which gave correct solutions without using any insight.

9 Decision Problems So to solve the deity existence problem would have to give a more general solution. EG: Given: A complete description of a physical universe and a complete description of a deity. Decide: Does the deity necessarily exist for the given universe? Now there are infinitely many possible inputs, so cannot just use a look-up table and this is a meaningful problem, though hopeless…

10 Decision Problems as Languages To study what it means for a decision problem to be solvable, need a method of standardizing such problems and making them easily understood by computers. Can’t just come over to a computer and ask “Does 2 + 2 = 4?” and even if could, dealing with voice recognition, natural language processing, etc. takes us far afield to topics irrelevant to answering “is the problem solvable?” Solution: represent each instance by a string: Thus instance is the string “Does 2 + 2 = 4?” or more simply “2+2=4”. Q: Define solvability language theoretically?

11 Decision Problems as Languages A: A decision problem P is solvable if its encoding by strings admits a computer program which inputs such encodings, always halts, and always outputs the correct answer. Furthermore, the set of encodings should be computable in its own right. Language theoretically: Let  be the alphabet in which the encoded instances of P exist. Let E be the set of encoded instance and let L be the set corresponding to the YES instances. 1 Then P is said to be decidable (or algorithmically solvable or computable) if L is decidable.

12 Decision Problems Sensible Encodings Some common sense is needed: The encoding scheme should be sensible but there is no way to define what sensible means. EG: “Is p a prime number?” Encoding scheme which results in the positive instances L = {2,3,5,7,…} is sensible. Encoding scheme which results in the positive instances L = {1,4,6,8,9,10,…} is not!

13 Decision Problems Sensible Encodings Sensible encoding of sensible decision problem would never result in an undecidable encoding set E but condition still needed so don’t solve impossible problems unwittingly. EG: “Does a given Java program with no infinite loops print out “I’m not loopy!” on some input?” Natural encoding set E is over Unicode alphabet, and is just the set of Java programs with no infinite loops. But we’ll see that E is itself undecidable so second part becomes meaningless.

14 Decision Problems Basic Problems Before tackling fundamental decision problems of CS, start introspectively by considering if there are algorithms for… …telling if a given string is accepted by a given language of a fixed type C such as DFA’s, NFA’s, PDA’s etc. Acceptance Problem A C …telling if a given language of type is C empty. Emptiness Problem E C …telling if two given languages of the same type C are equal. Equivalence Problem EQ C Each in turn, is encoded by a language.

15 Decision Problems Encoding Example Let’s see how A DFA may be encoded. A DFA is the problem of deciding Given: A DFA M and a string x. Decide: Does M accept x ? The language L of interest is the language of encodings of pairs (M, x ) where M accepts x. Encoding is denoted using the angled braces notation “ ”. In practice, will never really work with encodings as too unwieldy. However, for once it’s worth to see how this might actually be carried out.

16 Decision Problems Encoding Example Consider the DFA: Q: How can you represent this by a string? b a 0,1

17 Decision Problems Encoding Example A: DFA’s are 5-tuples, so just write down the 5 tuple, spelling out all details. Therefore = ({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a}) b a 0,1

18 Decision Problems Encoding Example A: DFA’s are 5-tuples, so just write down the 5 tuple, spelling out all details. Therefore = ({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a}) Q b a 0,1

19 Decision Problems Encoding Example A: DFA’s are 5-tuples, so just write down the 5 tuple, spelling out all details. Therefore = ({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})  b a 0,1

20 Decision Problems Encoding Example A: DFA’s are 5-tuples, so just write down the 5 tuple, spelling out all details. Therefore = ({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a})  b a 0,1

21 Decision Problems Encoding Example A: DFA’s are 5-tuples, so just write down the 5 tuple, spelling out all details. Therefore = ({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a}) q 0 b a 0,1

22 Decision Problems Encoding Example A: DFA’s are 5-tuples, so just write down the 5 tuple, spelling out all details. Therefore = ({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a}) F b a 0,1

23 Decision Problems Encoding Example A: DFA’s are 5-tuples, so just write down the 5 tuple, spelling out all details. Therefore = ({a,b},{0,1},{(a,0,b),(a,1,b),(b,0,a),(b,1,a)},a,{a}) Encoding amounts to just adding the comma and the input string w. Q: What’s the alphabet of encoding lang. E ? b a 0,1

24 Decision Problems Encoding Example A: TOO BIG! The number of letters in alphabet and number of states is unbounded. Using different characters for each state/letter would require an infinite alphabet if want to be able to encode every possible DFA! Instead, should fix some alphabet (e.g. binary or ASCII) and refer to states/letters using numbers. Thus the state a is denoted by 1, since it’s the first state, the state b by 2 and so on…

25 Decision Problems What You Really Do! After going to all this effort to define decision problems in terms of languages, Turing machines, encodings and so on, in practice, we use none of the above. The point was to argue that in principle this could be done, and remind ourselves that this ought to be achievable, when wondering whether a not purported pseudocode is really an algorithm. Sipser’s approach tries to maintain the veneer of this approach. I find this to be artificial at times so will just work directly with the natural data structure inherited by the problem.

26 A DFA For example, the easiest way to view a DFA is as a labeled graph. Encoding the DFA by a string and trying to apply some graph theoretic algorithm to the representative string is overly burdensome. You basically have to recreate the whole graph on the TM tape, which in principle can be done, but in practice is silly, especially since the starting point was a graph! Q: How do you solve A DFA ?

27 A DFA A: Just follow the path labeled by the input string from the start state. Accept if end was in F. More precisely. DFAaccept(DFA M, String x 1 x 2 x 3 … x n ) State q = q 0 //q 0 defined by M’s 5-tuple for(i = 1 to n) q =  (q,x i )//  defined by M’s 5-tuple if(q  F)// F defined by M’s 5-tuple return ACCEPT else return REJECT

28 A NFA Q: How about A NFA ?

29 A NFA A: Just determinize first and use solution to A DFA. NFAaccept(NFA N, String x 1 x 2 x 3 … x n ) DFA M = determinize(N)//Subset Constr’n return DFAaccept(M, x 1 x 2 x 3 … x n ) Q1: What’s the running time? Q2: What’s not ideal about this algorithm? Q3: What’s the fix?

30 A NFA A1: O(2 |N| +n) since determinizing might involve all the subsets of the states of N so is exponential, but once have determinized, simulating the DFA is O(n). Q2: The problem is the exponential running time. Q3: We already know the fix. It’s just the game “Determinize” which keeps track of active state without explicitly constructing all the subsets. Running time of O(|N | 2 ·n) since each update of the active states may involve looking at O(|N |) active states each of which may activate O(|N |) other states.

31 A NFA NFAaccept2(NFA N, String x 1 x 2 x 3 … x n ) StateSet S = {q 0 }// store S as bit string for(i = 1 to n) S =  (S,x i )// since NFA,  outputs a set if(S and F are not disjoint) return ACCEPT else return REJECT

32 E DFA Let’s now focus on emptiness problem. We can use the pumping lemma to solve the emptiness problem: Suppose we know what the pumping number p is for our DFA. Then if we found no accepted strings of length < p we could be sure that the DFA accepts no strings. Q: Why is this true?

33 E DFA A: Suppose language not empty. Since checked that all strings of length < p are rejected, smallest accepted string s must be of length  p. The string s is pumpable, so can be pumped down obtaining a shorter string s’, which contradicts the minimality of s ! This prompts the following algorithm:

34 E DFA DFAempty( DFA M ) integer p = |Q | for (all strings x over  with length < p) if( DFAaccepts(M,x) == ACCEPT ) return NONEMPTY //only got here if nothing was accepted return EMPTY Q: Running time? Improvements?

35 E DFA A: Running time: O(|  | |Q| ) –exponential. Improvements: Just see if any accept state can be reached from start state by performing a BFS or DFS as follows:

36 E DFA DFAempty2( DFA M ) State q = q 0 Stack S // Initialize a LIFO stack Set V =  // Set of visited states S.push(q 0 ) while(S   ) q = S.pop() if (q  F ) return NONEMPTY V = V  {q} for (States q’ satisfying q  q’ ) // an edge if (q’  V ) S.push(q’ ) return EMPTY // Only got here if F unreachable Q: Is this a BFS or DFS?

37 E DFA A: This is a DFS algorithm since stack is used. Queue would give BFS.

38 A CFG Acceptance problem is also solvable. There’s a rather horrible algorithm involving Chomsky Normal Form. Later we’ll see a much improved polynomial time algorithm (must have one, otherwise compilers would be useless!) Chomsky normal form gives the following fact: LEMMA: Suppose a grammar G is in Chomsky normal form. Let x be in L(G ), and let n be the length of x. Then x is generated by a derivation of length 2n-1, when n > 0.

39 A CFG LEMMA: Suppose a grammar G is in Chomsky normal form. Let x be in L(G ), and let n be the length of x. Then x is generated by a derivation of length 2n-1, when n > 0. Proof. The first n-1 productions are of the form A  BC and get us to the correct length. The last n production are unit terminating A  a and derive x. This gives rise to the following algorithm:

40 A CFG CFGaccept( CFG G, String x=x 1 x 2 x 3 … x n ) CFG G’ = ChomskyNormalForm(G ) for (all derivations from start variable of G’ of length  2n +1) if (derivation resulted in x) return ACCEPT return REJECT Q1: Why does this work for x =  ? Q2: Running time in terms of n ?

41 A CFG A1: Works for x =  because of “  2n+1” clause. A2: Exponential. If we consider the blow-up of CNF conversion, this is a truly horrendous algorithm.

42 Blackboard Exercises 1. EQ DFA 2. E CFG 3. CARD DFA

Download ppt "1 Algorithmic Problems Zeph Grunschlag. 2 Agenda Viewing algorithms as languages Algorithmic Decision Problems A TM, A CFG, A REX, E TM, … etc."

Similar presentations