1. David Evans http://www.cs.virginia.edu/evans CS588: Security and Privacy University of Virginia Computer Science Lecture 15: Complexity Notes This is selected from CS200 lectures 16, 30, 37, 38 from http://www.cs.virginia.edu/cs200/lectures/

2. CS588 Spring 20052 What does  really mean? O( x ) – it is no more than x work (upper bound)  ( x ) – work scales as x (tight bound)  ( x ) – it is at least x work (lower bound) If O( x ) and  ( x ) are true, then  ( x ) is true.

3. CS588 Spring 20053 Meaning of O (“big Oh”) f(x) is O ( g (x) ) means: There is a positive constant c such that c * f(x) < g(x) for all but a finite number of x values.

4. CS588 Spring 20054 O Examples f(x) is O ( g (x) ) means: There is a positive constant c such that c * f(x) < g(x) for all but a finite number of x values. x is O ( x 2 )? Yes, c = 1 works fine. 10x is O ( x )? Yes, c =.09 works fine. x 2 is O ( x )? No, no matter what c we pick, cx 2 > x for big enough x

5. CS588 Spring 20055 Lower Bound:  (Omega) f(x) is  ( g (x) ) means: There is a positive constant c such that c * f(x) > g(x) for all but a finite number of x values. Difference from O – this was <

6. CS588 Spring 20056 f(x) is  ( g (x) ) means: There is a positive constant c such that c * f(x) > g(x) for all but a finite number of x values. f(x) is O ( g (x) ) means: There is a positive constant c such that c * f(x) < g(x) for all but a finite number of x values. Examples x is  ( x ) –Yes, pick c = 2 10x is  ( x ) –Yes, pick c = 1 Is x 2  ( x )? –Yes! x is O( x ) –Yes, pick c =.5 10x is O( x ) –Yes, pick c =.09 x 2 is not O( x )

7. CS588 Spring 20057 Tight Bound:  (Theta) f(x) is  ( g (x) ) iff: f(x) is O ( g (x) ) and f(x) is  ( g (x) )

8. CS588 Spring 20058  Examples 10x is  ( x ) –Yes, since 10x is  ( x ) and 10x is O( x ) Doesn’t matter that you choose different c values for each part; they are independent x 2 is/is not  ( x )? –No, since x 2 is not O ( x ) x is/is not  ( x 2 )? –No, since x 2 is not  ( x )

9. CS588 Spring 20059 Measuring Work When we say a procedure or a problem is O( f(n) ) what are we counting? Need a model computer to count steps

10. CS588 Spring 200510 How should we model a Computer? Apollo Guidance Computer (1969) Colossus (1944) IBM 5100 (1975) Cray-1 (1976)

11. CS588 Spring 200511 Modeling Computers Input –Without it, we can’t describe a problem Output –Without it, we can’t get an answer Processing –Need some way of getting from the input to the output Memory –Need to keep track of what we are doing

12. CS588 Spring 200512 Modeling Input Engelbart’s mouse and keypad Punch Cards Altair BASIC Paper Tape, 1976

13. CS588 Spring 200513 Simplest Input Non-interactive: like punch cards and paper tape One-dimensional: just a single tape of values, pointer to one square on tape 0011001000 How long should the tape be? Infinitely long! We are modeling a computer, not building one. Our model should not have silly practical limitations (like a real computer does).

14. CS588 Spring 200514 Modeling Output Blinking lights are cool, but hard to model Output is what is written on the tape at the end of a computation Connection Machine CM-5, 1993

15. CS588 Spring 200515 Modeling Processing Evaluation Rules –Given an input on our tape, how do we evaluate to produce the output What do we need: –Read what is on the tape at the current square –Move the tape one square in either direction –Write into the current square 0011001000 Is that enough to model a computer?

16. CS588 Spring 200516 Modeling Processing Read, write and move is not enough We also need to keep track of what we are doing: –How do we know whether to read, write or move at each step? –How do we know when we’re done? What do we need for this?

17. CS588 Spring 200517 Finite State Machines 1 Start 2 HALT 1 0 1 # 0

18. CS588 Spring 200518 Hmmm…maybe we don’t need those infinite tapes after all? 1 Start 2 HALT ( not a paren ) # ) ERRO R What if the next input symbol is ( in state 2?

19. CS588 Spring 200519 How many states do we need? 1 Start 2 HALT ( not a paren ) # ) ERRO R 3 not a paren ( ) 4 ( ) (

20. CS588 Spring 200520 Finite State Machine There are lots of things we can’t compute with only a finite number of states Solutions: –Infinite State Machine Hard to describe and draw –Add a tape to the Finite State Machine

21. CS588 Spring 200521 FSM + Infinite Tape Start: –FSM in Start State –Input on Infinite Tape –Pointer to start of input Move: –Read one input symbol from tape –Follow transition rule from current state To next state Write symbol on tape, and move L or R one square Finish: –Transition to halt state

22. CS588 Spring 200522 Matching Parentheses Repeat until halt: –Find the leftmost ) If you don’t find one, the parentheses match, write a 1 at the tape head and halt. –Replace it with an X –Look left for the first ( If you find it, replace it with an X (they matched) If you don’t find it, the parentheses didn’t match – end write a 0 at the tape head and halt

23. CS588 Spring 200523 Matching Parentheses 1 Start HALT Input: ) Write: X Move: L ), X, L 2: look for ( X, X, R X, X, L (, X, R #, 0, # #, 1, # (, (, R Will this report the correct result for (() ?

24. CS588 Spring 200524 Matching Parentheses 1 Start HALT ), X, L 2: look for ( (, (, R (, X, R #, 0, # #, 1, # X, X, L X, X, R #, #, L 3: look for ( X, X, L #, 1, # (, 0, #

25. CS588 Spring 200525 Turing Machine Alan Turing, On computable numbers: With an application to the Entscheidungsproblem, 1936 –Turing developed the machine abstraction to show the halting problem really leads to a contradiction –Our informal argument, depended on assuming we could do if and everything else except halts?

26. CS588 Spring 200526 Describing Turing Machines zzzzzzzzzzzzzzzzzzzzzz TuringMachine ::= Alphabet ::= { Symbol* } Tape ::= OneSquare ::= Symbol | # Current ::= OneSquare LeftSide ::= [ Square* ] RightSide ::= [ Square* ] Everything to left of LeftSide is #. Everything to right of RightSide is #. 1 Start HALT ), X, L 2: look for ( #, 1, -  ), #, R  (, #, L (, X, R #, 0, - Finite State Machine

27. CS588 Spring 200527 Describing Finite State Machines TuringMachine ::= FSM ::= States ::= { StateName* } InitialState ::= StateName must be element of States HaltingStates ::= { StateName* }all must be elements of States TransitionRules ::= { TransitionRule* } TransitionRule ::= < StateName, ;; Current State OneSquare, ;; Current square StateName, ;; Next State OneSquare, ;; Write on tape Direction > ;; Move tape Direction ::= L, R, # 1 Start HALT ), X, L 2: look for (  ), #, R  (, #, L (, X, R #, 0, # #, 1, # Transition Rule is a procedure: StateName X OneSquare  StateName X OneSquare X Direction

28. CS588 Spring 200528 Example Turing Machine TuringMachine ::= FSM ::= Alphabet ::= { (, ), X } States ::= { 1, 2, HALT } InitialState ::= 1 HaltingStates ::= { HALT } TransitionRules ::= {,,,} 1 Start HALT ), X, L 2: look for (  ), #, R  (, #, L (, X, R #, 0, # #, 1, #

29. CS588 Spring 200529 Enumerating Turing Machines Now that we’ve decided how to describe Turing Machines, we can number them TM-5023582376 = balancing parens TM-57239683 = even number of 1s TM- 3523796834721038296738259873 = Photomosaic Program TM- 3672349872381692309875823987609823712347823 = WindowsXP Not the real numbers – they would be much bigger!

30. CS588 Spring 200530 Complexity Class P Class P: problems that can be solved in polynomial time by a deterministic TM. O (n k ) for some constant k. Easy problems like sorting, making a photomosaic using duplicate tiles, simulating the universe are all in P.

31. CS588 Spring 200531 Deterministic Computing All our computing models so far are deterministic: you know exactly what to do every step –TM: only one transition rule can match any machine configuration, only one way to follow that transition rule –Lambda Calculus: always  -reduce the outermost redex

32. CS588 Spring 200532 Nondeterministic Computing Allows computer to try different options at each step, and then use whichever one works best Make a Finite State Machine non- deterministic: doesn’t change computing time Make a Turing Machine non-deterministic: might change computing time No one knows whether it does or not This is the biggest open problem in Computer Science

33. CS588 Spring 200533 Making FSM’s Nondeterministic 12 3 01 0 0 1 HALT #

34. CS588 Spring 200534 Nondeterministic FSM 12 3 HALT # 0, 1 01 Input ReadPossible States 0{ 1, 2 } 00{ 1, 2 } 001{ 1, 3 } 0010{ 1, 2 } 00101{ 1, 3 } 00101#{ HALT }

35. CS588 Spring 200535 Power of NFSM Can NFSMs computing anything FSMs cannot compute? Can NFSMs compute faster than FSMs? No – you can always convert an NFSM to a FSM by making each possible set of states in the NFSM into one state in the new FSM. (Number of states may increase as 2 s ) No – both read input one letter at a time, and transition automatically to the next state. (Both are always  (n) where n is the number of symbols in the input.)

36. CS588 Spring 200536 Nondeterministic TMs Multiple transitions possible at every step Follow all possible steps: –Each possible execution maintains its own state and its own infinite tape –If any possible execution halts (in an accepting state), the NTM halts, and the tape output of that execution is the NTM output

37. CS588 Spring 200537 Complexity Classes Class P: problems that can be solved in polynomial time by a deterministic TM. O (n k ) for some constant k. Easy problems like simulating the universe are all in P. Class NP: problems that can be solved in polynomial time by a nondeterministic TM Hard problems like the pegboard puzzle sorting are in NP (as well as all problems in P).

38. CS588 Spring 200538 Problem Classes P NP Decidable Undecidable Sorting:  (n log n) Simulating Universe: O(n 3 ) Cracker Barrel: O(2 n ) and  (n) Fill tape with 2 n *s:  (2 n ) Halting Problem:  (  )

39. CS588 Spring 200539 P = NP? Is there a polynomial-time solution to the “hardest” problems in NP? No one knows the answer! The most famous unsolved problem in computer science and math Listed first on Millennium Prize Problems –win $1M if you can solve it –(also an automatic A+ in this course)

40. CS588 Spring 200540 If P  NP: P NP Decidable Undecidable Sorting:  (n log n) Simulating Universe: O(n 3 ) Cracker Barrel: O(2 n ) and  (n) Fill tape with 2 n *s:  (2 n ) Halting Problem:  (  )

41. CS588 Spring 200541 If P = NP: P NP Decidable Undecidable Sorting:  (n log n) Simulating Universe: O(n 3 ) Cracker Barrel: O(2 n ) and  (n) Fill tape with 2 n *s:  (2 n ) Halting Problem:  (  )

42. CS588 Spring 200542 Smileys Problem Input: n square tiles Output: Arrangement of the tiles in a square, where the colors and shapes match up, or “no, its impossible”.

43. Thanks to Peggy Reed for making the Smiley Puzzles!

44. CS588 Spring 200544 How much work is the Smiley’s Problem? Upper bound: (O) O ( n !) Try all possible permutations Lower bound: (  )  ( n ) Must at least look at every tile Tight bound: (  ) N o one knows!

45. CS588 Spring 200545 NP Problems Can be solved by just trying all possible answers until we find one that is right Easy to quickly check if an answer is right –Checking an answer is in P The smileys problem is in NP We can easily try n ! different answers We can quickly check if a guess is correct (check all n tiles)

46. CS588 Spring 200546 Is the Smiley’s Problem in P? No one knows! We can’t find a O( n k ) solution. We can’t prove one doesn’t exist.

47. CS588 Spring 200547 This makes a huge difference! n! 2n2n n2n2 n log n today 2032 time since “Big Bang” log-log scale Solving a large smileys problem either takes a few seconds, or more time than the universe has been in existence. But, no one knows which for sure!

48. CS588 Spring 200548 Who cares about Smiley puzzles? If we had a fast (polynomial time) procedure to solve the smiley puzzle, we would also have a fast procedure to solve the 3/stone/apple/tower puzzle: 3

49. CS588 Spring 200549 3SAT  Smiley     Step 1: Transform into smileys Step 2: Solve (using our fast smiley puzzle solving procedure) Step 3: Invert transform (back into 3SAT problem

50. CS588 Spring 200550 The Real 3SAT Problem (also can be quickly transformed into the Smileys Puzzle)

51. CS588 Spring 200551 Propositional Grammar Sentence ::= Clause Sentence Rule: Evaluates to value of Clause Clause ::= Clause 1  Clause 2 Or Rule: Evaluates to true if either clause is true Clause ::= Clause 1  Clause 2 And Rule: Evaluates to true iff both clauses are true

52. CS588 Spring 200552 Propositional Grammar Clause ::=  Clause Not Rule: Evaluates to the opposite value of clause (  true  false) Clause ::= ( Clause ) Group Rule: Evaluates to value of clause. Clause ::= Name Name Rule: Evaluates to value associated with Name.

53. CS588 Spring 200553 Proposition Example Sentence ::= Clause Clause ::= Clause 1  Clause 2 (or) Clause ::= Clause 1  Clause 2 (and) Clause ::=  Clause (not) Clause ::= ( Clause ) Clause ::= Name a  (b  c)   b  c

54. CS588 Spring 200554 The Satisfiability Problem (SAT) Input: a sentence in propositional grammar Output: Either a mapping from names to values that satisfies the input sentence or no way (meaning there is no possible assignment that satisfies the input sentence)

55. CS588 Spring 200555 SAT Example SAT (a  (b  c)   b  c )  { a: true, b: false, c: true }  { a: true, b: true, c: false } SAT (a   a )  no way Sentence ::= Clause Clause ::= Clause 1  Clause 2 (or) Clause ::= Clause 1  Clause 2 (and) Clause ::=  Clause (not) Clause ::= ( Clause ) Clause ::= Name

56. CS588 Spring 200556 The 3SAT Problem Input: a sentence in propositional grammar, where each clause is a disjunction of 3 names which may be negated. Output: Either a mapping from names to values that satisfies the input sentence or no way (meaning there is no possible assignment that satisfies the input sentence)

57. CS588 Spring 200557 3SAT / SAT Is 3SAT easier or harder than SAT? It is definitely not harder than SAT, since all 3SAT problems are also SAT problems. Some SAT problems are not 3SAT problems.

58. CS588 Spring 200558 3SAT Example 3SAT ( (a  b   c)  (  a   b  d)  (  a  b   d)  (b   c  d ) )  { a: true, b: false, c: false, d: false } Sentence ::= Clause Clause ::= Clause 1  Clause 2 (or) Clause ::= Clause 1  Clause 2 (and) Clause ::=  Clause (not) Clause ::= ( Clause ) Clause ::= Name

59. CS588 Spring 200559 3SAT  Smiley Like 3/stone/apple/tower puzzle, we can convert every 3SAT problem into a Smiley Puzzle problem! Transformation is more complicated, but still polynomial time. So, if we have a fast (P) solution to Smiley Puzzle, we have a fast solution to 3SAT also!

60. CS588 Spring 200560 NP Complete Cook and Levin proved that 3SAT was NP- Complete (1971) A problem is NP-complete if it is as hard as the hardest problem in NP If 3SAT can be transformed into a different problem in polynomial time, than that problem must also be NP-complete. Either all NP-complete problems are tractable (in P) or none of them are!

61. CS588 Spring 200561 NP-Complete Problems Easy way to solve by trying all possible guesses If given the “yes” answer, quick (in P) way to check if it is right –Solution to puzzle (see if it looks right) –Assignments of values to names (evaluate logical proposition in linear time) If given the “no” answer, no quick way to check if it is right –No solution (can’t tell there isn’t one) –No way (can’t tell there isn’t one)

62. CS588 Spring 200562 Traveling Salesperson Problem –Input: a graph of cities and roads with distance connecting them and a minimum total distant –Output: either a path that visits each with a cost less than the minimum, or “no”. If given a path, easy to check if it visits every city with less than minimum distance traveled

63. CS588 Spring 200563 Graph Coloring Problem –Input: a graph of nodes with edges connecting them and a minimum number of colors –Output: either a coloring of the nodes such that no connected nodes have the same color, or “no”. If given a coloring, easy to check if it no connected nodes have the same color, and the number of colors used.

64. CS588 Spring 200564 Pegboard Problem - Input: a configuration of n pegs on a cracker barrel style pegboard - Output: if there is a sequence of jumps that leaves a single peg, output that sequence of jumps. Otherwise, output false. If given the sequence of jumps, easy (O( n )) to check it is correct. If not, hard to know if there is a solution. Proof that variant of this problem is NP-Complete is attached to today’s notes.

65. CS588 Spring 200565 Minesweeper Consistency Problem –Input: a position of n squares in the game Minesweeper –Output: either a assignment of bombs to squares, or “no”. If given a bomb assignment, easy to check if it is consistent.

66. CS588 Spring 200566 Drug Discovery Problem –Input: a set of proteins, a desired 3D shape –Output: a sequence of proteins that produces the shape (or impossible) Note: US Drug sales = $200B/year If given a sequence, easy (not really) to check if sequence has the right shape. Caffeine

67. CS588 Spring 200567 Complexity and Cryptography Next class