On the Complexity of Search Problems George Pierrakos Mostly based on: On the Complexity of the Parity Argument and Other Insufficient Proofs of Existence.

On the Complexity of Search Problems George Pierrakos Mostly based on: On the Complexity of the Parity Argument and Other Insufficient Proofs of Existence [Pap94] On total functions, existence theorems and computational complexity [MP91] How easy is local search? [JPY88] Computational Complexity [Pap92] The complexity of computing a Nash equilibrium [DGP06] The complexity of pure Nash equilibria [FPT04] slides and scribe notes from many people…

Outline 1. Generally on Search Problems 2. The Class TFNP 3. Subclasses of TFNP part I: PPA, PPAD Problems in PPA, PPAD Completeness in PPAD 4. Subclasses of TFNP part II: PPP, PLS 5. PPAD-completeness of NASH & the complexity of computing equilibria in congestion games

Decision Problems vs Search (or “function”) Problems SAT Input: boolean CNF-formula φ Output: “yes” or “no” FSAT Input: boolean CNF-formula φ Output: satisfying assignment or “no” if none exist

Are search problems harder? They are definitely not easier: a poly-time algorithm for FSAT can be easily tweaked to give a poly-time algorithm for SAT …and vice versa, FSAT “reduces” to SAT: we can figure out a satisfying assignment by running poly-time algorithm for SAT n-times

Are search problems harder? SAT is self-reducible but TSP is not… Here is how we solve FTSP using TSP: Use binary search to figure out the cost C of the optimum tour, using TSP algorithm Run TSP algorithm n 2 times, always with cost parameter C, each time setting the weight of some edge to C+1

The Classes FP and FNP L € NP iff there exists poly-time computable R L (x,y) s.t. X € L  y { |y| ≤ p(|x|) & R L (x,y) } Note how R L defines the problem-language L The corresponding search problem Π R(L) € FNP is: given an x find any y s.t. R L (x,y) and reply “no” if none exist FSAT € FNP… what about FTSP? Are all FNP problems self-reducible like FSAT? [open?] FP is the subclass of FNP where we only consider problems for which a poly-time algorithm is known

Reductions and completeness A function problem Π R reduces to a function problem Π S if there exist log-space computable string functions f and g, s.t. R(x,g(y))  S(f(x),y) intuitively f reduces problem Π R to Π S and g transforms a solution of Π S to one of Π R Standard notion of completeness works fine…

FP FNP A proof a-la-Cook shows that FSAT is FNP-complete Hence, if FSAT € FP then FNP = FP But we showed self-reducibility for SAT, so the theorem follows: Theorem: FP = FNP iff P=NP So, why care for function problems anyway??

On total “functions”: the class TFNP What happens if the relation R is total? i.e., for each x there is at least one y s.t. R(x,y) Define TFNP to be the subclass of FNP where the relation R is total TFNP contains problems that always have a solution, e.g. factoring, fix-point theorems, graph-theoretic problems, … How do we know a solution exists? By an “inefficient proof of existence”, i.e. non-(efficiently)- constructive proof The idea is to categorize the problems in TFNP based on the type of inefficient argument that guarantees their solution

Basic stuff about TFNP 1. FP TFNP FNP 2. TFNP = F(NP coNP) NP = problems with “yes” certificate y s.t. R 1 (x,y) coNP = problems with “no” certificate z s.t. R 2 (x,y) for TFNP F(NP coNP) take R = R 1 U R 2 for F(NP coNP) TFNP take R 1 = R and R 2 = ø 3. There is an FNP-complete problem in TFNP iff NP = coNP  : If NP = coNP then trivially FNP = TFNP  : If the FNP-complete problem Π R is in TFNP then: FSAT reduces to Π R via f and g, hence any unsatisfiable formula φ has a “no” certificate y, s.t. R(f(φ),y) (y exists since Π R is in TFNP) and g(y)=“no” 4. TFNP is a semantic complexity class  no complete problems! note how telling whether a relation is total is undecidable (and not even RE!!)

Subclasses of TFNP: PPA, PPAD PPA stands for polynomial parity argument: Every graph has an even number of odd-degree nodes Every graph of degree two or less has an even number of leaves Both arguments turn out to be equally generic PPAD stands for polynomial parity argument directed Parity argument in directed graphs, where we now seek for a sink or a source

PPA ANOTHER HAMILTON CYCLE Thm: any graph with odd degrees has an even number of HC through edge xy Proof Idea: take a HC remove edge (1,2) & take a HP fix endpoint 1 and start “rotating” from the other end each HP has two “valid” neighbors (d=3) except for those paths with endpoints 1,2 Evoke the parity argument: at least two (different) such HPs must exist: add edge (1,2) et voila! time: worst case exponential…

PPAD 2D-SPERNER Given a triangulation of a triangle (castle) s.t. each original vertex obtains its own color no vertex on edge ij contains color 3-i-j there exists a trichromatic triangle Proof Idea: add external 0-1 edges traverse triangles (“rooms”) using 0- 1 edges (“doors”) If a “room” has no “door” out then it is trichromatic Note that the process cannot exit the castle and cannot fold upon itself our tour leaves 0 nodes to the right, hence direction, hence PPAD…

Definition of PPA Let A be a problem and M the (associated) poly-time TM Let x be an input of A Let C x = Σ p(|x|) be the configuration space of x, i.e. the set of all strings of length at most p(|x|) On input c € C x machine M outputs in time O(p(n)) a set of at most two configurations M(x,c) M(x,c) may well be empty if c is “rubbish” c and c’ are neighbors ([c,c’] € G(x)) iff c € M(x,c) and c’ € M(x,c’) G(x) is symmetric with degree at most 2 It is the implicit search graph of the problem Let M(x, 0…0) = {1…1} and 0…0 € M(x, 1…1), hence 0…0 is the standard leaf PPA is the class of problems defined as follows: “Given x, find a leaf of G(x) other than the standard leaf 0…0”

Definition of PPA (an example) Take A to be ANOTHER HAMILTON CYCLE. We show that A € PPA x is the input of A: a graph and a Hamilton Cycle C x is the configuration space of x: C x contains encodings of “valid” Hamilton Paths (poly-space) and other irrelevant stuff On input c € C x machine M outputs in time O(p(n)) a set M(x,c) of: the at most two neighbors of a valid Hamilton Path, or possibly nothing c and c’ are neighbors ([c,c’] € G(x)) iff c € M(x,c) and c’ € M(x,c’) remember the graph of neighboring “valid” Hamilton Paths the standard leaf is the initial Hamilton Path (from removal of (1,2))

Definition of PPAD (an example) Take A to be 2D-SPERNER. We show that A € PPAD x is the input of A: a triangle, a triangulation and a valid color assignment CAUTION: restricted input  “promise” problems: we usually need some sort of syntactic guarantee of the validity of the input C x is the configuration space of x: C x contains encodings of all sub-triangles in the triangulation M(x,c) = In(x,c) U Out (x,c) where: In(x,c) is the castle room where we came from Out(x,c) is the castle room where we are heading to c and c’ are neighbors ([c,c’] € G(x)) iff c € M(x,c) and c’ € M(x,c’) remember our tour through the triangular castle the standard leaf is the node lying outside the triangle

Couple of Theorems PPA as defined above makes use of the “even leaves” argument Why not modify the definition as follows (PPA’): …on input c € C x machine M outputs in time O(p(n)) a set M(x,c) s.t. |M(x,c)| = poly(n)… …“Given x, find an odd-degree node of G(x) other than the standard leaf 0…0”… and make use of the (more general?) parity argument? Quite surprisingly: PPA’ = PPA

Theorem 1: PPA’ = PPA (The Chessplayer Algorithm) Obviously PPA PPA’ For the inverse: For each node c of degree k ≥ 1, we create new nodes (c,i), where i = 0,…,k div 2 each node (c,i) is connected to the two nodes (c’,j) s.t. the lexicographically 2i, 2i+1 edges out of c are the 2j, 2j+1 edges into c’ nonstandard leaves of the new graph correspond to nonstandard odd-degree nodes of the former

Two more propositions Proposition 1: FP PPAD PPA FNP Proposition 2: Consider the more natural definition of PPA, where the leaf we search for is the one connected to the standard leaf (PPA’’) Then PPA’’ = PPAD’’ = FPSPACE just show FPSPACE PPA’’ any problem in FPSPACE can be solved by a reversible TM, i.e. one where each configuration has at most one predecessor and one successor [Be84] define M(x,c) to return the (at most) two configurations that are the predecessor and successor of c on input x hence, the nonstandard leaf on the same component as the standard leaf is the halting configuration with the desired output

PPAD BROUWER FIX POINT Brouwer’s fix-point theorem: let S be any n-dimensional simplex and let φ: S  S be a continuous function. Then φ has a fix point, i.e. an x* s.t. φ(x*) = x* Proof for the special case of a triangle, using Sperner lemma Let T 1, …, T i,… be a sequence of finer triangulations Define labels L 1, …, L i,… for the triangulations as depicted: L i is a valid labeling for T i Consider the sequence of centers c 1, …, c i,… of the resulting trichromatic triangles t 1, …, t i,…(guaranteed by Sperner’s Lemma) This is an infinite sequence in a closed convex space, hence it has a subsequence converging at x*: a point which is arbitrarily closed to the centroid of a trichromatic triangle x* is a fix point of φ: if not, then we can find a small triangle containing x* but not φ(x*). Because of the continuity of φ, we can show that this cannot be labeled with 3 colors Also need to take care of continuity for representation in TM

PPAD NASH & LINEAR COMPL/RITY Theorem[Nash51]: every game has a mixed Nash equilibrium Original proof of Nash uses Kakutani’s fix point theorem; KAKUTANI is in PPAD by a reduction from BROWER Nash’s theorem can be proved using Brower’s fix-point theorem too In [Pap94] they show that NASH is in PPAD by a reduction from LINEAR COMPLENTARITY

PPAD NASH & LINEAR COMPL/RITY LINEAR COMPLEMENTARITY We are given an nxn matrix A and an n-vector b We are seeking vectors x,y ≥ 0 s.t. Ax + b = y x·y = 0, i.e. at least one of x i, y i is 0 Solved by Lemke’s algorithm: some times it does not even converge if it converges it might well be (worst case) exponential In general, the problem is NP-complete (?) For the case of P-matrices A (matrices satisfying det(A S ) > 0 for all non- empty subsets S of {1,…,n}) the problem has a unique solution and Lemke’s algorithm converges to it. Is checking for the P-property for general matrices in P? Open… PPAD formulation of L-COMPL: Given A and b, find a solution x, y to the linear complementarity problem or a subset S s.t. det(A S ) ≤ 0

PPAD NASH & LINEAR COMPL/RITY Example: from bimatrix games to a linear complementarity problem Let are the probability and payoff of player 1 for row i are the probability and payoff of player 2 for column j is the payoff of player 1 is the payoff of player 2

PPAD NASH & LINEAR COMPL/RITY Approach 1: implies that the player has equal expected payoff from both his choices (else he would pick the one maximizing his profit with probability 1 and drop the other) Hence we can write down the following linear system: To compute all equilibria we need to solve a linear system (in poly-time) for all different supports of players 1 and 2 in time 2 n 2 m P(n,m) (support = subset of strategies played with p>0)

PPAD NASH & LINEAR COMPL/RITY Approach 2: reduce to L-COMPL

PPAD NASH & LINEAR COMPL/RITY Use Lemke’s algorithm: “double pivoting”

PPA ANOTHER HAMILTON PATH Given a graph G and a HP in it, find either another HP in G or in its complement G c Construct a bipartite graph, whose odd-degree nodes are HP of G and G c “left” nodes: all HP of the complete graph on the nodes of G “right” nodes: all subgraphs of G c -including the empty subgraph Edge from h (hpath) on left side to s (subgraph) on the right side iff s is a proper subgraph of h Claim: the only odd-degree nodes are the HP of G and G c on the left side If node on the left is not a HP of G or G c then it contains a non-empty subset of edges of G c and will be adjacent to all proper subsets of this set: these are 2 k A HP of G on the left will only be adjacent to the empty subgraph A HP of G c on the left will be adjacent to its 2 n-1 -1 proper subgraphs (excluding itself) and will also have odd degree Any subgraph (node on the right) will be connected to all its possible connections to a HP (usually 0): for graphs with more than 5 nodes this number is even (can be proved via an easy calculation…)

Other problems PPA: HAMILTON DECOMPOSITION TSP NONADJACENCY CHEVALLEY MOD 2 TOTALLY EVEN SUBGRAPH CUBIC SUBGRAPH PPAD: k-D SPERNER KAKUTANI COMPETITIVE & EXCHANGE EQUILIBRIUM TUCKER BORSUK-ULAM, DISCRETE HAM SANDWICH, NECKLAGE SPLITTING…

The Class PPP PPP stands for Polynomial Pigeonhole Principle Definition: A problem A is in PPP if it is defined in terms of a poly- time TM M Given input x define the set of “solutions” S(x) = Σ p(|x|) and For any y € S(x), M(x,y) € S(x) Desired output: y € S(x), s.t. M(x,y) = 0…0 or y, y’ € S(x) s.t. M(x,y) = M(x,y’) One of the two is guaranteed to exist by pigeonhole principle

A PPP-complete problem: PIGEONHOLE CIRCUIT Input: Boolean circuit C with n inputs and n output Output: either find an input y s.t. C(y) = 0 n or find two inputs y, y’ s.t. C(y) = C(y’) Obviously PIGEONHOLE CIRCUIT € PPP: M is the poly-time TM that simulates circuit C on input y Input x is the description of the circuit C S(x) = {0,1} n and for any y € S(x), M(x,y) € S(x) And any problem in PPP can be evaluated by such a circuit: just tweak the proof of CIRCUIT-VALUE P-completeness

PIGEONHOLE CIRCUIT for MONOTONE circuits? Monotone circuits are the circuits that do not contain “no” gates Property of monotone circuits: swapping x i from false to true cannot change any output bit from true to false Proof: trivial by induction on the depth of the tree MONOTONE PIGEONHOLE CIRCUIT € FP Proof: We need to consider only the input sequences of the form 0 i 1 n-i, i=0,…,n, i.e. only n input sequences Start with 1 n and start swapping bits to 0. The only way to have n distinct outputs is if: output starts at 0 n and each time one output bit swaps to 1 Hence, either output = 0 n for some input, or two inputs have same output

PPP EQUAL SUMS Input: n positive integers with sum less than 2 n -1 Output: two subsets with the same sum EQUAL SUMS in PPP: Two such subsets trivially exist since there are more subsets than possible sums The formal definitions utilizes a TM M that given the input x (set of all sets) and the description of a set y, outputs in polynomial time the sum of the set’s elements. Since the sum of all elements is less than 2 n -1 we cannot have M(x,y) = 0…0 Is EQUAL SUMS PPP-complete?

Two more propositions One-way permutation π is a poly-time algorithm mapping Σ n to itself, such that for all x≠y, π(x) ≠ π(y), s.t. it is hard to recover x from π(x). Prop1: If PPP=FP then one-way permutations do not exist Proof: suppose PIGEONHOLE CIRCUIT is in FP We wish to invert an alleged one-way permutation π, i.e. given y, find x s.t. y=π(x) Turn the algorithm for π into a circuit (CIRCUIT VAL is P-complete) and add bitwise XOR gates between output y and x; call this circuit C Apply the poly-time algorithm for PIGEONHOLE CIRCUIT to C. Since π is a permutation, the algorithm cannot return two inputs giving the same output, hence it returns y, s.t. C(y) = 0, i.e. π(x) = y

Two more propositions Prop2: PPAD PPP Proof: Reformulate any problem in PPAD with graph G as a problem in PPP by use of π(x) defined as  π(x) = x if x is a sink  π(x) = G 2 (x) if x is a nonstandard source  π(x) = G(x) otherwise Trivially, from any confluence of values of π we can recover a nonstandard source or a sink of G

The class PLS PLS originally defined in [Johnson, et. al. ‘88]: “find some local optimum in a reasonable search space”. Basic ingredient: a problem with a search space, i.e. a set of feasible solutions which has a neighborhood structure Every problem L (e.g. TSP) in PLS has a set of instances D L (e.g. encodings of adjacency matrix) Every instance x in D L has a finite set of solutions F L (x) (e.g. tours) considered wlog polynomial in size For each solution s in F L (x) we have A poly-time cost function c(x,s) that given an instance x and a solution s, it computes its cost (e.g. length of a tour) A poly-time neighbor function N(x,s) F L (x) that given an instance x and a solution s, returns its neighboring solutions (e.g. a tour that differs from the original one in at most two edges)

The class PLS Remaining constraints are expressed in terms of 3 polynomial time algorithms A L, B L, C L A L, given an instance x in D L produces a particular standard solution of L, A L (x) € F L (x) (e.g. a standard tour for TSP) B L, given an instance x in D L and a string s determines whether s is a solution (s € F L (x)?) and if so, it computes its cost c(x,s) C L, given an instance x € D L and a string s € F L (x) has two possible outcomes, depending on s: If there is any solution s’ € N(x,s) s.t. c(x,s’) < c(x,s) it outputs s’ Otherwise it outputs “no”, which implies that s is locally optimal

The class PLS The sense of local searching is obvious: starting at some arbitrary solution start searching locally for a better solution and move towards it. Standard local search algorithm: Given x, use A L to produce a starting solution s= A L (x) Repeat until locally optimal solution is found: Apply algorithm C L to x and s If C L yields a better cost neighbor s’ of then set s=s’ Since the solution space is finite the above algorithm always halts If F L (x) is polynomially bounded then the above algorithm is polynomial However, things can turn out pretty bad: Finding a local optimum reachable from a specific state can be proven PSPACE- complete There are instances with states exponentially far from any local optimum Correct question to ask: “Given x, find some locally optimum solution s”

The class PLS PLS-reduction: we must map the local optima of one problem to those of another, thus pairtaining the neighboring structure. Hence problem L reduces to K if there exist poly-time functions f and g s.t. f maps instances x of L to instances f(x) of K g maps (solutions of f(x), x) pairs to solutions of x for all instances x of L, if s is a local optimum for instance f(x) of K, then g(s,x) is a local optimum for x Basic PLS-complete problem: weighted CIRCUIT – SAT under input bitflips some relatives of TSP, MAXCUT and: POS-NAE-3SAT: [Shaeffer&Yannakakis-91] Given an instance of NAE-3SAT with weights on its clauses and with positive literals only, find a truth assignment, whose total weight (i.e. the weight of the unsatisfied and totally satisfied clauses) cannot be improved by flipping a variable

Congestion Games Definitions Idea: generalization of Selfish Task Allocation Intuition: Route traffic through a network Resource allocation We have a set of resources E (network edges…?) a set of players N the resource delays d e For each player i an action set A i 2 E a weight (traffic demand) w i

Congestion Games Definitions let A=(a 1,…, a n ), a i € A i, be a strategy profile denote by P e (A) the set of players picking resource e in profile A n e (A) the number of players picking resource e in profile A, n e (A)=|P e (A)| define the load of resource e to be L(e) = Σ i in P(e) w i for each strategy profile A, each player incurs a total delay/cost c i (A) = Σ e in ai L(e)

Congestion Games Equilibria and Potential Functions A Pure Nash Equilibrium (PNE) is a strategy profile A=(a 1,…, a n ), s.t. for all i € N and for all a i ’ € A i c(a 1,…, a i,…, a n ) ≤ c(a 1,…, a i ’,…, a n ) Notice the local optimality flavor of a PNE Exact Potential for a c.g. is a function Φ: Ε→R s.t. for all i € N, a i ’ € A i and A c i (A) - c i (A -i, a i ’) = Φ(A) - Φ(A -i, a i ’) It is easy to verify that every game that admits an exact potential possesses a PNE

Types of Congestion Games Symmetric vs Asymmetric Players have the same action set and the same weights, i.e. they are indistinguishable Network vs General Congestion Games the resources are graph edges each player has a source and a sink vertex (s i,t i ) the action sets A i are s i -t i paths Single vs multi-commodity (network) games the (s i,t i ) pairs are common for all players

Congestion Games Existence of PNE Theorem [Rosenthal ’73]: Every unweighted congestion game is an exact potential game The inverse holds [Mondere&Shapley]: Every finite exact potential game is isomorphic to an unweighted c.g. So it is only for c.g. that we can find an exact potential Does this mean that the potential technique (for guaranteeing existence of a local minimum) can be used only for c.g.?

Congestion Games Potential Functions We need to be less strict about the potential function. Consider the party affiliation game (HAPPYNET): n players (nodes in graph) Symmetric preference relations between players-nodes 2 actions per player: {-1,1} State S is a mapping V  {-1,1} (players pick sides) payoff of i: sgn[S(i) Σ j S(j)w ij ] (if the payoff is positive then we say “node i is happy”) The problem: “Find a state in which all nodes are happy” intuition: everyone wants to be with his friends application: convergence of Hopfield NN

Congestion Games Potential Functions HAPPYNET always admits a solution Proof: by use of a (general) potential function φ[S] = Σ i,j S(i)S(j) w ij Suppose that node i is unhappy, i.e. S(i) Σ j S(j)w ij = - δ < 0 Consider the state S’ where S is flipped, i.e. S’(i) = S(i) Obviously φ[S’] = φ[S] +2δ, i.e. the potential function is increased by at least two when we flip any unhappy node Since φ takes values in [-W, W] where W is the sum of all weights, and it is increased by two in each iteration, this process has to end at a state with no unhappy nodes, i.e. at a solution How many steps? Polynomial in W, exponential in |W| Connection with congestion games: The party affiliation game is NO congestion game therefore it admits NO exact potential; it admits a general potential s.t. sgn( c i (A) - c i (A’) ) = sgn( Φ(A) – Φ(A’) ) Using this we proved that the problem always has a solution, i.e. the underlying game always admits a PNE, which can be derived from the local optima of s i Σ j s j w ij.

Congestion Games Complexity of PNE Main results here from [Fabrikant/ Papadimitriou/ Talwar STOC 2004] They proved that the problem of finding a PNE of unweighted games: 1. is in P for symmetric network c.g. [irrelevant] 2. is PLS-complete for 1. General asymmetric c.g. 2. General symmetric c.g. 3. Asymmetric network c.g. [too damn hard]

Congestion Games General Asymmetric c.g. POS-NAE-3SAT ≤ PLS General Asymmetric c.g. variable x → player x clause c → resources e c, e c ’ A x → {{e c |x in e c }, {e c ’|x in e c ’}} x plays e c if x=0 in POS-NAE-3SAT x plays e c ’ if x=1 in POS-NAE-3SAT d e (1)=d e (2)=0, d e (3)=w(c) Every PNE of the c.g. corresponds to a local minimum of POS-NAE-3SAT, since bitflip corresponds to single- player deviation

Congestion Games General Symmetric c.g. General Asymmetric c.g. ≤ PLS General Symmetric c.g. Augment the network by adding n additional resources e 1,…, e n d ei (k)=0 if k=1 and d ei (k)=M otherwise (M suff. large number) A = U x { sU{e x } | s in A x } same number of players so any solution that uses a e x twice has an unused e x, so it is no PNE at PNE players arbitrarily take on the “roles” of the players in the general asymmetric game

(one) Hierarchy of search complexity classes FNP TFNP PLSPPPPPA PPAD FP

On the Complexity of Search Problems George Pierrakos Mostly based on: On the Complexity of the Parity Argument and Other Insufficient Proofs of Existence.

Similar presentations

Presentation on theme: "On the Complexity of Search Problems George Pierrakos Mostly based on: On the Complexity of the Parity Argument and Other Insufficient Proofs of Existence."— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

On the Complexity of Search Problems George Pierrakos Mostly based on: On the Complexity of the Parity Argument and Other Insufficient Proofs of Existence.

Similar presentations

Presentation on theme: "On the Complexity of Search Problems George Pierrakos Mostly based on: On the Complexity of the Parity Argument and Other Insufficient Proofs of Existence."— Presentation transcript:

Similar presentations

About project

Feedback