1. 1 Understanding Problem Hardness: Recent Developments and Directions Bart Selman Cornell University

2. 2 Introduction & Motivation Computational Challenges in Planning, Reasoning, Learning, and Adaptation. What are the characteristics of challenging computational problems?

3. A Few Examples Reasoning many forms of deduction abduction / diagnosis (e.g. de Kleer 1989) default reasoning (e.g. Kautz and Selman 1989) Bayesian inference (e.g. Dagum and Luby 1993) Planning domain-dependent and independent (STRIPS) (e.g. Chapman 1987; Gupta and Nau 1991; Bylander1994) Learning neural net “loading” problem (e.g. Blum and Rivest 1989) Bayesian net learning decision tree learning

4. An abundance of negative complexity results for many interesting tasks. Results often apply to very restricted formalisms, and also to finding approximate solutions. But worst-case, what about average-case? Sometimes “surprising” results. A closer look leads to new insights & algorithms and solution strategies.

5. 5 Outline A --- “Early’’ results: phase transitions & computational hardness B --- Current focus: --- problem mixtures (tractable / intractable) --- adding global structure C --- Future directions and prospects --- modeling resource constraints --- adaptive computing --- deeper theoretical understanding

6. 6 A. “Early” Results (‘90-’95)

7. Example Domain: Satisfiability SAT: Given a formula in propositional calculus, is there an assignment to its variables making it true? We consider clausal form, e.g.:  a b c  b d  b c e  The canonical NP-complete problem. (“exponential search space”) 

9. Generating Hard Random Formulas Key: Use fixed-clause-length model. (Mitchell, Selman, and Levesque 1992; Kirkpatrick and Selman 1994) Critical parameter: ratio of the number of clauses to the number of variables. Hardest 3SAT problems at ratio = 4.25

11. Intuition At low ratios: few clauses (constraints) many assignments easily found At high ratios: many clauses inconsistencies easily detected

13. 13 Phase transition 2-, 3-, 4-, 5-, and 6-SAT

14. Theoretical Status Of Threshold Very challenging problem... Current status: 3SAT threshold lies between 3.003 and 4.6. (Motwani et al. 1994; Broder et al. 1992; Frieze and Suen 1996; Dubois 1990, 1997; Kirousis et al. 1995; Friedgut 1997; Archlioptas et al. 1999 / related work: Beame, Karp, Pitassi, and Saks 1998; Bollobas, Borgs, Chayes, Han Kim, and Wilson 1999)

15. Phase transition and combinatorial problems is an active research area with fruitful interactions between computer science, physics (approaches from statistical mechanics), and mathematics (combinatorics / random structures). Also, a close interaction between experimental and theoretical work. (With experimental findings quite often confirmed by formal analysis within months to a few years.) Finally, relevance to applications via algorithmic advances and notion of “critically constrained problems”.

16. Consequences for Algorithm Design Phase transition work instances led to improvements in algorithms: --- local search methods (e.g., GSAT / Walksat) (Selman et al. 1992; 1996; Min Li 1996; Hoos 1998, etc.) --- backtrack-style methods (Davis-Putnam and variants / complete) (Crawford 1993; Dubois 1994; Bayardo 1997; Zane 1998, etc.)

17. 17 Progress Propositional reasoning and search (SAT): 1990: 100 variables / 200 clauses (constraints) 1998: 10,000 - 100,000 variables / 10^6 clauses Novel applications: e.g. in planning (Kautz & Selman), program debugging (Jackson), protocol verification (Clarke), and machine learning (Resende).

18. B. Current Focus --- mixtures of problem classes, e.g., 2-SAT and 3-SAT (“moving between P and NP”) the 2+p-SAT model --- structured instances perturbed quasi-group completion problems

19. Focus --- 1) mixtures: 2+p-SAT problem mixture of binary and ternary clauses p = fraction ternary p = 0.0 --- 2-SAT / p = 1.0 --- 3-SAT What happens in-between? ( Monasson, Zecchina, Kirkpatrick, Selman, and Troyansky, Nature, to appear)

20. 20 Phase Transition for 2+p-SAT

21. 21 Location Threshold

22. 22 Computational Cost

23. 23 Results for 2+p-SAT p < ~ 0.41 --- model essentially behaves as 2-SAT search proc. “sees” only binary constraints smooth, continuous phase transition p > ~ 0.41 --- behaves as 3-SAT (exponential scaling) abrupt, discontinuous scaling Many new, rigorous results (including scaling) by Achlioptas, Bollobas, Borgs, Chayes, Han Kim, and Wilson. (Next talk.)

24. Consequences for Algorithm Design 1) Strategies that exploit tractable substructure with propagation are most effective. (consistent with the best empirically discovered methods) 2) In addition, use early branching on critically constrained variables. (the “backbone variables” / suggests use of clustering and statistical learning methods) (Boyan and Moore 1998)

25. 25 Proposal: study the influence of global structure on problem hardness. structure on problem hardness. Focus --- 2) Structure (Gomes and Selman 1997; 1998)

26. 26 Defn.: a pair (Q, *) where Q is a set, and * is a binary operation on Q such that a * x = b ; y * a = b are uniquely solvable for every pair of elements a,b in Q. The multiplication table of its binary operation defines a latin square (i.e., each element of Q appears exactly once in each row/column). Example: Quasigroup of order 4 Quasigroups

27. 27 Given a partial latin square, can it be completed? Example: Quasigroup Completion Problem (QCP)

28. 28 Quasigroup Completion Problem A Framework for Studying Search NP-Complete (Colbourn 1983, 1984; Anderson 1985). Has a regular global structure not found in random instances. Leads to interesting search problems when structure is perturbed. similar to e.g. structure found in the channel assignment problem for cellular networks

29. 29 Computational Cost

30. 30

31. Consequences for Algorithm Design On these structured problems, backtrack search methods show so-called heavy-tailed probability distributions. (Gomes, Selman & Crato 1997, 1998). Both very short and very long runs occur much more frequent than one would expect.

32. 32 Standard Distribution

33. 33 Heavy Tailed Cost Distribution

34. 34 Fringe of Search Tree

35. Algorithmic Strategy: Rapid Random Restarts. Order of magnitude speedup. (Gomes et al. 1998; 1999) Related:. Algorithm portfolios (Huberman 1998; Gomes 1998). Universal strategies (Ertel and Luby 1993; Alt et al. 1996)

36. 36 Rapid Restarts --- Planning

37. 37 Portfolio for heavy-tailed search procedures (2-20 processors)

38. 38 C. Future directions and prospects Modeling resource constraints & user requirements / utility should be possible to identify optimal restart strategies, possibly adaptive --- may need way of “measuring progress” (Horvitz and Klein 1995; Gomes and Selman 1999)

39. 39 Adaptive Computing combine statistical learning methods with combinatorial search techniques. first success: STAGE system for local search. (Boyan and Moore 1998) extension: train a planner on small instances (Selman, Kautz, Huang 1999) Deeper theoretical understanding with continued interactions with experiments and applications

40. Summary During the past few years, we have obtained a much better understanding of the nature of computationally hard problems. Rich interactions between physics, computer science and mathematics, and between theory, experiments, and applications. Clear algorithmic progress with room for future improvements (possibly another level of scaling: 10^6 Boolean variables, 10^8 constraints. Further applications.)