1. Non-binary constraints Toby Walsh Dept of CS University of York England

2. Non-binary constraints Bibliography Bacchus & van Beek, AAAI-98 Chen, PhD thesis, UAlberta, 2000 Dechter, AAAI-90 Mohr & Masini, ECAI-88 Regin, AAAI-94 Stergiou & Walsh, AAAI-99 & IJCAI-99 ….

3. Non-binary constraints Major contribution to competitiveness of CP –despite academic focus on binary CSPs! Some classic examples: all-different, cardinality,...

4. Non-binary constraints Compact & declarative problem specifications –that often run with minimal “tweaking” Efficient algorithms for enforcing high levels of consistency –Regin’s specialized algorithm for a k-ary all-different constraint –O(k^2d^2) compared compared to O(d^k) for a generic algorithm

5. Non-binary constraints Comparison with binary constraints: –binary models without additional vars (binary decompositions) –binary models with additional hidden vars (binary encodings) Theoretical and empirical results

6. Binary decompositions Certain non-binary constraints decompose into binary constraints on same vars Sometimes called “network decomposable”

7. Binary decompositions Two examples: –all-different(x1,x2,x3) = x1<> x2, x1<>x3, x2<>x3 –monotone(x1,x2,x3) = x1 < x2 < x3 One non-example: – even(x1+x2+x3)

8. Binary decompositions Theoretical comparison direct –compare pruning of vars in binary decomposition with that in non-binary Empirical experiments reinforce theory –decomposing non-binary constraints can add orders of magnitude to solution cost

9. Binary consistencies Arc-consistency (AC) –any assignment can be extended to a second Path-consistency (PC) –any pair of assignments can be extended to a third Strong path-consistency (strong PC) –AC and PC

10. Binary consistencies Path-consistency rarely used in practice –time complexity –adds new (binary) constraints Path-consistency can be achieved with a (generalized) arc-consistency algorithm –see later

11. Binary consistencies Between AC and PC –inverse and singleton consistencies –just prune values (no additional binary constraints) Experiments suggest can be useful –good pruning/time

12. Binary consistencies Path inverse consistency (PIC) –any assignment can be extended to two more vars Neighbourhood inverse consistency (NIC) –any assignment can be extended to all vars in immediate neighbourhood

13. Binary consistencies Singleton arc-consistency (SAC) –after any assignment, problem can be made AC Restricted path-consistency (RPC) –problem is AC, and if any assignment is consistent with just a single value for a 2nd var then any 3rd var has a consistent value

14. Binary consistencies Simple relationships between them strongPC > SAC > PIC > RPC > AC NIC > PIC NIC ~ SAC NIC ~ strongPC

15. Non-binary consistencies Generalized arc-consistency (GAC) – any assignment can be extended to all other vars in same constraint Efficient algorithms exist for GAC on certain non-binary constraints –e.g. Regin’s all-different algorithm

16. Non-binary consistencies GAC v AC {1,2} x1 x2 x3 x1<>x2, x1<>x3, x2<>x3 AC all-different(x1,x2,x3) not GAC

17. Non-binary consistencies GAC can be used to enforce path- consistency –given a binary CSP –construct a ternary CSP’ –CSP is PC iff CSP’ is GAC

18. Non-binary consistencies Given CSP has: –vars xi with values a,b,c,... –binary constraints R(a,b) Construct CSP’ with: –vars xij with values,... –ternary constraints R’(,, ) iff R(a,b) & R (b,a) & R(c,a)

19. Non-binary consistencies Forward checking – forward checking (FC) on binary constraints enforces limited form of AC –most recently instantiated var and all futures vars are made AC Choices about how to generalize this to non-binary constraints

20. Non-binary consistencies nFC0 – any constraint with one un-instantianted var made AC nFC1 –apply one pass of AC to each constraint and projection involving current var and one future var … nFC5

21. Binary decompositions Upper and lower bound on FC nFC1 on non-binary > FC on decomposition > nFC0 on non-binary Gaps can be exponential Consider n-ary all-different with n-1 values nFC1 takes (n-1) branches FC on decomposition takes (n-1)! branches

22. Binary decompositions GAC lower bound GAC on non-binary > AC on decomposition Gap again can be exponential But if we decompose too much, GAC=AC! GAC upper bound In general, GAC ~ NIC, GAC ~ PIC.. BUT if decomposition to clique, NIC > GAC

23. Binary decompositions Tighter results provable for stricter classes Tree decomposable constraints –constraint graph is tree Triangle preserving constraints –non-binary constraints on all triangles

24. Binary decompositions Tree decomposable constraints –e.g. monotone(x1,x2,x3) –GAC=AC –not surprising as AC enough to solve! Decomposition here doesn’t lose us anything –but even one cycle is enough for GAC>AC

25. Binary decompositions Triangle preserving decomposition e.g. all-different(x1,x2,x3), quasigroups,... GAC > PIC, gap can again be exponential GAC ~ SAC, strongPC PIC is very strong consistency to be achieving at each node –GAC can do even better than this! –decomposition carries a very large price

26. Binary decompositions Experimental results –quasigroup completion –quasigroup existence Quasigroup is a Latin square –completion is completing partially filled square –existence is finding one with additional properties

27. Binary decompositions Modelling the quasigroup problem –n^2 vars, each with domain of size n Non-binary model –2n all-different constraints (one for each row and column Binary decomposition –2n cliques of not-equals constraints

28. Binary decompositions Quasigroup completion –Gomes & Selman report “heavy-tailed” distributions Maintaining AC on binary decomposition –problems often take long time to solve Maintaining GAC on all-different –almost all problems trivial

29. Binary decompositions Quasigroup existence –best paper at IJCAI-93 –of interest to design theory Open results first proved by computer –in some cases, only ever proved by computer Maintaining GAC very competitive –compared to specialized model finders like FINDER, SEM

30. Intermission Mid-point of talk Short commercial break CSPLib: a benchmark library for constraints http://csplib.cs.strath.ac.uk

32. Why build CSPLib? Benchmarking –shift field from random to more realistic problems Modelling Show-case Challenges

33. Why not build CSPLib? Over-fitting –CSPLib needs to be large & extensible Unrepresentative –CSPLib needs your “realistic” problems No standard modelling language –CSPLib uses natural language Hard work –CSPLib needs to be a “community” effort

34. What does CSPLib contain? Over 20 problem types in 5 categories –bin packing (e.g. vessel loading) –scheduling (e.g. basketball tournament) –frequency assignment (e.g. Golomb rulers) –combinatorics (e.g. perfect square) –puzzles (e.g. nonagrams) Other useful links –solvers, other benchmark libraries,...

35. What does CSPLib contain? Example entry: –NL specification –results –references –(data) for instances –code –anything else of use “A Golomb ruler is a set of m integers, 0=a1<a2<..<am such that the m(m-1)/2 differences, aj-ai distinct. Such a ruler is of length am. The objective is to find the smallest length ruler…”

36. How to add to CSPLib? Send us your entries. NOW! –it takes just 1/2 hour Everyone has at least one problem to contribute –get others to solve your open problems –a problem shared is a problem solved Email problems to csplib@cs.strath.ac.uk

37. End of intermission Now back to... Binary encodings

38. Not all non-binary constraint decompose into binary constraints –on the same set of variables Consider again –even(x1+x2+x3) But binary CSPs NP-complete

39. Binary encodings Every non-binary constraint can be encoded into binary constraints –using polynomial number of additional (hidden) variables Two popular encodings –hidden variable encoding –dual encoding

40. Binary encodings Hidden variable encoding –add hidden var for each non-binary constraint Dual encoding –add hidden var for each non-binary constraint –throw away original variables

41. Binary encodings Dual encoding –consider c1:even(x1+x2), c2:odd(x2+x3) {00,11} {01,10} c1 c2 R21 R21= or

42. Binary encodings Hidden variable encoding –consider c1:even(x1+x2), c2:odd(x2+x3) {0,1} {00,11} {01,10} {0,1} x1x2x3 c1 c2 r1 r2 r1= or r2= or

43. Binary encodings Hidden var encoding -> dual encoding –compose relations, discard original vars {0,1} {00,11} {01,10} {0,1} x1x3 c1 c2 R21 R21 = r2 + r1

44. Binary encodings Double encoding –dual + hidden var encoding –original vars + hidden vars –all constraints of dual and of hidden Also called “combined encoding”

45. Binary encodings Theoretical analysis complicated by: –encoding “builds” in GAC for hidden vars –must translate between (original and hidden) vars –pruning in dual can infer large arity nogoods in original

46. Binary encodings Hidden var encoding FC on hidden ~ nFC0 on original each can be exponentially better than the other FC+ propogates through hidden vars FC+ on hidden = nFC1 on original

47. Binary encodings Hidden var encoding AC on hidden = GAC on original Before looking for efficient (specialized) GAC algorithm –try AC on hidden var encoding

48. Binary encodings No point doing NIC on hidden var encoding NIC on hidden = AC on hidden due to star shaped topology of constraint graph Higher consistencies remain distinct strongPC on hidden > SAC on hidden > NIC, AC on hidden

49. Binary encodings Dual encoding FC on dual ~ nFC0 on original each can be exponentially better than the other Dual better for tight constraints – domains for hidden vars then small

50. Binary encodings Dual encoding AC on dual > GAC on original BUT domains of hidden vars very large when non-binary constraints loose – AC on dual prohibitively expensive

51. Binary encodings Dual v hidden encoding BT on dual = FC on hidden AC on dual > AC on hidden SAC on dual > SAC on hidden strongPC on dual = strongPC on hidden Path consistency is enough to get through the hidden star!

52. Binary encodings Experimental results –crossword puzzles –Golomb rulers –random CSPs –random 3-SAT

53. Binary encodings Crossword puzzles “Original” model –vars for letters, domains {A-Z} Dual model –vars for words, domains = dictionary Dual at least 1,000 times faster on larger problems

54. Binary encodings Golomb ruler –set of ticks, xi on a ruler –all inter-tick distances, dij different Ternary constraints, dij=xi-xj –encoded using hidden var/double encoding Competitive with non-binary model –runs on any binary CSP solver!

55. Binary encodings Random CSPs and SAT –Bacchus and van Beek plot contours to show constraint tightness where encoding pays –gives good predictions for performance on structured problems like crosswords, … –number of satisfying tuples in constraint is most important factor in such predictions

56. Conclusions Non-binary v binary decompositions –GAC on non-binary can be stronger than PIC on decomposition Non-binary v binary encodings –GAC on non-binary = AC on hidden –AC on dual > GAC on non-binary

57. Conclusions Non-binary v binary decompositions –decomposition can add significantly to search cost Non-binary v binary encodings –encoding pays in practice on tight constraints

58. Future directions Modelling with non-binary constraints –many more models possible –role of auxiliary vars and implied constraints Different levels of consistency within a non-binay model –e.g. GAC on all-different, but bounds consistency on arithmetic constraints

59. Future directions Mixed models –e.g. dual encoding for only some of the (non-binary) constraints Optimization –objective function typically non-binary