1. Klassische Themen der Computerwissenschaft -Constraint Satisfaction-
Alexander Felfernig
Institut für Softwaretechnologie
Inffeldgasse 16b/2
A-8010 Graz
Austria

2. Goals Part 1 Introduction Applications CSP Solving Part 2
Debugging Constraint Sets
Conflict Detection & Diagnosis

3. Debugging Constraint Sets

4. Engineering of CKBs: Scenario
Knowledge Engineers are responsible for constraint acquisition & maintenance. Domain experts articulate domain knowledge (often in terms of examples). Constraints are changed very frequently. The correctness of the knowledge base is crucial: faulty definitions would cause follow-up costs. New constraints have to be integrated and tested as soon as possible.

5. Engineering of CKBs ? ? ?
Goal: Effective testing & debugging of constraint sets
Approach:
induce conflicts in the constraint set
resolve the conflict sets using model-based diagnosis [AIJ1987]
Result: minimal diagnoses for constraint sets. [AIJ2004] [IJEC2006] [WebEng2007] [IUI2008] [IJCAI2009]

6. Diagnosing Inconsistent Constraint Sets
V={v1,v2,v3,v4} dvi=[1..3] c1: v1 = 4 c2: v1 ≠ v2 c3: v3 = 3 c4: v4 > v3
Solution?
Conflict Set CS = {c1, c2, …, cq}  C s.t.
CS inconsistent.
Minimal (CS):  CS’ with CS’  CS.
Diagnosis   C: C -  consistent.
CS1={c1}, CS2={c3, c4}

7. Conflict Sets and Diagnoses: Further Examples
V={v1,v2,v3}
dv1=dv2=dv3=[1..3]
c1:v1>v2
c2:v2>v3
c3:v3>v1
Conflict sets:
CS1={c1,c2,c3}
Diagnoses:
1={c1}
2={c2}
3={c3}
V={v1,v2,v3}
dv1=dv2=dv3=[1..3]
c1:v1>v2
c2:v1<v2
c3:v2=v1
Conflict sets:
CS1={c1,c2}
CS2={c1,c3}
CS3={c2,c3}
Diagnoses:
1={c1,c2}
2={c1,c3}
3={c2,c3}

8. Conflict Sets and Diagnoses: Further Examples
V={v1,v2,v3}
dv1=dv2=dv3=[1..3]
c1:v1>v2
c2:v2>v3
c3:v3>v1
Conflict sets:
CS1={c1,c2,c3}
Diagnoses:
1={c1}
2={c2}
3={c3}
V={v1,v2,v3}
dv1=dv2=dv3=[1..3]
c1:v1>v2
c2:v1<v2
c3:v2=v1
Conflict sets:
CS1={c1,c2}
CS2={c1,c3}
CS3={c2,c3}
Diagnoses:
1={c1,c2}
2={c1,c3}
3={c2,c3}

9. Testing & Diagnosing Constraint Sets
Test Cases (T)+ c1 c2 c3
Conflict Sets (CSi) c4
CS1
CSr
CS2 …
Constraint Set (C)
Diagnoses (i) t1 c1 c1 c4 1 c2 t2 … c3 … tp c4 …
should be:
consistent … c2 c3 k cn
but are:
inconsistent …
Conflict Set CS = {c1, c2, …, cq}  C s.t. tiT: CS  {ti} inconsistent.
Minimal (CS):  CS’ with CS’  CS.
[AIJ1987]
[AIJ2004]
[AAAI2004]
[IJEC2007]
Diagnosis   C: C -   {ti} consistent ti  T.

10. Diagnosing Constraint Sets: Example
Test Cases (T)+
Conflict Sets (CSi) c5
CS1
Constraint Set (C)
Diagnoses (i)
but are:
inconsistent t1 c1
CS1 c5
c5 (t2 ok) t2 c2 c3
should be:
consistent c2 c3
CS2 c2
CS2
1={c5, c2}
(t1 ok) c3
(t1 ok)
2={c5, c3} c4 c5
Constraints:
c1: wr=medium  ip<>short-t c2: wr=high  ip=long-t c3: ip=long-t  rr=>3-6%  rr=>6-9% c4: rr=>9%  wr=high c5: rr=>6-9%  wr<>low  wr<>medium
Test cases:
t1: wr=high  rr=‘>9%’
t2: rr =>6-9%  wr=medium

11. Determining Conflict Sets: Basic Approach
V={v1,v2,v3,v4}; dv1=[1..4], dv2=[1..4], dv3=[1..3], dv4=[1..2] c1: v1 > v2, c2: v3 > v1, c3: v3 = v1, c4: v4 < v3, c5: v3 > v2, c6: v4 < v1, c7: v4 <> v2 c1 c2
conflict set CS1: {c2, c3} c3 c3 c1 c2 c3 c2 c1 c2 c4 c5 c6 c7 c7 c1 c2 c4 c5 c6 c7 c6 c1 c2 c7 c6 c2 c1
conflict set CS2: {c1,c2,c6,c7} v1 v2 v3 v4 1 2 3 4 v1 v2 v3 v4 1 2 3 4 v1 v2 v3 v4 1 2 3 4 v1 v2 v3 v4 1 2 3 4

12. Determining Diagnoses: Simple Algorithm
CS1:{c2, c3} c2 c3
diagnosis 1 {c2}
CS2:{c1, c2, c6, c7} c1 c2 c6 c7
diagnosis 2 {c1,c3}
¬diagnosis  {c2,c3} (not minimal)
diagnosis 3 {c3,c6}
diagnosis 4 {c3,c7}

13. Example Knowledge Representation
Table v1 (v1) 1 2 3 4
Table v2 (v2) 1 2 3 4
Table v3 (v3) 1 2 3
Table v4 (v4) 1 2
variables
and
domains
SELECT *
FROM v1, v2, v3, v4
WHERE
v1.v1>v2.v2 And /*c1*/
v3.v3>v1.v1 And /*c2*/
v3.v3=v1.v1 And /*c3*/
v4.v4<v3.v3 And /*c4*/
v3.v3>v2.v2 And /*c5*/
v4.v4<v1.v1 And /*c6*/
v4.v4<>v2.v2; /*c7*/
/* original query …
no solution*/
SELECT *
FROM v1, v2, v3, v4
WHERE
v1.v1>v2.v2 And /*c1*/
v3.v3>v1.v1 And /*c2*/
v3.v3=v1.v1 And /*c3*/
v4.v4<v3.v3 And /*c4*/
v3.v3>v2.v2 And /*c5*/
v4.v4<v1.v1 And /*c6*/
v4.v4<>v2.v2; /*c7*/
/* query taking into
account 1*/
SELECT *
FROM v1, v2, v3, v4
WHERE
v1.v1>v2.v2 And /*c1*/
v3.v3>v1.v1 And /*c2*/
v3.v3=v1.v1 And /*c3*/
v4.v4<v3.v3 And /*c4*/
v3.v3>v2.v2 And /*c5*/
v4.v4<v1.v1 And /*c6*/
v4.v4<>v2.v2; /*c7*/
/* query taking into
account 4*/ …

14. Duality of Conflicts and Diagnoses

15. Algorithms for Conflict Detection & Diagnosis

16. Diagnosis as Optimization Problem
Each constraint ci is represented by a variable xi [0,1]
Conflict sets CSj are represented by constraints csj
Example: cs1: x1 + x4 + x5  1
Optimization function:

17. Diagnosis as Optimization Problem
cs1: x1 + x4 + x5  1

18. Simple Conflict Detection
C  {c1, c2, …, cn}; /* C: all constraints */ B  ; /* B: constraints part of the minimal conflict */ IF inconsistent(C) REPEAT  = B; /* : currently investigated constraint set */ c  element(C - ) /* c: one constraint */     {c}; UNTIL inconsistent() B  B  {c}; UNTIL inconsistent(B) RETURN (B);
#consistency checks:
worst case: O((n*(n+1))/2+n) consistency checks
best case: 3

19. Example V={v1,v2,v3}; dv1=dv2=dv3=[1..3]; C={c1,c2,c3}
c1:v1>v2; c2:v2>v3; c3:v3>v1
B: ; : 
c:c1; : {c1}
c:c2; : {c1, c2}
c:c3; : {c1, c2, c3}
B:{c3}; : {c3}
c:c1; : {c3,c1}
c:c2; : {c3,c1,c2}
B:{c3,c2}; : {c3,c2}
c:c1; : {c3,c2,c1}
B:{c3,c2,c1} /* minimal conflict set */

20. Example V={v1,v2,v3}; dv1=dv2=dv3=[1..3]; C={c1,c2,c3}
c1:v1>v2; c2:v2=v1; c3:v3<>v1
B: ; : 
c:c1; : {c1}
c:c2; : {c1, c2}
B:{c2}; : {c2}
c:c1; : {c2,c1}
B:{c2,c1} /* minimal conflict set */

21. Example V={v1,v2,v3}; dv1=dv2=dv3=[1..3]; C={c1,c2,c3}
c1:v1>v2; c2:v2>v2; c3:v3<>v1
B: ; : 
c:c1; : {c1}
c:c2; : {c1, c2}
B:{c2} /* minimal conflict set */

22. QuickXPlain Algorithm
QuickXPlain(C={c1,c2, …, cm}) IF (is empty(C) or consistent(C)) return  /* no inconsistency */ ELSE return QX(, , C); /* C inconsistent */ QX(B, , C={c1,c2, …, cn}) IF ( <>  AND inconsistent(B)) return(); /* pruning of C */ IF (C = {c}) return({c}); /* conflict element c  detected */ k=n/2; C1{c1, …, ck}; C2 {ck+1, …, cn}; /* B still consistent */ 2  QX(B  C1, C1, C2); 1  QX(B  2, 2, C1); return (2  1)
(adapted version of [AAAI2004])
#consistency checks:
worst case: O(2k*log(m/k)+2k)
best case: O(log(m/k)+2k

23. QuickXPlain Scenarios
{c1, c2, c3, c4, c5, c6,c7}
… {c1, c2, c3}
2. {c1, c2, c3, c4, c5, c6,c7}
… {c1, c2, c3, c4, c5, c6,c7} C1 C2 C1 C2 C1 C2

24. FastDiag Algorithm Basic principle of QuickXplain [JUN04] path length
Consistency checks (max):
d: diagnosis set size
n: number of constraints
path
length
checks
@leafs
checks

25. FastDiag Scenarios C1 C2 C1 C2 C1 C2 C1 C2
{c1, c2, c3, c4, c5, c6,c7} /* AC – C2 ¬consistent */
… {c1, c2, c3, c4, c5, c6, c7};
2. {c1, c2, c3, c4, c5, c6,c7} /* AC – C2 consistent */
… {c4, c5, c6,c7} C1 C2 C1 C2 C1 C2

26. FastDiag Performance - Example: Computer Configurator Knowledge Base
Replay of user sessions with computer configurator
Average #(Diagnoses): 5.3
Average #(MCS): 14.5
20 variables, 50 constraints
#(calculated diagnoses)
HSDAG Breath-First
HSDAG Best-First
FastDiag 1
127
118 26 5
192
182 32 10
201
200 35
all
202
297 41
Runtime comparison (runtime in msec; avg values over ~300 sessions).

27. Studie @TUGraz: Identifizierbarkeit fehlerhafter Constraints
Lehrveranstaltung: Advanced Topics in AI
#Teilnehmer: 14 Studierende
(21% weibl., 79% männl.)
#Constraints: 10 (binär)
#Variable: 10 (domain size: 3)
Ziel: Konfliktfindung & Diagnose A1 A2 B1 B2 C1 C2
2/2
3/2
2/3
3/3
2/6
3/6
Wissensbasen: #Konflikte/Kardinalität(Konflikte)
Weitere Studien: A. Felfernig, Reducing Development and Maintenance Efforts for Web-based Recommender Applications, International Journal of Web Engi- neering and Technology, 3(3), pp , 2007.

28. Heatmap Visualization
Overview of areas, knowledge engineers looked at.
Can be used, for example, for constraint ranking.

29. GazePlot Visualization
Minimal Diagnoses
Preferred
Based on „confusion patterns“
Sequence of points, knowledge engineers looked at.
Can be used for the detection of “confusion patterns”.

30. Online Games: „Learning To Solve Diagnosis Tasks“
ColorShooter for iOS: EatIt for Android:

31. StudyBattles Environment
Environment that includes example questions related to course-relevant topics (currently only supported for the part on „Constraint Programming“)

32. References
A. Felfernig, L. Hotz, C. Bagley, and J. Tiihonen. Knowledge-based Configuration – From Research to Business Cases, Elsevier/Morgan Kaufmann, pp. 1—357, 2014.

33. References
S. Brailsford, C. Potts, and B. Smith. Constraint Satisfaction Problems: Algorithms and Applications, Europ. Jrnl. of Operations Research, 119: , [OR1999]
A. Felfernig, G. Friedrich, K. Isak, K. Shchekotykhin, E. Teppan, and D. Jannach. Automated Debugging of Recommender User Interface Descriptions, Journal of Applied Intelligence, Springer Verlag, 31(1):1-14, [AI2009]
A. Felfernig, G. Friedrich, D. Jannach, and M. Zanker, An Environment for the Development of Knowledge-based Recommender Applications, International Journal of Electronic Commerce (IJEC), 11(2), pp , [IJEC2006]
A. Felfernig, Reducing Development and Maintenance Efforts for Web-based Recommender Applications, International Jrnl. of Web Engineering and Technology, 3(3): , [WebEng2007]
E. Teppan, A. Felfernig: Minimization of Product Utility Estimation Errors in Recommender Result Set Evaluations. Web Intelligence 2009: 20-27, [WebIntelligence2009]
A. Felfernig, G. Friedrich, D. Jannach, and M. Stumptner, Consistency-based Diagnosis of configuration knowledge bases, AI Journal, 152(2):213–234, [AIJ2004]

34. References
A. Felfernig, G. Friedrich, M. Schubert, M. Mandl, M. Mairitsch, and E. Teppan. Plausible Repairs for Inconsistent Requirements, 21st International Joint Conference on Artificial Intelligence (IJCAI'09), Pasadena, California, USA, pp , [IJCAI2009]
A. Felfernig and R. Burke. Constraint-based Recommender Systems: Technologies and Research Issues, ACM International Conference on Electronic Commerce (ICEC'08), Innsbruck, Austria, Aug , pp , [ICEC2008]
E.C. Freuder. In Pursuit of the Holy Grail. Constraints, 2(1): 57-61, [Constraints1997]
U. Junker. QUICKXPLAIN: Preferred Explanations and Relaxations for Over- Constrained Problems. 19th National Conference on AI (AAAI04), pages 167–172, [AAAI2004]
R. Reiter. A theory of diagnosis from first principles, AI Journal, 23(1):57– 95, [AIJ1987]
S. Russel and P. Norvig. Artificial Intelligence - A Modern Approach, Prentice Hall, 2nd edition, 2002, see also: [ArtInt2002]

35. Thank You!