1. Virtual Scientific Communities for Innovation Karl Lieberherr Northeastern University College of Computer and Information Science Boston, MA joint work with Ahmed Abdelmeged and Bryan Chadwick Karl Lieberherr Northeastern University College of Computer and Information Science Boston, MA joint work with Ahmed Abdelmeged and Bryan Chadwick Supported by Novartis and GMO

2. Introduction Problems to be solved: – Optimal assembly of a system from components hardware software – Maximum constraint satisfaction problem (MAX- CSP) – Transporting goods minimizing energy consumption – Schedule tasks minimizing cost 9/17/2015Innovation2

3. Introduction Solve optimization problems in a domain X (X- problems). – Find a feasible solution of good quality efficiently. Scholars to play the Specker Challenge Game (X) [SCG(X)]. Repeat a few times. Within the group of participating scholars, the winning scholar has the – best solver for X-problems – best supported knowledge about X 9/17/20153Innovation

4. Introduction Players share hypotheses about "the approximability of problems in certain niches of the problem domain" Administrator reconciles inconsistencies between shared hypotheses => Condensing knowledge/stirring progress Player with the strongest correct hypothesis gains reputation, the other player receives targeted feedback / gains knowledge

5. Introduction The game is designed to exclude situations where it is impossible to give useful targeted feedback and/or it's possible to gain reputation without sharing the strongest correct hypothesis, e.g.: proposing strong obvious hypothesis, avoiding involvement with other players, mirroring,... etc => Fair assessment

6. Benefits of SCG Social Welfare – Supported knowledge Hypotheses are challenged and strengthened. Better supported knowledge comes from better algorithms and software. 9/17/20156Innovation

7. SCG(X) Scholars Design Problem Solver Develop Software Deliver Agent Agent AliceAgent Bob Administrator SCG police I am the best No!! Let’s play constructively 9/17/20157Innovation Scholar Alice Scholar Bob

8. SCG 9/17/2015Innovation8 no automation human plays full automation agent plays degree of automation used by scholar our focus some automation human plays 0 1 more applications: test constructive knowledge transfer to reliable, efficient software agent robot Bob Alice

9. Social Engineering Why develop problem solving software through a virtual scientific community? – Evaluates fairly, frequently, constructively and dynamically. Encourages retrieval of state-of- the-art know-how, integration and discovery. – Challenges humans, drives innovation, both competitive and collaborative. – Agents point humans to what needs attention in problem solution / software. 9/17/20159Innovation

10. Software Development Software developers are knowledge integrators: Requirements, contextual information (lectures, papers), behavior of program in competition, etc. 9/17/2015Innovation10

11. Scholars and virtual Scholars! Are encouraged to 1.offer results that are not easily improved. 2.offer results that they can successfully support. 3.strengthen results, if possible. 4.publish results of an experimental nature with an appropriate confidence level. 5.stay active and publish new results or oppose current results. 6.be well-rounded: solve posed problems and pose difficult problems for others. 7.become famous! 9/17/201511Innovation

12. Agent Design How to Design an artificial organism? – needs introspection to give it an ego. – has a basic need: maximize reputation. – has a rhythm: every round the same activity. – interacts with other agents by proposing and opposing hypotheses. makes agent vulnerable. 9/17/201512Innovation

13. competitive / collaborative 9/17/2015Innovation13 Agent Alice: claims hypothesis H Agent Bob: challenges H, discounts: provides evidence for !H loses reputation rwins knowledge k wins reputation rmakes public knowledge k

14. Definitions A hypothesis H offered by Alice is constructively defendable by Alice against Bob if Alice supports H when Bob challenges H. The constructive defense is determined by an interactive protocol between Alice and Bob. A hypothesis H1 is stronger than hypothesis H2 if H1 implies H2. Successfully opposing is a form of proposing: strengthening a hypothesis means to propose a new one. Discounting a hypothesis means to propose its complement. 9/17/2015Innovation14

15. SCG is sound The SCG game is sound, i.e., agent Alice wins with proposed hypothesis H against opponent Bob iff – H is stronger than what Bob could constructively defend and – H is constructively defendable by Alice against Bob. 9/17/2015Innovation15

16. GIGO: Garbage in / Garbage out If all agents are weak, no useful solver created. WEAK against STRONG: – STRONG accepts a hypothesis that is not discountable but WEAK cannot support it. Correct knowledge might be discounted. – STRONG strengthens a hypothesis too much that it becomes discountable, but WEAK cannot discount it. Incorrect knowledge might be supported. 9/17/2015Innovation16

17. What is the purpose of SCG? The purpose of playing an SCG(X) contest is to assess the "skills" of the players in: – "approximating" optimization problems in domain X, – "figuring-out" the wall-clock-time-bounded approximability of niches in domain X, – "figuring-out" hardest problems in a specific niche, and – "being-aware" of the niches in which their own solution algorithm works best. This multi-faceted evaluation makes SCG(X) more superior to contests based on benchmarks that only test the player's skills in approximating optimization problems. During the game, players cross-test each others' skills. 9/17/2015Innovation17

18. Game Driven Software Development A number of autonomous teams. Each team develops an agent that embodies their own heuristics. Agents participate in a contest. Contest winners get an egoistic boost. Teams develop their agents for the next contest. 9/17/201518Innovation

19. How to use SCG(X) ABB needs new ideas about how to solve optimization problems in domain X. Define hypothesis language for X – X-problems – hypotheses, includes protocol Submit hypothesis language definition to SCG server. 9/17/201519Innovation

20. How to use SCG(X) Offer prize money for winner with conditions, e.g., performance must be at least 10% higher as performance of agent XY that ABB provides. 10 teams from 6 countries sign up, committing to 6 competitions. Player executables become known to other players after each competition. One team from ABB. The SCG server sends them the basic agent and the administrator for testing. 9/17/201520Innovation

21. How to use SCG(X) Game histories known to all. Data mining! First competition is at 23.59 on day 1. Registration starts at 18.00 on same day. The competition lasts 2.5 hours. Repeat on days 7, 14, … 42. The final winner is: Team Mumbai, winning 10000 Euro. Delivers source code and design document describing winning algorithm to ABB. 9/17/201521Innovation

22. Benefits for ABB of using SCG(X) Teams perform know-how retrieval and integration and maybe some research. – Participating teams try to find the best knowledge in the area. – Hypothesis language gives control! The non-discounted hypotheses give hints about new X-specific knowledge. A well-tested solver for X-problems that integrates the current algorithmic knowledge in field X. 9/17/201522Innovation

23. Disadvantages of SCG The game is addictive. After Bob having spent 4 hours to fix his agent and still losing against Alice, Bob really wants to know why! Overhead to learn to define and participate in competitions. The administrator for SCG(X) must perfectly supervise the game. Includes checking the legality of X-problems. – if admin does not, cheap play is possible. – watching over the admin. 9/17/201523Innovation I am perfect

24. How to compensate for disadvantages Warn the scholars. Use a gentleman’s security policy: report administrator problems, don’t exploit them to win. Occasionally have a non-counting “attack the administrator” competition to find vulnerabilities in administrator. – both generic as well as X-specific vulnerabilities. 9/17/201524Innovation

25. Experience block 9/17/2015Innovation25

26. Experience Used for 3 years in undergraduate Software Development course. Prerequisites: 2 semesters of Introductory Programming, Object-Oriented Design, Discrete Structures, Theory of Computation. – Collect and integrate knowledge from prerequisite courses, lectures, and literature. – Teach it to the agent. 9/17/201526Innovation

27. Experience MAX-CSP MAX-CSP Problem Decompositions T-Ball (one relation), Softball (several relations, one implication tree), Baseball (several relations). ALL, SECRET 9/17/201527Innovation

28. Stages for SECRET T-Ball MAXCUT – R(x,y)= x!=y – fair coin ½ – maximally biased coin ½ – semi-definite programming / eigenvalue minimization 0.878 9/17/201528Innovation

29. Stages for SECRET T-Ball One-in-three – R(x,y,z) = (x+y+z=1) – fair coin: 0.375 – optimally biased coin: 0.444 9/17/201529Innovation

30. Stages for ALL Baseball Propose/Oppose/Provide/Solve – based on fair coin – optimally biased coin correctly optimize polynomials – correctly eliminate noise relations – correctly implement weights – … 9/17/201530Innovation

31. Life with SCG with SCG structured collaboration between scholars, frequent feedback motivation: propose and oppose non-trivial hypotheses to gain reputation. Drive to win knowledge accumulation in undiscounted hypotheses target scholars on a topic without SCG collaboration is unstructured, less effective motivation: reputation gain is delayed knowledge is scattered in emails, programs and minds more management effort required 9/17/201531Innovation

32. The SCG(X) Game

33. How to model a scholar? Solve problems. Provide hard problems. Propose hypotheses about Solve and Provide (Introspection). Oppose hypotheses. – Strengthen hypotheses. – Challenge hypotheses. Supported challenge failed. Discounted challenge succeeded. 9/17/201533Innovation

34. How to model a hypothesis A problem space. A discounting predicate on the problem space. A protocol to set the predicate through alternating “moves” (decisions) by Alice and Bob. If the predicate becomes true, Alice wins. 9/17/201534Innovation

35. How to model a hypothesis Proposing and challenging a hypotheses is risky: your opponent has much freedom to choose its decisions within the game rules. Alternating quantifiers. Replace “exists” by agent algorithm kept by administrator. 9/17/201535Innovation

36. Hypothesis Alice’ Hypothesis: There exists a problem P in niche N of X s.t. for all solutions S Bob searched by the opponent Bob in T seconds. Quality(P, S Bob ) < AR * Quality(P, S Alice ). Hypotheses have an associated confidence [0,1]. Hypothesis:. SQ = Quality(P, S Alice ) 9/17/201536Innovation

37. 1in3 niche Only relation 1in3 is used. 1in3 problem P: v1 v2 v3 v4 v5 1in3( v1 v2 v3) 1in3( v2 v4 v5) 1in3( v1 v3 v4) 1in3( v3 v4 v5) secret 1 0 0 1 0 Truth Table 1in3 000 0 001 1 010 1 011 0 100 1 101 0 110 0 111 0 Secret quality SQ = 3/4 9/17/201537Innovation

38. 1in3 Hypothesis 1in3 hypothesis H proposed by Alice: exists P in 1in3 niche so that for all S Bob that opponent Bob searches in time t (small constant) seconds: Quality(P,S Bob ) < 0.4 * Quality(P,S Alice ). H = (niche = (1in3), AR =0.4, confidence = 0.8) Bob has clever knowledge that Alice does not have. He opposes the hypothesis H by challenging it using his randomized algorithm. 9/17/201538Innovation

39. Bob’s clever knowledge 4/9 for 1in3 4/9 for 1in3: For all P in 1in3 niche, exists S so that Quality(P,S) >= 0.444 * SQ. Proof: la(p)=3*p*(1-p) 2 has the maximum 4/9. argmax p in [0,1] la(p) = 1/3. Without search, in PTIME. Derandomize Bob successfully discounts Alice gets a hint – Was Bob just lucky? Truth Table 1in3 000 0 001 1 010 1 011 0 100 1 101 0 110 0 111 0 9/17/201539Innovation

40. End 9/17/2015Innovation40

41. 10/16/09Can DM and ML help? Properties of challenge language Doing discounting and supporting requires constructive skills. Uncertainty about which problem to be delivered. Optional: mathematical skills –When agents are perfect, supporting implies the statement is a theorem and discounting implies the statement is NOT a theorem (a counter example was found). 41

42. Reputation Gain Hypothesis have credibility [0, ∞ ]. The credibility of a hypothesis is proportional to agent’s confidence in the hypothesis and agent’s reputation. Reputation gain is proportional to the discounting factor and the hypothesis credibility. The discounting factor [-1,1]. 1 means the hypothesis is completely discounted. 9/17/201542Innovation

43. AR is too low AR is too high exists P for all S that opponent searches: Quality(P,S) < AR * SQ Quality(P,S’) - AR * SQ strengthens: AR - AR’. Discounting Factor

44. H1 = ((1in3), AR = 1.0, confidence = 1.0) H1 proposed by Alice: exists P in 1in3 niche so that for all S that opponent Bob searches: Quality(P,S) < 1.0 * SQ. This is a reasonable hypothesis if Alice is sure that her secret assignment is the maximum assignment when she provides a sufficiently big problem to Bob.

45. What we did not tell you so far A game defines some configuration constants: a maximum problem size For example, all problems in the niche can have at most 1 million constraints. A maximum time bound for all tasks (propose, oppose, provide, solve), e.g. 60 seconds. An initial reputation, e.g., 100. When reputation becomes negative, agent has lost.

46. Discounting Factor: ReputationGain for Strengthening H1 = ((1in3), AR = 1.0, confidence = 1.0) H1 proposed by Alice: exists P in 1in3 niche so that for all S that opponent Bob searches: Quality(P,S) < 1.0 * SQ. Bob thinks he can strengthen H1 to H2 = (MAXCSP, niche = secret ExistsForAll (1in3), AR = 0.9, confidence = 1.0). DiscountingFactor 1.0-0.9 = 0.1. ReputationGain for Bob = 0.1 * 1.0 * AliceReputation. Alice gets her reputation back if she discounts H2.

47. Discounting Factor ReputationGain for Discounting H = ((1in3), AR = 0.4, confidence = 1.0) H proposed by Alice: exists P in 1in3 niche so that for all S that opponent Bob searches: Quality(P,S) < 0.4 * SQ. Bob knows he can discount H based on this knowledge: 4/9 for 1in3. Let’s assume he achieves 0.45 on Alice’ problem. DiscountingFactor 0.45 – 0.4 = 0.05. ReputationGain for Bob = 0.05*1.0*AliceReputation.

48. Discounting Factor ReputationGain for Supporting H = ((1in3), AR = 0.4, confidence = 1.0) H proposed by Alice: exists P in 1in3 niche so that for all S that opponent Bob searches: Quality(P,S) < 0.4 * SQ. Bob knows he can discount H based on this knowledge: 4/9 for 1in3. Let’s assume he achieves 0.3 on Alice’ problem. Bob has a bug somewhere! DiscountingFactor 0.3 – 0.4 = -0.1 ReputationLoss for Bob = -0.1*1.0*AliceReputation.

49. Mechanism Design The exact SCG(X) mechanism is still a work in progress. SCG(X) mechanism must be sound: – Encourage productive behavior and discourage unproductive behavior of scientists. – The agent with best heuristics wins. 9/17/201549Innovation

50. Tools to facilitate use of SCG(X) Definition of X. Generate a client-server infrastructure for playing SCG(X) on the web. Administrator enforces SCG(X) rules: client. Baby agents: servers. They can communicate and play an uninteresting game. Baby agents get improved by their caregivers, register with Administrator and the game begins at midnight. 9/17/201550Innovation

51. Properties

52. SCIENTIFIC COMMUNITY

53. 10/16/09Can DM and ML help? SCG: a scientific market game Domain X (Problem Solving domain such as an NPO domain) Agents with a reputation: offer-accept-deliver-solve Agents offer challenges with a confidence Agents accept challenges Discounting protocol for challenges: deliver-solve Agent wins reputation –when it accepts and discounts a challenge of another agent (challenge confidence * offerer reputation * at-risk). –when it supports its own challenge that was accepted by an agent (challenge confidence * acceptor reputation * at-risk). 53

54. 10/16/09Can DM and ML help? Think of a scientific community about domain X Scientists have reputations Scientists offer statements with a confidence Scientists question statements (accept) Scientists use discounting protocol (deliver- solve) Scientists win and loose reputation 54

55. 10/16/09Can DM and ML help? Scientific Market SCG(X) Defined by –Generic SCG game Axioms A mechanism (game rules) satisfying the axioms – Description of NPO X Feasible solutions, objective function –Belief language for X Predicates for defining niches (subsets of problems in X) Belief predicates Purpose of game: Good scientific behavior in domain X is rewarded. 55

56. Scholars and virtual Scholars! Are encouraged to – offer results that are not easily improved. – offer results that they can successfully support. – quote related work and show how they improve on previous work. – publish results of an experimental nature with an appropriate confidence level. – stay active and publish new results or oppose current results. – be well-rounded: solve posed problems and pose difficult problems for others. – become famous! 9/17/201556Innovation

57. Productive Scientific Behavior (1) The agents propose hypotheses that are difficult to strengthen or challenge (i.e. non- trivial yet correct). Otherwise, they lose reputation to their opponents. Offer results that cannot be easily improved. Offer results that they can successfully support. 9/17/201557Innovation

58. Good scientific behavior (2) Opposing a belief comes in two flavors. The agents should share “tight” beliefs. Agents who share a belief that is not tight lose reputation and the agents who tighten a belief win reputation unless the tightened belief is discounted by some other agent. offer results that are not easily improved. quote related work and show how they improve on previous work. The agents should share beliefs that are difficult to discount. Agents who share beliefs that are discountable lose reputation and the challengers who successfully discount win reputation. offer results that they can successfully support. 9/17/201558Innovation

59. Productive Scientific Behavior (2) Agents are encouraged to propose hypotheses they are not sure about. But they need to fairly express their confidence in their hypotheses. – If the confidence is inappropriately high, they lose too much reputation if the hypothesis is successfully discounted. – If the confidence is inappropriately low, they don’t win enough reputation if the hypothesis is successfully supported. publish results of an experimental nature with an appropriate confidence level. 9/17/201559Innovation

60. Productive Scientific Behavior (3) Agents stay active. In each “round”, they must propose new hypotheses and oppose other agents hypotheses. stay active and publish new hypotheses or oppose current hypotheses. Agents maximize their reputation. become famous! 9/17/201560Innovation

61. Productive Scientific Behavior (4) When Alice loses reputation to Bob, Alice can learn from Bob: – Alice has a bug in her software. – Bob has skills superior to hers. Alice should try to acquire Bob’s skills. Learn from mistakes. Be careful how you oppose a Nobel Laureate. The risks are high. 9/17/201561Innovation

62. Unproductive Scientific Behavior Cheating is forbidden: you can only succeed through good scientific behavior (by adding useful hypotheses or by successfully opposing hypotheses in the knowledge base). 9/17/201562Innovation

63. Fair Scientific Community All agents start with the same initial reputation. The winner has the best skills in domain X within the set of participating agents. 9/17/201563Innovation

64. Properties Agents are penalized for unproductive behaviors. A behavior is unproductive if it does not possibly lead to the accumulation of new knowledge about the specific NPO problem. Equilibrium. Agent with the best heuristics wins the game. Two player games + tournament. 9/17/201564Innovation

65. Applications

66. Improving the research approach Problem to be solved: Develop the best practical algorithms for solving NPO X. Standard solution: Write hundreds of papers on the topic with isolated implementations. What are the best practical algorithms? Our solution: Use the virtual scientific agent community SCG(X) with a suitably designed hypotheses language to compare the algorithms. The winning agent has the best practical algorithms. 9/17/201566Innovation

67. Game works at the press of a button to determine the winner. The winner has the best skills in the chosen domain. Find the experientially best algorithms for solving problems in domain X. Evaluation tool. The feedback is constructive. Testing and Learning Tool. Grading Tool. Over time, the market will collect undiscounted challenges: Belief Maintenance System. Agents must be reliable: Teaching Software Engineering Tool. Grading Tool.

68. 10/16/09Can DM and ML help? What is a scientific virtual market game good for? Market works at the press of a button to determine the winner. –The winner has the best skills in the chosen domain. Evaluation tool. –The feedback is constructive. Testing and Learning Tool. Grading Tool. –Over time, the market will collect undiscounted challenges: Belief Maintenance System. –Agents must be reliable: Teaching Software Engineering Tool. Grading Tool. 68

69. Contributing to State of the art knowledge of domain X

70. 10/16/09Can DM and ML help? Applications of SCG(X) Find the experientially best algorithms for solving problems in X. 70

71. Teaching

72. Agent World for SCG(X) Agent Caregiver lives outside SCG(X) world – World-class experts in domain X. – Graduate and undergraduate students Studying domain X. – Studying material needed to solve problems in X. – Learning algorithms based on game histories. Agent lives inside SCG(X) world – Agent win and lose reputation. – Agent Caregiver prepares agent for next game. 9/17/201572Innovation

73. Teaching: Survival Skills in SCG(X) Needed when agent caregiver is human. Knowledge about domain X needs to be developed by students or taught to them and understood and put into algorithms (propose- oppose(strengthen-challenge)-provide-solve) that go into the agent. This tests both whether the knowledge about X is understood as well as the programming skills. 9/17/201573Innovation

74. Teaching: Survival Skills in SCG(X) [cont.] [Scientific Innovation in X] Agents get skills programmed into them by clever scientists in domain X. Scientists use data mining to learn from competitions and manually improve the agents. [Machine Learning Innovation in X] Agents get skills programmed into them by an agent caregiver programmed with learning skills and data mining skills for domain X. Agent gets updated automatically between competitions and they improve automatically. 9/17/201574Innovation

75. 10/16/09Can DM and ML help? Software Development Skills Needed when agent caregiver is human. Knowledge about domain X needs to be developed by students or taught to them and understood and put into algorithms (offer- accept-deliver-solve) that go into the agent. This tests both whether the knowledge about X is understood as well as the programming skills. 75

76. AthenaLightningSweetStepdadPeon Athena3030 Lightning0 Sweet0 Stepdad3 Peon1

77. 10/16/09Can DM and ML help? Skills needed to survive in SCG(X) [Scientific Innovation in X] Agents get skills programmed into them by clever scientists in domain X. Scientists use data mining to learn from competitions and manually improve the agents. [Machine Learning Innovation in X] Agents get skills programmed into them by an agent caregiver programmed with learning skills and data mining skills for domain X. Agent gets updated automatically between competitions and they improve automatically. 77

78. Possible Application Domain For DM/ML/AI

79. SCG(X) produces history Proposer’s reputation: 120 Hypothesis10 proposer1 opposer2 confidence 1 Problem delivered Solution found: discountFactor = 1 Opposer: increase in reputation: 1 * 1 * 120 = 120 9/17/201579Innovation

80. Blame assignment – Where is the proposer to blame? – Bad hypothesis that is discountable. – Bug in problem finding algorithm. – Bug in problem solving algorithm used to check proposed hypothesis. 9/17/201580Innovation

81. Creating Agents An agent is composed of 6 components: Agent =. Components can refer to each other. Given a set of agents: Agent 1... Agent n Composed agent is a 12-tuple:. Propose, Oppose, Strengthen, Challenge, Provide, Solve 1=own 0=other 9/17/201581Innovation

82. Creating Agents [cont.] PropI, OppI, StrI, ChaI, ProvI, SolI ∈ [1..n]. PropO consist of 5-bits, each denote one of the other components. The first bit describes whether to use the opposition component of agent PropI or agent OppI. 9/17/201582Innovation

83. IMPLEMENTATION

84. Tools to facilitate use of SCG(X) Definition of X. Generate a client-server infrastructure for playing SCG(X) on the web. Administrator enforces SCG(X) rules: client. Baby agents: servers. They can communicate and play an uninteresting game. Baby agents get improved by their caregivers, register with Administrator and the game begins at midnight. 9/17/201584Innovation

85. Conclusions We have shown how a virtual scientific community of agents can foster the development and innovation of heuristics for approximating NPOs. We need your input on how DM and ML could help with evolving the agents. 9/17/201585Innovation

86. Questions?

87. 10/16/09Can DM and ML help? Pending When belief is discounted: offer complement of belief. Belief holder = agent that successfully discounted. 87

88. 10/16/09Can DM and ML help? Discounting If Alice offers the belief (FourColorConjecture, confidence = 1.0), she must be ready to support it. –The opponent Bob gives Alice a planar graph. –Alice must deliver a 4-coloring. If she does not, Bob has successfully discounted Alice’ belief and Alice loses reputation and Bob gains. If she does, Alice has successfully defended her belief and Alice wins reputation and the opponent Bob loses. –Note that discounting is different from finding a counterexample. If Alice loses she has a “fault” in her coloring algorithm. 88

89. 10/16/09Can DM and ML help? Beliefs: Four color conjecture FourColorConjecture: For all graphs g satisfying the predicate planar(g) there exists a 4-coloring of the nodes of g such that no two adjacent nodes have the same color. ForAllExists belief: For all problems p satisfying predicate pred(p) there exists a solution s satisfying a property(p,s). 89

90. – Undiscounted beliefs represent the accumulated shared knowledge gained from the game. (Requires negation and reoffer of discounted beliefs?) 9/17/201590Innovation

91. Improving the research approach Problem to be solved: Develop the best practical algorithms for solving NPO X. Standard solution: Write hundreds of papers on the topic with isolated implementations. What are the best practical algorithms? Our solution: Use the virtual scientific agent community SCG(X) with a suitably designed hypotheses language to compare the algorithms. The winning agent has the best practical algorithms. 9/17/201591Innovation