1. The Scientific Community Game for STEM Innovation and Education (STEM: Science, Technology, Engineering and Mathematics) Karl Lieberherr Ahmed Abdelmeged 3/16/20111Open House 2011

2. 3/16/2011Open House 20112 SCG = Scientific Community Game

3. Why Scientific Community Game? Why are you here? … motives in academic publishing: – desire for recognition and respect from the people one regards as peers, – desire to have impact (on conclusions being reached, on the development of the discipline, etc.), and – desire to participate in significant knowledge- building discourse. e.g., Scardamalia, M., & Bereiter, C. (1994) Open House 201133/16/2011

4. Why do we model Scientific Communities? Scientific Communities create and disseminate new knowledge to help society. A computational model of scientific communities supports the same efforts for computational problems: – focused collaboration and competition – checking of the rules of a scientific community – knowledge maintenance and evaluation 3/16/20114Open House 2011

5. How it Works Scholars propose and oppose (refute or strengthen) or agree on claims. Strengthen and agree are reduced to refute. Claims predict the outcome of a refutation protocol. Parameterized by two structures: Domain and Protocol. Claim Example: Alice claims that she can solve problem instances in instance set I with quality at least q using resources at most r. 3/16/20115Open House 2011

6. Mathematics Example: Strengthen, Agree and Refute Alice claims Claim C(t): ForAll [x in [0,1]] Exists[y in [0,1]] (x*y + (1-x)*(1-y^2)) >= t Consider claims: C(0.55): Bob strengthens this claim (0.6). C(0.60): Bob agrees with this claim. C(0.65): Bob attempts to refute this claim. 3/16/2011Open House 20116 Roles: Alice claims Bob opposes or agrees

7. Refute Alice claims ForAll [x in [0,1]] Exists [y in [0,1]] (x*y + (1-x)*(1-y^2)) >= 0.65 Bob refutes Protocol: – Bob provides x – Alice provides y – Bob refutes iff (x*y + (1-x)*(1-y^2)) < 0.65 3/16/2011Open House 20117

8. Karl Popper 3/16/20118Open House 2011

9. SCG is a web-based implementation of Karl Popper’s science ideas One of the greatest philosophers of science of the 20th century.philosophers of science Falsifiability or refutability is the logical possibility that an assertion could be shown false by a particular observation or physical experiment. Error elimination (refutation), performs a similar function for science that natural selection performs for biological evolution.natural selectionbiological evolution Open House 20119 from Wikipedia 3/16/2011

10. Comparison Karl Popper: Conjectures and Refutations Scientific Community Game: Claims and Refutations Our claims are about computational problems. Open House 2011103/16/2011

11. Automating the refutations There can be “bugs” in refutations. With a computational model of scientific communities we can check for many “bugs”. Can detect misunderstandings students might have. Fair evaluation of scholars. 3/16/201111Open House 2011

12. Designers SCG Domain – Instance, Solution, InstanceSet, valid, quality – basic domain functionality, like standard solvers and solvers for niches. providing instances with “interesting” solutions Protocol: using protocol language – standard protocols: ForAllExists, PositiveSecret, etc. Playground: Configure – Research/Development Managers (Innovation) – Professors (Teaching) Avatar – researchers, developers – students 3/16/201112Open House 2011

13. Domain Instance (language) Solution (language) – boolean valid(Instance) – [0,1] quality(Instance) InstanceSet (language, subset of Instance) – boolean valid() – boolean belongsTo(Instance) Response = Instance union Solution 3/16/201113Open House 2011

14. SCG(Domain) Protocol (fixed language) Claim(Domain) – boolean strengthen(Claim other) // other strengthens this – Domain.InstanceSet getInstanceSet() – Protocol getProtocol() – [0,1] getQuality() – [-1..1] getResult(List(Domain.Response)) 3/16/201114Open House 2011

15. Refutations of Claim and !Claim are efficient Claim: F unsatisfiable if refuted: Bob finds satisfying J; proof of !Claim. If defended: baby step towards proof of Claim. Proof: long !Claim: F satisfiable if refuted: Alice does not find satisfying J; baby step towards proof of Claim. If defended: proof of !Claim. Proof: short 3/16/201115Open House 2011 Roles: Alice claims Bob attempts to refute

16. Both refutations are efficient Claim: Exists F in IS All J: fsat(F,J)<=t if refuted: Bob finds J; proof of !Claim assuming Alice is perfect. If defended: baby step towards proof of Claim. Proof short. !Claim: F has J: fsat(F,J)>=t All F in IS Exists J: fsat(F,J)>=t if refuted: Alice does not find J; baby step towards proof of Claim. If defended: proof of !Claim if Bob is perfect. Proof short. 3/16/201116Open House 2011

17. Claim involving Experiment Claim ExperimentalTechnique(X,Y,q,r) I claim, given raw materials x in X, I can produce product y in Y of quality q and using resources at most r. 17Bionetics 2010

18. Our vision Researchers and Professors come to the SCG website and configure a new playground X in which tournaments will take place. Participating teams get baby avatars generated for X that participate in daily competitions. Competition generates a wealth of information: educated employees/students, good (undefeated) software, good algorithms, good potential employees. Reward is given to the winner. 3/16/2011Open House 201118

19. Conclusions Computational Modeling of Scientific Communities is a good idea: – foster Innovation – improve education STEM domains: use the web effectively Current use: – Algorithms class – Software development class 3/16/2011Open House 201119

20. 3/16/2011Open House 201120 Thank you!

21. Thank you! 3/16/2011Open House 201121

22. Strengthening When claim C is strengthened by Bob to C', Alice must try to refute C' and the strengthening holds only if Bob defends C'. strengthenP(C,C') must hold. When scholar Bob successfully strengthens a claim of Alice, Bob wins reputation: – Bob + ClaimConfidence + |quality(C)-quality(C')| When scholar Alice successfully defends her own claim against Bob, Alice wins reputation. – Alice + ClaimConfidence 3/16/2011Open House 201122

23. Agreement When Bob agrees on claim C with Alice, – (1) Bob must defend C against Alice (if not, Bob loses) – (2) Bob must refute C' = C minimally strengthened along quality dimension (using the configuration file constant minStrengthen) with Alice as defender (if not, Bob loses). Then Alice must do the same: 3/16/2011Open House 201123

24. Agreement – (1) Alice must defend C against Bob (if not, Alice loses) – (2) Alice must refute C' with Bob as defender (if not, Alice loses) If all those protocols produce the result as described, the claim goes into the social welfare set (the knowledge base of claims believed to hold and having maximum strength). 3/16/2011Open House 201124

25. Refutations of Claim and !Claim are efficient Claim: F unsatisfiable if refuted: Bob finds satisfying J; proof of !Claim. If defended: baby step towards proof of Claim. Proof: long !Claim: F satisfiable if refuted: Alice does not find satisfying J; baby step towards proof of Claim. If defended: proof of !Claim. Proof: short Alice should never have made the claim! 3/16/201125Open House 2011

26. Both refutations are efficient Claim: Exists F in IS All J: fsat(F,J)<=t if refuted: Bob finds J; proof of !Claim assuming Alice is perfect. If defended: baby step towards proof of Claim. Proof short. !Claim: F has J: fsat(F,J)>=t All F in IS Exists J: fsat(F,J)>=t if refuted: Alice does not find J; baby step towards proof of Claim. If defended: proof of !Claim if Bob is perfect. Proof short. Alice should never have made the claim!? 3/16/201126Open House 2011

27. Designers SCG Domain – includes designing basic components for avatar like standard solvers. Example: HSR: linear search solver Protocol Playground: Goal: make playground designers configurators. Avatar designers 3/16/201127Open House 2011

28. Example Playground Design Highest Safe Rung Configuration: – domain HSR – claim 1: instanceSetClass protocolClass – claim 2: instanceSetClass !protocolClass – initialReputation = 100 – … 3/16/201128Open House 2011

29. Designers: what they produce SCG /scg – scg.cd, scg.beh – /protocol Java classes: Singleton Pattern Domain /domain – /hsr: hsr.cd, hsr.beh /avatar (components for avatar) Playground – config file: location of configuration file is given as argument to admin 3/16/201129Open House 2011

30. Config Config = // to configure admin SCGConfig Wrap(DomainConfigI). Example entries: – domain CSP – claim 1: instanceSetClass protocolClass – claim 2: instanceSetClass !protocolClass – initialReputation = 100 – … 3/16/201130Open House 2011

31. Where can we find configuration settings If there is a configuration file location given to the admin – in the configuration file If not: the default value given in the code. 3/16/201131Open House 2011

32. 3/16/201132Open House 2011

33. Designers SCG Domain – Instance, Solution, InstanceSet, valid, quality – basic domain functionality, like standard solvers and solvers for niches. providing instances with “interesting” solutions Protocol: using protocol language – standard protocols: ForAllExists, PositiveSecret, etc. Playground: Configurate – Research/Development Managers (Innovation) – Professors (Teaching) Avatar – researchers, developers – students 3/16/201133Open House 2011

34. Example Playground Design Highest Safe Rung Configuration: – domain HSR – claim 1: instanceSetClass protocolClass – claim 2: instanceSetClass !protocolClass – initialReputation = 100 – … 3/16/201134Open House 2011