Presentation is loading. Please wait.

Presentation is loading. Please wait.

Constraint Based Synthesis for Beginners PSY 2012 Armando Solar-Lezama.

Similar presentations


Presentation on theme: "Constraint Based Synthesis for Beginners PSY 2012 Armando Solar-Lezama."— Presentation transcript:

1 Constraint Based Synthesis for Beginners PSY 2012 Armando Solar-Lezama

2 CAP View of Synthesis Synthesis Methodology Code

3 A different approach Synthesis Methodology Code Domain Specific Tool

4 EXHIBIT A: SYNTHESIS OF SQL With Alvin Cheung and Sam Madden

5 Motivation It turns out SQL is challenging to learn  Who would have thought? Frameworks simplify program/DB interface  You can access the DB without using SQL  Which can lead to some interesting code...

6 Examples: Explicit Select public Set getUnfinishedProjects() { Set unfinishedP = new HashSet (); List projects = this.projectDao.getAllProjects(); for (Project project : projects) { if (!(project.getIsFinished())) { unfinishedP.add(project); } } return unfinishedP; } Get list of projects from the DB Select unfinished projects SELECT * FROM Projects WHERE isFinished=FALSE

7 Examples: Explicit Select public List getRoleUser(){ List listUser = new ArrayList (); List user= this.userDao.getUsers(); List role = this.roleDao.getRoles(); for(int i = 0; i < user.size(); i ++){ for(int a = 0; a < role.size(); a++){ if(user.get(i).getRole_id(). equals(role.get(a).getRole_id())){ WilosUser userok = user.get(i); listUser.add(userok); } return listUser; } Find users in the Roles list and add them to the output Start with users and roles SELECT u FROM users u, roles r WHERE u.roleId == r.id

8 Why is this so bad? These can be performance bottlenecks  Where performance matters, people write SQL More controversial arguments can be made

9 Is this a synthesis problem? It is Synthesizer Imperative code with loop nests Equivalent Formula in Relational Algebra Domain knowledge about relational algebra

10 Framing the problem List getUsersWithRoles () { List users = getUsersFromDB(); List roles = getRolesFromDB(); List results = []; int i = j = 0; while (i < users.size()) { while (j < roles.size()) { if (users[i].roleId == roles[j].id) results.add(users[i]); } return results; }

11 Framing the problem We want to synthesize a post-condition! Challenges  Defining the language  Solving the synthesis problem  Generating code

12 Defining the language Requirements  Should make reasoning about the program easy  Should have expressiveness comparable to SQL

13 Language Just like relational algebra......but with lists rather than sets  The program is written in terms of lists, not sets

14 Invariants List getUsersWithRoles () { List users = getUsersFromDB(); List roles = getRolesFromDB(); List results = []; int i = j = 0; while (i < users.size()) { while (j < roles.size()) { if (users[i].roleId == roles[j].id) results.add(users[i]); } return results; }

15 Verification

16 Synthesis Synthesis works best in a finite domain Invariant generation can be framed as a sketch 1) Model operations in terms of their source code 2) Construct the verification condition leaving invariant and post condition as unknowns 3) Create a sketch for unknowns 4) Solve!

17 Results so far We have tried this with a dozen loop nests from real open source projects All solve in less than 7 minutes  Slow, but not very optimized.

18 EXHIBIT B: AUTOMATED GRADING With Rishabh Singh and Sumit Gulwani

19 The real software problem The Software Quality problem is a symptom The real problem: The demand for software in our society far exceeds the supply of people skilled enough to produce it

20 Three pronged attack  Make programmers more productive  Make programming more accessible  Reduce the cost of training the next generation

21 Grading Programming Assignments Test-cases based grading  No precise correctness correlation  No student tailored feedback Manual grading by TAs  Error-prone  Time consuming  Expensive Manual grading will not scale to 100K students

22 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i < a.Length; i--) { b[count] = a[i]; count++; } return b; } 22 Buggy Program for Array Reverse 6:28::50 AM

23 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length-1; i < a.Length-1; i--) { b[count] = a[i]; count++; } return b; } 23 Buggy Program for Array Reverse 6:32::01 AM

24 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length-1; i < a.Length-1; i--) { b[count] = a[i]; count++; } return b; } 24 Buggy Program for Array Reverse 6:32::32 AM No change! Sign of Frustation?

25 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i <= a.Length; i--) { b[count] = a[i]; count++; } return b; } 25 Buggy Program for Array Reverse 6:33::19 AM

26 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i < a.Length; i--) {Console.Writeline(i); b[count] = a[i]; count++; } return b; } 26 Buggy Program for Array Reverse 6:33::55 AM Same as initial attempt except Console.Writeline!

27 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i < a.Length; i--) {Console.Writeline(i); b[count] = a[i]; count++; } return b; } 27 Buggy Program for Array Reverse 6:34::06 AM No change! Sign of Frustation?

28 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i <= a.Length; i--) {Console.Writeline(i); b[count] = a[i]; count++; } return b; } 28 Buggy Program for Array Reverse 6:34::56 AM The student has tried this before!

29 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i < a.Length; i--) { b[count] = a[i]; count++; } return b; } 29 Buggy Program for Array Reverse 6:36::24 AM Same as initial attempt!

30 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length-1; i < a.Length-1; i--) { b[count] = a[i]; count++; } return b; } 30 Buggy Program for Array Reverse 6:37::39 AM The student has tried this before!

31 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i > 0; i--) { b[count] = a[i]; count++; } return b; } 31 Buggy Program for Array Reverse 6:38::11 AM Almost correct! (a[i-1] instead of a[i] in loop body)

32 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i >= 0; i--) { b[count] = a[i]; count++; } return b; } 32 Buggy Program for Array Reverse 6:38::44 AM Student going in wrong direction!

33 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i < a.Length; i--) { b[count] = a[i]; count++; } return b; } 33 Buggy Program for Array Reverse 6:39::33 AM Back to bigger error!

34 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i < a.Length; i--) { b[count] = a[i]; count++; } return b; } 34 Buggy Program for Array Reverse 6:39::45 AM No change! Frustation!

35 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i < a.Length; i--) { b[count] = a[i]; count++; } return b; } 35 Buggy Program for Array Reverse 6:40::27 AM No change! More Frustation!!

36 using System; public class Program { public static int[] Puzzle(int[] a) { int[] b = new int[a.Length]; int count = 0; for(int i=a.Length; i < a.Length; i--) { b[count] = a[i]; count++; } return b; } 36 Buggy Program for Array Reverse 6:40::57 AM No change! Too Frustated now!!! Gives up.

37 AutoGrader Automate grading  Find semantic errors  Feedback to fix them Students make similar mistakes

38 Is this a synthesis problem? It is Synthesizer Buggy implementation Corrections for student program Error Model Reference Solution

39 Array Reverse i = 1 i <= a.Length

40 Challenge 1: Different Algorithms

41 Challenge 2: Scalability different possible candidate corrections

42 Our Approach

43 Aren’t rewrite systems hard? Can teachers really write rewrite rules? Angelic non-determinism helps  Ambiguities and redundancies no longer matter

44 Results: Problems Fixed

45 Results: Performance

46 Results: Generalization

47 Feedback for Tutoring System OK for Grading, But not ideal for teaching

48 Broad research agenda ahead Transformative for students in under-funded schools  Reduce the resources required to support quality instruction  Enable “true” distance education for programming courses Same technology could be used for automatic tutoring  Identify errors stemming from deep misconceptions (e.g. not understanding difference in values vs. references)  Synthesize small examples that make misconceptions explicit 48

49 Sketch Tutorial on Saturday


Download ppt "Constraint Based Synthesis for Beginners PSY 2012 Armando Solar-Lezama."

Similar presentations


Ads by Google