# Symbolic Execution with Mixed Concrete-Symbolic Solving

## Presentation on theme: "Symbolic Execution with Mixed Concrete-Symbolic Solving"— Presentation transcript:

Symbolic Execution with Mixed Concrete-Symbolic Solving
Corina Pasareanu1, Neha Rungta2 and Willem Visser3 1Carnegie Mellon, 2SGT Inc./NASA Ames 3University of Stellenbosch

Symbolic Execution Program analysis technique
King [Comm. ACM 1976] , Clarke [IEEE TSE 1976] Executes a program on symbolic inputs Maintains path condition (PC) – checked for satisfiablity with decision procedures Received renewed interest in recent years due to Algorithmic advances Increased availability of computational power and decision procedures Applications: Test-case generation, error detection, … Tools, many open-source UIUC: CUTE, jCUTE, Stanford: EXE, KLEE, UC Berkeley: CREST, BitBlaze Microsoft’s Pex, SAGE, YOGI, PREfix NASA’s Symbolic (Java) Pathfinder IBM’s Apollo, Parasoft’s testing tools etc. Say we use Omega lib

void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; S4; } S0, S1, S3, S4 = statements we wish to cover Symbolic Execution

statements we wish to cover Symbolic Execution Can not handle it!
void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; S4; } S0, S1, S3, S4 = statements we wish to cover Symbolic Execution Can not handle it! Solution: Mixed concrete-symbolic solving Assume hash is native or can not be handled by decision procedure

Mixed Concrete-Symbolic Solving
//hash(x)=10*x Example Mixed concrete-symbolic solving: all paths covered Assume hash can not be analyzed with our constraint solver EXE: stmt S3 not covered DART: all statements covered, path S0, S4 not covered Mixed concrete-symbolic solving: covers all paths Replace 6 with 7: DART and mixed c-s solving will miss S0, S3 PC to reach S0 is X>0 /\ y==hash(X) DART starts with a random value for x and y If x satisfies X>0 it fixes it and then picks a value for y such that it is equal to hash(x) EXE: solve the simple part and then keep that value Divergence: predicted path is different from the one taken Predicted path “S0;S4” != path taken “S1;S4” EXE results: stmt “S3” not covered DART results: path “S0;S4” not covered

Mixed Concrete-Symbolic Solving
Use un-interpreted functions for external library calls Split path condition PC into: simplePC – solvable constraints complexPC – non-linear constraints with un-interpreted functions Solve simplePC Use obtained solutions to simplify complexPC Check the result again for satisfiability

Mixed Concrete-Symbolic Solving
Assume hash(x) = 10 *x: PC: X>3 ∧ Y>10 ∧ Y=hash(X) simplePC complexPC Solve simplePC Use solution X=4 to compute h(4)=40 Simplify complexPC: Y=40 Solve again: simplified PC: X>3 ∧ Y>10 ∧ Y=40 Satisfiable!

Symbolic Execution … void test(int x, int y) { if (x > 0) {
if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; S4; } int hash(x) { if (0<=x<=10) return x*10; else return 0; PC: true PC: X>0 PC: X<=0 Solve X>0 hash(1)=10 Check X>0 & Y=10 PC: X>0 & Y=hash(X) S0 PC: X>3 & Y>10 & Y=hash(X) S3 PC: X>0 & X<=3 & Y=hash(X) S4 Solve X>3 & Y>10 hash(4)=40 Check X>3 & Y>10 & Y=40

Potential for Unsoundness
test (int x, int y) { if (x>=0 && x>y && y == x*x) S0; else S1; } Not Reachable PC: X>=0 & X > Y & Y = X*X S0 X>=0 & X>Y Y = X*X simplePC complexPC Must add constraints on the solutions back into simplified PC Not clear how to fix it; adding the constraints in DART will likely over-constrain the state space Add constraints on solutions used to simplify complexPC X=0, Y=-1 Y=0*0=0 DART/Concolic will diverge instead X>=0 & X>Y & Y=0 & X=0 Not SAT! Is SAT which implies S0 is Reachable! X>=0 & X>Y & Y=0 simplified PC

Directed Automated Random Testing (DART) Godefroid, Klarlund and Sen 2005
or Concolic Execution Collects path conditions along concrete executions Negates constraints on the PC after a run and Executes again with the newly found solutions Can overcome the weaknesses of classic symbolic execution

DART/Concolic Execution
X>0 & Y!=10 & X>3 void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; S4; } native int hash(x) { if (0<=x<=10) return x*10; else return 0; test(1,0) test(4,0) X > 0 X > 0 & Y != 40 S1 X>0 & Y!=40 & X>3 & Y<= 10 S4 X > 0 X > 0 & Y != 10 S1 X>0 & Y!=10 & X<=3 S4 DART/Concolic Execution

DART/Concolic Execution
X>0 & Y!=40 & X>3 & Y>10 X>0 & Y=40 & X>3 & Y>10 void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; S4; } native int hash(x) { if (0<=x<=10) return x*10; else return 0; test(4,11) X > 0 X > 0 & Y != 40 S1 X>0 & Y!=40 & X>3 & Y>10 S3 test(4,40) X > 0 X > 0 & Y = 40 S0 X>0 & Y=40 & X>3 & Y>10 S3 DART/Concolic Execution

DART/Concolic Execution
X>0 & Y=40 & X<=3 & Y>10 void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (x > 3 && y > 10) S3; S4; } native int hash(x) { if (0<=x<=10) return x*10; else return 0; test(1,40) X > 0 X > 0 & Y != 10 S1 X>0 & Y!=10 & X<=3 S4 Divergence! Aimed to get S0;S4 But reached S1;S4 Culprit in red; DART tries to satisfy a constraint specific to a previous run DART/Concolic Execution

Mixed Concrete-Symbolic Solving
vs DART Both incomplete Incomparable in power (see paper) Mixed concrete-symbolic solving can handle only “pure”, side-effect free functions DART does not have the limitation; will likely diverge

Incremental Solving User Annotations Random Solving

Incremental Solving … void test(int x, int y) { if (x > 0) {
if (y == hash(x)) S0; else S1; if (y > 10) S3; S4; } int hash(x) { if (0<=x<=10) return x*10; else return 0; PC: true PC: X>0 PC: X<=0 Solve X>0 hash(1)=10 Check X>0 & Y=10 PC: X>0 & Y=hash(X) S0 Solve X>0 & Y>10 Solution: X=1 hash(1)=10 Check X>0 & Y>10 & Y=10 PC: X>0 & Y>10 & Y=hash(X) S3 PC: X>0 & X<=3 & Y=hash(X) S4 Suggests future solvers to give multiple solutions Not SAT! Solution: X=2 hash(2)=20 Check X>0 & Y>10 & Y=20 Get another solution: SAT!

User Annotations … @Partition({“x>3”,”x<=3”})
void test(int x, int y) { if (x > 0) { if (y == hash(x)) S0; else S1; if (y > 10) S3; S4; } int hash(x) { if (0<=x<=10) return x*10; else return 0; PC: true PC: X>0 PC: X<=0 Solve X>0 hash(1)=10 Check X>0 & Y=10 PC: X>0 & Y=hash(X) S0 Solve X>0 & Y>10 & X>3 Hash(4)=40 Check X>0 & Y>10 & Y=40 SAT! Add user partitions one at a time PC: X>0 & Y>10 & Y=hash(X) S3 PC: X>0 & X<=3 & Y=hash(X) S4

Random Solving Pick solutions randomly from the solution space
Current implementation only picks randomly if the solution space is completely unconstrained

Implementation Mixed Concrete-Symbolic Solving Custom Listeners on SPF
Symbolic PathFinder SPF Symbolic Execution Extension for JPF (jpf-symbc) Model Checker for Java Open Source Java PathFinder Experience TSAFE (Tactical Separation Assisted Flight Environment) Apollo Lunar Pilot Example PC: 37 constraints in simplePC and 6 in complexPC

Related Work Tools that perform mixture of concrete and symbolic execution EXE, DART, CUTE, PEX, SAGE, … “Higher order test generation” – P. Godefroid [PLDI’11] Uses combination of validity checking and un-interpreted functions Generates tests from validity proofs Implementation challenge

Conclusions and Future Work
Mixed concrete-symbolic solving to address problems with classic symbolic execution Handling native libraries Incomplete decision procedures Open source implementation for Java Future Work More experiments More heuristics Handle data structures executed outside symbolic execution Use JPF’s serialization

Thank you!

Similar presentations