Race Directed Random Testing of Concurrent Programs KOUSHIK SEN - UNIVERSITY OF CALIFORNIA, BERKELEY PRESENTED BY – ARTHUR KIYANOVSKI – TECHNION, ISRAEL

Goal Verify Data Races  Automatically  Efficiently  No false alarms  Being able to reproduce races

Background

Data Race - Definition  Two or more threads access the same memory location concurrently  At least one of the accesses is for writing x=1 if (x==1) … Temporally next to each other

Data Race – Another Example Thread 2: X=X+1; Thread 1: X=X+1; What “actually happens” in x=x+1? 1.Read x to tmp 2.Add 1 to tmp 3.Write tmp to X

Not a Data Race! Thread 2: Lock(L); X=X+1; Unlock(L); Thread 1: Lock(L); X=X+1; Unlock(L);

Multithreaded Testing On One CPU  Lets say we have 2 threads – blue and red.  There is a data race between red 3 and blue 3.  Let’s show a way of running them on 1 CPU that can find the same Data Races that running them on 2 CPUs can Data Race

Once upon a time… Testing concurrent programs

The Simple Traditional Way Repeatedly execute the program with the hope that different test executions will result in different interleavings.

Advantages Of The Traditional Way  Inexpensive compared to sophisticated techniques such as model checking and verification

Problems With The Traditional Way  Outcome highly dependent on the test environment.  Some interleavings may only occur on heavily-loaded test systems.  Depends on the underlying operating system for thread scheduling  Often ends up executing the same interleaving many times.

Simple Randomized Algorithm Simple Scheduler At every state during a concurrent execution –Pick a thread randomly –Execute the next instruction of the thread Slide taken and adapted from the original slides of Koushik Sen for PLDI08

Simple Randomized Algorithm Simple Scheduler At every state during a concurrent execution –Pick a thread randomly –Execute the next instruction of the thread Slide taken and adapted from the original slides of Koushik Sen for PLDI08

Simple Randomized Algorithm Simple Scheduler At every state during a concurrent execution –Pick a thread randomly –Execute the next instruction of the thread Slide taken and adapted from the original slides of Koushik Sen for PLDI08

Simple Randomized Algorithm Simple Scheduler At every state during a concurrent execution –Pick a thread randomly –Execute the next instruction of the thread Slide taken and adapted from the original slides of Koushik Sen for PLDI08

Simple Randomized Algorithm Simple Scheduler At every state during a concurrent execution –Pick a thread randomly –Execute the next instruction of the thread Slide taken and adapted from the original slides of Koushik Sen for PLDI08

Simple Randomized Algorithm Simple Scheduler At every state during a concurrent execution –Pick a thread randomly –Execute the next instruction of the thread Slide taken and adapted from the original slides of Koushik Sen for PLDI08

Problems With Randomized Algorithm  Everything is left to chance – Randomized is not good enough for some code structures What are the chances that we randomly stumble upon the data race here? thread2 { x = x+1; } thread1 { for (i=0; i<1,000,000; ++i); x = x+1; } Data Race

Program Analysis Techniques  Very effective in finding data races  Static Analysis - No need to see an actual execution.  Dynamic Analysis - Need to see real concurrent execution. BUT  Often report many data races that are false warnings  Require manual inspection to see if a race is real or not

False Positive Example thread2 { 3: if (y == 1) 4: if (x != 1) 5: do something; } thread1 { 1: x = 1; 2: y = 1; } Current Analysis Techniques will report a data race between lines 1 and 7. Initially x = y = 0 Data Race?

RaceFuzzer Algorithm – Welcome to the Future

First Stage – Find Potential Races  Find Potential Candidates For Races  Use an existing technique to find a set of pairs of state transitions that could potentially race.  At the time of writing the paper an Hybrid Race Detection Algorithm was chosen. But any algorithm that can find races will do.

Second Stage – Verify Real Races Key Ideas:  Randomized Scheduler  Racing statements “wait for each other“ to increase chances to be detected.  Remember this example? If only thread2 could wait for the end of the for loop… thread2 { x = x+1; } thread1 { for (i=0; i<1,000,000; ++i); x = x+1; } Data Race

RaceFuzzer By Example

Definitions  Enabled(s) – not waiting for a lock held by another thread  Alive(s) – have not terminated execution  Execute(s, t) – returns state after executing next statement of t  NextStmt(s, t) – next statement that t will execute

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Run ERASER: Statement pair (s5,s6) are in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Run ERASER: Statement pair (s5,s6) are in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Goal: Create a trace exhibiting the race Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Goal: Create a trace exhibiting the race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s6: if (o1.f==1) s5: o1.f = 1;

Input and Initialization 1: Inputs: the initial state s0, a set of two racing statements RaceSet 2: s := s0 3: postponed := ∅ 4: while Enabled(s) != ∅ do 5: t := a random thread in Enabled(s) \ postponed 6: if NextStmt(s, t) ∈ RaceSet then 7: R := Racing (s, t, postponed) 8: if R != ∅ then /* Actual race detected */ 9: print “ERROR: actual race found” 10: /* Randomly resolve race */ 11: if random boolean then 12: s := Execute(s, t) 13: else 14: postponed := postponed ∪ {t } 15: for all t‘ ∈ R do 16: s := Execute(s, t‘) 17: postponed := postponed \{t’} 18: end for 19: end if 20: else /* Wait for a race to happen */ 21: postponed := postponed ∪ {t} 22: end if 23: else 24: s := Execute(s, t) 25: end if 26: if postponed = Enabled(s) then 27: remove a random element from postponed 28: end if 29: end while 30: if Active(s) != ∅ then 31: print “ERROR: actual deadlock found” 32: end if

Input and Initialization Inputs:  the initial state s0  a set of two racing statements RaceSet Initialization: 2: s := s0 3: postponed := ∅

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: (s5,s6) in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

While.. Choose a Random Thread 1: Inputs: the initial state s0, a set of two racing statements RaceSet 2: s := s0 3: postponed := ∅ 4: while Enabled(s) != ∅ do 5: t := a random thread in Enabled(s) \ postponed 6: if NextStmt(s, t) ∈ RaceSet then 7: R := Racing (s, t, postponed) 8: if R != ∅ then /* Actual race detected */ 9: print “ERROR: actual race found” 10: /* Randomly resolve race */ 11: if random boolean then 12: s := Execute(s, t) 13: else 14: postponed := postponed ∪ {t } 15: for all t‘ ∈ R do 16: s := Execute(s, t‘) 17: postponed := postponed \{t’} 18: end for 19: end if 20: else /* Wait for a race to happen */ 21: postponed := postponed ∪ {t} 22: end if 23: else 24: s := Execute(s, t) 25: end if 26: if postponed = Enabled(s) then 27: remove a random element from postponed 28: end if 29: end while 30: if Active(s) != ∅ then 31: print “ERROR: actual deadlock found” 32: end if

While.. Choose a Random Thread 4: while Enabled(s) != ∅ do 5: t := a random thread in Enabled(s) \ postponed … 29: end while

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: (s5,s6) in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: (s5,s6) in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

Next Statement in a race? 1: Inputs: the initial state s0, a set of two racing statements RaceSet 2: s := s0 3: postponed := ∅ 4: while Enabled(s) != ∅ do 5: t := a random thread in Enabled(s) \ postponed 6: if NextStmt(s, t) ∈ RaceSet then 7: R := Racing (s, t, postponed) 8: if R != ∅ then /* Actual race detected */ 9: print “ERROR: actual race found” 10: /* Randomly resolve race */ 11: if random boolean then 12: s := Execute(s, t) 13: else 14: postponed := postponed ∪ {t } 15: for all t‘ ∈ R do 16: s := Execute(s, t‘) 17: postponed := postponed \{t’} 18: end for 19: end if 20: else /* Wait for a race to happen */ 21: postponed := postponed ∪ {t} 22: end if 23: else 24: s := Execute(s, t) 25: end if 26: if postponed = Enabled(s) then 27: remove a random element from postponed 28: end if 29: end while 30: if Active(s) != ∅ then 31: print “ERROR: actual deadlock found” 32: end if

Next Statement in a race? 6: if NextStmt(s, t) ∈ RaceSet then … 23: else 24: s := Execute(s, t) 25: end if

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: (s5,s6) in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Execution: s1: g1(); Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); (s5,s6) in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); (s5,s6) in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); (s5,s6) in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Execution: s1: g1(); s6: if (o1.f==1) Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Execution: s1: g1(); s6: if (o1.f==1) Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { }

Actual Data Race? 1: Inputs: the initial state s0, a set of two racing statements RaceSet 2: s := s0 3: postponed := ∅ 4: while Enabled(s) != ∅ do 5: t := a random thread in Enabled(s) \ postponed 6: if NextStmt(s, t) ∈ RaceSet then 7: R := Racing (s, t, postponed) 8: if R != ∅ then /* Actual race detected */ 9: print “ERROR: actual race found” 10: /* Randomly resolve race */ 11: if random boolean then 12: s := Execute(s, t) 13: else 14: postponed := postponed ∪ {t } 15: for all t‘ ∈ R do 16: s := Execute(s, t‘) 17: postponed := postponed \{t’} 18: end for 19: end if 20: else /* Wait for a race to happen */ 21: postponed := postponed ∪ {t} 22: end if 23: else 24: s := Execute(s, t) 25: end if 26: if postponed = Enabled(s) then 27: remove a random element from postponed 28: end if 29: end while 30: if Active(s) != ∅ then 31: print “ERROR: actual deadlock found” 32: end if

Actual Data Race? 7: R := Racing (s, t, postponed) 8: if R != ∅ then /* Actual race*/ … 20: else /* Wait for a race*/ 21: postponed := postponed ∪ {t} 22: end if

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s6: if (o1.f==1) (s5,s6) in race Postponed = { } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Execution: s1: g1(); s6: if (o1.f==1) Postponed = { s6: if (o1.f==1) } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); (s5,s6) in race Postponed = { s6: if (o1.f==1) } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); (s5,s6) in race Postponed = { s6: if (o1.f==1) } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); (s5,s6) in race Postponed = { s6: if (o1.f==1) } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); (s5,s6) in race Postponed = { s6: if (o1.f==1) } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Postponed = { s6: if (o1.f==1) } Execution: s1: g1(); s2: g2(); s3: g3(); Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Postponed = { s6: if (o1.f==1) } Execution: s1: g1(); s2: g2(); s3: g3(); Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Postponed = { s6: if (o1.f==1) } Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Postponed = { s6: if (o1.f==1) } Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o2.f = 1; Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o2.f = 1; Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { s6: if (o1.f==1) }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Race? Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o2.f = 1; Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { s6: if (o1.f==1) }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Postponed = { s6: if (o1.f==1) } Race? NO o1.f ≠ o2.f Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o2.f = 1; Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o2.f = 1; Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { s6: if (o1.f==1), s5: o2.f = 1; }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Postponed = { s6: if (o1.f==1), s5: o2.f = 1; } Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Postponed = { s6: if (o1.f==1), s5: o2.f = 1; } Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; (s5,s6) in race Postponed = { s6: if (o1.f==1), s5: o2.f = 1; } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; (s5,s6) in race Postponed = { s6: if (o1.f==1), s5: o2.f = 1; } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Postponed = { s6: if (o1.f==1), s5: o2.f = 1; } Race? YES o1.f = o1.f Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; Slide taken and adapted from the original slides of Koushik Sen for PLDI08

Actual Data Race! 1: Inputs: the initial state s0, a set of two racing statements RaceSet 2: s := s0 3: postponed := ∅ 4: while Enabled(s) != ∅ do 5: t := a random thread in Enabled(s) \ postponed 6: if NextStmt(s, t) ∈ RaceSet then 7: R := Racing (s, t, postponed) 8: if R != ∅ then /* Actual race detected */ 9: print “ERROR: actual race found” 10: /* Randomly resolve race */ 11: if random boolean then 12: s := Execute(s, t) 13: else 14: postponed := postponed ∪ {t } 15: for all t‘ ∈ R do 16: s := Execute(s, t‘) 17: postponed := postponed \{t’} 18: end for 19: end if 20: else /* Wait for a race to happen */ 21: postponed := postponed ∪ {t} 22: end if 23: else 24: s := Execute(s, t) 25: end if 26: if postponed = Enabled(s) then 27: remove a random element from postponed 28: end if 29: end while 30: if Active(s) != ∅ then 31: print “ERROR: actual deadlock found” 32: end if

Actual Data Race! 9: print “ERROR: actual race found” 10: /* Randomly resolve race */ 11: if random boolean then 12: s := Execute(s, t) 13: else 14: postponed := postponed ∪ {t } 15: for all t‘ ∈ R do 16: s := Execute(s, t‘) 17: postponed := postponed \{t’} 18: end for 19: end if

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); (s5,s6) in race Postponed = { s6: if (o1.f==1), s5: o2.f = 1; } Race? YES o1.f = o1.f Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); (s5,s6) in race Postponed = { s5: o2.f = 1; } s6: if (o1.f==1) s5: o1.f = 1; Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; (s5,s6) in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { s6: if (o1.f==1), s5: o2.f = 1; }

Everybody Is Postponed? 1: Inputs: the initial state s0, a set of two racing statements RaceSet 2: s := s0 3: postponed := ∅ 4: while Enabled(s) != ∅ do 5: t := a random thread in Enabled(s) \ postponed 6: if NextStmt(s, t) ∈ RaceSet then 7: R := Racing (s, t, postponed) 8: if R != ∅ then /* Actual race detected */ 9: print “ERROR: actual race found” 10: /* Randomly resolve race */ 11: if random boolean then 12: s := Execute(s, t) 13: else 14: postponed := postponed ∪ {t } 15: for all t‘ ∈ R do 16: s := Execute(s, t‘) 17: postponed := postponed \{t’} 18: end for 19: end if 20: else /* Wait for a race to happen */ 21: postponed := postponed ∪ {t} 22: end if 23: else 24: s := Execute(s, t) 25: end if 26: if postponed = Enabled(s) then 27: remove a random element from postponed 28: end if 29: end while 30: if Active(s) != ∅ then 31: print “ERROR: actual deadlock found” 32: end if

Everybody Is Postponed? 26: if postponed = Enabled(s) then 27: remove a random element from postponed 28: end if

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; (s5,s6) in race Slide taken and adapted from the original slides of Koushik Sen for PLDI08 Postponed = { s6: if (o1.f==1), s5: o2.f = 1; }

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; s6: if (o1.f==1) (s5,s6) in race Postponed = { s5: o2.f = 1; } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; s6: if (o1.f==1) s7: ERROR; (s5,s6) in race Postponed = { s5: o2.f = 1; } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; s6: if (o1.f==1) s7: ERROR; s5: o2.f = 1; (s5,s6) in race Postponed = { } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

Deadlock? 1: Inputs: the initial state s0, a set of two racing statements RaceSet 2: s := s0 3: postponed := ∅ 4: while Enabled(s) != ∅ do 5: t := a random thread in Enabled(s) \ postponed 6: if NextStmt(s, t) ∈ RaceSet then 7: R := Racing (s, t, postponed) 8: if R != ∅ then /* Actual race detected */ 9: print “ERROR: actual race found” 10: /* Randomly resolve race */ 11: if random boolean then 12: s := Execute(s, t) 13: else 14: postponed := postponed ∪ {t } 15: for all t‘ ∈ R do 16: s := Execute(s, t‘) 17: postponed := postponed \{t’} 18: end for 19: end if 20: else /* Wait for a race to happen */ 21: postponed := postponed ∪ {t} 22: end if 23: else 24: s := Execute(s, t) 25: end if 26: if postponed = Enabled(s) then 27: remove a random element from postponed 28: end if 29: end while 30: if Active(s) != ∅ then 31: print “ERROR: actual deadlock found” 32: end if

Deadlock? 30: if Active(s) != ∅ then 31: print “ERROR: actual deadlock found” 32: end if

RACEFUZZER using an example Thread1 foo(o1); foo(C x) { s1: g1() s2: g2(); s3: g3(); s4: g4(); s5: x.f = 1; } Thread2 bar(o1); bar(C y) { s6: if (y.f==1) s7: ERROR; } Thread3 foo(o2); Execution: s1: g1(); s2: g2(); s3: g3(); s4: g4(); s5: o1.f = 1; s6: if (o1.f==1) s7: ERROR; s5: o2.f = 1; (s5,s6) in race Postponed = { } Slide taken and adapted from the original slides of Koushik Sen for PLDI08

Deterministic Replay There are 3 points of non-determinism in the algorithm: Both are because of calls to a random function. If we save the random function seed before we start running RaceFuzzer, we can later replay the same run over and over using the same seed. 5: t := a random thread in Enabled(s) \ postponed 11: if random boolean then 27: remove a random element from postponed

Evaluation

RaceFuzzer Limitations  Cannot detect all real races in a concurrent program.  Depends on the dynamic analysis done in the first stage.  Being random in nature, RaceFuzzer may not be able to separate all real races from potential races.  However, this did not happen in our experiments with existing benchmarks.

Conclusion  RaceFuzzer  Classifies real Data Races  Enables reproduction of races found  No false positives!  Very efficient  Finds some exceptions and Deadlocks as a bonus. 84

QUESTIONS? 85

False Positive? Thread1 foo(x,y); foo(int x, int y) { s1: x = 1; s2: lock(L); s3: y = 1; s4: unlock(L); } Thread2 bar(x,y); bar(int x, int y) { s6: lock(L); s7: if (y == 1) { s8: if (x != 1) { s9: do something; }} s10: unlock(L); } Slide taken and adapted from the original slides of Koushik Sen for PLDI08 (s1,s8) in race Postponed = { s1: x = 1; } Execution: s6: lock(L); s7: if (y == 1) { s8: if (x != 1) { s9: do something; }} s10: unlock(L); s1: x = 1; s2: lock(L); s3: y = 1; s4: unlock(L); Postponed = {}

Hard To Find Data Races? Thread1 foo(x,y); foo(int x) { s1: for (i=0; i<1,000,000; ++i); s2: x = x+1; } Thread2 bar(x,y); bar(int x) { s3: x = x+1; } (s2,s3) in race Postponed = { s3: x = x+1; } Execution: s1: for (i=0; i<1,000,000; ++i); s2: x = x+1;s3: x = x+1; Data Race