HARD: Hardware-Assisted lockset- based Race Detection P.Zhou, R.Teodorescu, Y.Zhou. HPCA’07 Shimin Chen LBA Reading Group Presentation

Motivation Data race detection important S/W solutions slow (not good for production runs) Previous H/W solutions focus on happens- before relation  Cannot detect potential races

Motivating Example

Solution: HARD (h/w lockset) Challenges: – How to efficiently store and maintain lockset for each variable in hardware? – How to efficiently perform the set operation in the lockset algorithm? Main ideas (will be detailed later) – h/w bloom filter – Piggybacking on cache coherence protocols – Reset all bloom filters after exiting a barrier

Outline LockSet (refresh our memory) HARD Evaluation Conclusion

Main Lockset Algorithm Idea: accesses to every shared variable should be protected by some common lock. Data structures: – Thread t’s current lock set: L(t) – Candidate set for a variable v: C(v) Algorithms – Modify L(t) upon lock acquire and release – Initiate C(v) to be a set of all locks – When t accesses v, C(v)=C(v)  L(t) – If C(v) ==  then report violation on variable v

Reducing False Positives

Outline LockSet (refresh our memory) HARD Evaluation Conclusion

HARD Overview LState: exclusive, shared, etc. BFVector: candidate lock set for the cache line Lock Register: Thread’s lockset Counter Register: used for resolving hash collisions (more detail later) 2bits16bits 32bits

HARD Overview: Operations A lock  a ‘1’ in bloom filter Fetching a line from memory: set the BFVector to all 1s, LState to exclusive Update BFVector and LState on accesses Communicate them through coherence protocol Lock register: thread’s lock set 2b16b 32b

Bloom Filter Bloom filter: A bit vector that represents a set of keys – A key is hashed d (e.g. d=3) times and represented by d bits Construct: for every key in the set, set its 3 bits in vector Membership Test: given a key, check if all its 3 bits are 1 – Definitely not in the set if some bits are 0 – May have false positives Bit 0 =H 0 (key)Bit 1 =H 1 (key)Bit 2 =H 2 (key) Filter

Representing LockSet as Bloom Filter 4 hash functions Lockset Intersection: bloom filter intersection Lockset empty: any of the 4bits are all 0

False Negative Caused by Bloom Filter

Prob of False Negatives Suppose the candidate set contains m locks Given a lock, probability of recognizing it as a member: prob_whole = prob_part k prob_part = 1 – (1-1/n) m When k=4, n=4: – (m=1), (m=2), (m=3) – Paper says: “experiments show that no races were missed” But what if the thread currently holds multiple locks? n bits k parts k=4, n=4

If threads hold 1 to 8 locks (not in the paper) n bits =4 k parts = m=1 m=2 m=3 m=4 t=1 : t=2 : t=3 : t=4 : t=5 : t=6 : t=7 : t=8 :

Try another design n bits =8 k parts = m=1 m=2 m=3 m=4 t=1 : t=2 : t=3 : t=4 : t=5 : t=6 : t=7 : t=8 :

Unlock operation  remove bit from bloom filter? 32 bit counter register each bloom filter bit has 2 bit counter Increment the 2-bit counter if the bloom filter bit is set Unlock: decrement the 2-bit counter, if 0, clear bloom filter bit 2b16b 32b

Candidate Set and LState Communications must broadcast changes to C(v) if cache line is in shared state

Handling Barriers Set BFVectors to all 1s after exiting a barrier (what if t2 does not hold any lock?)

Three Approximations Bloom filter to represent lockset Lockset info only in cache – Can only detect races in a short window of execution Cache line granularity – False sharing – Compiler to put shared variables to different lines? – Removing false sharing is generally good

Outline LockSet (refresh our memory) HARD Evaluation Conclusion

Methodology SESC: cycle-accurate execution-driven simulator (MIPS instruction set) Six SPLASH-2 benchmarks Randomly inject a data race: randomly remove a dynamic instance of lock and corresponding unlock Compare with happens-before, ideal lockset

Bug detected, false alarms Ideal: word-granularity, keep state in memory, perfect lockset # of false alarms is # of source code locations, dynamic errors are much more

Mainly bus traffic increase Note that HARD requires bloom filter operation per memory access in processor pipeline

Conclusion Main idea: bloom filter to represent lockset Three approximations: – Bloom filter to represent lockset – Lockset info only in cache – Cache line granularity Problems: – Lockset: false positives – Seems hard to add operations into processor pipeline – Are these the right approximations for monitoring production runs?