An Effective Method to Control Interrupt Handler for Data Race Detection Makoto Higashi †, Tetsuo Yamamoto ‡, Yasuhiro Hayase †, Takashi Ishio † and Katsuro Inoue † † Osaka University ‡ Ritsumeikan University

Outline Motivation –Embedded software and data race conditions Approach –Control of an interrupt handler Case study –uClinux Summary and future work 2010/5/4 AST2010 2

2010/5/4 AST Reliability of embedded software There are many safety-critical embedded software –Pacemaker: risk of losing human life –Mobile phone: risk of intercepting personal data Embedded system consists of external devices and control software –Embedded software becomes aware of inputs from external devices through interrupts or I/O memory

2010/5/4 4 Interrupt-driven software We focus on interrupt-driven software in embedded software, where processing is initiated when external devices signal the CPU –Interrupts add fine-grained concurrency to the software handler(void) op = 0 return Interrupt handler main(void) if (op == 1) … Main routine interrupt return interrupt

Particular fault to interrupt- driven software Main routine shares memory with interrupt handler There is a failure at the particular timing –Interrupt occurs at unexpected timing –Interrupt handler changes shared memory 2010/5/4 AST It is important to detect data race conditions

2010/5/4 AST Example of data race condition divide(void) no yes return x != 0 ret = 100 / x interrupt_handler(void) x = 0 return interrupt Interrupt handler Main routine variable x access Divide 100 by x in case of x != 0 Divide 100 by x in a state of x == 0 Detection of data race conditions through testing

2010/5/4 AST Testing Process Testing process of non interrupt-driven software –Input values to module –Check return value from the module Testing process of interrupt-driven software –Combination Interrupt handler with other module –Consider a wide variety of Interrupt timing

Key ideas 1/2 Condition of data race condition 1.Main routine accesses a variable twice 1st access is reading or writing 2nd access is reading 2.Main routine assumes that the value of the variable is unchanged 3.Interrupt occurs between 1st and 2nd access 4.Interrupt handler changes the value of the variable 2010/5/4 AST x = 3 ret = 100 / x x != 0 Assumes x != 0 no return interrupt ret = 100 / x interruput Assumes a == 3

Key ideas 2/2 Control of execution path of interrupt handler Embedded software uses memory- mapped I/O for communicating with external devices –Load instructions read the value of memory –Substitute user specified value for the value of memory 2010/5/4 AST2010 9

Our work Objective: Testing of interrupt-driven software Approach: Detection of faults related to interrupts (data race conditions) to cause interrupts automatically Result: Detection a fault not to cause interrupts manually 2010/5/4 AST

2010/5/4 AST Mechanism to cause interrupts CPU emulator Mechanism to cause interrupts interpreter A user specifies the kind of interrupt Configuration file Machine language instruction Check read instruction or write instruction LDR ADDR ADD MOV ・ Interrupt handler: ・ RET interrupt

2010/5/4 AST Prevention of infinite loop Compare current program counter with the saved counter If the saved counter is different from the current program counter, casuse an interrupt and save program counter Mechasin to cause interrupts Saved counter Current program counter interrupt LDR ADDR ADD MOV ・ Interrupt handler: ・ RET Result of comparison Current program counter Access memory Interrupt handler Prevent causing interrupts at the same location Saved counter

2010/5/4 AST Mechanism to substitute values Memory access LDR ADDR ADD MOV ・ Return user specified value Return value Read instrunction CPU emulator Memory access memory check if the memory address is user specified address If yes, Substitute the value Memory access Return value …..

2010/5/4 AST Configuration file 1/2 Interrupt –The kind of interrupt –Support only one kind of interrupt in single file Memory address –An address which is mapped to external device –If you know memory address of global variable, the address can be specified

2010/5/4 AST Configuration file 2/2 Function name –Substitute new value within only specified function Because it is very slow to substitute values within all functions New value –Constant –Global variables –Current value

Case study We have applied our method to software which contains data race condition –Aim Investigate the process to detect the data race condition –Target software uClinux 2010/5/4 AST

Data race condition on uClinux When sending characters of queue, the code accesses out of queue –After checking the count of queue, the routine sends characters –Just after the checking, interrupts occurs 2010/5/4 AST ・ if (xmit_cnt <= 0 || ……) return; ・ xmit_cnt--; ・ if (xmit_cnt <= 0 || ……) return; ・ xmit_cnt--; Main routineInterrupt handler interrupt The count of queue is 1 The count of queue is 0 Access out of queue

2010/5/4 AST Procedure to detect the data race condition Assign 5 to the count of queue –Because interrupts occurs 4 times before the count of queue is checked Call main routine ・ Static void rs_flush_chars(struct tty_struct *tty){ struct m68k_serial *info = ……; m68328_uart *uart = ……; ・ if (xmit_cnt <= 0 || ……) return; interrupt

2010/5/4 AST Testing process 1.Doubt the possibility to cause a data race condition within a certain module A 2.Assume an interrupt handler B to cause the data race condition in cooperation with module A 3.Specify the kind of the interrupt handler to configuration file 4.Test module A

Cost CPU cycles –Our method:72,417,488 –Normal execution:4,836,078 About 15 times The total number of cycles took in the interrupt handler:69,952, /5/4 AST

2010/5/4 AST Data race condition that our mechanisms cannot detect 1: unsigned int len = 0; 2: void str_cpy(char *buf, char *str); 3: { 4: len = strlen(str); 5: if((0 < len) && (len <= strlen(str))) 6: memcpy(buf,str,len+1); 7: } 8: 9: void interrupt_handler(void){ 10: len++; 11: } interrupt No data race condition data race condition

2010/5/4 AST Summary and future work We have implemented 2 mechanisms to a CPU emulator to test for data race conditions in interrupt-driven software –Causes interrupts automatically –Substitute values of memory Future work –More appropriate timing of interrupts –Multiple kinds of interrupts