1. Software Bugs Bite! Yuanyuan (YY) Zhou Associate Professor Dept. of Computer Science Univ. of Illinois, Urbana-Champaign yyzhou@cs.uiuc.edu

2. YY. Zhou University of Illinois, Urbana-Champaign Slide 2 My Story: Zig-zag my way to System When young--------------------parents want me to be a business woman College application-----------Applied for BioChemistry  But changed unknowingly to CS (Reason: parents were chemistry teachers) 1st graduate school----------Mathematics at Univ. of Virginia 1st research area at Princeton-------------Theory 1st year as a system student --------------Negative  Felt that system is trivial----you can actually understand system talks 2nd year at Princeton---Wanted to quit to go to medical school 1st job after Ph.D  Co-founded a startup for 2 years (realize that I am probably not a business woman) 1 st year at UIUC ------- in the architecture group Now-----try to be a doctor to cure software “diseases”

3. YY. Zhou University of Illinois, Urbana-Champaign Slide 3 Family & Fun A supportive husband + 2 girls  Husband is my motivation to work on software bugs  My daughters inspired several research ideas (e.g Rx) and help me realize life is much more than my career Hobby  Skiing, hiking.... but Illinois? Switched to ice skating and growing vegetables  Computer games (a Civilization fan)

4. YY. Zhou University of Illinois, Urbana-Champaign Slide 4 Health of Computers We want computers to be dependable  Aircraft control systems  Hospital monitor systems  Financial transactions systems  Internet services  Cell phones, PDAs, smart home appliances, cars

5. YY. Zhou University of Illinois, Urbana-Champaign Slide 5 Bad News: Software Bugs Software bug is a major concern  Counts for 40% system failure [Marcus2000]  Costs $59.5 billion annually [NIST]

6. YY. Zhou University of Illinois, Urbana-Champaign Slide 6 Severe Software Clinical Cases Therac-25 (radiation therapy machine ) accidents (1985-1987) Therac-25  the injured patients died from lethal dosage of radiation caused by a software bug Code Red worm exploited a buffer overflow in Microsoft ’ s IIS server  Cost over $2.6 billion and Infected 350,000 servers in <14 hours 2003 North America blackout  Caused by a race condition in GE Energy's XA/21 monitoring softwarerace condition

7. YY. Zhou University of Illinois, Urbana-Champaign Slide 7 Why Software Bugs? Why do we get ill?  Nature Answer: human nature  On average, programmers inject 10 bugs per thousand lines of code

8. YY. Zhou University of Illinois, Urbana-Champaign Slide 8 Debugging is Hard for large software Cure/diagnose a disease is hard… Impossible to test every cases  Many bugs are hidden, “latent bugs” Some bugs are hard to reproduce  Configuration, timing and running environment dependent Hard to find root causes  Root causes may be far away from the symptom

9. YY. Zhou University of Illinois, Urbana-Champaign Slide 9 Type of Software Defects Specification bugs  Even the specification is wrong Design bugs  The design is erroneous Implementation bugs  Buffer overflow, memory leak, memory corruption, etc.

10. YY. Zhou University of Illinois, Urbana-Champaign Slide 10 Classification of Implementation Bugs Deterministic vs. non-deterministic Root causes  Memory bugs Memory leaks ?? Memory corruption Buffer overflow ?? Dangling pointer ??..  Semantics Unhandled exceptions Copy-paste bugs …  Concurrency Data races ?? Atomicity violations ?? Deadlocks ?? …

11. YY. Zhou University of Illinois, Urbana-Champaign Slide 11 Bugs and Security Attacks? Why are we in the same session? Software bug is a major source of security vulnerabilities  Bouncer (Tuesday)  Memory safety (Wednesday) Example: Buffer overflow  Stack smashing attack Started from November 1996 Still the basis for many attacks Who knows this attack?

12. YY. Zhou University of Illinois, Urbana-Champaign Slide 12 The Stack

13. YY. Zhou University of Illinois, Urbana-Champaign Slide 13 Buggy Program void buggy(char * in){ int i; char buffer[4]; for(i=0; in[i]!=0; i++) buffer[i] = in[i]; } Where is the bug? What happens if in is “ Aleph One ”?

14. YY. Zhou University of Illinois, Urbana-Champaign Slide 14 Buggy Code’s Stack 0x6500000 0 0x00000000int i 0x6500000 4 0x00000000char buffer[4] 0x6500000 8 0x8000003Dsaved FP 0x6500000 C 0x02000000return address 0x6500001 0 0x10000000char * in 0x6500001 4 0x00000000old stack frame… A l e p h _ O n e \0 41 6C 65 70 68 20 4F 6E 65 00 Program will start copying…

15. YY. Zhou University of Illinois, Urbana-Champaign Slide 15 Buggy Code’s Stack 0x6500000 0 0x00000001int i 0x6500000 4 0x41000000char buffer[4] 0x6500000 8 0x8000003Dsaved FP 0x6500000 C 0x02000000return address 0x6500001 0 0x10000000char * in 0x6500001 4 0x00000000old stack frame… A l e p h _ O n e \0 41 6C 65 70 68 20 4F 6E 65 00 Program will start copying…

16. YY. Zhou University of Illinois, Urbana-Champaign Slide 16 Buggy Code’s Stack 0x6500000 0 0x00000002int i 0x6500000 4 0x416C0000char buffer[4] 0x6500000 8 0x8000003Dsaved FP 0x6500000 C 0x02000000return address 0x6500001 0 0x10000000char * in 0x6500001 4 0x00000000old stack frame… A l e p h _ O n e \0 41 6C 65 70 68 20 4F 6E 65 00 Program will start copying…

17. YY. Zhou University of Illinois, Urbana-Champaign Slide 17 Buggy Code’s Stack 0x6500000 0 0x00000003int i 0x6500000 4 0x416C6500char buffer[4] 0x6500000 8 0x8000003Dsaved FP 0x6500000 C 0x02000000return address 0x6500001 0 0x10000000char * in 0x6500001 4 0x00000000old stack frame… A l e p h _ O n e \0 41 6C 65 70 68 20 4F 6E 65 00 Program will start copying…

18. YY. Zhou University of Illinois, Urbana-Champaign Slide 18 Buggy Code’s Stack 0x6500000 0 0x00000004int i 0x6500000 4 0x416C6570char buffer[4] 0x6500000 8 0x8000003Dsaved FP 0x6500000 C 0x02000000return address 0x6500001 0 0x10000000char * in 0x6500001 4 0x00000000old stack frame… A l e p h _ O n e \0 41 6C 65 70 68 20 4F 6E 65 00 Program will start copying… and continue.

19. YY. Zhou University of Illinois, Urbana-Champaign Slide 19 Buggy Code’s Stack 0x6500000 0 0x00000005int i 0x6500000 4 0x416C6570char buffer[4] 0x6500000 8 0x6800003Dsaved FP 0x6500000 C 0x02000000return address 0x6500001 0 0x10000000char * in 0x6500001 4 0x00000000old stack frame… A l e p h _ O n e \0 41 6C 65 70 68 20 4F 6E 65 00 C isn’t bounds checked…

20. YY. Zhou University of Illinois, Urbana-Champaign Slide 20 Buggy Code’s Stack 0x6500000 0 0x00000008int i 0x6500000 4 0x416C6570char buffer[4] 0x6500000 8 0x68204F6Esaved FP 0x6500000 C 0x02000000return address 0x6500001 0 0x10000000char * in 0x6500001 4 0x00000000old stack frame… A l e p h _ O n e \0 41 6C 65 70 68 20 4F 6E 65 00 What happens next?

21. YY. Zhou University of Illinois, Urbana-Champaign Slide 21 Buggy Code’s Stack 0x6500000 0 0x00000009int i 0x6500000 4 0x416C6570char buffer[4] 0x6500000 8 0x68204F6Esaved FP 0x6500000 C 0x65000000return address 0x6500001 0 0x10000000char * in 0x6500001 4 0x00000000old stack frame… A l e p h _ O n e \0 41 6C 65 70 68 20 4F 6E 65 00 We’ve overwritten the return address!

22. YY. Zhou University of Illinois, Urbana-Champaign Slide 22 Buggy Code’s Stack 0x6500000 0 0x00000009int i 0x6500000 4 0x416C6570char buffer[4] 0x6500000 8 0x68204F6Esaved FP 0x6500000 C 0x65000000return address 0x6500001 0 0x10000000char * in 0x6500001 4 0x00000000old stack frame… A l e p h _ O n e \0 41 6C 65 70 68 20 4F 6E 65 00 How to use it to hijack the program to do whatever you want?

23. YY. Zhou University of Illinois, Urbana-Champaign Slide 23 So How to Deal with Bugs? Compile time Run time Recovery Safe Language  Example:  Java  Transactional Memory (Mon)  Analogy:  Eat healthy, exercise…  Pro: can prevent some bugs  Cons:  Other types of bugs still occur  May be inefficient for some apps: server, OS. On-line Debugging Off-line Design time Diagnosis

24. YY. Zhou University of Illinois, Urbana-Champaign Slide 24 So How to Deal with Bugs? Static Checking  Program analysis & Model checking  Example:  Engler’s group  iComment [Tuesday]  Analogy:  CAT Scan  Pros:  No run time overhead  Good Coverage  Cons:  No accurate information  Need specification, annotation Compile time Run time Recovery On-line Debugging Off-line Design time Diagnosis

25. YY. Zhou University of Illinois, Urbana-Champaign Slide 25 So How to Deal with Bugs?  Check during execution  Example:  MUVI [Monday]  Analogy:  Heart monitor  Pros:  Accurate run-time information  Cons:  Large overhead  Coverage limitation Dynamic Checking Compile time Run time Recovery On-line Debugging Off-line Design time Diagnosis

26. YY. Zhou University of Illinois, Urbana-Champaign Slide 26 So How to Deal with Bugs?  Example:  gdb  Time travel machine  Analogy:  Doctor’s appointment  Pros:  Program-specific  Cons:  Time and effort-consuming Interactive Debugging Compile time Run time Recovery On-line debugging Off-line Design time Diagnosis

27. YY. Zhou University of Illinois, Urbana-Champaign Slide 27 So How to Deal with Bugs?  Example:  simple restart  rollback and reexecute  Analogy:  Electric shock  Advantage:  Simple and general  Limitation:  Cannot recover from all failures  Output commit problem Generic Recovery Compile time Run time Recovery On-line Debugging Off-line Design time Diagnosis

28. YY. Zhou University of Illinois, Urbana-Champaign Slide 28 So How to Deal with Bugs?  Example:  Core dump  Execution traces  Triage (Tuesday)  Analogy:  Triage (self-diagnosis)  Advantage:  Simple and general  Limitation:  Require significant offline manual efforts  Privacy concerns Online Diagnosis Compile time Run time Recovery On-line Debugging Off-line Design time Diagnosis

29. YY. Zhou University of Illinois, Urbana-Champaign Slide 29 Evaluation Methodology Benchmarks  Siemens benchmark  Too small Bug injection  May not be representative Find new bugs in open source code Better to be confirmed by developers Use existing bugs  Especially dynamic checking----need bugs to manifest

30. YY. Zhou University of Illinois, Urbana-Champaign Slide 30 Evaluation Metrics for Bug Detection Soundness: No false negatives  Prove there is no such violation  May report many false bugs Completeness: No false positives  All violations detected are true  May not find all the bugs Scalability: scalable to real programs Very hard to achieve all  Always need to tradeoff one for the others

31. YY. Zhou University of Illinois, Urbana-Champaign Slide 31 Open Problems Bug Detection  Detecting semantic and concurrency bugs  Reduce false positives Bug diagnosis  On-site diagnosis  Distributed systems diagnosis  Deterministic replay on multicore /multiprocessors Bug recovery  Automatic bug fixing  Bug survival with reduced functionality

32. YY. Zhou University of Illinois, Urbana-Champaign Slide 32 Conclusions Software bugs have existed for decades and will continue to exist Fighting bugs is important Fighting bugs is a fun and interdisciplinary  Compiler, software engineering, systems, hardware, data mining, machine learning, statistics, …

33. YY. Zhou University of Illinois, Urbana-Champaign Slide 33 Thanks!

34. YY. Zhou University of Illinois, Urbana-Champaign Slide 34 More Bug’s Cartoon