1. Coding concerns, are they real? Fadi Wedyan, Dalal Alrmuny May 10 th, 2007

2. Outlines Objective. Motivation. Study method. Results. Threats to validity. Conclusions. Future work and suggested projects. Questions.

3. Objective Investigating the possible connection between coding concerns and software failures in OSS.

4. Motivation No existing study attempted to answer the research question we asked. Many inspection tools exist, are they useful, are the concerns “real”?

5. Study method 1) choose the OSS to be studied, 2) choose tools for code inspection, and 3) develop a methodology to map the coding concerns to the reported failures

6. Research Questions Q1: “Does a relationship between coding concerns and reported failures in OSS exist?” Q2: “Are there certain types of coding concerns that are more likely to produce failures in OSS?”

7. Hypotheses First Hypothesis H1: Null hypothesis H01: “There is no relationship between coding concerns and reported failures in OSS”. Alternative hypothesis HA1: “A relationship exists between coding concerns and reported failures in OSS”. Second Hypothesis H2: Null hypothesis H02: “There is no relationship between certain types of coding concerns and reported failures in OSS”. Alternative hypothesis HA2: “A relationship exists between certain types of coding concerns and reported failures in OSS”.

8. Studied Systems Project NameReleasesNo. DevelopersKLOCNumber of Files jEdit5 stable, 6 mature138140.6394 iText4-beta, 5 stable499.8685 Table 1: Main characteristics of the studied software systems

9. Code Inspection Tools ToolTypeAnalysisCode IntelliJ ® IDEACommercialStaticSource code FindBugs™OSSStaticBytecode Table 2: Main characteristics of code inspection tools used in the study

10. Sources of failure data Reported bugs in SourceForge –Incomplete. –Lot of faked bugs. CVS Repository –Complete. –Distinguish bugs fixing actions from others.

11. Results (1) VersionDate releasedReported failuresFindBugs™IntelliJ ® IDEA iText 1.3April 6, 200542511 iText 1.4March 3, 20063069 jEdit 3.0Dec 21, 20003801 jEdit 4.1March 27, 20032612 jEdit 4.2Aug 28, 20043112 Table 3: Reported failures compared to concerns

12. Results (2) iText 1.3iText 1.4jEdit 3.0jEdit 4.1jEdit 4.2 Bad Practice10811991110 Correctness1214134 Malicious234260122 Multithreaded45122 Dodgy7980288 Total4464781426 Table 4: Coding concerns reported by FindBugs™

13. Results (3) Table 5: Coding reported by IntelliJ® IDEA iText 1.3iText 1.4jEdit 3.0jEdit 4.1jEdit 4.2 Class Structure103 328450384 Error Handling3940324589 Numerical Issues141763028 Cloning Issues75444 Control Flow Issues7281150193374 Data Flow Issues3835658 Threading Issues55223337 Probable Bugs653649235286307 Total93193578310461231

14. Results (4) Table 6: Coding concerns categories compared to detected failures in iText 1.3 using IntelliJ® IDEA CategoryConcernsFailures Class Structure1030 Error Handling392 Numerical Issues141 Cloning Issues71 Control Flow Issues720 Data Flow Issues380 Threading Issues50 Probable Bugs6537 Total93111

15. Results (5) Table 7: Coding concerns categories compared to detected failures in iText 1.3 using FindBugs™ CategoryConcernsFailures Bad Practice1084 Correctness120 Malicious2340 Multithreaded40 Dodgy791 Total4465

16. Threats to Validity Internal validity: –The size of the studied systems. –The reported failures. External validity: –Size and type of the data sets. –Code inspection tools.

17. Conclusions A relationship exists between coding concerns and reported failures: the inspection tools were successful in pointing out 12-26% of the reported failures. Certain categories of concerns were successfully mapped into failures more than others.

18. Conclusions We recommend relying on the CVS repository instead of the reported failures in SourceForge. –About 40% of failures fixing found in CVS did not exist in the reported failures.

19. Future work and suggested projects More testing: extending the study on more software and using other tools. Studying early releases of software. Studying the “pattern of errors in Java codes”. –Results can be integrated in the inspection tools. For example, tracing the life cycle of objects.

20. Questions